Interrater, intra-rater and internal consistency reliability of the Theory Application Assessment Instrument

Aim: To investigate the interrater, intra-rater and internal consistency reliability of the Theory Application Assessment Instrument, a tool for testing competency in combining multiple theoretical conceptual practice models in client evaluation and treatment planning. Methods: The assessment was administered to 27 graduate students in two phases. The principal investigator rated the participants on the assessment scale on two separate occasions. Results: Interrater and intra-rater reliability were good for Phase 2 data. Internal consistency reliability for Phase 2 data was in the 'questionable' range. Conclusion: The instrument shows promise of being an efficient tool. However, this was an exploratory study with a small sample. Its replication with a larger more representative sample is needed to increase external validity.