Development and preliminary reliability testing of an assessment of patient independence in performing a treatment program: Standardized scenarios

Background: Physical therapists often assess patient independence through observation; however, it is not known if therapists make these judgments reliably. We have developed a standardized method to assess a patient's ability to perform his or her treatment program independently. Objectives: To develop a standardized assessment of patient independence in performance of a treatment program and examine the intra- and inter-rater reliability decisions made by two physical therapists. Design: Test-retest. Methods: An assessment of patient independence in performance was developed. Standardized patient scenarios were used to assess the intra- and inter-tester reliability of two physical therapists. Percentage of agreement (%) and kappa's coefficient (k and k _w indexed rater reliability. Results: Intra-rater reliability of therapist 1 was as follows: knowledge: %=95, k=0.90; performance: %=95, k _w=0.82. Intra-rater reliability of therapist 2 was as follows: knowledge: %=85, k=0.68; performance: %=94, k_w=0.80. Interrater reliability for knowledge was %=91 and k=0.79 and for performance was %=91 and k_w=0.72. Conclusion: Trained therapists displayed substantial to excellent intra-rater reliability and substantial inter-rater reliability in assessing a patient's independence in a treatment program.