Co-Designing a Web-Based and Tablet App to Evaluate Clinical Outcomes of Early Psychosis Service Users in a Learning Health Care Network: User-Centered Design Workshop and Pilot Study

Background: The Early Psychosis Intervention Network of California project, a learning health care network of California early psychosis intervention (EPI) programs, prioritized incorporation of community partner feedback while designing its eHealth app, Beehive. Though eHealth apps can support learning health care network data collection aims, low user acceptance or adoption can pose barriers to successful implementation. Adopting user-centered design (UCD) approaches, such as incorporation of user feedback, prototyping, iterative design, and continuous evaluation, can mitigate these potential barriers. Objective: We aimed to use UCD during development of a data collection and data visualization web-based and tablet app, Beehive, to promote engagement with Beehive as part of standard EPI care across a diverse user-base. Methods: Our UCD approach included incorporation of user feedback, prototyping, iterative design, and continuous evaluation. This started with user journey mapping to create storyboards, which were then presented in UCD workshops with service users, their support persons, and EPI providers. We incorporated feedback from these workshops into the alpha version of Beehive, which was also presented in a UCD workshop. Feedback was again incorporated into the beta version of Beehive. We provided Beehive training to 4 EPI programs who then piloted Beehive’s beta version. During piloting, service users, their support persons, and EPI program providers completed Beehive surveys at enrollment and every 6 months after treatment initiation. To examine preliminary user acceptance and adoption during the piloting phase, we assessed rates of participant enrollment and survey completion, with a particular focus on completion of a prioritized survey: the Modified Colorado Symptom Index. Results: UCD workshop feedback resulted in the creation of new workflows and interface changes in Beehive to improve the user experience. During piloting, 48 service users, 42 support persons, and 72 EPI program providers enrolled in Beehive. Data were available for 88% (n=42) of service users, including self-reported data for 79% (n=38), collateral-reported data for 42% (n=20), and clinician-entered data for 17% (n=8). The Modified Colorado Symptom Index was completed by 54% (n=26) of service users (total score: mean 24.16, SD 16.81). In addition, 35 service users had a support person who could complete the Modified Colorado Symptom Index, and 56% (n=19) of support persons completed it (mean 26.71, SD 14.43). Conclusions: Implementing UCD principles while developing the Beehive app resulted in early workflow changes and produced an app that was acceptable and feasible for collection of self-reported clinical outcomes data from service users. Additional support is needed to increase collateral-reported and clinician-entered data.