Evaluating the appropriateness of electronic information resources for learning

Objectives: Current US medical students have begun to rely on electronic information repositories— such as UpToDate, AccessMedicine, and Wikipedia—for their pre-clerkship medical education. However, it is unclear whether these resources are appropriate for this level of learning due to factors involving information quality, level of evidence, and the requisite knowledgebase. This study evaluated appropriateness of electronic information resources from a novel perspective: amount of mental effort learners invest in interactions with these resources and effects of the experienced mental effort on learning. Methods: Eighteen first-year medical students read about three unstudied diseases in the above-mentioned resources (a total of fifty-four observations). Their eye movement characteristics (i.e., fixation duration, fixation count, visit duration, and task-evoked pupillary response) were recorded and used as psychophysiological indicators of the experienced mental effort. Post reading, students’ learning was assessed with multiple-choice tests. Eye metrics and test results constituted quantitative data analyzed according to the repeated Latin square design. Students’ perceptions of interacting with the information resources were also collected. Participants’ feedback during semi-structured interviews constituted qualitative data and was reviewed, transcribed, and open coded for emergent themes. Results: Compared to AccessMedicine and Wikipedia, UpToDate was associated with significantly higher values of eye metrics, suggesting learners experienced higher mental effort. No statistically significant difference between the amount of mental effort and learning outcomes was found. More so, descriptive statistical analysis of the knowledge test scores suggested similar levels of learning regardless of the information resource used. Conclusions: Judging by the learning outcomes, all three information resources were found appropriate for learning. UpToDate, however, when used alone, may be less appropriate for firstyear medical students’ learning as it does not fully address their information needs and is more demanding in terms of cognitive resources invested.