PageGraph
A theory-driven real-time assistive extension for health credibility assessment | Jun 2024
Assessing the credibility of online health information (OHI) remains a considerable challenge for consumers. Few studies have explored how to assist this process in real-time.
Guided by nudge theory, we designed an assistive credibility evaluation tool, PageGraph, which displays quality indicators and values based on credibility evaluation models and empirical research. We conducted a two-stage mixed-method approach to test and improve the usability of PageGraph.
THEORY-DRIVEN DESIGN
Nudge theory employs subtle interventions to avoid intrusiveness and preserve individual choices within the environment. Nudges encompass various strategies, including incentives, default settings, feedback mechanisms, and structuring complex choices. Nudges have demonstrated effectiveness in encouraging responsible behaviors such as promoting savings, reducing energy consumption, and improving dietary habits.
A CHROME EXTENSION
PageGraph extension is shown on the right side of the screen. It includes 11 quality indicators: site owner, site type, contact information, site aim, editorial process, author, publication date, time of update, number of references, number of ads, and linguistic features.
Therefore, We designed a Chrome extension (PageGraph) that displays a list of pres-elected quality indicators aside from webpages. The selection of quality indicators was based on established rubrics by healthcare professionals and prior empirical evidence, including HONcode and JAMA.
USABILITY TESTS
We conducted 16 usability tests to explore users’ interactions with this tool in an attempt to offer design and theoretical implications for future studies. Participants were invited to complete 9 health information evaluation tasks and participate in follow-up interviews. In Stage 1, we invited 16 participants and collected their eye-tracking activities to see how they used and interacted with PageGraph. We presented the screen recordings of the 9 tasks and elicited explanations about their actions (e.g., why they opened/closed PageGraph) and how PageGraph did or did not help their evaluation.
In Stage 1, PageGraph helped decrease the 16 participants' mental workload during evaluation. However, their interactions with PageGraph (e.g., open frequency, fixation time, and eye-movement pattern) varied by website type. The interview data revealed that participants have different evaluation modes in health information evaluation and place varying emphasis on different indicators.
Pupil dilation collected in the eye-tracking experiment for usability tests suggest that when users are interacting with the PageGraph, their mental workload is significantly lower. The extension is effective in lower users’ mental workload while supporting credibility assessment during health-related information search.
Based on the findings in Stage 1, we moved to Stage 2 and focused on participants' indicator use preferences when evaluating health information. We recruited 60 participants to conduct the lab experiment and asked them to indicate their indicator use. In Stage 2, we identified the indicators that should be customized by the type of website, such as site owner/sponsor, article author, number of references, number of ads, and linguistic features.
The results show that participants use significantly different indicators from PageGraph based on the types of the websites (i.e., commercial, government or dicussion forum). This suggests that the indicators in the PageGraph are important since it can support users evaluations and decisions for diverse health-related webpages.
PageGraph can support participants' credibility evaluation by providing content summaries and highlighting key information, thereby decreasing their mental workload. Future studies should consider how to customize the presented indicators and their ranges according to the context (e.g., website type).
