%0 Journal Article
%@ 2564-1891
%I JMIR Publications
%V 5
%N 
%P e66416
%T Assessing the Reliability and Validity of Principles for Health-Related Information on Social Media (PRHISM) for Evaluating Breast Cancer Treatment Videos on YouTube: Instrument Validation Study
%A Kusama,Hiroki
%A Takahashi,Yoshimitsu
%A Orihara,Shunichiro
%A Adachi,Kayo
%A Ishizuka,Yumiko
%A Semba,Ryoko
%A Shima,Hidetaka
%A Horimoto,Yoshiya
%A Kaise,Hiroshi
%A Taguri,Masataka
%A Inoue,Sho
%A Nakayama,Takeo
%A Ishikawa,Takashi
%K information quality
%K social media
%K YouTube
%K PRHISM
%K breast cancer treatment
%K videos
%K reliability
%K validity
%K instrument validation study
%K medical information
%K online health information
%K cancer treatment
%K Japan
%K Principles for Health-Related Information on Social Media
%D 2025
%7 11.6.2025
%9 
%J JMIR Infodemiology
%G English
%X Background: There is breast cancer–related medical information on social media, but there is no established method for objectively evaluating the quality of this information. Principles for Health-Related Information on Social Media (PRHISM) is a newly developed tool for objectively assessing the quality of health-related information on social media; however, there have been no reports evaluating its reliability and validity. Objective: The purpose of this study was to statistically examine the reliability and validity of PRHISM using videos about breast cancer treatment on YouTube (Google). Methods: In total, 60 YouTube videos were selected on January 5, 2024, with the Japanese words for “breast cancer,” “treatment,” and “chemotherapy,” and assessed by 6 Japanese physicians with expertise in breast cancer. These evaluators independently evaluated the videos using PRHISM and an established tool for assessing the quality of health-related information, DISCERN, as well as through subjective assessments. We calculated interrater and intrarater agreement among evaluators with CIs, measuring agreement using weighted Cohen kappa. Results: The interrater agreement for PRHISM overall quality was κ=0.52 (90% CI 0.49-0.55), indicating that the expected level of agreement, statistically defined by the lower limit of the 90% CI exceeding 0.53, was not achieved. However, PRHISM demonstrated higher agreement compared with DISCERN overall quality, which had a κ=0.45 (90% CI 0.41-0.48). In terms of validity, the intrarater agreement between PRHISM and subjective assessments by breast experts was κ=0.37 (95% CI 0.14-0.60), while DISCERN showed an agreement of κ=0.27 (95% CI 0.07-0.48), indicating fair agreement and no significant difference in validity. Conclusions: PRHISM has demonstrated sufficient reliability and validity for evaluating the quality of health-related information on YouTube, making it a promising new metric. To further enhance objectivity, it is necessary to explore the use of artificial intelligence and other approaches. 
%R 10.2196/66416
%U https://infodemiology.jmir.org/2025/1/e66416
%U https://doi.org/10.2196/66416