This study is part of a project to develop an evidence-based informed consent form for treatment decisions related to knee osteoarthritis [38]. Using a descriptive cross-sectional design, this study assesses compliance with evidence-based quality standards in HIM about knee osteoarthritis treatment. Initially, the plan was to evaluate the quality of printed information leaflets distributed by doctors and received by patients enrolled in an exploratory study [12]. However, due to the low response rate to the leaflet collection, the methodology was adapted to include online HIM. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement was used for the reporting [39].

From January to July 2021, we invited doctors and patients participating in an exploratory study on information and decision-making processes to provide printed material on the treatment of knee osteoarthritis [12]. Specifically, we asked 3 general practitioners, 4 office-based orthopedic specialists, and 1 hospital in an urban metropolitan area in western Germany to contribute. In addition, 13 patients, recruited both regionally and nationwide, were invited to contribute material. Participants were asked to send their material to us electronically; for those unable to do so, prepaid return envelopes were provided. However, only sporadic printed material was received from doctors, and none from patients. To supplement the collected data, we conducted an exploratory internet search in April 2022 using the Google search engine. This search targeted German-language information provided specifically by medical practices and clinics. Using the German term for knee osteoarthritis, the first 5 pages of search results were reviewed, with browser settings reset to default. This first step aimed to provide an overview of the type and scope of health information available.

In May 2023, a systematic search was conducted to expand the scope of the existing data. This search was also performed using Google, the most widely used search engine in Germany [40], with browser settings reset and incognito mode activated to minimize personalization of search results. We used Google Trends analysis [41] to identify the most popular German search terms relating to treatments for knee osteoarthritis. As TKA was a popular topic, a separate search term was used for this treatment option. For knee osteoarthritis, the term “Kniearthrose” was used, while for TKA, the term “Knie TEP” was applied. The initial 50 search results for each search term were extracted and documented in a Microsoft Excel spreadsheet. Additionally, known sources of health information not captured in the Google search results were manually added to ensure comprehensiveness.

HIM was included if it met the following criteria: (1) it provided information about the treatment of knee osteoarthritis, (2) addressed lay audiences, and (3) was written in the German language.

For the internet search, additional criteria were applied, such as websites were included regardless of how the content was presented. Google results marked as advertisements were excluded. Websites were also excluded if (1) they consisted exclusively of patient testimonials or (2) the information was not publicly accessible.

HIM identified across all 3 searches was consolidated, and duplicates were removed. When multiple websites from the same provider addressed different search terms, they were counted as a single HIM if the content was clearly interconnected and there was overall coherence. We defined the coherence of HIM on the basis of the structure. We consider a system of hierarchically organized pages from the same provider to be one HIM, whereas a referral to another provider, for example, is not considered to be part of the same HIM.

Selection criteria were independently reviewed by 2 researchers, and disagreements were resolved by a third reviewer. For the internet searches, copies of the included and excluded HIM were saved as PDF files so that the original could be accessed in the event of website changes during the course of the study.

All HIMs were categorized based on 3 basic variables: provider, country, and certification. Providers were classified into 8 categories, including commercial providers, hospitals, medical practices, nonprofit organizations, publicly funded providers, specialist associations, health insurance companies, and self-help groups. As German-language HIM can come from various countries, a further classification was made according to the provider’s country of origin, that being Germany, Austria, or Switzerland. Regarding certification, distinctions were made based on whether the HIM held any external certification or recommendation and, if so, from which organization.

The quality assessment was carried out using the Mapping the Quality of Health Information (MAPPinfo) checklist [42], which is based on the Guideline EBHI [29]. It is the first tool to implement the recommendations of the guideline in a precise way, allows the assessment to be based only on HIM without the need for the search of additional information, and can be reliably used by people without special training or expertise in evidence-based practice. Therefore, the quality assessment process is much more transparent and accessible to a wider audience. The MAPPinfo checklist consists of 19 items that evaluate the following 4 key aspects of quality: definitions, which refer to the target audience and the purpose of the information; transparency, which addresses how the information was prepared, including details on authorship, funding, timeliness, and sources; content, which focuses on the informational content such as the explanation of options and the presentation of benefits and harms; and presentation, which examines the suitability of how the content is delivered. The checklist includes 11 items assessed using a dichotomous format (fulfilled/not fulfilled) and 8 items assessed using a trichotomous format (fulfilled/partially fulfilled/not fulfilled). The total number of applicable items depends on the specific content of the HIM, with certain items, such as C7/P4, used only for diagnostic decisions and others, such as P7, applied only when charts are used to visualize frequencies. The total score of the MAPPinfo checklist, which ranges from 0% to 100%, represents the information quality and reflects the extent to which the criteria of the Guideline EBHI are met.

The psychometric properties of the checklist had been validated in previous studies, confirming its suitability [42]. The assessment process and documentation were carried out using the Microsoft Excel–based MAPPinfo Evaluation Toolkit, which includes several worksheets. These enabled the collection of independent evaluator ratings, plausibility checks, identification of discrepancies, and documentation of consensus, as well as tables with calculations of reliability and information quality and the visualization of the results [43].

The MAPPinfo rating was carried out independently by 2 master students (LMR and JS), followed by a consensus process. In the case of disagreement, a third rater (SZ) was consulted, and consensus was reached through discussion. To ensure high interrater agreement, the 3 raters independently assessed a sample of 10 preselected HIMs and discussed discrepancies in their assessments, developing a common understanding of how the criteria should be applied to the specific health problem of knee osteoarthritis.

The MAPPinfo checklist already includes the aspects of trustworthiness and appropriate presentation of content. In addition, the content of the HIM was systematically extracted using predefined criteria, including information on the complications and benefits of TKA, treatment options for knee osteoarthritis, anesthesia options, and complications related to anesthetics. A subjective evaluation was also conducted to assess whether the treatment options were presented in a balanced manner. The criteria for complications of TKA were determined using the standardized list of the Knee Society [44], the criteria for benefits of TKA and the treatment options were defined using the developed evidence-based informed consent forms [38], and the definitions of the German consensus guideline on knee osteoarthritis [4]. The extraction process was organized in an Excel matrix, with both the criteria and the matrix itself piloted and refined based on an initial review of the 10 preselected HIMs. Where classification was not feasible, free-text entries were permitted to capture relevant details. The assessment was initially conducted by one rater and subsequently reviewed by a second rater to ensure consistency and accuracy.

Data were analyzed using the Excel-based MAPPinfo Evaluation Toolkit [43]. We calculated a score ranging from 0% to 100% for each HIM. Each item was assigned 1 point if it was fulfilled, 0.5 points if it was partially fulfilled, or 0 points if it was not fulfilled. Items C4/P1 to C7/P4, which reflect both content and presentation, were given double weighting. The total score was calculated by dividing the sum of all item points by the maximum possible points for all applicable items and expressing the result as a percentage. A score of 100% indicates full compliance with the quality criteria of the Guideline EBHI. Additionally, we determined the overall mean score for all HIMs, as well as the mean scores for each item, including the SD. The overall mean values were further analyzed descriptively for different values of the basic variables of the HIM. The content analysis was performed descriptively, using frequencies and proportions. Furthermore, a descriptive analysis was conducted to evaluate the presentation of treatment options by provider groups. The interrater reliabilities of the MAPPinfo ratings between the 2 raters were calculated at the item level using the T-coefficient. This method does not estimate the probabilities of expected agreement empirically from the given dataset but uses theoretical assumptions derived from the number of potential values that an item can receive [45]. Interrater reliabilities were considered moderate between 0.40 and 0.60, strong when higher than 0.60, and excellent when higher than 0.80 [46].

As this study does not involve participants or participant data, ethical approval and informed consent were not applicable.

Out of 159 identified HIMs, a total of 94 were included in the analysis (Figure 1). Within the total sample, commercial providers accounted for 37.2% (n=35), hospitals for 27.7% (n=26), medical practices for 25.5% (n=24), nonprofit organizations and publicly funded providers for 3.2% (n=3), professional associations for 2.1% (n=2), and health insurance companies for 1.1% (n=1). No HIMs from self-help groups were included. Certifications from 3 organizations were identified (Health Information System Action Forum [Aktionsforum Gesundheitsinformationssystem] (Afgis) [47], German Health Literacy Network [Deutsches Netzwerk Gesundheitskompetenz] (DNGK) [48], and Public Health Foundation [49]), most of which contain transparency criteria. In total, 13 of the analyzed HIMs were certified by at least one of these organizations, while 81 had no certification.

The agreement between the raters was excellent (T=0.91, T minimum=0.6, T maximum=1). The 94 HIMs demonstrated a mean compliance of 14.6% (SD 9.4%) with the criteria outlined in the Guideline EBHI. Compliance with the quality standards ranged from 0% to 69% among individual HIMs. None of the HIMs met all criteria.

At the item level (Figure 2 [43]), the 3 criteria with the highest compliance were the avoidance of using narratives (P6; mean 88.3%, SD 32.1%), the use of neutral language (P5; mean 48.9%, SD 50%), and the explanation of the health problem (C1; mean 45.7%, SD 49.8%). Compliance with the quality standards (Table 1) was very low in some of the criteria. These include, for example, the use of gain and loss framing (P8; 0%), the reporting of a systematic search strategy (T6; mean 1.1%, SD 10.3%), the reporting of a strategy for managing conflicts of interest (T3; mean 1.1%, SD 10.3%), and the presentation of benefits (C5/P2; mean 2.1%, SD 14.4%) and harms (C6/P3; mean 1.1%, SD 10.3%).

Different levels of information quality were observed between provider groups, as well as between HIMs with or without certification, and between HIMs with different reporting of treatment options. HIM provided by publicly funded providers (n=3) and nonprofit organizations (n=3) achieved higher information quality with an average of 40.1% (SD 3.6%) and 37.2% (SD 23.1%), respectively, compared to HIM from other providers (Table 2). HIMs with at least 1 certification (n=13) demonstrated higher information quality than uncertified HIMs (n=81), with an average of 26.8% (SD 16%) compared to 12.7% (SD 5.9%). However, information quality also varied within certified HIMs. Those provided by organizations recognized as reliable health portals by the DNGK [48] had higher information quality than other certifications. HIM providing balanced information on conservative and surgical treatment options (n=14) achieved higher information quality than those focusing on surgical (n=52) or conservative options (n=28). Information quality was comparable across countries of origin (Table 2).

TKA was mentioned as a treatment option by 78 of the 94 HIMs. Within this subgroup, a third of the HIM did not report any complications (n=28, 35.9%) or benefits (n=30, 38.5%). The most commonly reported complications were joint infection (n=39, 50%), prosthesis loosening (n=30, 38.5%), and revision (n=25, 32.1%). In terms of benefits, pain relief was the most commonly reported benefit (n=32, 41%), followed by nonspecific statements about mobility (n=24, 30.8%) and general nonspecific statements about benefits (n=20, 25.6%; Table 3).

Within the same subgroup, 21 (26.9%) HIMs addressed the topic of anesthetics. When anesthetics were mentioned, the available options (different types of anesthetics) were mostly described. A total of 2 (9.5%) HIMs reported associated complications (Table 3).

The HIMs mentioned a range of treatment options for knee osteoarthritis. Pharmacological therapy was most frequently reported (n=75, 79.8%), followed by physiotherapy (n=70, 74.5%) and orthopedic assistive devices (n=53, 56.4%; Table 4).

Treatment options were reported in a balanced way in 14 HIMs, while 52 HIMs focused on surgical options and 28 on conservative options. The focus varied depending on the type of provider (Table 5).

This study was designed to investigate the extent to which German-language HIM for the treatment of knee osteoarthritis can facilitate informed choice and to provide an in-depth analysis of contents published in this segment of health information. To this purpose, 94 HIMs from different providers were assessed using the validated MAPPinfo checklist, which operationalizes ethical and evidence-based minimum criteria for HIM aiming to facilitate informed choices. The content of the HIMs was then extracted and analyzed in a standardized manner.

The results indicate a concerning lack of information quality in the HIM analyzed, with an average of only 14.6% (SD 9.4%) of the minimum criteria necessary for informed choices being met. It is noteworthy that none of the HIMs met all the criteria in full (maximum score: 69%).

Deficits in the definition and content criteria are particularly critical. The definition of the target group and the formulation of the objective were fulfilled on average by only 1.1% (SD 10.3%) and 6.4% (SD 20.9%), respectively. Missing or insufficient information in these areas can cause uncertainty among users and contradict the requirements for an informed choice. The description of all possible options and the presentation of the choice between them was fulfilled on average by only 9% (SD 20.6%). Reporting on the natural course of osteoarthritis of the knee was fulfilled on average by only 3.2% (SD 17.6%). Similarly, the appropriate presentation of benefits and harms was fulfilled on average by only 2.1% (SD 14.4%) and 1.1% (SD 10.3%), respectively. It is crucial for patients with knee osteoarthritis to have a comprehensive picture of possible treatments and their consequences so that they can make a decision in line with their preferences and values. These areas were also identified by patients in previous studies as key information needs [11-13,17,18].

We found some differences in information quality between different providers. Health information from publicly funded and nonprofit providers had a higher average information quality of 40.1% (SD 3.6%) and 37.2% (SD 23.1%), respectively. The HIM with the highest quality (69%) also came from a nonprofit organization. On the one hand, this indicates that there is potential, but at the same time, it becomes clear that even these providers do not fulfil all the criteria. Overall, we therefore conclude that the HIM analyzed for the treatment of knee osteoarthritis does not support an informed choice.

According to the providers of the certifications, the certification of HIM is intended to reflect a quality standard. Our analysis showed that HIMs with at least 1 certification have a slightly higher information quality (mean 26.8%, SD 16%) than those without certification (mean 12.7%, SD 5.9%). Due to the predominant focus of these certificates on transparency criteria and the overall low information quality, the quality standard of the certifications appears insufficient. The DNGK certificate stands out here with an average information quality of 40.9% (SD 17.3%), which is mainly due to recommendations from publicly funded and nonprofit providers. However, the content requirements remain considerably below the requirements for an informed choice.

The latest update to the guideline on indication for knee arthroplasty reiterates that nonsurgical treatment should be attempted for at least 3 months before surgery is considered [50]. However, previous studies showed that conservative treatment options are often not sufficiently offered or used [12,13,15,16]. The analysis revealed that 85.1% (80/94) of HIMs mentioned at least 1 conservative treatment option, which at first glance appears to be helpful. However, due to the unbalanced presentation of this information, we have considerable doubts as to whether this really helps to meet information needs. Doctors are highly trusted and are often used as a source of information. [12,23,24]. However, only 3 HIMs from hospitals (n=26) and 1 HIM from medical practices (n=24) presented the treatment options in a balanced way. The majority of these presentations focused on surgical therapies, which could reinforce the impression that conservative options are less relevant and thus strengthen the impression of TKA as the only meaningful option [10,12,14,15]. Decision-making in the treatment of knee osteoarthritis is often complex due to the chronic and fluctuating nature of the condition. Patients are faced with the challenge of addressing their impairments or adapting their lives accordingly while weighing up the benefits and risks of the available treatment options. Their need for information may vary depending on the stage of the disease and their personal circumstances. [10] This emphasizes the importance of comprehensive, balanced, and timely information to ensure that no treatment option is overlooked.

In the 78 HIMs that mentioned TKA as a treatment option, one-third did not mention complications or benefits at all. Serious complications, including but not limited to infections, implant loosening, revision surgery, and thromboembolic events, were mentioned in only up to half of the HIMs.

It is well-documented that patients frequently seek additional sources of information when they are provided with insufficient details by their health care professionals [20]. However, these findings suggest that, due to the generally poor quality of available HIM, this additional search is likely to be unsuccessful and may even place an additional burden on health care systems as doctors have to address the possibly incorrect information given to patients.

Previous studies evaluating the quality of health information on knee osteoarthritis and TKA did not use assessment tools that fully meet the criteria of the Guideline EBHI [29], which defines the standards needed to support informed decision-making, nor have they been extensively validated. Consequently, a direct comparison with our results is not possible. Nevertheless, these previous studies have also identified substantial weaknesses in the HIM they examined [30-36].

The literature repeatedly highlights the issue of inadequate health literacy and the associated difficulties in accessing, interpreting, and using health information effectively to make informed decisions [51]. While we acknowledge and strongly support the need to improve health literacy, it remains essential to ensure that appropriate HIM is available to support informed decisions when adequate health literacy exists. The findings of this study are in concordance with those of previous analyses of HIM, which indicate that such HIMs are currently not available [52,53].

The findings of this study underline the urgent need for health information providers to improve the quality of available HIM. The criteria required to support informed choices should guide these improvements. To address these challenges, initiatives are needed to strategically allocate public resources to the production of high-quality HIM and to encourage greater collaboration between developers of HIM.

The current range of certifications, which are designed to guarantee the quality of HIM, do not meet this objective adequately with regard to the HIM analyzed for the treatment of knee osteoarthritis. Furthermore, these certifications do not equip users adequately with the ability to recognize the individual strengths and weaknesses of HIM. One potential solution might be to establish a certification system based on the MAPPinfo checklist, explicitly assessing criteria necessary for informed choices. Moreover, the incorporation of visual representations of information quality could facilitate transparency for users by clearly indicating which criteria are fulfilled and where deficiencies exist. While numerous tools already exist to help users easily assess the quality of health information, a graphical representation of MAPPinfo information quality could offer a straightforward approach. The findings of our study suggest that patients with knee osteoarthritis seeking treatment-related information should prioritize HIM from publicly funded or nonprofit providers.

The strengths of this study include a systematic search strategy, use of the validated MAPPinfo checklist, excellent interrater reliability, and rigorous, systematic documentation of all assessment steps. However, it is important to acknowledge the limitations of the study. First, the search was conducted exclusively using Google, which may introduce a selection bias in an unknown direction. Furthermore, even though the sample was extensive, not all available HIMs could be identified. Second, it could be argued that evaluating HIM against criteria for informed choice may not reflect the intentions of all information providers, as some may claim that they do not explicitly aim to support decision-making. This approach was justified on the basis that users typically cannot discern the intended purpose of a given health resource or determine what information is necessary to support an informed choice. Consequently, it is reasonable to assume that users generally perceive HIM as suitable for decision-making unless it is explicitly and clearly stated otherwise by the provider. A further limitation concerns the temporary nature of the criteria used. At the time of this study, the criteria used in the MAPPinfo checklist represented ethical and evidence-based minimum standards. However, it is conceivable that additional criteria could influence informed choice to varying degrees, although such criteria have not yet been sufficiently validated and therefore could not be assessed.

Overall, our findings highlight the urgent need to substantially improve the quality of German-language HIM on the treatment of knee osteoarthritis to support patients in making informed choices. The identified deficiencies are fundamental and affect all dimensions of information quality. Furthermore, a substantial proportion of the HIM reviewed does not address critical issues, such as complications and benefits associated with TKA, and only a few present treatment options in a balanced way. Such deficiencies may potentially result in a biased perception of available therapies, which could influence patient decision-making.

Moreover, of the HIM analyzed for the treatment of knee osteoarthritis, only 1 out of 3 available certifications exhibited substantially higher information quality. However, it did not meet the comprehensive requirements needed to support informed choice. The MAPPinfo checklist has the potential to serve as a valuable instrument for more differentiated certification, highlighting both the strengths and weaknesses of HIMs. Furthermore, the establishment of initiatives aimed at enhancing cooperation among providers and strategically aligning public resources should facilitate the development of HIM that meets all the criteria of the Guideline EBHI. Until such initiatives are implemented, patients seeking the highest quality information are advised to consult health information provided by publicly funded or nonprofit organizations.