Abstract
The increasing use of the internet has resulted in its gaining a more active role in both patient and professional education, including the YouTube platform, a popular platform for sharing and watching videos. This study aimed to evaluate the quality, usefulness, and reliability of videos on inflammatory muscle diseases on YouTube, as well as their determinants for both patient and healthcare professional education.
The keywords “Inflammatory muscle disease”, “Idiopathic inflammatory myositis”, “Inflammatory muscle disease treatment”, “Inflammatory myositis treatment”, “Dermatomyositis” and “Polymyositis” were searched on the site on April 22nd, 2022. According to the Global Quality scale, three categories (high quality, middle quality, and low quality) were created based on the educational value of YouTube videos. The reliability of the information was assessed using the DISCERN tool. The video parameters were compared between the quality groups to reveal the determinants of the quality.
Seventy-five videos of patients and 116 videos of healthcare professionals were included in the video after exclusion. The numbers of high, intermediate and low-quality videos were 27 (36%), 27 (36%), and 21 (28%) for videos for patients, and 20 (17.2%), 60 (51.8%), and 36 (31%), respectively. Only the number of likes was found to be a determinant of video quality for healthcare professionals (p<0.05), and none of the other determinants were found to predict the quality for both groups (p>0.05).
YouTube is an open platform for information, and it includes various educational videos of various quality. Experts, such as physicians, should be encouraged to provide more content to help both patients and healthcare professionals obtain better quality information.
Introduction
Inflammatory muscle diseases are a group of autoimmune disorders that manifest with the inflammation of the muscles as well as the involvement of other systems (1). The skin, lungs, heart, joints, and many other systems can be involved. These heterogeneous disorders can take many names depending on the clinical manifestations, and serological and histopathological features, including but not limited to dermatomyositis, polymyositis, inclusion body myositis, overlap myositis, and immune-mediated necrotizing myopathy (2). As the management also differs between these disorders, the clinicians involved in the diagnosis and management must have a good understanding of the mechanisms, characteristics, and treatment of these pathologies.
Learning and understanding the principles of diseases, especially the complex ones such as inflammatory muscle diseases have always been challenging for health practitioners. With the precious mission of teaching the elaborate mechanisms and treatment methods and simplifying when in need for the patients, they require a helping hand even more. Thanks to the introduction of novel methods through developing technology, many new modes of education methods have emerged in recent years. The use of the internet, videos, and interactive learning programs with the help of computers can be examples of such methods. Learning with these methods is not only being pursued by health professionals since patients and their caregivers also use them to seek information regarding their situations and possible solutions as well.
The internet has become a choice of modality for gaining information since it has been introduced. Thus, obtaining health information was not an exception. It has been claimed that almost half of the population used the internet to gain insights regarding their health status and diseases (3). Moreover, internet use is also getting popularity in medical education (4). One source of information is YouTube, a widely-used platform where users can upload and view videos. Its user-friendly design and accessibility make it a valuable educational resource for patients learning about their health conditions, as well as for medical students and professionals seeking information (5-7). However, the accuracy, quality, and reliability of information on these platforms can be questionable. Since any user can upload videos that adhere to the terms of service, there are no safeguards like peer review to ensure the content’s credibility. While scientifically valid and high-quality information can be of use to patients, misleading information can be harmful (8). Although most medical professionals and students can discriminate the accuracy of the information, low-quality information can result in a waste of time and resources, and even lead them to wrong beliefs. Madathil et al. (9) reported that the healthcare information videos on YouTube varied regarding quality, having higher quality ones as well as conflicting and questionable videos. Detection of the determinants of quality and reliability of the videos can help practitioners and learners to choose the right videos to avoid misleading information and save time.
The review of the literature has shown no studies investigating the quality and reliability of the healthcare information on inflammatory muscle diseases (as a group or individual disease) provided by YouTube. Thus, the potential of YouTube to provide healthcare information to both patients and healthcare professionals is not known regarding this topic. This study seeks to evaluate the quality of healthcare information available on YouTube. Additionally, it aims to identify factors or video characteristics that influence the information’s quality and reliability.
Materials and Methods
The chosen keywords, “Inflammatory muscle disease”, “Idiopathic inflammatory myositis”, “Inflammatory muscle disease treatment”, “Inflammatory myositis treatment”, “Dermatomyositis” and “Polymyositis” were written in the search bar in the site “www.youtube.com” in April 22nd, 2022. Since users typically view only the first three pages, all videos from these pages were included (10). The likelihood of a video appearing on these pages increases with its popularity, ensuring that the most popular videos are included. Our search method gave a total of 300 videos, and these videos were assessed by two researchers. Irrelevant videos, duplicates, non-English videos, or videos with serious technical flaws that made it impossible to assess were excluded. Included videos were then grouped as “for patients” and “for health professionals”, and were analyzed separately.
Global quality scale (GQS): The GQS is a scale designed to assess internet information sources. It evaluates aspects such as media flow and user-friendliness. Scores range from 1 to 5, where 1 or 2 signify low quality, 3 indicates intermediate quality, and 4 or 5 denote high quality (11). The details and the criteria of GQS was given in Table 1.
DISCERN tool: The modified DISCERN tool was used to evaluate the reliability of the included videos. Scores range from 0 to 5 points, with higher scores indicating greater reliability (12, 13). The details and the criteria of the modified DISCERN tool was given in Table 2.
Sources and the properties of the videos: Evaluators identified the video sources, categorizing them into the following groups: “Academic/University”, “Physician”, “Health-related website”, “Professional organization”, “Non-physician health personnel”, “Patient” and “Independent User” (14). Other video characteristics, such as the total number of likes, comments, views, and video duration, were also recorded and analyzed.
Since this study uses publicly available data and does not involve human subjects, it is exempt from committee of ethics approval. This exemption applies to other similar study designs as well (13, 15, 16).
The analysis was conducted using the Statistical Package for the Social Sciences version 26.0 (SPSS Inc., Chicago, IL, USA). Data were described using median (minimum-maximum), median (interquartile range), numbers, and percentages. The Shapiro-Wilk test assessed the normality of distributions. Depending on whether the data were categorical or continuous, chi-square tests and the Kruskal-Wallis test were used to compare the groups. The kappa coefficient was used to evaluate the agreement between the two researchers. The threshold for significance was considered p<0.05.
Results
The inter-rater correlation analysis for GQS scores gave the kappa score of 0.88, indicating an excellent agreement.
The duplicate videos (n=51) were removed after obtaining the initial 240 videos. Twenty-five videos were then excluded from the analysis due to language other than English (n=5), irrelevant content/advertisement (n=12), and technical issues that made the videos unassessable (n=8).
The videos were then marked for their audience, which gave 116 videos for health professionals, and 75 videos for patients, as 27 videos were found to be useful for both audiences. The general features of the videos were given in Table 3.
The sources of the videos intended for healthcare professionals were given in Table 4. Most of these videos were uploaded by physicians, and all high-quality videos were uploaded by physicians and academics/universities. The videos that were uploaded by non-physician healthcare professionals, independent users, and health-related websites were found to be of low quality. Still, most of the videos were found to be of intermediate quality, and as physicians contributed the most to the videos, they had the highest number of videos among all quality categories.
The sources of the videos intended for patients were given in Table 5. The physicians had the highest amount of numbers in these videos too, only to be seconded by the patients themselves. Most of the physicians’ content was found to be high in quality (41.9%), with intermediate-quality content being very close (39.5%). Although the patients contributed the most to the high-quality videos after physicians, most of their content was of low (38.5%) and intermediate (38.5%) quality. There were also contributions from independent users and non-physician healthcare professionals to the high-quality and low-quality content.
The inter-rater correlation analysis for GQS scores gave the kappa score of 0.88, indicating an excellent agreement.
There were no differences among quality groups regarding the number of views, comments, or video durations. In both groups of videos, DISCERN scores were found to differ among videos of different qualities (p<0.01). Interestingly, the number of likes was found to be an indicator of quality for videos intended for healthcare professionals (p=0.004), and not for the videos intended for patients (p>0.05). The findings were given in Table 6.
Discussion
As technology advances, the internet and associated information sources like websites have become increasingly popular. However, these sources, with their vast amounts of information, also carry the risk of providing biased, flawed, invalid, or potentially harmful content (17). Since platforms like YouTube currently lack protective measures such as expert reviews and inspections, mitigating the risk of inaccurate information relies on the vigilance of users and, if they choose to recommend such resources, healthcare professionals (18). Research efforts to evaluate the quality of healthcare information on YouTube are expanding and becoming more prevalent across various medical fields (10, 13, 15, 19). There is still no data available on the quality of videos uploaded about inflammatory muscle diseases. Moreover, most of the studies seem to solely focus on patient education. However, there seem to be many healthcare professionals, especially students (7, 20). Therefore, we have decided that our analyses should also cover them as well, and potentially lead them to better sources of information.
One of the aims of our study was to define the characteristics and the sources of videos aimed at healthcare professionals. Compared to the videos aimed at patients, they seemed to have fewer numbers of views, likes numbers of comments, and longer durations. As the patients are expected to form a bigger portion of information seekers through YouTube, these findings are not unusual. High-quality and intermediate-quality videos for the education of professionals were found to be coming from physicians or academic/university sources, with 2 intermediate quality videos from the patients themselves. Thus, health professionals should seek these sources if they wish to get better quality information, still keeping an eye out for the videos of the patients as well, as they can also find out precious insights that are beyond the books.
The videos that may address the patients were also assessed for their sources. These videos constituted the more popular and shorter videos in our study, with more views and shorter durations. The videos that were prepared by the physicians were found to have a high quality, with more than 80% of the videos being of intermediate or higher quality. A portion of the videos that were created by patients was also found to be of worthy quality, which hints to the users to give a chance at the patients who speak for themselves. While in little numbers, videos prepared by non-physician health personnel were rarely found to have a high quality, and rare videos from independent users were found to have high-quality videos, mostly. The literature for various medical conditions is also similar, since most of the higher-quality videos belong to healthcare professionals, mostly physicians, in that field (21, 22). These findings indicate that the patients may have a chance to find better quality videos if they sought videos prepared by the doctors. This also indicates that physicians should work on preparing more videos as well as academic sources such as universities, which may give even better results with a more professional approach.
Determinants of the quality of the videos were evaluated and defined for both videos that may appeal to the professionals and the patients. Our study yielded no determinants of the quality of the videos for the patients, and neither views, likes, comments nor the duration of the video were useful to predict the quality of a video. However, the number of likes was found to be useful to predict whether a video was of a higher quality for the videos aimed at healthcare professionals. Therefore, this audience may use this detail to seek better-quality videos. Similarly, neither comments, views nor durations were of use to predict the quality of these videos. Looking from the other side, the DISCERN scores of the higher-quality videos were significantly higher for both audiences. Thus, as the quality of a video increases, the information provided by the video gets more reliable and valid. These findings were similar to the ones found in the literature and back up the fact that the quality and the reliability of the information often coexist (13, 23).
The strengths of the study include the use of multiple evaluators to assess the videos, which gave excellent agreements on inter-rater correlations. Additionally, validated and reliable scoring systems, which have been widely used in previous research on various topics, were employed to assess the videos.
However, the study has several limitations. The videos were selected and evaluated at a single point in time, which may not represent the dynamic and ever-changing nature of YouTube. While the GQS has been extensively used in such studies, it remains a subjective scoring system, and finding an objective alternative is challenging. Furthermore, the video search was limited to those in English, so results may differ across other languages and regions.
Conclusion
With its increasing access, the internet serves as a source of information for almost anything, including health and education. YouTube is one of the most accessed and popular websites and has a mixed pool of videos regarding the quality and reliability for both patients and health professionals for inflammatory muscle diseases. Since better reliable information of higher quality matters in a platform where anyone can upload almost anything, academics, professional organizations, universities, associations, and healthcare practitioners should upload more videos to provide reliable, practical, and high-quality health-related information. Better education for the patients may help them adhere to the treatment and hopefully improve the quality of their lives.
