Abstract

Objective

In this study we aimed to investigate the reliability and quality of YouTube videos on familial mediterranean fever (FMF).

Materials and Methods

A total of 42 most viewed videos related to FMF were included in this study. The videos included were divided into two groups: professional and non-professional videos. The objective assessment was conducted using two commonly used DISCERN and global quality scale (GQS) instruments.

Results

Twenty-nine (69.0%) videos were uploaded by professionals (physicians, health channels) and 13 (31%) videos were uploaded by non-professionals. The mean video length was found as 13.5±12.7 minutes in the professional videos and 6.28±12.6 minutes. Based on the results of the DISCERN scoring, the reliability of the YouTube videos on FMF was poor in 17 (40.5%) videos, moderate in 4 (9.5%) patients, and good in 21 (50%) videos. According to the GQS results, the quality of the videos was poor in 12 (28.6%) videos, moderate in 4 (9.5%) videos, and good in 26 (61.9%) videos.

Conclusion

The reliability and quality of videos uploaded by healthcare professionals are higher. Healthcare professionals, including physical therapy and rehabilitation specialists and rheumatologists, should be encouraged to upload reliable video content to appropriately guide patients.

Introduction

Familial mediterranean fever (FMF) is an autosomal recessive inherited inflammatory disease characterized by fever and inflammation in pleura, peritoneum, skin or joints (1). The typical phenotype of FMF includes self-limiting fever and polyserositis, dermal involvement, arthritis and high acute phase response (2). The disease is caused by mutations in the MEFV gene that is especially prevalent in the Mediterranean basin (3). FMF is the most common hereditary autoinflammatory disease worldwide. FMF is most commonly seen in Jewish, Armenian, Turkish and Arab societies (1). In these tehnic groups, the incidence of FMF is 1/200-1000 with a higher carrier rate of 1/3-5 (4). Although the disease mainly affects populations from the East Mediterranean population, patients are reported from every region of the world due to travel and immigration (5, 6).

Although patients with FMF experience similar main manifestations of the disease, the disease experience differs among persons. The duration, severity and personal effects of symptoms are unpredictable, leading people to seek more information about their disease. The Internet and particularly YouTube are the first researched information sources for this purpose. Recently, the Internet has become a major source of health related information (7). Recent survey studies have found that 8 of each 10 Internet users access health related information online (8). Particularly patients with chronic diseases are increasingly relying on the Internet to gain more insight into their diseases and to manage their conditions (9).

YouTube is the most commonly used Internet sharing platform with 2 billion monthly active users, 15 million content creators and 38 million active channels as of 2021 [https://invideo.io/ blog/youtube-statistics/ (accessed: 09/11/2021)]. YouTube has a potential for sharing and disseminating health related information as well as as a tool for diagnostic aid and education (10). However, 86% of the online health seekers have concerns about the reliability and quality of health related information they obtained via the Internet, and especially YouTube (11). In addition, healthcare providers and government agencies have also expressed concerns about the quality of the information on this platform due to lack of any guidelines and/or intervention regulating the uploaded material on YouTube. This issue raises questions about the risk of disseminating misleading health related information (12). These concerns have prompted researchers to conduct analysis studies in order to evaluate the reliability and quality of YouTube videos in every field of medicine. A total of 58 YouTube analysis studies has been published only within the last year in PubMed [https://pubmed. ncbi.nlm.nih.gov/?term=youtube&filter=datesearch.y_1 (access date: 15/11/2021)]. Numerous studies have drawn attention to the importance of misleading information in YouTube videos on rheumatic diseases (13-16). However, to our knowledge, there is no study evaluating the content, reliability, and quality of YouTube videos regarding FMF. Therefore, in this study we aimed to investigate the reliability and quality of YouTube videos pertaining to FMF.

Materials and Methods

The YouTube website was searched for the term “familial mediterranean fever” by two physical medicine and rehabilitation specialists on 10/11/2021. From the filtering feature of YouTube, the “view count” option was selected and the most viewed videos were listed. The search yielded a total of 110 videos pertaining to FMF. Considering that English is viewed as a universal language by many countries in the world, only English videos were included. (10) Irrelevant videos (healing music, mits etc.), non-English, duplicate, and advertisements videos were excluded from the study. As a result, the remaining 42 most vieved videos were included in the study. Since there were only 110 videos regarding FMF, we analyzed all the YouTube videos on this issue. The links of these 42 videos were entered into a Microsoft Excel file and analyzed by the two researchers. The inclusion flowchart of the videos is shown in Figure 1.

The most common methodology in YouTube analysis studies is to focus on a fixed sample size such as “the most commonly viewed 50 videos” or “the first 100 videos” (17). Unlike these studies, we included all possible videos on FMF, but after the exclusion process only 42 relevant videos remained and we completed our analysis with these videos. In order to avoid bias, the evaluation was performed by the two researchers separately in different rooms, but at the same time.

The 42 videos’ length, image type, content, qualification of the uploaders, date of upload, time since the upload, view count, the number of daily views, like, dislike and comment counts were recorded. Popularity of an video was determined using the video power index (VPI) as described in the previous studies according to the following formula: (18).

VPI = (like count x 100/ [like count + dislike count]) x 100

The videos included in the study were divided into two groups according to the qualification of the uploaders as professional and non-professional videos and the variables were compared between these two groups. The professional videos included general information about the FMF, while the non-professional videos mainly included patient experience.

Evaluation of the videos was carried out on both a subjective and objective basis. In the subjective evaluation, the two researchers assessed the videos as useful and misleading. The videos containing scientifically unproven information were considered as misleading and those involving scientifically accurate information about the symptoms, clinical presentation, diagnosis, differential diagnosis, treatment, prevention of FMF etc. were accepted as useful. The decision was made with agreement between the two researchers. The objective assessment was made using two commonly used tools DISCERN and global quality scale (GQS). The reliability of the videos was evaluated with DISCERN, while the quality of the videos was evaluated using the GQS. These two instruments have been used commonly for the evaluation of YouTube videos in the previous studies (7, 16, 19).

DISCERN scoring: The DISCERN is a scoring tool used for assessing the reliability of consumer health information on treatment options. In this study, we used the shortened DISCERN tool that was adapted by Singh et al. (13) from the original form. The DISCERN scale includes 5 items scored using a 5-point Likert scale. The DISCERN items investigate the reliability of information sources, additional sources, aims, bias and areas of uncertainty. A video content considered good for DISCERN scores >3 points, moderate for a DISCERN score of 3 and poor for DISCERN scores <3 points (Figure 2) (20).

GQS: GQS, which was developed for the first time by Bernard et al. (21) is used to assess the quality of video contents based on the usefulness of the information presented. GQS consists of 5 items that question quality, ease of use and flow of the examined video contents with a 5-point Likert scale. The quality of a video content is scored between 1 point (very poor) to 5 points (excellent) (Figure 3).

Publicly available videos were evaluated for this study. Additionally, since no human participants or test animals were included in this study, ethics committee approval was not required. The study does not require patient consent.

Statistical Analysis

The statistical analysis of the data obtained in this study was carried out using SPSS version 24.0 (SPSS, Statistical Package for Social Sciences, IBM Inc., Chicago, IL, USA) statistical software. Normality of the data was analyzed with the Shapiro-Wilk test. Non-normally distributed continuous variables are given as mean ± standard deviation and categorical variables as frequency (n) and percentage (%). Mann-Whitney U was used for the comparison of numerical variables, while chi-square test was used for the comparison of categorical variables between the videos uploaded by professionals and non-professionals. Cronbach alpha coefficients were used to determine the inter-rater agreement consistency. P<0.05 values were considered statistically significant.

Results

YouTube search on FMF returned a total of 110 videos. Of these, 26 irrelevant videos, 19 non-English videos, 16 duplicate videos, 6 ads and 1 not-working videos were excluded and the remaining 42 videos were subjected to the analysis. Twenty-nine (69.0%) videos were uploaded by the professionals (physicians, health channels, nurses) and 13 (31%) videos by the non-professionals (patients, others). As a result of the subjective evaluation of the videos about FMF, 25 (59.5%) videos were found to be useful and 17 (40.5%) videos misleading. Uploaders and general quality of the videos are shown in Figure 4. Image type was found as animation in 1 (2.4%) video, presentations in 19 (45.2%) videos and real images in 22 (52.4%) videos. Video contents were general information about FMF (including symptoms, clinical presentation, diagnosis, treatment, prevention) in 24 (57.1%) videos, congress presentation in 7 (16.7%) videos, patient experience in 6 (14.3%) videos, myth about FMF in 2 (4.8%) videos, case presentation in 1 (2.4%) video, survey about the awareness of FMF in 1 (2.4%) video and relationship between FMF and coronavirus diseaese-19 (COVID-19) in 1 (2.4%) video.

The mean video length was found as 11.27±12.68 minutes. The mean video length was found as 13.5±12.7 minutes in the professional videos and 6.28±12.6 minutes in the non-professional videos. The mean video length was statistically significantly longer in the videos uploaded by professionals (p=0.024). The general characteristics of the reviewed videos are presented in Table 1.

The most viewed video was uploaded by a health channel on November 14, 2012 and included clinical information about FMF that was viewed 36.281 times. The most liked video was uploaded by a physician on December 12, 2020 and included general information on FMF that received 665 likes. The number of views was statistically significantly higher in the videos uploaded by the professionals (4629.59±7819.47) compared to the videos uploaded by the non-professionals (2831.15±3552.77) (p=0.014). The mean daily view was found as 5.41±12.02 in the videos uploaded by the professionals and 1.87±3.71. The mean daily views count was significantly higher in the videos uploaded by the professionals (p<0.001).

The mean VPI value was found as 91.21% in all videos. The mean VPI value was calculated as 95.62% in the useful videos and 85.14% in the misleading videos. The mean VPI value was statistically significantly higher in the videos evaluated as useful (p<0.001). The mean DISCERN score was found as 3.11±1.20 and the mean GQS score as 3.4±1.21 for all videos. The VPI, DISCERN and GQS scores according to the uploaders and general video quality are given in Table 2.

Based on the results of DISCERN scoring, reliability of the YouTube videos on FMF was found as poor in 17 (40.5%) videos, moderate in 4 (9.5%) patients and good in 21 (50%) videos. According to the results of GQS, quality of the videos was found as poor in 12 (28.6%) videos, moderate in 4 (9.5%) videos and good in 26 (61.9%) videos (Figure 5).

The mean DISCERN score given by the researcher 1 was found as 3.02±1.37 and the mean DISCERN score was given by the researcher 2 was found as 3.19±1.15. The mean GQS score given by the researcher 1 was found as 3.43±1.43 and the mean GQS score given by the researcher 2 was found as 3.55±1.06. Accordingly, there was a good agreement between the two independent researchers in terms of the DISCERN and GQS scores (Table 3).

Discussion

The Internet is the third most trusted source of health related information following physicians and official health care institutions and is considered more reliable compared to the traditional media (22). Hay et al. (23) reported that 87.5% of the patients with rheumatic disease refer to the Internet to obtain information about their condition before seeking medical help from physicians. Furthermore, it has been reported that patients with more pain tend to resort to the Internet more frequently (23). Because FMF is a painful illness, it is not unusual for these patients to seek information for their condition online. YouTube is an increasingly important source of health related information with a potentişal to influence its users (24). However, the information shown on YouTube lacks scientific inspection and supervision, because everyone can upload such content for free of charge and without being scientifically reviewed. For this reason, YouTube may contain many videos that may be misleading (25).

In the present study, we evaluated the reliability and quality of 42 YouTube videos pertaining to FMF with a total view count of 171,063 and a whole duration of 7.9 hours. In a study by Onder and Zengin (7) on YouTube videos pertaining to gout, the mean views count was reported as over 16 million. The difference may be attributed to FMF being a more specific disease with some ethnic origins. In our study, 69% of the videos were uploaded by health care professionals (physicians, health channels, nurses) and 31% of the videos by non-professionals (patients and others). Majority of the videos uploaded by non-professionals included patient experience as video content. Whereas, the YouTube videos uploaded by the professionals mostly contained general information about FMF, including symptoms, clinical presentation, diagnosis, treatment and prevention. The rate of videos uploaded by health care professionals varies among the studies depending on the characteristics of the disease being searched. Onder and Zengin (7) investigated YouTube videos as a source of information on gout and found that 97% of the videos were uploaded by the professionals (physicians, academic organizations, other healthcare professionals). In our previous YouTube analysis study about Behçet’s disease, 46% of the videos were uploaded by the professionals (16). In another study evaluating 42 YouTube videos regarding male infertility, 71.4% of the videos were uploaded by healthcare professionals (26). It is seen that the rate of the videos uploaded by healthcare professionals varies widely among the studies, mainly depending on the health topic searched. Because FMF is an autosomal hereditary inflammatory disease that lasts for a long time, the higher rate of videos uploaded by the professionals compared to patient experience is expected.

In this study, the general quality of the videos was assessed as “useful” or “misleading” based on the consensus achieved by the two researchers. Accordingly, 59.5% of the videos were evaluated as useful and 40.5% as misleading. In a recent study by Andika et al. (27), evaluating the YouTube videos on the COVID-19 pandemic, 23% of the videos were reported to be misleading and 44% useful. The difference between the studies may be resulted from the fact that subjective evaluation of the videos is primarily based on the type of the disease assessed.

In our study, the mean VPI value that shows popularity of a video content was found as 95.62% in the professional videos and 85.14% in the non-professional videos. In the present study, we clearly found that view and like counts were significantly higher in the videos uploaded by the professionals. DISCERN scoring is an important tool indicating the reliability of the videos based on several criteria as mentioned above. According to the DISCERN scoring, the reliability of YouTube videos was poor in 40.5% of the videos. The mean DISCERN score was found as 3.62±1.20 for the videos uploaded by the professionals and 1.96±1.22 for those uploaded by the non-professionals. The significant difference was mainly due to the videos containing patient experiences about FMF that do not make any contribution to the management of the disease. Similar to our results, in the study by Onder and Zengin (7), the mean DISCERN score was found as 2.0 in the videos uploaded by the non-professionals and 3.75 in those uploaded by the professionals. In another study analyzing YouTube videos pertaining to dysphagia, the mean DISCERN score was found as 1.6±1.14 in the low-quality videos and 3.39±0.74 in the high-quality videos (28). In another study by Aydın and Yılmaz (29) investigating YouTube videos about echocardiography, the mean DISCERN score was found as 3.0 for all videos. The results of our study and those of the other studies show higher DISCERN scores for the videos uploaded by the healthcare professionals, that mean a higher reliability.

Quality of the YouTube videos on FMF was measured using the GQS scale. Accordingly, the mean GQS score was found as 4.14±1.21 for the videos uploaded by the professionals and 2.04±1.25 for those uploaded by the non-professionals (p<0.001). In a study by Chang and Park (30) evaluating the most viewed 50 YouTube videos about epidural steroid injection, the mean GQS score of all videos was found as 2.3±1.1. In our previous study on Behçet’s disease, the mean GQS score was found as 4.09±0.72 in the videos uploaded by the professionals and 3.41±0.69 in those uploaded by the non-professionals (16).

Cohen’s kappa coefficient for inter-rater agreement was found as 0.745 for the DISCERN score, and 0.896 for the GQS scores, indicating near-perfect agreement for both. In the study of Onder and Zengin (31) Cohen’s kappa statistic demonstrated an inter-observer agreement of 0.925. In another study by the same author on the validity of health-related information on psoriatic arthritis, inter-rater agreement was 0.783 for the DISCERN score, and 0.862 for the GQS score. In this context, our finding is consistent with the literature.

There are a lot of health related YouTube videos about a wide diversity of diseases and medical conditions. When the previous studies were reviewed, the common aspect of all studies was the fact that the reliability and quality of the video contents uploaded by lay persons were poor. In our analysis there were even healing music videos for treating FMF. This indicates the necessity of regulations for uploading health related content in the Internet and particularly YouTube.

Study Limitations

Major limitation of the study was including a snap-shot evaluation of the videos at a certain time. Whereas, online information can be uploaded or removed at any time, and even simple searches can give different results depending on the fluctuating popularity of the video content. In addition, the number of viewed videos was relatively small. In addition, to minimize subjective bias in video scoring, future studies may include more reviewers of varying backgrounds such as patients as healthcare consumers and different age groups. It is remarkable that the number of all available videos about FMF was only 110, and 42 videos were eligible for the analysis. Finally, since YouTube video studies are on a wide range of diseases, direct comparison of the results are affected by the disease specific characteristics. For example, since popularity of COVID-19 is much higher than the other medical conditions, it is obvious that comparison of the view, like, dislike counts etc. between nay other disease and COVID-19 would be challenging.

Conclusion

A considerable portion of YouTube videos pertaining to FMF are of poor quality. The reliability and quality of the videos uploaded by health care professionals are higher. There is an urgent need for regulations/policies for health related YouTube videos. The health related videos on YouTube should be subjected to a professional review process before publishing. Health care professionals, including physical therapy and rehabilitation specialists and rheumatologists should be encouraged to upload reliable video contents to guide patients appropriately.