ABSTRACT
Objective:
This study aims to evaluate what information is given in Turkish videos about osteoporosis on YouTube and to determine the quality and reliability of the videos.
Materials and Methods:
The first 50 Turkish videos about osteoporosis on YouTube were evaluated in terms of quality, reliability, and information level. Two searches were conducted for related content on YouTube and two orthopedics surgeons evaluated the videos simultaneously. The Global Quality score (GQS) (1-5) and DISCERN (15-75) scoring systems were used to assess the quality of the video content. With the Osteoporosis Specific scale (1-29), it was questioned what information specifically about osteoporosis was given in the videos while the accuracy of the video source information was evaluated with the Journal of American Medical Association (JAMA) score (1-4). Descriptive data regarding the number of views, comments, likes, dislikes were recorded, as well as the upload date to YouTube and the duration of the videos. The popularity of videos was evaluated with the video power index.
Results:
Considering the messages given in the videos, the most frequent information was “decrease in bone mass” with 41 videos. This was followed by “osteoporosis is a risk of fracture” and “there are risk factors for osteoporosis.” The Osteoporosis Specific score was low 8.92. The mean DISCERN score was 25.020 (15-75) and the mean GQS was 1.98 (0-5), indicating low quality. The JAMA score (1-4) for which the video source was questioned showed a low level of reliability of 1.66. Videos about osteoporosis prepared by people other than healthcare professionals were more popular (82.25 vs. 56.80) (cc=0.296, p=0.037).
Conclusion:
The content of the videos on YouTube osteoporosis is generally inadequate or inaccurate. Higher quality and informative videos based on international guidelines can contribute to patient compliance and increase public awareness of osteoporosis.
Introduction
Osteoporosis is a progressive bone disease characterized by decreased bone density and deterioration in the microarchitecture of bone structure. It is usually asymptomatic and presents with fractures. The prevalence of osteoporosis increases with age and although it is known as a disease of the elderly, it can also occur in younger patients (1-3).
The presence of osteoporosis is known in 200 million women worldwide (4). According to the National Health and Nutrition Examination Survey conducted by the National Center for Disease Control and Prevention Health Statistics, it is estimated that more than 9.9 million Americans have osteoporosis and 43.1 million Americans have low bone mass (3,5). Women are affected by osteoporosis at a rate of 4/1 compared to men (3). In 2009, the prevalence of osteoporosis over the age of 50 in Turkey was 7.5% in men and 12.9% in women (6). In the Thrace Region, the prevalence of osteoporosis is 15.1% in women over 40 years old and 10.7% in men, while it reaches 25.7% over 55 years of age (7).
12% of patients with a history of fractures due to osteoporosis break another bone within one year, and 25% within five years. Bone resorption has a negative impact on the quality of life of patients in general. In addition, fractures caused by osteoporosis also create an economic burden for patients’ relatives and the health system (8). Osteoporotic fractures cost more in women over 55 years of age than myocardial infarction, stroke, or breast cancer (1,3). One out of every three patients with hip fractures who lived independently before need care within at least one year after the fracture (9). And one-fifth of these patients die within a year (5). For this reason, it is necessary to provide the necessary information and inform the patients in order to prevent osteoporosis and reduce the risk of falling.
It has been reported that 75% of people at risk for osteoporosis do research on their health on the internet (10). Based on information obtained from YouTube, the site is visited by more than one billion internet users every month and YouTube has become one of the most popular video-sharing websites (11). This rich content makes YouTube a huge online video library. Although easy access to information on YouTube seems to make life easier, the lack of verified sources and an expert-peer review process are important problems. This means that it is necessary to review the reliability quality and of the videos on YouTube.
The aim of this study is to determine the level of information about osteoporosis in Turkish videos on YouTube and to determine the quality or reliability of these videos.
Materials and Methods
In this cross-sectional study, Turkish YouTube videos about osteoporosis were evaluated. Google trends (https://trends.google.com/trends/?geo=TR) search terms were used to select the videos. A search was made for the word “osteoporosis”, and it was listed in the filters by setting “Turkey” as the region, “2008-Today” as the date, and “YouTube search” section. Turkish terms in the results were -in order of frequency- as follows: “kemik erimesi” (bone loss), “kemik erimesi belirtileri” (signs of bone loss), “kemik erimesine ne iyi gelir” (what is good for bone loss), “osteoporoz” (osteoporosis) and “kemik erimesi neden olur” (what causes bone loss). The resulting key terms were used when searching on YouTube.
Two searches were made on YouTube on May 8, 2021, in Tekirdağ, Turkey. Searches were made using a web browser with cleared history and cookies. Without logging in on YouTube and searches were made with the “sort by relevance” option selected. The first 50 URLs obtained were saved for each search term. The resulting videos were evaluated simultaneously by two orthopedic surgeons. The inclusion criterion was: Turkish videos with osteoporosis-related content. The exclusion criteria were: Videos that did not address the primary topic or had no audio or subtitles. Repeated videos were not evaluated. Data on the number of views of the videos, the number of comments, the number of “likes”, “dislikes”, the date of upload to YouTube, and the total duration of the video were recorded.
Global Quality score (GQS) (1-5) and DISCERN (15-75) scoring systems were used to determine the quality of video content (12,13). The accuracy of the video source information was evaluated with the Journal of American Medical Association (JAMA) score (1-4) (14).
DISCERN scale is a scoring system developed in Oxford, United Kingdom, which aims to measure written health information consisting of 15 questions in total and one additional question for overall evaluation (13). In the DISCERN scoring system, each question is scored between 1 and 5 points and the total score is between 16-75. Scores are evaluated as very poor between 16-26 points, poor between 27-38 points, fair between 39-50 points, good between 51-62 points and, excellent between 63-75 points (13). The JAMA scoring system consists of 4 criteria (authorship, attribution, clarity, currency), with 1 point for each and a maximum of 4 points. 1 point indicates the lowest quality information and 4 points the highest quality information (14). The GQS consists of 5 questions for evaluating the general quality and educational level of the content. In the scoring system, 1 point indicates poor quality and 5 points indicates excellent quality (12).
What information was provided in the videos was checked using a checklist based on international control guidelines called Osteoporosis Specific scale (3,8). If the information was mentioned in the video, a score was given for the presence of each message, ranging from 0 points to 29 points.
The popularity of videos was evaluated by view rate and video power index (VPI). View ratio was calculated using (number of views/time since upload) formula. The formula [number of likes×100/ (like+dislike)] was used to calculate the like ratio. VPI was calculated using the (like ratio×view ratio/100) formula (12). In particular, VPI was preferred to evaluate viewer-video interaction and to avoid YouTube’s ranking algorithm parameters that may contain commercial concerns.
This article does not contain any studies with human participants or animals performed by any of the authors and no ethical approvement is required for this study.
Statistical Analysis
IBM SPSS Statistics Version 25 (IBM Corp. Armonk, NY, USA) was used for statistical analysis. Descriptive statistical methods were used to evaluate study data. Whether the data were normally distributed or not was evaluated using the Shapiro-Wilk test and graphical examinations. The Mann-Whitney U test was used to compare two groups of quantitative variables that did not show normal distribution. Spearman correlation test was used to determine the correlation between variables. Chi-square test was conducted for each messages counting over the number of 5 to evaluate association between the groups and Fisher’s Exact test for the rest. Inter-observer agreement was determined by Interclass Correlation Coefficient (ICC). P-value of <0.05 was considered statistically significant.
Results
ICC was found to be high with a value of 0.922 and no significant difference was observed between observers.
There was a moderate correlation between DISCERN, JAMA (cc=0.437, p=0.002). There was also a very strong correlation between DISCERN, GQS (cc=0.843, p<0.001), and a weak correlation between GQS, JAMA (cc=0.379, p=0.007). In addition, strong, strong, and moderate correlations were observed with the Osteoporosis Specific score and DISCERN, GQS and JAMA scores, respectively (cc=0.742, 0.752, 0.385, p<0.05, respectively) (Table 1).
No correlation was found between DISCERN, JAMA, GQS, and VPI. In addition, there was a weak negative correlation between the Osteoporosis Specific score and VPI (cc=-0.323, p=0.022). It is concluded that the popularity of the video decreases as the information content of the video increases.
Videos prepared by healthcare professionals and non-healthcare professionals were compared with the Mann-Whitney U test in terms of scoring systems, VPI, and number of views. Although there was no significant difference between DISCERN, GQS and JAMA values, it was high in favor of healthcare professionals. The Osteoporosis Specific score was significantly higher in videos prepared by healthcare professionals (p=0.004, 10.58 vs 5.72). However, the number of views (16.425 vs 116.919, p=0.016) and mean VPI (56.80 vs 82.25) (cc=0.296, p=0.037) values in terms of popularity were higher in the other group. Significant and weak correlation was observed between VPI and video sources. The average VPI value of videos of non-healthcare professionals (n=14) was significantly higher than that of healthcare professionals (n=36). It was observed that the popularity of non-healthcare professionals in osteoporosis videos was higher (Figure 1).
No significant relationship was found between the like ratio and the video source (p=0.514). A weak correlation was observed between the view ratio and the video source, and the view ratio of healthcare professionals was low (p=0.035, cc=0.299).
In order of frequency, the messages given in the videos are “Decrease in bone mass” in 41 videos (82%), “Osteoporosis is a risk for fracture” in 37 videos (74%), “There are risk factors for osteoporosis (family history, age, sex, etc.)” in 36 videos (72%), “Sufficient calcium intake” in 32 (64%) videos and “Sufficient intake of vitamin D” in 31 (62%) videos (Table 2).
The least given messages were from low to high with 0 videos “Self medicating should be avoided”, 1 video with “Prolonged treatment”, 2 videos with “The importance of continuation of treatment even if there are difficulties”, 3 videos with “Vertebral images” and “Rule out fragility fractures” and “Specific bone disease”. Messages about the definition and diagnosis of osteoporosis and recommendations were found most frequently, while messages for treatment and follow-up were found to be less. The number of information given in Recommendations (133) and Definition categories (130) were more frequent than Diagnosis (95), Treatment (82), and Follow-up (6) categories (Figure 2).
For the messages over number of 5 chi-square test was conducted and significant difference was observed between the groups (p<0.001). Also Fisher’s Exact test was conducted for messages under number of 5 and significant difference was observed (p<0.001). While the number of messages was more than 5 in the health professionals group, it was the opposite in the other group. To exclude that non-homogenity, percantages were used and weighted as frequencies and chi-square test was conducted for all. The results were similar and consistent with the previous tests (p<0.001). Also for most common messages (Decrease in bone mass and Osteoporosis is a risk for fractures), 2x2 contingency table was used and chi-square test were conducted. There was no significant difference between the groups (p=0.741).
The mean DISCERN score was low with 25.020±6.625 (16-46), while the JAMA score was 1.66±0.658 (1-3) and the GQS was 1.98±1.097 (1-5). The Osteoporosis Specific score was found to be 8.920±5.91 (1-29). Descriptive statistics are presented in Table 3.
Discussion
The main finding of our study was that the Turkish videos about osteoporosis were of insufficient quality and did not provide sufficient information according to the scoring systems used.
Rozenfeld et al. (15) conducted a survey-based study involving 3,000 women aged 50-85 years and investigated what information about osteoporosis was searched on the internet and what surveyors wanted to found. They found that middle-aged women were more interested in the management of the disease on internet-based information. In line with this interest, more messages were given in the recommendations part of the videos and our results are consistent with the study.
While there are various scoring systems to evaluate the content of YouTube videos, there is no standard approach (16). The information about healthcare videos on YouTube is generally inaccurate and a patient is highly likely to find such misleading information (17). In a study, Gutlapally et al. (18) found that only 11 of 45 sites had reliable information about osteoporosis. This can be dangerous as it may affect public opinion and primary perception of the disease and the management. In addition, people trying to learn about osteoporosis on the Internet may also be affected when they receive false feedback from comments, which can lead to the adoption of behaviors that may affect treatment, such as lack of compliance (19).
In our study, it was questioned which of the 29 important messages (Table 1) were given in Turkish videos with the Osteoporosis Specific score, and it was seen that messages were given at a low rate with an average of 8.92. This is similar to the results of other studies and shows that the messages given are incomplete (8,18). But also Tejada-Llacsa et.al (8) found that most mentioned information in Spanish videos about osteoporosis was “Osteoporosis is a risk factor for fractures”. In contrast with that “Decrease in bone mass” was the most common information given in the videos in our study, and this was the most common search criterion with “kemik erimesi” (bone loss), which was accepted as a similar term in the Turkish society and indicates a decrease in the bone mass.
There are numerous studies investigating the quality of medical information videos on YouTube (11,12,16,17,20,21). In these studies, medical information videos on YouTube were of low quality.
In 2014, Brooks et al. (20) reviewed lumbar discectomy videos-which are an informative resource for patients on YouTube. Recently, Celik et al. (21) examined the information about rotator cuff injury on YouTube and found that it was of poor quality.
In our study, the DISCERN score was 25.020±6.625 (16-46), while the JAMA score was 1.66±0.658 (1-4) and the GQS was 1.98±1.097 (1-5). These results are consistent with the results of other studies in the literature and show that Turkish videos about osteoporosis are of low quality. In terms of messages separately presented in Table 2, there were significant differences between health-care prefesional and non-healthcare professionals (p<0.001). But in terms of the most common two messages (Decrease in bone mass and osteoporosis is a risk for fractures) there was no significant differences (p=0.741). It can be assumed that although the total number of messages were mentioned more in videos prepared by health-care professionals, the most common messages were given almost in the same manner between the two groups. In addition, there was a weak negative correlation between the Osteoporosis Specific score and VPI (cc=-0.323, p=0.022). The average VPI value of videos of non-healthcare professionals was significantly higher than that of healthcare professionals (82.25 vs 56.80) (cc=0.296, p=0.037) and also the view ratio of healthcare professionals was low (p=0.035, cc=0.299). Also mean Osteoporosis Specific score was significantly higher in healthcare professionals compared to other group (p=0.004, 10.58 vs 5.72). Although the video is more informative, it has been concluded that this situation does not increase the popularity of the video in the first place. Welbourne and Grant (22) analyzed 390 videos from 39 YouTube channels to explore factors affecting video popularity and found that user-created videos were more popular than those uploaded by professionals. It is mentioned that popularity is related to more interaction in user-sourced videos. It can be thought that the low popularity may be related to a proposition that health professionals are less interactive in videos. However, our study specifically examined videos in the field of health and the trust of the viewers towards the healthcare professional can be effective in their video selection. Viewers’ video choices in the field of health can be multifactorial, and this could be the subject of another study.
In conclusion, our study shows that the number of information in Turkish videos about osteoporosis and the video quality is low, which may cause people at risk to have incomplete information about osteoporosis and create a challenging environment for the patient-doctor relationship. Because it is easy and inexpensive to access the Internet, patients tend to obtain medical information from the Internet (19). For this reason, videos shared on a platform such as YouTube may be beneficial to be verified by an expert in order to ensure the optimum patient-doctor relationship, especially if the video is about health care or better quality videos may be recorded by healthcare professionals.
Our study has several strengths. First of all, our study is the first study to examine Turkish videos on osteoporosis, and to our knowledge, there are only three studies in the literature examining Turkish videos overall (23-25). YouTube searches are restricted to Turkey, which also provides information specific to the Turkish society’s approach to osteoporosis-related videos. The examined videos were in the native language of the surgeons without language a barrier and the interrater agreement was quite high. In addition, when compared with other studies in this field, multiple scoring systems evaluating quality and also a scale assessing what information specific to osteoporosis were used together and analyzed (8,11,12,16,17,20,21).
There are also some limitations of this study. Firstly, YouTube is a growing platform and the search results can change over time. Secondly, the first 50 videos that appeared after searching for the keywords were examined. However, we think that the videos that appeared at the top were watched more. Thirdly, we only examined Turkish videos. Although this is a limitation, it can also provide a cross-sectional benefit in terms of examining videos for Turkish Society, which we see as an important aspect. Fourth, only YouTube videos were evaluated in this study, and the quality and reliability of osteoporosis-related videos on other sites were not covered. Despite the limitations, we believe that our study shows beneficial information about the educational quality of videos and YouTube should be considered as a platform to improve public information and perception of osteoporosis.
Conclusion
Turkish YouTube videos about osteoporosis contain incomplete or incorrect information about osteoporosis and the quality of the videos is low. Especially, the videos prepared by healthcare professionals based on international guidelines and scoring systems can increase the quality of these videos. Considering that osteoporosis is a progressive and silent disease, as well as other platforms, it is important to provide preventive and informative videos on YouTube.
Ethics
Ethics Committee Approval: No ethical approvement is required for this study.
Informed Consent: This article does not contain any studies with human participants or animals performed by any of the authors.
Peer-review: Externally peer-reviewed.
Authorship Contributions
Concept: Y.M.D., E.C, Design: Y.M.D., Data Collection or Processing: E.C., M.A., M.M., B.K., Analysis or Interpretation: E.C., M.A., E.G., S.Ç., Literature Search: E.C., E.G., S.Ç., M.M., B.K., Writing: Y.M.D., E.C., M.A.
Conflict of Interest: The authors confirm that there are no known conflicts of interest associated with this publication.
Financial Disclosure: The authors declared that this study has received no financial support.