Educational value of surgical videos on YouTube – quality assessment of and set of standards for hysterectomy using vaginal natural orifice transluminal endoscopic surgery videos

Overview

Objective: This study aimed to assess the reliability and educational value of vaginal natural orifice transluminal endoscopic surgery (vNOTES) hysterectomy videos on YouTube and their suitability for training surgeons. Materials and methods: On June 12, 2024, YouTube was searched using the keywords “vNOTES hysterectomy,” “TVNOTES hysterectomy,” “transvaginal natural orifice transluminal endoscopic hysterectomy,” “vNOTES,” and “vaginal notes hysterectomy.” A total of 73 videos met the inclusion criteria. Viewer engagement metrics, such as time since upload, number of views, likes, dislikes, comments, and video duration were recorded. Ratios such as a view ratio, a like ratio, and Video Power Index (VPI) were calculated. The videos were categorized by the modified Global Quality Scale (GQS) and evaluated based on a scoring system derived from a standardized 10-step vNOTES hysterectomy procedure, with scores ranging from 0 to 15. Results: Out of 73 videos, 40 (53.8%) were categorized as poor quality, 13 (17.8%) as moderate, and 20 (27.4%) as good. No significant differences were found between groups in terms of time since upload, views, dislikes, comments, or a like ratio. However, videos in the good-quality group had a significantly higher number of likes and VPI scores. Critical elements such as patient preparation and positioning, setup of the operation room, circumcision of the cervix, and vault closure were inadequately addressed in lower-quality videos. Videos with a didactic voice had significantly more views, likes, and comments than those with music or no sound. No significant correlations were found between video length and engagement metrics. Conclusion: The majority of vNOTES hysterectomy videos (53.8%) on YouTube lack comprehensive educational content, with only a small fraction deemed appropriate for surgical training. The interest rates of the viewers may not be correlated with the usefulness rates of the videos. Surgeons and organizations should focus on producing high-quality, peer-reviewed instructional videos to improve the educational value of YouTube as a resource.

Keywords:

Hysterectomy – natural orifice endoscopic surgery – YouTube – educational technology

Introduction

In recent years, there has been increasing interest in natural orifice transluminal endoscopic surgery (NOTES), an idea of using the body’s natural orifices to access the abdominal cavity [1]. Over the years, gynecological surgeons worldwide have adopted this technique, and a wide range of benign gynecological procedures has been performed with the transvaginal NOTES (vNOTES) technique [2–7].

As a result of better cosmetic outcomes, a shorter duration of operation and hospital stay, significantly lower estimated blood loss values, and the lack of trocar-related complications [8], vNOTES surgery has become increasingly popular, resulting in an increase in internet users seeking information (Fig. 1).

Watching online surgical videos before entering an operating room is becoming a popular way for residents and senior surgeons to review rarely performed or any operations, check some technical details, observe how their colleagues work, and refresh their memory [9]. There are many platforms for surgeons to share their experiences with vNOTES surgery through the posting and viewing of videos such as International NOTES Society’s website (https: //www.notesurgery.org), Web-Surg, and Medtube, but it has been shown that YouTube remains by far the most popular source for surgical preparation [10].

YouTube (www.youtube.com) is one of the most visited websites with available educational and medical resources [11]. It is important to understand, however, that YouTube is primarily a social media platform with a commercial purpose. Uploaded content is primarily screened for copyright violations rather than for educational value, which can lead to multiple problems with educational content, and the quality and accuracy of the videos may vary significantly.

A number of studies have evaluated the educational value and quality of videos about various topics, including sleeve gastrectomy, endometrioma cystectomy, laparoscopic cholecystectomy, total extraperitoneal laparoscopic inguinal hernia repair, pulmonary lobectomy, transvaginal tension free vaginal tape-obturator (TVT-O) and transobturator vaginal tape (TOT) [12–17]. According to those studies, YouTube medical materials may be an effective tool for learning surgical skills. In most cases, the content provided by these sources is of poor quality and inaccurate, which needs to be addressed.

The objective of this study is to assess the reliability and quality of the vNOTES hysterectomy videos available on YouTube and whether these videos are appropriate for training surgeons.

Materials and methods

A search was performed on https: //www.youtube.com/ on June 12^th, 2024 with the keywords “vNOTES hysterectomy, TVNOTES hysterectomy, vaginal notes hysterectomy, transvaginal natural orifice transluminal endoscopic hysterectomy, vNOTES.’’ Search history and all cookies were deleted, and no personal Google or YouTube account had been logged in before searching.

The videos were listed by relevance sorting, which is the current default option on YouTube. Several studies about search engine user behavior demonstrates that most of the users click on a search result within the first page of the results, and 90% of search engine users click on a result within the first three pages of the results [14]. However, currently, YouTube search engine displays the results in the form of an infinite scrolling list not in the form of pages of results. Therefore, the first 50 videos for each keyword were analyzed to achieve the most reliable statistical analysis. Ethics committee approval was not required for this study as this was an observational study performed using collected data from publicly available YouTube videos.

Video analysis

Videos presenting solely gynecological vNOTES surgeries other than hysterectomy, such as salpingectomy, oophorectomy, ovarian cyst excision, myomectomy, uterosacral ligament suspension, and tubal ectopic pregnancy surgery were excluded. Videos describing the patients’ individual experiences about the surgery, duplicated videos, surgical animation videos, videos that were only about vNOTES hysterectomy-related recorded oral presentations, and videos with commercial purpose were excluded.

The videos were categorized according to their upload source as Surgeon/Practitioner, University/Hospital Channels, Professional Organizations, and Health information Websites. Two researchers (EA, PBI), both gynecology specialists, categorized the videos independently and discrepancies between the 2 researchers were resolved by further discussion with (MY).

Characteristics such as the time passed since video upload, number of views, number of likes and dislikes, number of comments, and duration of the video were recorded. Ratios such as view ratio (number of views/days), like ratio (like × 100/[like + dislike]), and Video Power Index (VPI; like ratio × view ratio/100) were calculated. The presence of didactic sound or music was also noted.

Surgical analysis

The surgical techniques used in vNOTES hysterectomy procedures were assessed based on the recently published international vNOTES consensus paper [15]. A scoring system was developed in accordance with the proposed standardized 10-step approach, which was described in the literature [15,16]. We added 5 more evaluation parameters that included case presentation, presence of didactic steps, setup of the operation room, patient preparation and positioning, and postoperative management (Tab. 1). Each video received a 1 or 0 if the item was met in the video. The total score for each video ranged between 0 and 15.

A 5-point global score modified from that of Singh et al. (Tab. 2) [17] was used to rate the overall quality of the video, based on the quality of the information and how useful the reviewer thought the particular video would be to a surgeon and it was then recorded on a Global Quality Scale (GQS). Videos were categorized into 3 groups: Poor (1–2), Moderate (3), or Good (4–5).

Assessment was carried out independently and blindly by two senior surgeons (ABT, MY) who were both experienced in vNOTES hysterectomy.

Statistical analysis

IBM SPSS Statistics for MacOS, version 21 (IBM Corp., Armonk, N.Y., USA), was used for the data analysis. Descriptive statistics were calculated to summarize the overall characteristics of the videos. To evaluate differences in categorical variables across different quality score categories, a Chi-square test was employed. For comparisons involving continuous engagement metrics across categories, an Analysis of Variance (ANOVA) was conducted, followed by Tukey’s Honest Significant Difference (HSD) test for post hoc comparisons to identify specific group differences. Point-biserial correlation coefficients were calculated to assess the relationship between binary video score items and continuous video engagement metrics, such as views, likes, dislikes, comments, view ratio, like ratio, and video length. Spearman rank correlation coefficients were used to analyze the correlations between the total video score and video demographics. Pearson correlation coefficients were used to assess the relationship between video length and video demographics. The agreement between two researchers was assessed using the Kappa coefficient. All statistical tests were two-sided, with results considered statistically significant at a 95% confidence interval and a P-value of less than 0.05.

Results

A total of 250 videos were initially identified using the search terms “vNOTES hysterectomy, TVNOTES hysterectomy, vaginal notes hysterectomy, transvaginal natural orifice transluminal endoscopic hysterectomy, add a keyword.’’ After excluding duplicates (N = 90), videos describing gynecological vNOTES surgeries other than hysterectomy (N = 30), vNOTES surgery-related oral presentations (N = 2), advertisements (N = 13), animations (N = 3), videos about patients’ experiences (N = 8), or videos describing non-vNOTES gynecological surgeries (N= 31), a total of 73 videos met the inclusion criteria and were included in the study. All included videos were in English.

Excellent inter-reviewer reliability was found between the reviewers (kappa scores 0.930 and 0.897, respectively) for video evaluation regarding the reviewers’ total video and modified-GQS scores.

The videos were categorized based on modified GQS scores into three groups: poor (Group 1; N = 40; 53.8%), moderate (Group 2; N = 13; 17.8%), and good (Group 3; N = 20; 27.4%) (Tab. 3).

No statistically significant differences were found between groups in the comparison made in terms of characteristics such as the time passed since video upload, number of views, number of dislikes, number of comments, duration of the video, and like ratio. However, Group 3 had a significantly higher number of likes (34.00 ± 35.86) compared to Group 1 (14.73 ± 18.67) and 2 (15.00 ± 12.74) (P = 0.012). Additionally, VPI was significantly higher in Group 3 (3.39 ± 5.99) than in Group 1 (1.03 ± 1.23) and Group 2 (1.56 ± 2.76) (P = 0.049). The presence of didactic voice was also higher in Group 3 (55.0%) compared to Group 1 (17.5%) and Group 2 (30.8%) (P = 0.12). Videos were uploaded mainly by surgeons or practitioners across all groups (85.0%; 92.3%; and 70.0%; resp.).

Tab. 4 presents the results of the Pearson correlation analysis between video length and various video demographics, including views, likes, view ratio, like ratio, VPI, and comments. No statistically significant relationships were found between the length of the videos and any of the video demographics studied.

Analysis of the video engagement metrics according to the type of video sound revealed significant differences among groups (Tab. 5). Videos with a didactic voice demonstrated the highest levels of engagement, with a significantly greater mean number of views (3,237.23 ± 3,338.30), likes (37.95 ± 33.68), and comments (3.41 ± 3.88) compared to videos with no sound or those with music (P = 0.048; P < 0.001; and P= 0.002; resp.). No significant differences were found in view ratio, like ratio, or VPI across the different sound types.

The distribution of video score items between groups is shown in Tab. 6 and visualized in Fig. 2. The mean total video score in Group 3 (9.65 ± 1.81) was higher than the rest of the groups with Group 2 being higher than Group 1 (5.55 ± 2.33 vs. 2.88 ± 1.27; resp.) (P < 0.001) (P < 0.001).

No significant differences were observed among groups for case presentation (P = 0.578), identification of ureter and transection of the parametrium (P = 0.231), transection of the infundibulopelvic or ovarian ligament (P = 0.658), and postoperative management (P = 0.684). However, videos in Group 3 exhibited a significantly higher presence of didactic steps (80.0%), setup of the operation room (30.0%), patient preparation and positioning (25.0%), circumcision of the cervix (95.0%), posterior colpotomy (90.0%), anterior colpotomy (85.0%), transection of uterosacral ligaments (95.0%), preparation and placing of the vNOTES-port (95.0%), hemostasis and port removal (50.0%), and vault closure (40.0%) (all P < 0.05).

None of the videos in Group 1 included steps for patient preparation and positioning, setup of the operation room, circumcision of the cervix, or vault closure. Postoperative management was nearly absent across all groups, being mentioned in only 2.5% of the videos in Group 1 and 5.0% of the videos in Group 3, and completely absent in Group 2. The step with the highest presence rate in all three groups was the transection of the infundibulopelvic or ovarian ligament, which was present in nearly all videos across groups.

The point-biserial correlation analysis between various video score items (VS1 to VS15) and video engagement metrics is detailed in Tab. 7. The presence of didactic steps in videos was associated with higher engagement, showing significant positive correlations with the number of views (R = 0.322; P = 0.005), likes (R = 0.429; P < 0.001), comments (R = 0.345; P = 0.003), view ratio (R = 0.247; P = 0.035), and VPI (R = 0.276; P = 0.018). Several video score items also showed significant positive correlations with both the number of views and the number of likes, such as setup of the operation room, circumcision of the cervix, anterior and posterior colpotomy, transection of uterosacral ligaments, hemostasis and port removal, and vault closure. In contrast, other video score items such as case presentation, identification of the ureter and transection of the parametrium, and transection of the infundibulopelvic or ovarian ligament did not show significant correlations with most engagement metrics.

A positive correlation was found between the total video scores and number of views, number of likes, view ratio, and VPI (R = 0.411, P < 0.001; R = 0.436, P < 0.001; R = 0.370, P = 0.001; R = 0.372, P = 0.001, resp.). Similarly, the modified GQS score was positively correlated with the same engagement metrics: number of views (R = 0.346; P = 0.003), number of likes (R = 0.271; P = 0.021), view ratio (R = 0.314; P = 0.007), and VPI (R = 0.340; P = 0.003). However, neither the total video scores nor the modified GQS scores showed significant correlations with the time since video upload (R = 0.055, P = 0.646 for total video scores; R = 0.028, P = 0.817 for modified GQS scores) or video length (R = 0.132, P = 0.267 for total video scores; R = 0.133, P = 0.262 for modified GQS scores) (Tab. 8).

Discussion

The present study revealed that only 27.4% of the vNOTES hysterectomy videos on YouTube had good educational value, which is lower than expected for a novel technique with significant potential for training. While vNOTES is becoming popular as a minimally invasive surgery technique, the majority of the videos evaluated in this study lacked critical surgical steps and instructional content required by surgeons who want to learn or improve their skills in performing this procedure. This underscores a broader issue: a lack of awareness among many surgeons and practitioners about what constitutes a good educational surgical video.

Several studies support these findings, highlighting the widespread variability in the educational quality of surgical videos across platforms like YouTube. A recent systematic review assessing the quality of content in YouTube videos for professional medical education evaluated 31 studies, demonstrating significant variation in the quality rating tool (QRT) scores with a high percentage of videos receiving low QRT scores [18]. Many of these studies concluded that included videos featured incomplete content and lacked sufficient educational depth to meet the standards of competent surgical teaching. For example, Rodriguez et al. [19] found that YouTube videos on laparoscopic cholecystectomy demonstrated suboptimal techniques; with approximately half involving unsafe maneuvers, and fewer than 10% providing adequate safety assessments. Similarly, Derakhshan et al. [20] reported that 62% of videos demonstrating facelift surgery failed to mention the importance of avoiding the facial nerve, a critical structure at risk of injury during the operation. Another study evaluating videos of total extraperitoneal laparoscopic inguinal hernia repair revealed that most of the techniques demonstrated were outdated [15].

In line with these findings, de’Angelis et al. [21] discovered that most YouTube videos of laparoscopic appendectomy omitted important technical details such as patient positioning and trocar placement, as well as educational components such as commentary and formal case presentation. Similarly, Derakhshan et al. [20] observed that the videos analyzed regarding rhytidectomy procedures lacked pre/postoperative information such as indications, complications, and selection of patients. These omissions, which compromise patient safety, are alarmingly similar to the deficiencies we found in our review of vNOTES hysterectomy videos. Our research revealed that only 2 out of all 73 videos included in the study presented any information on postoperative management, and none of the videos in the poor-quality group addressed critical steps such as patient preparation and positioning, setup of the operating room, circumcision of the cervix, or vault closure. Most of the poor-quality videos also failed to demonstrate key components of the vNOTES hysterectomy procedure, including posterior and anterior colpotomy, transection of the uterosacral ligaments, preparation and placement of the vNOTES port, hemostasis, and port removal. Given that 53.8% of the evaluated videos were categorized as poor quality, these findings are particularly concerning.

Several studies have suggested that the popularity of a video, as measured by views, likes, or comments, does not necessarily reflect its educational value [13–15,22–24]. Lee et al. [25] found that videos categorized as “very useful” for patient information on gallstone disease received significantly fewer views and likes compared to misleading or incomplete videos. This paradox – where lower-quality content garners higher engagement – is likely driven by factors such as sensationalism or easier accessibility of content that simplifies complex procedures. Our findings corroborate this trend, as no significant difference was found between video quality groups in terms of number of views, likes, or dislikes. However, a positive correlation was observed between video score and view ratio, likes, and VPI (Video Power Index), suggesting that better-quality videos can still engage audiences effectively, even if they are not the most viewed.

The correlation between video duration and quality has also been explored in previous studies. Some research suggests that longer videos tend to provide more comprehensive content, leading to higher educational value scores [12,26,27]. However, in our study, video duration did not significantly differ between the quality groups, reinforcing the notion that length alone is not a reliable indicator of educational value. This finding aligns with Keskinlılıc Yagiz et al. [15], who reported that while longer videos received more likes and comments, their overall quality was not necessarily higher. This suggests that the content structure and adherence to surgical principles, rather than video length or viewer interaction, are the key determinants of educational value.

It is also noteworthy that our study’s analysis of video sound types revealed significant differences in engagement. Videos with didactic voice performed better than those with music or no sound in terms of the number of views, likes, and comments. This aligns with prior research that emphasizes the importance of clear instructional narration for educational videos [28]. Despite this, many vNOTES videos still lack this crucial element, as seen in our data where a large proportion of the videos had either no sound or only background music.

The limitations of this study should be acknowledged. As we analyzed the videos at a single point in time, their content may have changed since our search and future analyses may yield different results. Additionally, our sample size of 73 videos is relatively small. Lastly, while we developed a scoring system based on expert recommendations, there is no universally validated tool for assessing the quality of vNOTES hysterectomy videos, which could limit the generalizability of our results.

In conclusion, the educational value of vNOTES hysterectomy videos on YouTube is currently inadequate for comprehensive surgical training, with only 27.4% of videos classified as having good educational quality. While videos featuring didactic narration and structured procedural steps tend to engage viewers more effectively, many still lack essential details, limiting their educational utility. To improve the educational potential of YouTube as a platform for surgical learning, content creators – particularly surgeons – must follow established educational guidelines. Incorporating comprehensive explanations, detailed procedural steps, and maintaining high standards of rigor can significantly enhance the quality of these videos. However, it is important to acknowledge that most of these videos are produced without peer review and contain gaps that may mislead learners.

While YouTube can be a valuable supplementary resource, it cannot replace formal, peer-reviewed educational content. There is a critical need for professional organizations to curate high-quality, instructional videos that meet the standards required for effective surgical education. As digital platforms continue to play a growing role in medical training, it is essential to ensure that these resources adhere to the highest levels of accuracy, reliability, and instructional value.