This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Interactive Journal of Medical Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.i-jmr.org/, as well as this copyright and license information must be included.
An increasing quantity of research in the domain of telemedicine show a growing popularity and acceptance of care over distance systems among both clinicians and patients. We focus on telementoring solutions, developed for providing remote guidance to less experienced surgeons. Telestration is often regarded as an extra functionality of some telementoring systems. However, we advocate that telestration must be viewed as a core feature of telementoring due to its advantages.
To analyze and define concepts, parameters, and measurement procedures to evaluate the impact of using telestration while telementoring.
A systematic review of research dealing with telestration during remote guidance sessions was performed by querying three major online research databases (MEDLINE, Association of Computing Machinery, and Institute of Electrical and Electronics Engineers) using a predefined set of keywords (“laparoscopy”, “annotate”, “telestrate”, “telestration”, “annotation”, “minimally invasive”, and “MIS”).
The keyword-based search identified 117 papers. Following the guidelines for performing a systematic review, only 8 publications were considered relevant for the final study. Moreover, a gap in research defining the impacts of telestration during telementoring was identified. To fill this niche, a framework for analyzing, reporting, and measuring the impacts of telestration was proposed.
The presented framework lays the basics for the structured analysis and reporting of telestration applied to telementoring systems. It is the first step toward building an evidence knowledge base documenting the advantages of live video content annotation and supporting the presented connections between the concepts.
The shift from open to laparoscopic surgery presents a fertile ground for an expansion of telemedicine (a set of medical practices without direct physician-patient interaction that are often realized via interactive audio-video communication [
Telementoring has been discussed extensively in the literature, with multiple solutions demonstrated for laparoscopic surgery [
In spite of the recent advances in mobile devices, the current body of literature tends to limit the telementoring approach to stationary platforms. However, the Mobile Medical Mentor project team advocates using mobile devices for telementoring due to additional benefits compared to the stationary ones [
Telestration is defined as a technique for enabling the drawing of freehand markups (annotations) over image or video (
The paper is structured as follows: after a short introduction to the developing domain of telemedicine, the motivation for using telementoring systems is presented. The telestration feature is analyzed in greater detail.
The Methods Section defines the procedures that were followed to summarize the current body of knowledge in using telestration and guidelines to develop the evaluation framework. The Results Section represents the achievements of a systematic review procedure and highlights the identified gap in the available research. The following section discusses the identified niche in research, introducing an evaluation framework for analyzing, measuring, and reporting the impacts of using telestration during telementoring sessions. Evaluation biases and confounders were analyzed in the Discussion Section.
Surgical telestration.
This section is divided into two sub-sections, the first of which describes the search process for relevant research. This search was performed to summarize the available body of knowledge in evaluating the impacts of telestration applied to telementoring systems. The second part studies the development of an evaluation framework in more detail by introducing the methodology for developing theoretical models.
To form a solid basis for this research, a systematic literature review of telementoring applications that include a telestration feature in MIS was performed. The review was carried out following the guidelines defined by Kitchenham [
The search for related studies was limited by publication date (1992-2012) due to technological progress. Search results (published before 1992) were considered too old to be relevant. Three major online research databases (MEDLINE, Association of Computing Machinery [ACM], and Institute of Electrical and Electronics Engineers [IEEE]) were selected as data sources for the review. The search was performed on the 13th of November, 2012. The selected databases were queried using the same set of keywords, containing combinations of the following terms: “laparoscopy”, “annotate”, “telestrate”, “telestration”, “annotation”, “minimally invasive”, and “MIS”. Keywords were selected based on earlier research in the domain [
The methodology presented by Sjøberg et al [
The keyword-based search resulted in 117 papers in total, as shown in
The review was performed to assess the impacts and benefits of telestration as applied to telementoring systems and then analyze these data in a systematic manner. The study revealed a gap of knowledge in assessing the outcomes of telestration. All of the selected studies reported the use of telestration; however, no analysis or assessment of its impacts was identified. The only outcome was a significant improvement of the mentoring session due to the ability to annotate graphic content, reported in 3 papers [
The purpose of the review was to assess the potential benefits of telestration in telementoring based on reported experiences. However, we were unable to achieve our goal due to the lack of existing research. The shortage of publications motivated the need to propose an evaluation framework for analyzing the outcomes of telestration in the scope of telementoring.
Systematic review flow diagram.
An extra layer of interaction and complexity is added by providing the ability to annotate the graphic content used in a telementoring session. On the one hand, telestration simplifies pointing actions, eliminating the need for discussion to define the exact location in the visual representation; while on the other hand, it introduces extra complexity into the workflow model of the procedure. This complexity includes controlling of the telementoring system and annotations as well as extra hardware in the operating room. In this Section, we aim to define the potential benefits of telestration and discuss the impact of telestration on the outcome of the overall procedure.
To define the influence of telestration while telementoring, the outcomes were divided into Educational and Clinical Impact groups, highlighting the direct influence of the education process on potentially improved patient outcome (see
From a clinical perspective, the overall goal is the improved patient outcome, defined quantitatively as the ratio and severity level of complications, as shown in
Another important aspect that should not be omitted is time consumption. From the perspective of the mentor, time savings contribute to a reduction of the workload as well as increased availability of the expert. The mentee should also experience decreased time consumption, resulting in shorter duration of the overall procedure. These issues pose the following question: given that the resources available in the hospital are constant, is it possible to treat more patients during the same period of time by decreasing the duration of the procedure? Moreover, the costs of the treatment should also diminish.
To summarize the Section, we assume that telestration is a feature that improves telementoring techniques. However, due to lack of studies on the possible impacts on the overall mentoring process, our claims remain poorly supported. Only the most obvious relations were discussed due to the simplicity of the models. Other dependencies may exist.
Impacts of telestration.
Educational outcomes of telestration.
Clinical outcomes of telestration.
To prove the hypotheses formulated in Section Definition of Concepts and Theoretical Models, a measurement system needs to be established. The purpose of such system is to assess if the introduction of a particular feature resulted in expected outcomes. This system calls for a comparison of measurements before and after the procedure workflow was supplemented by the new technology.
Evaluating the educational outcomes, shown in
To evaluate our hypotheses, a comparison of measures before and after introducing the technology to the operating room must be performed. ALFA Toolkit [
Start and end point of the procedure (knife time)
Number of guidance misinterpretations
Number of requests to clarify the guidance
Start and end point of every telementoring interaction between mentor and mentee
Initiator of each interaction
Duration and timing-based measures are automatically collected by ALFA Toolkit during the coding process, while the numerical data (number of misinterpretations and clarification requests, initiator of the interaction) is recorded by the coders in an MS Excel sheet, which is combined with the values exported from ALFA toolkit for further analysis. Average values together with dispersion coefficients are used to represent the final results.
Measurement system.
Parameter | Relationship in |
What is measured? | Measure unit |
Knife time | C | Duration: starting point–first incision, end point–end of the procedure. | Minutes, seconds |
Ratio and severity level of complications |
E, F | Number of complications / overall number of performed procedures and severity of the complications, if any. |
Ratio coefficient and a grade in the Clavien-Dindo classification scheme for surgical complications [ |
Score in rating scale (surgical education) | B | Scores in the predefined scale to measure the progress of education. | Rating score |
Duration (surgical education) | B | Duration required to reach the same level in the rating scale. | Hours |
Accuracy of mentoring | A | 1) Number of guidance misinterpretations (possibly leading to errors), |
Combined score |
Research biases were analyzed in detail by Hartman et al [
An accurate measure of the parameters defined in
From the point of view of the measurement process itself, ambiguity is also inevitable. To avoid biases in duration and score estimation based on video representation, only the consensus decision of the project team should be considered correct.
The presented framework addresses the theoretical side of the project. It was developed as a methodology section for the on-going research. Logical relations between the concepts are represented in order to understand the influence of the technology on procedural workflow and define it in a measurable manner. The research still suffers from the lack of evidence and validation on real life test cases.
To conclude the section, we admit the potential weaknesses of the proposed research caused by the mentioned biases. The list of the biases influencing the final outcome of the experiment is not exhaustive; however, we aimed to mention only the most obvious cases.
The paper aimed to summarize the experience of using telestration during telementoring sessions to highlight the impacts of this feature on the mentoring process and workflow of the procedure. However, an absence of research reflecting the use of telestration was identified. Therefore, a framework for analyzing the impact of live video content annotation was proposed. To keep the models simple and adaptable, easy-to-measure concepts were employed and only the most obvious dependencies were discussed. To support our claims and the presented models, an impact measurement procedure was defined.
The presented framework and impact measurement procedures form a methodology for the further analysis and reporting of research on telestration and telementoring systems. Having a more formalized method should increase the quality and quantity of publications in this field, providing an evidence-based knowledge base supporting the development of the telementoring domain and the introduction of new technologies and features with the aim of improving patient outcomes.
Association of Computing Machinery
Institute of Electrical and Electronics Engineers
minimally invasive surgery
This research was funded by a Helse Nord grant (ID 5614/HST1025-11) to the Norwegian Centre of Integrated Care and Telemedicine (NST), University Hospital of North Norway. The study would not have been possible without the support of the Mobile Medical Mentor (M3) project team at NST.
None declared.