CN113017549A

CN113017549A - Brain endoscope operation auxiliary system

Info

Publication number: CN113017549A
Application number: CN201911253454.9A
Authority: CN
Inventors: 乔霓丹
Original assignee: Huashan Hospital of Fudan University
Current assignee: Huashan Hospital of Fudan University
Priority date: 2019-12-09
Filing date: 2019-12-09
Publication date: 2021-06-25

Abstract

The invention relates to the field of medical instruments, and discloses a brain endoscope operation auxiliary system, which comprises: the system comprises an anatomical structure identification subsystem, a visual electrophysiology collector and an optic nerve electrophysiology early warning subsystem; the anatomical structure identification subsystem is used for receiving the operation image output by the brain endoscope and identifying the anatomical structure in the brain endoscope operation according to the operation image; the visual electrophysiology collector is used for collecting visual electrophysiology monitoring data of a patient in a cerebral endoscope operation; the optic nerve electrophysiology early warning subsystem is used for receiving the visual electrophysiology monitoring data output by the visual electrophysiology collector and realizing early warning on optic nerve functions of the patient according to the visual electrophysiology monitoring data. The convolution neural network is utilized to analyze the operation image and the nerve electrophysiology monitoring data, the operation image and the nerve electrophysiology monitoring data are fused to construct a safe brain endoscope operation auxiliary system, and the key anatomical structure instant identification and the timely early warning of the optic nerve function in the brain endoscope operation process can be realized.

Description

Brain endoscope operation auxiliary system

Technical Field

The invention relates to the field of medical instruments, in particular to a brain endoscope operation auxiliary system.

Background

Cerebral endoscopy is an emerging neurosurgical technique in recent years, and the development thereof enables surgeons to remove tumors originally called "forbidden surgical regions" -cranial fundus tumors, including pituitary tumors, craniopharyngiomas, chordomas, and the like. These tumors press the peripheral nervous tissue, seriously affecting the quality of life of the patient and even endangering life. The operating doctor inserts the endoscope into the skull base of the patient through the nasal passage of the patient to perform an operation, and the operation is performed by watching the video output by the display. The endoscope has the advantages of small wound, clear operation visual field and the like, and has obvious advantages compared with the traditional microscope operation. However, basicranial tumors often invade the surrounding critical anatomy: the anterior invasion of optic nerve, bilateral invasion of the cavernous sinus of the internal carotid artery, posterior invasion of the oblique slope venous plexus, and superior invasion of intracranial vessels and brain tissue are shown in fig. 1, wherein a is a schematic diagram of a cranial base operation of a cerebral endoscope: three surgical instruments coexist in the nasal cavity of a patient, Tumor: a tumor; b is seen under the endoscope, where CNII: the optic nerve; and (6) CarA: the internal carotid artery; clivus: a slope; middle oval part: a tumor. Therefore, accurate navigation and avoidance of accidental injury in brain endoscopic surgery are key to the success of the surgery. However, the following two problems often exist in clinic, which seriously affect the operation of the operating doctor:

first, the anatomy under a brain endoscope is quite different from that seen in craniotomy, and the normal anatomy is often destroyed by the cranial base tumor, which most neurosurgeons are not familiar with. The operation of the brain endoscope requires decades of training, and the current top brain endoscope doctors in China can curl fingers. The inexperienced surgeons encounter the key anatomical parts (such as internal carotid artery, optic nerve and the like) which are difficult to grasp. Once misinjured, the consequences can be catastrophic, endangering the life of the patient.

Second, surgical instruments are often touched with the visual cross in the process of brain endoscope operation, and the visual cross is damaged by traction and heat conduction of the surgical instruments in the process of operation. Intraoperative ocular electrophysiological monitoring is a current tool to prevent visual cross-lesions. However, the conventional visual electrophysiological monitoring needs a trained electrophysiological doctor to monitor continuously, so that a certain recognition error exists and the labor cost is high.

If the key anatomical parts in the brain endoscope operation can be early warned immediately and the visual electrophysiological monitoring can automatically warn, the method has very important clinical significance for improving the operation safety. Therefore, a new surgical auxiliary system is needed to be developed, which can immediately warn the vulnerable anatomical structure and warn the optic nerve functional injury, and provide safer technical support for the operation of the surgeon.

Disclosure of Invention

The invention aims to provide a brain endoscope operation auxiliary system which can immediately early warn an easily damaged anatomical structure and early warn optic nerve function damage and provide safer technical support for operation of an operator.

In order to solve the above technical problem, an embodiment of the present invention discloses a brain endoscope operation support system, including: the system comprises an anatomical structure identification subsystem, a visual electrophysiology collector and an optic nerve electrophysiology early warning subsystem;

the anatomical structure identification subsystem is used for receiving an operation image output by a brain endoscope and identifying an anatomical structure in the brain endoscope operation according to the operation image;

the visual electrophysiology collector is used for collecting visual electrophysiology monitoring data of a patient in a cerebral endoscope operation;

the optic nerve electrophysiology early warning subsystem is used for receiving the visual electrophysiology monitoring data output by the visual electrophysiology collector and realizing early warning on the optic nerve function of the patient according to the visual electrophysiology monitoring data.

In another preferred example, the anatomical structure recognition subsystem comprises:

the first marking module is used for converting the existing video or image of the brain endoscope operation into an image with a standard size and marking the name and the position of the anatomical structure on the converted image;

the first modeling module is used for performing machine learning modeling according to the image marked by the first marking module to obtain a first model;

and the identification module is used for inputting the real-time operation image output by the brain endoscope into the first model and outputting the name and the position of the anatomical structure in the real-time operation image.

In another preferred example, the anatomical structure includes: the nasal septum, the middle turbinate, the sphenoid sinus, the saddle base, the internal carotid artery, the optic nerve and the blood vessels.

In another preferred example, the first model is an inclusion v4 convolutional neural network structure.

In another preferred example, the neuro-electrophysiological warning subsystem comprises:

the second marking module is used for calculating the amplitude change of the patient visual electrophysiological monitoring data in the historical data between 90ms and 120ms by taking the average amplitude of the visual electrophysiological monitoring data in the historical data within 5 minutes after the patient is stably anesthetized as a baseline, and marking the amplitude change result;

the preprocessing module is used for converting the visual electrophysiological monitoring data marked by the second marking module into standard pixel size and carrying out vectorization by using the value of each pixel;

the second modeling module is used for performing machine learning modeling according to the data processed by the preprocessing module to obtain a second model;

and the early warning module is used for inputting the real-time visual electrophysiological monitoring data output by the visual electrophysiological collector into the second model, calculating an amplitude change result of the real-time visual electrophysiological monitoring data, and early warning according to the amplitude change result.

In another preferred example, the standard pixel size is 350 × 90.

In another preferred example, the amplitude variation result includes: amplitude increase, no change in amplitude, and amplitude decrease.

In another preferred example, the early warning module gives an early warning when the amplitude is reduced to a predetermined threshold value.

In another preferred embodiment, the threshold is 25%.

In another preferred example, the second model is a three-layer convolutional neural network followed by two recurrent neural network structures.

Compared with the prior art, the implementation mode of the invention has the main differences and the effects that:

the convolution neural network technology is utilized to analyze the operation image and the nerve electrophysiology monitoring data, the operation image and the nerve electrophysiology monitoring data are fused to construct a safe brain endoscope operation auxiliary system, and the key anatomical structure instant identification and the timely early warning of the optic nerve function in the brain endoscope operation process can be realized.

Drawings

FIG. 1 is a schematic representation of a brain endoscopic procedure and dissection;

FIG. 2 is a schematic structural diagram of an auxiliary system for a cerebral endoscope operation in a first embodiment of the present invention;

FIG. 3 is a diagram illustrating a CNN model trained to recognize important structural sections in a book according to a first embodiment of the present invention;

FIG. 4 is a diagram illustrating a method for training a neural network model to identify neuroelectrophysiological changes in a first embodiment of the present invention.

Detailed Description

In the following description, numerous technical details are set forth in order to provide a better understanding of the present application. However, it will be understood by those skilled in the art that the technical solutions claimed in the present application can be implemented without these technical details and with various changes and modifications based on the following embodiments.

In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The first embodiment of the present invention relates to a brain endoscope operation support system, and fig. 2 is a schematic structural view of the brain endoscope operation support system.

Specifically, as shown in fig. 2, the brain endoscope operation support system includes: an anatomical structure recognition subsystem, a visual electrophysiology collector and an optic nerve electrophysiology early warning subsystem.

The anatomical structure identification subsystem is used for receiving the operation image output by the brain endoscope and identifying the anatomical structure in the brain endoscope operation according to the operation image.

The visual electrophysiology collector is used for collecting visual electrophysiology monitoring data of a patient in a cerebral endoscope operation.

In one aspect, in this embodiment, preferably, the anatomical structure recognition subsystem includes:

the first marking module is used for converting the existing video or image of the brain endoscopic surgery into an image with a standard size and marking the name and the position of the anatomical structure on the converted image.

Further, preferably, the anatomical structure comprises: the nasal septum, the middle turbinate, the sphenoid sinus, the saddle base, the internal carotid artery, the optic nerve and the blood vessels.

The method comprises the steps of converting an existing brain endoscope operation video or image into an image with a standard size and a small video, framing an important anatomical structure on the image or video by using a rectangular frame, and recording the position (the distance between a central point and the edge of a transverse axis and the edge of a longitudinal axis of the image), the size (the length of the transverse axis and the length of the longitudinal axis) and the name of the anatomical structure in the frame by an operator. These anatomical structures include the nasal septum, middle turbinate, sphenoid sinus, saddle floor, internal carotid artery, optic nerve, blood vessels, and the like.

And the first modeling module is used for performing machine learning modeling according to the image marked by the first marking module to obtain a first model.

In this embodiment, preferably, the first model is an inclusion v4 convolutional neural network structure.

Further, preferably, 80% of the data in the image labeled by the first labeling module is used for training the first model, and the image is input into the convolutional neural network as an input layer (pixel values on three channels of RGB); 10% of the data is used for adjusting the hyper-parameters of the convolutional neural network; and taking the remaining 10% of data to verify the accuracy, sensitivity and specificity of the model, and establishing an ROC curve by taking the result marked by the brain endoscope expert as a gold standard.

The Convolutional Neural Network (CNN) technology is utilized to identify important anatomical structures in the brain-friendly endoscopic surgery, and an operating doctor is reminded at any time in the surgery to avoid accidental injuries.

Fig. 3 is a schematic diagram of the training of CNN models to identify important anatomical structures in books.

On the other hand, in this embodiment, preferably, the optic nerve electrophysiology early warning subsystem includes:

and the second marking module is used for calculating the amplitude change between 90ms and 120ms in the visual electrophysiological monitoring data of the patient in the historical data by taking the average amplitude of the visual electrophysiological monitoring data within 5 minutes after the patient is stably anesthetized in the historical data as a baseline, and marking the amplitude change result.

In this embodiment, preferably, the amplitude change result includes: amplitude increase, no change in amplitude, and amplitude decrease.

And the preprocessing module is used for converting the visual electrophysiological monitoring data marked by the second marking module into standard pixel size and performing vectorization by using the value of each pixel.

In this embodiment, preferably, the standard pixel size is 350 × 90.

And the second modeling module is used for performing machine learning modeling according to the data processed by the preprocessing module to obtain a second model.

In this embodiment, preferably, the second model is a three-layer convolutional neural network followed by two recurrent neural network structures.

Further, preferably, 80% of the data processed by the preprocessing module is used for training the second model, and the image is input into the convolutional neural network as an input layer (pixel value on a single channel). The convolutional neural network used was: the three layers of convolutional neural networks are connected with two recurrent neural networks; 10% of data is used for adjusting the hyper-parameters of the convolutional neural network; data of 10% of the remaining eating was taken for verifying the accuracy of the second model.

In this embodiment, preferably, the warning module warns when the amplitude decreases to a predetermined threshold. Wherein the threshold is preferably 25%.

The CNN technology is used for analyzing the neuroelectrophysiology monitoring data, an intelligent algorithm model is constructed, and the increase and decrease of the optic neuroelectrophysiology amplitude in the operation process can be automatically identified.

FIG. 4 is a diagram of training a neural network model to identify neuroelectrophysiological changes, where Time represents Time, Input Image represents Input Image, CNN: pattern represents CNN model, RNN: sequence indicates RNN Sequence, Output indicates Output, INCREASING indicates an increase in amplitude, NO CHANGE indicates NO CHANGE in amplitude, DECREASING indicates a decrease in amplitude.

And after the model training is finished, the model is on line, endoscope operation images collected in real time in the operation are received, the first model processes the images in real time in the background, and the recognition result is displayed on an additional display. The second model receives the nerve electrophysiology data acquired in real time in the operation, the second model processes the data in real time in the background, and once the amplitude is judged to be reduced by the algorithm, the sound alarm is immediately provided.

To sum up, this application brain endoscope operation auxiliary system utilize convolution neural network technology analysis operation image and nerve electrophysiology guardianship data, and both fuse, construct safe brain endoscope operation auxiliary system, can realize the timely early warning of the real-time discernment of the anatomical structure of key and optic nerve function among the brain endoscope operation process.

In order to better understand the technical solution of the present application, the following description is given with reference to a preferred embodiment, and the details listed in the preferred embodiment are mainly for understanding and are not intended to limit the scope of the present application.

For a brain endoscope, a high-definition image is one of the great advantages, and a Convolution Neural Network (CNN), which is an emerging artificial intelligence technology depending on images, opens a door for solving the problems encountered in clinic. However, no research report for analyzing the brain endoscope image and the visual electrophysiology by using the CNN is available in the medical aspects at home and abroad so far. The reason for this is that the first use of CNN technology requires a powerful computer and engineering foundation, and there are few such resources in domestic medical and academic units. Second, the start of the cerebral endoscope is late, related operations are started to be performed in China only within the last three or four years, the research using the CNN has high requirements on the number of images (more than million levels are needed), and a plurality of large medical centers are often needed to reach the number of case images needed by the research through years of clinical accumulation. At present, purposeful image collection and database construction are developed in few units in a plurality of regional large hospitals at home and abroad.

The technical problem to be solved by the preferred embodiment includes:

1. the CNN technology is used for identifying the anatomical structure which is easy to be accidentally injured, and an operating doctor is reminded at any time in the operation to avoid accidental injury;

2. the CNN technology is used for analyzing the neuroelectrophysiology monitoring data, and an intelligent algorithm is constructed, so that the increase and decrease of the optic neuroelectrophysiology amplitude in the operation process can be automatically identified.

The technical scheme adopted by the preferred embodiment mainly comprises the following steps:

firstly, establishing a model by using a CNN technology to identify important anatomical structures in cerebral endoscopy:

1) the existing video of the brain endoscope operation is converted into an image with a standard size and a small video, an operator frames an important anatomical structure on the image or the video by using a rectangular frame, and records the position (the distance between a central point and the edge of a transverse axis and the edge of a longitudinal axis of the image), the size (the length of the transverse axis and the length of the longitudinal axis of the frame) and the name of the anatomical structure in the frame. These anatomical structures include the nasal septum, middle turbinate, sphenoid sinus, saddle floor, internal carotid artery, optic nerve, blood vessels, and the like.

2) 80% of the resulting data was used to model, and the image was input to the convolutional neural network as an input layer (pixel values on the three channels of RGB). The convolutional neural network used in the technical scheme of the application is as follows: InceptitionV 4 convolutional neural network structure. 10% of the data is used for the adjustment of the hyper-parameters of the convolutional neural network.

3) And (3) model verification: and taking the remaining 10% of data to verify the accuracy, sensitivity and specificity of the model, and establishing an ROC curve by taking the result marked by the brain endoscope expert as a gold standard.

Secondly, a model for intelligently identifying the optic nerve electrophysiology wave amplitude is constructed by utilizing the neural network technology:

1) first we collected data on visual electrophysiological monitoring of multiple patients during surgery. We measure the amplitude of the peak amplitude between 90ms and 120ms in the electrophysiological signal. To minimize noise during monitoring, we measured the amplitude using the average response over 5 minutes after stable anesthesia. Then, the amplitude after anesthesia was taken as a baseline, and the degree of change in the amplitude was calculated. We have determined three amplitude variations: no change, increase and decrease (defined as 25% increase or 25% decrease in amplitude compared to baseline during surgical decompression)

2) The pre-analysis process of the vision-induced potential sequence includes extracting vision-induced potential images from all three channels of each eye, excluding images with large noise and artifacts. All sequences are converted to 350 x 90 pixel size and vectorized using the value of each pixel.

3) 80% of the data was used to model, and the image was input to the convolutional neural network as an input layer (pixel values on a single channel). The convolutional neural network used was: the three layers of convolutional neural networks are followed by two recurrent neural networks. 10% of the data is used for the adjustment of the hyper-parameters of the convolutional neural network.

4) And (3) model verification: the remaining 10% of the data was taken for verifying the accuracy of the model.

And thirdly, fusing an anatomical structure recognition model and an optic nerve electrophysiology early warning model:

1) after the two models are online in real time, endoscope operation images acquired in real time in an operation are received, the anatomical structure recognition model processes the images in real time at the background, the recognition result is displayed on an additional display and displayed, and an operator is reminded of key anatomical structures in time.

2) The other optic nerve electrophysiology early warning model receives nerve electrophysiology data acquired in real time in an operation, the model processes the data in real time at the background, and sound alarm is immediately provided once the amplitude is judged to be reduced by the algorithm.

As can be seen from the above description, in the technical scheme of the preferred embodiment, the CNN technology is used to analyze the operation image and the neuroelectrophysiological monitoring data, and the two are fused to construct a safe auxiliary system for the endoscopic brain surgery, so that the instant identification of the key anatomical structure and the timely early warning of the optic nerve function during the endoscopic brain surgery can be realized.

With the continuous improvement of the overall level of the medical industry in China, advanced imaging diagnosis technologies such as Magnetic Resonance Imaging (MRI) and the like are gradually popularized, and various provincial-level hospitals and some district-level and county-level hospitals in China have the equipment, so that the effective detection of clinical skull base tumor cases is objectively and greatly promoted, and the brain endoscope is a better operation mode for most cases. In view of the huge base of 14 hundred million people in China, the top encephaloscope doctors can flex fingers, so that doctors who master encephaloscope technology are trained to be a problem that neurosurgery needs to be looked upright. However, the current situation is that the brain endoscopy doctors only rely on the tips to perform one-to-one operation teaching, and the number of teachers is far from enough. According to the technical scheme, some important anatomical structures in the brain endoscope operation can be identified, the brain endoscope operation identification system not only can be popularized and applied to medical units with brain endoscopes nationwide as teaching tools and operation auxiliary equipment, and assist neurosurgeons who are learning or will learn the brain endoscopes so as to shorten the training time of the brain endoscope operation of the neurosurgeons, but also can be popularized to other endoscope centers so as to improve the safety of the endoscope operation. Therefore, huge social benefits and economic benefits are generated, and certain contribution is made to the development of neurosurgery in China and the life health of the nation.

It should be noted that, as will be understood by those skilled in the art, the functions of the modules shown in the embodiments of the present application may be implemented by a program (executable instructions) running on a processor, and may also be implemented by a specific logic circuit. When implemented in the form of software functional modules and sold or used as a stand-alone product, may also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present specification may be essentially implemented or portions thereof contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present specification. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Thus, embodiments of the present description are not limited to any specific combination of hardware and software.

It should be noted that, in the embodiment of the present invention, each module is a logic module, and physically, one logic module may be one physical module, or may be a part of one physical module, or may be implemented by a combination of multiple physical modules, where the physical implementation manner of the logic modules itself is not the most important, and the combination of the functions implemented by the logic modules is the key to solve the technical problem provided by the present invention. Furthermore, in order to highlight the innovative part of the present invention, the above-mentioned embodiments of the device of the present invention do not introduce modules which are not so closely related to solve the technical problems proposed by the present invention, which does not indicate that there are no other modules in the above-mentioned embodiments of the device.

It is to be noted that, in the present application, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, the use of the verb "comprise a" to define an element does not exclude the presence of another, same element in a process, method, article, or apparatus that comprises the element. In the present patent application, if it is mentioned that a certain action is executed according to a certain element, it means that the action is executed according to at least the element, and two cases are included: performing the action based only on the element, and performing the action based on the element and other elements. The expression of a plurality of, a plurality of and the like includes 2, 2 and more than 2, more than 2 and more than 2.

All documents mentioned in this specification are to be considered as being incorporated in their entirety into the disclosure of this specification so as to be subject to modification as necessary. It should be understood that the above description is only a preferred embodiment of the present disclosure, and is not intended to limit the scope of the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of the present disclosure should be included in the scope of protection of one or more embodiments of the present disclosure.

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

Claims

1. A brain endoscope surgery assisting system, characterized in that the assisting system comprises: the system comprises an anatomical structure identification subsystem, a visual electrophysiology collector and an optic nerve electrophysiology early warning subsystem;

2. The brain endoscopy surgery assistance system of claim 1, wherein the anatomy recognition subsystem comprises:

3. The brain endoscopic surgery assistance system according to claim 2, characterized in that the anatomical structure comprises: the nasal septum, the middle turbinate, the sphenoid sinus, the saddle base, the internal carotid artery, the optic nerve and the blood vessels.

4. The brain endoscope surgery assistance system according to claim 2, wherein the first model is an IncepotionV 4 convolutional neural network structure.

5. The brain endoscope surgery assistance system of claim 1, wherein the neuro-electrophysiology early warning subsystem comprises:

6. The brain endoscopy surgical assistant system of claim 5, wherein the standard pixel size is 350 x 90.

7. The brain endoscope surgery assistance system according to claim 5, wherein the amplitude variation result includes: amplitude increase, no change in amplitude, and amplitude decrease.

8. The brain endoscope surgery assistance system according to claim 7, wherein the early warning module performs early warning when the amplitude decreases to a predetermined threshold value.

9. The brain endoscope surgery assistance system according to claim 8, wherein the threshold value is 25%.

10. The brain endoscope surgery assistance system according to claim 5, wherein the second model is a three-layer convolutional neural network followed by two recurrent neural network structures.