JP2023176332A - Display control device, display control method, and program - Google Patents

Abstract

To capture an active portion and an inactive portion of a function of the brain as a whole in a bird's eye view manner.SOLUTION: A display control device comprises: an imaging data acquisition unit which acquires imaging data obtained by capturing an image of the brain subjected to craniotomy of a patient during a surgery; a first input unit which acquires electric stimulation response information expressing the change in the reaction of the patient to the electric stimulation applied to the brain of the patient in such a state that the patient is made to perform the prescribed work; an electric stimulation information acquisition unit which acquires electric stimulation information when the electric stimulation is applied; a reaction detection unit which detects for each region the state of the change of the reaction to the electric stimulation on the basis of the electric stimulation information and the electric stimulation response information; an image generation unit which generates a synthetic image obtained by superimposing imaging data on the detection result of the reaction detection unit; and a display control unit which causes the display unit to display the synthetic image.SELECTED DRAWING: Figure 1

Description

The present invention relates to a display control device, a display control method, and a program.

2. Description of the Related Art There is a known technique for displaying a combination of brain function data and image data of a patient's brain for examination and treatment of the patient's brain (for example, see Patent Document 1).

JP2017-202344A

During brain surgery, the medical team may proceed with the surgery while communicating with the patient through awake surgery and checking the patient's cognitive state by observing stimulus responses such as the patient's reactions to voice and written instructions and questions. . In this case, there is a need for a technology that can provide a bird's-eye view of functionally active and inactive parts of the brain as a whole.

An object of the present invention is to provide a display control device, a display control method, and a program that can provide a bird's-eye view of functionally active and inactive parts of the brain as a whole.

The display control device according to the present invention includes an imaging data acquisition unit that acquires imaging data obtained by photographing an image of the craniotomy brain of a patient undergoing surgery; a first input section that acquires electrical stimulation response information representing changes in a patient's response to electrical stimulation; an electrical stimulation information acquisition section that obtains electrical stimulation information when electrical stimulation is performed; a reaction detection unit that detects a state of change in response to electrical stimulation for each region based on response information; and an image generation unit that generates a composite image by superimposing the imaging data and the detection result of the reaction detection unit. , and a display control section that causes the display section to display the composite image.

The display control method according to the present invention includes the steps of acquiring imaging data obtained by photographing an image of the craniotomy brain of a patient undergoing surgery, and applying electrical stimulation to the patient's brain while the patient performs a predetermined task. a step of acquiring electrical stimulation response information representing a change in the patient's response to the electrical stimulation; a step of acquiring electrical stimulation information when the electrical stimulation is performed; and a step of obtaining electrical stimulation response information representing changes in the patient's response to the electrical stimulation. detecting the state of change in each region, generating a composite image by superimposing the imaging data and the detection result of the state of change in reaction, and displaying the composite image on a display unit. include.

The program according to the present invention includes a step of acquiring imaging data obtained by photographing an image of the craniotomy brain of a patient undergoing surgery, and a step of acquiring imaging data obtained by photographing an image of the craniotomy brain of a patient undergoing surgery. a step of acquiring electrical stimulation response information representing a change in response to the electrical stimulation; a step of acquiring electrical stimulation information when electrical stimulation is performed; and a step of obtaining electrical stimulation response information representing a change in the response to the electrical stimulation based on the electrical stimulation information and the electrical stimulation response information. a step of detecting the state of the brain for each region, a step of generating a composite image by superimposing the imaging data and the detection result of the brain state, and a step of displaying the composite image on a display unit. .

According to the present invention, active areas and inactive areas of the brain as a whole can be viewed from a bird's-eye view.

FIG. 1 is a block diagram showing a configuration example of a display device according to a first embodiment. FIG. 2 is a conceptual diagram for explaining the method of applying electrical stimulation to the brain according to the first embodiment. FIG. 3 is a flowchart showing the flow of processing of the display control device according to the first embodiment. FIG. 4 is a diagram showing an example of a surgical image according to the first embodiment. FIG. 5 is a diagram illustrating a configuration example of a display system according to the second embodiment. FIG. 6 is a block diagram showing a configuration example of a display device according to the second embodiment. FIG. 7 is a sequence diagram showing the flow of processing of the operator-side display device and the terminal-side display device according to the second embodiment. FIG. 8 is a diagram illustrating an example of a method for superimposing contact site instruction information on a surgical image according to the second embodiment. FIG. 9 is a sequence diagram showing the processing flow of the operator-side display device and the terminal-side display device according to the third embodiment. FIG. 10 is a diagram illustrating an example of a method for superimposing resection range instruction information on a surgical image according to the third embodiment.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to this embodiment, and in the following embodiments, the same parts are given the same reference numerals and redundant explanations will be omitted.

[First embodiment]
[Display device]
A configuration example of the display device according to the first embodiment will be described using FIG. 1. FIG. 1 is a block diagram showing a configuration example of a display device according to a first embodiment.

The display device 10 is placed in an operating room and is a device for sharing information between a surgeon and a patient. Specifically, the display device 10 allows a patient undergoing awake surgery to communicate with the medical team, and check the cognitive state by viewing stimulus responses such as the patient's reactions to the medical team's vocal and written instructions and questions. This device is used to proceed with the surgery.

The display device 10 is connected to an imaging device 20, an electroencephalogram measurement device 30, and an electrical stimulation device 40.

The imaging device 20 images the craniotomy of the patient's brain during surgery and generates an image. The imaging device 20 is comprised of medical equipment such as an operative field camera. The imaging device 20 outputs imaging data regarding images of the brain to the display device 10.

The electroencephalogram measuring device 30 is a device that measures a patient's brain waves. The brain wave measurement device 30 may be a well-known device that detects brain waves using electrical signals or magnetic signals. The electroencephalogram measuring device 30 measures electroencephalograms to confirm at least the visual, auditory, olfactory, language, and motor functions of the patient's brain, and generates electroencephalogram information.

The electroencephalogram measuring device 30 measures electroencephalograms related to visual function, for example, by asking a patient questions about the illustration while showing the illustration to the patient, and measuring electroencephalograms while the patient is answering the questions. The electroencephalogram measurement device 30 measures brain waves related to auditory function and language function, for example, by measuring brain waves when a patient is responding to a question from an operator or the like. The brain wave measurement device 30 measures brain waves related to olfactory function by measuring brain waves when a patient is sniffing something that stimulates the sense of smell, such as an aromatic agent. For example, the electroencephalogram measurement device 30 measures brain waves when the patient moves his/her limbs in response to an instruction such as "Please move your limbs" given to the patient by a surgeon, etc., thereby measuring the motor function. Measure brain waves.

The electrical stimulation device 40 is a device that applies electrical stimulation to an object by bringing electrodes into contact with the object. The electrical stimulation device 40 stimulates the patient's brain to detect active areas, which are areas where the brain is likely to be functioning, and inactive areas, which are areas where the brain is unlikely to be functioning. This device applies electrical stimulation to the patient and detects changes in the patient's response. FIG. 2 is a conceptual diagram for explaining the method of applying electrical stimulation to the brain according to the first embodiment. As shown in FIG. 2, the electrical stimulation device 40 applies electrical stimulation to the brain 90 of the patient by bringing the electrodes 31 into contact with two points, point P1 and point P2, on the patient's brain 90, for example. The electrical stimulation device 40 has a function of generating electrical stimulation information that is information regarding electrical stimulation given to a part of the brain. The electrical stimulation device 40 outputs electrical stimulation information to the display device 10. The areas of the brain to which electrical stimulation is applied include, for example, the part of the brain related to visual functions called the visual cortex (hereinafter referred to as the part of the visual cortex), the part of the brain related to auditory functions called the auditory cortex (hereinafter referred to as the part of the auditory cortex). (hereinafter referred to as the olfactory cortex), a part of the brain related to the language function called the language area or language center (hereinafter referred to as the part of the language area), a part of the brain related to the olfactory function called the olfactory cortex (hereinafter referred to as the part of the olfactory cortex), motor There is a part of the brain related to motor function called the motor cortex (hereinafter referred to as the motor cortex part), and while the patient is performing a specified task, electrical stimulation is applied to each part and changes in the patient's response are detected. I will do it. Here, the predetermined work performed by the patient includes, for example, responding to the way the patient sees things, responding to questions and inquiries from the patient, responding to the patient's hearing, uttering predetermined content by the patient, These include predetermined movements by patients, responses to patients' perceptions of smells and pain, and responses to what patients remember or recognize.

Similarly, electrical stimulation is applied to other parts of the brain while detecting changes in the patient's response. For example, if you want to detect changes in a patient's response depending on the presence or absence of electrical stimulation to a part of the visual cortex, you can show the patient a predetermined illustration and ask questions about the illustration without applying electrical stimulation. While looking at the illustration, electrical stimulation can be applied to the desired part of the visual cortex to detect changes in the patient's answers to questions about the illustration. For example, a question may be asked about the appearance of the illustration depending on the presence or absence of electrical stimulation, and changes in the patient's response may be detected based on whether or not the appearance of the illustration has changed.

Similarly, for example, when detecting changes in a patient's response depending on the presence or absence of electrical stimulation to a region of the auditory cortex, the patient's response to a predetermined question in the absence of electrical stimulation and the investigation in the region of the auditory cortex can be detected. It is preferable to detect a change in the patient's response to a predetermined question while applying electrical stimulation to the desired region as a change in the patient's response. For example, a question may be asked about the way the question is heard depending on the presence or absence of electrical stimulation, and changes in the patient's response may be detected based on whether or not the way the question is heard has changed.

Similarly, for example, if you want to detect changes in a patient's response depending on the presence or absence of electrical stimulation to a part of the language cortex, you can encourage the patient to make a predetermined utterance without applying electrical stimulation. While this is being performed, electrical stimulation can be applied to the part of the language cortex that is desired to be examined, and changes in the patient's response can be detected. For example, changes in the patient's response may be detected by determining whether the patient's speech continues or stops based on the presence or absence of electrical stimulation.

Similarly, for example, if you want to detect changes in a patient's response depending on the presence or absence of electrical stimulation to a region of the olfactory cortex, give the patient something that stimulates the sense of smell, such as a predetermined fragrance, without applying electrical stimulation. How a patient perceives an odor when smelling it, and how a patient smells something that stimulates the sense of smell, such as a prescribed fragrance, while electrical stimulation is applied to the part of the olfactory cortex that is desired to be investigated. It is best to detect changes in how the patient feels as changes in the patient's response. For example, a change in the patient's reaction may be detected based on whether or not the patient's sense of smell has changed depending on the presence or absence of electrical stimulation.

Similarly, for example, if you want to detect changes in a patient's response depending on the presence or absence of electrical stimulation to a part of the motor cortex, you can first encourage the patient to perform a predetermined movement without applying electrical stimulation, and then While this is being performed, electrical stimulation can be applied to the part of the motor cortex that is desired to be examined, and changes in the patient's response can be detected. For example, a change in the patient's response may be detected based on whether or not there is a change in the patient's motion due to the presence or absence of electrical stimulation.

Further, the electrical stimulation information may include time information for specifying the period during which electrical stimulation is applied, and information regarding the current value and voltage value when electrical stimulation is applied. From the time information included in this electrical stimulation information for specifying the period during which electrical stimulation is given, brain wave information during the period during which electrical stimulation is given, and the period during which electrical stimulation is given, can be obtained from the input unit 50. It is preferable that the information for specifying changes in the patient's motion be used as electrical stimulation response information, and used to detect active areas and inactive areas of the brain. The electrical stimulation response information obtained from the input unit 50 for identifying changes in the patient's motion includes operation information of the keyboard, mouse, touch panel, buttons, switches, etc. included in the input unit 50, audio information from a microphone, and a camera. It is preferable that the information is composed of image information from , or a combination thereof.

In mapping using electrical stimulation during awake surgery, for example, when mapping the language area of the brain, it is necessary to detect the cessation of normal speech, so it is better to use audio information from a microphone. This audio information may be used to detect active and inactive areas of the brain based on whether or not speech has stopped. Further, for example, in the case of the motor cortex of the brain, it is preferable that the input unit 50 includes voltage information obtained by an electromyograph, and that this voltage information is used. This voltage information may be used to detect active areas and inactive areas of the brain depending on whether the patient's movements have stopped. For example, in the case of the motor cortex of the brain, active areas of the brain can be determined based on whether or not the patient's movements have stopped, using operation information from the keyboard, mouse, touch panel, buttons, switches, etc. included in the input unit 50. It is recommended to use this method to detect inactive areas. Furthermore, if the area to be mapped in the brain is related to the brain waves of other areas, the brain wave information of the other related areas is used to determine the period during which the patient's brain waves are giving electrical stimulation, and the electrical stimulation. It may be used to detect active and inactive areas of the brain based on whether or not the patient's brain response was inhibited, depending on the difference from the period when no therapy was given.

The display device 10 includes an input section 50, a display section 52, an audio output section 54, a storage section 56, a communication section 58, and a control section (display control device) 60. The display device 10 is composed of an information processing device such as a personal computer.

The input unit 50 receives various input operations on the terminal device 12. The input unit 50 outputs an input signal corresponding to the received input operation to the control unit 60. The input unit 50 includes, for example, a keyboard, a mouse, a touch panel, a button, a switch, a microphone, a camera, an electromyograph, and the like. When a touch panel is used as the input unit 50, the input unit 50 is arranged on the display unit 52. The input unit 50 is preferably used for operating and controlling the terminal device 12. Further, the input unit 50 is preferably used when the operator inputs the resection area. Further, the input unit 50 is preferably used to obtain electrical stimulation response information representing a change in the patient's response to electrical stimulation applied to the patient's brain while the patient performs a predetermined task.

The display unit 52 displays various images. The display unit 52 is a display including, for example, a liquid crystal display (LCD), an organic EL (electro-luminescence), or the like. The display unit 52 is provided at a position where it can be viewed by the surgeon who performs the surgery and the patient who undergoes the surgery.

The audio output unit 54 outputs various types of audio. The audio output unit 54 is a speaker that outputs audio.

The storage unit 56 stores various information. The storage unit 56 stores information such as calculation contents of the control unit 60 and programs. The storage unit 56 includes, for example, at least one of a RAM (Random Access Memory), a main storage device such as a ROM (Read Only Memory), and an external storage device such as an HDD (Hard Disk Drive).

The communication unit 58 performs communication between the display device 10 and an external device. As will be described later, the communication unit 58 performs communication with other display devices 10.

The control section 60 controls each section of the display device 10. The control unit 60 includes, for example, an information processing device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage device such as a RAM (Random Access Memory) or a ROM (Read Only Memory). and has. The control unit 60 executes a program that controls the operation of the display device 10 according to the present invention. The control unit 60 may be realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 60 may be realized by a combination of hardware and software.

The control section 60 includes an imaging data acquisition section 70 , an electroencephalogram information acquisition section 72 , an electrical stimulation information acquisition section 74 , a reaction detection section 76 , an image generation section 78 , a display control section 80 , and a resection range specification section 82 and.

The imaging data acquisition unit 70 acquires imaging data obtained by imaging an object. Specifically, the imaging data acquisition unit 70 acquires imaging data from the imaging device 20 obtained by imaging the craniotomy brain of a patient undergoing surgery.

The brain wave information acquisition unit 72 acquires brain wave information indicating the measurement results of the patient's brain waves during surgery from the brain wave measurement device 30 . Specifically, the brain wave information acquisition unit 72 acquires brain wave information indicating visual, auditory, olfactory, language, and motor functions of the patient's brain from the brain wave measurement device 30 .

The electrical stimulation information acquisition unit 74 obtains electrical stimulation information, which is information regarding electrical stimulation given to a region of the patient's brain, from the electrical stimulation device 40 . Specifically, the electrical stimulation information acquisition unit 74 obtains electrical stimulation information regarding electronic stimulation given to the patient's brain in order to confirm the patient's brain function. Here, the electrical stimulation information may include time information for specifying the period during which electrical stimulation is applied, and information regarding the current value and voltage value when electrical stimulation is applied.

The reaction detection unit 76 defines a part of the brain where there is a high possibility that the brain is functioning as an active part of the brain, and a part of the brain where there is a low possibility that the brain is functioning as an inactive part of the brain. Detect as a location. Specifically, the reaction detection unit 76 detects the electroencephalogram information acquired by the electroencephalogram information acquisition unit 72, the electrical stimulation information acquired by the electrical stimulation information acquisition unit 74, and the change in the patient's reaction obtained from the input unit 50. Based on electrical stimulation response information, changes in the patient's response are detected, and active and inactive areas of the brain are detected. Here, from the time information for specifying the period during which electrical stimulation is being applied, which is included in the electrical stimulation information, the brain wave information during the period during which electrical stimulation is being applied, and the brain wave information during the period during which electrical stimulation is being applied are input from the input unit 50. Based on the electrical stimulation response information obtained, changes in the patient's response may be detected and used to detect active and inactive areas of the brain. The electrical stimulation response information obtained from the input section 50 includes operation information of the keyboard, mouse, touch panel, buttons, switches, etc. included in the input section 50, audio information from a microphone, image information from a camera, or It is preferable that the information is composed of such combinations.

In detecting active and inactive parts of the brain, the reaction detection unit 76 detects changes in the patient's reaction depending on the presence or absence of electrical stimulation, and if it is determined that the change in the patient's reaction is large, the brain is not functional. It is recommended to detect the part of the brain where the electrical stimulation was performed as the active part of the brain. In addition, the reaction detection unit 76 detects changes in the patient's reaction depending on the presence or absence of electrical stimulation, and if it is determined that the change in the patient's reaction is small, it is assumed that the brain is unlikely to be functioning. It is preferable to detect the part of the brain that has been electrically stimulated as an inactive part of the brain. Here, an evaluation value is obtained to judge the magnitude of the change in the patient's response, and the magnitude of the change in the patient's response is determined based on a predetermined threshold value that differs for each part of the brain that controls different functions. Good too. The evaluation value may be calculated based on the amount of change in the electrical stimulation response information, or may be calculated based on the correlation of the electrical stimulation response information.

In mapping using electrical stimulation during awake surgery, for example, when mapping the language area of the brain, it is necessary to detect the cessation of normal speech, so it is better to use audio information from a microphone. It is preferable to use this audio information to detect active areas and inactive areas of the brain based on whether speech has stopped or not. For example, if speech has stopped, it is determined that the change in the patient's response is significant, and the region of the brain to which electrical stimulation was performed may be detected as an active region of the brain. If speech has not stopped, it is determined that the change in the patient's response is small, and the region of the brain to which electrical stimulation has been applied may be detected as an inactive region of the brain.

Further, for example, in the case of the motor cortex of the brain, it is preferable that the input unit 50 includes voltage information obtained by an electromyograph, and that this voltage information is used. It is preferable to use this voltage information to detect active areas and inactive areas of the brain depending on whether or not the patient's movements have stopped. For example, in the case of the motor cortex of the brain, active areas of the brain can be determined based on whether or not the patient's movements have stopped, using operation information from the keyboard, mouse, touch panel, buttons, switches, etc. included in the input unit 50. It is preferable to perform detection between the active area and the inactive area. For example, if the patient's movements stop, it is determined that the change in the patient's response is large, and the region of the brain to which electrical stimulation was performed may be detected as an active region of the brain. If the patient's movements have not stopped, it is determined that the change in the patient's response is small, and the region of the brain that has been electrically stimulated may be detected as an inactive region of the brain.

Furthermore, if the area to be mapped in the brain is related to the brain waves of other areas, the brain wave information of the other related areas is used to determine the period during which the patient's brain waves are giving electrical stimulation, and the electrical stimulation. It is preferable to detect active areas and inactive areas of the brain based on the difference in electroencephalogram information from a period in which brain waves are not given. For example, if we calculate the correlation between the EEG information when electrical stimulation is applied and the EEG information when no electrical stimulation is applied, if the correlation value of the EEG information is small, the EEG information is significantly different and the patient's response changes. It is preferable that the part of the brain determined to be large and subjected to electrical stimulation be detected as an active part of the brain. If the correlation value of the electroencephalogram information is large, the difference in the electroencephalogram information is small, so it is determined that the change in the patient's response is small, and the part of the brain that was electrically stimulated may be detected as an inactive part of the brain. .

The image generation unit 78 generates various images. The image generation unit 78 superimposes a brain image and information indicating the presence or absence of a function in each brain region based on the imaging data acquired by the imaging data acquisition unit 70 and the detection result of the reaction detection unit 76. Generate a composite image (surgical image).

The display control section 80 causes the display section 52 to display various images. The display control unit 80 causes the display unit 52 to display the composite image generated by the image generation unit 78.

The resection range specifying unit 82 superimposes instruction information on the composite image in accordance with instructions input by the operator into the input unit 50. Specifically, the resection range specifying unit 82 selects a first resection range that includes the surgical site and a portion where the brain is unlikely to be functioning on the display unit in accordance with an instruction input by the operator into the input unit 50. It is superimposed on the composite image displayed at 52.

[Processing content]
The processing contents of the display control device according to the first embodiment will be explained using FIG. 3. FIG. 3 is a flowchart showing the flow of processing of the display control device according to the first embodiment. Rather than shown in FIG. 3, the process shown is performed during surgery on a patient's brain.

The imaging data acquisition unit 70 acquires imaging data obtained by imaging the craniotomy brain of a patient undergoing surgery from the imaging device 20 (step S10). Specifically, the imaging data acquisition unit 70 may acquire imaging data all the time or at predetermined intervals. The imaging data acquired by the imaging data acquisition unit 70 may be moving image data or still image data. Then, the process advances to step S12.

The brain wave information acquisition unit 72 acquires brain wave information indicating the measurement results of the patient's brain waves during surgery from the brain wave measurement device 30 (step S12). Specifically, the brain wave information acquisition unit 72 acquires brain wave information indicating visual, auditory, olfactory, language, and motor functions of the patient's brain from the brain wave measurement device 30 . Then, the process advances to step S14.

The electrical stimulation information acquisition unit 74 obtains electrical stimulation information, which is information regarding electrical stimulation given to a region of the patient's brain, from the electrical stimulation device 40 (step S14). Specifically, the electrical stimulation information acquisition unit 74 obtains electrical stimulation information regarding electronic stimulation given to the patient's brain in order to confirm the patient's brain function. The electrical stimulation information acquisition unit 74 obtains electrical stimulation information for each predetermined region in the entire brain of the patient. Then, the process advances to step S16.

The reaction detection unit 76 detects a change in the patient's reaction (step S16). Specifically, the reaction detection unit 76 detects the electroencephalogram information acquired by the electroencephalogram information acquisition unit 72, the electrical stimulation information acquired by the electrical stimulation information acquisition unit 74, and the change in the patient's reaction obtained from the input unit 50. Active and inactive areas of the brain are detected based on electrical stimulation response information. In detecting active and inactive parts of the brain, the reaction detection unit 76 detects changes in the patient's reaction depending on the presence or absence of electrical stimulation, and if it is determined that the change in the patient's reaction is large, the brain is not functional. It is recommended to detect the part of the brain where the electrical stimulation was performed as the active part of the brain. In addition, the reaction detection unit 76 detects changes in the patient's reaction depending on the presence or absence of electrical stimulation, and if it is determined that the change in the patient's reaction is small, it is assumed that the brain is unlikely to be functioning. It is preferable to detect the part of the brain that has been electrically stimulated as an inactive part of the brain. Here, an evaluation value is obtained to judge the magnitude of the change in the patient's response, and the magnitude of the change in the patient's response is determined based on a predetermined threshold value that differs for each part of the brain that controls different functions. Good too. The evaluation value may be calculated based on the amount of change in the electrical stimulation response information, or may be calculated based on the correlation of the electrical stimulation response information. Then, the process advances to step S18.

The image generation unit 78 generates a brain image and information indicating active and inactive areas for each region of the brain, based on the imaging data acquired by the imaging data acquisition unit 70 and the detection results of the reaction detection unit 76. A composite image is generated in which the images are superimposed (step S18). Then, the process advances to step S20.

The display control unit 80 causes the display unit 52 to display the composite image generated by the image generation unit 78 (step S20). Then, the process advances to step S22.

The resection range specifying unit 82 superimposes instruction information on the composite image in accordance with the instruction input by the operator into the input unit 50 (step S22). Specifically, the resection range specifying unit 82 superimposes a first resection range including the surgical site on the composite image displayed on the display unit 52 in accordance with an instruction input by the surgeon into the input unit 50. Here, the first resection range may include a portion where the brain is unlikely to be functioning as necessary in order to resect the surgical site rationally and economically.

FIG. 4 describes an example of a surgical image according to the first embodiment. FIG. 4 is a diagram showing an example of a surgical image according to the first embodiment.

The surgical image 100 includes a brain image 110, a first region 120, a second region 130, a tumor region 140, and a first resection range 150. Here, the first region 120, the second region 130, and the tumor region 140 are shown as circular regions for convenience, but they may be closed regions of any shape specified as regions.

Brain image 110 is a brain image of a patient undergoing surgery. The first region 120 is a region where the reaction detection unit 76 detects that the brain is not functioning. The second region 130 is a region where the reaction detection unit 76 detects that the brain is functioning. Tumor region 140 indicates a tumor to be removed. Tumor region 140 is a type of surgical site. The first resection range 150 is a region to be surgically removed. The first region 120, the second region 130, the tumor region 140, and the first resection range 150 are displayed as an overlay on the brain image 110, for example. Here, in mapping using electrical stimulation in awake surgery, the first region 120 is a region where the brain is detected as not functioning by the reaction detection unit 76, for example, the patient's brain 90 shown in FIG. The electrode 31 is brought into contact with two points, point P1 and point P2, to apply electrical stimulation to the brain 90, and if it is detected that the brain is not functioning, the region between the electrodes is included. It is best to set it like this. In addition, it is preferable to detect brain function using electrodes in the vicinity of points P1 and P2, identify areas between multiple electrodes where the brain is not functioning, and set the area to include the identified areas. . Furthermore, a region may be set larger by a predetermined amount along the region where the brain is actually detected to be not functioning. Alternatively, a region may be set smaller by a predetermined amount along the region where the brain is actually detected to be not functioning. Similarly, the second region 130 is a region where the brain is detected to be functioning by the reaction detection unit 76, and for example, two points, point P1 and point P2, of the patient's brain 90 shown in FIG. The electrode 31 may be brought into contact with the brain 90 to apply electrical stimulation to the brain 90, and if it is detected that the brain is functioning, the setting may be made to include the area between the electrodes. In addition, it is preferable to detect brain function using electrodes in the vicinity of points P1 and P2, identify areas between multiple electrodes where the brain is functioning, and set the area to include the identified areas. . Furthermore, the area may be set larger by a predetermined amount along the area where the brain is actually detected to be functioning. Alternatively, a region may be set smaller by a predetermined amount along the region where the brain is actually detected to be functioning. In addition, in mapping using electrical stimulation during awake surgery, it is recommended to specify a starting position from a predetermined reference position and mapping using electrical stimulation is performed within an area with a certain interval, as shown in Figure 4. . For example, if the upper left position of the brain image being photographed is set as a reference position for the electrical stimulation device 40, and a starting position from the reference position is set, electrical stimulation will be applied sequentially from the starting position within a region at regular intervals. The electrical stimulation device 40 may guide the mapping. The mapping position may be guided by an overlay display.

The tumor region 140 and the first resection range 150 are regions designated by the operator. The operator specifies the tumor region 140 and the first resection range 150 while the first region 120 and the second region 130 are displayed on the surgical image 100. The tumor region 140 can be identified by an operator's visual inspection, MRI (Magnetic Resonance Imaging), CT (Computed Tomography), or the like. After specifying the tumor region 140, the operator specifies the first resection range 150. By confirming the first region 120, the second region 130, and the tumor region 140 on the surgical image 100, the surgeon can excise the tumor region 140 while leaving the second region 130. An optimal first resection range 150 can be specified, including the first region 120, if necessary.

In this embodiment, the electrical stimulation device 40 is used to detect active areas, which are areas where the brain is likely to be functioning, and inactive areas, which are areas where the brain is unlikely to be functioning. , the patient is prompted to perform a predetermined task, electrical stimulation is applied to the patient's brain, and changes in the patient's response depending on the presence or absence of electrical stimulation are detected. For example, the patient is encouraged to repeatedly and continuously perform the actions of "grasping hands" and "opening hands" as predetermined tasks. It is known that the area of the brain that controls hand movements is the primary motor cortex of the precentral gyrus, a gyrus located just in front of the central sulcus that divides the cerebral hemispheres into anterior and posterior parts. This primary motor cortex includes areas related to hand movements. The electric stimulation device 40 is used to apply electrical stimulation to the parts related to hand movement, and the patient is encouraged to repeat and continuously perform the actions of "grasping the hand" and "opening the hand." It detects whether there are any changes in the patient's response, such as the patient stopping when the stimulus is applied, or feeling uncomfortable when holding or opening the hand. If it is determined that the change in the patient's response is large, it is assumed that the brain is likely to be functioning, and the part of the brain to which electrical stimulation was applied is detected as an active part of the brain. In addition, the reaction detection unit 76 detects changes in the patient's reaction depending on the presence or absence of electrical stimulation, and if it is determined that the change in the patient's reaction is small, it is assumed that the brain is unlikely to be functioning. The part of the brain that has been electrically stimulated is detected as an inactive part of the brain and displayed on the display section 52 as a composite image. In this way, by displaying the active region and inactive region of the brain as a composite image on the display unit 52, medical staff can appropriately understand the detected region.

In this embodiment, the surgeon and patient view the surgical image 100 during the surgery. Therefore, for example, the surgeon can explain to the patient, while looking at the surgical image 100, that the first resection range 150 will be resected and that the region where the brain is likely to be functioning will not be resected. This can reduce the patient's anxiety about surgery and improve motivation for rehabilitation.

As described above, in the first embodiment, by superimposing and displaying active areas and inactive areas on a brain image, it is possible to obtain a bird's-eye view of functionally active areas and inactive areas of the brain as a whole. This allows medical staff to appropriately understand which parts of the entire brain can be excised and which parts should be preserved.

Furthermore, in the first embodiment, by sharing the same surgical image with the surgeon, the patient can improve his or her understanding of the condition of his or her body and the surgical procedure. Thereby, the first embodiment can reduce the patient's anxiety about surgery and improve motivation for rehabilitation.

[Second embodiment]
A second embodiment of the present invention will be described. FIG. 5 is a diagram illustrating a configuration example of a display system according to the second embodiment.

The display system 1 includes a display device 10A-1 and a display device 10A-2. The display device 10A-1 is placed, for example, in an operating room where a surgeon performs surgery on a patient. The display device 10A-1 is also called an operator-side display device. The display device 10A-2 is a device used by a collaborator who gives advice regarding the treatment. In this embodiment, the collaborator is, for example, a veteran doctor with more experience than the surgeon. The display device 10A-1 and the display device 10A-2 are connected to each other via a network N so that they can communicate with each other. The display device 10A-1 and the display device 10A-2 are configured to send and receive surgical images to and from each other. The display device 10A-2 transmits various types of advice to the display device 10A-1 via the network N. The display device 10A-1 and the display device 10A-2 may be collectively referred to as the display device 10A.

[Display device]
A configuration example of the display device according to the second embodiment will be described using FIG. 6. FIG. 6 is a block diagram showing a configuration example of a display device according to the second embodiment.

As shown in FIG. 6, the display device 10A differs from the display device 10 shown in FIG.

The contact site instructing unit 84 is a contact site for instructing the surgical image received from another display device 10A via the communication unit 58 as a location where electrical stimulation is to be applied and electrical stimulation information is to be obtained, according to a user's instruction. Generate instruction information. The contact site instruction information may be information for instructing reexamination. The contact site instruction section 84 transmits the generated contact site instruction information to the other display device 10A via the communication section 58. Details of the contact site indicating section 84 will be described later.

The resection range instructing unit 86 instructs a surgical image indicating a first resection range received from another display device 10A via the communication unit 58 to instruct a second resection range different from the first resection range according to a user's instruction. Generates resection range instruction information indicating. The resection range instruction section 86 transmits the generated resection range instruction information to the other display device 10A via the communication section 58. Details of the resection range indicating section 86 will be described later.

[Processing content]
The processing contents of the operator side display device and the terminal person side display device according to the second embodiment will be explained using FIG. 7. FIG. 7 is a sequence diagram showing the flow of processing of the operator-side display device and the terminal-side display device according to the second embodiment.

The control unit 60A of the display device 10A-1 transmits the generated surgical image to the display device 10A-2 via the communication unit 58 (step S30). The control unit 60A of the display device 10A-2 receives the surgical image from the display device 10A-1 via the communication unit 58 (step S32).

The display control unit 80 of the display device 10A-2 displays the received surgical image on the display unit 52 of the display device 10A-2 (step S34). The display section 52 of the display device 10A-1 is sometimes called a first display section, and the display section 52 of the display device 10A-2 is sometimes called a second display section.

The contact site instructing unit 84 of the display device 10A-2 determines the position at which electrical stimulation is applied to the surgical image displayed on the display unit 52 to obtain electrical stimulation information according to instructions input by the collaborator into the input unit 50. Contact point instruction information is generated that instructs (step S36). The collaborator may include text information or audio information in the contact site information. The collaborator can include text information in the contact site information by inputting text using the input unit 50. In addition, the input unit 50 is equipped with a microphone, and by collecting the voice of the cooperator with the microphone and using it as voice information, the voice information can be included in the contact site information. For example, the collaborator may include the reason for instructing the position for acquiring the electrical stimulation information as text information in the contact site information.

The control unit 60A of the display device 10A-2 transmits the contact site instruction information to the display device 10A-1 via the communication unit 58 (step S38). The control unit 60A of the display device 10A-1 receives the contact site instruction information from the display device 10A-2 via the communication unit 58 (step S40).

The display control unit 80 of the display device 10A-2 displays the contact site instruction information superimposed on the surgical image (step S42). FIG. 8 is a diagram illustrating an example of a method for superimposing contact site instruction information on a surgical image according to the second embodiment. As shown in FIG. 8, the surgical image 100A includes a contact site designation area 160. For example, by viewing the surgical image 100A, the surgeon applies electrical stimulation information to the contact site designation area 160 using the electrical stimulation device 40. Thereby, the display device 10 can obtain electrical stimulation information of the contact site indication area 160. Note that if the contact site instruction information includes text information, the display control unit 80 may display the text information side by side with the surgical image 100A.

As described above, in the second embodiment, new electrical stimulation information is obtained by applying electrical stimulation based on instructions from an experienced collaborator. This allows the surgeon to designate a more appropriate resection range with reference to the opinions of his collaborators.

[Third embodiment]
A third embodiment of the present invention will be described. The display system and display device according to the third embodiment are respectively. Since it is the same as the display system shown in FIG. 5 and the display device shown in FIG. 6, the explanation will be omitted.

[Processing content]
The processing contents of the operator side display device and the terminal person side display device according to the third embodiment will be explained using FIG. 9 . FIG. 9 is a sequence diagram showing the processing flow of the operator-side display device and the terminal-side display device according to the third embodiment.

The processing from step S50 to step S54 is the same as the processing from step S30 to step S34 shown in FIG. 7, so a description thereof will be omitted.

The resection range instruction section 86 of the display device 10A-2 generates resection range instruction information indicating the second resection range based on the instruction input by the collaborator into the input section 50 with respect to the surgical image displayed on the display section 52. (Step S56). The collaborator may include text information in the resection range instruction information. By inputting characters using the input unit 50, the collaborator can include character information in the resection range instruction information. For example, the collaborator may include the reason for instructing the resection range as text information in the resection range instruction information.

The control unit 60A of the display device 10A-2 transmits the resection range instruction information to the display device 10A-1 via the communication unit 58 (step S58). The control unit 60A of the display device 10A-1 receives the resection range instruction information from the display device 10A-1 via the communication unit 58 (step S60).

The display control unit 80 of the display device 10A-2 displays the resection range instruction information superimposed on the surgical image (step S62). FIG. 10 is a diagram illustrating an example of a method for superimposing resection range instruction information on a surgical image according to the third embodiment. As shown in FIG. 10, the surgical image 100B includes a second resection range 170. For example, by visually checking the surgical image 100B, the surgeon can distinguish between the first resection range 150 designated by himself and the second resection range 170 designated by the collaborator. It will be possible to check different aspects on one screen so that they can be easily distinguished. Note that if the excision range instruction information includes text information, the display control unit 80 may display the text information side by side with the surgical image 100B.

As described above, in the third embodiment, the surgical range designated by the surgeon himself and the surgical range designated by an experienced collaborator can be displayed on a single screen of information. This allows the surgeon to specify a more appropriate surgical range by comparing the surgical range specified by the surgeon with the surgical range specified by the collaborator.

Each component of each device shown in the drawings is functionally conceptual, and does not necessarily need to be physically configured as shown in the drawings. In other words, the specific form of distributing and integrating each device is not limited to what is shown in the diagram, and all or part of the devices can be functionally or physically distributed or integrated in arbitrary units depending on various loads and usage conditions. Can be integrated and configured. Note that this distribution/integration configuration may be performed dynamically.

Although the embodiments of the present invention have been described above, the present invention is not limited to the contents of these embodiments. Furthermore, the above-mentioned components include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those that are in a so-called equivalent range. Furthermore, the aforementioned components can be combined as appropriate. Furthermore, various omissions, substitutions, or changes of the constituent elements can be made without departing from the gist of the embodiments described above.

This disclosure includes matters that contribute to the realization of the SDGs of "Good health and well-being for all" and contribute to value creation through healthcare products and services.

1 display system 10, 10A display device 20 imaging device 30 electroencephalogram measurement device 40 electrical stimulation device 50 input section 52 display section 54 audio output section 56 storage section 58 communication section 60, 60A control section (display control device)
70 Imaging data acquisition unit 72 Electroencephalogram information acquisition unit 74 Electrical stimulation information acquisition unit 76 Reaction detection unit 78 Image generation unit 80 Display control unit 82 Resection range designation unit 84 Contact site designation unit 86 Resection range designation unit

Claims (4)

an imaging data acquisition unit that acquires imaging data obtained by capturing an image of a craniotomy brain of a patient undergoing surgery;
a first input unit that acquires electrical stimulation response information representing a change in the patient's response to electrical stimulation applied to the patient's brain while the patient is performing a predetermined task;
an electrical stimulation information acquisition unit that obtains electrical stimulation information when the electrical stimulation is performed;
a reaction detection unit that detects a state of change in response to the electrical stimulation for each region based on the electrical stimulation information and the electrical stimulation response information;
an image generation unit that generates a composite image by superimposing the imaging data and the detection result of the reaction detection unit;
a display control unit that displays the composite image on a display unit;
A display control device comprising: a second input section for inputting the first resection range;
a communication unit that receives information regarding the second resection range input from an external input unit different from the second input unit;
The image generation unit superimposes the first resection range and the second resection range on the composite image in different ways,
The display control device according to claim 1. acquiring imaging data of an image of a craniotomy brain of a patient undergoing surgery;
obtaining electrical stimulation response information representing a change in the patient's response to electrical stimulation applied to the patient's brain while the patient is performing a predetermined task;
acquiring electrical stimulation information when performing the electrical stimulation;
detecting a state of change in response to the electrical stimulation for each region based on the electrical stimulation information and the electrical stimulation response information;
generating a composite image by superimposing the imaging data and the detection result of the state of change in the reaction;
Displaying the composite image on a display unit;
including display control methods. acquiring imaging data of an image of a craniotomy brain of a patient undergoing surgery;
obtaining electrical stimulation response information representing a change in the patient's response to electrical stimulation applied to the patient's brain while the patient is performing a predetermined task;
acquiring electrical stimulation information when performing the electrical stimulation;
identifying a state of change in response to the electrical stimulation for each region based on the electrical stimulation information and the electrical stimulation response information;
generating a composite image by superimposing the imaging data and the detection result of the state of change in the reaction;
Displaying the composite image on a display unit;
A program that causes a computer to execute.

Images