JP2011036370A - Medical image recording apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a medical image recording apparatus capable of displaying or reproducing useful moving images by short-time and easy operation. <P>SOLUTION: The medical image recording apparatus includes: a moving image acquisition part for acquiring medical moving images; an information acquisition part for acquiring medical information; a feature point detecting part for detecting feature points from the information detected by at least one of the moving image acquisition part and information acquisition part; a synchronizing part for synchronizing the acquired moving images and information; an input detecting part for detecting a command to reproduce the moving images input by an operator; and a control part for reproducing the moving images acquired by the moving image acquisition part if the input detecting part detects the input of the reproduction command with reference to the time when the feature points are detected by the feature point detecting part. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention belongs to the field of recording images and information related to the subject of the image, and particularly relates to a medical image recording apparatus that records and reproduces surgical images, biological data, and surgical information.

  In endoscopic surgery (surgery using a fiberscope endoscope or laparoscopic surgery), there is a surgery performed by taking an image, displaying the image on a monitor, and checking the image on the monitor. In addition, as an apparatus for recording an image photographed by these cameras, there is an apparatus for simultaneously recording an image photographed by a camera and patient biometric data (pulse and blood pressure) acquired by another device. In addition to an image taken with an endoscope, there is also an apparatus that records an image taken with a camera or the like in an operating room.

  For example, Patent Document 1 discloses a medical image creation apparatus that edits a moving image that has been photographed and converted into electronic information by a medical photographing device, an input unit that inputs the moving image photographed by the medical photographing device, and the input Temporary storage means for temporarily storing the moving picture input to the means, a display section for reproducibly displaying the moving picture stored in the temporary storage means, and an operation section for operating the moving picture displayed on the display section A medical image creation comprising: an extraction unit that extracts a desired still image from the moving image; and an output unit that displays a still image display unit that displays a list of the extracted still images on the same screen. An apparatus is described.

Japanese Patent Laying-Open No. 2005-103030

  By recording a medical image in this way, it is possible to reproduce an image on which a medical image and a plurality of information are displayed. Here, in surgery, it may be necessary to accurately grasp the progress of the surgery in order to solve the problem or to give an instruction from the instructor. In this case, the progress of the surgery can be grasped by reproducing the recorded image with the apparatus that recorded the image.

  However, if an image is simply reproduced, it is necessary to search for the necessary image by itself, and it takes time to obtain necessary information. Further, in the apparatus described in Patent Document 1, a desired still image can be extracted, but an operator viewing the monitor needs to perform an operation. In other words, even with the device described in Patent Document 1, it is necessary to search for a necessary image by displaying the image by itself. For this reason, even with the apparatus described in Patent Document 1, it takes time to acquire necessary information.

  The present invention has been made in view of the above, and it is an object of the present invention to provide a medical image recording apparatus capable of displaying or reproducing a useful moving image in a short time and with a simple operation.

  In order to solve the above-described problems and achieve the object, the present invention provides a moving image acquisition unit that acquires a medical moving image, an information acquisition unit that acquires medical information, the moving image acquisition unit, and the information acquisition A feature point detection unit that detects a feature point from information detected at least one of the units, a synchronization unit that synchronizes the acquired video and information, and an input detection unit that detects a playback instruction of the video input by an operator And when the input detection unit detects that the reproduction instruction is input, a control unit that reproduces the moving image acquired by the moving image acquisition unit with reference to a time point when the feature point is detected by the feature point detection unit; It is characterized by having.

  The information acquisition unit includes a detection function for detecting an operation of a surgical tool, and the feature point detection unit detects the operation of the surgical tool by the information acquisition unit, and uses the operation as a feature point. It is preferable to detect.

  In addition, the information acquisition unit has a detection function of detecting biological information of a patient, and the feature point detection unit is configured such that the biological information detected by the information acquisition unit is a value that is out of a preset range. It is preferable to detect it as a feature point.

  Moreover, it is preferable that the said moving image acquisition part acquires the image acquired with the endoscope.

  Moreover, it is preferable that the said control part reproduces | regenerates the moving image from the time before the said feature point for the preset time.

  Further, the display control unit includes a display control unit that generates an image to be displayed, and the display control unit includes an image before starting the playback and an image acquired by the moving image acquisition unit before detecting the playback instruction. It is preferable to display an image.

  Moreover, it is preferable that the said input detection part is a touch panel which detects the contact to the display part which displays the image produced | generated by the said display control part.

  The input detection unit is preferably a foot switch.

  The moving image acquisition unit is preferably connected in an electrically non-contact state with an image capturing device that captures a medical moving image.

  Moreover, it is preferable that the information acquisition unit is electrically connected to an information generation device that generates medical information.

  The medical image recording apparatus according to the present invention extracts a start point of an image that is highly likely to be useful as a feature point, so that an operator can view a useful image with a simple operation. There is an effect. In addition, since there are feature points, it is possible to easily search for a useful image, and it is possible to view a useful image in a short time even when searching for an image.

FIG. 1 is a block diagram showing a schematic configuration of an example of a medical system including a medical image recording apparatus according to the present invention. FIG. 2 is a block diagram showing a schematic configuration of the medical image recording apparatus shown in FIG. FIG. 3 is an explanatory diagram for explaining an example of an image and information recording method. FIG. 4 is a screen diagram illustrating an example of an image displayed on the display. FIG. 5 is a flowchart showing an example of processing by the medical image recording apparatus. FIG. 6A is a screen diagram for explaining an example of processing by the medical image recording apparatus. FIG. 6B is a screen diagram for explaining an example of processing by the medical image recording apparatus. FIG. 6C is a screen diagram for explaining an example of processing by the medical image recording apparatus. FIG. 7 is a flowchart showing another example of processing by the medical image recording apparatus. FIG. 8 is an explanatory diagram for explaining an example of a method for detecting a bleeding point. FIG. 9 is a flowchart showing another example of processing by the medical image recording apparatus. FIG. 10 is an explanatory diagram showing the relationship between the feature points detected by the medical image recording apparatus and the recording data.

  Embodiments of a medical image recording apparatus according to the present invention will be described below in detail with reference to the drawings. Note that the present invention is not limited to the embodiments. For example, in the following embodiment, the medical image recording apparatus is described as an apparatus that acquires and reproduces an image captured by an endoscope unit. However, the type of image that is acquired and reproduced is not limited, and an operative field is imaged. A camera image may be acquired and played back.

  FIG. 1 is a block diagram showing a schematic configuration of an example of a medical system including a medical image recording apparatus according to the present invention. The medical system 10 includes a medical image recording device 12, a display 14, an endoscope unit 16, a biological monitor 18, an operating room imaging unit 20, a surgical instrument 22, an output unit 24, and an operation detection unit 26. And a foot switch 28.

  The medical image recording device 12 acquires an image (moving image), biological information, and operation information from each unit to be described later, and records the acquired information. Further, the medical image recording device 12 generates and reproduces an image based on the acquired information. The medical image recording apparatus 12 will be described in detail later.

  The display 14 is a device that displays an image such as a liquid crystal display, and displays an image produced by the medical image recording device 12.

  The endoscope unit 16 is a device that is inserted into a patient to be operated and acquires an in-vivo image. The endoscope unit 16 is inserted into a patient and includes an insertion portion 16a provided with an imaging mechanism at the tip, an operation portion 16b operated by an operator (such as a surgeon, that is, a doctor), and an imaging mechanism of the insertion portion 16a. The processing unit 16c processes the acquired image information and generates an image. In addition to the fiberscope type endoscope in which the insertion portion is formed of a soft material, the endoscope of the endoscope unit 16 includes a cystoscope, thoracoscope, abdominal cavity, in which the insertion portion is formed of a hard material. A mirror or the like can be used. Further, the endoscope unit 16 sends the acquired image information to the medical image recording apparatus 12. Note that the endoscope unit 16 and the medical image recording apparatus 12 are electrically insulated and transmit and receive information by optical communication using, for example, an optical coupler.

  The biological monitor 18 measures or detects biological data of the patient and displays it on the display. Here, examples of the biological data measured or detected by the biological monitor 18 include a pulse, a heart rate, a blood pressure, a body temperature, an electroencephalogram, a blood oxygen concentration, and a blood carbon dioxide concentration. In addition, the biological monitor 18 sends the acquired biological data to the medical image recording device 12, similarly to the endoscope unit 16. The biological monitor 18 and the medical image recording apparatus 12 are also electrically insulated, and information is transmitted / received by optical communication using an optical coupler, for example.

  The operating room imaging unit 20 is a camera that images an operating room in which surgery is performed. The operating room photographing unit 20 can photograph a person in the operating room, an operating table, and various devices. Note that the operating room imaging unit 20 may be a fixed type, may be configured to capture an image of the same region, or may be movable, and the imaging region may be changeable. Similarly to the endoscope unit 16, the operating room imaging unit 20 also sends acquired image information to the medical image recording device 12.

  The surgical tool (surgical instrument) 22 is an instrument used for surgery such as an electric knife and an ultrasonic knife. In FIG. 1, two surgical tools 22 are shown, but the number of surgical tools is not limited. The output unit 24 is connected to the surgical tool 22 and drives the surgical tool 22 based on the operation of the operator. That is, the output unit 24 supplies power to the surgical instrument 22 such as an electric knife or an ultrasonic knife to drive it. The surgical tool 22 is supplied with electric power from the output unit 24 and is driven to cut the patient's tissue or burn the patient's tissue.

  The operation detection unit 26 is disposed in a cord portion that connects the surgical tool 22 and the output unit 24, detects current flowing between the surgical tool 22 and the output unit 24, and is the surgical tool 22 used? Detect whether or not. As the operation detection unit 26, for example, a clamp meter (overhead ammeter) that can detect whether a current flows only by sandwiching a cord through which the current flows can be used. The clamp meter detects the current flow by detecting the magnetic field generated around the cord when the current flows. The operation detection unit 26 sends the detected value to the medical image recording apparatus 12. The medical image recording apparatus 12 determines whether the surgical tool 22 has been used based on the detection value sent from the operation detection unit 26. The operation detection unit 26 may detect whether the surgical tool 22 has been used from the detected value, and send the detection result to the medical image recording apparatus 12.

  The foot switch 28 is an input unit through which an operator inputs instructions to the medical image recording apparatus 12 and is disposed at the operator's foot. The foot switch 28 sends an instruction input by the operator to the medical image recording apparatus 12. The medical system 10 is configured as described above.

  Next, the medical image recording apparatus 12 will be described with reference to FIG. FIG. 2 is a block diagram showing a schematic configuration of the medical image recording apparatus shown in FIG. As described above, the medical image recording device 12 is a device that records and reproduces images and data supplied from each unit, and includes a moving image acquisition unit 40, an information acquisition unit 42, a synchronization unit 44, and feature point detection. Unit 46, input detection unit 48, display control unit 50, storage unit 52, transmission / reception unit 54, and control unit 56.

  The moving image acquisition unit 40 acquires image data from each unit of the medical system 10. Specifically, the image data sent from the endoscope unit 16 and the image data sent from the operating room imaging unit 20 are acquired. That is, the moving image acquisition unit 40 acquires the image data acquired by the endoscope unit 16 from the endoscope unit 16 and acquires the image data acquired by the operating room imaging unit 20 from the operating room imaging unit 20.

  The information acquisition unit 42 acquires information acquired by each unit of the medical system 10. Specifically, the information acquisition unit 42 acquires biological data sent from the biological monitor 18. That is, the information acquisition unit 42 acquires the biological data acquired by the biological monitor 18 from the biological monitor 18. Moreover, the information acquisition part 42 acquires the value detected by the operation detection part 26, and acquires the information of whether the surgical tool 22 was operated based on the acquired value.

  The synchronization unit 44 synchronizes the image data acquired by the moving image acquisition unit 40 and various data acquired by the information acquisition unit 42. That is, the synchronization unit 44 matches the time axes of each image data and various data, and associates each image data with various data. By synchronizing with the synchronization unit 44, it is possible to prevent the time axis of each data from being shifted during data reproduction. Here, the method of synchronizing each data with the synchronizing part 44 is not specifically limited, For example, what is necessary is just to add flag data to each data. Further, data for every fixed time may be packetized and stored. Data of the same time may be used as one unit.

  The feature point detection unit 46 detects a time when a certain change has occurred from the image data acquired by the moving image acquisition unit 40 and various data acquired by the information acquisition unit 42, and the time when the change has occurred is a feature point. And In the present embodiment, the use of the surgical tool 22 is detected as a feature point, and the time point when the surgical tool 22 is determined to be used is detected as a special vertex based on the information acquired by the information acquisition unit 42.

  The input detection unit 48 detects a signal sent from the operation detection unit 26. That is, the input detection unit 48 detects a signal input when the operator operates the operation detection unit 26.

  The display control unit 50 generates an image to be displayed on the display 14 based on the data sent from the control unit 56. The storage unit 52 stores data sent from the control unit 56. Note that a cache memory can be used as the storage unit 52. By using a cache memory as the storage unit 52, data can be read out in a short time. The storage unit 52 may be a storage unit that can store a large amount of data although it takes time to read data from a hard disk or the like. Examples of the format for recording (encoding) the acquired moving picture information include MPEG (Moving Picture Experts Group) -1, MPEG-2, MPEG-4 (mainly in the case of mobile communication, H.261 (International For example, in MPEG-2, the encoded information is composed of PES (Packetized Elementary Stream) packets (variable length), which are packets of appropriate length units. The moving image information is acquired as a stream composed of a plurality of PES packets.

  The transmission / reception unit 54 transmits / receives data to / from an external network or an external storage device. The transmission / reception unit 54 sends data generated by each unit (image data to be reproduced) and data stored in the storage unit 52 to an external network or an external storage device, and stores the data in the external device. Also, data stored in the external device is read.

  The control unit 56 controls the operation of each unit of the medical image recording apparatus 12 described above, and performs reproduction, storage, readout, and the like of the image created from the acquired image data and various data. The medical image recording device 12 is configured as described above.

  Next, operations of the medical system 10 and the medical image recording device 12 will be described. First, a method for recording (storing) acquired data will be described with reference to FIG. FIG. 3 is an explanatory diagram for explaining an example of an image and information recording method. Here, in the diagram shown in FIG. 3, “video information 1” is image data acquired by the endoscope unit 16, and “video information 2” is image data acquired by the operating room imaging unit 20. Further, “biological information” is data acquired by the biological monitor 18, and “operation information” is operation data of the surgical tool 22 acquired based on a signal from the operation detection unit 26.

  As shown in FIG. 3, the data of “video information 1”, “video information 2”, “biological information”, and “operation information” are sent from the above-described units to the medical image recording apparatus 12 and processed by each unit. By doing so, it becomes data. Moreover, by synchronizing with the synchronizer 44, it matches with the same time axis, as shown in FIG. Each data is cut out for each predetermined time section in a state of being associated with each other. Specifically, data in a certain time range is cut into three parts, a cutout part 1, a cutout part 2, and a cutout part 3. When cutting out, each cutout part also cuts out data in the same time zone as the adjacent cutout part. For example, when data starting from 0 seconds is cut out every 20 seconds, the cut out portion 1 cuts out data from 0 seconds to 20 seconds, the cut out portion 2 cuts out data from 18 seconds to 38 seconds, The cutout part 3 cuts out data from 36 seconds to 56 seconds. Thereby, when connecting the cut-out data again, since a part of image becomes the same data, it becomes easy to collate and can connect data correctly.

  The data thus extracted includes data of “video information 1”, “video information 2”, “biological information”, and “operation information” in the same time period. For example, the cutout part 1 includes “video information 1-1”, “video information 2-1”, “biological information-1”, and “operation information-1”, and an aggregate of the data is a data unit. 1 is generated. The data unit 1 may also include synchronous data, that is, time axis flags and feature point data described later. The data unit generated in this way is stored in the storage unit 52 or stored in the external storage device from the transmission / reception unit 54. Further, by reading and reproducing the data unit by the control unit 56, an image can be reproduced on the display 14. In addition, it is possible to reproduce a plurality of data units continuously over a certain time by reproducing them continuously along the time axis.

  Next, an example of an image produced by the medical image recording device 12 and displayed on the display 14 will be described with reference to FIG. FIG. 4 is a screen diagram illustrating an example of an image displayed on the display. As shown in FIG. 4, an image 104, an image 106, a graph 108, a graph 110, and operation keys 112 are displayed on the screen 102 of the display 14. The image displayed on the screen 102 is generated by the control unit 56 and the display control unit 50 based on the data, information, and instructions acquired from the moving image acquisition unit 40, the information acquisition unit 42, and the input detection unit 48.

  The image 104 is an image acquired by the endoscope unit 16 and displays an image (moving image) of the surgical field where the operator is performing processing, that is, the surgical target organ or the like. The image 106 is an image acquired by the operating room imaging unit 20 and displays images (moving images) of operators in the operating room, patients, surgical tools, and the like.

  The graph 108 and the graph 110 are biological data acquired by the biological monitor 18. On the screen 102 shown in FIG. 4, the blood pressure detection result is displayed as a graph 108, and the pulse detection result is displayed as a graph 110. Note that the images and data displayed in the image 104, the image 106, the graph 108, and the graph 110 are data at the same time synchronized by the synchronization unit 44.

  Next, the operation key 112 displays a key for operating the medical image recording apparatus 12. For example, keys for inputting instructions such as playback, stop, fast forward, rewind, recording start, and feature point movement are displayed. The operation key 112 is displayed based on an input from the foot switch 28 so that the selected key can be confirmed. For example, only the selected key is displayed in a different color. In this way, by changing the display of the selected key, the operator can confirm which key is selected by input to the foot switch 28.

  In this way, the medical image recording apparatus 12 creates an image from various synchronized images and data and displays the image on the display 14. Here, the medical image recording apparatus 12 may display data currently acquired by each unit on the screen 102 in addition to the image obtained by reproducing the stored data. That is, a real-time image (image to be recorded) may be displayed. In addition, it is preferable that the screen 102 also displays a graph displaying the time of detection of the feature points with the horizontal axis as the time axis so that the operator can recognize the detection timing of the feature points.

  Next, an example of processing of the medical image recording apparatus 12 will be described with reference to FIGS. 5 and 6-1 to 6-3. Here, FIG. 5 is a flowchart showing an example of processing by the medical image recording apparatus. FIGS. 6A to 6C are screen diagrams for explaining an example of processing by the medical image recording apparatus. The process shown in FIG. 5 is an example of a recording operation.

  First, as a preparation, the operator connects the medical image recording device 12 and other parts of the medical system 10 to obtain a state where image data and various data can be acquired. Further, the medical image recording device 12 and the display 14 are connected so that an image generated by the medical image recording device 12 can be displayed on the display 14. Thereafter, the medical image recording device 12 is activated. Note that the order of preparation is not particularly limited, and each unit may be connected after the medical image recording apparatus 12 is activated.

  When the preparation is completed, the medical image recording apparatus 12 determines whether there is a recording start instruction as Step S12. Here, the recording start instruction is an instruction input when the operator operates the foot switch 28. If it is determined in step S12 that there is no recording start instruction (No), the medical image recording apparatus 12 performs the determination in step S12 again. That is, the medical image recording apparatus 12 repeats the determination in step S12 until a recording start instruction is issued.

  If it is determined in step S12 that there is a recording start instruction (Yes), the medical image recording apparatus 12 starts recording in step S14. That is, the images and information acquired from each unit are synchronized and stored in the storage unit 52. Moreover, it is memorize | stored in an external storage device via the transmission / reception part 54 as needed.

  When the medical image recording apparatus 12 starts recording in step S14, the medical image recording apparatus 12 determines whether or not the surgical instrument 22 has been used in step S16. That is, it is determined whether use of the surgical tool 22 is detected based on a signal from the operation detection unit 26 during recording of images and various data. The use of the surgical tool 22 can be detected by detecting the current flowing between the surgical tool 22 and the output unit 24 by the operation detection unit 26. In other words, in the present embodiment, it is determined that the surgical instrument 22 has been used when a current is passed through the surgical instrument 22 and there is a high possibility that the target portion of the patient has been incised or hemostatic. If it is determined in step S16 that the surgical tool 22 is not used (No), the medical image recording apparatus 12 proceeds to step S22.

  If the medical image recording apparatus 12 determines in step S16 that the surgical tool 22 has been used (Yes), it sets the time immediately before use as a feature point in step S18. That is, the time point immediately before the surgical instrument 22 is used and an incision is made or hemostasis is performed is set as a feature point. Specifically, as illustrated in FIG. 6A, a characteristic point is a time point immediately before starting the incision (removal) of the target portion with the surgical tool 22, that is, a time point when the image 104a is acquired. The setting immediately before use is not particularly limited, and may be set in any way. For example, a predetermined time (0.5 seconds before, 1 second before, 5 seconds before) that is determined to be used may be set as the immediately preceding time. The medical image recording device 12 attaches flag data to the time point set as the feature point so that it becomes the reference (reference for cueing) at the time of reproduction. Note that it suffices if the feature points can be identified, and in addition to attaching flag data, a table (table) of the correspondence between the time corresponding to the synchronized time axis and the feature points may be created.

  After setting the feature points in step S18, the medical image recording apparatus 12 displays an image of the feature points in step S20. In the present embodiment, only images acquired from the endoscope unit 16 are displayed at the feature points. Specifically, as shown in FIG. 6B, the image 104a acquired from the endoscope unit 16 at the feature point is displayed in the region displayed on the operating room imaging unit 20. In the area where the image acquired from the endoscope unit 16 is displayed, the image 104b acquired from the endoscope unit 16 at the present time, that is, the image corresponding to the data to be recorded is displayed. In addition, in the image 104b acquired from the endoscope unit 16 at the present time illustrated in FIG. 6-2, the excision of the target portion started after the image 104a is completed.

  When the medical image recording apparatus 12 displays the image of the feature point in step S20, or when it is determined No in step S16, the medical image recording apparatus 12 determines whether there is a reproduction instruction as step S22. Similar to the recording start instruction, the reproduction instruction is input by the operator operating the foot switch 28, and the medical image recording apparatus 12 determines whether this instruction is input. The reproduction instruction is an instruction to reproduce the recorded image (moving image) after the feature point. If the medical image recording apparatus 12 determines that there is no reproduction instruction (No) in step S22, the process proceeds to step S26.

  If it is determined in step S22 that there is a reproduction instruction (Yes), the medical image recording apparatus 12 reproduces the image using the feature point as a base point in step S24. Specifically, as shown in FIG. 6B, the image of the area where the image 104a acquired from the endoscope unit 16 at the feature point is displayed is reproduced as a moving image. Thereby, the moving image acquired from the endoscope unit 16 after that is used as the base point in the region. By reproducing the moving image in this way, after a predetermined time has elapsed, as shown in FIG. 6-3, the image 104b acquired from the endoscope unit 16 at the current time when the screen shown in FIG. Will be displayed. Further, the image 104c acquired from the endoscope unit 16 at the time of FIG. 6-3 is displayed in a region next to the region (region in which the image on the left side in FIG. 6-3 is displayed). When the medical image recording apparatus 12 reproduces an image in step S24, the medical image recording apparatus 12 reproduces the image until a stop instruction is input by the operator, even if the image for a preset time is reproduced. You may do it.

  If the medical image recording apparatus 12 displays an image in step S24, or if it is determined No in step S22, it determines whether there is a recording end instruction in step S26. Similar to the recording start instruction, the recording end instruction is input by the operator operating the foot switch 28, and the medical image recording apparatus 12 determines whether or not this instruction is input. The recording end instruction is an instruction to end recording of images and data acquired by each unit.

  If the medical image recording apparatus 12 determines in step S26 that there is no recording end instruction (No), it proceeds to step S16 and repeats the above operation. That is, the medical image recording apparatus 12 repeats the processing from step S16 to step S24 until a recording end instruction is input. If it is determined in step S26 that there is a recording end instruction (Yes), the medical image recording apparatus 12 performs a recording end process in step S28 and ends the process.

  As described above, the medical image recording apparatus 12 can detect a feature point and, when a reproduction instruction is input, reproduces an image from the feature point, thereby confirming a past image even during surgery. . Further, by detecting a feature point (use of a surgical tool in this embodiment) and reproducing from the feature point, reproduction can be started without searching for a point (time) at which the operator starts reproduction. . In addition, by detecting the use of surgical tools, which is an important act in surgery, and confirming images (movies) using surgical tools, the operator can grasp problems that occurred during the surgery. . That is, it is possible to easily reproduce the time when the surgical tool is used, which is highly likely to require confirmation with an image, so that the image requested by the operator can be easily reproduced.

  For example, advanced medical technology, especially endoscopic surgery, is difficult to learn, so an unskilled doctor (ie, surgeon or operator) can operate under the supervision of an instructor (a highly skilled doctor). May be performed. In such a case, even if a problem such as bleeding or sudden change in condition occurs, using the apparatus of the present invention, it is possible to easily display an image while the instructor performs an operation immediately before performing an operation using a surgical instrument. Can watch. As a result, the instructor can grasp the immediately preceding state and processing, can solve the problem effectively and in a short time, and can give an instruction to an unskilled doctor. In addition, since the image being processed can be reproduced with a simple operation, guidance can be performed while confirming the image even when guidance is performed. Thereby, even when a problem occurs, it becomes possible to grasp the problem in a short time, and the operation can be performed more safely.

  Also, by displaying both the current image and the image to be played back, the operator and / or the instructor can analyze the problem while comparing the situation before the treatment with the current video, and more The problem can be solved in a short time.

  Further, by displaying the feature point image together with the current image as in step S <b> 20, the operator can grasp the change of the processing target part (affected part). As a result, it is possible to grasp the change in the target site that occurs when the operator performs a treatment with a surgical tool (such as an electric knife or an ultrasonic knife) during the operation, and the contents of the treatment, that is, the size of the incised region. It is possible to easily confirm changes in bleeding. As a result, it is possible to easily perform the operation, and it is also easy to find out when a problem occurs.

  In addition, as shown in FIG. 3, by storing a plurality of data as data units and storing them separately, some images and data can be reproduced even during recording. In addition, storage can be performed in parallel with reproduction. In addition, since the synchronized data on the same time axis can be taken out, the data can be easily read out. The various effects described above can be obtained by recording data units as in this embodiment, but the present invention is not limited to this, and the data recording method is not particularly limited. For example, each data may be stored separately. Alternatively, a plurality of recording devices may be prepared, and when a reproduction instruction is input, an image may be reproduced by one of the plurality of recording devices.

  Here, in the above-described embodiment, since the above-described effects can be obtained, an image of feature points (image acquired by the endoscope unit 16) is displayed on the display 14 in step S20. The feature point image is not necessarily displayed. For example, an image may be reproduced on the display 14 after a reproduction instruction is input.

  In the above embodiment, since the operation status and the data to be recorded can be grasped more accurately, the image and data to be recorded, that is, the image and data received at that time are displayed. The present invention is not limited to this, and an image may be displayed only when a reproduction instruction is given.

  In the above embodiment, since the affected areas can be compared, only the image acquired by the endoscope unit 16 is reproduced, that is, the reproduced image is displayed only on a part of the screen. Is not limited to this, and the reproduced image may be displayed on the entire screen. That is, an image is created based on the acquired image and data, and the image and data (endoscopic image, operating room image, biological data, etc.) acquired using the feature point as a base point are displayed on the entire surface of the display 14, The acquired data may not be displayed.

  In the above embodiment, when a reproduction instruction is given, an image from the immediately preceding feature point is reproduced. However, the present invention is not limited to this, and when there are a plurality of feature points, an image to be reproduced is reproduced. May be selected. By making it possible to select in this way, it is possible to select and reproduce a necessary feature point image, that is, an image processed by the surgical instrument. Moreover, when selecting in this way, it is only necessary to display by a time axis or to display an image of feature points as a thumbnail image. Thus, by selecting from the time axis and the images at each time point, it is possible to select a necessary image in a short time.

  In the above-described embodiment, the image is reproduced using the feature point as a base point. However, the image may be reproduced from a time point that is a preset time before the feature point. By setting in this way, the feature point and the set time point can be confirmed as a moving image. Thereby, even when the use of the surgical tool is detected as the feature point, an image from before the surgical tool is used can be reproduced.

  In the above-described embodiment, the use of the surgical tool can be accurately and easily detected, and a surgical tool that uses electricity is often used for processing such as incision and hemostasis. Although the current flow was detected using, the method of detecting the use of the surgical tool is not limited to this. The use of the surgical tool may be detected by performing image processing on an image captured by the endoscope unit or the operating room photographing unit and detecting the movement of the surgical tool. When image analysis is used in this way, image processing is required, but the use of a surgical instrument that does not use electricity can also be detected. In addition, the surgical tool may be detected from the shape of the surgical tool, but an identification display such as a barcode or a QR (registered trademark, Quick Response) code is attached to the surgical tool, and the identification display is read. Good.

  Moreover, in the said embodiment, although use of the surgical tool was detected as a feature point, this invention is not limited to this. Biometric data (biological information) can also be used as the feature point. Hereinafter, the case where biometric data is used as a feature point will be described with reference to FIG. Here, FIG. 7 is a flowchart showing another example of processing by the medical image recording apparatus. The processing shown in FIG. 7 is the same as the processing shown in FIG. 5, except for the feature point detection processing. Will be omitted, and the points peculiar to the processing of FIG.

  First, preparation similar to the above processing is performed. When the preparation is completed, the medical image recording apparatus 12 determines whether there is a recording start instruction as Step S12. If it is determined in step S12 that there is no recording start instruction (No), the medical image recording apparatus 12 performs the determination in step S12 again. If it is determined in step S12 that there is a recording start instruction (Yes), the medical image recording apparatus 12 starts recording in step S14.

  When the medical image recording apparatus 12 starts recording in step S14, it determines in step S40 whether there is an abnormality in the biometric data. Here, the biometric data is a pulse, a heart rate, a blood pressure, a body temperature, an electroencephalogram, a blood oxygen concentration, and a blood carbon dioxide concentration as described above. Whether or not there is an abnormality in the biometric data is determined by whether or not the numerical value (detection value) of the biometric data is within a predetermined range. The determination of whether or not there is an abnormality may be determined to be abnormal if it is out of the predetermined range even for a moment, but the numerical value of the biological data is greater than or equal to the set time or the set number of times If it is out of the predetermined range, it may be determined that the biometric data is abnormal. If the medical image recording apparatus 12 determines in step S40 that the biometric data is not abnormal (No), the medical image recording apparatus 12 proceeds to step S22.

  If the medical image recording device 12 determines that the biometric data is abnormal (Yes) in step S40, the medical image recording device 12 sets, as step S42, the time point immediately before the determination as abnormal is a feature point. That is, the time point immediately before the biometric data falls outside the predetermined range is set as the feature point. In the present embodiment, the immediately preceding setting is not particularly limited and may be set in any manner. For example, a predetermined time (0.5 seconds before, 1 second before, 5 seconds before) when the biometric data is determined to be abnormal may be set as immediately before. In addition, when it is determined that the biometric data is abnormal when the numerical value of the biometric data is out of the predetermined range for the set time or more than the set number of times, the first time when the biometric data is out of the predetermined range May be used as the feature point immediately before the time when it first deviates from the predetermined range. If the medical image recording device 12 sets the previous time point as the feature point in step 42, the medical image recording device 12 proceeds to step S20. Note that the processing after step S20 is the same as the processing of the flowchart shown in FIG.

  As described above, by detecting the feature points based on the biometric data, it is possible to reproduce an image from the time when the change of the patient occurs. Thereby, when a problem occurs, the cause of the problem can be grasped in a shorter time. In addition, various effects similar to those described above can be obtained.

  Further, bleeding may be detected as a feature point. Hereinafter, a method for detecting a bleeding point by the medical image recording apparatus 12 will be described.

The control unit 56 of the medical image recording device 12 detects a motion region (region with motion) having a predetermined feature to be detected from the moving image information image acquired by the endoscope unit 16, and this motion region The state of the motion region is determined based on the amount of change. Here, the “motion region” is a region where the area changes in a series of moving images or a region where there is motion (that is, movement). Specifically, a surgical instrument for use in surgery, organ, and hemorrhage in the affected area. In the present embodiment, the bleeding area in the affected area is specified from the plurality of movement areas, and the state of the bleeding area, specifically, the change in the area of the bleeding area and the amount of bleeding are grasped. Hereinafter, a method for detecting bleeding will be described in detail.

  First, the control unit 56 of the medical image recording apparatus 12 has two images, that is, an image at a certain point in time and an image before a predetermined time (preset time) as images of moving image information acquired by the endoscope unit 16. Images are acquired, the acquired images are compared, and the degree of coincidence of the images is determined. That is, it is determined to what extent the area (common area) in which the same part of the patient is imaged is determined between an image at a certain time point and an image before a predetermined time. If the degree of coincidence of images is equal to or greater than a certain level (for example, 80% or more of the two images coincide), it is determined that the same part of the patient is being photographed, and motion determination is performed. If the degree of coincidence of images is less than a certain level, it is determined that the shooting scene has been switched, and no motion determination is performed. Note that the degree of coincidence of images is the area that can be detected as the corresponding part by detecting the corresponding part of both images by extracting the contour, color, etc. of the subject by image processing and comparing the whole images It is calculated from the ratio.

  Next, the control unit 56 detects all the motion areas having a predetermined size or more from the moving image information image acquired by the endoscope unit 16 and calculates the amount of change in these motion areas. Here, the “change amount of the motion region” is the change amount or change rate (change speed) of the area of the motion region. The change amount of the area of the motion region is the difference between the area of the region at a certain time t and the area of the region at a time t + Δt after a minute time. The change rate of the area of the motion region is the time difference Δt. Calculated by dividing. In the following description, for convenience, the number of images to be compared for explaining the case where the amount of change in area is calculated from images at two times is not limited. In addition, by increasing the number of images to be compared, for example, using images at three or more times and using a statistical average to determine the amount of change in the motion region, the detection accuracy of the amount of change in the motion region is further improved. Can be made.

  First, each area (size) of the subject shown in the image is detected. For example, for each of two images, an image at a certain point in time and an image before a predetermined time, the image is decomposed into a plurality of pixels, the chromaticity of each pixel is obtained, and a portion composed of pixels having similar chromaticity is obtained. Detect as a region. The region detection method is not limited to this, and the contour of the subject may be detected by image processing, and the region to be drawn may be detected from the contour. As described above, when the subject area of each image is detected, the area of the image at a certain point in time is compared with the area of the image before a predetermined time. As a result of the comparison, a region where the position of the region (relative position in the image) has moved or the size of the region has changed is detected as a motion region.

  Next, the control unit 56 removes, as a noise component, a motion region that is not related to the bleeding point (bleeding region) from the detected plurality of motion regions. Here, examples of the noise component include a movement of a surgical tool used during a surgery, a movement of an organ, and the like. The noise removal processing method is not similar to the preset blood chromaticity, the area of the area does not increase with the passage of time, the area does not continue to increase for a certain period of time, for all detected motion areas. It is determined whether or not the three conditions are met. Then, the motion region corresponding to any one of these three conditions is determined as noise, and is deleted (excluded) from the detected motion region. As a result, when all of the above three conditions are not met, that is, the area of the region increases with the passage of time, and the area continues to increase for a certain period of time, and the blood chromaticity set in advance is Only the motion region that satisfies the condition of similarity is extracted, and the extracted motion region is detected as a “bleeding point”.

  When removing the surgical tool region by noise removal, the shape data of the surgical tool used in the surgery is stored in the storage unit in advance, and the shape of the motion region detected by the image processing is stored in the storage unit. In comparison with the stored shape of the surgical tool, it is preferable to delete the movement region when both match. Thereby, noise can be removed more efficiently. Further, the area area increase rate, the area area increase duration, and the blood chromaticity range can be various ranges and can be set by the operator. The increase rate is large, the duration is long, and the chromaticity range is narrowed, so that the bleeding point detection condition becomes strict and only more serious bleeding can be detected. Further, a finer bleeding point can be detected by increasing the rate of increase, the duration time being short, and widening the chromaticity range. In this case, there is a high possibility that a false detection, that is, a portion that has not bleeding is detected as a bleeding point.

  The control unit 56 also estimates the amount of bleeding from the amount of change in the area of the bleeding region. Specifically, the control unit 56 stores in advance data indicating the relationship between the rate of change in the area of the bleeding region and the flow rate per unit time of blood bleeding from the affected area. Then, the amount of bleeding is estimated by comparing the rate of change in the area of the bleeding region calculated by the above-described procedure with the stored data.

  Moreover, it is preferable that the control part 56 displays a mark (mark) so that the area | region detected as a bleeding point may be enclosed. By displaying the mark in this way, it is possible to recognize the bleeding point. Further, it is preferable that the mark is displayed before bleeding starts when the moving image is reproduced.

  Hereinafter, an example of the detection method and the display method will be specifically described with reference to FIG. Here, FIG. 8 is an explanatory diagram for explaining an example of a method for detecting a bleeding point. In addition, FIG. 8 is displaying the image from the time T1 to the time T4, and has shown the case where it is bleeding from one place. First, when the image at time T1 is compared with the image at time T2, a region having a change in area is detected. Thereafter, at time T3 when time elapses, the area of the bleeding portion becomes larger. Thus, when the area of the bleeding region continues to increase for a certain period of time from time T1 to time T3, the control unit 56 detects the bleeding region as a bleeding point. The control unit 56 detects the bleeding point as described above. At this time, the time point at which bleeding is first confirmed (time T1) is set as the bleeding start time, and the time point (time T3) at which the bleeding is determined is set as the bleeding check time.

  During image reproduction, the bleeding point is surrounded by a dotted circle mark between the time T1 and the time T2 before the bleeding confirmation time, and the bleeding point is displayed between the time T3 and the time T4 after the bleeding confirmation time. Surround with a solid circle mark. Thus, by displaying the mark, the bleeding point can be easily recognized. Further, by switching the mark display method, it is possible to clearly know the bleeding confirmation time.

  Next, the case of using bleeding detection as a feature point will be described with reference to FIG. Here, FIG. 9 is a flowchart showing another example of processing by the medical image recording apparatus. Since the processing shown in FIG. 9 is the same as the processing shown in FIG. 5 except for the feature point detection processing, the same processing is denoted by the same step reference numerals, and detailed description thereof will be given. Will be omitted, and the points peculiar to the processing of FIG. 9 will be mainly described below.

  First, preparation similar to the above processing is performed. When the preparation is completed, the medical image recording apparatus 12 determines whether there is a recording start instruction as Step S12. If it is determined in step S12 that there is no recording start instruction (No), the medical image recording apparatus 12 performs the determination in step S12 again. If it is determined in step S12 that there is a recording start instruction (Yes), the medical image recording apparatus 12 starts recording in step S14.

  When the medical image recording apparatus 12 starts recording in step S14, it determines in step S50 whether there is bleeding. Whether or not there is bleeding is determined based on the analysis result of the acquired image as described with reference to FIG. If it is determined in step S50 that there is no bleeding (No), the medical image recording apparatus 12 proceeds to step S22.

  If it is determined in step S50 that there is bleeding (Yes), the medical image recording apparatus 12 sets a time point before bleeding as a feature point in step S52. In the case of this embodiment as well, the setting of the time point before bleeding is not particularly limited, and any setting may be made. For example, what is necessary is just to set the fixed time (0.5 second before, 1 second before, 5 seconds before) the time of the bleeding determination to be before bleeding. Further, the time point at which the determination of whether or not bleeding is started (that is, the time point when the comparison of the areas is started) may be made before bleeding. If the medical image recording device 12 determines that the time before bleeding is a feature point in step 52, the medical image recording device 12 proceeds to step S20. Note that the processing after step S20 is the same as the processing of the flowchart shown in FIG.

  In this manner, by detecting the feature points based on the occurrence of bleeding (presence / absence of bleeding), it is possible to reproduce an image from the time when a change in the affected area occurs. Thereby, when bleeding (that is, a problem) occurs, the cause of bleeding can be grasped in a shorter time by reproducing the image from before bleeding. In addition, various effects similar to those described above can be obtained. Further, not only the occurrence of bleeding (the time when bleeding starts) but also the time when bleeding reaches a certain amount (a certain area or more) may be detected as a feature point.

  Here, in the above-described embodiment, the feature point is any one of the use of the surgical tool, the abnormality of the biometric data, and the occurrence of bleeding. However, these multiple factors may be detected as the feature points. In other words, if a surgical tool is used or if there is an abnormality in biometric data, it may be detected as a feature point. If a surgical tool is used, if there is an abnormality in biometric data, or if bleeding occurs, it will be detected as a feature point. You may make it do.

  By detecting a plurality of factors as feature points in this way, the medical image recording apparatus 12 can more accurately cue images (moving images) that the operator may desire to reproduce during surgery. It is possible to reproduce a necessary image in a short time by reproducing a feature point selected from a plurality of feature points as a base point. Hereinafter, a description will be given with reference to FIG. Here, FIG. 10 is an explanatory diagram showing the relationship between the feature points detected by the medical image recording apparatus and the recording data. Note that FIG. 10 shows a case where a feature point is detected if there is use of a surgical tool, abnormality of biological data, or occurrence of bleeding.

  When various factors are detected, by detecting them as feature points, a plurality of feature points can be detected during the operation as shown in FIG. FIG. 10 shows record data and feature point data of surgery performed from 14:00 (PM2) to 16:00 (PM4). In the operation shown in FIG. 10, the medical image recording apparatus 12 detects that the instrument (surgical instrument) A is used at 14:10 as a feature point, and uses the instrument (surgical instrument) B at 14:40. This is detected as a feature point. In addition, bleeding at 14:20 is detected as a feature point. Furthermore, the fact that bleeding has been recognized at 14:21, that is, the time when the amount of bleeding has reached a certain level is detected as a feature point. Further, an increase in blood pressure at 15:00 (that is, the blood pressure has exceeded a predetermined range as biometric data) is detected as a feature point. In this way, by detecting a plurality of factors as feature points, an image from the time when each factor occurs during the operation can be reproduced, and the problem can be grasped more easily.

  In the above embodiment, the foot switch 28 is used as an input unit for an operator to input an operation. However, the present invention is not limited to this. As the input unit, a keyboard, a mouse, a touch panel, or the like can be used. By using a touch panel as an input unit, an operator can easily perform an operation. The touch panel is arranged on the surface of the display and detects an input instruction based on the relationship between the screen displayed on the display and the contact position by the operator.

  In the above embodiment, the use of a surgical tool, abnormal biological data, and the occurrence of bleeding are detected as characteristic points. However, the present invention is not limited to this, and features analysis results that may cause problems in patients. What is necessary is a dot.

  In the above embodiment, the medical image recording apparatus can reduce the possibility of affecting other devices in the medical system, and can smoothly use other devices, so that it is electrically insulated from other devices. However, the present invention is not limited to this. Although the above effect cannot be obtained, it may be connected in an electrically connected state, and for example, information may be exchanged by transmitting and receiving electrical signals.

  In the above embodiment, since the value and effectiveness as information can be further increased, an endoscope image, an operating room image, biological information, and information on the use of a surgical tool are used. The number of images to be acquired and the type of information are not particularly limited as long as at least one moving image and information capable of detecting feature points can be obtained. In addition, when a feature point can be detected from a moving image, the medical image recording apparatus may be configured to acquire only one moving image.

  As described above, the medical image recording apparatus according to the present invention is useful for recording and reproducing medical moving images, and is particularly suitable for reproducing surgical images.

DESCRIPTION OF SYMBOLS 10 Medical system 12 Medical image recording device 14 Display 16 Endoscope unit 18 Living body monitor 20 Operating room imaging | photography part 22 Operating tool 24 Output part 26 Operation detection part 28 Foot switch 40 Movie acquisition part 42 Information acquisition part 44 Synchronization part 46 Feature inspection Output unit 48 Input detection unit 50 Display control unit 52 Storage unit 54 Transmission / reception unit

Claims (10)

A video acquisition unit for acquiring medical videos;
An information acquisition unit for acquiring medical information;
A feature point detection unit that detects a feature point from information detected by at least one of the moving image acquisition unit and the information acquisition unit;
A synchronization unit for synchronizing the acquired video and information;
An input detection unit for detecting an instruction to reproduce the moving image input by an operator;
When detecting that the reproduction instruction is input by the input detection unit, a control unit that reproduces the moving image acquired by the moving image acquisition unit with reference to the time point when the feature point is detected by the feature point detection unit;
A medical image recording apparatus comprising: The information acquisition unit includes a detection function for detecting operation of a surgical tool,
The medical image recording apparatus according to claim 1, wherein the feature point detection unit detects the operation as a feature point when detecting that the operation tool is operated by the information acquisition unit. The information acquisition unit includes a detection function for detecting biological information of a patient,
The medical image according to claim 1, wherein the feature point detection unit detects a feature point when the biological information detected by the information acquisition unit is a value that is out of a preset range. Recording device.   The medical image recording apparatus according to any one of claims 1 to 3, wherein the moving image acquisition unit acquires an image acquired by an endoscope.   5. The medical image recording apparatus according to claim 1, wherein the control unit reproduces a moving image from a point of time that is a preset time before the feature point. Furthermore, it has a display control unit that generates an image to be displayed,
6. The display control unit according to claim 1, wherein the display control unit displays an image including an image before starting the reproduction and an image acquired by the moving image acquisition unit before detecting the reproduction instruction. The medical image recording apparatus according to any one of the above.   The medical image recording apparatus according to claim 6, wherein the input detection unit is a touch panel that detects contact with a display unit that displays an image generated by the display control unit.   The medical image recording apparatus according to claim 1, wherein the input detection unit is a foot switch.   The medical image recording according to any one of claims 1 to 8, wherein the moving image acquisition unit is connected in an electrically non-contact state with an image capturing device that captures a medical moving image. apparatus.   The medical image recording according to any one of claims 1 to 9, wherein the information acquisition unit is connected in an electrically non-contact state with an information generation device that generates medical information. apparatus.

Images