JP2012045419A - Medical image display device, medical image display system, medical image display method, and endoscope apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make an intelligible examination explanation without forcing extra burdens on both a physician and a patient.SOLUTION: A schema is displayed on a display screen 56 of a medical image display device in addition to a moving image and a still image recorded in an endoscope apparatus. The moving image and the still image are given recording position information of the still image in the moving image when photographed by the endoscope apparatus, and based on this information, the corresponding still image is accessed when reaching the recording position during the reproduction of the moving image. Further, identification information indicating a photographed part of the still image is recorded in a photographing time of the endoscope apparatus, and the medical image display device gives link information so that a specific position in the schema is associated with the still image based on the identification information. Consequently, when the specific position of the schema is pointed, the still image corresponding to the point is accessed.

Description

  The present invention relates to a medical image display device that displays a medical image obtained by imaging an in vivo with an endoscope, a medical image display system, a medical image display method, and an endoscope device used in the medical image display system. It is.

  In order to inspect the living body, endoscopy using an endoscopic device is performed. The endoscope apparatus includes an endoscope and a processor device. An endoscope is inserted into a living body (for example, a digestive tract, etc.) and includes a tube-shaped insertion portion that can be bent freely. An imaging element such as a CCD and an imaging portion are illuminated at the distal end of the insertion portion. A photographing unit having an illumination window for irradiating light is arranged. The endoscope sequentially transmits an image signal output from the image sensor at a predetermined frame rate to a processor device connected to the endoscope. The processor device sequentially performs image processing on the received imaging signal to generate an endoscopic image (medical image), and displays this on the monitor in real time. The doctor performs the examination while observing the inside of the living body through the image projected on the monitor.

  The endoscope also has a function of recording a photographed endoscopic image as high-definition still image data. A release switch is provided in the operation unit of the endoscope, and a still image of a required part is recorded by pressing the release switch as necessary. The endoscopic image recorded in this way is printed out by a printer or displayed on a display of a personal computer, so that it can be used for diagnosis after a doctor's examination, explanation of examination results for a patient, etc. (for example, , See Patent Document 1).

JP 2001-120498 A

  In recent years, the need for informed consent (explanation and consent of medical practice) has been socially recognized, and in the medical field, it is required to provide sufficient explanation regarding medical practice to patients. In the endoscopic examination, as described above, the examination results and the like are explained while showing the still image obtained by photographing the required part after the examination. There was a problem that it was difficult to grasp the location, etc., and it was difficult to understand the contents of the explanation accurately.

  Of course, in endoscopy, the image under examination is displayed on the monitor of the processor device in real time, so the doctor will explain the image to the patient and explain it appropriately. Supplements are also being made. However, during the examination, the patient is in a state where an endoscope is inserted into the body, and it is burdensome for the patient to watch the monitor image and give detailed explanations to the patient in such a state. In addition to being large, there are cases where there is no room to receive the explanation.

  Therefore, after the examination, based on the schematic diagram (schematic diagram) that schematically shows the internal structure of the living body and the captured still image, an explanatory document is created so that the relationship between them can be understood, and the explanation is used. Has also been done. However, it took a considerable amount of time to create easy-to-understand materials, so the burden on doctors was great, and such explanations tended to be avoided.

  The present invention has been made in view of such a background, and it is an object of the present invention to provide an easy-to-understand explanation of an examination without imposing an extra burden on both a doctor and a patient.

  In order to achieve the above object, a medical image display apparatus according to the present invention includes an insertion portion that is inserted into a living body and an operation portion that is connected to a proximal end portion of the insertion portion. In a medical image display device that displays a medical image recorded as image data on a display using a endoscope device, the medical image includes a still image, and the still image and the still image are captured. A schema diagram link information adding unit for adding schema diagram link information for associating each image data with a schema diagram schematically representing the internal structure of the living body including the imaging region; and the schema diagram and the still image A control unit that searches for the other on the basis of the schema diagram link information and displays the other on the display while one is displayed on the display, and identifies the imaging part of the still image Identification information input by recording with a microphone or operating the endoscope apparatus at the time of shooting the still image is attached to the image data of the still image, and the schema diagram link information The assigning unit associates the still image with a specific position in the schema based on the identification information.

  In the image data of the schema diagram, collation information for collating with the identification information is preliminarily attached for each specific position in the schema diagram, and the schema diagram link information adding unit includes the identification information and the collation It is preferable to associate the still image with the specific position by collating information.

  The identification information is, for example, voice data recorded by the microphone, and is collated with the collation information recorded as voice data corresponding to the identification information.

  The identification information is input by, for example, operating the operation unit or operating a console of a processor device included in the endoscope apparatus.

  The identification information is, for example, text data, and is collated with the collation information recorded as text data corresponding to the identification information.

  The control unit may search and display the still image corresponding to the specific position based on the schema diagram link information when the specific position in the schematic diagram displayed on the display is pointed. preferable.

  The control unit preferably displays the still image and the schema diagram side by side on the display.

  The medical image includes a moving image in addition to the still image, and the control unit is based on medical image link information that is recorded by the endoscope device and associates image data of the moving image and the still image. Then, while displaying one of the moving image and the still image, the other may be searched and displayed.

  The medical image link information includes recording position information indicating the recording position of the still image in the entire moving image, and the control unit determines whether the moving image is being reproduced based on the recording position information. The still image corresponding to the recording position may be displayed.

  It is preferable that the control unit displays the moving image side by side on the display in addition to the still image and the schema diagram.

  The medical image display system according to the present invention includes an endoscope device that has an insertion portion that is inserted into a living body and an operation portion that is connected to a proximal end portion of the insertion portion, and images the inside of the living body, and the endoscope In a medical image display system having a medical image display device that displays a medical image recorded as image data using a mirror device on a display, the medical image includes a still image, and the still image, A schema diagram link information providing unit for adding schema diagram link information for associating each image data with a schematic diagram schematically representing an internal structure of a living body including a photographing part photographed in a still image, and the schema diagram And a control unit that searches for one of the still images on the display and displays the other on the display based on the schema diagram link information. Identification information for identifying the imaging part of the image, and identification information input by recording with a microphone or operating the endoscope apparatus at the time of capturing the still image is attached to the image data of the still image The schema diagram link information assigning unit associates the still image with a specific position in the schema diagram based on the identification information.

  The endoscope apparatus includes a release button for inputting a still image shooting instruction and an identification information input unit for inputting the identification information. The microphone, the operation unit, and a processor apparatus included in the endoscope apparatus It is preferable that an identification information input unit including any one of the consoles and an identification information recording unit for recording the identification information attached to the image data of the still image.

  The medical image includes a moving image in addition to the still image, and the control unit is based on medical image link information that is recorded by the endoscope device and associates image data of the moving image and the still image. It is preferable that one of the moving image and the still image is searched and displayed while the other is displayed.

  The medical image link information includes recording position information indicating the recording position of the still image in the entire moving image, and the control unit determines whether the moving image is being reproduced based on the recording position information. It is preferable to display the still image corresponding to the recording position.

  The recording position information is preferably a time stamp recorded in moving image data during the recording of the moving image. It is preferable that the endoscope has a medical image link information recording unit that records the medical image link information. The still image is preferably recorded with a higher number of pixels than the number of pixels in one frame of the moving image.

  The medical image display method of the present invention records a medical image taken as an image data by an endoscope apparatus having an insertion portion to be inserted into a living body and an operation portion connected to a proximal end portion of the insertion portion. In the medical image display method for displaying the medical image on the display of the medical image display device, the medical image includes a still image, and recording of the still image is performed by a microphone or the endoscope device. An identification information input step for inputting identification information for identifying a photographing part of the still image by an operation, an identification information recording step for recording the identification information attached to the image data of the still image, and the still image And a data fetching step of fetching the identification information incidental thereto and the medical image display device, and a schema diagram schematically showing the internal structure of the living body. Based on the identification information, schema diagram link information for associating each image data of the schema diagram and the still image is assigned based on the identification information so that the specific position corresponding to the imaging part of the still image corresponds to the still image. A schema diagram link information adding step, and a display step of searching for one of the still image and the schema diagram on the display and displaying the other on the display based on the schema diagram link information. It is preferable.

  According to the present invention, when displaying a medical image photographed by an endoscope apparatus on a display, a still image photographed by the endoscope apparatus and a schema diagram schematically showing the internal structure of a living body. Since the schema diagram link information to be associated is given, and based on this information, one of the still image and the schema diagram is displayed on the display, and the other is searched and displayed on the display. Therefore, the examination results of the endoscope can be explained in an easy-to-understand manner without imposing an extra burden on both.

It is explanatory drawing of a medical image display system. It is a block diagram which shows the outline of an endoscope apparatus and a medical image display apparatus. It is explanatory drawing explaining the link condition of each file of a moving image, a still image, and a schema. It is explanatory drawing which shows the file structure of each file of a moving image and a still image. It is explanatory drawing which shows the file structure of a schema diagram file. It is explanatory drawing of the correlation process of a still image and a schema diagram. It is explanatory drawing of an operation screen. It is a flowchart which shows an endoscopic imaging | photography procedure. It is a flowchart which shows the procedure of the correlation process of an endoscopic image and a schema diagram. It is a flowchart which shows the display procedure of a moving image, a still image, and a schema. It is a flowchart which shows the display procedure of a schema and a still image.

  FIG. 1 shows an endoscopic device 10 that records an in-vivo image obtained by photographing an inside of a living body and records the image (medical image) as image data, and an explanation and image of the examination to a patient after the examination is completed. 1 shows a medical image display system including a medical image display device 11 that displays an endoscopic image recorded by the endoscope device 10 for making a diagnosis.

  The endoscope apparatus 10 includes an endoscope 13 including an imaging unit, a processor device 14 that generates an endoscopic image based on a signal output from the imaging unit, a console 16, and a headset 17. The console 16 includes an operation unit 18 including a keyboard and a mouse for inputting an operation signal for operating the processor device 14, and a monitor 19 for displaying an operation screen and displaying a captured endoscopic image in real time. The headset 17 is provided with a microphone 17 a for inputting an audio signal to the processor device 14.

  The endoscope 13 includes an insertion portion 21 that is inserted into a living body and an operation portion 22 that is connected to a proximal end portion of the insertion portion 21, and is connected to the processor device 14 by a cord 23. The distal end 21a of the insertion portion 21 is provided with a photographing unit, a forceps opening for inserting and removing forceps, a nozzle for supplying water and supplying air for cleaning the lens, and the like. The imaging unit includes a CCD 25 (see FIG. 2), an objective lens that collects the imaging light on the CCD 25, and an illumination window that irradiates the imaging site to be imaged with illumination light.

  Inside the insertion portion 21, an angle wire for changing the shooting angle of the CCD 25, a light guide made of an optical fiber for guiding the illumination light from the light source to the illumination window, an imaging signal output from the CCD 25, and an operation signal such as a shooting instruction A signal cable, a water supply pipe, an air supply pipe, and the like are inserted.

  The operation unit 22 includes an angle knob that winds and pulls the angle wire, a water supply switch that supplies water, an air supply switch that supplies air, a monitor display changeover switch, and the like. When the angle knob is pushed and pulled by operating the angle knob, the direction of the tip 21a is changed and the photographing angle is directed to a desired direction.

  The monitor display changeover switch displays a plurality of frames to be acquired that are output by the CCD 25 at a predetermined frame rate in real time on the monitor 19 in real time, and stops (freezes) the frame update to monitor one frame at the time of the stop. 19 is switched to the freeze display to be displayed on the screen.

  As shown in FIG. 2, the operation unit 22 has a release button 26 for inputting a still image shooting instruction for recording a still image as image data, and a moving image recording switch for inputting instructions for starting and ending moving image recording. 27 are provided. When the release button 26 is operated, the video imaged by the endoscope 13 is recorded as still image data. When recording of a moving image is started, for example, an imaging signal output from the CCD 25 for monitor display is recorded as moving image data. In still image shooting, the number of recorded pixels of the CCD 25 is higher than one frame of a moving image, and a high-definition image is recorded as compared with a moving image. Still image shooting is possible even during video recording. In that case, the acquisition of the frame of the moving image is temporarily stopped while still image shooting is performed, and is resumed when the still image shooting is completed.

  When performing an endoscopic inspection, an image of the entire inspection object range is observed on the monitor 19 and the inspection is performed, and a required part that is a particularly important part as an observation object is often photographed. For example, if the required site is the upper gastrointestinal tract (esophagus, stomach, duodenum), the cardia which is the entrance to the stomach from the esophagus, the gastric horn which is the bent portion of the upper stomach wall, the pylorus which is the exit from the stomach to the duodenum, This includes the duodenal bulb at the entrance of the duodenum, and other parts suspected of being lesioned by findings before the examination. Such a required part is recorded as a high-definition still image, and image diagnosis is performed after the inspection based on the image. In the case of the upper digestive tract, after inserting the insertion portion 21 from the patient's mouth and inserting the distal end 21a up to the duodenum, a still image of about 20 to 40 cuts is taken in the process of pulling out the insertion portion 21. .

  The microphone 17a is an identification information input unit that inputs sound serving as identification information for identifying a shooting part of a still image during such still image shooting. For example, when the release button 26 is pressed, the doctor utters the name of the imaging region (such as “Fumon”, “Ikaku”, “Yumon”, etc.) Recording is performed with the microphone 17a. The recorded audio data is stored in association with the image data of the corresponding still image, and is used for associating the still image with the schema diagram prepared in advance by the medical image display device 11 as will be described later. The The audio signal from the microphone 17a is sent to the processor device 14 by radio, for example. Of course, you may communicate by wire.

  The processor unit 14 generates an endoscopic image by performing image processing on a CPU 31 that controls the entire apparatus, a working memory 32 that is used when the CPU 31 executes a program, and an imaging signal that is input from the CCD 25. An endoscopic image generation unit 33, an audio signal processing unit 34 that converts an analog audio signal from the microphone 17a into digital audio data, a data storage 36 that stores image data and audio data of moving images and still images, a LAN, and the like Communication I / F 37 for performing data communication with other devices through the communication network.

  The CPU 31 converts image data and audio data into a predetermined file format, and stores them in the data storage 36 as a still image file, a moving image file, and an audio file. As described above, the CPU 31 associates audio data with a still image, and attaches one audio file to one still image file. In addition, when still image shooting is performed during moving image recording, the CPU 31 sets endoscope image link information (medical image link information) to each image file in order to associate each image file of the moving image and the still image. Give.

  As will be described later, this endoscopic image link information is information used in the medical image display device 11, and includes a file name for associating each image file of a moving image and a still image with each other in the entire movie. Recording position information indicating when the still image was taken is included. When recording a moving image, a time stamp is generally given as information indicating the playback position of each frame in addition to the frame data. As the recording position information, for example, such a time stamp is used.

  The endoscopic image and audio data in the data storage 36 are classified and stored based on patient information and examination information registered in advance before examination. Patient information includes patient ID, name, date of birth, age, sex, etc., and examination information includes examination date, examination site, and the like. Endoscopic images and audio data are classified and stored for each patient, for example. Further, the data for each patient is stored in a state of being classified and arranged as appropriate according to the examination date, examination type, clinical department, and the like. Such patient information and examination information are registered in the processor device 14 by manual input through the console 16 before the examination, for example. When patient information or the like has already been registered in other medical examination equipment or accounting database in the hospital, such registered information may be taken into the processor device 14 via a communication network.

  For example, the medical image display apparatus 11 has a medical image display program installed on the basis of a personal computer. The medical image display device 11 includes a device main body 38 and a console 39 connected to the device main body 38. The console 39 includes an operation unit 41 (see FIG. 1) including a keyboard and a mouse for inputting an operation signal for operating the apparatus main body 38, an operation screen, and a display for displaying a medical image recorded by the endoscope apparatus 10. 42 (see FIG. 1). The apparatus main body 38 performs data communication through a CPU 46 that controls the entire apparatus, a working memory 47 that is used when the CPU 46 executes a program, a data storage 48 that stores medical image display programs and various data, and a communication network. Communication I / F49 for performing is provided. When the medical image display program is activated, the CPU 46 executes various processes described in the program.

  The medical image display apparatus 11 takes in the image file and the audio file recorded by the endoscope apparatus 10 through the communication I / F 49 and stores them in the data storage 48. The data storage 48 stores a schema diagram file and an audio sample file in addition to the data fetched from the endoscope apparatus 10.

  As shown in FIG. 3, the schema diagram file is a schema diagram schematically representing the internal structure of a living body converted into digital data. This schema diagram file is read from the data storage 48 by the CPU 46, and the schema diagram is reproduced and displayed on the display 42. A plurality of schema diagram files are stored in accordance with examination sites such as the upper digestive tract and the lower digestive tract.

  The schema diagram SCM1, for example, mainly shows the entire structure of the upper digestive tract. When the examination of the upper gastrointestinal tract is performed, an easy-to-understand explanation can be given by referring to the recorded endoscopic image together with such a schema diagram SCM1 when explaining to the patient. Therefore, the CPU 46 gives schema diagram link information for associating the endoscopic image with the schema diagram file, and when one is displayed on the display 42 based on the schema diagram link information, the other is simply Search and display. As the schema diagram link information, link information between the moving image file and the schema diagram file and link information between the still image file and the schema diagram file are respectively given.

  In addition, on the schema diagram SCM1, the above-described required parts (for example, cardia, stomach corner, pylorus) are shown. Still images PIC1 to PIC3 obtained by photographing those required parts in the inspection are associated with specific positions P1 to P3 corresponding to the required parts on the schema diagram SCM1. Thereby, for example, when the specific position P1 on the schema diagram SCM1 displayed on the display 42 is pointed with the mouse, the still image file PIC1 can be retrieved and displayed. In this example, a three-cut still image has been described as an example. However, this is only an example, and the number of still images is not limited to this.

  Further, as described above, the still image file and the moving image file are associated by the endoscope image link information provided by the endoscope apparatus 10. Since this endoscopic image link information includes recording position information, the medical image display apparatus 11 reproduces moving image frame data and reaches reproduction positions T1 to T3 where still images are recorded, for example. The still images PIC1 to PIC3 corresponding to the positions T1 to T3 are retrieved and displayed.

  FIG. 4 is a configuration diagram of each file of a moving image, a still image, and a sound acquired by the endoscope apparatus 10. As shown in FIG. 4A, the moving image file includes captured frame data and incidental information, and the incidental information includes patient information, examination information, and link information. The link information includes endoscope image link information (link information with a still image) and schema diagram link information. The link information with the still image is obtained by associating the still image file name (PIC1 to PIC3) with the time stamps T1 to T3 indicating the recording positions. The schema diagram link information is the file name of the schema diagram SCM1. The schema diagram link information is given by the medical image display device 11. Therefore, at the time of recording with the endoscope apparatus 10, the area for storing the information is blank.

  As shown in FIG. 4B, the still image file is composed of image data and accompanying information, and the accompanying information includes patient information, examination information, and link information. The link information includes endoscope image link information (link information with a moving image) and schema diagram link information. Link information with a moving image is, for example, a moving image file name (MOV1) and a time stamp T1 indicating a recording position in the moving image. The still image schema diagram link information is obtained by associating the file name of the schema diagram SCM1 with the specific position information regarding the specific position. The specific position information includes, for example, coordinate information on the schema diagram SCM1. The area for storing the schema diagram link information is blank when it is recorded by the endoscope apparatus 10 as in the case of the moving image file.

As shown in FIG. 4C, the audio file includes audio data and a corresponding still image file name. In the case of the audio file VO1 corresponding to the still image file (PIC1) obtained by photographing the cardia, the audio data “Fumon” recorded at the time of photographing and the file name “PIC1” of the still image file are included. With this file name, the audio file is associated with the still image.

  FIG. 5 is a configuration diagram of a schema diagram file. The schema diagram file includes image data of the schema diagram SCM1 and accompanying information. The accompanying information includes the file names of the moving image file and the still image file as schema diagram link information. The link information with the still image file is associated with the specific position information regarding the specific positions P1 to P3 set in advance. Since the link information is generated by performing an association process by the medical image display device 11, the link information is blank until the process is performed.

  The schema diagram file is accompanied by audio sample files SVO1 to SVO3 corresponding to the specific positions P1 to P3. The audio sample file is collation information that is collated with an audio file attached to the still image in order to associate the still image with a specific position. The audio sample files SVO1 to SVO3 are obtained by recording the part names of the specific positions P1 to P3 as audio data. The audio sample file includes this audio data, a corresponding schema diagram file name (SCM1), and specific position information.

  As shown in FIG. 6, the CPU 46 of the medical image display device 11 first performs, for example, an association process between a still image file PIC1 and a schema diagram file, an audio file VO1 attached to the still image file PIC1, A plurality of audio sample files SVO1 to SVO3 attached to the schema diagram file are collated, and an audio sample file having audio data that matches the audio data of the audio file VO1 is checked. If a matching audio sample file SVO1 is found, the schema name file name SCM1 and specific position information P1 corresponding to the audio sample file SVO1 are stored in the still image file PIC1. On the other hand, the file name PIC1 of the still image file is stored in the schema diagram file in association with the specific position P1.

  In this example, as the link information of each file, the file name of the associated file is given, but when the link source file and the link destination file are stored in different folders or different devices, In addition to the file name, a path name indicating the storage location and address information (URL, etc.) are given.

  FIG. 7 shows an example of a medical image display screen (hereinafter referred to as a display screen) 56 displayed on the display 42. The display screen 56 is provided with a display window 57 for displaying moving images, a display window 58 for displaying still image files, a display window 59 for displaying schema diagrams, and a display window 61 for displaying examination information and patient information. ing. Each of these windows can be appropriately changed in display size. The display size is changed by operating the mouse pointer 62, for example. Further, as shown in the figure, these windows are displayed in parallel in the display screen 56, and one window can be enlarged and displayed over the entire area of the display screen 56. Further, a file button 63 is provided on the display screen 56. When the file button 63 is clicked with a mouse, a file selection screen is displayed, and a file to be displayed can be selected on the selection screen.

  The display window 57 is provided with buttons for instructing playback, pause, and end of playback of moving images. Further, a more detailed operation menu is displayed in a pop-up format by a mouse operation (for example, a right click operation), and it is also possible to fast forward, fast reverse, change the playback speed, and the like. Since the playback time of a moving image is long, it is played back fast by default.

  In the display window 59, marks such as circles are displayed at specific positions P1 to P3 on the schema diagram associated with the still image, and it is possible to identify that the still image is associated with the position. . When the mark is pointed with the pointer 62, a still image corresponding to the point position is called. Further, the display screen 56 may show a line or an arrow indicating the correspondence between the called still image and the specific position.

  Hereinafter, the operation of the above configuration will be described with reference to the flowcharts of FIGS. FIG. 8 shows an endoscopic image photographing procedure at the time of examination. First, patient information and examination information are registered in the processor device 14 before the examination. Then, the moving image recording switch 27 is pressed to start moving image recording. The insertion portion 21 is inserted into the patient's body, and the examination is performed while observing the inside of the digestive tract with the monitor 19. During this observation, a still image is appropriately captured for the required part. At the time of this imaging, the doctor utters the name of the imaging region and records the sound with the microphone 17a. As described above, the identification information of the imaging part can be input by voice, so there is no extra operation such as button operation, and it is easy. In addition, the act of issuing the name of the imaging region at the time of imaging is also intuitive and less uncomfortable than the button operation.

  The CPU 31 of the processor unit 14 stores the recorded audio data in association with the image data of the still image. Further, the CPU 31 adds patient information, examination information, and link information (endoscopic image link information) with a moving image to the incidental information of the still image file, and stores the still image file in the data storage 36. These procedures are repeated until the inspection is completed. By taking such a still image, it is possible to record a higher-definition image than a moving image frame at a required portion. When the inspection is finished, the moving image recording is finished. Link information (endoscopic link information) with a still image is added to the moving image data, and the moving image file is stored in the data storage 36.

  After completion of the examination, the endoscopic image and the audio file recorded by the endoscope apparatus 10 are taken into the medical image display apparatus 11 through the communication I / F 49 and stored in the data storage 48. The CPU 46 of the medical image display apparatus 11 starts the association process between the endoscopic image and the schema diagram based on the operation instruction for the association process. When there are a plurality of types of schema diagrams, the CPU 46 examines the examination site from the examination information in the moving image or still image file, and extracts the schema diagram corresponding to the examination site. Thereby, a schema diagram for performing the association process is selected. Of course, the doctor may select a desired schema diagram among a plurality of types.

  Then, schema diagram link information (file name) is assigned to each file of the endoscopic image and the schema diagram. In the case of a still image, as shown in FIG. 6, the audio file attached to the still image and the audio sample file attached to the schema diagram are collated and associated with a specific position on the schema diagram. The The endoscopic image thus subjected to the association process is stored in the data storage 48 again. When the association process with the schema diagram is performed, as shown in FIG. 3, the movie, the still image, and the schema diagram are associated with each other by the endoscope image link information and the schema diagram link information. become.

  After the association process is completed in this way, the doctor uses the medical image display device 11 to explain the examination while showing the patient the display screen 56 shown in FIG. When explaining, as shown in the flowchart of FIG. 10, the doctor first selects a moving image file to be reproduced, for example. When the moving image file is selected, the CPU 46 opens the display window 57 and displays the first frame of the moving image file. Further, the linked schema diagram SCM1 is searched based on the schema diagram link information, and the schema diagram SCM1 is displayed on the display window 59. The doctor explains the outline of the examination while referring to these moving images and the schema.

  Further, when the playback of the moving image is started, the CPU 46 monitors whether there is a still image corresponding to the frame being played back based on the endoscope link information during the playback of the moving image, and the still image exists. When reaching the frame to be displayed, the display window 58 is opened to display a still image. When a still image is displayed, for example, the moving image is automatically paused. Since the still image has better image quality than the frame of the moving image, detailed explanation is added while showing the still image to the patient. Such a procedure is repeated until the end of playback of the moving image.

  In addition, since the recording position in the entire movie is stored in the still image file, when an arbitrary still image is displayed, based on the recording position information, the vicinity of the recording position of the still image ( For example, it is of course possible to display a frame of a moving image of the portion before and after the still image. In this way, by linking a moving image and a still image, the search from one to the other can be facilitated, and seamless reproduction and display can be performed between the two.

  As an advantage of recording a moving image in addition to a still image, there is an advantage that after the examination, the state of the body under examination can be shown to the patient, and the doctor can confirm the examination status again. For example, in order to perform a biopsy (biopsy tissue diagnosis), a forceps is taken out from the distal end of the endoscope during the examination, and a part of the tissue at the lesion site may be removed. At the time of this extraction, a medicine containing a pigment is ejected from the forceps opening to make the extraction site undulate. By recording a moving image, it is possible to reconfirm the spread of the sprayed medicine, the process of spreading, etc., and it is also possible to check whether the extraction procedure has been performed properly. In addition, because of these advantages, many doctors insist that endoscopic diagnosis should be done with videos.

  As shown in the flowchart of FIG. 11, when the mouse is pointed at a specific position on the screen of the displayed schema diagram, the CPU 46 searches for a still image corresponding to the position based on the schema diagram link information. Are displayed on the display window 58. In this way, the state of the required part can be explained in detail while showing the partially examined still image while showing the examination part as a whole by the schematic diagram and clarifying the correspondence between them. it can.

  In this way, it is possible to explain the examination while referring to the schema diagram and the endoscopic image, so that it is easy to explain for the doctor and easy for the patient to understand. Become. In addition, since this easy-to-understand explanation can be received after the examination, the patient is freed from the burden of receiving a detailed explanation during the examination. In addition, since the endoscope apparatus 10 associates a moving image with a still image and the medical image display apparatus 11 associates a schema diagram with an endoscope image, a complicated procedure of creating explanatory materials is not necessary, and a doctor There is little burden for.

  In the above-described embodiment, endoscopic image link information and schema diagram link information are given to each file of a moving image, a still image, and a schema diagram, respectively. You may give as another file independent of each file. In this case, it is preferable to create a database by converting link information between files into table data.

  In the above embodiment, link information is given so that still images and moving images, and these endoscopic images and schema diagrams can be called (searched and displayed) with each other. For example, in the relationship between a still image and a schema diagram, it is possible to call only one direction so that a schema diagram file can only be called from a still image and a schema diagram cannot be called from a still image. You may give link information to.

  Also, when a specific position on the schema diagram is pointed, the corresponding still image is called. For example, when a schema diagram is displayed, it is associated with the schema diagram based on the schema diagram link information. Alternatively, a plurality of still images may be searched and displayed simultaneously around the display window of the schema diagram.

  Further, the still image or the schema diagram may be called by voice using the audio data attached to the still image or the schema diagram. For example, when “cardia” is uttered, the medical image display apparatus recognizes this sound and displays a still image obtained by photographing the cardia based on the input sound. According to this, the trouble of operating the mouse and the keyboard during the explanation to the patient can be saved.

  In the above embodiment, an example in which an endoscopic image is associated with one schema diagram has been described. However, for example, a plurality of schema diagrams are changed such that the associated schema diagram is changed for each imaging region of a still image. May be associated.

  Moreover, although the said embodiment demonstrated by the example using audio | voice data as identification information for matching a schema diagram and the specific position of a still image, it may not be audio | voice data. For example, a button for inputting identification information (for example, text data) corresponding to the type of the imaging region is provided on the operation unit of the endoscope or the console of the processor device, and the identification information corresponding to the imaging region by this button operation. May be input. Also, as used in capsule endoscopes, etc., a positioning device is provided for positioning the shooting position by transmitting a radio wave for positioning from the imaging unit of the endoscope and receiving it, and the positioning information is You may use for identification information.

  In addition, in the case where the number of still image shooting parts and the shooting order are patterned, text data such as a number indicating the shooting order may be used as the identification information. In this case, for example, when the release button of the endoscope apparatus is pressed, the endoscope apparatus automatically attaches a number indicating the photographing order to the still image. Since the identification information is collated with the collation information attached to the schema diagram, when text data is used as the identification information, the text data corresponding to the collation information of the schema diagram is also used.

  In the above-described embodiment, a moving image is recorded in addition to a still image by the endoscope device, and the moving image is also reproduced by the medical image display device. Only a figure is sufficient. However, as described above, since there is utility in recording a moving image, it is preferable to record and reproduce the moving image. Moreover, when recording a moving image, you may provide the image pick-up element which image | photographs a moving image and a still image separately.

  Moreover, although the said embodiment demonstrated in the example which used the planar figure as a schema figure, of course, you may use a three-dimensional figure. In addition to the drawn image, a photograph taken may be used as the schema diagram. In addition, when using a three-dimensional figure, for example, from a tomographic image acquired by an imaging photographing modality (medical device) such as an MRI (magnetic resonance imaging) apparatus or a CT (X-ray tomographic imaging) apparatus, You may use what comprised the internal structure in three dimensions. If such a three-dimensional diagram is used as a schema diagram, it is possible to show the internal structure not only from the front but also from the back by rotating the diagram three-dimensionally.

  In the above embodiment, each data of the endoscopic image and the schema diagram is stored in the data storage in the medical image display device, but the data storage location is not limited to this. For example, data may be stored in a data storage server that is connected to a medical image display device through a communication network and that exists inside or outside the hospital, and the medical image display device accesses the data storage server to read the data. .

  In the above embodiment, an electronic endoscope provided with only an image sensor has been described as an example of the endoscope apparatus. However, an ultrasonic endoscope provided with an ultrasonic probe may be used. In this case, in addition to the endoscopic image, an ultrasonic image captured by the ultrasonic probe may be linked as a medical image with the schema diagram.

DESCRIPTION OF SYMBOLS 10 Endoscope apparatus 11 Medical image display apparatus 13 Endoscope 14 Processor apparatus 17a Microphone (identification information input part)
22 Operation part 26 Release button 27 Movie recording switch 31 CPU (medical image link information recording part)
36 Data storage 42 Display 46 CPU (Schema diagram link information adding unit, control unit)

Claims (19)

A medical image recorded as image data is displayed using an endoscopic device that has an insertion portion to be inserted into a living body and an operation portion connected to a proximal end portion of the insertion portion and images the inside of the living body. In a medical image display device for displaying on
The medical image includes a still image, and associates the image data with the still image and a schematic diagram schematically representing the internal structure of the living body including the imaging region captured by the still image. A schema diagram link information adding unit for adding the schema diagram link information of
A control unit that searches for one of the schema diagram and the still image on the display and displays the other on the basis of the schema diagram link information.
Identification information for identifying the imaging part of the still image, and the identification information input by recording with a microphone or operating the endoscope apparatus at the time of capturing the still image is the image data of the still image It is attached,
The said schema diagram link information provision part matches the said still image with the specific position in the said schema diagram based on the said identification information, The medical image display apparatus characterized by the above-mentioned.   In the image data of the schema diagram, collation information for collating with the identification information is preliminarily attached for each specific position in the schema diagram, and the schema diagram link information adding unit includes the identification information and the collation The medical image display apparatus according to claim 1, wherein the still image and the specific position are associated by collating information.   3. The medical image according to claim 1, wherein the identification information is voice data recorded by the microphone and is collated with the collation information recorded as voice data corresponding to the identification information. Display device.   The medical image display apparatus according to claim 1, wherein the identification information is input by an operation of the operation unit or an operation of a console of a processor device included in the endoscope apparatus.   The medical image display apparatus according to claim 4, wherein the identification information is text data and is collated with the collation information recorded as text data corresponding to the identification information.   The control unit searches for and displays the still image corresponding to the specific position based on the schema diagram link information when the specific position in the schematic diagram displayed on the display is pointed to. The medical image display apparatus according to claim 1, wherein the medical image display apparatus is a medical image display apparatus.   The medical image display apparatus according to claim 1, wherein the control unit displays the still image and the schema diagram side by side on the display.   The medical image includes a moving image in addition to the still image, and the control unit is based on medical image link information that is recorded by the endoscope device and associates image data of the moving image and the still image. The medical image display device according to claim 1, wherein one of the moving image and the still image is searched and displayed while the other is displayed.   The medical image link information includes recording position information indicating the recording position of the still image in the entire moving image, and the control unit determines whether the moving image is being reproduced based on the recording position information. 9. The medical image display device according to claim 8, wherein the still image corresponding to the recording position is displayed.   The medical image display apparatus according to claim 8 or 9, wherein the control unit displays the moving image side by side on the display in addition to the still image and the schema diagram. An endoscope apparatus that has an insertion section that is inserted into a living body and an operation section that is connected to a proximal end portion of the insertion section, and records as image data using the endoscope apparatus. In a medical image display system having a medical image display device for displaying a medical image thus obtained on a display,
The medical image includes a still image, and associates the image data with the still image and a schematic diagram schematically representing the internal structure of the living body including the imaging region captured by the still image. A schema diagram link information adding unit for adding the schema diagram link information of
A control unit that searches for one of the schema diagram and the still image on the display and displays the other on the basis of the schema diagram link information.
Identification information for identifying the imaging part of the still image, and the identification information input by recording with a microphone or operating the endoscope apparatus at the time of capturing the still image is the image data of the still image It is attached,
The schema diagram link information adding unit associates the still image with a specific position in the schema diagram based on the identification information. The endoscope apparatus is
A release button for inputting the still image shooting instruction;
An identification information input unit for inputting the identification information, and an identification information input unit comprising any one of consoles of processor units included in the microphone, the operation unit, and the endoscope device;
12. The medical image display system according to claim 11, further comprising an identification information recording unit that records the identification information along with the image data of the still image.   The medical image includes a moving image in addition to the still image, and the control unit is based on medical image link information that is recorded by the endoscope device and associates image data of the moving image and the still image. The medical image display system according to claim 11 or 12, wherein one of the moving image and the still image is searched and displayed while the other is being displayed.   The medical image link information includes recording position information indicating the recording position of the still image in the entire moving image, and the control unit determines whether the moving image is being reproduced based on the recording position information. The medical image display system according to claim 13, wherein the still image corresponding to the recording position is displayed.   15. The medical image display system according to claim 14, wherein the recording position information uses a time stamp recorded in moving image data during recording of the moving image.   16. The medical image display system according to claim 13, wherein the endoscope has a medical image link information recording unit that records the medical image link information.   The medical image display system according to any one of claims 11 to 16, wherein the still image is recorded with a number of pixels higher than the number of pixels of one frame of the moving image.   The endoscope apparatus used for the medical image display system of any one of Claims 11-17. A medical image photographed by an endoscope apparatus having an insertion portion to be inserted into a living body and an operation portion connected to a proximal end portion of the insertion portion is recorded as image data, and the medical image is displayed as a medical image display device. In the medical image display method of displaying on the display of
The medical image includes a still image, and the identification information for identifying the imaging portion of the still image is input by recording with a microphone or operating the endoscope device at the time of capturing the still image. An information input step;
An identification information recording step for recording the identification information attached to the image data of the still image;
A data capturing step of capturing the still image and the identification information incidental thereto into the medical image display device;
Based on the identification information, the schema diagram is set such that a specific position corresponding to the imaging region of the still image and the still image correspond to each other and set in advance on a schematic diagram schematically showing the internal structure of the living body. A schema diagram link information providing step for adding schema diagram link information for associating each image data of the still image;
A medical image display method comprising: displaying one of the still image and the schema diagram on the display while searching for the other based on the schema diagram link information. .

Images