JP2018047067A - Image processing program, image processing method, and image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To associate an intestinal image with a designated zone in the intestine, and record it.SOLUTION: An image processing device 101 acquires a captured image captured by an imaging device 102 inserted into the intestine for capturing an intestinal image. The image processing device 101 receives the designation of which zone the intestine corresponds to, of the intestine divided into a plurality of zones. The designation of the zone is performed by a voice input, for example. The image processing device 101 stores in a storage part 120 one or a plurality of captured images acquired in a period when the designation of the zone is effective in association of the zone.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing program, an image processing method, and an image processing apparatus.

  Endoscopy is an examination that allows the doctor to see the inside of the body directly with the eyes by inserting the tip of the endoscope into the body and projecting the inside of the body with a camera mounted on the tip. It is. For example, in colonoscopy, the doctor first inserts the distal end of the endoscope up to the entrance of the small intestine, then pulls out and observes the intestine and takes a picture when a lesion such as a polyp is found. .

  As a prior art, at the time of endoscopy, a part selection screen, a disease name selection screen, and a property selection screen are sequentially displayed based on support information, and a part selected by a medical worker inputting voice Some names, disease names, and properties are highlighted on each screen. The support information includes a plurality of part name data of the hollow organ, a plurality of disease name data, and a plurality of property data associated with the disease name data.

Utility Model Registration No. 3194808

  However, in the prior art, it is difficult to record which part of the intestine corresponds to the image obtained by imaging the intestine during endoscopy.

  In one aspect, an object of the present invention is to record an intestinal image in association with a designated zone in the intestine.

  According to one aspect of the present invention, a captured image is acquired from an imaging device that is inserted into the intestine and captures an intestinal image, and is an intestine corresponding to any one of the intestines classified into a plurality of zones. An image processing program and an image processing apparatus are proposed that store one or a plurality of captured images acquired in a period in which the designation is valid and store them in a storage unit in association with the zone that accepted the designation.

  According to one aspect of the present invention, when it is detected that a voice having a predetermined content has been input from the microphone, until the detection that the voice having a predetermined other content has been input is performed. An image processing method is proposed in which a captured image acquired from an imaging device inserted in the intestine during the period is stored in a storage unit in association with a part of the intestine corresponding to the predetermined content.

  According to one aspect of the present invention, an intestinal image can be recorded in association with a designated zone in the intestine.

FIG. 1 is an explanatory diagram of an example of the image processing method according to the embodiment. FIG. 2 is an explanatory diagram showing the large intestine. FIG. 3 is an explanatory diagram illustrating a system configuration example of the medical support system 300. FIG. 4 is a block diagram illustrating a hardware configuration example of the image processing apparatus 101. FIG. 5 is an explanatory diagram showing an example of the stored contents of the intestinal image DB 320. FIG. 6 is an explanatory diagram showing an example of the contents stored in the zone information DB 330. FIG. 7 is a block diagram illustrating a functional configuration example of the image processing apparatus 101. FIG. 8 is an explanatory diagram showing an example of updating the stored contents of the intestinal image DB 320. FIG. 9 is an explanatory diagram illustrating a screen example of the inspection result screen 900. FIG. 10 is a flowchart illustrating an example of an image acquisition processing procedure of the image processing apparatus 101. FIG. 11 is a flowchart illustrating an example of a zone designation receiving process procedure of the image processing apparatus 101. FIG. 12 is a flowchart illustrating an example of an image processing procedure of the image processing apparatus 101.

  Hereinafter, embodiments of an image processing program, an image processing method, and an image processing apparatus according to the present invention will be described in detail with reference to the drawings.

(Embodiment)
FIG. 1 is an explanatory diagram of an example of the image processing method according to the embodiment. In FIG. 1, an image processing apparatus 101 is a computer that is inserted into the intestine and acquires a captured image captured by the imaging apparatus 102 that captures an intestinal image. The imaging device 102 is a device that captures a still image or a moving image, and is, for example, a CCD (Charge Coupled Device) mounted at the distal end portion of the endoscope 110.

  The endoscope 110 is a medical instrument for observing a cavity in the body. In endoscopy, the distal end portion of the endoscope 110 is inserted into the body, and the state of the body is projected on the monitor by the imaging device 102 mounted on the distal end portion, so that the doctor can directly see the state of the body with the eyes. Can be inspected. Examples of organs to be examined include the large intestine, the small intestine, and the duodenum.

  In the following description, “colon” will be described as an example of an organ to be examined.

  FIG. 2 is an explanatory diagram showing the large intestine. In FIG. 2, the large intestine is a part of the vertebrate digestive tract, and is a tube having a total length of about 1.6 m located between the small intestine and the anus. The large intestine is classified into a plurality of zones such as, for example, the cecum, ascending colon, right colonic curvature, transverse colon, left colonic curvature, descending colon, sigmoid colon, and rectum.

  As shown in FIG. 1, in an endoscopic examination, the doctor D inserts the endoscope 110 from the anus of the patient K, inserts the distal end portion of the endoscope 110 to the entrance portion of the small intestine, and then pulls out the intestine. When the inside is observed and a lesion such as a polyp is found, an intestinal image is taken. In addition, from the viewpoint of leaving evidence, a part where there is no problem in the intestine may be photographed.

  It is desirable to record each image taken in the endoscopy so that it can be seen which part of the intestine corresponds to it. However, during the examination, the doctor D often has both hands occupied to operate the endoscope 110, and it is not possible to manually record which part of the intestine each image corresponds to. difficult.

  Also, during the examination, it is desirable that the work other than the operation of the endoscope 110 be as small as possible so that the doctor D can concentrate on the examination. Furthermore, since the large intestine expands and contracts, it is difficult for the computer to automatically determine in which part of the intestine the distal end portion of the endoscope 110 is located.

  For this reason, it is conceivable that the doctor D remembers in which part each image taken in the intestine corresponds to and records the examination record based on the memory after the examination is performed. However, there may be several tens of images taken by one endoscopy, and doctor D cannot remember which part of the intestine each image corresponds to. There is a risk of writing errors.

  Therefore, in the present embodiment, an image processing method capable of automatically recording an intestinal image captured in an endoscopic examination in association with a designated site in the intestine will be described. Hereinafter, a processing example of the image processing apparatus 101 will be described.

  (1) The image processing apparatus 101 acquires a captured image that is inserted into the intestine and captured by the imaging apparatus 102 that captures the intestinal image. The intestinal image is taken at an arbitrary imaging timing based on a doctor's judgment during the examination. Specifically, for example, the image processing apparatus 101 acquires a captured image captured from the imaging apparatus 102 every time the intestine is captured by the imaging apparatus 102. Further, the image processing apparatus 101 may acquire the captured images captured by the imaging apparatus 102 during the inspection from the imaging apparatus 102 in a lump after the inspection is performed.

  In the example of FIG. 1, it is assumed that the captured images 111, 112, and 113 are captured by the judgment of the doctor D who is an operator in the endoscopic examination. In this case, for example, the image processing apparatus 101 acquires the captured images 111, 112, and 113 from the imaging apparatus 102 every time the captured images 111, 112, and 113 are captured. In FIG. 1, dotted arrows extending from the captured images 111, 112, and 113 indicate acquisition timings of the captured images 111, 112, and 113.

  (2) The image processing apparatus 101 receives designation of which intestine corresponds to which of the intestines classified into a plurality of zones. A zone is a site (segment) that divides the intestine. In the case of the large intestine, as shown in FIG. 2, the colon is classified into a plurality of zones such as the cecum, ascending colon, right colonic curve, transverse colon, left colonic curve, descending colon, sigmoid colon, and rectum.

  The zone is specified during the examination and is performed by the surgeon and assistant doctor. In addition, the nurse may designate the zone in accordance with the doctor's instruction. The designation of the zone may be performed by voice input, for example, or may be performed by a keyboard, a mouse, or a touch panel.

  Specifically, for example, when a doctor starts an examination of a certain part in the intestine, the zone corresponding to the part may be designated (prior designation). In addition, when the examination of a certain part in the intestine is completed, the doctor may designate a zone corresponding to the part (post-designation).

  In the example of FIG. 1, when the doctor D starts an examination of a certain part in the intestine, the zone corresponding to the part is designated (preliminary designation). Here, it is assumed that the doctor D designates “ascending colon” at time t1, designates “transverse colon” at time t2, and designates “descending colon” at time t3 (t1 → t2). → t3).

  (3) The image processing apparatus 101 stores in the storage unit 120 one or more captured images acquired during a period in which the zone designation is valid, in association with the zone. For example, assume that when examination of a certain part in the intestine is started, a zone corresponding to that part is designated (prior designation). In this case, the period in which the designation of the zone is valid is a period from the designation of the zone to the designation of another zone. In addition, when the inspection of a certain part in the intestine is completed, a zone corresponding to the part is designated (post-designation). In this case, the period during which the zone is specified is the period before the zone is specified.

  In the example of FIG. 1, when the doctor D starts an examination of a certain part in the intestine, the zone corresponding to the part is designated (preliminary designation). Therefore, the period in which the designation of “ascending colon” is valid is a period from time t1 to time t2 until designation of “transverse colon” is made after designation of “ascending colon”. For this reason, the image processing apparatus 101 stores the captured images 111 and 112 acquired in the period from the time point t1 to the time point t2 in the storage unit 120 in association with the “ascending colon”.

  The period during which the designation of “transverse colon” is valid is the period from time t2 to time t3 after designation of “transverse colon” until designation of “descending colon”. For this reason, the image processing apparatus 101 stores the captured image 113 acquired in the period from the time point t2 to the time point t3 in the storage unit 120 in association with the “transverse colon”.

  As described above, according to the image processing apparatus 101, any of the intestines classified into the plurality of zones is acquired while acquiring the captured image that is inserted into the intestine and captured by the imaging apparatus 102 that captures the intestinal image. It is possible to accept the designation of the intestine corresponding to the zone. Then, according to the image processing apparatus 101, it is possible to store one or a plurality of captured images acquired in a period in which the designation of the zone is effective in the storage unit 120 in association with the zone.

  Thereby, the intestinal image imaged in the endoscopic examination can be automatically associated with a designated site in the intestine and recorded. This eliminates the need for the operator to remember which part of the intestine each image corresponds to during the examination and to write it in the examination record after the examination. Leaks and mistakes can be prevented.

(System configuration example of medical support system 300)
Next, a system configuration example of the medical support system 300 according to the embodiment will be described. The medical support system 300 is a computer system that supports medical care in a medical institution. The medical institution is, for example, a hospital or a clinic.

  FIG. 3 is an explanatory diagram illustrating a system configuration example of the medical support system 300. In FIG. 3, the medical support system 300 includes an image processing apparatus 101 and an endoscope system 301. In the medical support system 300, the image processing apparatus 101 and the endoscope system 301 are connected via a wired or wireless network 310. The network 310 is, for example, a local area network (LAN), a wide area network (WAN), or the Internet.

  Here, the image processing apparatus 101 includes an intestinal image DB (Database) 320 and a zone information DB 330, and acquires a captured image from the endoscope system 301. The image processing apparatus 101 may be, for example, a PC (Personal Computer), a tablet PC, or the like used by a medical worker, or may be a server.

  In addition, the image processing apparatus 101 includes a microphone 302. The microphone 302 is a device that converts sound into an electrical signal. For example, the microphone 302 is wirelessly connected to the apparatus main body by short-range wireless communication such as Bluetooth (registered trademark). Specifically, for example, the microphone 302 is used by an operator who performs endoscopy, and may be a tie pin type or a headset type.

  The contents stored in the intestinal image DB 320 and the zone information DB 330 will be described later with reference to FIGS.

  The endoscope system 301 includes an endoscope device 340 and a control device 350. The endoscope device 340 is connected to the control device 350 and includes an operation unit 341 and an insertion unit 342. The operation unit 341 is provided with various buttons such as an angle knob connected to the distal end of the insertion unit 342 by a wire, a shutter button, an air / water supply button, and a suction button.

  For example, the doctor operates the angle knob of the operation unit 341, inserts the distal end portion of the insertion portion 342 into the body, and bends the distal end portion of the insertion portion 342 vertically and horizontally to observe the inside of the body cavity at 360 degrees. Can do. In addition, the doctor can capture an image of the intestine by pressing the shutter button of the operation unit 341.

  Moreover, the insertion part 342 is a part inserted in the intestines. At the distal end portion of the insertion portion 342, a camera 343 (lens, imaging device, etc.) for imaging the intestine, an illumination lens (not shown), a forceps port, a nozzle, and the like are provided. The endoscope 110 illustrated in FIG. 1 corresponds to the endoscope apparatus 340, for example.

  The control device 350 is a computer having a monitor 351, and supplies light, air, and water to the insertion unit 342, converts an electrical signal captured by the image sensor of the camera 343 into a video signal, and displays it on the monitor 351. To do. Further, the control device 350 transmits the captured image captured by the camera 343 to the image processing device 101.

  In the example of FIG. 3, the image processing apparatus 101 and the endoscope system 301 are provided separately, but the present invention is not limited to this. For example, the image processing apparatus 101 may be realized by the control apparatus 350 of the endoscope system 301. In the example of FIG. 3, the image processing apparatus 101 has the microphone 302. However, the present invention is not limited to this. For example, another computer connected to the image processing apparatus 101 may have the microphone 302. In this case, the image processing apparatus 101 acquires audio data received by the microphone 302 from another computer.

(Hardware configuration example of image processing apparatus 101)
FIG. 4 is a block diagram illustrating a hardware configuration example of the image processing apparatus 101. In FIG. 4, the image processing apparatus 101 includes a CPU (Central Processing Unit) 401, a memory 402, a disk drive 403, a disk 404, an I / F (Interface) 405, a display 406, an input device 407, And a microphone 302. Each component is connected by a bus 400.

  Here, the CPU 401 controls the entire image processing apparatus 101. The memory 402 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a flash ROM, and the like. Specifically, for example, a flash ROM or ROM stores various programs, and the RAM is used as a work area of the CPU 401. The program stored in the memory 402 is loaded on the CPU 401 to cause the CPU 401 to execute the coded process.

  The disk drive 403 controls reading / writing of data with respect to the disk 404 according to the control of the CPU 401. The disk 404 stores data written under the control of the disk drive 403. Examples of the disk 404 include a magnetic disk and an optical disk.

  The I / F 405 is connected to the network 310 via a communication line, and is connected to another computer (for example, the control device 350 of the endoscope system 301 shown in FIG. 3) via the network 310. The I / F 405 controls the interface between the network 310 and the own apparatus, and controls input / output of data from other apparatuses.

  The display 406 displays data such as a document, an image, and function information as well as a cursor, an icon, or a tool box. As the display 406, for example, a liquid crystal display, an organic EL (electroluminescence) display, a CRT (Cathode Ray Tube), or the like can be employed.

  The input device 407 has keys for inputting characters, numbers, various instructions, and the like, and inputs data. The input device 407 may be a keyboard or a mouse, or may be a touch panel type input pad or a numeric keypad. As described above, the microphone 302 is a device that converts sound into an electric signal. For example, the microphone 302 may be a tie pin type or a headset type.

(Contents stored in the intestinal image DB 320)
Next, the contents stored in the intestinal image DB 320 of the image processing apparatus 101 will be described. The intestinal image DB 320 is realized by a storage device such as the memory 402 and the disk 404 shown in FIG.

  FIG. 5 is an explanatory diagram showing an example of the stored contents of the intestinal image DB 320. In FIG. 5, the intestinal image DB 320 has fields of examination ID, surgeon ID, imaging time, zone name, and captured image. By setting information in each field, intestinal image information (for example, intestinal image information) Image information 500-1 to 500-4) is stored as a record.

  The examination ID (identification) is an identifier that uniquely identifies the endoscopic examination. The surgeon ID is an identifier that uniquely identifies the surgeon performing endoscopy. The imaging time indicates the date and time when the intestinal image was captured. The zone name is the name of the intestinal zone (part) corresponding to the intestinal image. In the initial state, the zone name is “-(null)”. The captured image is image data of an intestinal image.

  For example, the intestinal image information 500-1 indicates that the imaging time “2016/03 / of the captured image P1 captured by the operator (doctor) with the operator ID“ D1 ”in the endoscopic inspection with the inspection ID“ T1 ”. 28 10:29:35 "and the zone name"-". Each intestinal image information in the intestinal image DB 320 may store patient information such as patient ID, patient name, examination name, age, and disease name in association with each other.

(Storage contents of the zone information DB 330)
Next, the contents stored in the zone information DB 330 included in the image processing apparatus 101 will be described. The zone information DB 330 is realized by a storage device such as the memory 402 and the disk 404 shown in FIG.

  FIG. 6 is an explanatory diagram showing an example of the contents stored in the zone information DB 330. In FIG. 6, the zone information DB 330 has fields of examination ID, surgeon ID, zone name, examination start time, examination end time, and examination time. By setting information in each field, zone information (for example, Zone information 600-1 to 600-4) is stored as a record.

  Here, the examination ID is an identifier for uniquely identifying an endoscopic examination that is being performed or has been performed. The surgeon ID is an identifier that uniquely identifies the surgeon performing endoscopy. The zone name is the name of a zone (part) in the intestine. The examination start time indicates the date and time when the examination of the intestinal zone corresponding to the zone name is started. The examination end time indicates the date and time when the examination of the intestinal zone corresponding to the zone name is finished. The inspection time indicates time [s (seconds)] from the inspection start time to the inspection end time.

  For example, the zone information 600-1 indicates that the examination start time “2016/03/28 10:29:30” by the surgeon (doctor) with the surgeon ID “D1” in the endoscopic examination with the examination ID “T1”. From this time to the examination end time “2016/03/28 10:30:42”, the zone (site) of the zone name “terminal ileum” is examined.

(Functional configuration example of the image processing apparatus 101)
FIG. 7 is a block diagram illustrating a functional configuration example of the image processing apparatus 101. In FIG. 7, the image processing apparatus 101 includes an acquisition unit 701, a reception unit 702, an association unit 703, a determination unit 704, and an output unit 705. The acquisition unit 701 to the output unit 705 are functions serving as a control unit. Specifically, for example, by causing the CPU 401 to execute a program stored in a storage device such as the memory 402 and the disk 404 illustrated in FIG. Alternatively, the function is realized by the I / F 405. The processing result of each functional unit is stored in a storage device such as the memory 402 and the disk 404, for example.

  The acquisition unit 701 acquires a captured image that is inserted into the intestine and captured by a camera 343 that captures an intestinal image (see FIG. 3). Specifically, for example, the acquisition unit 701 acquires the captured image from the control device 350 of the endoscope system 301 every time a captured image is captured by the camera 343.

  The acquired captured image is stored in, for example, the intestinal image DB 320 illustrated in FIG. Specifically, for example, the acquisition unit 701 adds a time stamp to the acquired captured image, and stores it in the intestinal image DB 320 in association with the examination ID and the operator ID. This time stamp indicates the imaging time of the captured image, and indicates, for example, the date and time (Japan Standard Time) when the acquisition unit 701 acquires the captured image.

  Thereby, new intestinal image information is registered in the intestinal image DB 320. Even if the endoscope system 301 does not have a timekeeping function or is not synchronized in time with the image processing apparatus 101, the image processing apparatus 101 gives an imaging time synchronized with Japanese standard time to the captured image. be able to. However, at this time, the zone name of the intestinal image information is “−”. The examination ID and the operator ID are input, for example, by a user operation input using the input device 407 shown in FIG. 4 when starting an endoscopic examination.

  The accepting unit 702 accepts designation of which intestine corresponds to which of the intestines classified into a plurality of zones. Here, the plurality of zones include, for example, the ascending colon, the transverse colon, the descending colon, and the sigmoid colon. The designation of the zone may be performed, for example, by voice input of a user (for example, a doctor) using the microphone 302 or by operation input of a user (for example, an assistant or nurse) using the input device 407. It may be broken.

  In the following description, a plurality of zones for classifying the intestines are expressed as “zones Z1 to Zn” (n is a natural number of 2 or more), and an arbitrary zone among the zones Z1 to Zn is expressed as “zone Zi”. There are cases (i = 1, 2,..., N). In addition, unless otherwise specified, the designation of the zone Zi assumes a case where the zone Zi corresponding to the part is designated (preliminary designation) when examination of a certain part in the intestine is started.

  Specifically, for example, the accepting unit 702 accepts the designation of the zone Zi when the result of the speech recognition processing on the speech data input from the microphone 302 is the content indicating any zone Zi of the intestine. As the speech recognition technology, any existing technology may be used. For example, a technology for converting speech data into words by word spotting or a subword method may be used.

  In other words, for example, the reception unit 702 refers to a medical term dictionary in which a site (zone name) in the intestine is registered, and the result of performing speech recognition processing on the speech data indicates any zone name in the intestine. The designation of the zone Zi indicated by the zone name is accepted. In addition, for example, the reception unit 702 refers to the medical term dictionary and designates the zone Zi when the character string input using the input device 407 has contents indicating any zone Zi of the intestine. You may decide to accept.

  Information on the designated zone Zi is stored, for example, in the zone information DB 330 shown in FIG. Specifically, for example, the reception unit 702 adds a time stamp to the designated zone Zi, and stores the zone name of the zone Zi in the zone information DB 330 in association with the examination ID and the operator ID. This time stamp indicates the inspection start time of the designated zone Zi. For example, the time stamp indicates the date (Japan Standard Time) when the accepting unit 702 accepts designation of the zone Zi.

  As a result, new zone information is registered in the zone information DB 330. However, at this time, the inspection end time and the inspection time of the zone information are “−”. The inspection end time and the inspection time are set when the next zone information is registered. Specifically, the time immediately before the inspection start time of the next zone information (for example, the time one second before) is set as the inspection end time of the zone information. In addition, a time based on the inspection start time and the inspection end time of the zone information is set as the inspection time of the zone information.

  The accepting unit 702 may output information indicating that the designation of the zone Zi is accepted in response to the designation of the zone Zi. Specifically, for example, the reception unit 702 may display the zone name on the display 406 when the result of voice recognition processing of the voice data indicates any zone name in the intestine. Thereby, the doctor can visually confirm that designation of the zone Zi has been accepted.

  Further, the reception unit 702 may output a synthesized sound indicating the zone name from a speaker (not shown) when the result of the speech recognition processing on the audio data indicates any zone name in the intestine. . For example, when the result of the voice recognition processing indicates the zone name “ascending colon”, the reception unit 702 may output the synthesized sound “ascending colon” from the speaker of the headset worn by the doctor.

  Thereby, since the doctor can confirm that the designation of the zone Zi is accepted without looking at the display 406, the doctor can concentrate on the medical treatment. Furthermore, when there is no response such as a voice output of the synthesized sound, the doctor knows that the designation of the zone Zi has not been accepted, and can perform a voice instruction or the like again to prevent specification omission.

  The associating unit 703 associates one or a plurality of captured images acquired during a period in which the designation of the zone Zi is valid with the designated zone Zi. Here, the period in which the designation of the zone Zi is valid is a period from the designation of the zone Zi to the designation of another zone Zj (j ≠ i, j = 1, 2,..., N). . That is, the period during which the designation of the zone Zi is valid corresponds to the period from the inspection start time to the inspection end time of the zone Zi.

  Here, taking the zone information 600-2 in the zone information DB 330 (see FIG. 6) as an example, the captured image captured in the endoscopic examination with the examination ID “T1” is designated as the designated zone “ascending colon”. The process of associating with will be described.

  First, the associating unit 703 refers to the zone information 600-2, from the inspection start time “2016/03/28 10:30:43” to the inspection end time “2016/03/28 10:33:03”. Specify the inspection time zone. Next, the associating unit 703 searches the intestinal image DB 320 for intestinal image information whose imaging time is included in the specified examination time zone. In the example of FIG. 5, intestinal image information 500-2 is searched. Then, the associating unit 703 sets the zone name “ascending colon” of the zone information 600-2 as the zone name of the searched intestinal image information 500-2.

  Thereby, the captured image P2 acquired in the period (inspection time zone) in which the designation of the zone “ascending colon” is valid can be stored in the intestinal image DB 320 in association with the zone “ascending colon”. Here, an example of updating the stored contents of the intestinal image DB 320 will be described.

  FIG. 8 is an explanatory diagram showing an example of updating the stored contents of the intestinal image DB 320. In FIG. 8, zone names corresponding to the captured images P1 to P4 are set in the zone name fields of the intestinal image information 500-1 to 500-4, respectively. Thereby, the captured image acquired in the period in which the designation of each zone Zi is valid can be stored in the intestinal image DB 320 in association with each zone Zi.

  Returning to the description of FIG. 7, the determination unit 704 determines whether the designation of the zone Zi is performed in a predetermined order. Here, the predetermined order can be arbitrarily set. For example, after inserting the distal end portion of the insertion portion 342 to the entrance portion of the small intestine (for example, the end of the ileum) and then pulling it out, the intestine is pulled out. In this case, the order of passage of the distal end of the insertion portion 342 (for example, the terminal ileum ⇒ ascending colon ⇒ transverse colon ⇒ descending colon) is set.

  Specifically, for example, first, the determination unit 704 extracts zone information of a predetermined inspection ID from the zone information DB 330. The predetermined examination ID may be an examination ID of an endoscopic examination that is being performed, or may be an examination ID of an endoscopic examination that has been performed. Taking the inspection ID “T1” as an example, the zone information 600-1 to 600-4 is extracted from the zone information DB 330.

  Next, the determination unit 704 sorts the zone names of the extracted zone information 600-1 to 600-4 in order from the oldest test start time, and designates the zone designation order “terminal of ileum ⇒ ascending colon ⇒ transverse colon ⇒ descending colon”. Is identified. Then, the determination unit 704 determines whether or not the specified order of the specified zones is in a predetermined order.

  Here, when the predetermined order is “terminal ileum ⇒ ascending colon ⇒ transverse colon ⇒ descending colon”, the determination unit 704 determines that the specified order of the specified zone “terminal ileum ⇒ ascending colon ⇒ transverse colon ⇒ descending colon” is predetermined. It is determined that the order is correct. Thereby, it is possible to determine whether or not the intestinal examination is performed in a predetermined order.

  Further, the determination unit 704 determines whether or not a captured image is associated with each predetermined zone. Here, the predetermined zone can be arbitrarily set, and may be, for example, all the zones Z1 to Zn, or one or a plurality of zones selected from the zones Z1 to Zn. Good.

  Specifically, for example, first, the determination unit 704 extracts intestinal image information of a predetermined examination ID from the intestinal image DB 320. The predetermined examination ID may be an examination ID of an endoscopic examination that is being performed, or may be an examination ID of an endoscopic examination that has been performed. Taking the examination ID “T1” as an example, intestinal image information 500-1 to 500-4 is extracted from the intestinal image DB 320 (see FIG. 8).

  Next, the determination unit 704 refers to the extracted intestinal image information 500-1 to 500-4, and identifies the zone “terminal ileum, ascending colon, transverse colon, descending colon” associated with the captured image. . Here, the predetermined zone is defined as “ascending colon, transverse colon, descending colon”. In this case, the determination unit 704 determines that the captured image is associated with each predetermined zone. Thereby, for example, it is possible to determine whether or not an intestinal image of the zone Zi in which imaging is recommended or required in endoscopy is taken.

  Further, the determination unit 704 may determine whether or not a predetermined zone is specified in a predetermined order and whether or not a captured image is associated with each of the predetermined zones. Accordingly, it is possible to determine whether or not an intestinal image of the zone Zi in which intestinal inspection is performed in a predetermined order and imaging is recommended or required is taken.

  The output unit 705 outputs the determination result determined by the determination unit 704. As an output format of the output unit 705, for example, display on the display 406, transmission to an external device by the I / F 405, storage in a storage device such as the memory 402 and the disk 404, audio output from a speaker (not illustrated), There is a print output to the illustrated printer.

  Specifically, for example, the output unit 705 outputs a determination result as to whether or not the designation of the zone Zi has been made in a predetermined order. Thereby, it is possible to present information capable of determining whether or not the intestinal examination is performed in a predetermined order.

  For example, the output unit 705 outputs a determination result as to whether or not a captured image is associated with each predetermined zone. Thereby, it is possible to present information capable of determining whether or not an intestinal image of the zone Zi that is recommended or required to be imaged in the endoscopic examination is taken.

  Further, for example, the output unit 705 outputs a determination result as to whether or not a predetermined zone is designated in a predetermined order and a captured image is associated with each of the predetermined zones. As a result, intestinal examinations are performed in a predetermined order, and information capable of determining whether or not an intestinal image of a zone where imaging is recommended or required is taken can be presented. .

  The output unit 705 outputs a captured image acquired in a period in which the designation of the zone Zi is valid in association with information indicating the zone Zi. Specifically, for example, when the output unit 705 displays the acquired captured images on the display 406 in chronological order, the output unit 705 indicates which intestinal zone Zi each captured image to be displayed in chronological order is associated with. Information to be displayed may be displayed.

  More specifically, for example, the output unit 705 extracts intestinal image information and zone information of a predetermined examination ID from the intestinal image DB 320 and the zone information DB 330. Next, the output unit 705 displays a test result screen 900 as shown in FIG. 9 described later on the display 406 based on the extracted intestinal image information and zone information.

  In the above description, a time stamp is given to the acquired captured image, but the present invention is not limited to this. For example, when time synchronization is performed between the image processing apparatus 101 and the endoscope system 301, information indicating the shooting time of the captured image is previously included in the captured image transmitted from the endoscope system 301. It may be added.

  Further, in the above description, when an examination of a certain part in the intestine is started, the case where the zone Zi corresponding to the part is designated (preliminary designation) has been described as an example, but the present invention is not limited thereto. For example, when the inspection of a certain part in the intestine is completed, the zone Zi corresponding to the part may be designated (post-designation). In this case, for example, when the designation of the zone Zi is accepted, the accepting unit 702 adds a time stamp indicating the examination end time of the zone Zi and stores it in the zone information DB 330. Further, the time immediately after the inspection end time of the zone Zi (for example, the time after one second) is set as the inspection start time of the next zone information. For example, the start time of the endoscopic examination is set as the examination start time of the first zone information.

(Screen example of inspection result screen 900)
Next, a screen example of the inspection result screen 900 displayed on the display 406 will be described with reference to FIG.

  FIG. 9 is an explanatory diagram illustrating a screen example of the inspection result screen 900. In FIG. 9, an examination result screen 900 is an operation screen that displays captured images (“intestinal images” in FIG. 9) that are captured in the endoscopic examination with the examination ID “T1” in chronological order.

  The inspection result screen 900 displays information (“zone” in FIG. 9) indicating which zone Zi each captured image P1 to P5 is associated with. The inspection result screen 900 displays an inspection time zone (“Time” in FIG. 9) from the inspection start time to the inspection end time of each zone Zi.

  According to the inspection result screen 900, the user can intuitively grasp which zone Zi in the intestine each captured image P1 to P5 captured in the endoscopic inspection. . For example, the user can grasp whether the captured image P1 is an image of the “terminal end of the ileum” in the intestine.

  In addition, in the inspection result screen 900, an area having a width corresponding to the inspection time of each zone Zi is secured, and the captured images P1 to P5 are displayed. For this reason, the user can intuitively grasp how long it took to examine each zone Zi in the intestine. For example, the user can grasp that the “transverse colon” in the intestine has been examined for a longer time than the “terminal ileum” or the “ascending colon”.

  In the inspection result screen 900, for example, when a cursor C is moved and a captured image is selected by a user operation input using the input device 407, detailed information such as an imaging time of the captured image may be confirmed. it can. In the example of FIG. 9, as a result of selecting the captured image P2, detailed information 910 indicating the imaging time and the like of the captured image P2 is displayed.

(Various processing procedures of the image processing apparatus 101)
Next, various processing procedures of the image processing apparatus 101 will be described with reference to FIGS. First, the image acquisition processing procedure of the image processing apparatus 101 will be described.

  FIG. 10 is a flowchart illustrating an example of an image acquisition processing procedure of the image processing apparatus 101. In the flowchart of FIG. 10, first, the image processing apparatus 101 determines whether or not an inspection start instruction for endoscopic inspection has been received (step S <b> 1001). The inspection start instruction is given by a user operation input using the input device 407, for example.

  Here, the image processing apparatus 101 waits to receive an inspection start instruction (step S1001: No). When the image processing apparatus 101 receives an inspection start instruction (step S1001: Yes), the image processing apparatus 101 determines whether a captured image captured by the camera 343 has been acquired from the control apparatus 350 of the endoscope system 301. (Step S1002).

  Here, the image processing apparatus 101 waits for acquiring a captured image (step S1002: No). When the captured image is acquired (step S1002: Yes), the image processing apparatus 101 adds a time stamp to the acquired captured image (step S1003), and associates with the examination ID and the operator ID in the intestinal image. Register in the DB 320 (step S1004).

  Next, the image processing apparatus 101 determines whether or not an inspection end instruction for endoscopy has been received (step S1005). The inspection end instruction is given by, for example, a user operation input using the input device 407.

  Here, when the inspection end instruction has not been received (step S1005: No), the image processing apparatus 101 returns to step S1002. On the other hand, when an inspection end instruction is received (step S1005: Yes), the image processing apparatus 101 ends a series of processes according to this flowchart.

  Thereby, the intestinal image imaged in the endoscopy can be recorded in association with the imaging time.

  Next, a zone designation receiving process procedure of the image processing apparatus 101 will be described.

  FIG. 11 is a flowchart illustrating an example of a zone designation receiving process procedure of the image processing apparatus 101. In the flowchart of FIG. 11, first, the image processing apparatus 101 determines whether an inspection start instruction for endoscopic inspection has been received (step S <b> 1101). Here, the image processing apparatus 101 waits to receive an inspection start instruction (step S1101: No).

  If the image processing apparatus 101 receives an inspection start instruction (step S1101: Yes), the image processing apparatus 101 determines whether or not the audio data input from the microphone 302 has been acquired (step S1102). Here, the image processing apparatus 101 waits for acquisition of audio data (step S1102: No).

  When the audio data is acquired (step S1102: Yes), the image processing apparatus 101 performs audio recognition processing on the acquired audio data (step S1103). Next, the image processing apparatus 101 determines whether any zone name in the intestine is detected as a result of the voice recognition processing (step S1104).

  If the zone name is not detected (step S1104: NO), the image processing apparatus 101 proceeds to step S1109. On the other hand, when a zone name is detected (step S1104: Yes), the image processing apparatus 101 adds a time stamp to the detected zone name (step S1105) and registers new zone information in the zone information DB 330 ( Step S1106).

  Next, the image processing apparatus 101 sets the time immediately before the time indicated by the time stamp set in step S1105 (immediately before the inspection start time of new zone information) as the inspection end time of the immediately preceding zone information in the zone information DB 330. (Step S1107). The immediately preceding zone information is zone information registered immediately before the new zone information registered in step S1106.

  Then, the image processing apparatus 101 sets the time from the inspection start time to the inspection end time of the zone information as the inspection time of the immediately preceding zone information in the zone information DB 330 (step S1108). Next, the image processing apparatus 101 determines whether or not an inspection end instruction for endoscopy has been received (step S1109).

  Here, when the inspection end instruction is not received (step S1109: No), the image processing apparatus 101 returns to step S1102. On the other hand, when the inspection end instruction is received (step S1109: Yes), the image processing apparatus 101 sets the current time as the inspection end time of the new zone information in the zone information DB 330 (step S1110).

  Then, the image processing apparatus 101 sets the time from the inspection start time to the inspection end time of the zone information as the inspection time of the new zone information in the zone information DB 330 (step S1111), and a series of steps according to this flowchart. End the process.

  Thereby, the zone name of the zone Zi designated by voice input among the intestinal zones Z1 to Zn can be recorded in association with the examination start time, examination end time, and the like. If there is no previous zone information in the zone information DB 330 in step S1107, the image processing apparatus 101 proceeds to step S1109.

  In step S1102, for example, the image processing apparatus 101 may acquire only audio data whose sound pressure level exceeds a predetermined reference, that is, audio data when an utterance is detected. In addition, the image processing apparatus 101 may acquire audio data every certain time (for example, 10 seconds) (however, continuous audio data may partially overlap).

  Next, an image processing procedure of the image processing apparatus 101 will be described.

  FIG. 12 is a flowchart illustrating an example of an image processing procedure of the image processing apparatus 101. In the flowchart of FIG. 12, first, the image processing apparatus 101 determines whether or not an image display request for a captured image captured in the endoscopic examination has been received (step S1201).

  The image display request is made, for example, by a user operation input using the input device 407. The image display request includes an inspection ID for endoscopic inspection.

  Here, the image processing apparatus 101 waits to receive an image display request (step S1201: No). When the image processing apparatus 101 receives an image display request (step S1201: Yes), the image processing apparatus 101 extracts zone information corresponding to the examination ID included in the image display request from the zone information DB 330 (step S1202).

  Next, the image processing apparatus 101 selects unselected zone information that has not been selected from the extracted zone information (step S1203). Then, the image processing apparatus 101 refers to the selected zone information and specifies an examination time zone from the examination start time to the examination end time (step S1204).

  Next, the image processing apparatus 101 obtains intestinal image information corresponding to the examination ID included in the image display request from the intestinal image DB 320 and including the imaging time in the specified examination time zone. Search is performed (step S1205). The image processing apparatus 101 determines whether intestinal image information has been searched (step S1206).

  If the intestinal image information is not retrieved (step S1206: No), the image processing apparatus 101 proceeds to step S1208. On the other hand, when the intestinal image information is retrieved (step S1206: Yes), the image processing apparatus 101 sets the zone name of the selected zone information as the zone name of the retrieved intestinal image information (step S1207).

  Next, the image processing apparatus 101 determines whether there is unselected zone information that has not been selected among the extracted zone information (step S1208). If there is unselected zone information (step S1208: YES), the image processing apparatus 101 returns to step S1203.

  On the other hand, when there is no unselected zone information (step S1208: No), the image processing apparatus 101 reads the intestinal image information and the zone corresponding to the examination ID included in the image display request from the intestinal image DB 320 and the zone information DB 330. Information is extracted (step S1209). Next, the image processing apparatus 101 generates screen information of a test result screen based on the extracted intestinal image information and zone information (step S1210).

  Then, the image processing apparatus 101 displays an inspection result screen on the display 406 based on the generated screen information (step S1211), and ends a series of processing according to this flowchart. Thereby, the captured image acquired in the period in which the designation of each zone Zi is valid can be associated with each zone Zi, and each captured image captured in the endoscopy corresponds to the intestine of which zone Zi It is possible to display an inspection result screen that can grasp the above.

  As described above, according to the image processing apparatus 101 according to the embodiment, a captured image that is inserted into the intestine and captured by the camera 343 that captures the intestinal image can be acquired. In addition, according to the image processing apparatus 101, it is possible to receive designation of which intestine corresponds to which zone Zi among the intestines classified into the zones Z1 to Zn. Then, according to the image processing apparatus 101, one or a plurality of captured images acquired in a period in which the designation of the zone Zi is valid can be stored in association with the designated zone Zi.

  Thereby, the intestinal image imaged in the endoscopic examination can be automatically associated with a designated site in the intestine and recorded. This eliminates the need for the operator to remember which part of the intestine each image corresponds to during the examination and to write it in the examination record after the examination. Leaks and mistakes can be prevented.

  Further, according to the image processing apparatus 101, the designation of the zone Zi can be received by voice input using the microphone 302. As a result, even if the surgeon is often occupied with both hands during the examination, the zone Zi can be designated simply by uttering a word indicating the site in the intestine, improving convenience. Provide an environment where it is easy to concentrate on inspection.

  Further, according to the image processing apparatus 101, when the result of performing voice recognition processing on the voice data input from the microphone 302 indicates any zone name in the intestine, the synthesis indicating the zone name is performed from a speaker (not shown). Sound can be output. Thereby, since the doctor can confirm that the designation of the zone Zi is accepted without looking at the display 406, the doctor can concentrate on the medical treatment. Furthermore, when there is no output of synthesized sound, the doctor knows that the designation of the zone Zi has not been accepted, and can perform a voice instruction again to prevent designation omission.

  Further, according to the image processing apparatus 101, the designation of the zone Zi can be accepted by a user operation input using the input device 407. Thus, a nurse or the like who supports the surgeon can designate the zone Zi simply by inputting a word indicating a site in the intestine in accordance with the surgeon's instruction.

  Further, according to the image processing apparatus 101, it is possible to output a determination result as to whether or not the designation of the zone Zi has been made in a predetermined order. Thereby, the information which can judge whether the test | intestine in the intestine is performed in the predetermined order can be shown, and a test | inspection omission can be prevented.

  Further, according to the image processing apparatus 101, it is possible to output a determination result as to whether or not a captured image is associated with each predetermined zone. This presents information that can be used to determine whether or not an intestinal image of the zone Zi that is recommended or required for imaging in endoscopy is displayed, and prevents forgetting to take an intestinal image. Can do.

  Further, according to the image processing apparatus 101, it is possible to output a determination result as to whether or not a predetermined zone is designated in a predetermined order and whether a captured image is associated with each of the predetermined zones. Thereby, intestinal examinations are performed in a predetermined order, and information that can determine whether or not an intestinal image of the zone Zi for which imaging is recommended or required is presented. it can.

  In addition, according to the image processing apparatus 101, when the acquired captured images are displayed on the display 406 in time series order, information indicating which zone Zi is associated with each intestine image to be displayed in time series order. Can be displayed. Thereby, the user can grasp | ascertain which site | part in the intestine each captured image imaged sequentially in the endoscopic examination imaged.

  The image processing method described in the present embodiment can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. This image processing program is recorded on a computer-readable recording medium such as a hard disk, flexible disk, CD-ROM, MO (Magneto-Optical disk), DVD (Digital Versatile Disk), USB (Universal Serial Bus) memory, etc. It is executed by being read from the recording medium by a computer. The image processing program may be distributed via a network such as the Internet.

  The following additional notes are disclosed with respect to the embodiment described above.

(Additional remark 1) It acquires in the intestine, acquires a captured image from the imaging device which images an intestinal image,
Accepting the designation of which intestine corresponds to which of the intestines classified into multiple zones,
Storing one or a plurality of captured images acquired in a period in which the designation is valid in a storage unit in association with the zone in which the designation is received;
An image processing program for causing a computer to execute processing.

(Supplementary note 2) The period in which the designation is valid is a period until designation of another zone after the designation.
The image processing program according to supplementary note 1, wherein

(Supplementary Note 3) The period in which the designation is valid is a period before the designation is made.
The image processing program according to supplementary note 1, wherein

(Supplementary Note 4) The designation is performed by voice input.
The image processing program according to any one of appendices 1 to 3, wherein:

(Additional remark 5) The said designation | designated is detected by the analysis result of the audio | voice data input from the microphone showing any zone of an intestine.
The image processing program according to any one of appendices 1 to 4, characterized in that:

(Supplementary Note 6) The designation is performed by a keyboard, a mouse, or a touch panel.
The image processing program according to any one of appendices 1 to 3, wherein:

(Supplementary note 7) The plurality of zones includes at least one of the ascending colon, the transverse colon, the descending colon, and the sigmoid colon.
The image processing program according to any one of appendices 1 to 6, characterized in that:

(Additional remark 8) The determination result of whether the designation | designated of the zone was made in the predetermined order is output,
The image processing program according to any one of appendices 1 to 7, characterized in that:

(Additional remark 9) The determination result of whether the captured image was matched with each of the predetermined zone is output,
The image processing program according to any one of appendices 1 to 7, characterized in that:

(Supplementary Note 10) A predetermined zone is designated in a predetermined order, and a determination result as to whether or not a captured image is associated with each of the predetermined zones is output.
The image processing program according to any one of appendices 1 to 7, characterized in that:

(Additional remark 11) When displaying the acquired captured image in time series order, the information which shows to which intestine of which zone each captured image to display in time series order is displayed is displayed.
The image processing program according to any one of appendices 1 to 10, characterized in that:

(Additional remark 12) When the analysis result of the audio | speech data input from the said microphone shows any zone of an intestine, the synthetic | combination sound which shows the name of the said zone is output.
The image processing program according to appendix 5, characterized in that:

(Supplementary note 13) When it is detected that a sound having a predetermined content is input from the microphone, the intestine is detected between the detection and the detection that a sound having a predetermined other content is input. Storing the captured image acquired from the imaging device inserted into the storage unit in association with the site in the intestine corresponding to the predetermined content,
An image processing method, wherein the computer executes the processing.

(Supplementary Note 14) The predetermined content indicates a first part of the intestine,
The predetermined other content indicates a second part of the intestine;
14. The image processing method according to appendix 13, wherein

(Supplementary note 15) The first region of the intestine is at least one of the ascending colon, the transverse colon, the descending colon, and the sigmoid colon.
Item 15. The image processing method according to appendix 14, wherein

(Supplementary Note 16) A captured image is acquired from an imaging device that is inserted into the intestine and captures the intestinal image,
Accepting the designation of which intestine corresponds to which of the intestines classified into multiple zones,
Storing one or a plurality of captured images acquired in a period in which the designation is valid in a storage unit in association with the zone in which the designation is received;
An image processing apparatus comprising a control unit having a function of executing processing.

DESCRIPTION OF SYMBOLS 101 Image processing apparatus 102 Imaging device 110 Endoscope 111,112,113, P1-P5 Captured image 120 Storage part 300 Medical assistance system 301 Endoscope system 302 Microphone 320 Intestinal image DB
330 Zone information DB
340 Endoscope device 341 Operation unit 342 Insertion unit 343 Camera 350 Control device 351 Monitor 400 Bus 401 CPU
402 Memory 403 Disk drive 404 Disk 405 I / F
406 Display 407 Input device 701 Acquisition unit 702 Reception unit 703 Association unit 704 Determination unit 705 Output unit 900 Inspection result screen 910 Detailed information

Claims (14)

Obtain a captured image from an imaging device that is inserted into the intestine and captures the intestinal image,
Accepting the designation of which intestine corresponds to which of the intestines classified into multiple zones,
Storing one or a plurality of captured images acquired in a period in which the designation is valid in a storage unit in association with the zone in which the designation is received;
An image processing program for causing a computer to execute processing. The period during which the designation is valid is a period until designation of another zone after the designation.
The image processing program according to claim 1, wherein: The period during which the designation is valid is a period before the designation is made.
The image processing program according to claim 1, wherein: The designation is performed by voice input.
The image processing program according to any one of claims 1 to 3. The designation is detected by the analysis result of the voice data input from the microphone indicating any zone of the intestine.
The image processing program according to any one of claims 1 to 4, wherein The designation is performed by a keyboard, mouse or touch panel.
The image processing program according to any one of claims 1 to 3. The plurality of zones includes at least one of the ascending colon, the transverse colon, the descending colon, and the sigmoid colon.
An image processing program according to any one of claims 1 to 6. Outputs the determination result of whether or not the zone is specified in a predetermined order.
An image processing program according to any one of claims 1 to 7, wherein Outputting a determination result as to whether or not a captured image is associated with each of the predetermined zones;
An image processing program according to any one of claims 1 to 7, wherein When displaying the acquired captured images in chronological order, each captured image to be displayed in chronological order is displayed with information indicating which zone the intestine is associated with.
An image processing program according to any one of claims 1 to 9, wherein When it is detected that a sound having a predetermined content is input from the microphone, the sound is inserted into the intestine between the detection and the detection that the sound having a predetermined other content is input. The captured image acquired from the imaging device is stored in the storage unit in association with the site in the intestine corresponding to the predetermined content,
An image processing method, wherein the computer executes the processing. The predetermined content indicates a first part of the intestine;
The predetermined other content indicates a second part of the intestine;
The image processing method according to claim 11. The first part of the intestine is at least one of the ascending colon, the transverse colon, the descending colon, and the sigmoid colon;
The image processing method according to claim 12. Obtain a captured image from an imaging device that is inserted into the intestine and captures the intestinal image,
Accepting the designation of which intestine corresponds to which of the intestines classified into multiple zones,
Storing one or a plurality of captured images acquired in a period in which the designation is valid in a storage unit in association with the zone in which the designation is received;
An image processing apparatus comprising a control unit having a function of executing processing.

Images