JP6644530B2 - Endoscope business support system - Google Patents

Description

  The present invention relates to a system for supporting an endoscope operation.

  Currently, data recorded for endoscopy is mainly still images, moving images, and report information created by a doctor after the examination. Also, it is possible to record very general numerical information such as the overall time required for the inspection together with the report information.

  In recent years, many attempts have been made to obtain new knowledge and knowledge by realizing big data of various kinds of events in the real world using output information from devices, sensing technology, etc., and to realize various improvements and efficiency. It has been done. If you are going to do the same thing in endoscopic medicine, for example, how endoscopy is effective in finding lesions, what is the difference between a skilled doctor and an unskilled doctor It is expected that various kinds of knowledge and knowledge will be discovered, such as what appears in such a place, whether it is compliant with the so-called routine inspection protocol, and how to improve the inspection protocol.

  Patent Literature 1 collects state information indicating a state of a medical device and / or a person related to a medical practice in a time-series manner, and performs an execution order and a start timing of a plurality of stages based on an occurrence state of an event determined from the state information. And a medical information system that generates procedural information indicating an execution order and a start timing is disclosed.

JP 2006-164251 A

  In medical facilities, it is effective to define an examination protocol that defines examination procedures in order to prevent differences in patient burden and organ observation methods due to differences in doctors' experiences and skills. However, there is no point in defining a test protocol in cases where a physician forgets the test protocol or does not remember the test protocol correctly.

  In addition, if an inexperienced unskilled physician performed an endoscopy, the skilled physician also provides guidance after the examination, refer to the report created by the unskilled physician, do not overlook the findings or make no errors in diagnosis It is common to check whether or not. In a medical facility that defines a test protocol, it is preferable to confirm that an unskilled physician performed the test in accordance with the test protocol.However, although the findings are recorded in the test report, Is not recorded, it is not possible to confirm that the test protocol is currently being complied with.

  The present invention has been made in view of such a situation, and an object of the present invention is to provide a technique that allows a medical facility that defines a test protocol to perform a test that complies with the test protocol.

  In order to solve the above-mentioned problems, an endoscope operation support system according to an embodiment of the present invention includes a plurality of actions (endoscopic operation, organ observation, imaging, operation, operation, and the like) to be performed by a doctor in an endoscopy. (Hereinafter referred to as “task”), an inspection protocol storage unit that stores a plurality of tasks stored in the inspection protocol storage unit on a display device, and detects that the task has been performed. An execution task detection unit and a recording unit that records that the task has been executed are provided.

  It is to be noted that any combination of the above-described components and any conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, and the like are also effective as embodiments of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to make the test | inspection which obeys a test protocol performed in the medical facility which defines a test protocol.

FIG. 1 is a diagram illustrating a configuration of an endoscope operation support system according to a first embodiment of the present invention. FIG. 1 is a diagram showing a configuration of an endoscope system according to Embodiment 1. FIG. 1 is a diagram showing a configuration of an endoscope inspection data recording device according to a first embodiment. FIG. 3 is a diagram showing a configuration of a terminal device according to Embodiment 1. It is a figure showing an example of an endoscope image containing a biopsy forceps. FIGS. 6A to 6C are diagrams illustrating examples of the master information table held in the master information holding unit. FIG. 4 is a diagram illustrating an example of test data recorded in a test data holding unit. FIG. 9 is a diagram illustrating another example of the test data recorded in the test data holding unit. 5 is a flowchart for explaining an operation example of the endoscope system according to the first embodiment. 5 is a flowchart for explaining an operation example of the endoscope inspection data recording device according to the first embodiment. FIG. 9 is a diagram showing a configuration of an endoscope system according to Embodiment 2. FIG. 6 is a diagram showing a configuration of an endoscope inspection data recording device according to a second embodiment. FIGS. 13A and 13B are diagrams illustrating an example of an imaging distance master table held in a master information holding unit and an example of inspection data recorded in an inspection data holding unit. FIG. 9 is a diagram showing a configuration of an endoscope system according to Embodiment 3. FIG. 9 is a diagram showing a configuration of an endoscope inspection data recording device according to a third embodiment. FIG. 4 is a diagram illustrating an example of an imaging state master table held in a master information holding unit. FIG. 4 is a diagram illustrating an example of test data recorded in a test data holding unit. FIG. 4 is a diagram illustrating an example of a change evaluation value / imaging state master table held in a master information holding unit. FIG. 4 is a diagram illustrating an example of test data recorded in a test data holding unit. 15 is a flowchart for explaining an operation example of the endoscope system according to an application example of Embodiment 3. FIG. 9 is a diagram showing a configuration of an endoscope inspection data recording device according to a fourth embodiment. FIG. 4 is a diagram illustrating an example of statistical data recorded in a statistical data holding unit. 15 is a flowchart for explaining an operation example of the endoscope inspection data recording device according to the fourth embodiment. FIG. 14 is a diagram showing a configuration of an endoscope inspection data recording device according to a fifth embodiment. FIGS. 25A and 25B are diagrams illustrating an example of a part specifying screen for adding part information to an endoscopic image in the order of imaging. 15 is a flowchart for explaining an operation example of the endoscope inspection data recording device according to the fifth embodiment. It is a figure showing an example of a specification screen of two points in an endoscope examination. It is a figure showing an example of a selection screen between parts in an endoscope examination. It is a figure showing the 1st state (immediately after an examination start) of an example of a screen which expressed transition of an endoscope examination by data. It is a figure showing the 2nd state (after a lapse of 5 minutes from the examination start) of the example of a screen which expressed transition of the endoscope examination by data. It is a figure showing the 3rd state (at the time of 11 minutes after an examination start) of the example of a screen which expressed transition of an endoscope examination by data. It is a figure showing the 4th state (at the time of an examination end) of the example of a screen showing transition of an endoscope examination by data. FIG. 15 is a diagram showing a configuration of an endoscope system according to Embodiment 8. FIG. 15 is a diagram showing a configuration of an endoscope inspection data recording device according to an eighth embodiment. It is a figure showing an example of an inspection protocol screen displayed on a display. It is a figure showing another example of an inspection protocol screen displayed on a display. It is a figure showing another example of an inspection protocol screen displayed on a display. It is a figure showing an example of a display mode of a completed task. It is a figure showing another example of a display form of a completed task. It is a figure showing another example of a display form of a completed task. It is a figure showing another example of a display form of a completed task. It is a figure showing another example of an inspection protocol screen. It is a figure showing another example of an inspection protocol screen.

(Embodiment 1)
FIG. 1 is a diagram showing a configuration of an endoscope service support system 1 according to Embodiment 1 of the present invention. In Embodiment 1, the endoscope service support system 1 is an endoscope inspection data recording system installed in the endoscope department, and operates in cooperation with a task support system (not shown). The endoscope service support system 1 includes an endoscope system 10, an endoscope inspection data recording device 20, and a terminal device 30, which are interconnected by a network 2 such as a LAN.

  The endoscope service support system 1 can cooperate with another system in a medical facility. For example, a gateway device (not shown) is connected to the network 2, and the endoscope service support system 1 can cooperate with an ordering system, an electronic medical record system, a medical accounting system, and the like via the gateway device.

  FIG. 2 is a diagram illustrating a configuration of the endoscope system 10 according to the first embodiment. The endoscope system 10 includes an endoscope 11, an endoscope processing device (also referred to as a camera control unit) 12, a display device 13, and a light source device 14. The endoscope 11 is inserted into the body of a patient and photographs the inside of the patient. The endoscope 11 includes an imaging element 111, a forceps channel 112, and an operation unit 113.

  The imaging element 111 includes a CCD image sensor, a CMD image sensor, or a CMOS image sensor, and photoelectrically converts incident light. The image signal generated by the photoelectric conversion is subjected to signal processing such as A / D conversion and noise removal by a signal processing circuit (not shown), and is output to the endoscope processing device 12. The forceps channel 112 is a channel for passing a treatment tool such as forceps. The operation unit 113 is provided with a release button, an angle knob for bending the end of the endoscope, and the like.

  The display device 13 displays an image input from the endoscope processing device 12. For example, an image of the inside of the patient's body imaged by the endoscope 11 is displayed in real time. The light source device 14 includes a light source such as a xenon lamp and sends light to the tip of the endoscope 11. The light source device 14 can irradiate the endoscope 11 with a plurality of types of observation light. For example, white light, narrow band imaging (NBI), fluorescence, and near-infrared light can be emitted.

  The endoscope processing device 12 controls the entire endoscope system 10. The endoscope processing device 12 includes a control unit 121 and a communication unit 126. The control unit 121 includes an irradiation light switching control unit 122 and an image recognition unit 123. The image recognition unit 123 includes a medicine scattering detection unit 123a and a treatment execution detection unit 123b. Only the functional blocks related to the processing of the first embodiment are illustrated in the control unit 121 of FIG.

  The function of the control unit 121 can be realized by cooperation of hardware resources and software resources, or only by hardware resources. Processors, ROMs, RAMs, and other LSIs can be used as hardware resources. Programs such as an operating system and applications can be used as software resources. For example, the image recognition unit 123 may be configured by an FPGA.

  FIG. 3 is a diagram showing a configuration of the endoscope inspection data recording device 20 according to the first embodiment. The endoscope inspection data recording device 20 is composed of, for example, a server or a PC. The endoscope inspection data recording device 20 includes a communication unit 21, a control unit 22, a storage unit 23, and a console unit 24. The control unit 22 includes a data acquisition unit 221, an operation reception unit 222, a display control unit 223, a data calculation unit 224, and a data recording unit 225. The data acquisition unit 221 includes an irradiation light type data acquisition unit 221a, a medicine scattering data acquisition unit 221b, and a treatment execution data acquisition unit 221c. The control unit 22 of FIG. 3 also illustrates only functional blocks related to the processing of the first embodiment. The function of the control unit 22 can also be realized by cooperation of hardware resources and software resources, or only by hardware resources.

  The storage unit 23 includes a recording medium such as an HDD and an SSD, and includes a master information holding unit 231 and a test data holding unit 232. Only the functional blocks related to the processing of the first embodiment are also illustrated in the storage unit 23 of FIG.

  FIG. 4 is a diagram showing a configuration of the terminal device 30 according to Embodiment 1. The terminal device 30 is a terminal device used by medical personnel engaged in medical institutions such as doctors and nurses, and includes, for example, a PC, a tablet, a PDA, and the like. When a mobile terminal device such as a tablet or a PDA is used, an access point (not shown) is installed in the network 2 and connected to the network 2 by a wireless LAN. The terminal device 30 includes a communication unit 31, a control unit 32, a storage unit 33, a display unit 34, and an operation input unit 35. Hereinafter, a specific description will be given with reference to FIGS.

  An endoscope image obtained by photographing the inside of the patient with the endoscope 11 is input to the endoscope processing device 12 as a moving image. For example, it is input at a frame rate of 30 Hz. The display control unit (not shown) of the endoscope processing device 12 causes the display device 13 to display the input endoscope moving image. An operation receiving unit (not shown) of the endoscope processing device 12 receives a doctor's operation performed on the operation unit 113 of the endoscope 11. For example, a press of a release button by a doctor is accepted. An image extraction unit (not shown) of the endoscope processing device 12 extracts a still image from the endoscope moving image at the timing when the release button is pressed. The extracted still image is stored in a storage device (not shown) of the endoscope system 10 or transferred to and stored in an endoscope image recording device (not shown) via the network 2. It may be transferred to the endoscope inspection data recording device 20 and stored, or may be transferred to a PACS (Picture Archiving and Communication System) (not shown) and stored.

  The irradiation light switching control unit 122 receives the switching operation of the observation light performed by the doctor on the operation unit (not shown) of the endoscope processing device 12, and emits the observation light of the type instructed by the switching operation. The light source device 14 is controlled as follows. The observation light emitted by the light source device 14 is guided into the body cavity of the patient through the endoscope 11 and irradiates the inside of the body cavity.

  The medicine scattering detection unit 123a detects the medicine used in the endoscopic inspection from the frame image included in the endoscope moving image by image recognition. The medicine scattering detection unit 123a periodically searches for medicines at predetermined time intervals from the frame image by image recognition. For example, the search is performed every N seconds or every N frames. For example, the search is performed every one second or every five seconds.

  Drugs to be searched are pigments and stains that have been sprayed into the patient's body. The coloring agent is used for observing the shape and unevenness by storing on the surface without being absorbed by the living mucous membrane. Staining agents are used to observe differences in mucosal surface properties and cell structures depending on the presence or absence of absorption by tissues and differences in reactions.

  Representative dyes and dyes include indigo carmine, lugol (iodine), pioctanine (crystal violet), methylene blue, acetic acid, and ink. The main scattered organs of indigo carmine are the stomach and large intestine, and are used for differential diagnosis and range diagnosis of neoplastic lesions (cancer, adenoma, etc.) and non-neoplastic lesions (hyperplastic polyps). The main scattered organ of Lugor is the esophagus, which is used for differential diagnosis and range diagnosis of esophageal cancer. The main organ to which picotanine is applied is the large intestine, which is used for differential diagnosis and deepening diagnosis of neoplastic and non-neoplastic lesions including colorectal cancer. The main organs in which methylene blue is sprayed are the stomach and large intestine, and are used to diagnose intestinal metaplasia of the stomach. In recent years, it has also been used for nuclear staining for ultra-magnifying endoscopy (EC) observation. In some cases, picotanine is used for this purpose. The main organs to which acetic acid is applied are the stomach and large intestine, and are used for differential diagnosis and range diagnosis of neoplastic lesions (cancer, adenoma, etc.) and non-neoplastic lesions (hyperplastic polyps). Used in combination with indigo carmine to increase the accuracy of range diagnosis.

  The medicine scattering detection unit 123a detects medicines (for example, indigo carmine (blue to green blue), lugol (brown to yellowish brown), picotanine (purple to blue purple), methylene blue (blue), acetic acid (white) from the endoscope image. ) Is detected based on the color tone feature quantity. For example, a method described in Japanese Patent Application No. Hei 7-116138 and Japanese Patent Application No. 2014-191781 can be used as a detection method based on the color tone feature amount.

  The medicine scattering detection unit 123a calculates a color tone feature amount for each of the endoscopic images to be searched, and detects the presence or absence of the medicine in each of the endoscopic images to be searched. When the medicine is sprayed, the type of the medicine is specified. The medicine spraying detector 123a sends medicine spraying data indicating the spraying state of the medicine in the patient's body to the data transmitter 125 every N seconds or every N frames.

  The treatment execution detection unit 123b detects a treatment tool used for endoscopy by image recognition from a frame image included in the endoscope moving image, and detects a treatment being performed based on the detected treatment tool. I do. The treatment execution detection unit 123b periodically searches for a treatment tool from the frame image by image recognition. Normally, the processing is performed in synchronization with the image recognition processing of the medicine by the medicine scattering detection unit 123a.

  The treatment tool includes forceps. The forceps include biopsy forceps, grasping forceps, hot biopsy forceps, and the like. A hot biopsy forceps is a forceps that can be energized at a high frequency and can pinch, cut off, and collect foreign matter. Further, the treatment tool includes an injection needle. The injection needle includes a biopsy needle and a local injection needle for local injection of physiological saline. The treatment tools include high-frequency treatment tools. The high-frequency treatment instrument includes a snare, an incision forceps, a knife, a hemostat. The high frequency snare is used for EMR (Endoscopic Mucosal Resection). The high-frequency knife is used for ESD (Endoscopic Submucosal Dissection).

  The treatment instrument includes a clip used for hemostasis. In addition, the treatment tool includes a ligating device for ligating a polyp in the digestive tract. The treatment tool includes a treatment tool for a duodenal scope. The treatment instrument for duodenal scope includes a cannula, a contrast tube, a lithotripter, and the like. The cannula is used for ERCP (Endoscopic Retrograde Cholangiopancreatography).

  The treatment tool includes a guide wire, a brush, a basket, a tube, a stent, a balloon, a catheter, and the like. Guidewires and catheters are used for ERCP. The brush is used for cytology. The basket is used for quarrying and collecting foreign objects. Tubes are used for dye application. Stents are used for placement procedures. The balloon is used for dilatation.

  Hereinafter, a specific method for detecting the treatment tool will be described. In the endoscope 11, the position of the forceps channel 112 of the treatment tool is determined depending on the model. Some models have two forceps channels 112. The endoscope image obtained when each treatment tool is inserted through the forceps channel 112 is used as a template image, and the endoscope image at the time of examination is compared with the endoscope image to detect which treatment tool has been used. As the template image, a plurality of images having different forceps channel directions, protrusion lengths, and open / close states are prepared for each treatment tool. In addition, for a treatment tool having an asymmetric shape whose shape on an image changes due to rotation, a plurality of images having different rotation angles are prepared.

  In order to detect a treatment tool from an endoscope image, an edge is first detected from the endoscope image. A red (R) image or a green (G) image is used as an image for edge detection. When the treatment instrument sheath is red, a green (G) image may be used. Next, a line segment shape is detected from the edge image using template matching, Hough transform, or the like. Next, the detected line segment shape is compared with the template image to calculate the degree of coincidence. The treatment tool of the template image having the highest degree of coincidence is set as the detection result.

  FIG. 5 is a diagram illustrating an example of an endoscope image 40 including the biopsy forceps 41. The treatment instrument basically enters the frame from the direction of 4 to 8 o'clock as shown in FIG. Therefore, the template image of the treatment tool only needs to be prepared in the direction of the frame-in from the direction of 4 to 8 o'clock. This can reduce the number of template images and the amount of calculation required for collation.

  In detecting the treatment tool, it is also possible to consider the color information of the sheath. Further, an algorithm using a feature amount such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Up Robust Features) used in pattern detection or the like in recent years may be used.

  The treatment execution detection unit 123b detects the treatment being performed based on the treatment tool detected by the image recognition. For example, if a biopsy forceps is detected, it is determined that a biopsy is being performed. The treatment execution detection unit 123b passes the treatment execution data indicating the state of the treatment performed by the doctor to the data transmission unit 125 every N seconds or every N frames.

  The irradiation light switching control unit 122 synchronizes with the detection timing of the medicine spraying and the treatment execution by the medicine scattering detection unit 123a and the treatment execution detection unit 123b, and the observation light used by the light source device 14 every N seconds or every N frames. Is transmitted to the data transmission unit 125.

  The data transmission unit 125 transmits the irradiation light type data passed from the irradiation light switching control unit 122, the medicine scattering data passed from the medicine scattering detection unit 123a, and the treatment execution data passed from the treatment execution detection unit 123b to a network. 2 to the endoscope inspection data recording device 20. The data transmission unit 125 may transmit the irradiation light type data, the medicine dispersion data, and the treatment execution data delivered at predetermined time intervals, or may temporarily store the data in a buffer (not shown). They may be sent together. For example, after the endoscope inspection, irradiation light type data, drug dispersion data, and treatment execution data of the endoscope inspection may be transmitted collectively. If the capacity of the buffer (not shown) is designed to be small, the accumulated data may be transmitted collectively when the free space of the buffer (not shown) is exhausted.

  The irradiation light type data acquisition unit 221a, the medicine scatter data acquisition unit 221b, and the treatment execution data acquisition unit 221c of the endoscope inspection data recording device 20 convert the irradiation light type data, the medicine dispersion data, and the treatment execution data into an endoscope. Each is obtained from the system 10. The data recording unit 225 records the transition of the data acquired by the irradiation light type data acquiring unit 221a, the medicine scattering data acquiring unit 221b, and the treatment execution data acquiring unit 22c in the test data holding unit 232.

  In the present embodiment, the irradiation light type data acquisition unit 221a, the medicine scatter data acquisition unit 221b, and the treatment execution data acquisition unit 221c acquire the irradiation light type data, the medicine scatter data, and the time series data of the treatment execution data, respectively. . The data recording unit 225 records the acquired time-series data as it is or after processing it in the inspection data holding unit 232.

  FIGS. 6A to 6C are diagrams illustrating examples of the master information table held in the master information holding unit 231. FIG. 6A shows an example of the observation light type master table 213a, FIG. 6B shows an example of the medicine type master table 213b, and FIG. 6C shows an example of the treatment type master table 213c. .

  In the observation light type master table 213a shown in FIG. 6A, four types of observation light types are defined: white light = 1, narrow band light = 2, fluorescence = 3, and near infrared light = 4. In the medicine type master table 213b shown in FIG. 6B, five kinds of medicine types, indigo carmine = 1, Lugol = 2, biotanine = 3, methylene blue = 4, and acetic acid = 5, are defined. The number 0 indicates that no medicine is detected. In the treatment type master table 213c shown in FIG. 6C, nine types of treatment types, ie, biopsy = 1, polypectomy = 2, local injection = 3,..., Stent = 9 are defined as treatment types. Note that the number 0 indicates that no treatment is detected. Although not shown, a treatment tool master table is provided in which each treatment and a treatment tool used in each treatment are linked, and a treatment type is specified based on the treatment tool detected by image recognition.

  The same table as the observation light type master table 213a is also provided in the irradiation light switching control unit 122 of the endoscope system 10. The same table as the medicine type master table 213b is also provided in the medicine scattering detection section 123a of the endoscope system 10. The same table as the treatment type master table 213c and the treatment tool master table is also provided in the treatment execution detection unit 123b of the endoscope system 10.

  The irradiation light switching control unit 122 checks the type of the observation light at predetermined time intervals, converts the checked type of the observation light into a number with reference to the observation light type master table, and passes the number to the data transmission unit 125. The medicine scattering detection unit 123a searches for a medicine in the endoscope image at predetermined time intervals, converts the search result into a number with reference to the medicine type master table, and passes the result to the data transmission unit 125. The treatment execution detection unit 123b searches for a treatment tool in the endoscope image at predetermined time intervals, converts the search result into a number with reference to the treatment tool master table and the treatment type master table, and sends the data to the data transmission unit 125. hand over.

  FIG. 7 is a diagram illustrating an example of the test data 232a recorded in the test data holding unit 232. In the example illustrated in FIG. 7, the data recording unit 225 includes the irradiation light type data, the medicine dispersion data, and the treatment execution data acquired by the irradiation light type data acquisition unit 221a, the medicine dispersion data acquisition unit 221b, and the treatment execution data acquisition unit 221c. This is an example in which time-series data of data is directly recorded in the inspection data holding unit 232.

  In this example, data on observation light, spraying of medicine, and treatment are recorded in a time series from the start of the examination to the end of the examination in units of one second. Note that the time-series data may be data in units of frames instead of data in units of time. In the example shown in FIG. 7, the observation light is switched from white light to narrow-band light at the point of time when 2 seconds have elapsed from the start of the inspection. The biopsy has been started 100 seconds after the start of the test. Lugor is sprayed 1000 seconds after the start of the inspection.

  In order to record the endoscopy more accurately, in addition to the start time and end time of the endoscopy, the time when the endoscope 11 is inserted into the body cavity and the time when the endoscope 11 is removed outside the body cavity are recorded. It is desired.

  An examination start / end detection unit (not shown) of the endoscope system 10 detects the start of an endoscope examination when a doctor presses an examination start button, and sends examination start detection data including the detection time. Hand over to section 125. The start of the inspection may be detected by attaching the endoscope 11 to the light source device 14 or turning on the power of the light source device 14. In addition, the examination start / end detection unit (not shown) detects the end of the endoscopic examination when the doctor presses the examination end button, and passes the examination end detection data including the detection time to the data transmission unit 125. The end of the examination may be detected due to the removal of the endoscope 11 from the light source device 14 or the power off of the light source device 14.

  When detecting that the endoscope 11 has been inserted into the body cavity, the insertion state detecting unit (not shown) of the endoscope system 10 transmits endoscope insertion detection data including the detection time to the data transmission unit 125. Pass to The insertion of the endoscope 11 into the body cavity and the removal from the body cavity can be detected by the method described in Japanese Patent Application No. Hei 7-116138. Further, the body cavity insertion state detection unit (not shown) may detect that the endoscope 11 has been inserted into the body cavity based on, for example, the increase rate of the red (R) component of the endoscope image. Further, it may be detected that the endoscope 11 has passed through the mouthpiece and inserted into the body cavity based on a detection signal of a sensor attached to the mouthpiece of the patient.

  When detecting that the endoscope 11 has been removed outside the body cavity, the insertion state detecting unit (not shown) in the body cavity passes endoscope removal detection data including the detection time to the data transmission unit 125. The insertion state detection unit (not shown) in the body cavity may detect that the endoscope 11 has been removed from the body cavity based on, for example, the reduction rate of the red (R) component of the endoscope image. Further, it may be detected that the endoscope 11 has passed through the mouthpiece and has been removed from the body cavity based on a detection signal of a sensor attached to the mouthpiece of the patient.

  The data sending unit 125 is provided with the inspection start detection data and the inspection end detection data passed from the inspection start / end detection unit (not shown), and the endoscope insertion detection passed from the body cavity insertion state detection unit (not shown). The data and the endoscope removal detection data are transmitted to the endoscope inspection data recording device 20 via the network 2.

  The examination start / end data acquisition unit (not shown) and the body cavity insertion state detection data acquisition unit (not shown) of the endoscope examination data recording device 20 include examination start detection data, examination end detection data, endoscope insertion detection Data and endoscope removal detection data are acquired. The data recording unit 225 records the inspection start detection data, the inspection end detection data, the endoscope insertion detection data, and the endoscope removal detection data in the inspection data holding unit 232.

  The data calculation unit 224 of the endoscope inspection data recording device 20 uses the time of use of each observation light in the endoscope inspection based on the time-series data of the irradiation light type data acquired by the irradiation light type data acquisition unit 221a. And / or calculating the number of times each observation light is used. The usage time of the observation light can be calculated from the time during which the irradiation of the observation light is continuous. Although details will be described later, the observation start using the irradiation light information included in the supplementary information of the moving image or the still image captured by the endoscope 11 or the specific observation light manually input by the user. It can be calculated from the time difference between the time and the observation end time.

  In addition, the data calculation unit 224 determines the observation time using each medicine in the endoscopic examination based on the time series data of the medicine dispersion data acquired by the medicine dispersion data acquisition unit 221b, and / or the use of each medicine. Calculate the number of times. The observation time using the medicine can be calculated from the time during which the medicine is framed in the moving image. As will be described later in detail, it can be calculated from the time difference between the observation start time and the observation end time using a specific medicine, which is manually input by the user.

  In addition, the data calculation unit 224 calculates the execution time of each treatment and / or the number of times each treatment is performed in the endoscopy based on the time-series data of the treatment execution data acquired by the treatment execution data acquisition unit 221c. . The execution time of the treatment can be calculated from the time during which the treatment tool is framed in the moving image. As will be described later in detail, it can be calculated from the time difference between the treatment start time and the treatment end time manually input by the user.

  The data recording unit 225 includes the use time of each observation light calculated by the data calculation unit 224, the number of use of each observation light, the observation time using each medicine, the number of use of each medicine, the execution time of each treatment, In addition, at least one of the number of times each treatment is performed is recorded in the inspection data holding unit 232.

  FIG. 8 is a diagram illustrating another example of the test data 232b recorded in the test data holding unit 232. In the example illustrated in FIG. 8, the data recording unit 225 uses the observation time using each observation light, the number of times each observation light is used, the observation time using each medicine in the endoscopy calculated by the data calculation unit 224, This is an example in which the number of times of use of each medicine, the execution time of each treatment, and the number of executions of each treatment are recorded in the test data holding unit 232. The inspection data 232b shown in FIG. 8 is data generated by processing the raw inspection data 232a shown in FIG.

  In the example shown in FIG. 8, “Endoscope” in the detection information type column indicates that the number 0 indicates insertion into the body cavity and the number 1 indicates removal from the body cavity. The “observation light” is white light for 0 to 1 second from the start of the test, narrow-band light for 2 to 98 seconds from the start of the test, and white light (second time) for 99 to 2000 seconds from the start of the test. As a procedure, a biopsy is performed for a period of 100 to 600 seconds from the start of the test. Lugor is sprayed as a drug spray in a period of 1000 to 1500 seconds from the start of the test.

  In this manner, the elementary time-series data indicating the transition of the endoscopy can be processed and recorded into the type, the start time, the end time, and the number of events that have occurred in the endoscopy. When the processed data is generated, the raw time-series data may be deleted to reduce the data amount. The processing method of the elementary time-series data may be any method as long as the elementary time-series data can be restored, and the processing method and the recording method are not limited.

  In the above description, an example has been given in which the application of medicine and the execution of treatment are detected by image recognition. In this regard, the user may manually input medicine spraying and treatment execution. For example, a doctor or a nurse looks at an endoscopic image displayed on the display device 13 of the endoscope system 10 or the display unit 34 of the terminal device 30 during or after an endoscopic examination, and sprays a medicine and / or Information regarding the procedure may be input.

  Specifically, the name of the sprayed medicine is input, and the frame images at the observation start time and the observation end time in the medicine spray state are designated. Also, the name of the executed treatment is input, and the frame images at the treatment start time and the treatment end time are designated. The operation accepting unit 222 of the endoscopic examination data recording device 20 accepts information on the application of medicine and / or the execution of the treatment input by the doctor or the nurse, and transfers the information to the data recording unit 225 and the data calculating unit 224. The accuracy of image recognition is reduced in some parts of the body cavity, and when erroneous data is detected by image recognition, it is manually corrected by a doctor or nurse.

  As for the type of observation light, a doctor or a nurse can manually input the switching time of observation light and the type of observation light after switching. In some cases, the type of observation light and the switching point can be specified from the accompanying information of the moving image or frame image captured by the endoscope 11.

  FIG. 9 is a flowchart for explaining an operation example of the endoscope system 10 according to the first embodiment. When the inspection start / end detection unit (not shown) of the endoscope system 10 detects the start of the endoscope inspection (Y in S10), the detection time is notified to the data transmission unit 125. Thereafter, when the insertion state detection unit (not shown) detects the insertion of the endoscope 11 into the body cavity (Y in S11), the detection time is passed to the data transmission unit 125.

  The image recognition unit 123 acquires an endoscope image captured by the endoscope 11 (S12). The medicine scattering detection unit 123a searches for a medicine in the obtained endoscope image, and the treatment execution detection unit 123b searches for a treatment tool in the obtained endoscope image (S13). The medicine scattering detection unit 123a and the treatment execution detection unit 123b pass the search result to the data transmission unit 125. The irradiation light switching control unit 122 checks the type of the observation light (S14). The irradiation light switching control unit 122 passes the confirmation result to the data transmission unit 125. The data sending unit 125 temporarily stores data relating to the observation light, the medicine, and the treatment in a buffer (not shown) (S15).

  If the insertion state detecting unit (not shown) in the body cavity does not detect the removal of the endoscope 11 out of the body cavity (N in S16), and N seconds elapse from the acquisition of the endoscope image (Y in S17), Returning to step S12, a new endoscopic image is obtained (S12). When the insertion state detection unit (not shown) in the body cavity detects that the endoscope 11 has been removed from the body cavity (Y in S16), the detection time is passed to the data transmission unit 125. Thereafter, when the inspection start / end detection unit (not shown) detects the end of the endoscopic inspection (Y in S18), the detection time is notified to the data transmission unit 125. The data transmission unit 125 transmits the data stored in the buffer (not shown) to the endoscope inspection data recording device 20 as time-series data (S19).

  FIG. 10 is a flowchart for explaining an operation example of the endoscope inspection data recording device 20 according to the first embodiment. The data acquisition unit 221 of the endoscope inspection data recording device 20 acquires various time-series data from the endoscope system 10 (S20). The data calculation unit 224 calculates the use time and the number of times of use of the observation light for each type based on the acquired time-series data on the observation light (S21). In addition, the data calculation unit 224 calculates the observation time and the number of times of using the medicine for each type based on the acquired time-series data on the application of the medicine (S22). In addition, the data calculation unit 224 calculates the execution time and the number of times of the treatment for each type based on the acquired time-series data on the execution of the treatment (S23). The data recording unit 225 records the acquired time-series data and the calculated processed data in the inspection data holding unit 232 (S24).

  As described above, according to the first embodiment, it is possible to record, as data at each time point, the type of observation light used in the endoscopy, the execution of various treatments such as precise diagnosis by spraying various kinds of medicine, and biopsy. Therefore, it is possible to grasp the time and the number of times required for spraying the medicine and performing the treatment, to grasp the order of appearance, quantitative evaluation such as calculation of the average value, and to compare with other examinations. Can be accurately and objectively evaluated. In the present embodiment, the endoscope operation support system that records the observation light, the time-series data acquired for the application of the medicine, and the calculated processing data has been described, but any one of these data may be used. It is of course possible to selectively acquire or record the data. Alternatively, the data may be appropriately acquired and recorded according to the purpose of use of the data or the available computing resources.

  Conventionally, information relating to switching of observation light, application of a medicine, and execution of treatment during an endoscopy has been manually recorded afterward by a doctor or nurse based on the memory of the doctor or nurse. However, since the information to be recorded is diversified, the accuracy and granularity of the recording tend to vary from one recorder to another, and unified data recording has not been realized. Also, manual recording by doctors and nurses is a complicated task. On the other hand, according to the first embodiment, the trajectory of the endoscopy can be automatically recorded in time series based on information detected by image recognition from the endoscopic image during the endoscopy. . Therefore, the workload of doctors and nurses can be reduced, and unified data recording becomes possible.

  In addition, the procedure used in the endoscopy can be specified based on the detection order of each event such as drug spraying. For example, if a high-frequency snare is detected after the local injection needle is detected, it can be estimated that EMR has been performed. In addition, if a magnified observation is performed while using the narrow band light (NBI), the medical treatment fee is added. According to the present embodiment, the switching time of the observation light and the execution time of the treatment are automatically recorded. It is also possible to calculate the addition of medical fees.

  In addition, an enormous number of cases of endoscopy are performed daily, both in Japan and overseas, and big data can be generated by accumulating such data. The accumulated big data can be used for various statistical analyses, data mining, and the like.

(Embodiment 2)
In the first embodiment, an example has been described in which data relating to observation light switching, medicine spraying, and treatment execution is recorded in time series. In the second embodiment, in addition to them, an example in which data indicating the distance between the surface of an organ in a patient's body and the endoscope 11 and / or data indicating the magnification of the endoscope 11 is recorded in chronological order. explain.

  FIG. 11 is a diagram illustrating a configuration of the endoscope system 10 according to the second embodiment. The endoscope system 10 according to the second embodiment has a configuration in which an imaging distance estimation unit 123c and a magnification control unit 124 are added to the endoscope system 10 according to the first embodiment illustrated in FIG. Hereinafter, differences from the first embodiment will be described.

  The imaging distance estimating unit 123c estimates the imaging distance between the surface of the internal organ of the patient and the end of the endoscope 11 from the frame image included in the endoscope moving image. The imaging distance estimating unit 123c estimates the imaging distance using a method described in, for example, Japanese Patent No. 5028138. In the present embodiment, the estimated imaging distance is classified into “far”, “medium”, and “near”. It should be noted that the classification is not limited to three categories, and may be classified into two categories or four or more categories.

  The imaging distance estimation unit 123c estimates the imaging distance every N seconds or every N frames in synchronization with the image recognition processing of the medicine and the treatment tool by the medicine scattering detection unit 123a and the treatment execution detection unit 123b, and sends the estimation result to data. Hand over to section 125. As the estimation start timing of the imaging distance, any one of the start timing of recording of the endoscope moving image, the start timing of stopwatch measurement in the image recording device, and the timing of the first release can be used. In addition, any one of the timing of ending the recording of the endoscope moving image, the timing of ending the stopwatch measurement in the image recording device, and the timing of pressing the examination end button can be used as the timing of ending the estimation of the imaging distance.

  The magnification control unit 124 controls the magnification of the magnifying endoscope in accordance with a doctor's operation on the operation unit 113 when the magnifying endoscope having the magnifying observation function is mounted on the light source device 14. The magnification control unit 124 checks the magnification of the magnifying endoscope every N seconds or every N frames, and passes the confirmation result to the data transmission unit 125. The magnification control unit 124 may pass the zoom magnification as it is, or may pass it as a percentage value with 0% at the maximum wide angle and 100% at the maximum telephoto. Note that, instead of acquiring the zoom magnification from the magnifying endoscope, whether wide-angle observation or close-up magnification observation may be detected by image recognition.

  The data transmitting unit 125 includes imaging distance data indicating the imaging distance passed from the imaging distance estimating unit 123c, and the enlargement passed from the magnification control unit 124, in addition to the irradiation light type data, the medicine scatter data, and the treatment execution data. The magnification data indicating the magnification of the endoscope is transmitted to the endoscope inspection data recording device 20 via the network 2.

  FIG. 12 is a diagram showing a configuration of the endoscope inspection data recording device 20 according to the second embodiment. The endoscopic inspection data recording device 20 according to the second embodiment has an imaging distance data acquisition unit 221d and a magnification data acquisition unit 221e added to the endoscopic inspection data recording device 20 according to the first embodiment illustrated in FIG. Configuration. Hereinafter, differences from the first embodiment will be described.

  The imaging distance data acquisition unit 221d and the magnification data acquisition unit 221e acquire imaging distance data and magnification data from the endoscope system 10, respectively. The data recording unit 225 records the transition of the data acquired by the imaging distance data acquisition unit 221d and the magnification data acquisition unit 221e in the inspection data holding unit 232. In the present embodiment, the imaging distance data acquisition unit 221d and the magnification data acquisition unit 221e acquire time-series data of the imaging distance data and the magnification data, and the data recording unit 225 uses the acquired time-series data as it is or The data is processed and recorded in the inspection data holding unit 232.

  FIGS. 13A and 13B are diagrams illustrating an example of the imaging distance master table 231 d held in the master information holding unit 231 and an example of the test data 232 c recorded in the test data holding unit 232. In the imaging distance master table 231d shown in FIG. 13A, three categories of distance = 1, middle = 2, and near = 3 are defined as the imaging distance classification.

  The same table as the imaging distance master table 231d is also provided in the imaging distance estimating unit 123c of the endoscope system 10. The imaging distance estimating unit 123c estimates the imaging distance from the endoscope image at predetermined time intervals, converts the estimation result into a number with reference to the imaging distance master table, and passes the result to the data transmission unit 125.

  In the example illustrated in FIG. 13B, the data recording unit 225 includes the irradiation light type data acquisition unit 221a, the medicine scatter data acquisition unit 221b, the treatment execution data acquisition unit 221c, the imaging distance data acquisition unit 221d, and the magnification data acquisition unit 221e. This is an example in which time-series data of irradiation light type data, medicine scatter data, treatment execution data, imaging distance data, and magnification data acquired by the above are recorded in the inspection data holding unit 232 as they are.

  In this example, data on observation light, medicine spraying, treatment, imaging distance, and magnification are recorded in a time-series manner in units of one second from the start to the end of the examination. In the example shown in FIG. 13B, the imaging distance changes from “far” to “medium” at the time when 2 seconds have elapsed from the start of the examination (ie, the endoscope 11 approaches the organ surface), and at the time when 100 seconds have elapsed. Has changed from “medium” to “close” (that is, the endoscope 11 is closer to the organ surface). When the biopsy is started 100 seconds after the start of the examination, the magnification of the magnifying endoscope is 80%.

  As described above, according to the second embodiment, the following effects are obtained in addition to the effects of the first embodiment. By recording the imaging distance to the organ surface and the magnification when using the magnifying endoscope in a state that can be handled quantitatively and objectively, it is possible to evaluate and compare scrutiny such as range diagnosis and differential diagnosis.

  Generally, observations for finding a lesion tend to be observed in a distant view, and detailed observations after the discovery of a lesion tend to be observed in a near view. Therefore, it is possible to estimate whether observation is being performed for finding a lesion or detailed observation is being performed based on the imaging distance data. In addition, by accumulating the imaging distance data when the medicine is sprayed, when each treatment is performed, and when the narrow band light is used, the position of the endoscope 11 at the time of performing the processing can be stored in a database to derive a tendency.

  In an examination using a magnifying endoscope, generally, observation for finding a lesion tends to be observed at a wide angle, and detailed observation after finding a lesion tends to be observed in an enlarged state. Therefore, based on the magnification data of the magnifying endoscope, it can be estimated whether the observation for finding the lesion or the detailed observation is being performed. In recent years, a bifocal switching type magnifying endoscope that switches the focus by button operation has also been put into practical use. When such a model is used, it is not a continuous magnification change but a wide-angle or magnifying state. Observation. In addition, by accumulating magnification data of the magnifying endoscope at the time of spraying the medicine, performing each treatment, and using the narrow band light, the magnification of the magnifying endoscope at the time of performing them can be compiled into a database to derive a tendency. .

(Embodiment 3)
In the first embodiment, an example has been described in which data relating to observation light switching, medicine spraying, and treatment execution is recorded in chronological order. In the third embodiment, in addition to the above, an example in which data indicating an imaging state of an image captured by the endoscope 11 is recorded in chronological order will be described.

  FIG. 14 is a diagram showing a configuration of the endoscope system 10 according to the third embodiment. The endoscope system 10 according to the third embodiment has a configuration in which an imaging state classification unit 123d is added to the endoscope system 10 according to the first embodiment illustrated in FIG. Hereinafter, differences from the first embodiment will be described.

  The imaging state classification unit 123d classifies the imaging state of each endoscope image captured in time series by the endoscope 11. Specifically, it is classified whether or not the image is an endoscope image in a state where the surface of the organ in the body cavity of the patient is appropriately imaged. That is, the image is classified into an image in which the organ in the body cavity can be sufficiently observed and an image in which the organ in the body cavity cannot be sufficiently observed. Hereinafter, the former is referred to as category A, and the latter as category B. The images of Class B include images containing halation, color shift, red balls (close proximity to the large intestine), a large amount of bubbles, residues, liquids, defocus (blurring), blurring, water supply, and the like. As a method of classifying the imaging state of the endoscope image, for example, the methods described in Japanese Patent No. 4615963, Japanese Patent No. 4624841, and Japanese Patent No. 5530406 can be used. Using these methods, an endoscope image corresponding to Class B is detected, and an endoscope image not corresponding to Class B is classified into Class A.

  The imaging state classification unit 123d classifies the imaging state of the endoscope image every N seconds or every N frames in synchronization with the image recognition processing of the medicine and the treatment tool by the medicine scattering detection unit 123a and the treatment execution detection unit 123b. The classification result is passed to the data sending unit 125.

  The data transmission unit 125 transmits the imaging state data passed from the imaging state classification unit 123d in addition to the irradiation light type data, the medicine scattering data, and the treatment execution data to the endoscope inspection data recording device 20 via the network 2. To send to.

  FIG. 15 is a diagram showing a configuration of the endoscope inspection data recording device 20 according to the third embodiment. The endoscope inspection data recording device 20 according to the third embodiment has a configuration in which an imaging state data acquisition unit 221f is added to the endoscope inspection data recording device 20 according to the first embodiment illustrated in FIG. Hereinafter, differences from the first embodiment will be described.

  The imaging state data acquisition unit 221f acquires imaging state data from the endoscope system 10. The data recording unit 225 records the transition of the imaging state data acquired by the imaging state data acquisition unit 221f in the inspection data holding unit 232. In the present embodiment, the imaging state data acquisition unit 221f acquires the time series data of the imaging state data, and the data recording unit 225 records the acquired time series data as it is or after processing it in the inspection data holding unit 232.

  FIG. 16 is a diagram illustrating an example of the imaging state master table 231f held in the master information holding unit 231. In the example illustrated in FIG. 16, the “image in which the internal organs of the body cavity can be sufficiently observed” of the classification A is further subdivided into two types: good observation = 0 and observable = 1. In addition, classification B “images in which internal organs in the body cavity cannot be observed sufficiently” further includes residue, liquid, water supply = 2, halation = 3, color shift = 4, defocus (blurring), blur = 5, and judgment impossible = 6. It is subdivided into five types. Therefore, the imaging state is classified into seven categories in total. Residues reflect the degree of success of pretreatment such as intestinal lavage, and are sometimes regarded as important in the evaluation of lower endoscopy quality. It may be classified.

  The same table as the imaging state master table 231f is also provided in the imaging state classification unit 123d of the endoscope system 10. The imaging state classification unit 123d detects the imaging state of the endoscope image at predetermined time intervals, converts the detection result into a number with reference to the imaging state master table, and passes the result to the data transmission unit 125.

  FIG. 17 is a diagram illustrating an example of the test data 232 c recorded in the test data holding unit 232. In the example illustrated in FIG. 17, the data recording unit 225 includes the irradiation light type data acquired by the irradiation light type data acquisition unit 221a, the medicine scattering data acquisition unit 221b, the treatment execution data acquisition unit 221c, and the imaging state data acquisition unit 221f. This is an example in which time-series data of medicine spray data, treatment execution data, and imaging state data are recorded in the test data holding unit 232 as they are.

  In this example, data on the observation light, the application of the medicine, the treatment, and the imaging state are recorded in a time-series manner in units of one second from the start to the end of the examination. In the example shown in FIG. 17, out-of-focus or blurring is detected at the start of the inspection, color misregistration is detected at the lapse of one second from the start of the inspection, and good observation is obtained at the lapse of two seconds after the start of the inspection. While the endoscope 11 moves from the mouth to the esophagus immediately after the start of the examination, the captured image is likely to be blurred. Further, water is supplied at the point of time 700 seconds after the start of the inspection, and the imaging state is such that sufficient observation cannot be performed.

(Application Example of Third Embodiment)
In the above description according to Embodiment 3, the imaging state classification unit 123d classifies the imaging states by detecting a static imaging state from a single frame image. In an application example described below, a dynamic imaging state is detected based on a change between frame images to classify the imaging state.

  The imaging state classifying unit 123d calculates an evaluation value based on a change between frame images in an endoscope image captured in time series. An evaluation value based on a change between frame images can be calculated by, for example, a method described in Japanese Patent No. 5147308. Hereinafter, the calculated evaluation value is referred to as a change evaluation value P.

  The calculation of the change evaluation value P is executed every N seconds or every N frames in synchronization with the image recognition processing of the medicine and the treatment tool by the medicine scattering detection unit 123a and the treatment execution detection unit 123b. At this time, the change evaluation value P may be calculated based on the change between the current frame image and the frame image N seconds or N frames in the past, or may be calculated based on the change between the current frame image and the immediately preceding frame image. The evaluation value P may be calculated. The imaging state classification unit 123d detects and classifies a dynamic imaging state based on the calculated change evaluation value P.

  The change evaluation value P takes a smaller value as the change between frame images is smaller, indicating that the movement of the endoscope 11 is smaller. When the doctor stops moving the endoscope 11 and observes a specific part, the change evaluation value P takes a value close to zero.

  The imaging state classification unit 123d synchronizes with the image recognition processing of the medicine and the treatment tool by the medicine scattering detection unit 123a and the treatment execution detection unit 123b, and performs static and dynamic endoscope images every N seconds or every N frames. The imaging states are detected and classified into categories, and the classification results are passed to the data transmission unit 125. The data transmission unit 125 transmits, via the network 2, the static imaging state data and the dynamic imaging state data passed from the imaging state classification unit 123d, in addition to the irradiation light type data, the medicine scattering data, and the treatment execution data. The data is sent to the endoscope inspection data recording device 20.

  The imaging state data acquisition unit 221f acquires static imaging state data and dynamic imaging state data from the endoscope system 10. The data recording unit 225 records the static imaging state data acquired by the imaging state data acquiring unit 221f and the transition of the imaging state data in the inspection data holding unit 232. In the present embodiment, the imaging state data acquisition unit 221f acquires time-series data of static imaging state data and dynamic imaging state data, and the data recording unit 225 processes the acquired time-series data as it is or as it is. It is recorded in the inspection data holding unit 232.

  FIG. 18 is a diagram illustrating an example of the change evaluation value / imaging state master table 231g held in the master information holding unit 231. In the example illustrated in FIG. 18, the image classified into the classification A in the above-described static imaging state classification processing is subjected to the dynamic imaging state classification processing, and is classified into the classification B in the static imaging state classification processing. The classified images are not subjected to the dynamic imaging state classification processing.

  Among the “images capable of sufficiently observing the internal organs of the body cavity” classified into the class A, an image (class Aa) in which the image evaluation value P is in the range of 0.0 ≦ P <0.3 has a dynamic imaging state. Good observation = 0. Images (classification Ab) in which the image evaluation value P is in the range of 0.3 ≦ P <0.8 are classified as 1 in which the dynamic imaging state is observable. The image (classification Ac) in which the image evaluation value P is in the range of 0.8 ≦ P ≦ 1.0 is classified into the dynamic imaging state of inappropriate observation = 2.

  The same table as the change evaluation value / imaging state master table 231g is also provided in the imaging state classification unit 123d of the endoscope system 10. The imaging state classifying unit 123d calculates the change evaluation value P between the endoscope images at predetermined time intervals, converts the evaluation value P into a number with reference to the change evaluation value / imaging state master table, and passes the number to the data transmission unit 125. Note that the calculation of the change evaluation value P may be skipped for an endoscope image in which a static imaging state is classified into Class A.

  FIG. 19 is a diagram illustrating an example of the test data 232d recorded in the test data holding unit 232. In the example illustrated in FIG. 19, the data recording unit 225 performs irradiation light type data acquisition by the irradiation light type data acquisition unit 221a, the medicine scattering data acquisition unit 221b, the treatment execution data acquisition unit 221c, and the imaging state data acquisition unit 221f. This is an example in which time-series data of medicine spray data, treatment execution data, static imaging state data, change evaluation value P, and dynamic imaging state data are recorded in the inspection data holding unit 232 as they are. In this example, the change evaluation value P and the dynamic imaging state data are not recorded for the endoscope image in which the static imaging state is classified into the class B. For the mirror image, the change evaluation value P and the dynamic imaging state data may be calculated, classified, and recorded.

  In the example shown in FIG. 19, data on observation light, medicine spraying, treatment, a static imaging state, a change evaluation value, and a dynamic imaging state are recorded in a time-series manner in units of one second from the start of the examination to the end of the examination. . In the example shown in FIG. 19, the endoscope 11 has been moved at the time when 4 seconds and 1300 seconds have passed since the start of the examination, and the observation is inappropriate.

  FIG. 20 is a flowchart for explaining an operation example of the endoscope system 10 according to the application example of the third embodiment. When the inspection start / end detection unit (not shown) of the endoscope system 10 detects the start of the endoscope inspection (Y in S10), the detection time is notified to the data transmission unit 125. Thereafter, when the insertion state detection unit (not shown) detects the insertion of the endoscope 11 into the body cavity (Y in S11), the detection time is passed to the data transmission unit 125.

  The image recognition unit 123 acquires an endoscope image captured by the endoscope 11 (S12). The imaging state classification unit 123d detects and classifies a static imaging state of the obtained endoscope image, and passes the classification result to the data transmission unit 125 (S121). The imaging state classification unit 123d calculates the change evaluation value P based on the difference between the currently obtained endoscope image and the previously obtained endoscope image, and passes the change evaluation value P to the data transmission unit 125. (S122). The imaging state classification unit 123d detects and classifies the dynamic imaging state of the endoscope image acquired this time based on the calculated change evaluation value P, and passes the classification result to the data transmission unit 125 (S123). .

  The medicine scattering detection unit 123a searches for a medicine in the endoscope image acquired this time, and the treatment execution detection unit 123b searches for a treatment tool in the endoscope image acquired this time (S13). The medicine scattering detection unit 123a and the treatment execution detection unit 123b pass the search result to the data transmission unit 125. The irradiation light switching control unit 122 checks the type of the observation light (S14). The irradiation light switching control unit 122 passes the confirmation result to the data transmission unit 125. The data sending unit 125 temporarily stores data relating to the observation light, the medicine, the treatment, the static imaging state, the change evaluation value P, and the dynamic imaging state in a buffer (not shown) (S15).

  When N seconds have elapsed from the current acquisition of the endoscope image (Y in S17), the insertion state detecting unit (not shown) in the body cavity does not detect the removal of the endoscope 11 out of the body cavity (N in S16). ), Returning to step S12 to acquire a new endoscope image (S12). When the insertion state detection unit (not shown) in the body cavity detects that the endoscope 11 has been removed from the body cavity (Y in S16), the detection time is passed to the data transmission unit 125. Thereafter, when the inspection start / end detection unit (not shown) detects the end of the endoscopic inspection (Y in S18), the detection time is notified to the data transmission unit 125. The data transmission unit 125 transmits the time-series data stored in the buffer (not shown) to the endoscope inspection data recording device 20 (S19).

  Note that the processes in steps S122 and S123 can be skipped when the currently acquired endoscope image is classified into the category B in the static imaging state classification process in step S121.

  Further, even when the change evaluation value P and the dynamic imaging state data are generated for the endoscope image classified into the class B, it may be excluded from transmission targets in the transmission processing of the time-series data in step S19. .

  As described above, according to the third embodiment, the following effects are obtained in addition to the effects of the first embodiment. By classifying and recording the imaging state of each of the plurality of endoscopic images taken in time series, it is determined whether or not the organ surface or the like is appropriately imaged (that is, the organ surface can be sufficiently observed. State information such as whether or not there is) can be recorded quantitatively and objectively. Further, the movement of the endoscope 11 can be recorded by calculating and recording the change evaluation value between adjacent frame images. For example, it can be inferred from the transition of the change evaluation value that the endoscope 11 is performing the insertion operation (moving), that the doctor is observing the organ surface, and the like.

  The recording of the imaging state does not presuppose the preservation of the endoscope image, but the imaging state of the endoscope moving image section that is not stored and the endoscope image that is discarded are also recorded. The recording of the time-series data of the imaging state is mainly intended to record the handling of the endoscope 11 by the doctor during the endoscopy in a time-series manner, and the quality of the stored endoscope image. It is not intended to record

(Embodiment 4)
In the fourth embodiment, an example of a method of using the endoscopic inspection data stored in the endoscopic inspection data recording device 20 will be described.

  FIG. 21 is a diagram showing a configuration of an endoscope inspection data recording device 20 according to the fourth embodiment. The endoscopy data recording device 20 according to the fourth embodiment has a data analysis unit 224a and a statistical data holding unit 233 added to the endoscopy data recording device 20 according to the third embodiment shown in FIG. Configuration. Hereinafter, processing according to the fourth embodiment will be described.

  The (a) examination time per examination in the endoscopy is (b) the time from the start of the examination until the endoscope 11 is inserted into the body cavity of the patient, (c) the non-observation time, and (d) the normal time. The observation time, (e) detailed observation time, (f) treatment time, and (g) the total time from the removal of the endoscope 11 to the outside of the patient's body cavity until the end of the examination can be defined.

  (C) The non-observation time refers to a time during which a doctor cannot perform sufficient observation due to pulsation of a patient's organ, expiration, or image disturbance (halation or the like) due to contraction of an observation site. (D) The normal observation time refers to a time for observing the presence or absence of a lesion from a relatively long observation distance while moving the observation region relatively slowly or with the movement of the scope stopped. Except for those times categorized as). (E) The detailed observation time refers to a time for observing carefully when a lesion such as a close-up observation, an NBI observation, an enlargement observation, or a medicine spraying is found. (F) Treatment time refers to the time during which various procedures such as biopsy, stent placement / removal, and EMR are performed.

  The data calculation unit 224 calculates the examination time (a) from the examination start time and the examination end time of the endoscopic examination to be held in the examination data holding unit 232. Further, the data calculation unit 224 determines (b) the time from the start of the examination to the insertion of the endoscope 11 into the body cavity of the patient with the examination start time of the endoscope examination held in the examination data holding unit 232 and the Calculated from the endoscope insertion time.

  Further, the data calculation unit 224 calculates (c) the non-observation time based on the imaging state data of the endoscope inspection held in the inspection data holding unit 232. For example, in the static imaging state classification processing, “an image in which the internal organs of the body cavity cannot be sufficiently observed” = the endoscope image classified into the classification B, and the observation processing is inappropriately classified in the dynamic imaging state classification processing. The total time of the section in which the endoscope image is captured is defined as (c) non-observation time.

  Further, the data calculation unit 224 uses (e) the detailed observation time, the irradiation time of the special light in the endoscopic examination held in the examination data holding unit 232, the enlarged observation time using the magnifying endoscope, and the medicine. It is calculated based on the observation time etc. Further, the data calculation unit 224 calculates (f) the treatment time based on the execution time of the treatment performed in the endoscopic examination held in the examination data holding unit 232. The data calculation unit 224 also sets (g) the time from the removal of the endoscope 11 to the outside of the patient's body cavity until the end of the examination, by using the endoscope removal time of the endoscope examination held in the examination data holding unit 232. And the inspection end time.

  Further, the data calculation unit 224 calculates (d) the normal observation time, (a) the examination time, (b) the time from the start of the examination until the endoscope 11 is inserted into the body cavity of the patient, and (c) the non-observation time. (D) normal observation time, (e) detailed observation time, (f) treatment time, (g) by subtracting the total of the time from the removal of the endoscope 11 to the outside of the patient's body cavity until the end of the examination. calculate.

  The data calculation unit 224 calculates the (a) examination time, (b) the time from the start of the examination until the endoscope 11 is inserted into the body cavity of the patient, (c) the non-observation time, and (d) the normal observation time. , (E) the detailed observation time, (f) the treatment time, and (g) the time from the removal of the endoscope 11 to the outside of the patient's body cavity until the end of the examination, is recorded in the examination data holding unit 232.

  The data analysis unit 224a performs a statistical analysis based on the data recorded in the test data storage unit 232, and records the generated statistical data in the statistical data storage unit 233.

  FIG. 22 is a diagram illustrating an example of the statistical data 233a recorded in the statistical data holding unit 233. In this example, the data analysis unit 224a classifies the inspection data of the endoscopy by the inspection item, site, and lesion type, and (c) non-observation time, (d) normal observation time, (e) details The average value of each of the observation time and (f) the treatment time is calculated.

  In the example shown in FIG. 22, (c) non-observation time, (d) normal observation time, and (e) of endoscopic examination classified into inspection item = upper endoscopy, observation site = stomach, lesion = polyp (C) Non-observation time of endoscopy classified as detailed observation time, (f) average treatment time, examination item = lower endoscopy, observation site = sigmoid colon, lesion = adenoma, The average time of (d) the normal observation time, (e) the detailed observation time, and (f) the treatment time is calculated. Inspection item = Upper endoscopy, Observation site = Stomach, Lesion = Lesion detection rate of endoscopy classified into polyps, and Inspection item = Lower endoscopy, Observation site = Sigmoid colon, Lesion = Lesion detection rates of endoscopy classified as adenoma are calculated respectively.

  By generating such statistical data, it is possible to investigate the characteristics of the endoscopy for each classification of the endoscopy. In addition, the difficulty level of the operation and the procedure of the endoscope 11 can be quantified for each classification of the endoscope examination, which can be used for creating an examination protocol and improving the procedure. In addition, it is possible to examine and investigate which time is more important for finding a lesion.

  FIG. 23 is a flowchart for explaining an operation example of the endoscope inspection data recording device 20 according to the fourth embodiment. The data calculation unit 224 reads various data of the endoscope inspection from the inspection data holding unit 232 (S2240). The data calculation unit 224 calculates various required times ((a) examination time, (b) time from the start of the examination to insertion of the endoscope 11 into the body cavity of the patient, (c) The non-observation time, (e) the detailed observation time, (f) the treatment time, and (g) the time from the removal of the endoscope 11 to the outside of the patient's body cavity until the end of the examination) are calculated (S2241).

  The data calculation unit 224 calculates (a) examination time − ｛(b) time from the start of examination to insertion of the endoscope 11 into the body cavity of the patient + (c) non-observation time + (e) detailed observation time + (F) The treatment time, (g) The time｝ from the removal of the endoscope 11 to the outside of the patient's body cavity until the end of the examination is calculated, and (d) the normal observation time is calculated (S2242). The data calculation unit 224 stores the calculated various required times in the inspection data storage unit 232 (S2243). The data calculation unit 224 basically executes the above-described processing for inspection data of all endoscopy.

  The data analysis unit 224a reads various required times from the examination data holding unit 232 for each classification of the endoscopic examination (S2244). The data calculation unit 224 calculates the statistical values (for example, the average value and the median value) of the read various required times (S2245). The data analysis unit 224a stores the calculated statistical values of the various required times in the statistical data holding unit 233 (S2246). The data analysis unit 224a executes the above processing for the specified classification of the endoscopy.

  As described above, according to the fourth embodiment, since the data indicating the state at each time point in the endoscope inspection is recorded in chronological order, it is possible to accurately calculate various required times in the endoscope inspection. Therefore, the process of the endoscopy can be objectively evaluated, and can be used for finding a doctor's habits and points of improvement. Further, since data of the same specification is collected in a plurality of endoscopy, objective statistical data in the endoscopy can be calculated. Therefore, it can be utilized for creating and improving the inspection protocol. It can also be used as academic data.

(Embodiment 5)
In the fifth embodiment, a method of adding part information to an endoscope image captured by the endoscope 11 will be described.

  FIG. 24 is a diagram showing a configuration of the endoscope inspection data recording device 20 according to the fifth embodiment. The endoscopic examination data recording device 20 according to the fifth embodiment is different from the endoscopic examination data recording device 20 according to the first embodiment shown in FIG. 3 in that an image acquisition unit 221g and a site information adding unit 226 are added. Configuration. Hereinafter, processing according to the fifth embodiment will be described.

  In the first to fourth embodiments, the process of transferring the endoscope image captured by the endoscope 11 from the endoscope system 10 to the endoscope inspection data recording device 20 is not essential. Then, the transfer of the endoscope image is indispensable. The endoscope image to be transferred is a still image captured by a doctor releasing a shutter or operating a foot switch to release a shutter. The still image is an image for storage, and among those images, an image selected by the doctor is attached to the report. Note that transfer of the endoscope moving image from the endoscope system 10 to the endoscope inspection data recording device 20 is not essential.

  The image acquisition unit 221g acquires, from the endoscope system 10, a plurality of endoscope images captured by the endoscope 11 by a doctor's operation. The site information providing unit 226 provides site information (for example, duodenum, stomach, and esophagus) to the acquired endoscope image. The site information can be provided, for example, by the following method.

  The site information providing unit 226 provides site information manually input by a doctor or a nurse. Specifically, the display control unit 223 causes the display device 13 of the endoscope system 10 or the display unit 34 of the terminal device 30 to display the endoscopic image acquired by the image acquiring unit 221g during or after the examination. The doctor or the nurse manually inputs, into the displayed endoscope image, which part of the displayed endoscope image is an image. The operation receiving unit 222 of the endoscope inspection data recording device 20 receives the site information input by the doctor or the nurse, and passes the site information to the site information providing unit 226.

  Further, the part information providing unit 226 can also specify a part by image recognition from the endoscope image. For example, the part is specified by matching the acquired endoscope image with the pattern data of each part. Note that a doctor or a nurse manually inputs an endoscope image for which a part cannot be specified by image recognition.

  Further, the part information adding unit 226 can also add the part information to the endoscopic image according to the imaging order of the part specified in the examination protocol. The inspection protocol is a procedure in which a site to be observed, precautions at the time of observation, conditions of a still image to be recorded, and the like are ordered and used, and are used when performing a routine inspection. For example, in upper endoscopy, a procedure is used in which the endoscope 11 is first inserted into the duodenum, and observation is performed so that the endoscope 11 is returned in the direction of the esophagus.

  FIGS. 25A and 25B are diagrams illustrating an example of a part specifying screen 50 for adding part information to an endoscopic image in the order of imaging. The display control unit 223 of the endoscope inspection data recording device 20 displays the site specifying screen 50 on the display device 13 of the endoscope system 10 or the display unit 34 of the terminal device 30. The part specifying screen 50 shown in FIG. 25A is a screen before the part information is added to the endoscope image. In the examination protocol of the upper endoscopy shown in this example, (1) duodenum, (2) duodenum, (3) stomach (pylorus), (4) stomach (gastric body), (5) stomach (corneal corner) ), (6) stomach (cardia), (7) esophagus (lower), (8) esophagus (middle), (9) esophagus (upper), (10) throat, It is prescribed to shoot.

  The duodenum is set in the region information column 51a of the first endoscope image column 51, and the stomach (gastric body) is set in the region information column 54a of the fourth endoscope image column 54. ,... The throat is set in the region information column 510a of the tenth endoscope image column 510.

  The doctor inserts the endoscope 11 deep into the duodenum in accordance with the inspection protocol of the upper endoscopy, and sequentially captures images of the specified parts. As shown in FIG. 25B, the first captured image is assigned to the first endoscopic image column 51. When the doctor presses the OK button 50a, the image assignment is determined. If the part shown in the assigned endoscope image does not match the part information in the assigned column, the assignment can be canceled by pressing the cancel button 50b.

  In the example shown in FIG. 25B, the doctor skips imaging of the stomach (corneal stomach). For example, at the stage where the third endoscopic image of the stomach (pylorus) is allocated (the stage where the endoscope image of the fourth stomach (corneal stomach) is allocated), the OK button 50a is pressed. When pressed, assignment of an endoscopic image of the stomach (corneal corner) can be skipped. According to the imaging order of the part defined in the inspection protocol described above, the processing of adding the part information to the endoscope image captured by the endoscope 11 may be performed simultaneously and concurrently during the endoscopic inspection. And may be performed after the inspection.

  FIG. 26 is a flowchart for explaining an operation example of the endoscope inspection data recording device 20 according to the fifth embodiment. This operation example is an example in which site information is added to an endoscope image actually shot in accordance with the imaging order of the site defined by the examination protocol. The image acquisition unit 221g acquires the endoscopic images photographed in the endoscopy from the endoscope system 10 in the photographing order (S2260). The part information adding unit 226 adds part information to the endoscopic image acquired from the endoscope system 10 in accordance with the imaging order of the part specified by the examination protocol (S2261). The part information adding unit 226 transfers the endoscope image to which the part information has been added to an image recording device (not shown) via the network 2 (S2262). When an endoscope image is also stored in the endoscope inspection data recording device 20, the image is stored in an image holding unit (not shown) in the storage unit 23.

  As described above, according to the fifth embodiment, it is possible to easily add the part information to the storage endoscope image captured in the endoscopic examination. In particular, the method of adding the region information to the endoscopic image in accordance with the imaging order of the region defined by the examination protocol also motivates the doctor to consciously adhere to the examination protocol. Since there is no need to manually input site information, complicated work by doctors and nurses is not required. Also, the method of detecting a part from an endoscope image by image recognition processing may cause erroneous recognition of the part, but in the method of adding part information to the endoscope image according to the imaging order, the doctor adheres to the imaging order. Erroneous site information will not be added.

(Embodiment 6)
The data calculation unit 224 of the endoscope inspection data recording device 20 according to Embodiment 1 uses the time of use of each irradiation light, the number of times of use of each irradiation light, and each medicine based on various time-series data. The observation time, the number of times each drug was used, the execution time of each treatment, and the number of executions of each treatment were calculated. The time-series data used as a basis for this calculation is data of the entire period of the target endoscopic examination (the period from the examination start time to the examination end time).

  In the fifth embodiment, the data calculation unit 224 determines the use time of each irradiation light, the number of use of each irradiation light, the observation time using each medicine, and the time of each medicine between two designated points in the endoscopy. Is calculated, at least one of the number of times of use, the execution time of each treatment, and the number of times of each treatment is calculated. The two points in the endoscope inspection can be designated by time, detection information, a captured image, and the like.

  FIG. 27 is a diagram illustrating an example of a designation screen 60 for two points in an endoscopy. The display control unit 223 of the endoscope inspection data recording device 20 causes the display device 13 of the endoscope system 10 or the display unit 34 of the terminal device 30 to display the two-point designation screen 60. The designation screen 60 includes an arbitrary button 61, an observation light button 62, a medicine button 63, and a treatment button 64.

  The arbitrary button 61 is a button for designating two points by time. The observation light button 62 is a button for selecting the type of observation light and designating the use start time and use end time of the selected observation light as two points. The medicine button 63 is a button for selecting a medicine type and designating an observation start time and an observation end time using the selected medicine as two points. The treatment button 64 is a button for selecting a treatment type and designating the execution start time and the execution end time of the selected treatment as two points.

  FIG. 27 illustrates a state in which the medicine button 63 is pressed by a doctor or a nurse, and a medicine selection window 63a is displayed. For example, when Lugol is selected, the observation start time and the observation end time in a state where Lugol is scattered are designated as two points.

  Two points in the endoscopy can also be designated by selecting two pieces of part information. As described in the fifth embodiment, when site information is added to an endoscope image, two points can be designated by selecting two endoscope images to which site information has been added.

  FIG. 28 is a diagram illustrating an example of the region selection screen 70 in the endoscopic examination. The display control unit 223 of the endoscope inspection data recording device 20 displays the inter-part selection screen 70 on the display device 13 of the endoscope system 10 or the display unit 34 of the terminal device 30. The inter-part selection screen 70 has a first area 71 and a second area 72.

  The first area 71 is an area for selecting between two parts by selecting two part names. FIG. 28 illustrates an example in which the stomach (pylorus) is selected from the pull-down menu of the first selection field 71a and the stomach (cardia) is selected from the pull-down menu of the second selection field 71b.

  The second area 72 is an area in which two endoscopic images are selected from a plurality of endoscopic images to which the part information is added, two parts are selected, and a part is selected. FIG. 28 illustrates an example in which the third captured endoscopic image 72a of the stomach (pylorus) and the sixth captured endoscopic image 72b of the stomach (cardia) are selected.

  As described above, according to the sixth embodiment, the observation time and the like between any two points in the endoscope inspection can be calculated, so that the detailed analysis and evaluation of the endoscope inspection can be performed. For example, in observing the stomach, the observation time for each site such as the cardia, stomach body, stomach corner, etc. can be calculated, so that detailed analysis and evaluation are possible. For example, it can be analyzed whether the observation procedure is performed according to a standard time according to the inspection protocol.

(Embodiment 7)
If an inexperienced unskilled physician performs an endoscopy, the skilled physician will also provide guidance after the examination, and make sure that no findings are overlooked, that there is no diagnosis error, that imaging / examination is performed correctly, etc. It is common to perform a check (double check). For this purpose, a skilled doctor must be present at the examination of an unskilled doctor, but in an actual medical practice, it is difficult and inefficient that a skilled doctor is always present. Therefore, the inspection status of all the inspection rooms is collectively displayed in a monitor room or the like. In this case, a moving image captured by the endoscope 11 is often displayed in real time. Even if information of a plurality of examination rooms is displayed in parallel, it is difficult to grasp all the information in detail. It is also important to evaluate the process during the examination from the viewpoint of improving the skill of the endoscopy of the unskilled doctor.

  Therefore, in the seventh embodiment, data indicating the state at each time point of the endoscope inspection is recorded and displayed on a monitor in real time. As a result, the evaluation (double check) and guidance of the unskilled doctor can be efficiently supported, and the skill of the unskilled doctor can be improved.

  In the seventh embodiment, the data transmission unit 125 of the endoscope system 10 includes the irradiation light type data passed from the irradiation light switching control unit 122, the medicine dispersion data passed from the medicine dispersion detection unit 123a, and the treatment execution detection unit. The procedure execution data passed from 123b is transmitted to the endoscope inspection data recording device 20 via the network 2 in real time. That is, the passed data is sequentially transmitted without storing the data in a buffer (not shown).

  As described above, the transition of the data acquired by the irradiation light type data acquisition unit 221a, the medicine scatter data acquisition unit 221b, and the treatment execution data acquisition unit 221c of the endoscope inspection data recording device 20 is changed by the data recording unit 225. It is recorded in the inspection data holding unit 232. At the same time, the transition of the data is transferred to the terminal device 30 by the display control unit 223 and displayed on the screen of the display unit 34. The display control unit 223 may cause the image captured by the endoscopy to be displayed together with the transition of the data indicating the state at each time point of the endoscopy on the screen. The endoscope moving image may be continuously displayed in real time, or a still image captured by a doctor's operation may be displayed. Note that the same screen as the screen may be displayed on the display device 13 of the endoscope system 10. In this case, the doctor who is performing the examination can see the screen.

  FIG. 29 is a diagram illustrating a first state (immediately after the start of the examination) of a screen example in which the transition of the endoscope examination is represented by data. FIG. 30 is a diagram illustrating a second state (after a lapse of 5 minutes from the start of the examination) of a screen example in which the transition of the endoscope examination is represented by data. FIG. 31 is a diagram illustrating a third state (11 minutes after the start of the examination) of a screen example in which the transition of the endoscope examination is represented by data. FIG. 32 is a diagram illustrating a fourth state (at the end of the examination) of a screen example in which the transition of the endoscope examination is represented by data.

  On the screens 34a to 34d shown in FIGS. 29 to 32, the type of observation light in use is expressed by a pattern. Note that the type of observation light may be represented by color coding. When an event occurs, such as when the release is released or when the medicine is sprayed, the event content and time information are displayed. Further, an endoscope image taken with the release released is displayed at a position corresponding to the time when the release is released. The data indicating the state at each time point of the endoscopic examination as described above is added to the screen as the endoscopic examination progresses.

  Note that an inspection protocol (insertion method, observation site, observation method, imaging timing, etc.) may be displayed on the screens 34a to 34d so that it is possible to confirm whether the endoscopic inspection is performed according to the inspection protocol.

  When the endoscopy is performed simultaneously and in parallel in a plurality of examination rooms, data indicating the transition of each endoscopy may be displayed in a divided screen. Alternatively, a plurality of monitors may be prepared and displayed on each monitor.

  As described above, according to the seventh embodiment, in addition to recording the time-series data indicating the transition of the endoscopic examination, how the examination is performed by displaying the data on the monitor over time in real time Can be monitored, and the state of compliance with the inspection protocol, the degree of progress over time, and the like can be confirmed. Therefore, it is possible to prevent mistakes and grasp the progress and efficiency of the inspection. Also, when displayed on the display device 13 of the endoscope system 10, the doctor performing the examination can confirm the process and can also perform self-evaluation.

(Embodiment 8)
Embodiment 8 proposes a mechanism for a doctor to perform a test in compliance with a test protocol. As described in the fifth embodiment, in a medical facility that defines an examination protocol that defines an examination procedure, a doctor is required to perform an examination according to the examination protocol. Therefore, in the eighth embodiment, a display device different from the display device 13 that displays the in-vivo image of the patient input from the endoscope processing device 12 in real time is prepared in the examination room. A plurality of prescribed tasks are displayed, and the physician provides guidance support for performing an examination according to the examination protocol.

  FIG. 33 is a diagram illustrating a configuration of the endoscope system 10 according to the eighth embodiment. The endoscope system 10 includes an endoscope 11, an endoscope processing device 12, a display device 13, a light source device 14, and a display device 15.

  During the examination, the display device 13 displays an image of the inside of the patient's body imaged by the endoscope 11 in real time, and the doctor performs the examination while looking at the image displayed on the display device 13. In the eighth embodiment, the display device 15 is arranged in the examination room, and presents the examination protocol of the endoscopy to the doctor. Therefore, the display device 15 is preferably arranged near the display device 13 so that the doctor can view the in-vivo image and the examination protocol at the same time or without greatly changing the line of sight. Although the display device 13 and the display device 15 are shown as separate display devices in FIG. 33, a single display device may be used. In this case, the inspection protocol is displayed in a partial display area of the display device 13. In particular, if the display device 13 has a large screen display, it is preferable to display the inspection protocol in a partial display area of the display device 13.

  In the eighth embodiment, the endoscope processing device 12 is provided with the examination protocol screen from the endoscopic examination data recording device 20 and causes the display device 15 to display the examination protocol screen. A plurality of tasks are listed on the examination protocol screen in the order of execution. A task is an action that a physician should perform during an endoscopy, and constitutes an element of an examination protocol. When the doctor performs the task, information indicating that the task has been performed is transmitted to and recorded in the endoscopy data recording device 20, and the examination protocol screen allows the doctor to understand that the task has been performed. Is changed to the display mode.

  Basically, tasks are presented to a physician in the form of text that defines where and what to do in the patient's body. In order to express “where”, the task includes information indicating a specific part in the body. In order to express "what to do", the task includes information indicating an action to be performed by the doctor. Actions to be performed by the doctor are classified into a plurality of categories. In the eighth embodiment, the actions are classified into operation, observation, and imaging categories. In the eighth embodiment, the categories of operation, observation, and photographing are set, but other categories such as treatment and execution may be set. Each task serving as an element of the inspection protocol is associated with attribute information indicating that the task belongs to one category.

  The tasks belonging to the operation category are tasks related to the operation behavior of the endoscope 11, and define an insertion operation into a specific part of the endoscope 11, a bending operation and an air supply operation at the specific part. The task belonging to the observation category is a task related to a doctor's observation behavior, and determines observation of a specific part. The task belonging to the imaging category is a task related to the imaging action of the doctor, and determines imaging of a specific part. Here, the specific part means a part determined by the part information included in the task. Hereinafter, the endoscopic examination data recording apparatus 20 will be described as having an examination protocol screen generation function. However, another server in the endoscope operation support system 1 may have an examination protocol screen generation function. .

  FIG. 34 is a diagram showing a configuration of the endoscope inspection data recording device 20 according to the eighth embodiment. In the endoscope inspection data recording device 20 according to the eighth embodiment, the control unit 22 includes a data acquisition unit 221, an operation reception unit 222, a display control unit 223, a data recording unit 225, an execution task detection unit 240, an unexecuted task It comprises a detection unit 241, a situation change detection unit 242, and a display setting unit 243. The data acquisition unit 221 includes an irradiation light type data acquisition unit 221a, an image acquisition unit 221g, a patient data acquisition unit 221h, and a completion data acquisition unit 221i. The storage unit 23 includes an inspection data holding unit 232, an inspection protocol holding unit 234, and a setting information holding unit 235. The examination protocol holding unit 234 stores a plurality of tasks to be performed by a doctor in an endoscopic examination.

  Before the start of the examination, the endoscope processing device 12 transmits a display request for the examination protocol screen to the endoscopic examination data recording device 20. Upon receiving the display request, the endoscope inspection data recording device 20 generates an inspection protocol screen and transmits it to the endoscope processing device 12. At the start of the examination, the endoscope examination data recording device 20 may generate an examination protocol screen and transmit a display instruction of the examination protocol screen to the endoscope processing device 12. In this case, when acquiring the examination start detection data, the endoscopic examination data recording apparatus 20 may generate an examination protocol screen and transmit the examination protocol screen to the endoscope processing apparatus 12 together with a display instruction. Further, when acquiring the examination start detection data from the endoscope processing apparatus 12, the endoscope examination data recording apparatus 20 transmits a display instruction of an examination protocol screen to the endoscope processing apparatus 12, and receives the instruction. When the endoscope processing device 12 transmits a display request to the endoscope inspection data recording device 20, the endoscope inspection data recording device 20 generates an inspection protocol screen and transmits it to the endoscope processing device 12. You may. The endoscope processing device 12 displays the examination protocol screen on the display device 15 by any of these procedures, so that the doctor can confirm the examination protocol. Hereinafter, the details will be described.

  In the endoscope inspection data recording device 20, when the operation receiving unit 222 receives the display request of the inspection protocol, it instructs the display control unit 223 to generate an inspection protocol screen. The operation receiving unit 222 may instruct the display control unit 223 to generate an inspection protocol screen by using an inspection start operation such as inspection distribution as a trigger. In addition, when the examination start / end data acquisition unit (not shown) of the endoscope examination data recording device 20 receives the examination start detection data, the display control unit 223 may be instructed to generate an examination protocol screen. The examination protocol holding unit 234 stores a plurality of tasks to be performed by the doctor in the endoscopy in association with the execution order. Further, the inspection protocol holding unit 234 stores each task in association with one of a plurality of categories. The category of each task may be stored as task attribute information. The display control unit 223 reads a plurality of tasks stored in the examination protocol storage unit 234, generates an examination protocol screen, transmits the screen to the endoscope processing device 12, and causes the display device 15 to display the screen. In the eighth embodiment, display device 15 is a touch panel including a touch pad for detecting a touch position and a display, and is capable of receiving a touch operation from a medical worker such as a doctor or a nurse.

  FIG. 35 shows an example of the inspection protocol screen displayed on the display device 15. The inspection protocol screen is configured by being divided into a plurality of class columns. The class is defined by, for example, the position of a scope or an organ. The class column 300 displays the task at the start of the insertion. In the class field 300, tasks belonging to the “operation” category are displayed in the task display field 301a, and in this example, “insert an endoscope with the mouth deepened” is displayed.

  The class column 302 displays tasks related to the esophagus. In the class column 302, a plurality of task display columns 303a to 303e are provided. Tasks belonging to the “operation” category are displayed in the task display column 303a, and “The esophagus entrance is confirmed and the endoscope is inserted into the esophagus” is displayed. Tasks belonging to the “observation” category are displayed in the task display column 303b, and “observation of the upper esophagus” is displayed. Tasks belonging to the “shooting” category are displayed in the task display column 303c, and “Shooting the central esophagus” is displayed. Tasks belonging to the “shooting” category are displayed in the task display column 303d, and “Shooting the lower esophagus” is displayed. In the task display column 303e, tasks belonging to the “imaging” category are displayed, and “imaging the esophagogastric junction” is displayed.

  The class column 304 displays tasks related to the stomach. Here, a fold mark 304a is displayed in the class field 304, which indicates that the class field 304 is displayed in a folded state. When a medical worker such as a doctor performs a touch operation on the folding mark 304a, a task related to the stomach is expanded and displayed in the class column 304. Note that the folded display can be displayed in an arbitrary class, and in the example shown in FIG. 35, the class column 306 is also displayed in a folded state. When the display size of the display device 15 is not large, the class space irrelevant to the latest task is folded and displayed, so that the screen space can be effectively used.

  As described above, the inspection protocol holding unit 234 stores the plurality of tasks in association with the execution order, and the display control unit 223 generates an inspection protocol screen in which the plurality of tasks are arranged according to the execution order. Then, the image is displayed on the display device 15. This allows the physician to recognize the tasks to be performed from the examination protocol screen and understand the order of the tasks. Note that each task has one of the attributes of an operation category, an observation category, and a shooting category, but it is preferable that the display control unit 223 change the display mode of the task for each category when displaying the task. For example, by making the color of the task display column different for each category, the doctor can intuitively grasp the category of the task to be performed next. For example, the task display column of the operation category may be colored blue, the task display column of the observation category may be colored orange, and the task display column of the photography category may be colored green. Further, the display control unit 223 may apply a color scheme to a text sentence included in each task display column.

  In the eighth embodiment, the examination protocol specifies that the examination is performed in the order of the esophagus, the stomach, and the duodenum. This order is an example, and as described in the fifth embodiment, it may be defined that the examination is performed in the order of duodenum, stomach, and esophagus. The test protocol is defined for each medical facility, and the order of observation follows the test operation of the medical facility.

  The doctor checks the examination protocol screen, recognizes the task to be performed, and proceeds with the examination. After performing the task, the doctor touches the displayed task display column. When the task display column is touched, the task display column is displayed in a mode different from the original display mode so that it can be understood that the task has been performed. The display returns to the original display mode. Thereby, when the touch operation is mistaken, the display mode returns to the original display mode, and the display mode indicates that the task is an unexecuted task. The touch operation by the doctor is transmitted from the endoscope processing device 12 to the endoscope inspection data recording device 20 together with the touch position coordinates.

  The performed task detection unit 240 detects that the task has been performed. When information indicating that a touch operation has been performed by the doctor is transmitted from the endoscope processing device 12, the completion data obtaining unit 221i obtains touch operation information including touch position coordinates, and the completed task detecting unit 240 provide. The task-to-be-executed detecting unit 240 specifies the task display field where the touch operation has been performed based on the touch position coordinates, and detects that the task has been performed. When the performed task detection unit 240 detects that the task has been performed, the data recording unit 225 records that the task has been performed in the test data holding unit 232. Further, the executed task detection unit 240 notifies the display control unit 223 of the detected executed task, and in response thereto, the display control unit 223 changes the display mode of the executed task from the display mode of the unexecuted task. The display form of the executed task and the unexecuted task will be described later.

  In the above example, the doctor explained that the performed task was specified by touching the touch panel.However, for example, the performed task can be specified by stepping on the foot switch installed on the floor. You may. When the execution task detection unit 240 has an image recognition function and the image acquisition unit 221g acquires a still image captured by the endoscope 11 in real time, the execution task detection unit 240 acquires the still image captured by the image acquisition unit 221g. The imaged part may be specified by image analysis of the obtained still image. At this time, when it is determined that the imaging part is the part specified by the imaging task, the execution task detection unit 240 may detect that the imaging task using the part as the imaging part has been performed. .

  FIG. 36 shows another example of the inspection protocol screen displayed on the display device 15. As compared with the inspection protocol screen shown in FIG. 35, FIG. 36 is different in that a caution information display column 301b is provided in the class column 300 and an auxiliary information display column 303f is provided in the class column 302.

  First, the auxiliary information display field 303f will be described. The inspection protocol holding unit 234 stores auxiliary information related to the task. This auxiliary information corresponds to the task, and indicates the point of performing the task. The supplementary information display column 303f displays an execution procedure for the task displayed in the task display column 303a. In this example, a text sentence “Be careful not to enter the pear-shaped depression” is displayed. ing. When auxiliary information is stored in association with the task in the inspection protocol holding unit 234, the display control unit 223 displays the auxiliary information in association with the task. Here, auxiliary information is displayed near the corresponding task (esophagus 1). In FIG. 36, a list display format in which tasks are arranged from top to bottom according to the execution order is adopted. Immediately below, an auxiliary information display column 303f is arranged. If a list display format in which tasks are arranged from left to right in the order of execution is adopted, the auxiliary information display column 303f is arranged immediately to the right of the task (esophagus 1).

  The auxiliary information is information that is presented to a doctor so that the task can be smoothly performed. For example, the auxiliary information is useful for an unskilled physician, but may be unnecessary for a skilled physician. Therefore, the display setting unit 243 may be configured to be able to set whether to display the auxiliary information. This setting is performed for each doctor, and the display setting unit 243 records, in the setting information holding unit 235, setting information set for each doctor, that is, information that determines whether to display auxiliary information. When acquiring information on the inspector performing the examination from the endoscope processing device 12, the display control unit 223 refers to the setting information in the setting information holding unit 235 to determine whether to include the auxiliary information in the examination protocol screen. Identify. Even a skilled physician may include auxiliary information in the examination protocol screen, and how to determine the setting information may be in accordance with the operation guideline of the medical facility.

  Next, the attention information display column 301b will be described. The attention information display column 301b is displayed according to the attribute information of the patient. The patient data acquisition unit 221h acquires patient data from a management server (not shown). The management server is a server that holds examination order information, patient attribute information, and the like, and may include a database that records an endoscope image captured by the endoscope 11. In the endoscopy, there is a matter to be noted depending on the state of the patient, and the examination protocol holding unit 234 stores the attention information on the task in association with the attribute information of the patient.

  For example, heavy alcohol drinkers and smokers are known to be at increased risk for pharyngeal cancer. Therefore, when information such as a heavy alcohol drinker and a smoker is included as the attribute information of the patient, the display control unit 223 transmits the attention information associated with the attribute information from the examination protocol holding unit 234. Read it out and include it in the inspection protocol screen. In FIG. 36, in the attention information display column 301b, attention information regarding the task displayed in the task display column 301a is displayed. In this example, a text that states, "Patients who are at risk for pharyngeal cancer, often observe pharyngeal cancer." The sentence is displayed. As described above, the display control unit 223 displays the attention information in association with the task in accordance with the attribute information of the patient, so that the doctor can clearly understand the points to be noted.

  With reference to FIG. 35, it has been described that the display control unit 223 changes the display mode of the task for each category. However, the display modes of the attention information display column and the auxiliary information display column may be different.

  FIG. 37 shows another example of the inspection protocol screen displayed on the display device 15. In comparison with the inspection protocol screen shown in FIG. 36, in FIG. 37, a sample image 310 is displayed in the task display column of the imaging task. The display control unit 223 may display a sample endoscope image when displaying a task related to imaging. Here, a sample image 310a is displayed in the task display column 303c, a sample image 310b is displayed in the task display column 303d, and a sample image 310c is displayed in the task display column 303e. By including the sample image 310 in the examination protocol screen in this way, an unskilled doctor can refer to what image should be taken. The display setting unit 243 may be configured to be able to set whether or not to display a sample image. This setting is performed for each doctor, and the display setting unit 243 records setting information set for each doctor, that is, information that determines whether to display a sample image in the setting information holding unit 235. When the display control unit 223 acquires information on the inspector performing the examination from the endoscope processing apparatus 12, the display control unit 223 refers to the setting information of the setting information holding unit 235 and determines whether to include the sample image in the examination protocol screen. Identify. This makes it possible to present the sample image 310 only to the examiner who needs the sample image 310.

Hereinafter, a display mode of the executed task and the unexecuted task will be described.
When the performed task detection unit 240 detects the execution of the task as described above, the display control unit 223 changes the display mode of the completed task from the display mode of the unexecuted task.

  FIG. 38 illustrates an example of a display form of the completed task. In this example, it is determined that the task in the task display column 301a, the task in the task display column 303a, and the task in the task display column 303b have been performed, and the attention information in the attention information display column 301b and the auxiliary information in the auxiliary information display column 303f are confirmed. It has been done. The display control unit 223 causes the doctor to recognize the completed task by making the color of the display column different from the color of the display column of the unexecuted task. For example, the display section of the executed task, the confirmed attention information, and the confirmed auxiliary information may be displayed in the same color (for example, gray). Thereby, the doctor can easily distinguish the performed task from the unexecuted task. If the doctor touches the display field displayed in gray, the color of the task is changed to the color of the unexecuted task, and the state selected as completed may be canceled.

  FIG. 39 shows another example of a display mode of the completed task. Also in this example, it is determined that the task in the task display column 301a, the task in the task display column 303a, and the task in the task display column 303b have been performed, and the attention information in the attention information display column 301b and the auxiliary information in the auxiliary information display column 303f are confirmed. It has been done. The display control unit 223 adds executed marks 311a to 311e that indicate that the display has been executed, to the display column. Thereby, the doctor can recognize the performed task. Note that the display control unit 223 may express the completed task by adding a strikethrough to the text in the display column instead of the completed mark 311.

  FIG. 40 shows another example of the display mode of the completed task. Also in this example, it is determined that the task in the task display column 301a, the task in the task display column 303a, and the task in the task display column 303b have been performed, and the attention information in the attention information display column 301b and the auxiliary information in the auxiliary information display column 303f are confirmed. It has been done. The display control unit 223 deletes such a display column from the inspection protocol screen. This allows the doctor to recognize only the unexecuted tasks, and can simplify the examination protocol screen. In particular, when the screen size of the display device 15 is small, the efficiency of information provision can be realized by displaying only unexecuted tasks.

  FIG. 41 shows another example of a display mode of the completed task. Also in this example, it is determined that the task in the task display column 301a, the task in the task display column 303a, and the task in the task display column 303b have been performed, and the attention information in the attention information display column 301b and the auxiliary information in the auxiliary information display column 303f are confirmed. It has been done. In this example, the display control unit 223 displays a time table 312 in which task marks indicating tasks by category are arranged in chronological order, and changes the color of the executed task. By also displaying the time table 312 in this way, the doctor can recognize the temporal relationship between the implemented task and the unexecuted task.

  FIG. 42 shows another example of the inspection protocol screen. In this example, it is determined that the task in the task display column 301a, the task in the task display column 303a, the task in the task display column 303b, and the task in the task display column 303d have been performed, and the attention information and the auxiliary information display in the attention information display column 301b. The auxiliary information in the column 303f has been confirmed.

  In the example shown in FIG. 42, the doctor performs the task in the task display column 303d before performing the task in the task display column 303c. The examination protocol defines the procedure of the examination, and it is not appropriate to take an image of the lower esophagus before imaging the middle esophagus. However, in the case where the lower esophagus has been photographed first for some reason, it is preferable to be able to select the task that has been performed. In order to strictly comply with the inspection procedure defined in the inspection protocol, when a task that should be performed later is performed first, the performed task may not be selected. Accordingly, the doctor needs to select the previous unexecuted task in order to select the executed task, so that it is possible to prevent the doctor from forgetting to execute the unexecuted task.

  FIG. 43 shows another example of the inspection protocol screen. In this example, a task display column 303g is added to the examination protocol screen. This shows a state in which the observation light is switched to the narrow band light (NBI) during the examination, and a task related to the switched narrow band light is presented to the doctor.

  Returning to FIG. 34, the data acquisition unit 221 acquires the execution data in the endoscopy. Here, an example is shown in which the irradiation light type data acquisition unit 221a acquires irradiation light type data. When the irradiation light type data acquiring unit 221a acquires the irradiation light type data of the narrow band light, the status change detection unit 242 detects that the inspection execution status has changed. In this example, the situation change detection unit 242 detects a change in the irradiation light type data that is the execution data, that is, a change from white light to narrowband light. When the situation change detection unit 242 detects a change in the execution data, the display control unit 223 displays a task corresponding to the change on the inspection protocol screen at that timing. Here, a text sentence “Be close” is displayed in the task display field 303g. In this way, the doctor recognizes that close observation is to be performed, and, by approaching, selects the task display field 303g and determines that the task has been executed.

  The display control unit 223 may highlight the task display column 303g of the newly added task. This task was not scheduled by the standard protocol, so the physician may not be aware that the task has occurred. At this time, a certain notification sound may be generated to alert the doctor to the occurrence of a new task.

  As described above, the physician proceeds with the examination according to the examination protocol, and at the end of the examination, ideally, all tasks have been performed. However, during the inspection, there may be some tasks that could not be performed due to some reasons. Therefore, the unexecuted task detection unit 241 investigates the existence of the unexecuted task at the end of the examination, and detects that the unexecuted task exists when the unexecuted task remains. When the unexecuted task detection unit 241 detects the presence of the unexecuted task, the display control unit 223 may display the unexecuted task. The physician may view the displayed unexecuted task and perform a retest if necessary.

  After the examination is completed, the doctor creates an examination report. At this time, a task that has not been performed may be displayed, and a column may be provided for writing the reason why the doctor did not perform the task. By inputting the report on the unexecuted task in this way, it is possible to ensure compliance with the inspection protocol.

  It has been described that, during the inspection, when the execution task detection unit 240 detects that the task has been executed, the data recording unit 225 records the execution of the task in the inspection data holding unit 232. The performance record of this task may be used to confirm that the test was performed according to the test protocol. For example, when the examiner is an unskilled physician, the instructor can confirm whether or not the examination has been performed according to the procedure of the examination protocol from the execution record of the task in the examination. For example, as shown in the time table 312 (see FIG. 41), the display control unit 223 generates a screen in which the performed tasks are arranged in a table format in a time series, and the instructor selects the screen from the screen according to the examination protocol. It may be confirmed whether the inspection has been carried out.

  The present invention has been described based on the embodiments. This embodiment is an exemplification, and it is understood by those skilled in the art that various modifications can be made to the combination of each component and each processing process, and that such modifications are also within the scope of the present invention. is there. The methods described in Embodiment 1-7 can be variously combined, and are not limited to the combinations illustrated in FIGS. 1 to 32.

  Although the configuration in which the image recognition unit 123 is provided in the endoscope processing device 12 has been described in Embodiment 1-7 above, the image recognition unit 123 may be provided in the endoscope inspection data recording device 20, It may be provided in another device other than the endoscope processing device 12 and the endoscope inspection data recording device 20.

  In Embodiment 1-7 described above, various types of data are detected from the light source device 14 and the endoscope image every N seconds or every N frames, and transmitted to the endoscope inspection data recording device 20. In this regard, depending on the type of data, only the data at the time of the event occurrence may be detected and transmitted to the endoscope inspection data recording device 20. For example, it is possible to set that the data of the period in which the medicine is not sprayed in the medicine spraying data is not transmitted to the endoscope inspection data recording device 20.

DESCRIPTION OF SYMBOLS 1 ... Endoscope business support system, 10 ... Endoscope system, 15 ... Display device, 20 ... Endoscope inspection data recording device, 221 ... Data acquisition part, 221a ... -Irradiation light type data acquisition unit, 221g ... image acquisition unit, 221h ... patient data acquisition unit, 221i ... completion data acquisition unit, 222 ... operation reception unit, 223 ... display control unit, 225: data recording unit, 232: inspection data holding unit, 234: inspection protocol holding unit, 235: setting information holding unit, 240: executed task detection unit, 241: not executed Task detection unit, 242... Situation change detection unit, 243.

Claims (10)

An examination protocol holding unit that stores a plurality of tasks to be performed by a physician in endoscopy,
A display control unit that displays a plurality of tasks stored in the inspection protocol holding unit on a display device,
An execution task detection unit that detects that the task has been executed;
A recording unit for recording that the task has been performed;
A data acquisition unit for acquiring execution data in endoscopy,
A situation change detection unit that detects a change in the execution data ,
The display control unit, when a change in the execution data is detected, displays a task according to the change,
An endoscope business support system characterized by the following. When the performed task detection unit detects execution of a task, the display control unit causes the display mode of the performed task to be different from the display mode of the unexecuted task,
The endoscope service support system according to claim 1, wherein: The inspection protocol holding unit stores a plurality of tasks in association with their execution order,
The display control unit displays a plurality of tasks on a display device according to an execution order.
The endoscope service support system according to claim 1 or 2, wherein: The test protocol holding unit stores each task in association with one of a plurality of categories,
The display control unit changes the display mode of the task for each category,
4. The endoscope work support system according to claim 1, wherein: The display control unit, when displaying a task related to imaging, displays an endoscope image as a sample,
The endoscope service support system according to any one of claims 1 to 4, wherein: The inspection protocol holding unit stores auxiliary information about a task,
The display control unit displays auxiliary information in association with the task,
The endoscope service support system according to any one of claims 1 to 5, wherein: Further comprising a display setting unit to set whether to display the auxiliary information,
The endoscope operation support system according to claim 6, wherein: Further comprising a patient data acquisition unit for acquiring patient data,
The display control unit displays the attention information according to the patient data in association with the task,
The endoscope service support system according to any one of claims 1 to 7, wherein: An unexecuted task detection unit that detects that an unexecuted task exists,
When the unexecuted task detection unit detects the presence of the unexecuted task, the display control unit displays the unexecuted task,
The endoscope service system according to any one of claims 1 to 8, wherein: The unexecuted task detection unit, at the end of the inspection, detects that an unexecuted task exists,
The endoscope service support system according to claim 9, wherein: