WO2022064998A1 - Inspection assistance device, inspection assistance method, and inspection assistance program - Google Patents

Inspection assistance device, inspection assistance method, and inspection assistance program Download PDF

Info

Publication number
WO2022064998A1
WO2022064998A1 PCT/JP2021/032384 JP2021032384W WO2022064998A1 WO 2022064998 A1 WO2022064998 A1 WO 2022064998A1 JP 2021032384 W JP2021032384 W JP 2021032384W WO 2022064998 A1 WO2022064998 A1 WO 2022064998A1
Authority
WO
WIPO (PCT)
Prior art keywords
display
monitor
unit
inspection support
support device
Prior art date
Application number
PCT/JP2021/032384
Other languages
French (fr)
Japanese (ja)
Inventor
正晃 伊與田
Original Assignee
株式会社Ａｉメディカルサービス
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社Ａｉメディカルサービス filed Critical 株式会社Ａｉメディカルサービス
Publication of WO2022064998A1 publication Critical patent/WO2022064998A1/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/045—Control thereof

Definitions

  • the present invention relates to an inspection support device, an inspection support method, and an inspection support program.
  • the system in which the examination support device subsequently captures and analyzes the internal image of the subject captured by the endoscope system is not immediate.
  • the doctor who diagnoses the subject wants to know the analysis result immediately in conjunction with the imaging of the inside of the body by the endoscopic system. Therefore, it is conceivable to connect the inspection support device to the endoscope system and have the inspection support device analyze the internal image captured by the endoscope system in almost real time. In this case, the doctor will perform the examination while visually observing the monitor provided in the endoscope system and the monitor provided in the examination support device. If the monitor provided in the inspection support device always displays the analysis result in a certain display mode, the visibility may be significantly inferior depending on the positional relationship between the two monitors. However, the menu operation for specifying the display layout in advance is troublesome and may increase the time required for inspection.
  • the present invention has been made to solve such a problem, and in relation to the system monitor provided in the endoscope system, the present invention automatically displays a display mode for displaying inspection information and the like on a display monitor in an easy-to-see manner. It provides an inspection support device or the like for determining.
  • the inspection support device is an inspection support device used by connecting to an endoscope system including a camera unit inserted into the body of a subject, and is imaged by the camera unit.
  • An acquisition unit that acquires captured images, a display monitor for displaying inspection information related to the captured image and internal examination, and a display monitor and endoscope that are reflected in the captured image acquired by the acquisition unit prior to the execution of the internal examination.
  • An analysis unit that analyzes the interrelationships of the system monitors attached to the system, a determination unit that determines the display mode of the display monitor based on the interrelationships analyzed by the analysis unit, and a display monitor display according to the display mode determined by the determination unit. It is provided with a display control unit for controlling.
  • the inspection support method is an inspection support method using an inspection support device used by connecting to an endoscope system including a camera unit inserted into the body of the subject.
  • the inspection support program is an inspection support program for controlling an inspection support device used by connecting to an endoscope system including a camera unit inserted into the body of a subject.
  • the computer is made to execute a display control step for controlling the display of the display monitor according to the display mode determined in the step.
  • FIG. 1 is a diagram showing a state of endoscopy using the endoscope system 200 and the inspection device 100 according to the present embodiment.
  • Both the endoscope system 200 and the inspection device 100 are installed in the examination space.
  • the endoscope system 200 includes a camera unit 210, which is inserted from the oral cavity of a lying subject into the stomach and transmits an image signal of an image of the inside of the stomach to the system body as shown in the figure. do.
  • the insertion of the camera unit 210 into the stomach and the imaging operation are performed by a doctor.
  • the endoscope system 200 includes, for example, a system monitor 220 composed of a liquid crystal panel, processes an image signal sent from the camera unit 210, and displays it on the system monitor 220 as a visible captured image 221. Further, the endoscope system 200 displays the inspection information 222 including the subject information and the camera information of the camera unit 210 on the system monitor 220.
  • the inspection device 100 is connected to the endoscope system 200 by a connection cable 250.
  • the endoscope system 200 also transmits a display signal to be transmitted to the system monitor 220 to the inspection device 100 via the connection cable 250.
  • the inspection device 100 includes, for example, a display monitor 120 composed of a liquid crystal panel, and extracts an image signal corresponding to the captured image 221 from the display signal sent from the endoscope system 200 to obtain a visible captured image 121. It is displayed on the display monitor 120. Further, the inspection device 100 generates image data of the captured image 121, analyzes it, outputs diagnostic assistance information 122, and displays it on the display monitor 120.
  • the inspection device 100 displays the captured image 121 and the diagnostic assistance information 122 as soon as the display signal is received from the endoscope system 200 or in synchronization with the timing when the operation of the camera unit 210 by the doctor is estimated. That is, the doctor can sequentially confirm the captured image 121 and the diagnostic assistance information 122 in synchronization with the progress of the examination.
  • the system monitor 220 of the endoscope system 200 and the display monitor 120 of the inspection device 100 are juxtaposed in the examination space. .. Therefore, the doctor will proceed with the examination while visually observing the two monitors while operating the camera unit 210 inserted into the stomach of the subject.
  • the two monitors may be arranged in various ways depending on the convenience of the examination space and the like, and the amount of movement of the line of sight between the two monitors may vary depending on the arrangement relationship between the two monitors and the display mode of the display monitor 120. The visibility of the displayed information may be significantly reduced, such as when the size is increased.
  • the inspection device 100 in the present embodiment analyzes the mutual relationship between the display monitor 120 provided by the inspection device 100 and the system monitor 220 attached to the endoscope system 200, and automatically optimizes the display mode in the display monitor 120. ..
  • the configuration of the inspection device 100, the processing procedure, the determination of the display mode, and the like will be sequentially described.
  • FIG. 2 is a hardware configuration diagram of the inspection device 100.
  • the inspection device 100 is mainly composed of an arithmetic processing unit 110, a display monitor 120, an input / output interface 130, an input device 140, and a storage unit 150.
  • the arithmetic processing unit 110 is a processor (CPU: Central Processing Unit) that controls the inspection device 100 and executes a program.
  • the processor may be configured to cooperate with an arithmetic processing chip such as an ASIC (Application Specific Integrated Circuit) or a GPU (Graphics Processing Unit).
  • the arithmetic processing unit 110 reads out the inspection support program stored in the storage unit 150 and executes various processes related to inspection support.
  • the display monitor 120 is a monitor provided with, for example, a liquid crystal panel, and visually displays the captured image 121 and the diagnostic assistance information 122.
  • the input / output interface 130 is a connection interface for exchanging information with an external device, including a connector for connecting the connection cable 250.
  • the input / output interface 130 includes, for example, a LAN unit, and takes in the update data of the inspection support program and the analysis neural network 151 described later from an external device and delivers it to the arithmetic processing unit 110.
  • the input device 140 is, for example, a touch panel superimposed on a keyboard, a mouse, or a display monitor 120, and a doctor or an assistant operates these to change the settings of the inspection device 100 or input information necessary for the inspection. Or something.
  • the storage unit 150 is a non-volatile storage medium, and is composed of, for example, an HDD (Hard Disk Drive).
  • the storage unit 150 can store various parameter values, functions, display element data, look-up tables, etc. used for control and calculation, in addition to a program that executes control and processing of the inspection device 100.
  • the storage unit 150 stores, in particular, the neural network 151 for analysis.
  • the neural network 151 for analysis is a trained model that calculates the probability that a lesion exists in the image when the image data captured by the camera unit 210 is input.
  • the storage unit 150 may be composed of a plurality of hardware.
  • the storage medium for storing the program and the storage medium for storing the analysis neural network 151 may be composed of different hardware.
  • the arithmetic processing unit 110 also plays a role as a functional arithmetic unit that executes various operations according to the processing instructed by the inspection support program.
  • the arithmetic processing unit 110 can function as an acquisition unit 111, an analysis unit 112, a determination unit 113, a diagnostic assistance unit 114, and a display control unit 115.
  • the acquisition unit 111 reproduces and acquires the image captured by the camera unit 210 as the captured image 121 by processing the display signal sent from the endoscope system 200.
  • the analysis unit 112 analyzes the interrelationship between the display monitor 120 and the system monitor 220 reflected in the captured image 121 acquired by the acquisition unit 111 prior to the execution of the examination of the stomach of the subject.
  • the determination unit 113 determines the display mode of the display monitor 120 based on the interrelationship analyzed by the analysis unit 112.
  • the diagnostic assistance unit 114 inputs the captured image 121 acquired by the acquisition unit 111 during the execution of the gastric examination into the analytical neural network 151 read from the storage unit 150, calculates the probability of the presence of a lesion, and assists the diagnosis. Generate information.
  • the display control unit 115 controls the display of the display monitor 120 according to the display mode determined by the determination unit 113.
  • FIG. 3 is a diagram showing a state in which the camera unit 210 of the endoscope system 200 images the endoscope system 200 and the inspection device 100.
  • the doctor said that at least a part of the system monitor 220 and a part of the display monitor 120 were in the angle of view after starting the endoscope system 200 and the inspection device 100 and before starting the examination of the stomach of the subject.
  • the endoscope system 200 and the inspection device 100 are imaged so as to fit.
  • FIG. 3 shows how all of the system monitor 220 and all of the display monitor 120 are within the angle of view.
  • the image captured by the camera unit 210 is processed by the endoscope system 200 and displayed as a visible captured image 221 in a part of the display area of the system monitor 220.
  • the display control unit 115 of the inspection device 100 displays the display signal received from the endoscope system 200 on the display monitor 120 as it is in the same manner as the display mode displayed on the system monitor 220.
  • the acquisition unit 111 extracts the area of the captured image 121 from the display screen generated from the display signal.
  • the display control unit 115 superimposes and displays a frame-shaped area index 123 on the four corners of the region so that the region of the captured image 121 extracted by the acquisition unit 111 can be seen. Once the area index 123 is superimposed and displayed, the area index 123 will appear on the display monitor 120 which is reflected as a subject image in the captured image subsequently captured by the camera unit 210.
  • the image pickup by the camera unit 210 is continuously executed during this period, and the captured images are sequentially sent to the endoscope system 200 at, for example, 60 fps. Further, the endoscope system 200 sequentially generates display signals accordingly. Therefore, the display screen generated from the display signal is a frame image, and the acquisition unit 111 of the inspection device 100 extracts the region of the captured image 121 from the frame image. Specifically, by utilizing the fact that the captured image captured by the camera unit 210 changes sequentially to some extent, the difference between a plurality of frame images before and after is detected, and a region in which a change of a predetermined amount or more is detected is detected. It is determined that the area is the captured image 121.
  • the analysis unit 112 analyzes the mutual relationship between the system monitor 220 and the display monitor 120 by using the captured image 121 extracted from the frame image in this way.
  • FIG. 4 is a diagram for explaining a procedure for analyzing the mutual relationship between the system monitor 220 and the display monitor 120.
  • the acquisition unit 111 extracts and cuts out the region of the captured image 121 from the frame image, and the analysis unit 112. Hand over to.
  • the analysis unit 112 determines the display monitor area 120a and the system monitor area 220a, which are the respective areas of the display monitor 120 and the system monitor 220 reflected in the captured image 121. Specifically, both monitor areas may be determined by matching processing with a general monitor appearance image prepared in advance, or the captured image 121 is input to a trained model in which a large number of monitor images have been learned in advance. Both monitor areas may be determined.
  • the analysis unit 112 determines the area in which the area index 123 is reflected as the display monitor area 120a and the area in which the area index 123 is not reflected as the system monitor area 220a among the two areas extracted as the monitor area. ..
  • the display control unit 115 superimposes the area index 123 on the four corners of the captured image 121.
  • the area index 123 is superimposed in this way, the captured image 121 cut out from the frame image and the image of the area surrounded by the area index 123 have a substantially similar relationship, so that the display monitor area 120a by the analysis unit 112 It becomes easier to make a decision.
  • the area index 123 to be superimposed does not have to have a frame shape as in the present embodiment, and may not be an index showing the captured image 121 unless the display monitor area 120a is determined by using the similarity relationship. I do not care.
  • a large " ⁇ " mark may be superimposed on the entire display screen, or the index printed on the bezel portion of the display monitor 120 may be attached instead of superimposing the index on the display area. .. Any mode may be used as long as it is an index that can distinguish between the two monitors in the captured image 121.
  • the analysis unit 112 analyzes the mutual relationship between the display monitor 120 and the system monitor 220.
  • the positional relationship between the display monitor 120 and the system monitor 220 is analyzed.
  • the analysis unit 112 finds that the display monitor 120 is located substantially to the right of the system monitor 220 and delivers the result to the determination unit 113.
  • the coordinates of the center of gravity of each of the display monitor area 120a and the system monitor area 220a are calculated, and the vector value connecting the coordinates of the center of gravity is passed to the determination unit 113 as the analysis result of the positional relationship.
  • the determination unit 113 determines the display mode of the display monitor 120 based on the analysis result received from the analysis unit 112.
  • FIG. 5 is a diagram showing an example of a display mode determined by the determination unit 113.
  • the display control unit 115 When the determination unit 113 determines the display mode of the display element as described below, the display control unit 115 generates a display signal according to the determination and causes each display element to be displayed on the display monitor 120.
  • FIG. 5A shows a display mode when the display monitor 120 is located on the right side with respect to the system monitor 220
  • FIG. 5B shows a display mode when the display monitor 120 is located on the left side with respect to the system monitor 220. Indicates the display mode of.
  • a display mode of the display screen while the camera unit 210 is inserted into the stomach of the subject and the examination is being performed will be described.
  • the unique information is information that is not displayed on the system monitor 220 but is displayed only on the display monitor 120, and specifically, is information generated by the processing of the arithmetic processing unit 110 of the inspection device 100.
  • the display monitor 120 displays the estimated probability of the presence of a lesion in the captured image 121 calculated by the diagnostic assistance unit 114 as diagnostic assistance information.
  • the diagnostic aid information includes numerical information 122a indicating an estimated probability, title text 122b, and pie chart graphics 122c.
  • the doctor can visually recognize both monitors. It can enhance the sex. That is, it is possible to shorten the movement of the line of sight of the doctor who visually recognizes important information, which in turn contributes to the realization of a comfortable examination environment.
  • the captured image 121 displayed on the display monitor 120 may be the original image used by the diagnostic assistance unit 114 to calculate the estimation probability.
  • the display control unit 115 may display the captured image 121 as a still image for a certain period of time for displaying the diagnostic assistance information 122.
  • the display monitor 120 when the display monitor 120 is located on the left side with respect to the system monitor 220, the display area of the display monitor 120 is displaced from the center to the right side where the system monitor 220 is located. Display unique information.
  • the diagnostic assistance information 122 is displayed closer to the system monitor 220 side as unique information, and the captured image 121 is displayed at a position far from the system monitor 220.
  • the display element displayed by deviating to the side of the system monitor 220 is not limited to the diagnostic auxiliary information analyzed based on the captured image, but may be other inspection information.
  • the examination information may be, for example, an examination guide that sequentially indicates a portion in the stomach to be imaged by a sample image or the like.
  • the information is not limited to the inspection information, and may be unique information generated by the inspection device 100 that is not displayed on the system monitor 220. For example, it may be the subject information input by the input device 140. It may be configured so that the doctor who is the user can select in advance which display element is shifted to the system monitor 220 side for display.
  • FIG. 6 is a diagram showing another example of the display mode determined by the determination unit 113. Specifically, the display mode when the display monitor 120 is located above the system monitor 220 is shown.
  • the display mode of FIG. 6 is a display mode in which the camera unit 210 is inserted into the stomach of the subject and the examination is being performed, as in the case of FIGS. 5 (A) and 5 (B). , The displayed display elements are the same.
  • the diagnostic assistance information 122 is displayed closer to the system monitor 220 side as unique information, and the captured image 121 is displayed at a position far from the system monitor 220.
  • the diagnostic assistance information 122 can be displaced toward the system monitor 220, so that the movement of the doctor's line of sight is shortened. be able to.
  • FIG. 7 is a diagram showing another example of the display mode determined by the determination unit 113. Specifically, the display mode when the display monitor 120 is located on the upper left side with respect to the system monitor 220 is shown.
  • the display mode of FIG. 7 is a display mode in which the camera unit 210 is inserted into the stomach of the subject and the examination is being performed, as in the case of FIGS. 5 (A) and 5 (B). , The displayed display elements are the same.
  • the display monitor 120 when the display monitor 120 is located on the upper left side of the system monitor 220, it is located in the lower right side of the display area of the display monitor 120 where the system monitor 220 is located.
  • Display unique information Here, the diagnostic assistance information 122 is displayed closer to the system monitor 220 side as unique information, and the captured image 121 is displayed at a position far from the system monitor 220.
  • the numerical information 122a which is more important information, is displayed closer to the system monitor 220 side.
  • the movement of the doctor's line of sight can be further shortened.
  • other elements may be displayed at the reference position and only the numerical information 122a may be shifted to the system monitor 220 side and displayed.
  • the display mode in which only the numerical information 122a is displaced toward the system monitor 220 is not limited to the case where the positional relationship between the two monitors is in the diagonal direction, but also when they are arranged in the vertical and horizontal directions as shown in FIGS. 5 and 6. Can be adopted.
  • FIG. 8 is a diagram showing another example of the display mode determined by the determination unit 113.
  • the display monitor 120 and the system monitor 220 are monitors having substantially the same size as each other and are arranged at positions approximately equidistant from the doctor.
  • the display monitor 120 and the system monitor 220 may be different in size from each other, or may be arranged so as to be displaced in the depth direction.
  • the example of the display mode shown in FIG. 8 is similar to FIG. 5 (A) in that the display monitor 120 is located on the right side of the system monitor 220, but the display monitor 120 is larger than the system monitor 220 for the doctor. This is the case when it is placed at a distant position.
  • the analysis unit 112 analyzes from the captured image 121 that the display monitor area 120a is on the right side of the system monitor area 220a and that the display monitor area 120a is smaller than the system monitor area 220a, the analysis unit 112 determines the result. Hand over to. From the result, the determination unit 113 determines that the display monitor 120 is located on the right side of the system monitor 220, the display monitor 120 is located farther than the system monitor 220, or the display monitor 120 is located closer to the system monitor 220. Recognizing that it is small, the display mode shown in FIG. 8 is determined.
  • the ratio of the diagnostic assistance information 122 to the display area is increased as compared with the case where the display monitors 120 having the same size as the system monitor 220 are arranged adjacent to each other, and the enlarged diagnostic assistance information 122 is provided.
  • the display mode is determined so that the system monitor 220 is displaced to the left side where it is located.
  • the layout of the internal elements constituting the diagnostic auxiliary information 122 may be changed. For example, as shown in FIG. 8, the size of the title text 122b may not be changed and the position may be shifted to the system monitor 220 side.
  • the diagnostic assistance information 122 may be displayed so as to cover a part of the captured image 121 displayed at a position far from the system monitor 220.
  • the determination unit 113 indicates that the display monitor 120 is located closer to the system monitor 220, or the display monitor 120 is larger than the system monitor 220. If it is recognized, the display mode may be determined so that the ratio of the diagnostic aid information 122 to the display area is reduced. In this case, the unique information may be further displayed in the generated margin.
  • FIG. 9 is a diagram showing still another example of the display mode determined by the determination unit 113.
  • the diagnostic auxiliary information 122 is displaced toward the system monitor 220 as the unique information, but in the example of FIG. 9, the unique information is superimposed on the captured image 121. This is a display mode.
  • the analytical neural network 151 can also identify the suspected lesion site, the superimposed information 121a is superimposed on that site of the captured image 121.
  • the captured image 121 on which the superimposed information 121a is superimposed is displaced toward the system monitor 220 side.
  • the display monitor 120 since the display monitor 120 is located on the right side with respect to the system monitor 220, the captured image 121 in which the superimposed information 121a is superimposed on the region shifted to the left from the center in the display region of the display monitor 120. Is displayed.
  • the diagnostic auxiliary information 122 displayed at a position far from the system monitor 220 is also unique information, but which unique information is shifted to the system monitor 220 side and displayed is set in advance. It is determined by the priorities that have been set. For example, when there is a plurality of unique information for which priority is set, the highest-ranked unique information among the selected unique information to be displayed is biased and displayed on the system monitor 220 side. Therefore, if the unique information displayed by shifting to the system monitor 220 side at a certain point is deleted after a certain period of time, the next higher priority unique information is shifted to the system monitor 220 side again. Is displayed.
  • FIG. 10 is a diagram showing still another example of the display mode determined by the determination unit 113.
  • the analysis unit 112 may analyze the color tone adopted for the display of the system monitor 220. For example, as shown in FIG. 10, when dark black is adopted as the background color of the system monitor 220, the determination unit 113 makes the background area 124 of the display monitor 120 dark black. At this time, the coloring of the diagnostic assistance information 122 may be changed to one having good visibility according to the color tone of the background area 124. Further, the size of the display element may be changed.
  • the deviation of the unique information is also performed in the example of FIG. 10, the deviation of the unique information may be omitted when the visibility of the unique information can be improved by adjusting the color tone or the like.
  • the analysis unit 112 does not have to analyze the relative positional relationship between the display monitor area 120a and the system monitor area 220a.
  • the determination unit 113 may determine to select and use a text font that is close to the text font.
  • the determination unit 113 may make the size of the display element larger than usual. Further, depending on the size and distance of the display monitor 120, emphasizing means such as blinking of display elements, color change, and combined use of reading voice may be adopted.
  • FIG. 11 is a diagram illustrating a procedure of analysis processing using the neural network 151 for analysis, which is a trained model.
  • the neural network 151 for analysis is created in advance by supervised learning in which an image including a lesion site and an image not including the lesion site are given in large quantities together with the correct answer.
  • the neural network 151 for analysis created in this way is stored in the storage unit 150, appropriately read out by the diagnostic assistance unit 114, and used.
  • the endoscope system 200 receives a captured image from the camera unit 210 inserted into the stomach of the subject at, for example, 60 fps as described above, but when the doctor generates a release signal, a still image is displayed. Receive as an captured image.
  • the display signal received by the inspection device 100 from the endoscope system 200 is a frame signal of 60 fps as a display signal, but the captured image 121 developed from this is a pseudo still image frozen for a certain period of time. ..
  • the acquisition unit 111 When the acquisition unit 111 detects this pseudo still image and generates the captured image data, the acquisition unit 111 delivers the captured image data to the diagnosis assisting unit 114 as the image data to be diagnosed.
  • the diagnostic assistance unit 114 inputs the captured image 121 developed from the received captured image data to the analysis neural network 151, and causes the estimation probability to be calculated.
  • FIG. 12 is a flow chart illustrating a processing procedure executed by the arithmetic processing unit 110. The flow starts, for example, from the time when the endoscope system 200 and the inspection device 100 are activated.
  • step S101 the acquisition unit 111 acquires the display signal sent from the endoscope system 200 and develops it into a frame image. Then, in step S102, the captured image 121 is cut out from the expanded frame image and handed over to the analysis unit 112.
  • step S103 the analysis unit 112 searches the display monitor area 120a and the system monitor area 220a from the captured image 121 as described with reference to FIG. After finding these areas, the interrelationship between the display monitor 120 and the system monitor 220 is analyzed using these coordinates and the like.
  • the analysis unit 112 proceeds to step S104 and determines whether or not the mutual relationship analysis has been successful. If the display monitor area 120a and the system monitor area 220a cannot be found from the captured image 121, or if the mutual relationship cannot be analyzed even if they can be found, the process returns to step S101. If not, the analysis result is handed over to the determination unit 113, and the process proceeds to step S105.
  • the determination unit 113 determines the display mode of the display monitor 120 based on the mutual relationship between the display monitor 120 and the system monitor 220 indicated by the analysis result in step S105. After that, in step S106, the arithmetic processing unit 110 waits until the inspection is started. The arithmetic processing unit 110 determines, for example, that the examination is started when the acquired captured image 121 is confirmed to be an image in the oral cavity.
  • the display control unit 115 When the inspection is started, the display control unit 115 generates a display signal according to the display mode determined by the determination unit 113 in step S107, and displays each display element including the inspection information on the display monitor 120.
  • the acquisition unit 111 cuts out the captured image 121 from the frame image developed from the display signal and delivers the captured image 121 to the diagnostic assistance unit 114.
  • the diagnosis assisting unit 114 calculates the estimated probability as described above and hands it over to the display control unit 115.
  • the arithmetic processing unit 110 repeats steps S107 to S109 until an instruction to end the inspection is received in step S110. When the instruction to end the inspection is received, the series of processes is completed.
  • the endoscope system 200 and the inspection device 100 are connected via the connection cable 250, but a wireless connection may be used instead of a wired connection.
  • the endoscope system 200 outputs a display signal to the outside, and the inspection device 100 has described an embodiment using this display signal.
  • the signal output by the endoscope system 200 to the outside is captured by the camera unit 210.
  • the format of the output signal does not matter as long as it includes the image signal of the captured image. If the camera unit 210 directly transmits an image signal, the inspection device 100 may receive the image signal.
  • the inspection device 100 may be configured to transmit some request signal to the endoscope system 200.
  • the inspection device 100 may be provided with a change request unit that generates and transmits a request signal requesting a change in the display mode of the system monitor 220 based on the display mode determined by the determination unit 113.
  • the change request unit transmits, for example, a request signal requesting that the brightness of the system monitor 220 be reduced during the first predetermined time during which each diagnostic aid information 122 is displayed on the display monitor 120. In this way, if the display of the display monitor 120 and the display of the system monitor 220 can be linked, the visibility can be further improved.
  • the inspection support device 100 connected to the endoscope system 200 for inspecting the duodenum is a device that calculates and displays the probability that a lesion site is present in the captured image of the duodenum.
  • the neural network 151 for analysis is prepared in advance according to the site to be inspected.
  • the description has been made on the assumption that the camera unit 210 included in the endoscope system 200 is a flexible endoscope, but even if the camera unit 210 is a rigid endoscope, it may be described. There is no difference in the configuration or processing procedure of the inspection support device 100.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biomedical Technology (AREA)
  • Optics & Photonics (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Biophysics (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Endoscopes (AREA)

Abstract

This inspection assistance device is used while being connected to an endoscope system provided with a camera unit that is inserted into the body of a subject. The inspection assistance device is provided with: an acquisition unit for acquiring captured images captured by the camera unit; a display monitor for displaying the captured image and inspection information relating to in vivo inspection; an analysis unit for analyzing the correlation between the display monitor appearing in the captured image acquired by the acquisition unit before the execution of the in vivo inspection and a system monitor accompanying the endoscope system; a determination unit for determining a manner of display on the display monitor on the basis of the correlation analyzed by the analysis unit; and a display control unit for controlling the display on the display monitor on the basis of the display manner determined by the determination unit. Such an inspection device can automatically determine a display manner for displaying inspection information and the like on the display monitor so as to be easily recognized in relation to the system monitor provided in the endoscope system.

Description

検査支援装置、検査支援方法および検査支援プログラムInspection support equipment, inspection support methods and inspection support programs
　本発明は、検査支援装置、検査支援方法および検査支援プログラムに関する。 The present invention relates to an inspection support device, an inspection support method, and an inspection support program.
　内視鏡により被検者の例えば胃内部を撮像してその画像をモニタに表示する内視鏡システムが知られている。最近では、内視鏡システムで撮像された画像を分析して、その結果を医師へ知らせる検査支援装置も普及しつつある（例えば、特許文献１参照）。 There is known an endoscope system that images the inside of the stomach of a subject with an endoscope and displays the image on a monitor. Recently, an inspection support device that analyzes an image captured by an endoscope system and informs a doctor of the result is becoming widespread (see, for example, Patent Document 1).
特開２０１９－４２１５６号公報JP-A-2019-42156
　内視鏡システムが撮像した被検者の体内内画像を検査支援装置が事後的に取り込んで分析するシステムは、即時性に乏しい。被検者を診断する医師は、内視鏡システムによる体内の撮像に連動して即時に分析結果を知りたい。そこで、検査支援装置を内視鏡システムに接続して、内視鏡システムが撮像した体内画像をほぼリアルタイムで検査支援装置に分析させる利用態様が考えられる。この場合、医師は、内視鏡システムが備えるモニタと検査支援装置が備えるモニタを視認しながら診察を行うことになる。検査支援装置が備えるモニタが常に一定の表示態様により分析結果を表示するのであれば、２つのモニタの位置関係によっては著しく視認性が劣ることがある。しかし、表示レイアウトを事前に指定するメニュー操作は煩わしく、検査に要する時間を引き延ばす原因となることもある。 The system in which the examination support device subsequently captures and analyzes the internal image of the subject captured by the endoscope system is not immediate. The doctor who diagnoses the subject wants to know the analysis result immediately in conjunction with the imaging of the inside of the body by the endoscopic system. Therefore, it is conceivable to connect the inspection support device to the endoscope system and have the inspection support device analyze the internal image captured by the endoscope system in almost real time. In this case, the doctor will perform the examination while visually observing the monitor provided in the endoscope system and the monitor provided in the examination support device. If the monitor provided in the inspection support device always displays the analysis result in a certain display mode, the visibility may be significantly inferior depending on the positional relationship between the two monitors. However, the menu operation for specifying the display layout in advance is troublesome and may increase the time required for inspection.
　本発明は、このような問題を解決するためになされたものであり、内視鏡システムが備えるシステムモニタとの関係において、検査情報等を表示モニタに見やすく表示するための表示態様を自動的に決定する検査支援装置等を提供するものである。 The present invention has been made to solve such a problem, and in relation to the system monitor provided in the endoscope system, the present invention automatically displays a display mode for displaying inspection information and the like on a display monitor in an easy-to-see manner. It provides an inspection support device or the like for determining.
　本発明の第１の態様における検査支援装置は、被検者の体内に挿通されるカメラユニットを備える内視鏡システムに接続して利用される検査支援装置であって、カメラユニットで撮像された撮像画像を取得する取得部と、撮像画像と体内検査に関する検査情報を表示するための表示モニタと、体内検査の実行に先立って取得部が取得した撮像画像に写り込んだ表示モニタと内視鏡システムに付随するシステムモニタの相互関係を解析する解析部と、解析部が解析した相互関係に基づいて表示モニタの表示態様を決定する決定部と、決定部が決定した表示態様に従って表示モニタの表示を制御する表示制御部とを備える。 The inspection support device according to the first aspect of the present invention is an inspection support device used by connecting to an endoscope system including a camera unit inserted into the body of a subject, and is imaged by the camera unit. An acquisition unit that acquires captured images, a display monitor for displaying inspection information related to the captured image and internal examination, and a display monitor and endoscope that are reflected in the captured image acquired by the acquisition unit prior to the execution of the internal examination. An analysis unit that analyzes the interrelationships of the system monitors attached to the system, a determination unit that determines the display mode of the display monitor based on the interrelationships analyzed by the analysis unit, and a display monitor display according to the display mode determined by the determination unit. It is provided with a display control unit for controlling.
　また、本発明の第２の態様における検査支援方法は、被検者の体内に挿通されるカメラユニットを備える内視鏡システムに接続して利用される検査支援装置を用いた検査支援方法であって、体内検査の実行に先立ってカメラユニットで撮像された撮像画像を取得する取得ステップと、撮像画像に写り込んだ、検査支援装置が備える表示モニタと内視鏡システムに付随するシステムモニタの相互関係を解析する解析ステップと、解析ステップで解析された相互関係に基づいて、カメラユニットで撮像された撮像画像と体内検査に関する検査情報を表示する表示モニタの表示態様を決定する決定ステップと、決定ステップで決定された表示態様に従って表示モニタの表示を制御する表示制御ステップとを有する。 Further, the inspection support method according to the second aspect of the present invention is an inspection support method using an inspection support device used by connecting to an endoscope system including a camera unit inserted into the body of the subject. The acquisition step of acquiring the captured image captured by the camera unit prior to the execution of the internal examination, and the mutual display monitor of the inspection support device and the system monitor attached to the endoscope system reflected in the captured image. An analysis step for analyzing the relationship, and a determination step for determining the display mode of the display monitor for displaying the image captured by the camera unit and the inspection information related to the internal examination based on the interrelationship analyzed in the analysis step. It has a display control step that controls the display of the display monitor according to the display mode determined in the step.
　さらに、本発明の第３の態様における検査支援プログラムは、被検者の体内に挿通されるカメラユニットを備える内視鏡システムに接続して利用される検査支援装置を制御する検査支援プログラムであって、体内検査の実行に先立ってカメラユニットで撮像された撮像画像を取得する取得ステップと、撮像画像に写り込んだ、検査支援装置が備える表示モニタと内視鏡システムに付随するシステムモニタの相互関係を解析する解析ステップと、解析ステップで解析された相互関係に基づいて、カメラユニットで撮像された撮像画像と体内検査に関する検査情報を表示する表示モニタの表示態様を決定する決定ステップと、決定ステップで決定された表示態様に従って表示モニタの表示を制御する表示制御ステップとをコンピュータに実行させる。 Further, the inspection support program according to the third aspect of the present invention is an inspection support program for controlling an inspection support device used by connecting to an endoscope system including a camera unit inserted into the body of a subject. The acquisition step of acquiring the captured image captured by the camera unit prior to the execution of the internal examination, and the mutual display monitor of the inspection support device and the system monitor attached to the endoscope system reflected in the captured image. An analysis step for analyzing the relationship, and a determination step for determining the display mode of the display monitor for displaying the image captured by the camera unit and the inspection information related to the internal examination based on the interrelationship analyzed in the analysis step. The computer is made to execute a display control step for controlling the display of the display monitor according to the display mode determined in the step.
　本発明により、内視鏡システムが備えるシステムモニタとの関係において、検査情報等を表示モニタに見やすく表示するための表示態様を自動的に決定する検査支援装置等を提供することができる。　 INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide an inspection support device or the like that automatically determines a display mode for displaying inspection information or the like on a display monitor in an easy-to-see manner in relation to a system monitor included in the endoscope system. It was
内視鏡システムと本実施形態に係る胃検査支援装置を用いた内視鏡検査の様子を示す図である。It is a figure which shows the state of the endoscopy using the endoscopy system and the gastric examination support device which concerns on this embodiment. 胃検査支援装置のハードウェア構成図である。It is a hardware block diagram of the gastric examination support device. 内視鏡システムのカメラユニットで内視鏡システムと胃検査支援装置を撮像する様子を示す図である。It is a figure which shows the state of taking an image of an endoscope system and a gastric examination support device with a camera unit of an endoscope system. システムモニタと表示モニタの相互関係を解析する手順を説明するための図である。It is a figure for demonstrating the procedure for analyzing the interrelationship between a system monitor and a display monitor. 決定される表示態様の一例を示す図である。It is a figure which shows an example of the display mode to be determined. 決定される表示態様の別の一例を示す図である。It is a figure which shows another example of the display mode to be determined. 決定される表示態様の別の一例を示す図である。It is a figure which shows another example of the display mode to be determined. 決定される表示態様の別の一例を示す図である。It is a figure which shows another example of the display mode to be determined. 決定される表示態様の別の一例を示す図である。It is a figure which shows another example of the display mode to be determined. 決定される表示態様の別の一例を示す図である。It is a figure which shows another example of the display mode to be determined. 学習済みモデルを用いた分析処理の手順を説明する図である。It is a figure explaining the procedure of the analysis process using a trained model. 演算処理部の処理手順を説明するフロー図である。It is a flow diagram explaining the processing procedure of the arithmetic processing part.
　以下、発明の実施の形態を通じて本発明を説明するが、特許請求の範囲に係る発明を以下の実施形態に限定するものではない。また、実施形態で説明する構成の全てが課題を解決するための手段として必須であるとは限らない。 Hereinafter, the present invention will be described through embodiments of the invention, but the invention according to the claims is not limited to the following embodiments. Moreover, not all of the configurations described in the embodiments are indispensable as means for solving the problem.
　図１は、内視鏡システム２００と本実施形態に係る検査装置１００を用いた内視鏡検査の様子を示す図である。内視鏡システム２００と検査装置１００は、共に診察スペースに設置されている。本実施形態においては、被検者の体内のうち胃を検査する場合を想定する。内視鏡システム２００は、カメラユニット２１０を備え、カメラユニット２１０は、図示するように、横たわる被検者の口腔から胃へ挿通され、胃の内部を撮像した画像の画像信号をシステム本体へ送信する。カメラユニット２１０の胃内への挿通や撮像操作は、医師によって行われる。 FIG. 1 is a diagram showing a state of endoscopy using the endoscope system 200 and the inspection device 100 according to the present embodiment. Both the endoscope system 200 and the inspection device 100 are installed in the examination space. In this embodiment, it is assumed that the stomach of the subject's body is examined. The endoscope system 200 includes a camera unit 210, which is inserted from the oral cavity of a lying subject into the stomach and transmits an image signal of an image of the inside of the stomach to the system body as shown in the figure. do. The insertion of the camera unit 210 into the stomach and the imaging operation are performed by a doctor.
　内視鏡システム２００は、例えば液晶パネルによって構成されるシステムモニタ２２０を備え、カメラユニット２１０から送られてきた画像信号を処理して視認可能な撮像画像２２１としてシステムモニタ２２０に表示する。また、内視鏡システム２００は、被検者情報やカメラユニット２１０のカメラ情報などを含む検査情報２２２をシステムモニタ２２０に表示する。 The endoscope system 200 includes, for example, a system monitor 220 composed of a liquid crystal panel, processes an image signal sent from the camera unit 210, and displays it on the system monitor 220 as a visible captured image 221. Further, the endoscope system 200 displays the inspection information 222 including the subject information and the camera information of the camera unit 210 on the system monitor 220.
　検査装置１００は、接続ケーブル２５０によって内視鏡システム２００に接続されている。内視鏡システム２００は、システムモニタ２２０へ送信する表示信号を、接続ケーブル２５０を介して検査装置１００へも送信する。検査装置１００は、例えば液晶パネルによって構成される表示モニタ１２０を備え、内視鏡システム２００から送られてきた表示信号から撮像画像２２１に対応する画像信号を抽出して視認可能な撮像画像１２１として表示モニタ１２０に表示する。また、検査装置１００は、撮像画像１２１の画像データを生成し、これを分析して診断補助情報１２２を出力し、表示モニタ１２０に表示する。検査装置１００は、内視鏡システム２００から表示信号を受け取り次第、あるいは医師によるカメラユニット２１０の操作を推定したタイミングに同期して撮像画像１２１や診断補助情報１２２の表示を実行する。すなわち、医師は検査の進行に同期して逐次的に撮像画像１２１や診断補助情報１２２を確認することができる。 The inspection device 100 is connected to the endoscope system 200 by a connection cable 250. The endoscope system 200 also transmits a display signal to be transmitted to the system monitor 220 to the inspection device 100 via the connection cable 250. The inspection device 100 includes, for example, a display monitor 120 composed of a liquid crystal panel, and extracts an image signal corresponding to the captured image 221 from the display signal sent from the endoscope system 200 to obtain a visible captured image 121. It is displayed on the display monitor 120. Further, the inspection device 100 generates image data of the captured image 121, analyzes it, outputs diagnostic assistance information 122, and displays it on the display monitor 120. The inspection device 100 displays the captured image 121 and the diagnostic assistance information 122 as soon as the display signal is received from the endoscope system 200 or in synchronization with the timing when the operation of the camera unit 210 by the doctor is estimated. That is, the doctor can sequentially confirm the captured image 121 and the diagnostic assistance information 122 in synchronization with the progress of the examination.
　さて、既存の内視鏡システム２００に検査装置１００を接続して利用する場合には、内視鏡システム２００のシステムモニタ２２０と検査装置１００の表示モニタ１２０が診察スペースに並置されることになる。したがって、医師は、被検者の胃内へ挿通するカメラユニット２１０を操作しながら２台のモニタを視認しつつ検査を進めることになる。ここで、２台のモニタは、診察スペースの都合等によって様々に配置される可能性があり、両モニタの配置関係と表示モニタ１２０の表示態様によっては、両モニタを行き来する視線の移動量が大きくなるなど、表示される情報の視認性が著しく低下する場合がある。そこで、本実施形態における検査装置１００は、自身が備える表示モニタ１２０と内視鏡システム２００に付随するシステムモニタ２２０との相互関係を解析し、表示モニタ１２０における表示態様を自動的に最適化する。以下に、検査装置１００の構成、処理手順、表示態様の決定等について順次説明する。 When the inspection device 100 is connected to the existing endoscope system 200 and used, the system monitor 220 of the endoscope system 200 and the display monitor 120 of the inspection device 100 are juxtaposed in the examination space. .. Therefore, the doctor will proceed with the examination while visually observing the two monitors while operating the camera unit 210 inserted into the stomach of the subject. Here, the two monitors may be arranged in various ways depending on the convenience of the examination space and the like, and the amount of movement of the line of sight between the two monitors may vary depending on the arrangement relationship between the two monitors and the display mode of the display monitor 120. The visibility of the displayed information may be significantly reduced, such as when the size is increased. Therefore, the inspection device 100 in the present embodiment analyzes the mutual relationship between the display monitor 120 provided by the inspection device 100 and the system monitor 220 attached to the endoscope system 200, and automatically optimizes the display mode in the display monitor 120. .. Hereinafter, the configuration of the inspection device 100, the processing procedure, the determination of the display mode, and the like will be sequentially described.
　図２は、検査装置１００のハードウェア構成図である。検査装置１００は、主に、演算処理部１１０、表示モニタ１２０、入出力インタフェース１３０、入力デバイス１４０、記憶部１５０によって構成される。演算処理部１１０は、検査装置１００の制御とプログラムの実行処理を行うプロセッサ（ＣＰＵ：Central Processing Unit）である。プロセッサは、ＡＳＩＣ（Application Specific Integrated Circuit）やＧＰＵ（Graphics Processing Unit）等の演算処理チップと連携する構成であってもよい。演算処理部１１０は、記憶部１５０に記憶された検査支援プログラムを読み出して、検査の支援に関する様々な処理を実行する。 FIG. 2 is a hardware configuration diagram of the inspection device 100. The inspection device 100 is mainly composed of an arithmetic processing unit 110, a display monitor 120, an input / output interface 130, an input device 140, and a storage unit 150. The arithmetic processing unit 110 is a processor (CPU: Central Processing Unit) that controls the inspection device 100 and executes a program. The processor may be configured to cooperate with an arithmetic processing chip such as an ASIC (Application Specific Integrated Circuit) or a GPU (Graphics Processing Unit). The arithmetic processing unit 110 reads out the inspection support program stored in the storage unit 150 and executes various processes related to inspection support.
　表示モニタ１２０は、上述のように、例えば液晶パネルを備えるモニタであり、撮像画像１２１や診断補助情報１２２を視認可能に表示する。入出力インタフェース１３０は、接続ケーブル２５０を接続するためのコネクタを含む、外部機器との間で情報を授受するための接続インタフェースである。入出力インタフェース１３０は、例えばＬＡＮユニットを含み、検査支援プログラムや後述する分析用ニューラルネットワーク１５１の更新データを外部機器から取り込んで演算処理部１１０へ引き渡す。 As described above, the display monitor 120 is a monitor provided with, for example, a liquid crystal panel, and visually displays the captured image 121 and the diagnostic assistance information 122. The input / output interface 130 is a connection interface for exchanging information with an external device, including a connector for connecting the connection cable 250. The input / output interface 130 includes, for example, a LAN unit, and takes in the update data of the inspection support program and the analysis neural network 151 described later from an external device and delivers it to the arithmetic processing unit 110.
　入力デバイス１４０は、例えばキーボードやマウス、表示モニタ１２０に重畳されたタッチパネルであり、医師や補助者は、これらを操作して検査装置１００の設定を変更したり、検査に必要な情報を入力したりする。 The input device 140 is, for example, a touch panel superimposed on a keyboard, a mouse, or a display monitor 120, and a doctor or an assistant operates these to change the settings of the inspection device 100 or input information necessary for the inspection. Or something.
　記憶部１５０は、不揮発性の記憶媒体であり、例えばＨＤＤ（Hard Disk Drive）によって構成されている。記憶部１５０は、検査装置１００の制御や処理を実行するプログラムの他にも、制御や演算に用いられる様々なパラメータ値、関数、表示要素データ、ルックアップテーブル等を記憶し得る。記憶部１５０は、特に、分析用ニューラルネットワーク１５１を記憶している。分析用ニューラルネットワーク１５１は、カメラユニット２１０が撮像した画像データを入力すると、その画像内に病変が存在する確率を算出する学習済みモデルである。なお、記憶部１５０は、複数のハードウェアで構成されていても良く、例えば、プログラムを記憶する記憶媒体と分析用ニューラルネットワーク１５１を記憶する記憶媒体が別々のハードウェアで構成されてもよい。 The storage unit 150 is a non-volatile storage medium, and is composed of, for example, an HDD (Hard Disk Drive). The storage unit 150 can store various parameter values, functions, display element data, look-up tables, etc. used for control and calculation, in addition to a program that executes control and processing of the inspection device 100. The storage unit 150 stores, in particular, the neural network 151 for analysis. The neural network 151 for analysis is a trained model that calculates the probability that a lesion exists in the image when the image data captured by the camera unit 210 is input. The storage unit 150 may be composed of a plurality of hardware. For example, the storage medium for storing the program and the storage medium for storing the analysis neural network 151 may be composed of different hardware.
　演算処理部１１０は、検査支援プログラムが指示する処理に応じて様々な演算を実行する機能演算部としての役割も担う。演算処理部１１０は、取得部１１１、解析部１１２、決定部１１３、診断補助部１１４、表示制御部１１５として機能し得る。取得部１１１は、内視鏡システム２００から送られてくる表示信号を処理することにより、カメラユニット２１０で撮像された画像を撮像画像１２１として再現し取得する。解析部１１２は、被検者の胃の検査の実行に先立って取得部１１１が取得した撮像画像１２１に写り込んだ表示モニタ１２０とシステムモニタ２２０の相互関係を解析する。決定部１１３は、解析部１１２が解析した相互関係に基づいて表示モニタ１２０の表示態様を決定する。診断補助部１１４は、記憶部１５０から読み出した分析用ニューラルネットワーク１５１へ胃の検査の実行中に取得部１１１が取得した撮像画像１２１を入力し、病変が存在する確率を算出させて、診断補助情報を生成する。表示制御部１１５は、決定部１１３が決定した表示態様に従って表示モニタ１２０の表示を制御する。 The arithmetic processing unit 110 also plays a role as a functional arithmetic unit that executes various operations according to the processing instructed by the inspection support program. The arithmetic processing unit 110 can function as an acquisition unit 111, an analysis unit 112, a determination unit 113, a diagnostic assistance unit 114, and a display control unit 115. The acquisition unit 111 reproduces and acquires the image captured by the camera unit 210 as the captured image 121 by processing the display signal sent from the endoscope system 200. The analysis unit 112 analyzes the interrelationship between the display monitor 120 and the system monitor 220 reflected in the captured image 121 acquired by the acquisition unit 111 prior to the execution of the examination of the stomach of the subject. The determination unit 113 determines the display mode of the display monitor 120 based on the interrelationship analyzed by the analysis unit 112. The diagnostic assistance unit 114 inputs the captured image 121 acquired by the acquisition unit 111 during the execution of the gastric examination into the analytical neural network 151 read from the storage unit 150, calculates the probability of the presence of a lesion, and assists the diagnosis. Generate information. The display control unit 115 controls the display of the display monitor 120 according to the display mode determined by the determination unit 113.
　図３は、内視鏡システム２００のカメラユニット２１０で内視鏡システム２００と検査装置１００を撮像する様子を示す図である。医師は、内視鏡システム２００および検査装置１００の起動後であって被検者の胃の検査を始める前に、少なくともシステムモニタ２２０の一部と表示モニタ１２０の一部が共に画角内に収まるように、内視鏡システム２００と検査装置１００を撮像する。図３においては、システムモニタ２２０の全部と表示モニタ１２０の全部が共に画角内に収まっている様子を示している。 FIG. 3 is a diagram showing a state in which the camera unit 210 of the endoscope system 200 images the endoscope system 200 and the inspection device 100. The doctor said that at least a part of the system monitor 220 and a part of the display monitor 120 were in the angle of view after starting the endoscope system 200 and the inspection device 100 and before starting the examination of the stomach of the subject. The endoscope system 200 and the inspection device 100 are imaged so as to fit. FIG. 3 shows how all of the system monitor 220 and all of the display monitor 120 are within the angle of view.
　カメラユニット２１０で撮像された画像は、内視鏡システム２００で処理されて、システムモニタ２２０の表示領域の一部に視認可能な撮像画像２２１として表示される。検査装置１００の表示制御部１１５は、内視鏡システム２００から受け取った表示信号を、システムモニタ２２０で表示されている表示態様と同様にそのまま表示モニタ１２０に表示する。 The image captured by the camera unit 210 is processed by the endoscope system 200 and displayed as a visible captured image 221 in a part of the display area of the system monitor 220. The display control unit 115 of the inspection device 100 displays the display signal received from the endoscope system 200 on the display monitor 120 as it is in the same manner as the display mode displayed on the system monitor 220.
　並行して取得部１１１は、表示信号から生成される表示画面から撮像画像１２１の領域を抽出する。表示制御部１１５は、取得部１１１が抽出した撮像画像１２１の領域がわかるように、当該領域の四隅に枠状の領域指標１２３を重畳表示させる。領域指標１２３が一旦重畳表示されると、その後にカメラユニット２１０で撮像される撮像画像には、被写体像として写り込む表示モニタ１２０に領域指標１２３が現れることになる。 In parallel, the acquisition unit 111 extracts the area of the captured image 121 from the display screen generated from the display signal. The display control unit 115 superimposes and displays a frame-shaped area index 123 on the four corners of the region so that the region of the captured image 121 extracted by the acquisition unit 111 can be seen. Once the area index 123 is superimposed and displayed, the area index 123 will appear on the display monitor 120 which is reflected as a subject image in the captured image subsequently captured by the camera unit 210.
　カメラユニット２１０による撮像は、この間継続して実行されており、例えば６０ｆｐｓで撮像画像が内視鏡システム２００へ逐次送られる。また、内視鏡システム２００はこれに応じて表示信号を逐次生成している。したがって、表示信号から生成される表示画面はフレーム画像であり、検査装置１００の取得部１１１は、フレーム画像から撮像画像１２１の領域を抽出する。具体的には、カメラユニット２１０で撮像される撮像画像が多少なりとも逐次変化することを利用して、前後する複数のフレーム画像の差分を検出し、所定量以上の変化が検出される領域を撮像画像１２１の領域であると判断する。解析部１１２は、このようにフレーム画像から抽出された撮像画像１２１を利用してシステムモニタ２２０と表示モニタ１２０の相互関係を解析する。 The image pickup by the camera unit 210 is continuously executed during this period, and the captured images are sequentially sent to the endoscope system 200 at, for example, 60 fps. Further, the endoscope system 200 sequentially generates display signals accordingly. Therefore, the display screen generated from the display signal is a frame image, and the acquisition unit 111 of the inspection device 100 extracts the region of the captured image 121 from the frame image. Specifically, by utilizing the fact that the captured image captured by the camera unit 210 changes sequentially to some extent, the difference between a plurality of frame images before and after is detected, and a region in which a change of a predetermined amount or more is detected is detected. It is determined that the area is the captured image 121. The analysis unit 112 analyzes the mutual relationship between the system monitor 220 and the display monitor 120 by using the captured image 121 extracted from the frame image in this way.
　図４は、システムモニタ２２０と表示モニタ１２０の相互関係を解析する手順を説明するための図である。上述のように、内視鏡システム２００から送られてくる表示信号を処理してフレーム画像を取得したら、取得部１１１は、当該フレーム画像から撮像画像１２１の領域を抽出して切り出し、解析部１１２へ引き渡す。 FIG. 4 is a diagram for explaining a procedure for analyzing the mutual relationship between the system monitor 220 and the display monitor 120. As described above, after processing the display signal sent from the endoscope system 200 to acquire the frame image, the acquisition unit 111 extracts and cuts out the region of the captured image 121 from the frame image, and the analysis unit 112. Hand over to.
　解析部１１２は、撮像画像１２１に写り込んだ表示モニタ１２０とシステムモニタ２２０のそれぞれの領域である表示モニタ領域１２０ａとシステムモニタ領域２２０ａを決定する。具体的には、予め用意された一般的なモニタ外観画像とのマッチング処理によって両モニタ領域を決定してもよいし、予め多数のモニタ画像を学習した学習済みモデルに撮像画像１２１を入力して両モニタ領域を決定してもよい。ここで、解析部１１２は、モニタ領域として抽出した２つの領域のうち、領域指標１２３が写り込んでいる領域を表示モニタ領域１２０ａと決定し、写り込んでいない領域をシステムモニタ領域２２０ａと決定する。 The analysis unit 112 determines the display monitor area 120a and the system monitor area 220a, which are the respective areas of the display monitor 120 and the system monitor 220 reflected in the captured image 121. Specifically, both monitor areas may be determined by matching processing with a general monitor appearance image prepared in advance, or the captured image 121 is input to a trained model in which a large number of monitor images have been learned in advance. Both monitor areas may be determined. Here, the analysis unit 112 determines the area in which the area index 123 is reflected as the display monitor area 120a and the area in which the area index 123 is not reflected as the system monitor area 220a among the two areas extracted as the monitor area. ..
　本実施形態においては上述のように、表示制御部１１５は、撮像画像１２１の四隅に領域指標１２３を重畳させている。このように領域指標１２３が重畳されていると、フレーム画像から切り出された撮像画像１２１と領域指標１２３に囲まれた領域の画像とがほぼ相似関係となるので、解析部１１２による表示モニタ領域１２０ａの決定がしやすくなる。なお、重畳させる領域指標１２３は本実施形態のような枠状でなくてもよいし、相似関係を利用して表示モニタ領域１２０ａを決定するのでなければ、撮像画像１２１をしめす指標でなくても構わない。例えば、表示画面の全体に大きな「〇」印を重畳してもよいし、表示領域に指標を重畳させるのではなく、表示モニタ１２０のベゼル部に印刷した指標を貼着しておいてもよい。撮像画像１２１において両モニタを区別し得る目印となる指標であれば、いずれの態様であっても構わない。 In the present embodiment, as described above, the display control unit 115 superimposes the area index 123 on the four corners of the captured image 121. When the area index 123 is superimposed in this way, the captured image 121 cut out from the frame image and the image of the area surrounded by the area index 123 have a substantially similar relationship, so that the display monitor area 120a by the analysis unit 112 It becomes easier to make a decision. The area index 123 to be superimposed does not have to have a frame shape as in the present embodiment, and may not be an index showing the captured image 121 unless the display monitor area 120a is determined by using the similarity relationship. I do not care. For example, a large "○" mark may be superimposed on the entire display screen, or the index printed on the bezel portion of the display monitor 120 may be attached instead of superimposing the index on the display area. .. Any mode may be used as long as it is an index that can distinguish between the two monitors in the captured image 121.
　解析部１１２は、撮像画像１２１から表示モニタ領域１２０ａとシステムモニタ領域２２０ａを決定したら、表示モニタ１２０とシステムモニタ２２０の相互関係を解析する。ここでは、表示モニタ１２０とシステムモニタ２２０の位置関係を解析する。図４の例では、解析部１１２は、表示モニタ１２０がシステムモニタ２２０の略右側に位置することを見出してその結果を決定部１１３へ引き渡す。例えば、表示モニタ領域１２０ａとシステムモニタ領域２２０ａのそれぞれの重心座標を算出し、その重心座標間を結ぶベクトル値を位置関係の解析結果として決定部１１３へ引き渡す。決定部１１３は、解析部１１２から受け取った解析結果に基づいて表示モニタ１２０の表示態様を決定する。以下に表示態様のいくつかの例を説明する。 After determining the display monitor area 120a and the system monitor area 220a from the captured image 121, the analysis unit 112 analyzes the mutual relationship between the display monitor 120 and the system monitor 220. Here, the positional relationship between the display monitor 120 and the system monitor 220 is analyzed. In the example of FIG. 4, the analysis unit 112 finds that the display monitor 120 is located substantially to the right of the system monitor 220 and delivers the result to the determination unit 113. For example, the coordinates of the center of gravity of each of the display monitor area 120a and the system monitor area 220a are calculated, and the vector value connecting the coordinates of the center of gravity is passed to the determination unit 113 as the analysis result of the positional relationship. The determination unit 113 determines the display mode of the display monitor 120 based on the analysis result received from the analysis unit 112. Some examples of display modes will be described below.
　図５は、決定部１１３によって決定される表示態様の一例を示す図である。決定部１１３が以下に説明するように表示要素の表示態様を決定すると、表示制御部１１５は、その決定に従って表示信号を生成し、各表示要素を表示モニタ１２０に表示させる。図５（Ａ）は、表示モニタ１２０がシステムモニタ２２０に対して右側に位置する場合の表示態様を示し、図５（Ｂ）は、表示モニタ１２０がシステムモニタ２２０に対して左側に位置する場合の表示態様を示す。ここでは、カメラユニット２１０が被検者の胃内に挿通され検査を実行している最中の表示画面の表示態様について説明する。 FIG. 5 is a diagram showing an example of a display mode determined by the determination unit 113. When the determination unit 113 determines the display mode of the display element as described below, the display control unit 115 generates a display signal according to the determination and causes each display element to be displayed on the display monitor 120. FIG. 5A shows a display mode when the display monitor 120 is located on the right side with respect to the system monitor 220, and FIG. 5B shows a display mode when the display monitor 120 is located on the left side with respect to the system monitor 220. Indicates the display mode of. Here, a display mode of the display screen while the camera unit 210 is inserted into the stomach of the subject and the examination is being performed will be described.
　図５（Ａ）に示すように、表示モニタ１２０がシステムモニタ２２０に対して右側に位置する場合は、表示モニタ１２０の表示領域のうち中心よりシステムモニタ２２０が位置する左側へ偏位した領域に固有情報を表示させる。固有情報は、システムモニタ２２０に表示されておらず表示モニタ１２０にのみ表示する情報であり、具体的には、検査装置１００の演算処理部１１０の処理によって生成された情報である。 As shown in FIG. 5A, when the display monitor 120 is located on the right side with respect to the system monitor 220, the display area of the display monitor 120 is displaced to the left side where the system monitor 220 is located from the center. Display unique information. The unique information is information that is not displayed on the system monitor 220 but is displayed only on the display monitor 120, and specifically, is information generated by the processing of the arithmetic processing unit 110 of the inspection device 100.
　ここでは、固有情報が、胃の検査情報の少なくとも一部であって、胃の検査の実行中に取得される撮像画像１２１に基づいて分析された診断補助情報である場合について説明する。表示モニタ１２０は、診断補助部１１４によって算出された撮像画像１２１内に病変が存在する推定確率を診断補助情報として表示する。診断補助情報は、図示するように、推定確率を示す数値情報１２２ａ、タイトルテキスト１２２ｂ、円グラフ状のグラフィックス１２２ｃを含む。 Here, a case where the unique information is at least a part of the gastric examination information and is the diagnostic auxiliary information analyzed based on the captured image 121 acquired during the execution of the gastric examination will be described. The display monitor 120 displays the estimated probability of the presence of a lesion in the captured image 121 calculated by the diagnostic assistance unit 114 as diagnostic assistance information. As shown in the figure, the diagnostic aid information includes numerical information 122a indicating an estimated probability, title text 122b, and pie chart graphics 122c.
　このように診断補助情報１２２をシステムモニタ２２０側へ寄せて表示し、システムモニタ２２０にもほぼ同様の表示がある撮像画像１２１をシステムモニタ２２０から遠い位置に表示すれば、両モニタに対する医師の視認性を高めることができる。すなわち、重要な情報を視認する医師の視線移動を短くすることができ、ひいては快適な検査環境の実現に寄与する。なお、表示モニタ１２０に表示する撮像画像１２１は、診断補助部１１４が推定確率の算出に用いた元画像であってもよい。この場合、表示制御部１１５は、診断補助情報１２２を表示する一定期間の間、撮像画像１２１を静止画像として表示するとよい。 In this way, if the diagnostic assistance information 122 is displayed closer to the system monitor 220 side and the captured image 121 having almost the same display on the system monitor 220 is displayed at a position far from the system monitor 220, the doctor can visually recognize both monitors. It can enhance the sex. That is, it is possible to shorten the movement of the line of sight of the doctor who visually recognizes important information, which in turn contributes to the realization of a comfortable examination environment. The captured image 121 displayed on the display monitor 120 may be the original image used by the diagnostic assistance unit 114 to calculate the estimation probability. In this case, the display control unit 115 may display the captured image 121 as a still image for a certain period of time for displaying the diagnostic assistance information 122.
　図５（Ｂ）に示すように、表示モニタ１２０がシステムモニタ２２０に対して左側に位置する場合は、表示モニタ１２０の表示領域のうち中心よりシステムモニタ２２０が位置する右側へ偏位した領域に固有情報を表示させる。ここでは、図５（Ａ）の場合と同様に、固有情報として診断補助情報１２２をシステムモニタ２２０側へ寄せて表示し、撮像画像１２１をシステムモニタ２２０から遠い位置に表示している。 As shown in FIG. 5B, when the display monitor 120 is located on the left side with respect to the system monitor 220, the display area of the display monitor 120 is displaced from the center to the right side where the system monitor 220 is located. Display unique information. Here, as in the case of FIG. 5A, the diagnostic assistance information 122 is displayed closer to the system monitor 220 side as unique information, and the captured image 121 is displayed at a position far from the system monitor 220.
　ここで、システムモニタ２２０の側へ偏位させて表示させる表示要素は、撮像画像に基づいて分析された診断補助情報に限らず、他の検査情報であってもよい。検査情報としては、例えば、撮像すべき胃内の部位をサンプル画像等によって順次示す検査ガイドであってもよい。また、検査情報に限らず、システムモニタ２２０に表示されない、検査装置１００で生成された固有情報であってもよい。例えば、入力デバイス１４０によって入力された被検者情報であってもよい。いずれの表示要素をシステムモニタ２２０側へ偏位させて表示させるかについて、使用者である医師が事前に選択できるように構成してもよい。 Here, the display element displayed by deviating to the side of the system monitor 220 is not limited to the diagnostic auxiliary information analyzed based on the captured image, but may be other inspection information. The examination information may be, for example, an examination guide that sequentially indicates a portion in the stomach to be imaged by a sample image or the like. Further, the information is not limited to the inspection information, and may be unique information generated by the inspection device 100 that is not displayed on the system monitor 220. For example, it may be the subject information input by the input device 140. It may be configured so that the doctor who is the user can select in advance which display element is shifted to the system monitor 220 side for display.
　図６は、決定部１１３によって決定される表示態様の別の一例を示す図である。具体的には、表示モニタ１２０がシステムモニタ２２０に対して上側に位置する場合の表示態様を示す。図６の表示態様は、図５（Ａ）および図５（Ｂ）の場合と同様に、カメラユニット２１０が被検者の胃内に挿通され検査を実行している最中の表示態様であり、表示される表示要素も同様である。 FIG. 6 is a diagram showing another example of the display mode determined by the determination unit 113. Specifically, the display mode when the display monitor 120 is located above the system monitor 220 is shown. The display mode of FIG. 6 is a display mode in which the camera unit 210 is inserted into the stomach of the subject and the examination is being performed, as in the case of FIGS. 5 (A) and 5 (B). , The displayed display elements are the same.
　図６に示すように、表示モニタ１２０がシステムモニタ２２０に対して上側に位置する場合は、表示モニタ１２０の表示領域のうち中心よりシステムモニタ２２０が位置する下側へ偏位した領域に固有情報を表示させる。ここでは、固有情報として診断補助情報１２２をシステムモニタ２２０側へ寄せて表示し、撮像画像１２１をシステムモニタ２２０から遠い位置に表示している。ただし、両表示要素が重なり合わないように、大きさを変えずにそれぞれを左右にずらして表示している。このように、表示モニタ１２０がシステムモニタ２２０に対して上下方向に位置する場合であっても、診断補助情報１２２をシステムモニタ２２０側へ偏位させることができるので、医師の視線移動を短くすることができる。 As shown in FIG. 6, when the display monitor 120 is located above the system monitor 220, the information unique to the display area of the display monitor 120 that is displaced downward from the center where the system monitor 220 is located. Is displayed. Here, the diagnostic assistance information 122 is displayed closer to the system monitor 220 side as unique information, and the captured image 121 is displayed at a position far from the system monitor 220. However, in order to prevent both display elements from overlapping, they are displayed by shifting them to the left and right without changing their sizes. In this way, even when the display monitor 120 is positioned in the vertical direction with respect to the system monitor 220, the diagnostic assistance information 122 can be displaced toward the system monitor 220, so that the movement of the doctor's line of sight is shortened. be able to.
　図７は、決定部１１３によって決定される表示態様の別の一例を示す図である。具体的には、表示モニタ１２０がシステムモニタ２２０に対して左上側に位置する場合の表示態様を示す。図７の表示態様は、図５（Ａ）および図５（Ｂ）の場合と同様に、カメラユニット２１０が被検者の胃内に挿通され検査を実行している最中の表示態様であり、表示される表示要素も同様である。 FIG. 7 is a diagram showing another example of the display mode determined by the determination unit 113. Specifically, the display mode when the display monitor 120 is located on the upper left side with respect to the system monitor 220 is shown. The display mode of FIG. 7 is a display mode in which the camera unit 210 is inserted into the stomach of the subject and the examination is being performed, as in the case of FIGS. 5 (A) and 5 (B). , The displayed display elements are the same.
　図７に示すように、表示モニタ１２０がシステムモニタ２２０に対して左上側に位置する場合は、表示モニタ１２０の表示領域のうち中心よりシステムモニタ２２０が位置する右下側へ偏位した領域に固有情報を表示させる。ここでは、固有情報として診断補助情報１２２をシステムモニタ２２０側へ寄せて表示し、撮像画像１２１をシステムモニタ２２０から遠い位置に表示している。特に、より重要な情報である数値情報１２２ａをさらにシステムモニタ２２０側へ寄せて表示している。このように表示することにより、表示モニタ１２０がシステムモニタ２２０に対して斜め方向に位置する場合であっても、医師の視線移動を短くすることができる。特に、数値情報１２２ａをシステムモニタ２２０側へより偏位させることにより、医師の視線移動を更に短くすることができる。なお、診断補助情報１２２のうち、他の要素は基準位置に表示させて数値情報１２２ａのみをシステムモニタ２２０側へ偏位させて表示してもよい。また、数値情報１２２ａのみをシステムモニタ２２０側へ偏位させる表示態様は、両モニタの位置関係が斜め方向である場合に限らず、図５や図６のような上下左右方向に並ぶ場合であっても採用し得る。 As shown in FIG. 7, when the display monitor 120 is located on the upper left side of the system monitor 220, it is located in the lower right side of the display area of the display monitor 120 where the system monitor 220 is located. Display unique information. Here, the diagnostic assistance information 122 is displayed closer to the system monitor 220 side as unique information, and the captured image 121 is displayed at a position far from the system monitor 220. In particular, the numerical information 122a, which is more important information, is displayed closer to the system monitor 220 side. By displaying in this way, even when the display monitor 120 is positioned obliquely with respect to the system monitor 220, the movement of the line of sight of the doctor can be shortened. In particular, by further deviating the numerical information 122a toward the system monitor 220, the movement of the doctor's line of sight can be further shortened. In addition, among the diagnostic aid information 122, other elements may be displayed at the reference position and only the numerical information 122a may be shifted to the system monitor 220 side and displayed. Further, the display mode in which only the numerical information 122a is displaced toward the system monitor 220 is not limited to the case where the positional relationship between the two monitors is in the diagonal direction, but also when they are arranged in the vertical and horizontal directions as shown in FIGS. 5 and 6. Can be adopted.
　図８は、決定部１１３によって決定される表示態様の別の一例を示す図である。図５から図７に示す表示態様の例は、表示モニタ１２０とシステムモニタ２２０が互いにほぼ同じ大きさのモニタであって、それぞれが医師からほぼ等距離の位置に並べられる場合を想定した。しかし、表示モニタ１２０とシステムモニタ２２０は互いに大きさが異なる場合もあるし、奥行き方向にずれて配置される場合もある。図８に示す表示態様の例は、表示モニタ１２０がシステムモニタ２２０に対して右側に位置する点で図５（Ａ）と同様であるものの、表示モニタ１２０が医師に対してシステムモニタ２２０よりも遠い位置に配置されている場合におけるものである。 FIG. 8 is a diagram showing another example of the display mode determined by the determination unit 113. In the example of the display mode shown in FIGS. 5 to 7, it is assumed that the display monitor 120 and the system monitor 220 are monitors having substantially the same size as each other and are arranged at positions approximately equidistant from the doctor. However, the display monitor 120 and the system monitor 220 may be different in size from each other, or may be arranged so as to be displaced in the depth direction. The example of the display mode shown in FIG. 8 is similar to FIG. 5 (A) in that the display monitor 120 is located on the right side of the system monitor 220, but the display monitor 120 is larger than the system monitor 220 for the doctor. This is the case when it is placed at a distant position.
　解析部１１２は、撮像画像１２１から表示モニタ領域１２０ａがシステムモニタ領域２２０ａよりも右側に存在すること、および表示モニタ領域１２０ａがシステムモニタ領域２２０ａよりも小さいことを解析したら、その結果を決定部１１３へ引き渡す。決定部１１３は、当該結果から、表示モニタ１２０がシステムモニタ２２０に対して右側に位置することと、表示モニタ１２０がシステムモニタ２２０よりも遠くに位置する、または表示モニタ１２０がシステムモニタ２２０よりも小さいことを認識し、図８に示す表示態様を決定する。 When the analysis unit 112 analyzes from the captured image 121 that the display monitor area 120a is on the right side of the system monitor area 220a and that the display monitor area 120a is smaller than the system monitor area 220a, the analysis unit 112 determines the result. Hand over to. From the result, the determination unit 113 determines that the display monitor 120 is located on the right side of the system monitor 220, the display monitor 120 is located farther than the system monitor 220, or the display monitor 120 is located closer to the system monitor 220. Recognizing that it is small, the display mode shown in FIG. 8 is determined.
　具体的には、システムモニタ２２０と同じ大きさの表示モニタ１２０が互いに隣接して配置されている場合に比べて診断補助情報１２２が表示領域に占める割合を大きくし、大きくした診断補助情報１２２をシステムモニタ２２０が位置する左側へ偏位させる表示態様に決定する。このとき、診断補助情報１２２を構成する内部要素のレイアウトを変更してもよい。例えば、図８に示すように、タイトルテキスト１２２ｂについては大きさを変更せず、位置をシステムモニタ２２０側へずらしてもよい。また、システムモニタ２２０から遠い位置に表示する撮像画像１２１の一部を覆うように診断補助情報１２２を表示してもよい。 Specifically, the ratio of the diagnostic assistance information 122 to the display area is increased as compared with the case where the display monitors 120 having the same size as the system monitor 220 are arranged adjacent to each other, and the enlarged diagnostic assistance information 122 is provided. The display mode is determined so that the system monitor 220 is displaced to the left side where it is located. At this time, the layout of the internal elements constituting the diagnostic auxiliary information 122 may be changed. For example, as shown in FIG. 8, the size of the title text 122b may not be changed and the position may be shifted to the system monitor 220 side. Further, the diagnostic assistance information 122 may be displayed so as to cover a part of the captured image 121 displayed at a position far from the system monitor 220.
　図８の例とは逆に、解析部１１２から受け取った解析結果から決定部１１３が、表示モニタ１２０がシステムモニタ２２０よりも近くに位置する、または表示モニタ１２０がシステムモニタ２２０よりも大きいことを認識した場合には、診断補助情報１２２が表示領域に占める割合を小さくした表示態様に決定してもよい。この場合、生じた余白に更に固有情報を表示してもよい。 Contrary to the example of FIG. 8, from the analysis result received from the analysis unit 112, the determination unit 113 indicates that the display monitor 120 is located closer to the system monitor 220, or the display monitor 120 is larger than the system monitor 220. If it is recognized, the display mode may be determined so that the ratio of the diagnostic aid information 122 to the display area is reduced. In this case, the unique information may be further displayed in the generated margin.
　図９は、決定部１１３によって決定される表示態様の更に別の一例を示す図である。図５から図８の例においては、固有情報として診断補助情報１２２をシステムモニタ２２０側へ偏位させる表示態様であったが、図９の例は、撮像画像１２１に固有情報を重畳させる場合の表示態様である。 FIG. 9 is a diagram showing still another example of the display mode determined by the determination unit 113. In the examples of FIGS. 5 to 8, the diagnostic auxiliary information 122 is displaced toward the system monitor 220 as the unique information, but in the example of FIG. 9, the unique information is superimposed on the captured image 121. This is a display mode.
　分析用ニューラルネットワーク１５１が病変の嫌疑部位も特定できるものである場合には、撮像画像１２１のその部位に対して重畳情報１２１ａを重畳する。このように、撮像画像１２１に重畳情報１２１ａを重畳する場合には、重畳情報１２１ａを重畳した撮像画像１２１をシステムモニタ２２０側へ偏位させる。図９の例では、表示モニタ１２０がシステムモニタ２２０に対して右側に位置しているので、表示モニタ１２０の表示領域のうち中心より左側へ偏位した領域に重畳情報１２１ａを重畳した撮像画像１２１を表示している。 If the analytical neural network 151 can also identify the suspected lesion site, the superimposed information 121a is superimposed on that site of the captured image 121. In this way, when the superimposed information 121a is superimposed on the captured image 121, the captured image 121 on which the superimposed information 121a is superimposed is displaced toward the system monitor 220 side. In the example of FIG. 9, since the display monitor 120 is located on the right side with respect to the system monitor 220, the captured image 121 in which the superimposed information 121a is superimposed on the region shifted to the left from the center in the display region of the display monitor 120. Is displayed.
　図９の例では、システムモニタ２２０から遠い位置に表示している診断補助情報１２２も固有情報であるが、いずれの固有情報をシステムモニタ２２０側へ偏位して表示するかについては、予め設定されている優先順位によって決定されている。例えば、優先順位が設定された複数の固有情報が存在する場合には、表示対象となる選択された固有情報のうち最も順位の高い固有情報がシステムモニタ２２０側へ偏位して表示される。したがって、ある時点でシステムモニタ２２０側へ偏位して表示されていた固有情報が一定時間の後に消去される場合には、次に優先順位の高い固有情報が改めてシステムモニタ２２０側へ偏位して表示される。 In the example of FIG. 9, the diagnostic auxiliary information 122 displayed at a position far from the system monitor 220 is also unique information, but which unique information is shifted to the system monitor 220 side and displayed is set in advance. It is determined by the priorities that have been set. For example, when there is a plurality of unique information for which priority is set, the highest-ranked unique information among the selected unique information to be displayed is biased and displayed on the system monitor 220 side. Therefore, if the unique information displayed by shifting to the system monitor 220 side at a certain point is deleted after a certain period of time, the next higher priority unique information is shifted to the system monitor 220 side again. Is displayed.
　図１０は、決定部１１３によって決定される表示態様の更に別の一例を示す図である。解析部１１２は、撮像画像１２１から表示モニタ領域１２０ａとシステムモニタ領域２２０ａの相対的な位置関係を解析することに加えて、システムモニタ２２０の表示に採用されている色調を解析してもよい。例えば、図１０に示すように、システムモニタ２２０の背景色として暗黒色が採用されている場合には、決定部１１３は、表示モニタ１２０の背景領域１２４を暗黒色にする。このとき、診断補助情報１２２の彩色も、背景領域１２４の色調に合わせて視認性の良いものに変更してもよい。また、表示要素の大きさを変更してもよい。 FIG. 10 is a diagram showing still another example of the display mode determined by the determination unit 113. In addition to analyzing the relative positional relationship between the display monitor area 120a and the system monitor area 220a from the captured image 121, the analysis unit 112 may analyze the color tone adopted for the display of the system monitor 220. For example, as shown in FIG. 10, when dark black is adopted as the background color of the system monitor 220, the determination unit 113 makes the background area 124 of the display monitor 120 dark black. At this time, the coloring of the diagnostic assistance information 122 may be changed to one having good visibility according to the color tone of the background area 124. Further, the size of the display element may be changed.
　なお、図１０の例では固有情報の偏位も行っているが、色調の調整等によって固有情報の視認性を高めることができる場合などにおいては、固有情報の偏位を省いてもよい。この場合、解析部１１２は、表示モニタ領域１２０ａとシステムモニタ領域２２０ａの相対的な位置関係の解析を行わなくてもよい。 Although the deviation of the unique information is also performed in the example of FIG. 10, the deviation of the unique information may be omitted when the visibility of the unique information can be improved by adjusting the color tone or the like. In this case, the analysis unit 112 does not have to analyze the relative positional relationship between the display monitor area 120a and the system monitor area 220a.
　また、システムモニタ２２０の表示態様を考慮する場合は、システムモニタ２２０の表示に採用されている色調に限らず、他の要素を考慮してもよい。例えば、解析部１１２がシステムモニタ２２０の表示に用いられているテキストフォントを解析すれば、決定部１１３は、当該テキストフォントに近似するテキストフォントを選択して用いるように決定してもよい。 Further, when considering the display mode of the system monitor 220, not only the color tone adopted for the display of the system monitor 220 but also other factors may be considered. For example, if the analysis unit 112 analyzes the text font used for displaying the system monitor 220, the determination unit 113 may determine to select and use a text font that is close to the text font.
　また、解析部１１２が表示モニタ１２０の解像度がシステムモニタ２２０の解像度よりも低いことを解析した場合には、決定部１１３は、表示要素の大きさを通常よりも大きくしてもよい。また、表示モニタ１２０の大きさや距離によって、表示要素の点滅、色彩変更、読上げ音声の併用などの強調手段を採用してもよい。 Further, when the analysis unit 112 analyzes that the resolution of the display monitor 120 is lower than the resolution of the system monitor 220, the determination unit 113 may make the size of the display element larger than usual. Further, depending on the size and distance of the display monitor 120, emphasizing means such as blinking of display elements, color change, and combined use of reading voice may be adopted.
　次に、診断補助部１１４による分析処理について説明する。図１１は、学習済みモデルである分析用ニューラルネットワーク１５１を用いた分析処理の手順を説明する図である。分析用ニューラルネットワーク１５１は、病変部位を含む画像と含まない画像とを、その正解と共に大量に与えられる教師あり学習によって予め作成されたものである。このように作成された分析用ニューラルネットワーク１５１は、記憶部１５０に格納され、診断補助部１１４により適宜読み出されて利用に供される。 Next, the analysis process by the diagnostic assistance unit 114 will be described. FIG. 11 is a diagram illustrating a procedure of analysis processing using the neural network 151 for analysis, which is a trained model. The neural network 151 for analysis is created in advance by supervised learning in which an image including a lesion site and an image not including the lesion site are given in large quantities together with the correct answer. The neural network 151 for analysis created in this way is stored in the storage unit 150, appropriately read out by the diagnostic assistance unit 114, and used.
　内視鏡システム２００は、被検者の胃内に挿通されたカメラユニット２１０から、上述のように例えば６０ｆｐｓで撮像画像を受け取るが、医師がレリーズ信号を発生させた場合には、静止画像を撮像画像として受け取る。この場合に検査装置１００が内視鏡システム２００から受信する表示信号は、表示信号としては６０ｆｐｓのフレーム信号であるものの、ここから展開される撮像画像１２１は、一定期間フリーズした疑似静止画像となる。 The endoscope system 200 receives a captured image from the camera unit 210 inserted into the stomach of the subject at, for example, 60 fps as described above, but when the doctor generates a release signal, a still image is displayed. Receive as an captured image. In this case, the display signal received by the inspection device 100 from the endoscope system 200 is a frame signal of 60 fps as a display signal, but the captured image 121 developed from this is a pseudo still image frozen for a certain period of time. ..
　取得部１１１は、この疑似静止画像を検知して撮像画像データを生成したら、当該撮像画像データを診断対象画像データとして診断補助部１１４へ引き渡す。診断補助部１１４は、受け取った撮像画像データを展開した撮像画像１２１を分析用ニューラルネットワーク１５１へ入力し、推定確率を算出させる。 When the acquisition unit 111 detects this pseudo still image and generates the captured image data, the acquisition unit 111 delivers the captured image data to the diagnosis assisting unit 114 as the image data to be diagnosed. The diagnostic assistance unit 114 inputs the captured image 121 developed from the received captured image data to the analysis neural network 151, and causes the estimation probability to be calculated.
　次に、検査装置１００を用いた検査支援方法の処理手順について説明する。図１２は、演算処理部１１０によって実行される処理手順を説明するフロー図である。フローは、例えば、内視鏡システム２００および検査装置１００が起動された時点から開始される。 Next, the processing procedure of the inspection support method using the inspection device 100 will be described. FIG. 12 is a flow chart illustrating a processing procedure executed by the arithmetic processing unit 110. The flow starts, for example, from the time when the endoscope system 200 and the inspection device 100 are activated.
　取得部１１１は、ステップＳ１０１で、内視鏡システム２００から送られてくる表示信号を取得しフレーム画像に展開する。そして、ステップＳ１０２で、展開されたフレーム画像から撮像画像１２１を切り出して解析部１１２へ引き渡す。 In step S101, the acquisition unit 111 acquires the display signal sent from the endoscope system 200 and develops it into a frame image. Then, in step S102, the captured image 121 is cut out from the expanded frame image and handed over to the analysis unit 112.
　解析部１１２は、ステップＳ１０３で、図４を用いて説明したように撮像画像１２１から表示モニタ領域１２０ａとシステムモニタ領域２２０ａを探索する。これらの領域を見つけ出したら、これらの座標等を用いて表示モニタ１２０とシステムモニタ２２０の相互関係を解析する。解析部１１２は、ステップＳ１０４へ進み、相互関係の解析に成功したか否かを判断する。撮像画像１２１から表示モニタ領域１２０ａとシステムモニタ領域２２０ａを発見できなかったり、発見できても相互関係を解析できなかったりした場合には、ステップＳ１０１へ戻る。そうでない場合には、解析結果を決定部１１３へ引き渡して、ステップＳ１０５へ進む。 In step S103, the analysis unit 112 searches the display monitor area 120a and the system monitor area 220a from the captured image 121 as described with reference to FIG. After finding these areas, the interrelationship between the display monitor 120 and the system monitor 220 is analyzed using these coordinates and the like. The analysis unit 112 proceeds to step S104 and determines whether or not the mutual relationship analysis has been successful. If the display monitor area 120a and the system monitor area 220a cannot be found from the captured image 121, or if the mutual relationship cannot be analyzed even if they can be found, the process returns to step S101. If not, the analysis result is handed over to the determination unit 113, and the process proceeds to step S105.
　決定部１１３は、解析部１１２から解析結果を受け取ると、ステップＳ１０５で、当該解析結果が示す表示モニタ１２０とシステムモニタ２２０の相互関係に基づいて表示モニタ１２０の表示態様を決定する。その後、ステップＳ１０６で、演算処理部１１０は、検査が開始されるまで待機する。演算処理部１１０は、例えば、取得された撮像画像１２１が口腔内の画像であると確認された時点で検査が開始されたと判断する。 Upon receiving the analysis result from the analysis unit 112, the determination unit 113 determines the display mode of the display monitor 120 based on the mutual relationship between the display monitor 120 and the system monitor 220 indicated by the analysis result in step S105. After that, in step S106, the arithmetic processing unit 110 waits until the inspection is started. The arithmetic processing unit 110 determines, for example, that the examination is started when the acquired captured image 121 is confirmed to be an image in the oral cavity.
　検査が開始されると、表示制御部１１５は、ステップＳ１０７で、決定部１１３によって決定された表示態様に従って表示信号を生成し、検査情報を含む各表示要素を表示モニタ１２０へ表示する。取得部１１１は、ステップＳ１０８で、表示信号から展開されたフレーム画像から撮像画像１２１を切り出して診断補助部１１４へ当該撮像画像１２１を引き渡す。診断補助部１１４は、ステップＳ１０９で、上述のように推定確率を算出し、表示制御部１１５へ引き渡す。演算処理部１１０は、ステップＳ１１０で、検査終了の指示を受け付けるまで、ステップＳ１０７からステップＳ１０９を繰り返す。検査終了の指示を受け付けたら、一連の処理を終了する。 When the inspection is started, the display control unit 115 generates a display signal according to the display mode determined by the determination unit 113 in step S107, and displays each display element including the inspection information on the display monitor 120. In step S108, the acquisition unit 111 cuts out the captured image 121 from the frame image developed from the display signal and delivers the captured image 121 to the diagnostic assistance unit 114. In step S109, the diagnosis assisting unit 114 calculates the estimated probability as described above and hands it over to the display control unit 115. The arithmetic processing unit 110 repeats steps S107 to S109 until an instruction to end the inspection is received in step S110. When the instruction to end the inspection is received, the series of processes is completed.
　以上説明した本実施形態においては、内視鏡システム２００と検査装置１００が接続ケーブル２５０を介して接続される場合を想定したが、有線接続でなく無線接続であっても構わない。また、内視鏡システム２００は表示信号を外部に出力し、検査装置１００はこの表示信号を利用する実施形態を説明したが、内視鏡システム２００が外部に出力する信号がカメラユニット２１０で撮像された撮像画像の画像信号を包含するのであれば、出力信号の形式は問わない。カメラユニット２１０が直接的に画像信号を発信するのであれば、検査装置１００は当該画像信号を受信するようにしてもよい。 In the present embodiment described above, it is assumed that the endoscope system 200 and the inspection device 100 are connected via the connection cable 250, but a wireless connection may be used instead of a wired connection. Further, the endoscope system 200 outputs a display signal to the outside, and the inspection device 100 has described an embodiment using this display signal. However, the signal output by the endoscope system 200 to the outside is captured by the camera unit 210. The format of the output signal does not matter as long as it includes the image signal of the captured image. If the camera unit 210 directly transmits an image signal, the inspection device 100 may receive the image signal.
　また、以上説明した本実施形態においては、内視鏡システム２００から検査装置１００へ一方向に信号を送信する場合を説明したが、内視鏡システム２００が対応可能であれば、検査装置１００から内視鏡システム２００へ何らかの要求信号を送信するように構成してもよい。例えば、検査装置１００に決定部１１３が決定した表示態様に基づいて、システムモニタ２２０の表示態様の変更を要求する要求信号を生成して送信する変更要求部を設けてもよい。変更要求部は、例えば、各々の診断補助情報１２２が表示モニタ１２０に表示される最初の所定時間の間は、システムモニタ２２０の輝度を下げるように要求する要求信号を送信する。このように、表示モニタ１２０の表示とシステムモニタ２２０の表示を連動できれば、より視認性を高めることができる。 Further, in the present embodiment described above, the case where the signal is transmitted from the endoscope system 200 to the inspection device 100 in one direction has been described, but if the endoscope system 200 can handle it, the inspection device 100 has been described. It may be configured to transmit some request signal to the endoscope system 200. For example, the inspection device 100 may be provided with a change request unit that generates and transmits a request signal requesting a change in the display mode of the system monitor 220 based on the display mode determined by the determination unit 113. The change request unit transmits, for example, a request signal requesting that the brightness of the system monitor 220 be reduced during the first predetermined time during which each diagnostic aid information 122 is displayed on the display monitor 120. In this way, if the display of the display monitor 120 and the display of the system monitor 220 can be linked, the visibility can be further improved.
　また、以上説明した本実施形態においては、検査対象が胃の例を説明したが、内視鏡システム２００は、胃内検査に特化したものでなくてもよく、他の部位を検査するものであってもよい。例えば十二指腸を検査する内視鏡システム２００に接続する検査支援装置１００は、十二指腸の撮像画像に病変部位が存在する確率を算出して表示する装置である。この場合、分析用ニューラルネットワーク１５１は、検査対象となる部位に応じたものが予め用意される。また、以上説明した本実施形態においては、内視鏡システム２００が備えるカメラユニット２１０が軟性内視鏡であることを想定して説明したが、カメラユニット２１０が硬性内視鏡であっても、検査支援装置１００の構成や処理手順に違いは何ら生じない。 Further, in the present embodiment described above, an example in which the examination target is the stomach has been described, but the endoscope system 200 does not have to be specialized for the gastric examination and inspects other parts. May be. For example, the inspection support device 100 connected to the endoscope system 200 for inspecting the duodenum is a device that calculates and displays the probability that a lesion site is present in the captured image of the duodenum. In this case, the neural network 151 for analysis is prepared in advance according to the site to be inspected. Further, in the present embodiment described above, the description has been made on the assumption that the camera unit 210 included in the endoscope system 200 is a flexible endoscope, but even if the camera unit 210 is a rigid endoscope, it may be described. There is no difference in the configuration or processing procedure of the inspection support device 100.
１００…検査支援装置、１１０…演算処理部、１１１…取得部、１１２…解析部、１１３…決定部、１１４…診断補助部、１１５…表示制御部、１２０…表示モニタ、１２０ａ…表示モニタ領域、１２１…撮像画像、１２１ａ…重畳情報、１２２…診断補助情報、１２２ａ…数値情報、１２２ｂ…タイトルテキスト、１２２ｃ…グラフィックス、１２３…領域指標、１２４…背景領域、１３０…入出力インタフェース、１４０…入力デバイス、１５０…記憶部、１５１…分析用ニューラルネットワーク、２００…内視鏡システム、２１０…カメラユニット、２２０…システムモニタ、２２０ａ…システムモニタ領域、２２１…撮像画像、２２２…検査情報、２５０…接続ケーブル 100 ... Inspection support device, 110 ... Arithmetic processing unit, 111 ... Acquisition unit, 112 ... Analysis unit, 113 ... Decision unit, 114 ... Diagnosis assistance unit, 115 ... Display control unit, 120 ... Display monitor, 120a ... Display monitor area, 121 ... captured image, 121a ... superimposed information, 122 ... diagnostic assistance information, 122a ... numerical information, 122b ... title text, 122c ... graphics, 123 ... area index, 124 ... background area, 130 ... input / output interface, 140 ... input Device, 150 ... Storage unit, 151 ... Neural network for analysis, 200 ... Endoscope system, 210 ... Camera unit, 220 ... System monitor, 220a ... System monitor area, 221 ... Image captured image, 222 ... Inspection information, 250 ... Connection cable

Claims (12)

  1. 　被検者の体内に挿通されるカメラユニットを備える内視鏡システムに接続して利用される検査支援装置であって、
    　前記カメラユニットで撮像された撮像画像を取得する取得部と、
    　前記撮像画像と体内検査に関する検査情報を表示するための表示モニタと、
    　前記体内検査の実行に先立って前記取得部が取得した前記撮像画像に写り込んだ前記表示モニタと前記内視鏡システムに付随するシステムモニタの相互関係を解析する解析部と、
    　前記解析部が解析した前記相互関係に基づいて前記表示モニタの表示態様を決定する決定部と、
    　前記決定部が決定した前記表示態様に従って前記表示モニタの表示を制御する表示制御部と
    を備える検査支援装置。     It is an inspection support device used by connecting to an endoscope system equipped with a camera unit to be inserted into the body of the subject.
    An acquisition unit that acquires an image captured by the camera unit,
    A display monitor for displaying the captured image and examination information related to the internal examination,
    An analysis unit that analyzes the interrelationship between the display monitor and the system monitor attached to the endoscope system, which are reflected in the captured image acquired by the acquisition unit prior to the execution of the internal examination.
    A determination unit that determines the display mode of the display monitor based on the interrelationship analyzed by the analysis unit, and a determination unit.
    An inspection support device including a display control unit that controls the display of the display monitor according to the display mode determined by the determination unit.
  2. 　前記解析部は、前記相互関係として前記表示モニタと前記システムモニタの位置関係を解析する請求項１に記載の検査支援装置。 The inspection support device according to claim 1, wherein the analysis unit analyzes the positional relationship between the display monitor and the system monitor as the mutual relationship.
  3. 　前記決定部は、前記システムモニタに表示されていない固有情報を、前記位置関係に基づいて前記表示モニタの表示領域のうち前記システムモニタの側へ偏位した領域に表示させる表示態様に決定する請求項２に記載の検査支援装置。 The determination unit determines a display mode in which unique information that is not displayed on the system monitor is displayed in an area of the display area of the display monitor that is displaced toward the system monitor based on the positional relationship. Item 2. The inspection support device according to item 2.
  4. 　前記固有情報は、前記検査情報の少なくとも一部である、前記体内検査の実行中に取得される前記撮像画像に基づいて分析された診断補助情報である請求項３に記載の検査支援装置。 The test support device according to claim 3, wherein the unique information is at least a part of the test information and is diagnostic assistance information analyzed based on the captured image acquired during the execution of the internal test.
  5. 　前記診断補助情報は、少なくとも病変の確率を示す数値を含む請求項４に記載の検査支援装置。 The test support device according to claim 4, wherein the diagnostic assistance information includes at least a numerical value indicating the probability of a lesion.
  6. 　前記決定部は、前記位置関係に基づいて表示要素の大きさを変更する請求項２から５のいずれか１項に記載の検査支援装置。 The inspection support device according to any one of claims 2 to 5, wherein the determination unit changes the size of a display element based on the positional relationship.
  7. 　前記決定部は、前記システムモニタにおける表示態様を考慮して前記表示モニタにおける表示態様を決定する請求項１から６のいずれか１項に記載の検査支援装置。 The inspection support device according to any one of claims 1 to 6, wherein the determination unit determines the display mode on the display monitor in consideration of the display mode on the system monitor.
  8. 　前記決定部は、前記システムモニタの表示に採用されている色調を考慮して前記表示モニタにおける色調を決定する請求項７に記載の検査支援装置。 The inspection support device according to claim 7, wherein the determination unit determines the color tone of the display monitor in consideration of the color tone adopted for the display of the system monitor.
  9. 　前記決定部が決定した表示態様に基づいて、前記システムモニタの表示態様の変更を要求する要求信号を生成して前記内視鏡システムへ送信する変更要求部を備える請求項１から８のいずれか１項に記載の検査支援装置。 Any of claims 1 to 8 comprising a change requesting unit that generates a request signal requesting a change in the display mode of the system monitor based on the display mode determined by the determination unit and transmits the request signal to the endoscope system. The inspection support device according to item 1.
  10. 　前記表示制御部は、前記解析部による解析に先立って、前記表示モニタに指標を表示させる請求項１から９のいずれか１項に記載の検査支援装置。 The inspection support device according to any one of claims 1 to 9, wherein the display control unit displays an index on the display monitor prior to analysis by the analysis unit.
  11. 　被検者の体内に挿通されるカメラユニットを備える内視鏡システムに接続して利用される検査支援装置を用いた検査支援方法であって、
    　体内検査の実行に先立って前記カメラユニットで撮像された撮像画像を取得する取得ステップと、
    　前記撮像画像に写り込んだ、前記検査支援装置が備える表示モニタと前記内視鏡システムに付随するシステムモニタの相互関係を解析する解析ステップと、
    　前記解析ステップで解析された前記相互関係に基づいて、前記カメラユニットで撮像された撮像画像と前記体内検査に関する検査情報を表示する前記表示モニタの表示態様を決定する決定ステップと、
    　前記決定ステップで決定された前記表示態様に従って前記表示モニタの表示を制御する表示制御ステップと
    を有する検査支援方法。     It is an inspection support method using an inspection support device used by connecting to an endoscope system equipped with a camera unit inserted into the body of the subject.
    The acquisition step of acquiring the captured image captured by the camera unit prior to the execution of the internal examination, and
    An analysis step for analyzing the interrelationship between the display monitor included in the inspection support device and the system monitor attached to the endoscope system reflected in the captured image, and
    A determination step for determining a display mode of the display monitor for displaying an image captured by the camera unit and examination information related to the internal examination based on the interrelationship analyzed in the analysis step.
    An inspection support method having a display control step for controlling the display of the display monitor according to the display mode determined in the determination step.
  12. 　被検者の体内に挿通されるカメラユニットを備える内視鏡システムに接続して利用される検査支援装置を制御する検査支援プログラムであって、
    　体内検査の実行に先立って前記カメラユニットで撮像された撮像画像を取得する取得ステップと、
    　前記撮像画像に写り込んだ、前記検査支援装置が備える表示モニタと前記内視鏡システムに付随するシステムモニタの相互関係を解析する解析ステップと、
    　前記解析ステップで解析された前記相互関係に基づいて、前記カメラユニットで撮像された撮像画像と前記体内検査に関する検査情報を表示する前記表示モニタの表示態様を決定する決定ステップと、
    　前記決定ステップで決定された前記表示態様に従って前記表示モニタの表示を制御する表示制御ステップと
    をコンピュータに実行させる検査支援プログラム。     It is an inspection support program that controls an inspection support device that is used by connecting to an endoscope system equipped with a camera unit that is inserted into the body of the subject.
    The acquisition step of acquiring the captured image captured by the camera unit prior to the execution of the internal examination, and
    An analysis step for analyzing the interrelationship between the display monitor included in the inspection support device and the system monitor attached to the endoscope system reflected in the captured image, and
    A determination step for determining a display mode of the display monitor for displaying an image captured by the camera unit and examination information related to the internal examination based on the interrelationship analyzed in the analysis step.
    An inspection support program that causes a computer to execute a display control step that controls the display of the display monitor according to the display mode determined in the determination step.
PCT/JP2021/032384 2020-09-23 2021-09-03 Inspection assistance device, inspection assistance method, and inspection assistance program WO2022064998A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020-158802 2020-09-23
JP2020158802A JP2022052417A (en) 2020-09-23 2020-09-23 Examination support device, examination support method, and examination support program

Publications (1)

Publication Number Publication Date
WO2022064998A1 true WO2022064998A1 (en) 2022-03-31

Family

ID=80845208

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/032384 WO2022064998A1 (en) 2020-09-23 2021-09-03 Inspection assistance device, inspection assistance method, and inspection assistance program

Country Status (2)

Country Link
JP (1) JP2022052417A (en)
WO (1) WO2022064998A1 (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011000173A (en) * 2009-06-16 2011-01-06 Toshiba Corp Endoscopic examination supporting system
JP2014188310A (en) * 2013-03-28 2014-10-06 Sony Corp Image processing device, image processing method, program, and endoscope system
WO2017122541A1 (en) * 2016-01-13 2017-07-20 ソニー株式会社 Image processing device, image processing method, program, and surgical system
WO2018163644A1 (en) * 2017-03-07 2018-09-13 ソニー株式会社 Information processing device, assist system, and information processing method
JP2019042156A (en) * 2017-09-01 2019-03-22 富士フイルム株式会社 Medical image processing apparatus, endoscope apparatus, diagnosis support apparatus, and medical service support apparatus
JP2019154883A (en) * 2018-03-15 2019-09-19 ソニー・オリンパスメディカルソリューションズ株式会社 Medical image processing apparatus, medical observation apparatus, and image processing method
WO2019176556A1 (en) * 2018-03-13 2019-09-19 ソニー株式会社 Medical instrument management system and medical instrument management method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011000173A (en) * 2009-06-16 2011-01-06 Toshiba Corp Endoscopic examination supporting system
JP2014188310A (en) * 2013-03-28 2014-10-06 Sony Corp Image processing device, image processing method, program, and endoscope system
WO2017122541A1 (en) * 2016-01-13 2017-07-20 ソニー株式会社 Image processing device, image processing method, program, and surgical system
WO2018163644A1 (en) * 2017-03-07 2018-09-13 ソニー株式会社 Information processing device, assist system, and information processing method
JP2019042156A (en) * 2017-09-01 2019-03-22 富士フイルム株式会社 Medical image processing apparatus, endoscope apparatus, diagnosis support apparatus, and medical service support apparatus
WO2019176556A1 (en) * 2018-03-13 2019-09-19 ソニー株式会社 Medical instrument management system and medical instrument management method
JP2019154883A (en) * 2018-03-15 2019-09-19 ソニー・オリンパスメディカルソリューションズ株式会社 Medical image processing apparatus, medical observation apparatus, and image processing method

Also Published As

Publication number Publication date
JP2022052417A (en) 2022-04-04

Similar Documents

Publication Publication Date Title
US8004560B2 (en) Endoscope apparatus
JP4656988B2 (en) Endoscope insertion shape analysis apparatus and endoscope insertion shape analysis method
JP4914574B2 (en) Endoscope shape detection device
JP4676021B2 (en) Diagnosis support apparatus, diagnosis support program, and diagnosis support method
US8558879B2 (en) Endoscope apparatus and measuring method
EP1994878A1 (en) Image processing device for medical use and image processing method for medical use
JP5486432B2 (en) Image processing apparatus, operating method thereof, and program
US20210065370A1 (en) Information processing apparatus, inspection system, information processing method, and storage medium that are used in a diagnosis based on a medical image
JP6956853B2 (en) Diagnostic support device, diagnostic support program, and diagnostic support method
JP7194889B2 (en) Computer program, learning model generation method, surgery support device, and information processing method
JP2021029676A (en) Information processor, inspection system, and information processing method
KR101595962B1 (en) Colnoscopy surgery simulation system
JP2008119259A (en) Endoscope insertion shape analysis system
EP1566140B1 (en) Endoscope insertion direction detecting device
WO2022064998A1 (en) Inspection assistance device, inspection assistance method, and inspection assistance program
US20220361739A1 (en) Image processing apparatus, image processing method, and endoscope apparatus
WO2022065189A1 (en) Examination assistance device, examination assistance method, and examination assistance program
JP2022145395A (en) Examination support device, examination support method, and examination support program
WO2022270582A1 (en) Examination assistance device, examination assistance method, and examination assistance program
JP4464640B2 (en) Industrial endoscope equipment
JP2005111081A (en) Endoscopic image display processor
JP7391113B2 (en) Learning medical image data creation device, learning medical image data creation method, and program
US20220346632A1 (en) Image processing apparatus, image processing method, and non-transitory storage medium storing computer program
WO2023218523A1 (en) Second endoscopic system, first endoscopic system, and endoscopic inspection method
JP2023106327A (en) Inspection support device, inspection support method, and inspection support program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21872129

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21872129

Country of ref document: EP

Kind code of ref document: A1