JPWO2017199657A1 - Endoscope apparatus, measurement method, and program - Google Patents

Abstract

In the endoscope apparatus, the reliability map generation unit generates a reliability map indicating reliability information corresponding to each of a plurality of positions on the observation image. The measurement index display control unit displays the measurement index on the observation image, and moves the measurement index on the observation image based on a user instruction input via the input unit. The reliability map display controller controls the reliability map display ratio so that the ratio of the display size of the reliability map to the display size of the observation image is a predetermined value at a position different from the position where the measurement index is displayed on the display unit. Show the degree map.

Description

The present invention relates to an endoscope apparatus, a measurement method, and a program.
This application claims priority based on Japanese Patent Application No. 2006-101339 for which it applied to Japan on May 20, 2016, and uses the content here.

  In recent years, industrial endoscopes are widely used to observe internal flaws and corrosion in inspection of boilers, turbines, engines, chemical plants, and the like. As an example of an industrial endoscope, a measurement endoscope apparatus is disclosed in Patent Document 1. In the measurement endoscope apparatus, two optical systems are provided at the distal end of the endoscope. The measurement endoscope apparatus performs three-dimensional measurement by stereo measurement using the principle of triangulation. When a defective part such as a scratch is found inside the inspection object in the inspection, the user measures the size of the wound using such a measurement endoscope apparatus. Thus, the user can determine whether or not the inspection object needs to be disassembled and repaired. Therefore, it is very important that the measurement endoscope apparatus can perform three-dimensional measurement as accurately as possible. Devices that perform three-dimensional measurement other than stereo measurement are also used. For example, an apparatus is used in which means for projecting pattern light is provided at the tip of the insertion portion. This apparatus can perform three-dimensional measurement by analyzing a pattern projected on a subject.

  As described above, it is very important that the measurement endoscope apparatus can perform measurement as accurately as possible. In general, the user wants to perform measurement with as high a measurement accuracy as possible. At the time of measurement, the user adjusts the position of the insertion portion so that the position to be measured in the subject is reliably imaged. The user performs measurement by designating a measurement point on the still image after the image being observed is stopped.

  Patent Document 2 discloses a measurement endoscope apparatus that displays an overlay of measurement reliability on a measurement image acquired by an imager. The user performs measurement by designating measurement points on a measurement image on which measurement reliability is displayed in an overlay manner. According to the measurement endoscope apparatus disclosed in Patent Literature 2, the user can grasp the measurement reliability on the measurement image at the stage of specifying the measurement point.

  Patent Document 3 discloses an endoscope apparatus that displays an uncertainty estimation image in which information indicating the uncertainty of a measurement value is combined with an image acquired by a solid-state imaging device. According to the endoscope device disclosed in Patent Document 3, the user can know the accuracy of the measurement value in advance.

Japanese Unexamined Patent Publication No. 2004-33487 Japanese Unexamined Patent Publication No. 2012-220696 Japanese Unexamined Patent Publication No. 2013-25292

  In Patent Document 2, the measurement reliability is overlaid on the measurement image acquired by the imager. Therefore, the user can confirm the measurement reliability. However, since the measurement image is hidden in the portion where the measurement reliability is displayed as an overlay, the visibility is lowered. It is difficult for the user to confirm the state of the portion where the measurement reliability is overlaid. Further, when the measurement reliability is displayed in the vicinity of the measurement position in the measurement image, the visibility near the measurement position is degraded.

  Patent Document 3 discloses displaying an uncertainty estimation image corresponding to measurement reliability, but does not mention designation of measurement points. If the user designates a measurement point on the uncertainty estimation image, the visibility is reduced as in Patent Document 2. For this reason, it becomes inconvenient for the user.

  The present invention provides an endoscope apparatus, a measurement method, and a program that can suppress a decrease in the visibility of a measurement position in an observation image.

  According to the first aspect of the present invention, an endoscope apparatus includes an imaging unit, an observation image generation unit, a reliability calculation unit, a reliability map generation unit, a display unit, and a measurement index display control unit. And a measurement point determination unit, a measurement unit, and a reliability map display control unit. The imaging unit images a subject and generates an imaging signal. The observation image generation unit generates an observation image based on the imaging signal. The reliability calculation unit calculates reliability information related to the reliability of the calculation result of the three-dimensional coordinates corresponding to each of the plurality of positions in the observation image. The reliability map generation unit generates a reliability map indicating the reliability information corresponding to each of the plurality of positions on the observation image. The display unit displays the observation image and the reliability map. The measurement index display control unit displays a measurement index for the user to specify the position of a measurement point on the observation image on the observation image, and the user's instruction input via the input unit Based on the above, the measurement index is moved on the observation image. The measurement point determination unit determines the position of the measurement point based on the position of the measurement index. The measurement unit calculates a three-dimensional coordinate of the position of the measurement point determined by the measurement point determination unit. The reliability map display control unit is configured such that a ratio of the display size of the reliability map to the display size of the observation image becomes a predetermined value at a position different from a position where the measurement index is displayed on the display unit. The reliability map is displayed.

  According to the second aspect of the present invention, in the first aspect, the endoscope apparatus may further include a corresponding position calculation unit and a reliability information notification unit. The corresponding position calculation unit calculates a corresponding position on the reliability map corresponding to the position of the measurement index in the observation image. The reliability information notification unit notifies the user of the reliability information at the corresponding position calculated by the corresponding position calculation unit.

  According to a third aspect of the present invention, in the second aspect, the reliability information notification unit is configured to display a reference index at the corresponding position calculated by the corresponding position calculation unit in the reliability map. An index display control unit may be provided.

  According to a fourth aspect of the present invention, in the third aspect, the reliability information at the corresponding position calculated by the corresponding position calculation unit includes a plurality of trusts classified according to the degree of reliability. It may correspond to any one of the degree levels.

  According to a fifth aspect of the present invention, in the fourth aspect, the display form of the reference indicator may be any one of a plurality of display forms corresponding to the plurality of reliability levels. The reference index display control unit may display the reference index in a display form corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit.

  According to a sixth aspect of the present invention, in the fourth aspect, the reliability map generation unit is configured such that each of the corresponding positions calculated by the corresponding position calculation unit in the reliability map is in the corresponding position. The reliability map that is displayed in a color corresponding to the reliability level may be generated. The reference index display control unit displays the reference index in any one of a plurality of colors, and displays the display color of the reference index at the corresponding position calculated by the corresponding position calculation unit. The color corresponding to the reliability level may be different.

  According to a seventh aspect of the present invention, in the second aspect, the reliability information notification unit may include the measurement index display control unit.

  According to an eighth aspect of the present invention, in the seventh aspect, the reliability information at the corresponding position calculated by the corresponding position calculation unit includes a plurality of trusts classified according to the degree of reliability. It may correspond to any one of the degree levels.

  According to a ninth aspect of the present invention, in the eighth aspect, the display form of the measurement index may be any one of a plurality of display forms corresponding to the plurality of reliability levels. The measurement index display control unit may display the measurement index in a display form corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit.

  According to a tenth aspect of the present invention, in the eighth aspect, the reliability map generation unit is configured such that each of the corresponding positions calculated by the corresponding position calculation unit in the reliability map is in the corresponding position. The reliability map that is displayed in a color corresponding to the reliability level may be generated. The measurement index display control unit displays the measurement index in any one of a plurality of colors, and displays the display color of the measurement index at the corresponding position calculated by the corresponding position calculation unit. You may make it the same as the color corresponding to a reliability level.

  According to an eleventh aspect of the present invention, in the second aspect, the reliability information notifying unit controls an outer edge display control unit that controls a display form of an outer edge of the display area of at least one of the observation image and the reliability map. You may have.

  According to a twelfth aspect of the present invention, in the eleventh aspect, the reliability information at the corresponding position calculated by the corresponding position calculation unit is a plurality of reliability classified according to the degree of the reliability. It may correspond to any one of the degree levels.

  According to a thirteenth aspect of the present invention, in the twelfth aspect, the display form of the outer edge of the display area of at least one of the observation image and the reliability map is a plurality of displays corresponding to the plurality of reliability levels. Any one of the forms may be used. The outer edge display control unit displays the outer edge of the display area of at least one of the observation image and the reliability map in a display form corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit. Also good.

  According to a fourteenth aspect of the present invention, in the thirteenth aspect, the outer edge display control unit makes the display forms of the display area outer edge of the observation image and the display area outer edge of the reliability map the same. May be.

  According to a fifteenth aspect of the present invention, in the twelfth aspect, the reliability map generation unit is configured such that each of the corresponding positions calculated by the corresponding position calculation unit in the reliability map is in the corresponding position. The reliability map that is displayed in a color corresponding to the reliability level may be generated. The outer edge display control unit displays the outer edge of the display area of at least one of the observation image and the reliability map in any one of a plurality of colors, and the at least one of the observation image and the reliability map The display color of the outer edge of the display area may be the same as the color corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit.

  According to a sixteenth aspect of the present invention, in the fifteenth aspect, the outer edge display control unit makes the display colors of the display image outer edge of the observation image and the display region outer edge of the reliability map the same. May be.

  According to a seventeenth aspect of the present invention, in the second aspect, the reliability information notification unit may include the reliability map display control unit. The reliability map display control unit may control a display size of the reliability map.

  According to an eighteenth aspect of the present invention, in the seventeenth aspect, the reliability information at the corresponding position calculated by the corresponding position calculation unit includes a plurality of trusts classified according to the degree of reliability. It may correspond to any one of the degree levels.

  According to a nineteenth aspect of the present invention, in the eighteenth aspect, the ratio may be any one of a plurality of values corresponding to the plurality of reliability levels. The reliability map display control unit may display the reliability map so that the ratio is a value corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit.

  According to a twentieth aspect of the present invention, in the first aspect, the endoscope apparatus displays a first grid line at a first position on the observation image, and at the first position. You may further have a grid line display control part which displays a 2nd grid line in the 2nd position on the said reliability map corresponding.

  According to a twenty-first aspect of the present invention, in the twentieth aspect, the endoscope apparatus calculates a corresponding position on the reliability map corresponding to the position of the measurement index in the observation image. You may further have a calculation part. The reliability information at the corresponding position calculated by the corresponding position calculation unit may correspond to any one of a plurality of reliability levels divided according to the degree of reliability.

  According to a twenty-second aspect of the present invention, in the twenty-first aspect, the endoscope apparatus calculates a position on the reliability map corresponding to the position of the measurement index in the observation image. It may have a part. The display form of at least one of the first grid line and the second grid line may be any one of a plurality of display forms corresponding to the plurality of reliability levels. The grid line display control unit displays at least one of the first grid line and the second grid line in a display form corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit. May be.

  According to a twenty-third aspect of the present invention, in the twenty-first aspect, the endoscope apparatus calculates a position on the reliability map corresponding to the position of the measurement index in the observation image. It may have a part. The reliability map generation unit is configured to display the reliability such that each of the corresponding positions calculated by the corresponding position calculation unit in the reliability map is displayed in a color corresponding to the reliability level at the corresponding position. A map may be generated. The grid line display control unit displays the first grid line in any one of a plurality of colors, and the display color of the first grid line is calculated by the corresponding position calculation unit. You may make it the same as the color corresponding to the said reliability level in a position.

  According to the twenty-fourth aspect of the present invention, the measurement method includes an imaging step, an observation image generation step, a reliability calculation step, a reliability map generation step, a display step, a measurement index display step, and a measurement. It has a point determination step and a measurement step. In the imaging step, the subject is imaged and an imaging signal is generated. The observation image generation step generates an observation image based on the imaging signal. In the reliability calculation step, reliability information related to the reliability of the calculation result of the three-dimensional coordinates corresponding to each of the plurality of positions in the observation image is calculated. The reliability map generation step generates a reliability map indicating the reliability information corresponding to each of the plurality of positions on the observation image. In the display step, the observation image and the reliability map are displayed on a display unit. In the measurement index display step, a measurement index for a user to specify a position of a measurement point on the observation image is displayed on the observation image, and an instruction from the user input via the input unit is displayed. Based on this, the measurement index is moved on the observation image. The measurement point determination step determines the position of the measurement point based on the position of the measurement index. The measurement step calculates a three-dimensional coordinate of the position of the measurement point determined by the measurement point determination step. In the display step, the reliability map is set such that a ratio of a display size of the reliability map to a display size of the observation image becomes a predetermined value at a position different from a position where the measurement index is displayed on the display unit. Is displayed.

  According to the twenty-fifth aspect of the present invention, the program includes an observation image acquisition function, a reliability calculation function, a reliability map generation function, a display function, a measurement index display function, a measurement point determination function, The computer implements the measurement function and the reliability map display control function. The observation image acquisition function acquires an observation image of a subject. The reliability calculation function calculates reliability information related to the reliability of the calculation result of the three-dimensional coordinates corresponding to each of the plurality of positions in the observation image. The reliability map generation function generates a reliability map indicating the reliability information corresponding to each of the plurality of positions on the observation image. The display device displays the observation image and the reliability map on a display unit. The measurement index display function displays a measurement index for the user to specify the position of a measurement point on the observation image on the observation image and responds to the user's instruction input via the input unit. Based on this, the measurement index is moved on the observation image. The measurement point determination function determines the position of the measurement point based on the position of the measurement index. The measurement function calculates a three-dimensional coordinate of the position of the measurement point determined by the measurement point determination function. The reliability map display control function is configured such that the ratio of the display size of the reliability map to the display size of the observation image is a predetermined value at a position different from the position where the measurement index is displayed on the display unit. The reliability map is displayed.

  According to each aspect described above, the endoscope apparatus, the measurement method, and the program can suppress a decrease in the visibility of the measurement position in the observation image.

1 is a perspective view showing an overall configuration of an endoscope apparatus according to a first embodiment of the present invention. It is a block diagram which shows the internal structure of the endoscope apparatus by the 1st Embodiment of this invention. It is a block diagram which shows the function structure of CPU by the 1st Embodiment of this invention. It is a reference figure for demonstrating the calculation method of the three-dimensional coordinate of the measurement point by the stereo measurement of the 1st Embodiment of this invention. It is a flowchart which shows the procedure of the measurement process in the 1st Embodiment of this invention. It is a flowchart which shows the procedure of the measurement process in the 1st Embodiment of this invention. It is a reference figure which shows the reliability map in the 1st Embodiment of this invention. It is a reference figure which shows the reliability map in the 1st Embodiment of this invention. It is a reference figure which shows the observation image and reliability map in the 1st Embodiment of this invention. It is a reference figure which shows the observation image and reliability map in the 1st Embodiment of this invention. It is a reference figure which shows the observation image and reliability map in the 1st Embodiment of this invention. It is a reference figure which shows the observation image and reliability map in the 1st Embodiment of this invention. It is a reference figure which shows the reliability map in the 1st Embodiment of this invention. It is a reference figure which shows the reliability map in the 1st Embodiment of this invention. It is a reference figure which shows the reliability map in the 1st Embodiment of this invention. It is a reference figure which shows the reliability map in the 1st Embodiment of this invention. It is a flowchart which shows the procedure of the measurement process in the modification of the 1st Embodiment of this invention. It is a reference view showing an observation image and a reliability map in a modification of the first embodiment of the present invention. It is a reference view showing an observation image and a reliability map in a modification of the first embodiment of the present invention. It is a reference view showing an observation image and a reliability map in a modification of the first embodiment of the present invention. It is a block diagram which shows the function structure of CPU by the 2nd Embodiment of this invention. It is a flowchart which shows the procedure of the measurement process in the 2nd Embodiment of this invention. It is a reference figure which shows the observation image and reliability map in the 2nd Embodiment of this invention. It is a reference figure which shows the observation image and reliability map in the 2nd Embodiment of this invention. It is a reference figure which shows the observation image and reliability map in the 2nd Embodiment of this invention. It is a block diagram which shows the function structure of CPU by the 3rd Embodiment of this invention. It is a flowchart which shows the procedure of the measurement process in the 3rd Embodiment of this invention. It is a reference figure which shows the observation image and reliability map in the 3rd Embodiment of this invention. It is a reference figure which shows the observation image and reliability map in the 3rd Embodiment of this invention. It is a reference figure which shows the observation image and reliability map in the 3rd Embodiment of this invention. It is a block diagram which shows the function structure of CPU by the 4th Embodiment of this invention. It is a flowchart which shows the procedure of the measurement process in the 4th Embodiment of this invention. It is a reference figure which shows the observation image and reliability map in the 4th Embodiment of this invention. It is a reference figure which shows the observation image and reliability map in the 4th Embodiment of this invention. It is a block diagram which shows the function structure of CPU by the 5th Embodiment of this invention. It is a flowchart which shows the procedure of the measurement process in the 5th Embodiment of this invention. It is a reference diagram showing an observation image and a reliability map in the fifth embodiment of the present invention. It is a reference diagram showing an observation image and a reliability map in the fifth embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 shows the overall configuration of an endoscope apparatus 1 according to the first embodiment of the present invention. FIG. 2 shows the internal configuration of the endoscope apparatus 1. As shown in FIG. 1, the endoscope apparatus 1 includes an endoscope 2 and an apparatus body 3. The endoscope 2 includes an elongated insertion unit 20 and an operation unit 6 (input unit) for a user to perform an operation necessary for controlling the entire apparatus. The apparatus body 3 is connected to the endoscope 2. The apparatus main body 3 includes a monitor 4 and a housing 5. The monitor 4 displays an image of the subject imaged by the endoscope 2 and an operation menu. The housing 5 has a control unit 10 (see FIG. 2) inside.

  The insertion unit 20 is inserted into the subject. The insertion portion 20 includes a hard distal end portion 21, a bendable bending portion 22, and a flexible tube portion 23 having flexibility. A distal end portion 21 is disposed on the distal end side in the insertion portion 20. A flexible tube portion 23 is disposed on the main body side in the insertion portion 20. A bending portion 22 is disposed between the distal end portion 21 and the flexible tube portion 23. An optical adapter for forming a subject image can be attached to and detached from the distal end portion 21.

  As shown in FIG. 2, the housing 5 includes an endoscope unit 8, a CCU (camera control unit) 9, and a control unit 10. The endoscope unit 8 includes a light source device that supplies illumination light necessary for observation, and a bending device that bends the bending portion 22. The CCU 9 drives the image sensor 28. An imaging signal output from the imaging element 28 is input to the CCU 9. The CCU 9 performs preprocessing including amplification and noise removal on the image signal acquired by the image sensor 28. The CCU 9 converts the pre-processed imaging signal into a video signal such as an NTSC signal.

  The control unit 10 includes a video signal processing circuit 12, a ROM (Read Only Memory) 13, a RAM (Random Access Memory) 14, a card interface 15, an external device interface 16, a control interface 17, and a CPU (Central Processing). Unit) 18a.

  The video signal processing circuit 12 performs predetermined video processing on the video signal output from the CCU 9. For example, the video signal processing circuit 12 combines the video signal output from the CCU 9 and the graphic image signal generated by the CPU 18a. The graphic image signal includes an image of the operation screen, measurement information, and the like. The measurement information includes a measurement index image, a reference index image, a measurement result, and the like. The video signal processing circuit 12 outputs the synthesized video signal to the monitor 4.

  The ROM 13 is a non-volatile recording medium on which a program for the CPU 18a to control the operation of the endoscope apparatus 1 is recorded. The RAM 14 is a volatile recording medium that temporarily stores information used by the CPU 18 a for controlling the endoscope apparatus 1. The CPU 18 a controls the operation of the endoscope apparatus 1 based on a program recorded in the ROM 13.

  A memory card 32 that is a detachable recording medium is connected to the card interface 15. The card interface 15 captures control processing information, image information, and the like stored in the memory card 32 into the control unit 10. The card interface 15 records control processing information, image information, and the like generated by the endoscope device 1 in the memory card 32.

  An external device such as a USB device is connected to the external device interface 16. For example, the personal computer 31 is connected to the external device interface 16. The external device interface 16 transmits information to the personal computer 31 and receives information from the personal computer 31. Thereby, the monitor of the personal computer 31 can display information. Further, the user can perform operations related to the control of the endoscope apparatus 1 via the personal computer 31.

  The control interface 17 communicates with the operation unit 6, the endoscope unit 8, and the CCU 9 for operation control. The control interface 17 notifies the CPU 18a of an instruction input by the user via the operation unit 6. The control interface 17 outputs a control signal for controlling the light source device and the bending device to the endoscope unit 8. The control interface 17 outputs a control signal for controlling the image sensor 28 to the CCU 9.

  The program executed by the CPU 18a may be recorded on a computer-readable recording medium. The program recorded on the recording medium may be read and executed by a computer other than the endoscope apparatus 1. For example, the personal computer 31 may read and execute the program. The personal computer 31 may control the endoscope apparatus 1 by transmitting control information for controlling the endoscope apparatus 1 to the endoscope apparatus 1 according to a program. Alternatively, the personal computer 31 may acquire a video signal from the endoscope apparatus 1 and perform measurement using the acquired video signal.

  The above-described program may be transmitted to the endoscope apparatus 1 from a computer having a storage device or the like in which the program is stored via a transmission medium or by a transmission wave in the transmission medium. A “transmission medium” for transmitting a program is a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the above-described program may realize a part of the functions described above. Further, the above-described program may be a difference file (difference program) that can realize the above-described function in combination with a program already recorded in the computer.

  As described above, the endoscope apparatus 1 includes the imaging element 28 (imaging unit), the CCU 9 (observation image generation unit), the monitor 4 (display unit), and the CPU 18a. The image sensor 28 images a subject and generates an image signal. The CCU 9 generates a video signal based on the imaging signal. The video signal includes an image of a subject, that is, an observation image. Therefore, the CCU 9 generates an observation image. The monitor 4 displays an observation image based on the video signal and a reliability map described later.

  FIG. 3 shows a functional configuration of the CPU 18a. A reliability calculation unit 70, a reliability map generation unit 71, a measurement index display control unit 72, a control unit 73, a measurement point determination unit 74, a measurement unit 75, a corresponding position calculation unit 76, a reliability map display control unit 77, and The reliability information notification unit 78a constitutes the function of the CPU 18a. At least one of the blocks shown in FIG. 3 may be configured by a circuit different from the CPU 18a.

  The reliability calculation unit 70 calculates (generates) reliability information related to the reliability of the calculation result of the three-dimensional coordinates corresponding to each of a plurality of positions in the observation image based on the video signal. The reliability map generation unit 71 generates a reliability map indicating reliability information corresponding to each of a plurality of positions on the observation image. Specifically, the reliability map generation unit 71 first acquires reliability information from the reliability calculation unit 70 via the control unit 73. The reliability map generation unit 71 classifies the reliability at each position on the observation image into each level based on the acquired reliability information. For example, the reliability is classified into two types: a low level and a high level. The reliability map generation unit 71 generates a reliability map that allows the user to visually confirm the reliability classified into each level. For example, the reliability map generation unit 71 generates a reliability map in which each position is indicated by a color corresponding to the reliability level.

  The reliability map is an image showing a two-dimensional distribution of reliability information. The positions of the observation image and the reliability map correspond to each other so that the user can confirm where the plurality of positions on the observation image correspond to the reliability map. For example, a conversion formula for converting two-dimensional coordinates on the observation image into two-dimensional coordinates on the reliability map is stored in the RAM 14. A table in which the two-dimensional coordinates on the observation image are associated with the two-dimensional coordinates on the reliability map may be stored in the RAM 14. Based on the above conversion formula or table, a position on the reliability map corresponding to the position on the observation image is calculated.

  The measurement index display control unit 72 displays a measurement index on the observation image for the user to specify the position of the measurement point on the observation image. The user can move the measurement index freely by operating the operation unit 6. The measurement index display control unit 72 moves the measurement index on the observation image based on a user instruction input via the operation unit 6. The measurement index is a mark for the user to specify a specific location on the screen. There is no limitation on the shape or the like of the measurement index. The control unit 73 controls processing performed by each unit. The measurement point determination unit 74 determines the position of the measurement point based on the position of the measurement index. For example, after moving the measurement index, the user instructs designation of a measurement point via the operation unit 6. At this time, the measurement point determination unit 74 determines the position of the measurement index in the observation image as the position of the measurement point. The measurement unit 75 calculates the three-dimensional coordinates of the position of the measurement point determined by the measurement point determination unit 74. The measurement unit 75 measures the size of the three-dimensional shape of the subject based on the calculated three-dimensional coordinates.

  The corresponding position calculation unit 76 calculates a corresponding position on the reliability map corresponding to the position of the measurement index in the observation image. The positional relationship between the observation image and the reliability map is obtained in advance. Therefore, the corresponding position calculation unit 76 can easily calculate the corresponding position on the reliability map corresponding to the predetermined position on the observation image. The reliability map display control unit 77 sets the reliability map at a position different from the position where the measurement index is displayed on the monitor 4 so that the ratio of the display size of the reliability map to the display size of the observation image becomes a predetermined value. Display. Reliability information at each of a plurality of positions on the observation image is displayed on the reliability map.

  In the endoscope apparatus disclosed in Patent Document 3, when the measurement image and the uncertainty estimation image are displayed separately, the user determines which position on the uncertainty estimation image the measurement point on the measurement image is located on. There is a need to check frequently. For this reason, it becomes inconvenient for the user. Therefore, the first embodiment is an endoscope that can suppress a decrease in the visibility of the measurement position in the observation image and can make the user easily determine whether or not the measurement position is suitable for measurement. An object is to provide a device 1 and a program.

  The reliability information notification unit 78 a notifies the user of reliability information at the corresponding position calculated by the corresponding position calculation unit 76. Since the reliability information is displayed on the reliability map, the reliability information notification unit 78a notifies the user of the reliability information by means other than the means for displaying the reliability map. The reliability information notification unit 78a is a component for providing reliability information to the user as auxiliary information. By providing auxiliary information, the reliability state of the point that the user is trying to measure is easily communicated to the user. For example, the auxiliary information is visual information, auditory information, and tactile information. The visual information includes a change in the display screen. Auditory information includes warning sounds and the like. The tactile information includes warning vibration and the like. The reliability information notification unit 78a of the first embodiment includes a reference index display control unit 79 that displays a reference index at the corresponding position calculated by the corresponding position calculation unit 76 in the reliability map.

  The notification of reliability information by means other than the means for displaying the reliability map is not essential. Therefore, the CPU 18a may not have the functions of the corresponding position calculation unit 76 and the reliability information notification unit 78a.

  A method by which the measurement unit 75 calculates three-dimensional coordinates will be described. The measurement unit 75 calculates three-dimensional coordinates corresponding to a position designated on the observation image based on the video signal (image data) input to the video signal processing circuit 12. The position where the three-dimensional coordinates are calculated is a measurement position (measurement point) designated by the user in the observation image. The three-dimensional coordinates of the measurement point designated by the user are used to obtain the size (measurement result) of the three-dimensional shape of the subject. As an example, a method for calculating three-dimensional coordinates by stereo measurement will be described.

  An image is acquired in a state where a stereo optical adapter having two observation fields on the left and right is attached to the distal end portion 21. The stereo optical adapter is a dedicated lens for performing stereo measurement. Light from the subject enters the stereo optical adapter. The left and right subject images corresponding to the same subject are formed by the stereo optical adapter. Two subject images enter the image sensor 28 and are captured by the image sensor 28. The acquired image includes two subject images. Of the acquired images, an image of an area corresponding to two subject images is defined as a left image and a right image.

  The principle of stereo measurement will be described with reference to FIG. In stereo measurement, the three-dimensional coordinates of a subject are calculated using the principle of triangulation based on the coordinates of two optical ranging points when the subject image is captured by two optical systems. The midpoint of the line segment connecting the left optical center (first optical center 63) and the right optical center (second optical center 64) is defined as the origin O. In addition, an x-axis in which the right direction is positive and a y-axis in which the downward direction is positive are defined. Further, a z-axis is defined in which the direction away from the optical system in parallel with the optical axis is positive.

With respect to the image including the subject image obtained through the left optical system and the right optical system, the three-dimensional coordinates (X, Y, Z) of the measurement point 60 are represented by the following (1) by the principle of triangulation. It is calculated from equation (3). However, the two-dimensional coordinates of the measurement point 61 on the left image plane subjected to distortion correction and the corresponding point 62 on the right image plane subjected to distortion correction are (X _L , Y _L ) and (X _{R 1} , Y _R ). Each origin of these two-dimensional coordinates, the intersection O _L between the optical axis and the image plane of the left optical system and the right optical _system, which is the intersection O _R. The distance between the first optical center 63 and the second optical center 64 is D. The focal length is F. The parameter t is D / (X _R −X _L ).
X = t × X _R + D / 2 (1)
Y = −t × Y _R (2)
Z = t × F (3)

  When the coordinates of the measurement point 61 and the corresponding point 62 on the image plane are determined as described above, the three-dimensional coordinates of the measurement point 60 can be obtained using the parameter D and the parameter F. In general, in stereo measurement, corresponding points are calculated by a matching process. In the matching process, a second position of the subject corresponding to the first position of the subject is detected. The first position of the subject is the position of the subject in the image based on the light passing through one optical system. The second position of the subject is the position of the subject in the image based on the light passing through the other optical system. There are various methods for matching processing. For example, there is a matching method for performing template matching. There is also a matching method that performs matching using feature points such as shading edges. There is also a matching method using a phase only correlation method (POC: Phase Only Correlation) that calculates a correlation of phase components obtained by Fourier transforming an image. The matching method applied to the embodiment of the present invention is not limited to the above matching method.

  By obtaining the three-dimensional coordinates of several points, various measurement functions can be realized. For example, a distance between two points, a distance between a line connecting two points and one point, an area of a region surrounded by a line connecting a plurality of points, a depth of a reference surface, a surface shape, and the like are measured. The user can select a desired measurement function from various measurement functions. It is also possible to obtain the distance (object distance) from the first optical center 63 or the second optical center 64 to the subject. In order to perform the above stereo measurement, optical data indicating the characteristics of an optical system including the distal end portion 21 of the insertion portion 20 and the stereo optical adapter is required. For example, the details of the matching process and the optical data are described in Japanese Patent Application Laid-Open No. 2004-49638, and the description thereof is omitted.

  The measurement method applicable to the embodiment of the present invention is not limited to stereo measurement. For example, a pattern projection method that analyzes a pattern image acquired by imaging pattern light projected onto a subject and calculates three-dimensional coordinates may be applied to the embodiment of the present invention.

  Details of the measurement processing in the first embodiment will be described. In the first embodiment, measurement processing is performed according to the procedure shown in FIGS. Observation images based on video signals are acquired in various observation situations. Depending on the observation situation, an image that is easy to measure is acquired, or an image that is difficult to measure is acquired. The acquired observation image is an image recorded in the memory card 32 for measurement or an image (freeze image) temporarily recorded in the apparatus main body 3. The observation image is not limited to a still image. For example, the observation image may be a video signal (LIVE moving image) acquired in real time. In the first embodiment, the flow of measurement processing when the observation image is a still image will be described.

  In response to a user instruction, the CPU 18a acquires an observation image in a state where an object that the user wants to measure is captured (step S1). At this time, the image sensor 28 images the subject and generates an image signal. The CCU 9 generates a video signal, that is, an observation image based on the imaging signal. The CPU 18a acquires a video signal, that is, an observation image from the CCU 9.

  After the observation image is acquired, the reliability calculation unit 70 calculates reliability information related to the reliability of the calculation result of the three-dimensional coordinates based on the observation image (step S2). For example, the reliability calculation unit 70 detects a state that is not suitable for the matching process. Specifically, pixels whose luminance value in the observation image is greater than or equal to a predetermined value and pixels whose texture is thin are not suitable for the matching process. Pixels with different left and right appearance depending on occlusion are also not suitable for the matching process. Pixels with similar patterns are not suitable for matching. The reliability calculation unit 70 determines that the reliability of the pixels not suitable for the matching process is low as described above, and generates reliability information corresponding to the low reliability for each pixel. The reliability calculation unit 70 generates reliability information corresponding to high reliability for each pixel for pixels other than the pixels that are not suitable for the matching process. The generated reliability information is stored in the RAM 14.

  In the matching process, a plurality of corresponding position candidates on the second image corresponding to the measurement point designated by the user in the first image are detected. For example, the first image is an image corresponding to the left observation visual field in the stereo optical adapter, that is, the left image. For example, the second image is an image corresponding to the right observation visual field in the stereo optical adapter, that is, the right image. The similarity with the measurement position is calculated for each of the plurality of corresponding position candidates. The corresponding position candidate with the highest similarity is determined as the corresponding position. The reliability calculation unit 70 may determine that the reliability of the pixel at the measurement point is low when the similarity of the determined corresponding position is equal to or less than a predetermined value.

  The reliability calculation unit 70 may determine that the reliability of a pixel with a long distance between the distal end portion 21 of the insertion unit 20 and the subject is low. The reliability calculation unit 70 may determine that the reliability of a pixel in which the calculation result of the three-dimensional coordinate is significantly different from the calculation result acquired in the surrounding pixels is low. The reliability calculation unit 70 may comprehensively evaluate the information related to reliability as described above, and may calculate reliability information for each pixel of the observation image.

  In the measurement method using the pattern projection method, when the diffuse reflectance of the subject surface is small, an image in which the pattern projected on the subject is unclear is acquired. When the pattern can be detected from the image, the reliability calculation unit 70 may determine that the reliability of the pixel in the region where the pattern is projected is high. When at least a part of the pattern cannot be detected from the image, the reliability calculation unit 70 may determine that the reliability of the pixel in the region where the pattern including the unclear part is projected is low.

  As described above, the reliability information varies depending on each measurement method. The reliability information is not limited to the above example.

  After the reliability information is calculated, the reliability map generation unit 71 generates a reliability map for presenting the reliability information calculated by the reliability calculation unit 70 to the user (step S3). Each position on the observation image and each position on the reliability map are associated with each other. The reliability information at the corresponding position calculated by the corresponding position calculation unit 76 may correspond to any one of a plurality of reliability levels divided according to the degree of reliability. For example, the plurality of confidence levels includes first and second confidence levels. The first reliability level is a state where the reliability is high and can be measured. The second reliability level is a state in which the reliability is extremely low and measurement is not possible.

  FIG. 7 shows a first example of the reliability map 100. In the reliability map 100 shown in FIG. 7, the region 110 and the region 111 are configured by pixels having reliability information corresponding to the second reliability level. That is, the reliability of each pixel in the region 110 and the region 111 is extremely low. In the reliability map 100, regions other than the region 110 and the region 111 are configured by pixels having reliability information corresponding to the first reliability level. That is, the reliability of each pixel in the region other than the region 110 and the region 111 is high.

  The reliability map generation unit 71 generates a reliability map such that each of the corresponding positions calculated by the corresponding position calculation unit 76 in the reliability map is displayed in a color corresponding to the reliability level at the corresponding position. For example, in the reliability map 100 shown in FIG. 7, each pixel in the area 110 and the area 111 is displayed in red in order to prompt the user to avoid performing measurement in the area 110 and the area 111 with extremely low reliability. The In the reliability map 100 shown in FIG. 7, each pixel in a region other than the region 110 and the region 111 is displayed in a color other than red, for example, white.

  There may be more than two confidence levels. For example, the plurality of reliability levels includes first to third reliability levels. The first reliability level is a state where the reliability is high and can be measured. The second reliability level is a state in which the reliability is low to some extent. In the observation image, measurement is possible in the region corresponding to the second reliability level, but the measurement accuracy is low. The third reliability level is a state in which the reliability is extremely low and measurement is not possible.

  FIG. 8 shows a second example of the reliability map 100. In the reliability map 100 shown in FIG. 8, the region 110 is composed of pixels having reliability information corresponding to the third reliability level. That is, the reliability of each pixel in the region 110 is extremely low. In the reliability map 100, the region 111 and the region 112 are configured by pixels having reliability information corresponding to the second reliability level. That is, the reliability of each pixel in the region 111 and the region 112 is low. In the reliability map 100, regions other than the region 110, the region 111, and the region 112 are configured by pixels having reliability information corresponding to the first reliability level. That is, the reliability of each pixel in the region other than the region 110, the region 111, and the region 112 is high.

  For example, in the reliability map 100 shown in FIG. 8, each pixel in the area 110 is displayed in red in order to prompt the user to avoid performing measurement in the area 110 with extremely low reliability. In the reliability map 100 shown in FIG. 8, each pixel in the region 111 and the region 112 is displayed in yellow in order to avoid performing measurement in the region 111 and the region 112 with a certain degree of reliability as much as possible. In the reliability map 100 shown in FIG. 8, each pixel in the region other than the region 110, the region 111, and the region 112 is displayed in a color other than red and yellow, for example, white.

  The plurality of reliability levels may include at least two reliability levels. One of the two reliability levels is a state in which the reliability is relatively low. The other of the two reliability levels is a state in which the reliability is relatively high. When the reliability at the corresponding position on the reliability map is high for the entire observation image, or when the reliability at the corresponding position on the reliability map is low for the entire observation image, there is only one reliability level. The color corresponding to each reliability level is not limited to the above example.

  In the reliability map, each area may be distinguished by a display form other than color according to the reliability. For example, each area may be displayed in a pattern according to the reliability. Alternatively, characters, symbols, or marks corresponding to the reliability may be displayed in each area.

  A flow of processing after the reliability map is generated will be described. After the reliability map is generated, the monitor 4 displays an observation image based on the video signal (step S4). In the case of stereo measurement, two right and left images are acquired, but one of the two images is displayed as an observation image.

  The measurement index display control unit 72 displays the measurement index on the observation image (step S5). Specifically, the measurement index display control unit 72 generates a graphic image signal of the measurement index. The measurement index display control unit 72 outputs the generated graphic image signal to the video signal processing circuit 12. The video signal processing circuit 12 combines the video signal output from the CCU 9 and the graphic image signal output from the CPU 18a. Thereby, the measurement index is superimposed on the observation image. The video signal processing circuit 12 outputs the synthesized video signal to the monitor 4. The monitor 4 displays an observation image on which a measurement index is superimposed. For example, a measurement index is displayed at the center of the observation image. The position where the measurement index is initially displayed may be a position other than the center of the observation image. For example, the measurement index can be moved only within the observation image.

  After the measurement index is displayed, the reliability map display control unit 77 displays the reliability map at a position different from the position where the measurement index is displayed on the monitor 4, and the reliability map for the display size of the observation image. The reliability map is displayed so that the display size ratio (ratio) becomes a predetermined value (step S6). Specifically, the reliability map display control unit 77 generates a graphic image signal for displaying the reliability map generated by the reliability map generation unit 71. The reliability map display control unit 77 outputs the generated graphic image signal to the video signal processing circuit 12. The video signal processing circuit 12 combines the video signal output from the CCU 9 and the graphic image signal output from the CPU 18a. As a result, the reliability map is superimposed on the observation image. The video signal processing circuit 12 outputs the synthesized video signal to the monitor 4. The monitor 4 displays the observation image on which the reliability map is superimposed.

  For example, the display size of the observation image and the reliability map is the area of the display area or the number of pixels. The display size of the observation image and the reliability map may be the vertical or horizontal length of the display area. The display size of the observation image and the reliability map may be the number of pixels in the vertical or horizontal direction of the display area. The ratio of the reliability map to the display size of the observation image is a predetermined value and is not particularly limited. Therefore, the ratio of the display size of the reliability map to the display size of the observation image may be an arbitrary value larger than zero. However, the ratio of the display size of the reliability map to the display size of the observation image is preferably greater than 0 and 1 or less. In order not to impair the visibility of the observation image, it is preferable that the reliability map is displayed with the same size as the observation image or smaller than the observation image. For example, when the display size of the reliability map is smaller than the display size of the observation image, the reliability map generation unit 71 generates a reliability map whose display size is smaller than the display size of the observation image. The reliability map generation unit 71 generates a reliability map whose display size is the same as the display size of the observation image, and the reliability map display control unit 77 has a display size of the reliability map that is greater than the display size of the observation image. The reliability map may be displayed so that the value becomes smaller. Since the display area of the observation image is secured so that the display size of the reliability map does not exceed the display size of the observation image, the visibility of the observation image does not deteriorate.

  For example, the observation image and the reliability map have a rectangular shape. For example, the ratio of the vertical and horizontal lengths of the observation image is equal to the ratio of the vertical and horizontal lengths of the reliability map. The shape of the observation image and the reliability map is not limited to a rectangle.

  For example, the reliability map includes a region corresponding to each position of the entire observation image. The reliability map may include only the region corresponding to each position of the region including the center of the observation image and smaller than the entire observation image. A part of the observation image may overlap with at least a part of the reliability map. When the observation image and the reliability map overlap, the reliability map is displayed in front of the observation image on the screen of the monitor 4. In the observation image, a portion overlapping the reliability map cannot be visually recognized. When the observation image and the reliability map overlap, the reliability map is displayed so that the measurement index is not hidden by the reliability map.

  The reliability map may be displayed so that a part of the observation image and a part of the reliability map overlap outside the central region of the observation image. For example, the central area of the observation image is an inner area excluding an area of about 30 pixels from the end of the observation image. The occurrence of optical distortion and shading differs depending on the optical system. For this reason, the central region is not limited to the above region. As an example of the operation shown in FIGS. 5 and 6, an example will be described in which the moving range of the measurement index is within the central region of the observation image, and the reliability map is displayed outside the central region of the observation image. In this case, the measurement index and the reliability map do not overlap. The reliability map may be displayed outside the observation image.

  9 and 10 show the observation image 120 and the reliability map 100 displayed on the monitor 4. The measurement index 130 is superimposed on the observation image 120. In the observation image 120, there is a region where luminance is saturated on the surface of the subject. That is, halation has occurred in the observed image 120.

  FIG. 9 shows a first example. In FIG. 9, the reliability map 100 is displayed outside the observation image 120. As shown in FIG. 9, the observation image 120 and the reliability map 100 do not overlap. FIG. 10 shows a second example. As shown in FIG. 10, the reliability map 100 is displayed outside the central region 125 of the observation image 120 at the end of the observation image 120. In FIG. 10, a part of the observation image 120 and a part of the reliability map 100 overlap. The position where the reliability map 100 is displayed is not limited to the position where the reliability map 100 is displayed in FIGS. 9 and 10.

  As shown in FIGS. 9 and 10, the reliability map 100 and the observation image 120 are displayed on the monitor 4. For this reason, the user can grasp the reliability at the measurement position on the observation image 120 on the reliability map 100. As shown in FIGS. 9 and 10, the reliability map 100 is displayed at a position different from the position where the measurement index 130 is displayed. For this reason, the user can grasp the state of the subject at the measurement position on the observation image 120.

  The order in which the processes in steps S2 to S6 are executed is not limited to the order shown in FIG. As long as the observation image and the reliability map are finally displayed, the order of the processes may be changed. Or each process may be performed simultaneously.

  After the reliability map is displayed, the corresponding position calculation unit 76 acquires the position information of the measurement index in the observation image from the measurement index display control unit 72 via the control unit 73. The corresponding position calculation unit 76 calculates a corresponding position on the reliability map corresponding to the position indicated by the position information of the acquired measurement index (step S7).

  After the corresponding position on the reliability map is calculated, the reference indicator display control unit 79 acquires the reliability of the corresponding position (step S8). For example, the reference indicator display control unit 79 acquires the reliability information of the corresponding position from the RAM 14 and determines the reliability based on the reliability information.

  The display form (display pattern) of the reference index is any one of a plurality of display forms corresponding to a plurality of reliability levels. The reference index display control unit 79 displays the reference index in a display form corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit 76.

  The reference index display control unit 79 displays the reference index at the corresponding position calculated by the corresponding position calculation unit 76 in the reliability map. When the reliability of the corresponding position is high, the reference index display control unit 79 displays the reference index in a display form corresponding to the high reliability (Step S9). When the reliability of the corresponding position is low, the reference index display control unit 79 displays the reference index in a display form corresponding to the low reliability (Step S10). Specifically, the reference indicator display control unit 79 generates a graphic image signal of the reference indicator. The reference indicator display control unit 79 outputs the generated graphic image signal to the video signal processing circuit 12. The video signal processing circuit 12 combines the video signal output from the CCU 9 and the graphic image signal output from the CPU 18a. As a result, the reliability map is superimposed on the observation image, and the reference index is superimposed on the reliability map. The video signal processing circuit 12 outputs the synthesized video signal to the monitor 4. The monitor 4 displays an observation image on which a reliability map including a reference index is superimposed.

  An example in which the display form of the reference index changes according to the reliability level at the corresponding position will be described. 11 and 12 show the observation image 120 and the reliability map 100 displayed on the monitor 4. The measurement index 130 is superimposed on the observation image 120, and the reference index 140 is superimposed on the reliability map 100.

  In FIG. 11, the measurement index 130 is in a region where no halation has occurred. In FIG. 11, the reference index 140 is in a region with high reliability. In this case, the reference index 140 is displayed in a circle shape. In FIG. 12, the measurement index 130 is in a region where halation is occurring. In FIG. 12, the reference index 140 is in a region 110 where the reliability is extremely low. In this case, the reference index 140 is displayed in the form of X. 11 and 12, the shape of the reference index 140 is different. In the example shown in FIGS. 11 and 12, the reference indicator display control unit 79 changes the shape of the reference indicator 140 according to the reliability level at the corresponding position.

  As shown in FIGS. 11 and 12, a reference index 140 is displayed on the reliability map 100. For this reason, the user can easily recognize the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130. Therefore, the user can easily grasp the reliability information at the position of the measurement index 130.

  After the measurement index is displayed, the user can move the measurement index to a desired measurement position by operating the operation unit 6. Further, the user can designate a measurement point at a desired measurement position by operating the operation unit 6. The measurement point determination unit 74 determines whether a measurement point is designated by the user by monitoring the state of the operation unit 6 (step S11).

  In step S11, when the measurement point is not designated by the user, the measurement index display control unit 72 determines whether or not the movement of the measurement index is instructed by the user by monitoring the state of the operation unit 6. (Step S12). In step S12, when the movement of the measurement index is not instructed by the user, the process in step S11 is performed.

  In step S12, when the movement of the measurement index is instructed by the user, the measurement index display control unit 72 moves the measurement index on the observation image based on the user instruction input via the operation unit 6. (Step S19). After the measurement index moves, the process in step S7 is performed. In step S7, the corresponding position calculation unit 76 calculates a corresponding position on the reliability map corresponding to the position of the measurement index in accordance with the movement of the measurement index. In step S9 and step S10, the reference index display control unit 79 displays the reference index at the corresponding position calculated by the corresponding position calculation unit 76 in the reliability map. For this reason, the reference index moves on the reliability map in accordance with the movement of the measurement index.

  In step S11, when a measurement point is designated by the user, the measurement point determination unit 74 determines the position of the measurement index as the position of the measurement point (step S13). After the position of the measurement point is determined, the measurement unit 75 performs three-dimensional measurement. In the case of stereo measurement, three-dimensional measurement is executed according to the measurement principle described with reference to FIG. First, the measurement unit 75 calculates the three-dimensional coordinates of the position of the measurement point determined by the measurement point determination unit 74 based on the observation image (step S14). Subsequently, the measurement unit 75 determines whether or not the specification of the measurement point by the user has been completed (step S15). For example, when two measurement points are designated in measuring the distance between two points, the measurement unit 75 can determine that the designation of the measurement points by the user has been completed. For example, in the measurement of the area, when the user inputs a measurement point designation end instruction via the operation unit 6, the measurement unit 75 can determine that the user has specified the measurement point.

  In step S15, when the measurement point designation is not completed, the measurement index display control unit 72 displays the next measurement index on the observation image (step S16). The process in step S16 is the same as the process in step S5. After the next measurement index is displayed, the process in step S7 is performed.

  In step S15, when the measurement point designation is completed, the measurement unit 75 performs three-dimensional measurement based on the three-dimensional coordinates of the position of each measurement point (step S17). At this time, three-dimensional measurement is performed according to a measurement function selected in advance by the user. Thus, the distance between two points, the distance between the two points and the distance between one point, the area, the depth, the surface shape, and the like are calculated.

  After the three-dimensional measurement is completed, the measurement unit 75 displays the measurement result (step S18). Specifically, the measurement unit 75 generates a graphic image signal for displaying the measurement result. The measurement unit 75 outputs the generated graphic image signal to the video signal processing circuit 12. The video signal processing circuit 12 combines the video signal output from the CCU 9 and the graphic image signal output from the CPU 18a. Thereby, the measurement result is superimposed on the observation image. The video signal processing circuit 12 outputs the synthesized video signal to the monitor 4. The monitor 4 displays an observation image on which the measurement result is superimposed. When the process in step S18 ends, the measurement process ends.

  The reference indicator display control unit 79 may change the color of the reference indicator according to the reliability level at the corresponding position. For example, when there is a reference index in an area with high reliability, the reference index display control unit 79 displays the reference index in blue. When there is a reference index in an area with low reliability, the reference index display control unit 79 displays the reference index in red. The reference indicator display control unit 79 may change the size of the reference indicator according to the reliability level at the corresponding position. For example, when there is a reference index in an area with high reliability, the reference index display control unit 79 displays the reference index small. When there is a reference index in an area with low reliability, the reference index display control unit 79 displays the reference index in a large size. The reference indicator display control unit 79 may blink the reference indicator only when the reliability level at the corresponding position is low.

  As described above, the reliability map generation unit 71 is a reliability that displays each of the corresponding positions calculated by the corresponding position calculation unit 76 in the reliability map in a color corresponding to the reliability level at the corresponding position. Generate a map. The reference index display control unit 79 displays the reference index in any one of a plurality of colors, and sets the display color of the reference index to the reliability level at the corresponding position calculated by the corresponding position calculation unit 76. It may be different from the corresponding color.

  13 to 16 show the reliability map 100 displayed on the monitor 4. 13 to 16, each pixel in the region 110 is displayed in red, and each pixel in the region 111 and the region 112 is displayed in yellow. In FIG. 13, the reference index 140 is in a state where it overlaps the area 111. In this state, when the reference index 140 is displayed in yellow, the color of the region 111 and the reference index 140 is the same in yellow. For this reason, it may be difficult for the user to notice that the reference index 140 is in the region 111. Therefore, for example, as shown in FIG. 14, when the reference index 140 overlaps the area 111, the reference index 140 is set so that the color of the reference index 140 is different from the color (yellow) of the area 111, for example, green. The color of is changed.

  In FIG. 15, the reference index 140 is in a state where it overlaps the area 110. In this state, when the reference index is displayed in red, the color of the area 110 and the reference index 140 is the same in red. For this reason, it may be difficult for the user to notice that the reference index 140 is in the area 110. Therefore, for example, as shown in FIG. 16, when the reference index 140 overlaps the area 110, the reference index 140 has a color different from the color (red) of the area 110, for example, green so that the color of the reference index is The color is changed.

  The reference indicator display control unit 79 generates a graphic image signal of the reference indicator according to the display form of the reference indicator. The display form of the reference index changes in real time according to the position of the measurement index in the observation image. The change in the display form of the reference index is not limited to the above example. As long as the change in the reliability level can be easily notified to the user, the change in the display form of the reference index may be any change. By changing the display form of the reference index, the user can easily notice the change in the reliability level at the corresponding position.

  The measurement method of the first embodiment includes an imaging step (step S1), an observation image generation step (step S1), a reliability calculation step (step S2), a reliability map generation step (step S3), and a display. It has a step (step S4 and step S6), a measurement index display step (step S5), a measurement point determination step (step S13), and a measurement step (step S17).

  As described above, by displaying the reliability map indicating the reliability due to various factors together with the observation image, the user can designate a measurement point on the observation image while confirming the reliability. The user can immediately grasp the reliability of a desired measurement position by checking the reliability map. In addition, the user can easily perform measurement while avoiding a region with low reliability. In addition, the reliability map is displayed at a position different from the position where the measurement index is displayed on the monitor 4. For this reason, the fall of the visibility of a measurement position can be suppressed in an observation image.

  As described above, the corresponding position on the reliability map corresponding to the position of the measurement index in the observation image is calculated, and the reference index is displayed at the corresponding position. The user can easily grasp the reliability of the calculation result of the three-dimensional coordinates at the position of the measurement index by confirming the position of the reference index in the reliability map. Further, since the reference index moves in conjunction with the measurement index, the user can accurately grasp the reliability at the position of the measurement index. For this reason, the user can easily perform measurement while avoiding a region with low reliability.

  As described above, the reliability information corresponds to one of a plurality of reliability levels divided according to the degree of reliability, and each of the corresponding positions corresponds to the reliability level at the corresponding position. A confidence map that is displayed in color is generated. For this reason, the user can easily grasp the reliability of the entire observation image. The user can easily grasp the reliability at each corresponding position on the reliability map by checking the visually easy-to-understand reliability map as described above. Therefore, the endoscope apparatus 1 can make the user easily determine whether or not the measurement position is suitable for measurement.

  As described above, the reference index is displayed in a display form corresponding to the reliability level at the corresponding position, so that the user can easily notice the change in the reliability according to the position of the measurement index. In particular, when the measurement index is in an area with low reliability, the display form of the reference index changes, so that the user can immediately know that the reliability at the position of the measurement index is low. At this time, since the color of the reference index changes, the user can easily recognize that the reliability at the position of the measurement index is low. As a result, the user can easily perform measurement while avoiding a region with low reliability.

  As described above, the reference index is displayed in a color different from the color corresponding to the reliability level at the corresponding position, so that it is easy to visually distinguish the reference index from the low reliability area. For this reason, when there is a reference index in a region with low reliability, the user can clearly grasp the position of the reference index.

(Modification of the first embodiment)
In the modification of the first embodiment, when the position where the reliability map is displayed is fixed, a situation occurs in which the user cannot move the measurement index to the position where the reliability map is displayed. For this reason, the reliability map display control unit 77 displays the reliability map so that the measurement index and the reliability map do not interfere with each other. That is, when the measurement index and the reliability map overlap, the reliability map display control unit 77 moves the reliability map so that the measurement index and the reliability map do not overlap.

  In the modification of the first embodiment, the measurement process is performed according to the procedure shown in FIGS. In the modification of the first embodiment, the process shown in FIG. 6 is changed to the process shown in FIG. In the modification of the first embodiment, the processing shown in FIG. 5 is not changed. The process shown in FIG. 17 will be described while referring to differences from the process shown in FIG.

  In FIG. 17, the difference from FIG. 6 is step S20 and step S21. After the measurement index moves in step S19, the reliability map display control unit 77 determines whether or not the measurement index overlaps with the reliability map (step S20). In step S20, if the measurement index and the reliability map do not overlap, the process in step S7 is performed.

  If the measurement index and the reliability map overlap in step S20, the reliability map display control unit 77 moves the reliability map to a position that does not overlap the measurement index (step S21). For example, the reliability map display control unit 77 displays the reliability map at a position different from a predetermined area including the position where the measurement index is displayed. The predetermined region is a region that includes the measurement index in the observation image and is smaller than the central region of the observation image. After the reliability map is moved, the process in step S7 is performed.

  Regarding the points other than the above, the process shown in FIG. 17 is the same as the process shown in FIG.

  18 to 20 show the observation image 120 and the reliability map 100 displayed on the monitor 4. The measurement index 130 is superimposed on the observation image 120, and the reference index 140 is superimposed on the reliability map 100.

  In FIG. 18, the reliability map 100 is displayed on the right side of the measurement index 130. When the measurement index 130 moves from the position shown in FIG. 18 to the position shown in FIG. 19, that is, in the downward direction, the measurement index 130 does not overlap the reliability map 100. For this reason, the position where the reliability map 100 is displayed in FIG. 19 is the same as the position where the reliability map 100 is displayed in FIG. On the other hand, as shown in FIG. 20, when the measurement index 130 moves rightward and moves to a position overlapping the reliability map 100 in FIG. 18, the reliability map 100 moves. At this time, the reliability map 100 moves to a position that does not overlap the region 131 centered on the position of the measurement index 130.

  The modification of the first embodiment can also be applied to second to fifth embodiments described later.

  As described above, the reliability map moves so that the measurement index and the reliability map do not overlap. For this reason, it is possible to suppress a decrease in the visibility of the area around the measurement index in the observation image and to secure a display area for the observation image. It is important that the area around the measurement index is visible when the user designates the measurement point. Also, a wide observation image display area is important in endoscopy.

(Second Embodiment)
In the second embodiment of the present invention, the CPU 18a in the first embodiment is changed to a CPU 18b shown in FIG. FIG. 21 shows a functional configuration of the CPU 18b. The configuration shown in FIG. 21 will be described while referring to differences from the configuration shown in FIG.

  In the CPU 18b, the reliability information notification unit 78a in the CPU 18a shown in FIG. 3 is changed to a reliability information notification unit 78b. The reliability information notification unit 78 b includes a measurement index display control unit 72. As in the first embodiment, the measurement index display control unit 72 displays a measurement index on the observation image for the user to specify the position of the measurement point on the observation image. As in the first embodiment, the measurement index display control unit 72 moves the measurement index on the observation image based on a user instruction input via the operation unit 6.

  As in the first embodiment, the reliability information at the corresponding position calculated by the corresponding position calculation unit 76 corresponds to any one of a plurality of reliability levels divided according to the degree of reliability. In the second embodiment, the display form (display pattern) of the measurement index is any one of a plurality of display forms corresponding to a plurality of reliability levels. In the second embodiment, the measurement index display control unit 72 displays the measurement index in a display form corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit 76.

  With respect to points other than the above, the configuration shown in FIG. 21 is the same as the configuration shown in FIG.

  Details of the measurement process in the second embodiment will be described. In the second embodiment, measurement processing is performed according to the procedure shown in FIGS. 22 and 6. In the second embodiment, the process shown in FIG. 5 is changed to the process shown in FIG. In the second embodiment, the process shown in FIG. 6 is not changed. The process shown in FIG. 22 will be described while referring to differences from the process shown in FIG.

  In FIG. 22, step S9 in FIG. 5 is changed to step S22. In FIG. 22, step S10 in FIG. 5 is changed to step S23.

  When the reliability of the corresponding position calculated by the corresponding position calculation unit 76 is high, the measurement index display control unit 72 displays the measurement index in a display form corresponding to the high reliability (step S22). When the reliability of the corresponding position is low, the measurement index display control unit 72 displays the measurement index in a display form corresponding to the low reliability (step S23). Specifically, the measurement index display control unit 72 generates a graphic image signal of the measurement index. The measurement index display control unit 72 outputs the generated graphic image signal to the video signal processing circuit 12. The video signal processing circuit 12 synthesizes the video signal output from the CCU 9 and the graphic image signal output from the CPU 18b. Thereby, the measurement index is superimposed on the observation image. The video signal processing circuit 12 outputs the synthesized video signal to the monitor 4. The monitor 4 displays an observation image on which a measurement index is superimposed.

  Regarding the points other than the above, the process shown in FIG. 22 is the same as the process shown in FIG.

  An example in which the display form of the measurement index changes in accordance with the reliability level at the corresponding position will be described. 23 and 24 show the observation image 120 and the reliability map 100 displayed on the monitor 4. The measurement index 130 is superimposed on the observation image 120.

  In FIG. 23, the measurement index 130 is in a region where no halation occurs. In FIG. 23, the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130 is in a region with high reliability. In this case, the measurement index 130 is displayed in a circle. In FIG. 24, the measurement index 130 is in a region where halation occurs. In FIG. 24, the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130 is in the region 110 where the reliability is extremely low. In this case, the measurement index 130 is displayed in the form of X. 23 and 24, the shape of the measurement index 130 is different. In the example shown in FIGS. 23 and 24, the measurement index display control unit 72 changes the shape of the measurement index 130 according to the reliability level at the corresponding position.

  As in the first embodiment, the reliability map generation unit 71 displays each of the corresponding positions calculated by the corresponding position calculation unit 76 in the reliability map in a color corresponding to the reliability level at the corresponding position. A reliability map like this is generated. The measurement index display control unit 72 may display the measurement index in any one of a plurality of colors. The measurement index display control unit 72 may make the display color of the measurement index the same as the color corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit 76.

  The measurement index display control unit 72 may change the color of the measurement index according to the reliability level at the corresponding position. FIG. 25 shows the observation image 120 and the reliability map 100 displayed on the monitor 4. The measurement index 130 is superimposed on the observation image 120.

  In FIG. 25, the measurement index 130 is in a region where halation is occurring. In FIG. 25, the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130 is in the region 110 where the reliability is extremely low. For example, when the area 110 of the reliability map 100 is displayed in red, the display color of the measurement index 130 is changed so that the measurement index 130 is displayed in red. Similarly, when the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130 is in the area 111 or the area 112, the measurement index 130 is displayed in the same color as the color of the area 111 or the area 112. In this way, the display color of the measurement index 130 is changed. For example, when the area 111 and the area 112 are displayed in yellow, the display color of the measurement index 130 is also changed to yellow. When the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130 is in a region different from any of the region 110, the region 111, and the region 112, the measurement index 130 has a color different from the colors of these regions. For example, it is displayed in blue.

  The measurement index display control unit 72 may change the size of the measurement index according to the reliability level at the corresponding position. For example, when there is a measurement index in an area with high reliability, the measurement index display control unit 72 displays the measurement index small. When there is a measurement index in an area with low reliability, the measurement index display control unit 72 displays the measurement index large. The measurement index display control unit 72 may blink the measurement index only when the reliability level at the corresponding position is low.

  The measurement index display control unit 72 generates a graphic image signal of the measurement index according to the display form of the measurement index. The display form of the measurement index changes in real time according to the position of the measurement index in the observation image. The change in the display form of the measurement index is not limited to the above example. As long as the change in the reliability level can be easily notified to the user, the change in the display form of the measurement index may be any change. By changing the display form of the measurement index, the user can easily notice the change in the reliability level at the corresponding position.

  As described above, the measurement index is displayed in a display form corresponding to the reliability level at the corresponding position, so that the user can easily notice the change in the reliability according to the position of the measurement index. In particular, when the measurement index is in a region with low reliability, the display form of the measurement index changes, so that the user can immediately know that the reliability at the position of the measurement index is low. At this time, since the color of the measurement index changes, the user can easily recognize that the reliability at the position of the measurement index is low. As a result, the user can easily perform measurement while avoiding a region with low reliability.

  As described above, by displaying the measurement index in the same color as the color corresponding to the reliability level at the corresponding position, the user can easily grasp the reliability from the color of the measurement index. The user can easily grasp the reliability of the entire observation image by the reliability map, and can grasp the reliability of the position that the user wants to measure from only the measurement index. The user only needs to check the reliability map when the color of the measurement index is the same as the color of the area with low reliability. For this reason, the complicated operation | work which a user always compares an observation image with a reliability map is unnecessary.

(Third embodiment)
In the third embodiment of the present invention, the CPU 18a in the first embodiment is changed to a CPU 18c shown in FIG. FIG. 26 shows a functional configuration of the CPU 18c. The configuration shown in FIG. 26 will be described while referring to differences from the configuration shown in FIG.

  In the CPU 18c, the reliability information notification unit 78a in the CPU 18a shown in FIG. 3 is changed to a reliability information notification unit 78c. The reliability information notification unit 78c includes an outer edge display control unit 80 that controls the display form of the outer edge of the display area of at least one of the observation image and the reliability map.

  As in the first embodiment, the reliability information at the corresponding position calculated by the corresponding position calculation unit 76 corresponds to any one of a plurality of reliability levels divided according to the degree of reliability. In the third embodiment, the display form (display pattern) of the outer edge of the display area of at least one of the observation image and the reliability map is any one of a plurality of display forms corresponding to a plurality of reliability levels. The outer edge display control unit 80 displays the outer edge of the display area of at least one of the observation image and the reliability map in a display form corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit 76.

  With respect to points other than those described above, the configuration illustrated in FIG. 26 is the same as the configuration illustrated in FIG.

  Details of the measurement process in the third embodiment will be described. In the third embodiment, measurement processing is performed according to the procedure shown in FIGS. 27 and 6. In the third embodiment, the process shown in FIG. 5 is changed to the process shown in FIG. In the third embodiment, the process shown in FIG. 6 is not changed. The process shown in FIG. 27 is different from the process shown in FIG.

  In FIG. 27, step S9 in FIG. 5 is changed to step S24. In FIG. 27, step S10 in FIG. 5 is changed to step S25.

  When the reliability of the corresponding position calculated by the corresponding position calculation unit 76 is high, the outer edge display control unit 80 displays the outer edge of at least one of the observation image and the reliability map in a display form corresponding to the high reliability. (Step S24). When the reliability of the corresponding position is low, the outer edge display control unit 80 displays the outer edge of the display area of at least one of the observation image and the reliability map in a display form corresponding to the low reliability (step S25). Specifically, the outer edge display control unit 80 generates a graphic image signal of the outer edge of the display area. The outer edge display control unit 80 outputs the generated graphic image signal to the video signal processing circuit 12. The video signal processing circuit 12 synthesizes the video signal output from the CCU 9 and the graphic image signal output from the CPU 18c. Thereby, the display form of the outer edge of the display area of at least one of the observation image and the reliability map is controlled. The video signal processing circuit 12 outputs the synthesized video signal to the monitor 4. The monitor 4 displays an observation image and a reliability map.

  Regarding the points other than the above, the process shown in FIG. 27 is the same as the process shown in FIG.

  The outer edge display control unit 80 may make the display forms of the observation image display area outer edge and the reliability map display area outer edge the same. For example, the outer edge display control unit 80 may make the display colors of the observation image display area outer edge and the reliability map display area outer edge the same. In this case, the outer edge of the display area of the observation image and the reliability map is displayed in the same display form corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit 76.

  As in the first embodiment, the reliability map generation unit 71 displays each of the corresponding positions calculated by the corresponding position calculation unit 76 in the reliability map in a color corresponding to the reliability level at the corresponding position. A reliability map like this is generated. The outer edge display control unit 80 may display the outer edge of at least one of the observation image and the reliability map in any one of a plurality of colors. The outer edge display control unit 80 may cause the display color of the outer edge of at least one of the observation image and the reliability map to be the same as the color corresponding to the reliability level at the corresponding position calculated by the corresponding position calculating unit 76. Good.

  An example in which the display form of the outer edge of the display area of at least one of the observation image and the reliability map changes according to the reliability level at the corresponding position will be described. 28 to 30 show the observation image 120 and the reliability map 100 displayed on the monitor 4. The measurement index 130 is superimposed on the observation image 120.

  In FIG. 28, the measurement index 130 is in a region where no halation has occurred. In FIG. 28, the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130 is in a region with high reliability. In this case, the display region outer edge 121 of the observation image 120 is displayed in the first color corresponding to high reliability. The first color may be the same as the color at the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130. In FIG. 29, the measurement index 130 is in a region where halation is occurring. In FIG. 29, the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130 is in the region 110 where the reliability is extremely low. In this case, the display area outer edge 121 of the observation image 120 is displayed in the second color corresponding to the remarkably low reliability. The second color is different from the first color. The second color may be the same as the color at the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130. For example, when the area 110 is displayed in red, the display area outer edge 121 of the observation image 120 may be displayed in red.

  In the example shown in FIGS. 28 and 29, only the display area outer edge 121 of the observation image 120 is displayed in a color corresponding to the reliability at the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130. Only the outer edge of the display area of the reliability map 100 may be displayed in a color corresponding to the reliability at the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130.

  In FIG. 30, the measurement index 130 is in a region where halation is occurring. In FIG. 30, the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130 is in the region 110 where the reliability is extremely low. In this case, the display area outer edge 121 of the observation image 120 and the display area outer edge 101 of the reliability map 100 are displayed in the second color corresponding to the remarkably low reliability. In the example shown in FIG. 30, the display area outer edge 121 of the observation image 120 and the display area outer edge 101 of the reliability map 100 correspond to the reliability at the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130. Displayed in color. As described above, the second color may be the same as the color at the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130. For example, when the area 110 is displayed in red, the display area outer edge 121 of the observation image 120 and the display area outer edge 101 of the reliability map 100 may be displayed in red.

  In the example illustrated in FIG. 30, the outer edge display control unit 80 changes the color of the outer edge of the display area of at least one of the observation image 120 and the reliability map 100 according to the reliability level at the corresponding position.

  The outer edge display control unit 80 may change the type of lines constituting the outer edge of the display area according to the reliability level at the corresponding position. For example, when there is a measurement index in an area with high reliability, the outer edge display control unit 80 displays a line constituting the outer edge of the display area with a solid line. When there is a measurement index in an area with low reliability, a line that forms the outer edge of the display area is displayed as a broken line. The outer edge display control unit 80 may change the thickness of the outer edge of the display area according to the reliability level at the corresponding position. For example, when there is a measurement index in an area with high reliability, the outer edge display control unit 80 displays a thin line that forms the outer edge of the display area. When there is a measurement index in an area with low reliability, the outer edge display control unit 80 displays a line forming the outer edge of the display area thickly. The outer edge display control unit 80 may blink the outer edge of the display area only when the reliability level at the corresponding position is low.

  The outer edge display control unit 80 generates a graphic image signal of the outer edge of the display area according to the display form of the outer edge of the display area of at least one of the observation image and the reliability map. The display form of at least one of the observation image and the reliability map changes in real time according to the position of the measurement index in the observation image. The change in the display form of the outer edge of the display area is not limited to the above example. As long as the change in the reliability level can be easily notified to the user, the change in the display form of the outer edge of the display area of at least one of the observation image and the reliability map may be any change. By changing the display form of the outer edge of the display area, the user can easily notice the change in the reliability level at the corresponding position.

  As described above, since the outer edge of the display area of at least one of the observation image and the reliability map is displayed in a display form corresponding to the reliability level at the corresponding position, the user can trust the position corresponding to the position of the measurement index. Easy to notice changes in degree. In particular, when the measurement index is in an area with low reliability, the display form of the outer edge of the display area changes, so that the user can immediately know that the reliability at the position of the measurement index is low. At this time, since the color of the outer edge of the display area changes, the user can easily recognize that the reliability at the position of the measurement index is low. As a result, the user can easily perform measurement while avoiding a region with low reliability.

  As described above, the display area outer edge is displayed in the same color as the color corresponding to the reliability level at the corresponding position, so that the user can easily grasp the reliability from the color of the display area outer edge. The user can easily grasp the reliability of the entire observation image by the reliability map, and can grasp the reliability of the position that the user wants to measure from only the outer edge of the display area. The user only needs to check the reliability map when the color of the outer edge of the display area is the same as the color of the area with low reliability.

  As described above, since the display forms of the outer edge of the display area of the observation image and the outer edge of the display area of the reliability map are the same, the user can more easily grasp the reliability. For example, since the display colors of the outer edge of the display area of the observation image and the outer edge of the display area of the reliability map are the same, the user can more easily grasp the reliability.

(Fourth embodiment)
In the fourth embodiment of the present invention, the CPU 18a in the first embodiment is changed to a CPU 18d shown in FIG. FIG. 31 shows a functional configuration of the CPU 18d. The configuration shown in FIG. 31 will be described while referring to differences from the configuration shown in FIG.

  In the CPU 18d, the reliability information notification unit 78a in the CPU 18a shown in FIG. 3 is changed to a reliability information notification unit 78d. The reliability information notification unit 78 d includes a reliability map display control unit 77. As in the first embodiment, the reliability map display control unit 77 has a predetermined ratio of the display size of the reliability map to the display size of the observation image at a position different from the position where the measurement index is displayed on the monitor 4. Display the confidence map so that it is a value. In the fourth embodiment, the reliability map display control unit 77 controls the display size of the reliability map. Specifically, the reliability map display control unit 77 controls the ratio of the display size of the reliability map to the display size of the observation image.

  As in the first embodiment, the reliability information at the corresponding position calculated by the corresponding position calculation unit 76 corresponds to any one of a plurality of reliability levels divided according to the degree of reliability. In the fourth embodiment, the ratio of the display size of the reliability map to the display size of the observation image is any one of a plurality of values corresponding to a plurality of reliability levels. The reliability map display control unit 77 displays the reliability map so that the above ratio becomes a value corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit 76.

  With respect to points other than the above, the configuration shown in FIG. 31 is the same as the configuration shown in FIG.

  Details of the measurement processing in the fourth embodiment will be described. In the fourth embodiment, measurement processing is performed according to the procedure shown in FIGS. 32 and 6. In the fourth embodiment, the process shown in FIG. 5 is changed to the process shown in FIG. In the fourth embodiment, the process shown in FIG. 6 is not changed. The process shown in FIG. 32 will be described while referring to differences from the process shown in FIG.

  32, step S9 in FIG. 5 is changed to step S26. 32, step S10 in FIG. 5 is changed to step S27.

  When the reliability of the corresponding position calculated by the corresponding position calculation unit 76 is high, the reliability map display control unit 77 displays the reliability map with a display size corresponding to the high reliability (step S26). When the reliability of the corresponding position is low, the reliability map display control unit 77 displays the reliability map with a display size corresponding to the low reliability (step S27). Specifically, the reliability map display control unit 77 generates a graphic image signal for displaying the reliability map with a display size corresponding to the reliability. The measurement index display control unit 72 outputs the generated graphic image signal to the video signal processing circuit 12. The video signal processing circuit 12 synthesizes the video signal output from the CCU 9 and the graphic image signal output from the CPU 18d. As a result, the reliability map is superimposed on the observation image. The video signal processing circuit 12 outputs the synthesized video signal to the monitor 4. The monitor 4 displays the observation image on which the reliability map is superimposed.

  Regarding the points other than the above, the process shown in FIG. 32 is the same as the process shown in FIG.

  An example in which the display size of the reliability map changes in accordance with the reliability level at the corresponding position will be described. 33 and 34 show the observation image 120 and the reliability map 100 displayed on the monitor 4. The measurement index 130 is superimposed on the observation image 120.

  In FIG. 33, the measurement index 130 is in a region where no halation has occurred. In FIG. 33, the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130 is in a region with high reliability. In this case, the reliability map 100 is displayed in the first display size corresponding to the high reliability. In this case, the ratio of the display size of the reliability map 100 to the display size of the observation image 120 is the first value. In FIG. 34, the measurement index 130 is in a region where halation occurs. In FIG. 34, the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130 is in the region 110 where the reliability is extremely low. In this case, the reliability map 100 is displayed in the second display size corresponding to the remarkably low reliability. The second display size is different from the first display size. For example, the second display size is larger than the first display size. In this case, the ratio of the display size of the reliability map 100 to the display size of the observation image 120 is the second value. The second value is different from the first value. For example, the second value is larger than the first value.

  In the fourth embodiment, the display size of the observation image 120 in FIG. 33 and the display size of the observation image 120 in FIG. 34 are both fixed to a constant value. That is, the display size of the observation image 120 is not changed, and only the display size of the reliability map 100 is changed. However, it is not limited to this. That is, the endoscope apparatus 1 may change the ratio of the display size of the reliability map to the observation image by changing not only the reliability map 100 but also the display size of the observation image 120.

  The reliability map display control unit 77 generates a graphic image signal of the reliability map according to the display size of the reliability map. The display size of the reliability map changes in real time according to the position of the measurement index in the observation image. By changing the display size of the reliability map, the user can easily notice the change in the reliability level at the corresponding position.

  As described above, the reliability map is displayed with a display size corresponding to the reliability level at the corresponding position, so that the user can easily notice the change in the reliability according to the position of the measurement index. In particular, when the display index of the reliability map is changed when the measurement index is in a low reliability area, the user can immediately know that the reliability at the position of the measurement index is low. As a result, the user can easily perform measurement while avoiding a region with low reliability. The user only has to check the reliability map when the display size of the reliability map changes. For this reason, the complicated operation | work which a user always compares an observation image with a reliability map is unnecessary.

(Fifth embodiment)
In the fifth embodiment of the present invention, the CPU 18a in the first embodiment is changed to a CPU 18e shown in FIG. FIG. 35 shows a functional configuration of the CPU 18e. The configuration shown in FIG. 35 will be described while referring to differences from the configuration shown in FIG.

  The CPU 18e has a grid line display control unit 81 instead of the reliability information notification unit 78a in the CPU 18a shown in FIG. The grid line display control unit 81 displays the first grid line at the first position on the observation image. In addition, the grid line display control unit 81 displays the second grid line at the second position on the reliability map corresponding to the first position.

  As in the first embodiment, the reliability information at the corresponding position calculated by the corresponding position calculation unit 76 corresponds to any one of a plurality of reliability levels divided according to the degree of reliability. In the fifth embodiment, the display form (display pattern) of at least one of the first grid line and the second grid line is any one of a plurality of display forms corresponding to a plurality of reliability levels. In the fifth embodiment, the grid line display control unit 81 displays at least one of the first grid line and the second grid line in a display form corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit 76. Is displayed.

  Regarding the points other than the above, the configuration shown in FIG. 35 is the same as the configuration shown in FIG.

  Details of the measurement processing in the fifth embodiment will be described. In the fifth embodiment, measurement processing is performed according to the procedure shown in FIGS. 36 and 6. In the fifth embodiment, the process shown in FIG. 5 is changed to the process shown in FIG. In the fifth embodiment, the process shown in FIG. 6 is not changed. The process shown in FIG. 36 will be described while referring to differences from the process shown in FIG.

  36, step S9 in FIG. 5 is changed to step S28. 36, step S10 in FIG. 5 is changed to step S29.

  When the reliability of the corresponding position calculated by the corresponding position calculation unit 76 is high, the grid line display control unit 81 displays at least one of the first grid line and the second grid line in a display form corresponding to the high reliability. It is displayed (step S28). When the reliability of the corresponding position is low, the grid line display control unit 81 displays at least one of the first grid line and the second grid line in a display form corresponding to the low reliability (step S29). In step S28 and step S29, the grid line display control unit 81 displays the first grid line at the first position on the observation image, and the second position on the reliability map corresponding to the first position. To display the second grid line. Specifically, the grid line display control unit 81 generates a graphic image signal of the first grid line and the second grid line. The grid line display control unit 81 outputs the generated graphic image signal to the video signal processing circuit 12. The video signal processing circuit 12 synthesizes the video signal output from the CCU 9 and the graphic image signal output from the CPU 18e. As a result, the first grid line is superimposed on the observation image, and the second grid line is superimposed on the reliability map. The video signal processing circuit 12 outputs the synthesized video signal to the monitor 4. The monitor 4 displays an observation image on which the first grid lines are superimposed and a reliability map on which the second grid lines are superimposed. Since the positional relationship between the observation image and the reliability map is obtained in advance, the grid line display control unit 81 easily calculates the second position on the reliability map corresponding to the first position on the observation image. can do.

  When the display size of the observation image is different from the display size of the reliability map, the interval between the first grid lines is different from the interval between the second grid lines. For example, the ratio between the interval between the first grid lines and the interval between the second grid lines is the same as the ratio between the display size of the observation image and the display size of the reliability map. For example, when the vertical or horizontal length of the reliability map is ½ of the vertical or horizontal length of the observation image, the interval between the second grid lines is ½ of the interval between the first grid lines. It is.

  Regarding the points other than the above, the process shown in FIG. 36 is the same as the process shown in FIG.

  An example will be described in which the display form of at least one of the first grid line and the second grid line changes according to the reliability level at the corresponding position. 37 and 38 show the observation image 120 and the reliability map 100 displayed on the monitor 4.

  In the observation image 120, a measurement index 130, a plurality of first grid lines 160, and a plurality of first grid lines 161 are displayed. In the reliability map 100, a plurality of second grid lines 170 and a plurality of second grid lines 171 are displayed. The first grid line 160, the first grid line 161, the second grid line 170, and the second grid line 171 are indices of positions in the observed image 120 and the reliability map 100, respectively.

  The first grid line 160 and the second grid line 170 are grid lines arranged in the vertical direction. The first grid line 161 and the second grid line 171 are grid lines arranged in the horizontal direction. The second grid line 170 is displayed at a second position on the reliability map 100 corresponding to the first position on the observation image 120 on which the first grid line 160 is displayed. The second grid line 171 is displayed at a second position on the reliability map 100 corresponding to the first position on the observation image 120 on which the first grid line 161 is displayed. In the observation image 120, the first grid lines 160 and the first grid lines 161 are displayed at regular intervals. In the reliability map 100, the second grid lines 170 and the second grid lines 171 are displayed at equal intervals. The display size of the reliability map 100 is smaller than the display size of the observation image 120. Therefore, the interval between the second grid line 170 and the second grid line 171 in the reliability map 100 is larger than the interval between the first grid line 160 and the first grid line 161 in the observation image 120. small.

  In FIG. 37, the measurement index 130 is in a region where no halation occurs. In FIG. 37, the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130 is in a region with high reliability. In this case, the first grid line 160 and the first grid line 161 are displayed with a first thickness corresponding to high reliability. In FIG. 38, the measurement index 130 is in a region where halation occurs. In FIG. 38, the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130 is in the region 110 where the reliability is extremely low. In this case, the first grid line 160 and the first grid line 161 are displayed with the second thickness corresponding to the remarkably low reliability. The second thickness is different from the first thickness. For example, the second thickness is thicker than the first thickness.

  Of the first grid line 160 and the first grid line 161, only the first grid line 160 is displayed in the observation image 120, and the second grid line 170 and the second grid line 171 are displayed in the reliability map 100. Of these, only the second grid line 170 may be displayed. Alternatively, only the first grid line 161 among the first grid line 160 and the first grid line 161 is displayed in the observation image 120, and the second grid line 170 and the second grid line in the reliability map 100 are displayed. Of the 171, only the second grid line 171 may be displayed. In the observation image 120, only one first grid line 160 or first grid line 161 may be displayed. In the reliability map 100, only one second grid line 170 or second grid line 171 may be displayed. In the example shown in FIGS. 37 and 38, the first grid line 160, the first grid line 161, the second grid line 170, and the second grid line 171 are straight lines. The first grid line 160, the first grid line 161, the second grid line 170, and the second grid line 171 may be lines other than straight lines. The first grid line 160 and the first grid line 161 may be lines that divide the display area of the observation image 120 into at least two. The second grid line 170 and the second grid line 171 may be lines that divide the display area of the reliability map 100 into at least two.

  In the example shown in FIGS. 37 and 38, only the thickness of the first grid line of the observation image 120 changes according to the reliability at the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130. . Only the thickness of the second grid line of the reliability map 100 may change according to the reliability at the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130. The thickness of the first grid line of the observation image 120 and the second grid line of the reliability map 100 changes according to the reliability at the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130. May be.

  In the example shown in FIGS. 37 and 38, the grid line display control unit 81 changes the thickness of at least one of the first grid line and the second grid line according to the reliability level at the corresponding position.

  As in the first embodiment, the reliability map generation unit 71 displays each of the corresponding positions calculated by the corresponding position calculation unit 76 in the reliability map in a color corresponding to the reliability level at the corresponding position. A reliability map like this is generated. The grid line display control unit 81 may display the first grid line in any one of a plurality of colors. The grid line display control unit 81 may make the display color of the first grid line the same as the color corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit 76.

  For example, in the reliability map 100 shown in FIG. 37, when the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130 is in the area 110, the first grid line 160 and the first grid line 161 are Are displayed in the same color as the color of the area 110. For example, when the area 110 is displayed in red, the first grid line 160 and the first grid line 161 are displayed in red. Similarly, when the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130 is in the region 111 or the region 112, the first grid line 160 and the first grid line 161 are the region 111 or the region 112. Displayed in the same color as. For example, when the region 111 and the region 112 are displayed in yellow, the first grid line 160 and the first grid line 161 are displayed in yellow. When the corresponding position on the reliability map 100 corresponding to the position of the measurement index 130 is in a region different from any of the region 110, the region 111, and the region 112, the first grid line 160 and the first grid line 161 are , These areas are displayed in a different color. In any of the above cases, the second grid line 170 and the second grid line 171 may be displayed in the same color as the first grid line 160 and the first grid line 161.

  The grid line display control unit 81 may change at least one of the first grid line and the second grid line according to the reliability level at the corresponding position. For example, when there is a measurement index in an area with high reliability, the grid line display control unit 81 displays at least one of the first grid line and the second grid line with a broken line. When there is a measurement index in an area with low reliability, the grid line display control unit 81 displays at least one of the first grid line and the second grid line as a solid line. The grid line display control unit 81 may change the interval of at least one of the first grid line and the second grid line according to the reliability level at the corresponding position. For example, when there is a measurement index in an area with high reliability, the grid line display control unit 81 widens the interval between at least one of the first grid line and the second grid line. When there is a measurement index in an area with low reliability, the grid line display control unit 81 narrows the interval between at least one of the first grid line and the second grid line. The grid line display control unit 81 may blink at least one of the first grid line and the second grid line only when the reliability level at the corresponding position is low.

  In only the area around the measurement index, the color of the first grid line may be different from the color of the first grid line in the other area. The first grid line and the second grid line may not be displayed depending on the reliability status.

  The grid line display control unit 81 generates a graphic image signal of the grid line according to the display form of the grid line. The display form of the grid lines changes in real time according to the position of the measurement index in the observation image. The change in the display form of the grid lines is not limited to the above example. As long as the change in the reliability level can be easily notified to the user, the change in the display form of the grid lines may be any change. By changing the display form of the grid lines, the user can easily notice the change in the reliability level at the corresponding position.

  As described above, by displaying the first grid line and the second grid line, the user can indicate the correspondence between the position on the observation image and the position on the reliability map by using the first grid line and the second grid line. 2 grid lines. That is, the user can easily grasp the correspondence between the regions divided by the grid lines in the observation image and the reliability map. The user can grasp the corresponding position on the reliability map corresponding to the measurement index on the observation image by referring to the grid line.

  As described above, by displaying at least one of the first grid line and the second grid line in a display form corresponding to the reliability level at the corresponding position, the user can respond to the position of the measurement index. Easy to notice changes in reliability. Particularly, when the display form of the grid line is changed when the measurement index is in a region with low reliability, the user can immediately know that the reliability at the position of the measurement index is low. As a result, the user can easily perform measurement while avoiding a region with low reliability.

  As described above, since the first grid line is displayed in the same color as the color corresponding to the reliability level at the corresponding position, the user can easily grasp the reliability from the color of the first grid line. Can do. The user can easily grasp the reliability of the entire observation image by the reliability map, and can grasp the reliability of the position that the user wants to measure from only the first grid line. The user only has to check the reliability map when the color of the first grid line is the same as the color of the region with low reliability. For this reason, the complicated operation | work which a user always compares an observation image with a reliability map is unnecessary.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment and its modification. Additions, omissions, substitutions, and other modifications can be made without departing from the spirit of the present invention. Further, the present invention is not limited by the above description, and is limited only by the scope of the appended claims.

  According to each embodiment of the present invention, the endoscope apparatus, the measurement method, and the program can suppress a decrease in the visibility of the measurement position in the observation image.

DESCRIPTION OF SYMBOLS 1 Endoscope apparatus 2 Endoscope 3 Apparatus main body 4 Monitor 5 Case 6 Operation part 8 Endoscope unit 9 CCU
10 Control unit 12 Video signal processing circuit 13 ROM
14 RAM
15 Card interface 16 External device interface 17 Control interface 18a, 18b, 18c, 18d, 18e CPU
20 Insertion section 21 Tip section 22 Bending section 23 Flexible tube section 28 Imaging element 70 Reliability calculation section 71 Reliability map generation section 72 Measurement index display control section 73 Control section 74 Measurement point determination section 75 Measurement section 76 Corresponding position calculation Unit 77 Reliability map display control unit 78a, 78b, 78c, 78d Reliability information notification unit 79 Reference index display control unit 80 Outer edge display control unit 81 Grid line display control unit

Claims (25)

An imaging unit for imaging a subject and generating an imaging signal;
An observation image generation unit that generates an observation image based on the imaging signal;
A reliability calculation unit that calculates reliability information related to the reliability of the calculation result of the three-dimensional coordinates corresponding to each of the plurality of positions in the observation image;
A reliability map generation unit that generates a reliability map indicating the reliability information corresponding to each of the plurality of positions on the observation image;
A display unit for displaying the observation image and the reliability map;
A measurement index for the user to specify the position of the measurement point on the observation image is displayed on the observation image, and on the observation image based on the user instruction input via the input unit A measurement index display control unit for moving the measurement index;
A measurement point determination unit for determining the position of the measurement point based on the position of the measurement index;
A measurement unit that calculates three-dimensional coordinates of the position of the measurement point determined by the measurement point determination unit;
The reliability for displaying the reliability map so that the ratio of the display size of the reliability map to the display size of the observation image becomes a predetermined value at a position different from the position where the measurement index is displayed on the display unit. A map display control unit;
An endoscope apparatus having A corresponding position calculation unit for calculating a corresponding position on the reliability map corresponding to the position of the measurement index in the observation image;
A reliability information notification unit that notifies the user of the reliability information at the corresponding position calculated by the corresponding position calculation unit;
The endoscope apparatus according to claim 1, further comprising: The endoscope apparatus according to claim 2, wherein the reliability information notification unit includes a reference index display control unit that displays a reference index at the corresponding position calculated by the corresponding position calculation unit in the reliability map. . 4. The reliability information at the corresponding position calculated by the corresponding position calculation unit corresponds to any one of a plurality of reliability levels divided according to the degree of reliability. Endoscopic device. The reference indicator display form is any one of a plurality of display forms corresponding to the plurality of reliability levels,
The endoscope apparatus according to claim 4, wherein the reference index display control unit displays the reference index in a display form corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit. The reliability map generation unit is configured to display the reliability such that each of the corresponding positions calculated by the corresponding position calculation unit in the reliability map is displayed in a color corresponding to the reliability level at the corresponding position. Generate a map
The reference index display control unit displays the reference index in any one of a plurality of colors, and displays the display color of the reference index at the corresponding position calculated by the corresponding position calculation unit. The endoscope apparatus according to claim 4, wherein the color is different from the color corresponding to the reliability level.   The endoscope apparatus according to claim 2, wherein the reliability information notification unit includes the measurement index display control unit. 8. The reliability information at the corresponding position calculated by the corresponding position calculation unit corresponds to any one of a plurality of reliability levels classified according to the degree of reliability. Endoscopic device. The display form of the measurement index is any one of a plurality of display forms corresponding to the plurality of reliability levels.
The endoscope apparatus according to claim 8, wherein the measurement index display control unit displays the measurement index in a display form corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit. The reliability map generation unit is configured to display the reliability such that each of the corresponding positions calculated by the corresponding position calculation unit in the reliability map is displayed in a color corresponding to the reliability level at the corresponding position. Generate a map
The measurement index display control unit displays the measurement index in any one of a plurality of colors, and displays the display color of the measurement index at the corresponding position calculated by the corresponding position calculation unit. The endoscope apparatus according to claim 8, wherein the same color as that corresponding to the reliability level is used. The endoscope apparatus according to claim 2, wherein the reliability information notification unit includes an outer edge display control unit that controls a display form of an outer edge of the display area of at least one of the observation image and the reliability map. 12. The reliability information at the corresponding position calculated by the corresponding position calculation unit corresponds to any one of a plurality of reliability levels divided according to the degree of reliability. Endoscopic device. The display form of the display region outer edge of at least one of the observation image and the reliability map is any one of a plurality of display forms corresponding to the plurality of reliability levels,
The outer edge display control unit displays the outer edge of the display area of at least one of the observation image and the reliability map in a display form corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit. Item 15. The endoscope apparatus according to Item 12. The endoscope apparatus according to claim 13, wherein the outer edge display control unit makes the display forms of the display area outer edge of the observation image and the display area outer edge of the reliability map the same. The reliability map generation unit is configured to display the reliability such that each of the corresponding positions calculated by the corresponding position calculation unit in the reliability map is displayed in a color corresponding to the reliability level at the corresponding position. Generate a map
The outer edge display control unit displays the outer edge of the display area of at least one of the observation image and the reliability map in any one of a plurality of colors, and the at least one of the observation image and the reliability map The endoscope apparatus according to claim 12, wherein a display color of an outer edge of the display area is made the same as a color corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit. The endoscope apparatus according to claim 15, wherein the outer edge display control unit makes the display colors of the display area outer edge of the observation image and the display area outer edge of the reliability map the same. The reliability information notification unit includes the reliability map display control unit,
The endoscope apparatus according to claim 2, wherein the reliability map display control unit controls a display size of the reliability map. 18. The reliability information at the corresponding position calculated by the corresponding position calculation unit corresponds to any one of a plurality of reliability levels divided according to the degree of reliability. Endoscopic device. The ratio is any one of a plurality of values corresponding to the plurality of confidence levels;
The reliability map display control unit displays the reliability map so that the ratio becomes a value corresponding to the reliability level at the corresponding position calculated by the corresponding position calculation unit. Endoscope device. Grid line display for displaying a first grid line at a first position on the observation image and displaying a second grid line at a second position on the reliability map corresponding to the first position The endoscope apparatus according to claim 1, further comprising a control unit. A corresponding position calculating unit that calculates a corresponding position on the reliability map corresponding to the position of the measurement index in the observation image;
The reliability information at the corresponding position calculated by the corresponding position calculation unit corresponds to any one of a plurality of reliability levels classified according to the degree of reliability. Endoscopic device. A corresponding position calculation unit that calculates a position on the reliability map corresponding to the position of the measurement index in the observation image;
The display form of at least one of the first grid line and the second grid line is any one of a plurality of display forms corresponding to the plurality of reliability levels,
The grid line display control unit displays at least one of the first grid line and the second grid line in a display form corresponding to the reliability level at the corresponding position calculated by the corresponding position calculating unit. The endoscope apparatus according to claim 21. A corresponding position calculation unit that calculates a position on the reliability map corresponding to the position of the measurement index in the observation image;
The reliability map generation unit is configured to display the reliability such that each of the corresponding positions calculated by the corresponding position calculation unit in the reliability map is displayed in a color corresponding to the reliability level at the corresponding position. Generate a map
The grid line display control unit displays the first grid line in any one of a plurality of colors, and the display color of the first grid line is calculated by the corresponding position calculation unit. The endoscope apparatus according to claim 21, wherein the same color as that corresponding to the reliability level at a position is used. An imaging step of imaging a subject and generating an imaging signal;
An observation image generation step of generating an observation image based on the imaging signal;
A reliability calculation step of calculating reliability information related to the reliability of the calculation result of the three-dimensional coordinates corresponding to each of the plurality of positions in the observation image;
A reliability map generation step of generating a reliability map indicating the reliability information corresponding to each of the plurality of positions on the observation image;
A display step of displaying the observation image and the reliability map on a display unit;
A measurement index for the user to specify the position of the measurement point on the observation image is displayed on the observation image, and on the observation image based on the user instruction input via the input unit A measurement index display step for moving the measurement index;
A measurement point determination step for determining the position of the measurement point based on the position of the measurement index;
A measurement step of calculating three-dimensional coordinates of the position of the measurement point determined by the measurement point determination step;
In the display step, the ratio of the display size of the reliability map to the display size of the observation image is a predetermined value at a position different from the position where the measurement index is displayed on the display unit. A measurement method for displaying the reliability map. An observation image acquisition function for acquiring an observation image of a subject;
A reliability calculation function for calculating reliability information regarding the reliability of the calculation result of the three-dimensional coordinates corresponding to each of the plurality of positions in the observation image;
A reliability map generation function for generating a reliability map indicating the reliability information corresponding to each of the plurality of positions on the observation image;
A display function for displaying the observation image and the reliability map on a display unit;
A measurement index for the user to specify the position of the measurement point on the observation image is displayed on the observation image, and on the observation image based on the user instruction input via the input unit A measurement index display function for moving the measurement index;
A measurement point determination function for determining the position of the measurement point based on the position of the measurement index;
A measurement function for calculating the three-dimensional coordinates of the position of the measurement point determined by the measurement point determination function;
The reliability for displaying the reliability map so that the ratio of the display size of the reliability map to the display size of the observation image becomes a predetermined value at a position different from the position where the measurement index is displayed on the display unit. Map display control function,
A program to make a computer realize.

Images