JP6081209B2 - Endoscope apparatus and program - Google Patents

Description

  The present invention relates to an endoscope apparatus having a measurement function. The present invention also relates to a program for realizing a measurement function with a computer.

  In recent years, industrial endoscopes have been widely used for observation and inspection of internal scratches and corrosion of boilers, turbines, engines, chemical plants and the like. In recent industrial endoscopes, it is possible to attach a stereo optical adapter that captures the subject from different viewpoints to the tip of the endoscope, and measure various spatial characteristics of the subject based on the principle of triangulation (stereo) Measurement) (see, for example, Patent Document 1). In stereo measurement, for example, a pair of left and right subject images are acquired, and matching processing is performed in which a point on the other subject image corresponding to a point designated on one subject image is detected by image processing. Furthermore, based on the coordinates of the corresponding left and right points, the length, depth, area, etc. of the subject are calculated by the principle of triangulation.

  The result of the above matching process affects the accuracy of stereo measurement. In particular, in the case of erroneous matching of matching in which the positions of the corresponding left and right points are greatly different, the measurement results are completely different, and it is necessary to re-designate the points and re-read the image. For example, a technique for using a plurality of templates to use the most reliable result (see Patent Document 2) or a technique for evaluating reliability with a low resolution image as a technique for reducing erroneous matching. A method of determining the window size of a high-resolution image for point calculation (see Patent Document 3) is known. In addition, as a method for prompting the user to confirm whether or not a matching mismatch has occurred, a method of highlighting and displaying a region related to matching in a subject image is also known (see Patent Document 4).

JP 2004-33487 A JP 2001-109879 A JP 2009-92551 A JP 2011-170276 A

  In an inspection using an industrial endoscope, an image recorded at an observation site may be inspected by stereo measurement outside the observation site. However, in the case where a matching error occurs in the inspection image recorded at the observation site, it is inconvenient because it is necessary to re-acquire the inspection image at the observation site.

  What kind of matching method is used, not all of the images are mismatched, and there are images that are good at each method and images that are not good at each method. However, since there are a wide variety of subjects for industrial endoscopes, it has been difficult to provide matching methods suitable for various subjects.

  Although the techniques described in Patent Document 2 and Patent Document 3 may be able to reduce mismatching of matching, the best matching algorithm is selected for a wide variety of images such as industrial endoscopes. ,Have difficulty. Further, with the technique described in Patent Document 4, even if it is found that an incorrect matching is occurring, it takes time to re-read the image or to manually correct the corresponding points by the user. .

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide an endoscope apparatus and a program that can reduce erroneous correspondence of matching and reduce user trouble.

  The present invention has been made in order to solve the above-described problems. An imaging unit that captures a first subject image and a second subject image corresponding to the same subject and generates image data; and the image data A display unit that displays an image based on the image and a corresponding point candidate that is a point on the second subject image corresponding to a measurement point set on the first subject image using a plurality of matching methods A corresponding point candidate calculating unit to calculate, a reliability calculating unit for calculating a matching reliability based on the corresponding point candidates corresponding to each of the plurality of matching methods, and the reliability is less than a predetermined value In addition, when the measurement point and the corresponding point candidate corresponding to each of the plurality of matching methods are displayed on the image, and the reliability is a predetermined value or more, Multiple matches The corresponding point is determined based on the corresponding point candidate corresponding to each of the matching methods, and the corresponding point candidate corresponding to each of the plurality of matching methods is determined when the reliability is less than a predetermined value. A corresponding point determination unit that determines the specified corresponding point candidate as a corresponding point, and a measurement processing unit that measures the size of the subject based on the measured point and the determined corresponding point. It is an endoscope apparatus characterized by having.

  Further, in the endoscope apparatus according to the present invention, the reliability calculation unit is configured to determine the reliability based on a distance on the second subject image between the corresponding point candidates corresponding to each of the plurality of matching methods. Is calculated.

  Further, in the endoscope apparatus according to the present invention, the reliability calculation unit includes a position of the corresponding point candidate corresponding to each of the plurality of matching methods and a priority coefficient corresponding to each of the plurality of matching methods. The reliability is calculated based on the above.

  In the endoscope apparatus of the present invention, the corresponding point determination unit, when the reliability is equal to or higher than a predetermined value, the position of the corresponding point candidate corresponding to each of the plurality of matching methods, The corresponding point is determined based on a priority coefficient corresponding to each of a plurality of matching methods.

  Further, in the endoscope apparatus according to the present invention, the reliability calculation unit may determine each of the plurality of matching methods based on a result of calculating the corresponding point candidates corresponding to each of the plurality of matching methods in the past. The priority coefficient corresponding to is calculated.

  In the endoscope apparatus according to the present invention, the priority coefficient is provided for each condition when the corresponding point candidates are calculated.

  Moreover, in the endoscope apparatus according to the present invention, the display processing unit displays, in the same display form, overlapping corresponding candidate candidates among the corresponding point candidates corresponding to each of the plurality of matching methods. It is characterized by performing a process of displaying on the screen.

  In the endoscope apparatus of the present invention, the display processing unit performs a process of displaying the number of overlapping corresponding point candidates in the vicinity of the corresponding point candidate.

  In the endoscope apparatus of the present invention, the display processing unit determines the corresponding point candidates corresponding to each of the plurality of matching methods according to a priority coefficient corresponding to each of the plurality of matching methods. A process of displaying the image in a display form is performed.

  In the endoscope apparatus according to the present invention, the corresponding point candidate calculation unit further calculates the corresponding point candidate using a matching method different from the plurality of matching methods, and the display processing unit When the display of the corresponding point candidates calculated using a matching method different from the plurality of matching methods is instructed, the measurement point and the plurality of matching methods The corresponding point candidate corresponding to each of the image and the corresponding point candidate calculated using a matching method different from the plurality of matching methods are displayed on the image, and the corresponding point determining unit When the display of the corresponding point candidates calculated using a matching method different from the plurality of matching methods is instructed when the reliability is less than a predetermined value, The candidate for the designated corresponding point among the candidate for the corresponding point corresponding to each of the plurality of matching methods and the candidate for the corresponding point calculated using a matching method different from the plurality of matching methods. It is characterized by determining the corresponding point.

  In the endoscope apparatus of the present invention, the display processing unit may display the plurality of matching points when displaying the corresponding point candidates calculated using a matching method different from the plurality of matching methods. Among the corresponding point candidates calculated using a matching method different from the method, matching that overlaps with the corresponding point candidates calculated using the plurality of matching methods or is different from the plurality of matching methods Only the corresponding point candidates that overlap with the other corresponding point candidates calculated using a technique are displayed.

  In the endoscope apparatus of the present invention, the display processing unit is already displayed when displaying the corresponding point candidates calculated using a matching method different from the plurality of matching methods on the image. By not displaying any of the corresponding point candidates, the number of displayed corresponding point candidates is kept constant.

  Further, the present invention provides measurement points set on the first subject image of an image based on image data generated by capturing the first subject image and the second subject image corresponding to the same subject. A step of calculating a corresponding point candidate that is a corresponding point on the second subject image using a plurality of matching methods, and matching based on the corresponding point candidates corresponding to each of the plurality of matching methods And calculating the reliability of the measurement point and, when the reliability is less than a predetermined value, displaying the measurement point and the corresponding point candidate corresponding to each of the plurality of matching methods on the image. And when the reliability is equal to or higher than a predetermined value, the corresponding point is determined based on the corresponding point candidate corresponding to each of the plurality of matching methods, and the reliability is a predetermined value If it is full, a step of determining a corresponding point candidate designated among the corresponding point candidates corresponding to each of the plurality of matching methods as a corresponding point, the measurement point, and the determined correspondence A program for causing a computer to execute the step of measuring the size of the subject based on a point.

  According to the present invention, by using a plurality of matching methods, it becomes easier to obtain a matching result that does not cause mismatching of matching, and the need to re-image is reduced. Further, when the matching reliability is equal to or higher than a predetermined value, the corresponding point is automatically determined, so that the user's labor is not increased. If the matching reliability is less than the predetermined value, the corresponding corresponding point candidate among the corresponding corresponding point candidates corresponding to each of the plurality of matching methods is determined as the corresponding point. Compared to point correction, user effort is reduced. Accordingly, it is possible to reduce erroneous matching and reduce the user's trouble.

1 is a perspective view showing an overall configuration of an endoscope apparatus according to an embodiment of the present invention. It is a block diagram which shows the internal structure of the endoscope apparatus by one Embodiment of this invention. It is a block diagram which shows the function structure of CPU which the endoscope apparatus by one Embodiment of this invention has. It is a flowchart which shows the procedure of operation | movement of the endoscope apparatus by one Embodiment of this invention. It is a flowchart which shows the procedure of operation | movement of the endoscope apparatus by one Embodiment of this invention. It is a reference figure showing an image which an endoscope apparatus by one embodiment of the present invention acquired. It is a reference figure showing an image which an endoscope apparatus by one embodiment of the present invention acquired. It is a reference figure showing an image which an endoscope apparatus by one embodiment of the present invention acquired. It is a reference figure showing an image which an endoscope apparatus by one embodiment of the present invention acquired. It is a reference figure showing an image which an endoscope apparatus by one embodiment of the present invention acquired. It is a reference figure showing an image which an endoscope apparatus by one embodiment of the present invention acquired. It is a reference figure showing an image which an endoscope apparatus by one embodiment of the present invention acquired. It is a reference figure for demonstrating how to obtain | require the three-dimensional coordinate of the measurement point by stereo measurement.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows an overall configuration of an endoscope apparatus according to an embodiment of the present invention. As shown in FIG. 1, the endoscope apparatus 1 includes an endoscope 2 and an apparatus main body 3 connected to the endoscope 2. The endoscope 2 includes an elongated insertion unit 20 and an operation unit 6 for performing operations necessary for performing various operation controls of the entire apparatus. The apparatus main body 3 includes a monitor 4 that displays an image of a subject imaged by the endoscope 2, operation control content (for example, a processing menu), and the like, and a housing 5 that has a control unit 10 (see FIG. 2) inside. I have.

  The insertion portion 20 is configured by connecting a hard distal end portion 21, a bending portion 22 that can be bent vertically and horizontally, and a flexible tube portion 23 having flexibility in order from the distal end side. Various optical adapters such as a stereo optical adapter having two observation fields and a normal observation optical adapter having one observation field are detachably attached to the distal end portion 21.

  As shown in FIG. 2, an endoscope unit 8, a CCU 9 (camera control unit), and a control unit 10 are provided in the housing 5. A proximal end portion of the insertion portion 20 is connected to the endoscope unit 8. The endoscope unit 8 includes a light source device (not shown) that supplies illumination light necessary for observation, and a bending device (not shown) that bends the bending portion 22 that constitutes the insertion portion 20. . The CCU 9 includes a drive device (not shown) that drives the image sensor 28.

  An image sensor 28 is built in the distal end portion 21. The image sensor 28 photoelectrically converts the subject image formed through the optical adapter to generate an image signal. The imaging signal is converted into a video signal (image data) such as an NTSC signal in the CCU 9 and supplied to the control unit 10.

  In the control unit 10, a video signal processing circuit 12, to which a video signal is input, a ROM 13, a RAM 14, a card I / F 15 (card interface), a USB I / F 16 (USB interface), an RS-232C I / F 17 (RS-) 232C interface) and CPU 18 are provided.

  The RS-232C I / F 17 is connected to the CCU 9 and the endoscope unit 8, and to the operation unit 6 that performs control and operation instructions for the CCU 9 and the endoscope unit 8. When the user operates the operation unit 6, communication necessary for controlling the operation of the CCU 9 and the endoscope unit 8 is performed based on the operation content.

  The USB I / F 16 is an interface for electrically connecting the control unit 10 and the personal computer 31. By connecting the control unit 10 and the personal computer 31 via the USB I / F 16, control based on various instructions such as an endoscope image display instruction and image processing at the time of measurement is performed on the personal computer 31 side. In addition, it is possible to input and output control information and data necessary for various processes between the control unit 10 and the personal computer 31.

  Further, the memory card 32 can be freely attached to and detached from the card I / F 15. By attaching the memory card 32 to the card I / F 15, in accordance with control by the CPU 18, control processing information and image information stored in the memory card 32 are taken into the control unit 10, or control processing information or It is possible to record data such as image information in the memory card 32.

  The video signal processing circuit 12 displays a composite image obtained by synthesizing an endoscopic image based on the video signal supplied from the CCU 9 and a graphic operation menu, so that the graphic image signal based on the operation menu generated by the CPU 18 is displayed. And processing for synthesizing the video signals from the CCU 9 and processing necessary for display on the screen of the monitor 4, and the display signal is supplied to the monitor 4. Further, the video signal processing circuit 12 can simply perform processing for displaying an endoscopic image or an image such as an operation menu alone. Therefore, an endoscope image, an operation menu image, a composite image of the endoscope image and the operation menu image, and the like are displayed on the screen of the monitor 4.

  The CPU 18 executes the program stored in the ROM 13 to control various circuit units and the like so as to perform processing according to the purpose, thereby controlling the operation of the entire endoscope apparatus 1. The RAM 14 is used by the CPU 18 as a work area for temporarily storing data.

  The program executed by the CPU 18 may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by a computer other than the endoscope apparatus 1 and executed. For example, the personal computer 31 reads and executes a program, and according to the program, transmits control information for controlling the endoscope apparatus 1 to the endoscope apparatus 1 to control the endoscope apparatus 1, and the endoscope A video signal may be acquired from the apparatus 1 and measurement may be performed using the acquired video signal.

  Here, the “computer” includes a homepage providing environment (or display environment) if the WWW system is used. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, a DVD-ROM, and a flash memory, and a storage device such as a hard disk built in the computer. Say. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program described above may be transmitted from a computer storing the program in a storage device or the like to another computer via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting a program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the above-described program may be for realizing a part of the above-described function. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer, what is called a difference file (difference program) may be sufficient.

  In the present embodiment, an image is acquired in a state in which a stereo optical adapter having two observation fields on the left and right is attached to the distal end portion 21. Light from the subject enters the stereo optical adapter, and the left and right subject images (a first subject image and a second subject image) corresponding to the same subject are formed by the stereo optical adapter. Two subject images enter the image sensor 28 and are captured by the image sensor 28. The acquired image is an image including two subject images. In the present embodiment, among the acquired images, the images of the areas corresponding to the two subject images are defined as the left image and the right image.

  FIG. 3 shows a functional configuration of a portion of the CPU 18 that is the center of the description of the present embodiment. The control unit 180, the measurement point setting unit 181, the corresponding point candidate calculation unit 182, the reliability calculation unit 183, the display processing unit 184, the corresponding point determination unit 185, and the measurement processing unit 186 constitute the function of the CPU 18.

  The control unit 180 controls processing performed by each unit. The measurement point setting unit 181 operates the operation unit 6 by the user and sets a measurement point at a measurement position designated on the left image based on the image data. Corresponding point candidate calculation unit 182 uses a plurality of matching techniques to generate corresponding point candidates (hereinafter referred to as corresponding point candidates) corresponding to the measurement points set on the left image in the right image based on the image data. calculate. The reliability calculation unit 183 calculates matching reliability based on corresponding point candidates corresponding to each of the plurality of matching methods.

  In order to allow the user to confirm the position of the corresponding point candidate, the display processing unit 184 displays a graphic (mark) of the measurement point and the corresponding point candidate corresponding to each of the plurality of matching methods for the image based on the image data. Processing for superimposing is performed. The corresponding point determination unit 185 determines a corresponding point used for the measurement process based on the calculated corresponding point candidate. The measurement processing unit 186 calculates three-dimensional coordinates on the space based on the two-dimensional coordinates on the image of the measurement points and the determined corresponding points, and based on the three-dimensional coordinates corresponding to the plurality of measurement points, Measure the size of the subject.

Next, the principle of stereo measurement in this embodiment will be described. By using a stereo optical adapter and determining the three-dimensional spatial coordinates of the subject using the principle of triangulation based on the coordinates of the left and right optical ranging points when the subject image is captured by the left and right optical systems, Various measurements (stereo measurement) are possible. Hereinafter, a method of obtaining the three-dimensional coordinates of the measurement point by stereo measurement will be described with reference to FIG. With respect to the images captured by the left and right optical systems, the right direction is set to the x axis and the lower side by the triangulation method with the midpoint of the line segment connecting the left and right optical centers 63 and 64 as the origin O. The three-dimensional coordinates (X, Y, Z) of the measurement point 60 when the direction is defined as the y-axis and the direction away from the optical system parallel to the optical axis as the z-axis are defined as the following equations (1) to ( 3) Calculated by the equation. However, the measurement points 61 on the image plane of the right and left distortion correction has been performed, the two-dimensional coordinates of the corresponding point 62, the intersection O _L of the right and left optical axis and the image plane of the respective optical _systems, the O _R as the origin (X _L , Y _L ), (X _R , Y _R ), D is the distance between the left and right optical centers 63, 64, F is the focal length, and t = D / (X _R −X _L ). To do.
X = t × X _R + D / 2 (1)
Y = −t × Y _R (2)
Z = t × F (3)

  When the coordinates of the measurement point 61 and the corresponding point 62 on the image plane on the original image are determined as described above, the three-dimensional coordinates of the measurement point 60 are obtained using the parameters D and F. By obtaining three-dimensional coordinates of several points, various measurements such as the distance between two points, the distance between two points and the distance between one point, area, depth, and surface shape can be performed. It is also possible to obtain the distance (object distance) from the left optical center 63 or the right optical center 64 to the subject. The object distance is a distance from the distal end portion 21 to the subject, for example, a distance from the image sensor 28 or the observation optical system to the subject. In order to perform the above stereo measurement, optical data indicating the characteristics of the optical system including the tip 21 and the stereo optical adapter is required. The details of the matching process and the optical data are described in, for example, Japanese Patent Application Laid-Open No. 2004-49638, and thus the description thereof is omitted.

  Next, details of the measurement process in the present embodiment will be described. FIG. 4 shows an operation procedure of the endoscope apparatus 1 at the time of measurement. After the image (still image) acquired by the endoscope 2 is displayed on the monitor 4, when the user designates the measurement position on the left image based on the image data via the operation unit 6, the measurement point setting unit 181. Calculates the two-dimensional coordinates of the designated measurement position, and sets a measurement point at that position (step S1). Subsequently, matching processing (step S2) is performed, and corresponding points on the right image based on the image data are determined. Details of the matching process will be described later.

  After the matching process is completed, the measurement processing unit 186 calculates a three-dimensional coordinate in the space based on the two-dimensional coordinates on the image of the measurement point and the corresponding point (step S3). Subsequently, the control unit 180 determines whether or not the measurement point setting is completed (step S4). If the user continues to specify the measurement position, the next measurement point is set in step S1. When the measurement position designation by the user is completed, the measurement processing unit 186 performs three-dimensional measurement for calculating the distance between two points based on the three-dimensional coordinates calculated in step S3 (step S5). . Subsequently, the display processing unit 184 performs processing for displaying, on the monitor 4, an image in which the measurement result calculated by the measurement processing unit 186 is superimposed with graphics (characters) on the image based on the image data (step S <b> 6). . As a result, the measurement result is displayed on the monitor 4. When the measurement result is displayed, the measurement process ends.

  FIG. 5 shows the procedure of the matching process (step S2). First, an outline of the matching process will be described. In the matching process, a plurality of corresponding point candidates are calculated by a plurality of matching methods. When the plurality of corresponding point candidates have high reliability, corresponding points are automatically determined according to a predetermined criterion unrelated to the user's instruction. In addition, when the reliability of the plurality of corresponding point candidates is low, the user selects a corresponding point candidate suitable for the corresponding point from the plurality of corresponding point candidates, and the selected corresponding point candidate is determined as the corresponding point. The

  Details of the matching process will be described below. The control unit 180 sets the value of the variable i corresponding to the matching method used for calculating the corresponding points to 0 (step S200). Subsequently, the corresponding point candidate calculation unit 182 calculates the two-dimensional coordinates of the corresponding point candidate on the right image corresponding to the measurement point set on the left image, using a matching method corresponding to the value of the variable i. (Step S205).

  In this embodiment, three types of matching methods are used as an example. For example, a matching method A that performs template matching, a matching method B that performs matching using feature points such as shading edge portions, and a phase-only correlation method (POC: Phase Only Correlation) that calculates the correlation of phase components obtained by Fourier transforming an image. The matching method C used is used. In addition to the above-described matching methods, methods using different correlation functions used in template matching (for example, NCC (Normalized Cross-Correlation), SSD (Sum of Squared Difference), SAD (Sum of Absolute Difference), etc.) may be used. Good. In addition, methods with different template sizes and shapes may be used. Furthermore, a method with different image resolution for template matching, a method with different image processing (binarization, noise removal, various filtering), or a method using color information as well as image luminance information may be used. May be used. Although the matching method used for calculating the corresponding point candidate is set in advance, the user may be able to change the matching method.

  In the present embodiment, priority coefficients corresponding to the respective matching methods are provided. For example, the priority coefficient of the matching technique A is 1.0, the priority coefficient of the matching technique B is 0.8, and the priority coefficient of the matching technique C is 0.6. The priority coefficient indicates the magnitude (priority) of the influence of each corresponding point candidate on the process of calculating the matching reliability and the process of determining the corresponding point from a plurality of corresponding point candidates. . The higher the priority coefficient, the higher the priority. Although the priority coefficient of each matching method is set in advance, the value may be changed by the user.

  After the corresponding point candidate is calculated, the control unit 180 determines whether the corresponding point candidate is calculated using all the matching methods by determining whether the value of the variable i is equal to the predetermined value. (Step S210). When three types of matching methods are used, the predetermined value is 2. If there is a matching method that does not calculate the corresponding point candidate, the control unit 180 increases the value of the variable i by 1 (step S215). Subsequently, the corresponding point candidate is calculated again in step S205.

  When the corresponding point candidates are calculated using all the matching methods, the reliability calculating unit 183 calculates the matching reliability based on the two-dimensional coordinates of the corresponding point candidates corresponding to each of the plurality of matching methods (step S215). This reliability does not indicate the reliability of matching for each matching method, but indicates the reliability of the entire matching performed using a plurality of matching methods.

In step S215, the matching reliability is calculated as follows. For example, the matching reliability is calculated based on the distance on the image between each corresponding point candidate. The closer the distance between corresponding point candidates on the image, the higher the reliability of matching. More specifically, the matching reliability is a corresponding point that falls within a predetermined distance R (for example, 3 pixels) centered on a corresponding point candidate whose parallax amount is a median value among the calculated corresponding point candidates. Calculated as the sum of the candidate priority coefficients. The two-dimensional coordinates of the measurement point _{(X L, Y L),} (X R, Y R) of the two-dimensional coordinates of the corresponding point candidate When parallax amount is _X R -X _L. Although the value of the predetermined distance R is set in advance, the user may be able to change the value.

  By calculating the matching reliability as the sum of the priority coefficients, it is possible to make a determination according to the priority of the corresponding point candidate when making a determination by setting a threshold for the matching reliability, and more accurately Judgment can be made. For example, the sum of priority coefficients of corresponding point candidates calculated using the matching methods A, B, and C is 2.4, and the sum of priority coefficients of corresponding point candidates calculated using the matching methods A and B is 1.8. Yes, the sum of priority coefficients of corresponding point candidates calculated using the matching methods A and C is 1.6, and the sum of priority coefficients of corresponding point candidates calculated using the matching methods B and C is 1.4. When the threshold value is 1.5, the sum of the priority coefficients when the matching methods A, B, and C are used, when the matching methods A and B are used, and when the matching methods A and C are used is the threshold value. However, the sum of the priority coefficients when the matching methods B and C having a low priority coefficient are used does not exceed the threshold value.

  Of the calculated corresponding point candidates, the sum of the priority coefficients of corresponding point candidates within a predetermined distance R centered on the corresponding point candidate having the highest priority coefficient may be used as the matching reliability. As described above, by calculating the reliability of matching using only corresponding point candidates that are close to each other, it is possible to reduce the influence of corresponding point candidates that are likely to be not true corresponding points. Can be eliminated.

  The priority coefficient may not be used for calculating the matching reliability. For example, the ratio n / the number n of corresponding point candidates that fall within a predetermined distance R centered on the corresponding point candidate having the median parallax amount and the number N of matching methods used for calculating the corresponding point candidate N may be used as the matching reliability.

  When the matching reliability is equal to or higher than the predetermined value, the corresponding point determination unit 185 determines a corresponding point based on a predetermined criterion based on the plurality of corresponding point candidates (step S225). In step S225, the corresponding points are determined as follows. For example, the two-dimensional coordinates of all corresponding point candidates within the predetermined distance R centered on the corresponding point candidate whose parallax amount is the median obtained when the matching reliability is calculated in step S215 match. When doing so, the point of the two-dimensional coordinate is determined as the corresponding point.

  Further, when the two-dimensional coordinates of all corresponding point candidates within the predetermined distance R centering on the corresponding point candidate having the median parallax amount do not completely match, 2 of the corresponding point candidates A two-dimensional coordinate point obtained by averaging the dimensional coordinates is determined as a corresponding point. By averaging the two-dimensional coordinates of the corresponding point candidates, the position of the corresponding point can be obtained with an accuracy of 1 pixel or less, and a result with higher measurement accuracy can be obtained.

  The two-dimensional coordinates of all corresponding point candidates that fall within a predetermined distance R centering on the corresponding point candidate having the median parallax amount are weighted by a priority coefficient, and the weighted two-dimensional coordinates are obtained. The averaged two-dimensional coordinate point may be used as the corresponding point. Alternatively, the weighting may be performed using a value indicating the reliability of the result of matching using each matching method (for example, a correlation coefficient in the case of template matching) instead of the priority coefficient.

  Corresponding point candidates obtained using a matching method having the highest priority coefficient may be used as corresponding points. Or it is good also considering the corresponding point candidate with the highest value (for example, a correlation coefficient in the case of template matching) which shows the reliability of the result of having matched using each matching method as a corresponding point.

  When the matching reliability is less than the predetermined value, the display processing unit 184 generates a graphic image signal for displaying the measurement point and the plurality of corresponding point candidates and outputs the graphic image signal to the video signal processing circuit 12. The video signal processing circuit 12 outputs a display signal obtained by synthesizing the graphic image signal and the video signal from the CCU 9 to the monitor 4. The monitor 4 displays an image on which the measurement point and the plurality of corresponding point candidates are superimposed by displaying an image based on the display signal (step S235).

  FIG. 6 shows an example of an image in which measurement points and a plurality of corresponding point candidates are superimposed. In the image 600 displayed on the screen of the monitor 4, the measurement point PL designated by the user is displayed in the left image 600L, and the corresponding point candidate PR1 corresponding to the matching method A and the matching method B are displayed in the right image 600R. Corresponding corresponding point candidate PR2 and corresponding point candidate PR3 corresponding to matching method C are displayed. Each corresponding point candidate is displayed in a different display form for each matching method used to calculate the corresponding point candidate. In the example shown in FIG. 6, each corresponding point candidate is displayed in a different shape for each matching method. Each corresponding point candidate may be displayed in a different color for each matching method used to calculate the corresponding point candidate.

  In addition, each corresponding point candidate may be displayed in a different display form for each priority coefficient of the matching method used for calculating the corresponding point candidate. For example, the corresponding point candidate may be displayed with a size corresponding to the priority coefficient so that the corresponding point candidate is displayed larger as the priority coefficient is larger. Alternatively, only corresponding point candidates having a large priority coefficient may be displayed in different colors, or only corresponding point candidates having a large priority coefficient may be displayed in a highlighted manner.

  When the corresponding point candidates are calculated using template matching, the corresponding point candidates may be displayed in a display form corresponding to the correlation coefficient. Or you may display each corresponding point candidate with the display form according to the deviation | shift amount from the epipolar line of each corresponding point candidate.

  You may display each corresponding point candidate with the display form according to the characteristic of the matching method. For example, in template matching, the matching reliability for a subject having a step in a direction parallel to the optical axis of the optical system is relatively low, and the matching reliability for a subject having a pattern with low periodicity is relatively high. There is. In addition, in matching with feature points such as light and shaded edge portions, the matching reliability for a subject having a step in a direction parallel to the optical axis of the optical system is relatively high, and there is no pattern with clear features. There is a characteristic that the reliability of matching with a subject is relatively low. For example, in response to a user instruction, only the corresponding point candidates calculated using template matching are highlighted and displayed, or only the corresponding point candidates calculated using feature point matching are highlighted and displayed. Also good.

  After a plurality of corresponding point candidates are displayed, the user selects a corresponding point candidate that is considered to be correct via the operation unit 6. When any corresponding point candidate is selected, the corresponding point determination unit 185 determines the selected corresponding point candidate as a corresponding point (step S240). By displaying the corresponding point candidates in various display forms as described above, the user can confirm the likelihood of the corresponding points using various indices, and can select the corresponding point candidates more accurately.

  After the corresponding points are determined in steps S225 and S240, the display processing unit 184 generates a graphic image signal for displaying the determined corresponding points and outputs the graphic image signal to the video signal processing circuit 12. The video signal processing circuit 12 outputs a display signal obtained by synthesizing the graphic image signal and the video signal from the CCU 9 to the monitor 4. The monitor 4 displays an image on which the measurement point and the corresponding point are superimposed by displaying an image based on the display signal (step S230).

  FIG. 7 shows an example of an image in which measurement points and corresponding points are superimposed. In the image 600 displayed on the screen of the monitor 4, the measurement point PL designated by the user is displayed in the left image 600L, and the determined corresponding point PR is displayed in the right image 600R. When the corresponding point is displayed, the matching process ends.

  In the present embodiment, by using a plurality of matching methods, it becomes easy to obtain a matching result that does not cause a matching mismatch. Even if an image that is likely to cause a matching error depending on the matching method is acquired, if the matching error does not occur using a matching method that is suitable for the image, the image may not be read again. Also, when the matching reliability is high, the corresponding points are automatically determined, so that the labor of the user does not increase. In addition, when matching reliability is low, the corresponding point candidate specified by the user is determined as the corresponding point among the corresponding point candidates corresponding to each of the plurality of matching methods. This reduces the time and effort of the user. Accordingly, it is possible to reduce erroneous matching and reduce the user's trouble.

  Next, a modification of this embodiment will be described.

(First modification)
The first modification relates to a method for displaying a plurality of corresponding point candidates in step S235. In the first modified example, the processing after three corresponding point candidates are calculated using three types of matching methods and the three corresponding point candidates are displayed in step S235 will be described. FIG. 8 shows an example of the image displayed in step S235. A measurement point PL and three corresponding point candidates PR1, PR2, and PR3 are displayed. None of the corresponding point candidates is at a position that accurately corresponds to the position of the measurement point PL. The user can increase corresponding point candidates by inputting instructions via the operation unit 6.

  When an instruction for calculating a new corresponding point candidate is input, the corresponding point candidate calculating unit 182 calculates a new corresponding point candidate using a matching method different from the matching method that has already been used. At this time, a plurality of corresponding point candidates may be newly calculated using a plurality of matching methods. The display processing unit 184 generates a graphic image signal for displaying the newly calculated corresponding point candidate and outputs it to the video signal processing circuit 12. Thereafter, an image is displayed on the screen of the monitor 4 in the same manner as described above.

  FIG. 9 shows an example of an image including newly calculated corresponding point candidates. Compared with the image shown in FIG. 8, corresponding point candidates PR4, PR5, and PR6 are added. It can be seen that the newly calculated corresponding point candidate PR4 is at a position corresponding to the measurement point PL. In this case, when the user selects the corresponding point candidate PR4, the corresponding point determination unit 185 determines the selected corresponding point candidate PR4 as a corresponding point.

  A new corresponding point candidate may be calculated as follows. In a matching method that performs template matching, a point with the highest correlation coefficient is a corresponding point candidate. When the corresponding point candidate calculation unit 182 calculates a corresponding point candidate by template matching in step S205, the corresponding point candidate having the highest correlation coefficient calculated at that time is set as a new corresponding point candidate.

  Increasing the number of corresponding point candidates as described above increases the possibility that a corresponding point candidate at a position that accurately corresponds to the position of the measurement point will be calculated. Users such as image re-handling or manual correction of corresponding points Is less likely to occur. After displaying a plurality of corresponding point candidates in step S235, a corresponding point candidate may be newly calculated when the user instructs the calculation of a new corresponding point candidate. In step S205, the corresponding point candidate is displayed in step S235. Corresponding point candidates other than the corresponding point candidates are calculated in advance, and after a plurality of corresponding point candidates are displayed in step S235, a new corresponding point candidate is displayed when the user is instructed to display a new corresponding point candidate. It may be displayed. Further, it may be possible to repeatedly calculate and display a new corresponding point candidate based on a user instruction.

  As a result of increasing the number of corresponding point candidates as described above, if the number of corresponding point candidates increases too much, it becomes troublesome for the user to select corresponding point candidates. For this reason, it may be as follows.

  When a plurality of corresponding point candidates overlap (when the two-dimensional coordinates of the plurality of corresponding point candidates match or the distance on the image is equal to or less than a predetermined distance), the user indicates that the corresponding point candidates have high reliability. Therefore, overlapping corresponding candidate points may be highlighted and displayed. For example, the corresponding point candidate calculation unit 182 calculates the distance between all the corresponding point candidates including the plurality of corresponding point candidates displayed in step S235 and the new corresponding point candidate, and whether or not there are overlapping corresponding point candidates. Determine whether. When a plurality of corresponding point candidates overlap, the display processing unit 184 generates a graphic image signal for emphasizing and displaying the overlapping corresponding point candidates, and outputs the graphic image signal to the video signal processing circuit 12. Thereafter, an image is displayed on the screen of the monitor 4 in the same manner as described above.

  FIG. 10 shows an example of an image including newly calculated corresponding point candidates. Compared with the image shown in FIG. 8, corresponding point candidates PR4, PR5, PR6 are added, and the corresponding point candidates PR4, PR5 are overlapped and displayed larger than the other corresponding point candidates. Other methods may be used to highlight and display a plurality of overlapping corresponding point candidates, and methods such as displaying in different colors or blinking may be used.

  As described above, since the overlapping corresponding point candidates are emphasized, the user can easily identify the corresponding corresponding point candidates with high reliability. Therefore, even if the number of corresponding point candidates increases, the user can easily select corresponding point candidates.

  Among the corresponding point candidates newly calculated as described above, corresponding point candidates (two-dimensional coordinates are overlapped with any of the plurality of corresponding point candidates displayed in step S235 and other newly calculated corresponding point candidates. Only matching corresponding point candidates or corresponding point candidates whose distance on the image is equal to or smaller than a predetermined distance) may be displayed, and other corresponding point candidates may not be displayed. That is, the plurality of corresponding point candidates displayed in step S235 continue to be displayed as they are, but the newly calculated corresponding point candidates are displayed only when they overlap with any of the corresponding point candidates.

  For example, the corresponding point candidate calculation unit 182 calculates the distance between the newly calculated corresponding point candidate and the plurality of corresponding point candidates displayed in step S235 and other newly calculated corresponding point candidates. Then, it is determined whether the newly calculated corresponding point candidate overlaps with any corresponding point candidate. When the newly calculated corresponding point candidate overlaps with any one of the corresponding point candidates, the display processing unit 184 generates a graphic image signal for displaying the corresponding corresponding point candidate and outputs it to the video signal processing circuit 12. To do. Thereafter, an image is displayed on the screen of the monitor 4 in the same manner as described above.

  As described above, the newly calculated corresponding point candidate is displayed only when it overlaps with one of the corresponding point candidates, so that the user can easily select the corresponding point candidate even if the number of corresponding point candidates increases. Become. When repeatedly calculating and displaying a new corresponding point candidate, when displaying the newly calculated corresponding point candidate, the newly calculated corresponding point candidate, the corresponding corresponding point candidate being displayed, and the newly calculated corresponding point candidate are newly calculated. What is necessary is just to calculate the distance between other corresponding point candidates, and to determine whether the newly calculated corresponding point candidate overlaps with any corresponding point candidate.

  An upper limit may be set for the number of corresponding point candidates to be displayed, and a certain number of corresponding point candidates may be displayed. For example, the corresponding point candidate calculation unit 182 determines whether the total number of the corresponding point candidates displayed in step S235 and the newly calculated corresponding point candidate is equal to or greater than a predetermined number. When the total is less than the predetermined number, all corresponding point candidates including the newly calculated corresponding point candidates are displayed. When the total is equal to or greater than the predetermined number, the display processing unit 184 displays a number of corresponding point candidates equal to the difference between the total and the predetermined number among the plurality of corresponding point candidates displayed in step S235. Is displayed, a graphic image signal for displaying the newly calculated corresponding point candidate is generated and output to the video signal processing circuit 12. Thereafter, an image is displayed on the screen of the monitor 4 in the same manner as described above. The user may be allowed to set an upper limit on the number of corresponding point candidates to be displayed.

  As described above, by keeping the number of displayed corresponding point candidates constant, even if the number of corresponding point candidates increases, it is possible to prevent a decrease in image visibility, and the user selects a corresponding point candidate. It becomes easy to do. When calculating and displaying new corresponding point candidates repeatedly, when displaying newly calculated corresponding point candidates, the corresponding point candidate being displayed is displayed so that the number of displayed corresponding point candidates is constant. (For example, the oldest corresponding point candidate, the corresponding point candidate having the smallest correlation coefficient, the candidate having a large deviation from the epipolar line, etc.) may be hidden.

(Second modification)
The second modification relates to a method for displaying a plurality of corresponding point candidates in step S235. In the second modification, processing after calculating five corresponding point candidates using five types of matching methods and displaying the five corresponding point candidates in step S235 will be described. FIG. 11 shows an example of the image displayed in step S235. A measurement point PL and five corresponding point candidates PR1, PR2, PR3, PR4, and PR5 are displayed. The three corresponding point candidates PR1, PR2, and PR4 are at the same position. In FIG. 11, since the three corresponding point candidates overlap, the visibility of the image decreases. For this reason, when a plurality of corresponding point candidates overlap (when the two-dimensional coordinates of the plurality of corresponding point candidates match or the distance on the image is a predetermined distance or less), the corresponding point candidates are displayed in the same display form. May be displayed.

  For example, the corresponding point candidate calculation unit 182 calculates a distance on the image between a plurality of corresponding point candidates, and determines whether or not the corresponding point candidates overlap. When there are a plurality of overlapping corresponding point candidates, the display processing unit 184 generates a graphic image signal for displaying the overlapping corresponding point candidates with the same graphic (mark), and outputs the graphic image signal to the video signal processing circuit 12. Thereafter, an image is displayed on the screen of the monitor 4 in the same manner as described above.

  FIG. 12 shows an example of an image including overlapping corresponding point candidates. In FIG. 12, a mark M1 indicating three overlapping corresponding point candidates, a mark M2 indicating the corresponding point candidate PR3 in FIG. 11, and a mark M3 indicating the corresponding point candidate PR5 in FIG. 11 are displayed. Three overlapping corresponding point candidates are displayed as the same mark M1, and the mark M1 is displayed as a thicker line than the lines constituting the marks M2 and M3 and is emphasized. In addition, in the vicinity of each mark, the number of corresponding point candidates corresponding to the mark is displayed. For example, “3” is displayed near the mark M1 indicating three corresponding point candidates, and “1” is displayed near the marks M2 and M3 indicating one corresponding point candidate.

  As described above, since overlapping corresponding point candidates are displayed with the same mark, it is possible to reduce degradation of image visibility. Further, since the number of corresponding point candidates is displayed in the vicinity of the mark, it is possible to inform the user in an easy-to-understand manner whether or not the corresponding point candidates are overlapping, that is, which corresponding point candidate has high reliability.

(Third Modification)
The third modified example relates to a method for calculating the priority coefficient of the matching method. The priority coefficient may be calculated based on the calculation result of past corresponding point candidates. For example, when the reliability calculation unit 183 calculates corresponding point candidates using a plurality of matching methods, the predetermined distance R centered on the corresponding point candidate whose parallax amount is the median value among the calculated corresponding point candidates. Corresponding point candidates within (for example, 3 pixels) are considered to have been matched successfully, and other corresponding point candidates are considered to have failed to match. The reliability calculation unit 183 holds the result of the success / failure of the matching for each of the plurality of matching methods, and based on the calculation result of the corresponding point candidates for the plurality of measurement points, Each priority coefficient is calculated.

  For example, when corresponding point candidates for 10 or more measurement points are calculated, the reliability calculation unit 183 calculates a priority coefficient for each matching method. The priority coefficient is calculated as a matching success rate, which is a ratio of the number of times matching is successful to the number of times corresponding point candidates are calculated.

  As described above, by calculating the priority coefficient based on the calculation result of the past corresponding point candidate, the past matching result is reflected in the calculation of the reliability and the determination of the corresponding point, thereby improving the matching reliability. be able to.

  The priority coefficient may be calculated for each condition affecting the success rate of matching using the above method. Conditions that affect the success rate of matching include measurement targets and conditions based on image data. The measurement target may be a specific subject or a specific area in the subject. Conditions based on image data are object distance, texture value, luminance value, and the like. The condition based on the image data may be a condition whether or not halation is included in the image. The reliability calculation unit 183 calculates and manages a priority coefficient for each of the above conditions for each of a plurality of matching methods. That is, for each of the plurality of matching methods, a priority coefficient for each measurement target, a priority coefficient for each object distance, a priority coefficient for each texture value, and a priority coefficient for each luminance value are calculated.

  For example, when the corresponding point candidate is calculated using a plurality of matching methods, the reliability calculation unit 183 determines whether matching has succeeded or failed for each matching method as described above. For each matching method, the reliability calculation unit 183 holds the result of the success / failure of the matching in association with the information to be measured at that time. Alternatively, the reliability calculation unit 183 calculates an object distance, a texture value, or a luminance value based on the image data used for calculating the corresponding point candidate at that time, and calculates the calculated value (or calculation) for each matching method. The result of matching success / failure is held in association with a predetermined range including the determined value. When calculating the object distance, it is necessary to calculate three-dimensional coordinates. Therefore, the process performed in step S3 in FIG. 4 is performed in the matching process in step S2.

  When calculating the priority coefficient of each matching method, the reliability calculation unit 183 performs matching for the same measurement target, or for the same object distance, texture value, and luminance value (or the same range related to them). Based on the result of success / failure, a priority coefficient is calculated for each condition. For example, when measuring a specific measurement target, the user repeatedly sets corresponding measurement points in advance for the same measurement target and calculates corresponding point candidates, and each priority coefficient of a plurality of matching methods Is calculated.

  As described above, by providing a priority coefficient for each condition, matching reliability can be improved.

  As described above, according to the present embodiment, a plurality of corresponding point candidates are calculated using a plurality of matching methods, and if the matching reliability is low, the corresponding point candidate selected by the user is determined as the corresponding point. However, when the reliability of matching is high, by determining a corresponding point candidate automatically selected from a plurality of corresponding point candidates as a corresponding point, it is possible to reduce incorrect matching and reduce the user's effort. .

  In addition, by calculating the matching reliability or determining the corresponding point using the priority coefficient of the matching method, the matching result by the specific matching method is made larger for the matching reliability or the corresponding point. It can be reflected. For example, when an image obtained by photographing a specific measurement target is used, the matching result suitable for the specific measurement target can be more greatly reflected on the matching reliability or the corresponding point.

  In addition, by calculating the priority coefficient of the matching method based on the past corresponding point candidate calculation results, the past matching results are reflected in the calculation of matching reliability and the determination of corresponding points. Can be improved. Furthermore, the reliability of matching can be improved by providing a priority coefficient for each condition such as the measurement target, object distance, texture value, and luminance value when the corresponding point candidate is calculated.

  In addition, when displaying a plurality of corresponding point candidates calculated using a plurality of matching methods, it is possible to reduce degradation in image visibility by displaying overlapping corresponding point candidates in the same display form. Further, by displaying the number of overlapping corresponding point candidates in the vicinity of the corresponding point candidates, it is possible to inform the user in an easy-to-understand manner whether the corresponding point candidates are overlapping, that is, which corresponding point candidate has high reliability. Can do.

  In addition, when displaying a plurality of corresponding point candidates calculated using a plurality of matching methods, the corresponding point candidates are displayed in a display form corresponding to the priority coefficient corresponding to each of the plurality of matching methods, thereby allowing the user to On the other hand, the result of matching by a specific matching method can be notified more clearly. For example, when using an image obtained by capturing a specific measurement target, the corresponding point candidate calculated using a matching method suitable for the specific measurement target is displayed in a larger size so that the user can have a more reliable corresponding point. This makes it easier to select candidates.

  In addition, after calculating a plurality of corresponding point candidates using a plurality of matching methods, a new matching point candidate is calculated using a matching method different from the plurality of matching methods when the user instructs the calculation of a new corresponding point candidate. By calculating the corresponding point candidates, it is possible to reduce the user's troubles such as re-taking the image or manually correcting the corresponding points.

  Further, when displaying newly calculated corresponding point candidates, by displaying only the corresponding point candidates that overlap with other corresponding point candidates among the newly calculated corresponding point candidates, the number of corresponding point candidates is reduced. Even if it increases, it becomes easy for the user to select corresponding point candidates.

  In addition, when displaying newly calculated corresponding point candidates, the number of corresponding point candidates displayed can be kept constant, thereby preventing a reduction in image visibility, and the user can select corresponding point candidates. It becomes easy to do.

  As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to the above-described embodiments, and includes design changes and the like without departing from the gist of the present invention. .

  DESCRIPTION OF SYMBOLS 1 Endoscope apparatus, 2 Endoscope, 3 apparatus main body, 4 monitor (display part), 5 housing | casing, 6 operation part, 8 endoscope unit, 9 CCU, 10 control unit, 12 video signal processing circuit, 13 ROM, 14 RAM, 18 CPU, 28 Image sensor (imaging unit), 180 control unit, 181 measurement point setting unit, 182 corresponding point candidate calculation unit, 183 reliability calculation unit, 184 display processing unit, 185 corresponding point determination unit, 186 Measurement processing unit

Claims (13)

An imaging unit that captures a first subject image and a second subject image corresponding to the same subject to generate image data;
A display unit for displaying an image based on the image data;
A corresponding point candidate calculation unit that calculates a corresponding point candidate that is a point on the second subject image corresponding to the measurement point set on the first subject image using a plurality of matching methods;
A reliability calculation unit that calculates the reliability of matching based on the corresponding point candidates corresponding to each of the plurality of matching methods;
When the reliability is less than a predetermined value, a display processing unit that performs processing for displaying the measurement point and the corresponding point candidate corresponding to each of the plurality of matching methods on the image;
When the reliability is greater than or equal to a predetermined value, the corresponding points are determined based on the corresponding point candidates corresponding to each of the plurality of matching methods, and when the reliability is less than a predetermined value, A corresponding point determination unit that determines the corresponding point candidate specified among the corresponding point candidates corresponding to each of a plurality of matching methods as a corresponding point;
A measurement processing unit that measures the size of the subject based on the measurement points and the determined corresponding points;
An endoscope apparatus characterized by comprising:   The reliability calculation unit calculates the reliability based on a distance on the second subject image between the corresponding point candidates corresponding to each of the plurality of matching methods. The endoscope apparatus described in 1.   The reliability calculation unit calculates the reliability based on a position of the corresponding point candidate corresponding to each of the plurality of matching methods and a priority coefficient corresponding to each of the plurality of matching methods. The endoscope apparatus according to claim 1.   The corresponding point determination unit, when the reliability is equal to or higher than a predetermined value, the position of the corresponding point candidate corresponding to each of the plurality of matching methods and the priority corresponding to each of the plurality of matching methods The endoscope apparatus according to claim 1, wherein the corresponding points are determined based on a coefficient.   The reliability calculation unit calculates a priority coefficient corresponding to each of the plurality of matching methods based on a result of calculating the corresponding point candidate corresponding to each of the plurality of matching methods in the past. The endoscope apparatus according to claim 3 or 4, wherein the endoscope apparatus is characterized.   The endoscope apparatus according to claim 3 or 4, wherein the priority coefficient is provided for each condition when the corresponding point candidates are calculated.   The display processing unit performs a process of displaying overlapping corresponding candidate candidates on the image in the same display form among the corresponding point candidates corresponding to each of the plurality of matching methods. The endoscope apparatus according to claim 1.   The endoscope apparatus according to claim 7, wherein the display processing unit performs a process of displaying the number of overlapping corresponding point candidates in the vicinity of the corresponding point candidate.   The display processing unit performs processing of displaying the corresponding point candidates corresponding to each of the plurality of matching methods on the image in a display form according to a priority coefficient corresponding to each of the plurality of matching methods. The endoscope apparatus according to claim 1. The corresponding point candidate calculation unit further calculates the corresponding point candidate using a matching method different from the plurality of matching methods,
The display processing unit performs the measurement when the reliability is less than a predetermined value and the display of the corresponding point candidates calculated using a matching method different from the plurality of matching methods is instructed. A process of displaying on the image a point, the corresponding point candidate corresponding to each of the plurality of matching methods, and the corresponding point candidate calculated using a matching method different from the plurality of matching methods Done
The corresponding point determination unit, when the reliability is less than a predetermined value and the display of the corresponding point candidate calculated using a matching method different from the plurality of matching methods is instructed, Corresponding to the corresponding point candidate specified among the corresponding point candidates corresponding to each of a plurality of matching methods and the corresponding point candidates calculated using a matching method different from the plurality of matching methods The endoscope apparatus according to claim 1, wherein the endoscope apparatus is determined as a point.   The display processing unit is calculated using a matching method different from the plurality of matching methods when displaying the corresponding point candidate calculated using a matching method different from the plurality of matching methods on the image. Among the corresponding point candidates, the other corresponding points calculated using a matching method that overlaps with the corresponding point candidates calculated using the plurality of matching methods or different from the plurality of matching methods. The endoscope apparatus according to claim 10, wherein only the corresponding point candidates that overlap with the point candidates are displayed.   When the display processing unit displays the corresponding point candidate calculated by using a matching method different from the plurality of matching methods on the image, any one of the already displayed corresponding point candidates is displayed. The endoscope apparatus according to claim 10, wherein the number of candidates for the corresponding points to be displayed is kept constant by being hidden. The second corresponding to the measurement point set on the first subject image of the image based on the image data generated by capturing the first subject image and the second subject image corresponding to the same subject. Calculating corresponding point candidates that are points on the subject image using a plurality of matching methods;
Calculating a matching reliability based on the corresponding point candidates corresponding to each of the plurality of matching methods;
When the reliability is less than a predetermined value, performing a process of displaying the measurement point and the corresponding point candidate corresponding to each of the plurality of matching methods on the image;
When the reliability is greater than or equal to a predetermined value, the corresponding points are determined based on the corresponding point candidates corresponding to each of the plurality of matching methods, and when the reliability is less than a predetermined value, Determining the designated corresponding point candidate among the corresponding point candidates corresponding to each of a plurality of matching methods as a corresponding point;
Measuring the size of the subject based on the measurement points and the determined corresponding points;
A program that causes a computer to execute.

Images