JP6730029B2 - Image processing device, image processing method, image processing program, and imaging device - Google Patents

Description

The present invention relates to a technique for measuring the length of a subject from an image captured by an image capturing device.

A method (stereo measurement) of measuring the length and three-dimensional position (coordinates) of a subject using two cameras (stereo cameras) is known. In stereo measurement, a subject is imaged using two cameras arranged at different positions, the three-dimensional position of the subject is calculated based on the parallax of the subject between the two images captured by the two cameras, Measure the length of the subject and the distance to the subject. The measurement accuracy of the stereo measurement is the distance to the subject, the baseline length that is the distance between the two cameras, the focal length of the lens, the camera specifications such as the number of pixels, the calculation accuracy of the parallax, and the image of the measurement point. It depends on the position specification accuracy of.

There is a technique described in Patent Document 1 as a technique for improving measurement accuracy in measurement using a stereo camera. Patent Document 1 discloses a technique for improving the measurement accuracy by calculating the ratio (relative error) of the distance to the subject and the accuracy of depth, and when the relative error is large, re-imaging from a position where the relative error becomes smaller. Is listed.

JP, 2011-232330, A (Published November 17, 2011)

However, the technique described in Patent Document 1 has the following problems. The problem of the technique described in Patent Document 1 will be described with reference to FIG. FIG. 11 is a diagram for explaining a problem in the conventional stereo measurement, FIG. 11A is a diagram showing an example of the stereo measurement, and FIG. 11B is another example of the stereo measurement. FIG.

In FIG. 11A, rectangular areas 1102 and 1103 surrounding the measurement points 1100 and 1101 respectively indicate error ranges when the measurement points 1100 and 1101 are stereo-measured. The error range will be described later. In addition, in FIG. 11B, since the distance Z to the measurement point is shorter than that in FIG. It is smaller than the areas 1102 and 1103 shown.

As shown in FIG. 11, when the distance to the measurement point becomes shorter, the theoretical error range becomes smaller, and the relative error that is the ratio of the distance Z to the subject and the depth accuracy ΔZ becomes smaller. In Patent Document 1, the measurement accuracy is improved by photographing from a position where the relative error becomes small. However, the measurement accuracy depends on the sizes of the areas 1102, 1103, 1104, and 1105 which are theoretical error ranges. Therefore, even if an image is taken while changing the shooting distance, it is not possible to perform measurement with higher accuracy than the theoretical error range in the shooting distance.

The present invention has been made in view of the above problems, and an object thereof is to provide a technique for improving the measurement accuracy of the three-dimensional coordinates of a measurement point.

In order to solve the above problems, an image processing device according to an aspect of the present invention is an image processing device that calculates three-dimensional coordinates of measurement points that are commonly included in a plurality of images. A selection unit that selects a included point as a measurement reference point and a correction unit that refers to the measurement reference point and corrects the coordinates of the measurement point are provided.

In order to solve the above problems, an image processing method according to an aspect of the present invention is an image processing method for calculating three-dimensional coordinates of measurement points that are commonly included in a plurality of images. , A selection step of selecting a point included in the image as a measurement reference point, and a correction step of correcting the coordinates of the measurement point with reference to the measurement reference point.

According to the present invention, it is possible to improve the measurement accuracy of the three-dimensional coordinates of the measurement point.

1 is a block diagram showing a configuration example of an image processing apparatus according to a first embodiment. It is a figure which shows the principle of stereo measurement. It is a figure which shows the error of the distance between two points calculated by stereo measurement, (a) has shown the to-be-photographed object, (b) has shown the figure which looked at (a) of FIG. 3 from the top. (C) is a diagram showing an example of the measured distance, (d) is a diagram showing another example of the measured distance, and (e) is still another example of the measured distance. FIG. FIG. 4 is a diagram showing a method of calculating three-dimensional coordinates of a measurement point according to the first embodiment of the present invention, where (a) shows a part of an image of the subject shown in (a) of FIG. 3; (B) is a diagram showing a plurality of points (measurement reference points) designated by the user and a straight line obtained by fitting, and (c) is a case where only one coordinate on the image of the measurement point is designated. Error range and the error range of the correction point which is the intersection of the fitted straight line, and (d) shows the stereo measurement when the specified accuracy of the measurement point is within the error range shown in (c) of FIG. 3 shows the error range of the three-dimensional coordinates. FIG. 4 is an XZ plan view of the subject shown in FIG. 3A viewed from the Y direction. FIG. 4 is an XZ plan view of the subject shown in FIG. 3A viewed from the Y direction. The error range at the time of carrying out the stereo measurement of the three-dimensional coordinates of the measurement point in Embodiment 1 of the present invention is shown. The figure shows the case where the point on the subject is stereo-measured, and the straight line indicated by the dotted line shows the position of the three-dimensional coordinate obtained by the stereo measurement when the coordinate of the measurement point on the image is designated. The case where a point on the subject is measured is shown, and the straight line indicated by the dotted line is the line on which the three-dimensional coordinates of the measurement point are located. It is a figure which shows the error range and straight line in Embodiment 2 of this invention, (a) is an example which shows the relationship between an error range and a straight line, (b) shows the relationship between an error range and a straight line. Another example. It is a figure for demonstrating the subject in the conventional stereo measurement, (a) is a figure which shows an example of a mode of stereo measurement, (b) is a figure which shows another example of a mode of stereo measurement. ..

[First Embodiment]
An embodiment of the present invention will be described below with reference to the drawings.

(Image processing device)
FIG. 1 is a block diagram showing a configuration example of an image processing apparatus 100 according to this embodiment. As shown in FIG. 1, the image processing apparatus 100 includes an image acquisition unit 101, a measurement unit 102, a storage unit 103, and a measurement point reception unit 104.

The image acquisition unit 101 acquires an image captured by a stereo camera as an input image.

The measurement unit 102 performs measurement based on the image acquired by the image acquisition unit. The measurement unit 102 also includes a measurement value calculation unit 105 and a measurement correction unit 106 (selection unit, correction unit).

The measurement value calculation unit 105 measures the distance and length to the subject that is the measurement target, and calculates the measurement value.

The measurement correction unit 106 performs correction so that the measurement accuracy of the measurement value calculated by the measurement value calculation unit 105 is improved.

The storage unit 103 stores camera parameters necessary for measuring a baseline length, a focal length, and the like. When the measuring unit 102 measures, the information stored in the storage unit 103 is referred to.

The measurement point accepting unit 104 has a function of accepting an operation of selecting a subject (measurement point) to be measured from a user (operator). The measurement point reception unit 104 acquires measurement point information indicating the measurement point selected by the operator, and outputs the acquired measurement point information to the measurement unit 102.

Details of the processing performed by each unit will be described later. The image processing apparatus 100 shown in FIG. 1 may be included in an imaging device (stereo camera).

(Measurement error of stereo measurement)
FIG. 2 is a diagram showing the principle of stereo measurement. The imaging device 200a is arranged on the left side and the imaging device 200b is arranged on the right side. The imaging device 200a includes an imaging element 201a and a lens 202a, and the imaging device 200b includes an imaging element 201b and a lens 202b. The baseline lengths of the two image pickup devices are B, and the focal lengths thereof are both f. In FIG. 2, when the imaging device 200a is used as a reference during measurement, the three-dimensional coordinates (X, Y, Z) of the measurement point 203 have a parallax d (d=d2-d1) corresponding to the measurement point 203. , Focal length f, and base line length B are used to calculate the following equations (1), (2), and (3).

The x in the equation (2) and the y in the equation (3) are the x coordinate and the y coordinate of the measurement point 203 on the image captured by the image capturing apparatus 200a, respectively. When measuring the distance between two points, the distance between the two points can be calculated by calculating the three-dimensional coordinates of each of the two measurement points. As shown in Expression (1), Expression (2), and Expression (3), in order to calculate the three-dimensional coordinates of the measurement point by stereo measurement, in addition to camera parameters such as the camera base length and focal length, , And the coordinates (two-dimensional coordinates) on the image of the parallax and the measurement target point are required.

Therefore, in stereo measurement, the accuracy of parallax calculation and the accuracy of specifying the coordinates of the measurement points on the image affect the measurement accuracy. When the coordinates of the measurement point specified on the parallax or the image include an error with respect to each true value, the calculated three-dimensional coordinate also has an error range corresponding to the error of the coordinate of the measurement point on the parallax or the image. I will have it. That is, the three-dimensional coordinates of the measurement point obtained by the stereo measurement may include an error corresponding to the error range with respect to the true value of the three-dimensional coordinate of the measurement point. An area 204 shown in FIG. 2 indicates an error range, and when the three-dimensional coordinates obtained by the stereo measurement are the points 203 shown in FIG. 2, the true value of the three-dimensional coordinates is located anywhere in the area 204. It will be.

As described above, the three-dimensional coordinate calculated by the stereo measurement has a theoretical error corresponding to the parallax of the measurement point or the error of the coordinate on the image. Therefore, even when measuring the distance between the two points, A measured value including a theoretical error can be obtained. The fact that the distance between two points measured by stereo measurement includes an error will be described with reference to FIG.

FIG. 3 is a diagram showing an error in the distance between two points calculated by stereo measurement. FIG. 3A shows a subject 300 and a subject 301, and the two subjects are located at the front and rear. In FIG. 3A, a case will be described in which two points, a point 302 on the subject 300 and a point 303 on the subject 301, are set as measurement points and the distance between the two points is measured.

FIG. 3B is a top view of FIG. 3A and is an XZ plan view. An area 304 in FIG. 3B is an error range of the measurement point 302, and an area 305 is an error range of the measurement point 303.

3C is a diagram showing an example of the measured distance d1, FIG. 3D is a diagram showing another example of the measured distance d2, and FIG. 3E is a diagram showing the measured distance d3. It is a figure which shows another example. The distance d1 shown in FIG. 3(c), the distance d2 shown in FIG. 3(d), and the distance d3 shown in FIG. 3(e) are all the distances between the measurement points 302 and 303. The measured values are shown in the case. Further, the positions of the measurement points 302 and 303 indicate the positions of the three-dimensional coordinates calculated by the stereo measurement (the positions in the XZ plane in (c), (d) and (e) of FIG. 3). ing. The positions of the measurement points 302 and 303 shown in FIG. 3C are correct three-dimensional coordinate positions, and the distance d1 is the true value of the distance between the measurement points 302 and 303. As shown in (d) of FIG. 3 and (e) of FIG. 3, since the three-dimensional coordinates calculated in stereo measurement include an error, the distance d2 and the distance are measured even when measuring the distance between two points. An error occurs with respect to the true value like d3. Therefore, in the present embodiment, a method of improving the measurement accuracy by performing a process of reducing an error in parallax or coordinates on an image and reducing an error range will be described.

FIG. 4 is a diagram showing a method of calculating the three-dimensional coordinates of a measurement point in the first embodiment of the present invention, and FIG. 4A is a part of an image obtained by imaging the subject 301 shown in FIG. Is shown. The measurement point 401 is a measurement point that the user wants to measure, and points to the corner portion of the subject 301. The measurement point acceptance unit 104 accepts the coordinates of the measurement point 401 on the image and outputs the accepted coordinates to the measurement unit 102. As a method of designating the coordinates of the measurement point 401 on the image, for example, the cursor on the screen of the display device that displays the image may be moved to the measurement point and then the enter key or the like may be pressed to select it. If the display device is provided with a touch panel, the measurement point may be designated by the user's touch operation. Further, in order to accurately specify the coordinates, the measurement point may be specified after the image is enlarged and displayed.

However, the user cannot always correctly specify the coordinates of the measurement point 401, and an error may occur in the coordinates specified by the user with respect to the coordinates of the measurement point 401. The designated point 402 shown in FIG. 4A indicates the position on the image designated by the user. As shown in FIG. 4A, when the coordinates of the designated point 402 are deviated from the coordinates of the measurement point 401, an error also occurs in the three-dimensional coordinates of the measurement point 401 calculated by stereo measurement. ..

Therefore, in the present embodiment, the measurement correction unit 106 corrects the coordinates of the designated point 402 by using the information around the designated point 402, and obtains a correction point close to the measured point 401, thereby improving the designated accuracy of the measured point. A method (image processing method) for improving the measurement accuracy will be described. As a method of accurately specifying a measurement point, for example, there is a method in which the measurement correction unit 106 detects two straight lines passing near the specified point 402 and uses the intersection as a correction point.

In FIG. 4A, the measurement correction unit 106 detects a straight line 403 on the left side of the subject 301 and a straight line 404 on the front side of the subject 301 as straight lines around the designated point 402 (selection step). .. Then, the measurement correction unit 106 sets the intersection of the detected straight line 403 and the detected straight line 404 as a correction point (correction step).

As a method of detecting a straight line, a known method such as Hough transform can be used. Further, the user can specify a plurality of points on a straight line around the specified point 402, and the measurement correction unit 106 can obtain a straight line (approximate straight line) by fitting the specified plurality of points. When the user specifies it, a message such as “Specify a point on the straight line through which the measurement point passes” is displayed on the display device, and the measurement point receiving unit 104 receives the coordinates on the straight line. The straight line fitting can be performed by a known method such as the least square method.

In order to explain the effect of using the intersection of the straight line around the designated point 402 as the correction point, the case where the user designates a plurality of points on the straight line will be described in detail. FIG. 4B is a diagram showing a plurality of points (measurement reference points) designated by the user and a straight line obtained by fitting. Points 405, 406, 407, and 408 shown in FIG. 4B indicate points (measurement reference points) designated by the user as points on the straight line 403 shown in FIG. 4A. ing. The measurement correction unit 106 obtains a straight line 414 indicated by a dotted line by fitting the designated point 402, the point 405, the point 406, the point 407, and the point 408.

Similarly, a point 409, a point 410, a point 411, a point 412, and a point 413 shown in FIG. 4B are designated by the user as points on the straight line 404 shown in FIG. The measurement reference point) is shown. The measurement correcting unit 106 obtains a straight line 415 indicated by a dotted line by fitting the designated point 402, the point 409, the point 410, the point 411, the point 412, and the point 413. Then, the measurement correction unit 106 sets the intersection of the straight line 414 and the straight line 415 (two approximate straight lines) as a correction point (two-dimensional coordinate on the image of the measurement point).

As shown in FIG. 4B, the intersection (correction point) of the straight line 414 and the straight line 415 is closer to the measurement point 401 than the designated point 402 on the image designated by the user. Although each point designated by the user is displaced from the desired position (position on the straight line 403 or the straight line 404) like the designated point 402, by fitting a plurality of points, the position of the straight line ( The deviation between the straight line 403 and the straight line 404) and the straight line obtained by the fitting (the straight line 404 and the straight line 415) becomes small, and the designation accuracy of the correction point, which is the intersection of the straight lines, improves.

FIG. 4C is an error range showing the deviation of the coordinates of the designated point on the image from the coordinates of the original measurement point 401 on the image when only one coordinate of the measurement point 401 is designated on the image. 416 and an error range 417 indicating the deviation of the correction point on the image of the correction point, which is the intersection of the fitted straight line, from the coordinate on the image of the original measurement point 401. For example, the error range is ±0.5 pixels when the user correctly specifies the coordinates of the image of the measurement point with integer pixel precision, and the error range is large when the user's specified position deviates by integer pixel precision or more. As shown in (c) of FIG. 4, the fitting reduces the deviation of the original measurement point from the coordinate on the image, and reduces the error range of the coordinate on the image.

FIG. 4D shows an error range of three-dimensional coordinates in stereo measurement in the case where the designated accuracy of the measurement point is the error range shown in FIG. 4C. The three-dimensional coordinates are calculated by the formula (1), the formula (2), and the formula (3). When the coordinates on the image include an error, the value of x in the formula (2) and the value of y in the formula (3) are Since it changes within the error range, an error occurs in the corresponding X coordinate and Y coordinate. As shown in FIG. 4D, when the error range of the coordinates on the image is the error range 416, the error range of the three-dimensional coordinates is the error range 418 shown in FIG. 4D. When the error range of the coordinates on the image is the error range 417, the error range of the three-dimensional coordinates is the error range 419 shown in (d) of FIG. The error in the Z direction in (d) of FIG. 4 is determined by the error of the parallax d. For example, when the parallax d includes an error of ±0.5 pixel, the error ΔZ in the Z direction is calculated from the equation (1) as follows.

It is expressed as. Since the formula (2) and the formula (3) include the coordinate Z in the Z direction, the error ΔZ in the Z direction also affects the three-dimensional coordinates in the X direction and the Y direction. As shown in (d) of FIG. 4, when the accuracy of specifying the coordinates of the measurement point on the image increases, the error range in the XY directions of the three-dimensional coordinates calculated by stereo measurement decreases, and the measurement accuracy improves.

In the example described with reference to FIG. 4B, the intersection of two straight lines is the correction point, but there is only one straight line that passes through the designated point 402 (straight line around the designated point 402). In this case, the one straight line may be detected, and a point on the detected one straight line which is the closest to the designated point may be used as the correction point. If there is no straight line passing through the designated point 402 (straight line around the designated point 402) and the designated point 402 is a point on the curve (when there is a curved line around the designated point 402), Of the detected points on the curve, the points closest to the designated point 402 may be used as the correction points.

Next, a method of reducing the error range caused by the parallax error and improving the measurement accuracy will be described. FIG. 5 shows an XZ plan view of the subject 301 shown in FIG. 3A viewed from the Y direction. The measurement point is the measurement point 303, and an area 501 indicated by a square indicates an error range when the measurement point 303 is stereo-measured. A straight line 502 indicated by a dotted line indicates the position of the three-dimensional coordinate of the measurement point 303 obtained when stereo measurement is performed in the situation where the coordinates of the measurement point 303 on the image are designated. In other words, the straight line 502 indicates the possible range of the three-dimensional coordinates of the measurement point 303 obtained by stereo measurement in the situation where the coordinates of the measurement point 303 on the image are designated.

When the two-dimensional coordinate of the measurement point 303 on the image is determined, the three-dimensional coordinate of the measurement point 303 moves on the straight line 502 based on the parallax value. The dimensional coordinate is the position of the measurement point 303. On the other hand, when the parallax is calculated to be larger than the true value, it becomes a position before the actual position of the measurement point, for example, the position of the point 503 shown in FIG. When the parallax is calculated to be smaller than the true value, the position is deeper than the actual position of the measurement point, for example, the position of the point 504 shown in FIG. That is, when the calculated parallax includes an error with respect to the true value, the calculated three-dimensional coordinates also include an error with respect to the true value. Therefore, in the present embodiment, a method of improving the measurement accuracy by performing a process of reducing the parallax error and reducing the error range caused by the parallax error will be described.

FIG. 6 shows an XZ plan view of the subject 301 shown in FIG. 3A viewed from the Y direction, and the measurement point is the measurement point 303. The measuring unit 102 obtains the three-dimensional coordinates of the calculation point 601 by measuring the measurement point 303 in stereo. Here, since the stereo measurement by the measurement unit 102 includes the error range, the actual position of the measurement point 303 (the true value of the three-dimensional coordinate of the measurement point 303) and the three-dimensional coordinate of the calculation point 601 calculated by the stereo measurement are calculated. The position is incorrect. Further, a straight line 602 indicated by a dotted line indicates a possible range of three-dimensional coordinates of the measurement point 303 calculated when the coordinates of the measurement point 303 on the image are designated. The reason why the straight line 602 does not pass through the measurement point 303 is because it shows a diagram assuming that the coordinates of the measurement point 303 specified on the image have an error, and if the coordinates specified on the image are correct. The straight line 602 passes through the measurement point 303.

Regarding points 603, 604, 605, 606, 607, 608, and 609 in FIG. 6, the measurement correction unit 106 stereo-measures points (measurement reference points) on the straight line 610 that the measurement point 303 passes through. The three-dimensional coordinates in the case are shown. Since the stereo measurement includes the error range, the three-dimensional coordinates of the points calculated by the measurement correction unit 106 are not always located on the straight line 610. A straight line 611 indicated by a dotted line in FIG. 6 is obtained by fitting the three-dimensional coordinates of the points 603, 604, 605, 606, 607, 608, and 609 obtained by the measurement correction unit 106 performing stereo measurement. The straight line obtained by is shown. As shown in FIG. 6, with respect to the shift between the three-dimensional coordinates of each point calculated by the measurement correction unit 106 and the straight line 610, the shift between the straight line 611 fitted to each point calculated by the measurement correction unit 106 and the straight line 610. Becomes smaller.

Therefore, the measurement unit 102 calculates the three-dimensional coordinates assuming that the three-dimensional coordinates of the measurement point 303, which is a point on the straight line 610, are located on the straight line 611. That is, the intersection of the straight line 602 and the straight line 611 is set as the correction point 612. In the case of FIG. 6, since the calculated three-dimensional coordinates are the three-dimensional coordinates of the correction point 612, the error with the measurement point 303 is smaller than that of the calculation point 601 before correction. As described above, by correcting the three-dimensional coordinates of the measurement points with reference to the three-dimensional coordinates of the points around the measurement point 303, the error caused by the parallax error is reduced and the measurement accuracy is improved.

FIG. 7 shows an error range in the case where the three-dimensional coordinates of the measurement point 701 are stereo-measured in the first embodiment of the present invention. An area 702 shown in FIG. 7 shows an error range when only the three-dimensional coordinates of the measurement point 701 are calculated, and an area 703 refers to the three-dimensional coordinates of points around the measurement point 701 to measure the measurement point 701. The error range when the three-dimensional coordinate of is corrected is shown. Since the measurement correction unit 106 can reduce the error range by referring to the three-dimensional coordinates of points around the measurement point 701, the measurement accuracy can be improved. It is preferable to increase the number of points used for fitting because the accuracy of fitting is improved. Further, in order to increase the number of points used for fitting, the three-dimensional coordinates of the measuring point itself may be included in the three-dimensional coordinates used for fitting.

Here, the correction of the three-dimensional coordinates of the measurement point will be described in more detail. Since FIG. 6 shows the inside of the XZ plane, the straight line 611 calculated by the measurement correction unit 106 by fitting the three-dimensional coordinates of the points on the straight line 611 and the possible range of the three-dimensional coordinates of the measurement point 303 are shown. There is an intersection with the straight line 602. However, when defining a straight line in a three-dimensional space, the intersection does not always exist. Therefore, when the two straight lines do not have an intersection point, among the points on the straight line 602 where the measurement points are located, the point (closest contact point) that is closest to the straight line 611 is the three-dimensional point after correction of the measurement points. You can use coordinates. Alternatively, the three-dimensional coordinates of the measurement points may be corrected by fitting a plane from the three-dimensional coordinates of the points around the measurement points.

FIG. 8 shows a case where a point 802 on the subject 801 is stereo-measured in the first embodiment of the present invention, and a straight line 803 indicated by a dotted line indicates a case where the coordinates of the measurement point 802 on the image are designated. The possible range of three-dimensional coordinates obtained by stereo measurement is shown. Points other than the measurement points 802 shown in FIG. 8 indicate three-dimensional coordinates in which the points (measurement reference points) on the sides around the quadrangular surface 804 on the front side of the subject 801 are stereo-measured. A surface 805 indicated by a dotted line represents a plane (approximate plane) obtained by fitting the three-dimensional coordinates obtained by stereo-measuring points on the sides around the surface 804 with a plane. Then, the intersection of the surface 805 and the straight line 803 is used as the correction point, and the corrected three-dimensional coordinate of the measurement point is set. When calculating the intersection of a surface and a straight line, it is preferable that the intersection always exists unless the surface and the straight line are parallel. When fitting a straight line or a surface with reference to the three-dimensional coordinates of points around the measurement point, it is preferable that the positional relationship with the straight line where the three-dimensional coordinates of the measurement point are located is close to orthogonal.

FIG. 9 shows a case where the measurement point 902 on the subject 901 according to the first embodiment of the present invention is measured, and a straight line 903 indicated by a dotted line indicates a case where the coordinates of the measurement point 902 on the image are designated. The possible range of three-dimensional coordinates obtained by stereo measurement is shown. A straight line 904 indicated by a dotted line is a straight line calculated by fitting three-dimensional coordinates of points on the straight line on the right side of the subject 901, and a straight line 905 indicated by a dotted line is three points on the straight line above the subject 901. It is a straight line calculated by fitting the dimensional coordinates. A point 906 is an intersection of the straight line 903 and the straight line 904, and a point 907 is an intersection of the straight line 903 and the straight line 905.

When the errors of the points 906 and 907 from the measurement point 902 are compared, the errors of the points 907 and 902 are smaller. Since the straight line 903 and the straight line 904 are almost parallel to each other, the position of the intersection of the two straight lines largely changes due to a slight difference in the direction and position of the straight lines. On the other hand, since the straight line 903 and the straight line 905 are nearly orthogonal, the change in the position of the intersection point due to the difference in the direction and the position of the straight line is small. That is, the change in the three-dimensional coordinates after the correction of the measurement point is small, and the measurement error is small, which is preferable. Although the case of a straight line and a straight line has been described with reference to FIG. 9, the same applies to the case of calculating the intersection point of the surface and the straight line, in order to correct the three-dimensional coordinate of the measurement point for the surface where the surface and the straight line are nearly orthogonal. If it is selected as the surface to be referred to, the measurement accuracy becomes high, which is preferable.

As described above, in the present embodiment, the image processing apparatus 100 performs stereo measurement (when calculating three-dimensional coordinates of measurement points commonly included in a plurality of images) on the image of the measurement points. The coordinates of the specified point of 3 and the three-dimensional coordinates of the calculation point calculated based on the parallax are selected from the points around the measurement point on the image (points included in the image, measurement reference point), and information on the selected point The three-dimensional coordinates of the measurement point are corrected by using the correction. Therefore, the image processing device 100 can improve the measurement accuracy of the measurement point.

In addition, in the above-described embodiment, the configuration in which the image processing apparatus 100 corrects the three-dimensional coordinates of the measurement point is described, but the image processing apparatus 100 corrects only the designated point on the image of the measurement point (the image of the measurement point). The measurement accuracy can be improved by correcting the above two-dimensional coordinates) and performing stereo measurement on the corrected coordinates to calculate the three-dimensional coordinates. Further, the image processing apparatus 100 can improve the measurement accuracy even if the designated point on the image of the measurement point is set as the coordinate designated by the user and the three-dimensional coordinate of the measurement point calculated based on the parallax is corrected. ..

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In the second embodiment, the measurement accuracy is improved by referring to more suitable information regarding the information referred to when correcting the coordinates of the measurement point on the image and the three-dimensional coordinates of the measurement point. Here, in order to make the description clearer, in the following description, a point referred to when correcting the three-dimensional coordinates of the measurement point is referred to as a measurement reference point, and is distinguished from the measurement point.

As described above, in the present invention, the image processing apparatus 100 determines the three-dimensional coordinates of the measurement reference point to be a straight line or a plane through which the measurement point passes (including a straight line or a plane calculated by referring to points around the measurement point). The measurement accuracy can be improved by calculating from. However, the calculated straight line or plane is not always located on the same straight line or the same plane as the measurement point. Therefore, in the second embodiment, a configuration for evaluating whether or not the acquired measurement reference point and the reference information calculated from the measurement reference point are suitable information for correcting the measurement point will be described.

FIG. 10 is a diagram showing an error range and a straight line in the second embodiment of the present invention in the second embodiment of the present invention, FIG. 10A is an example showing a relationship between the error range and the straight line, and FIG. Is another example showing the relationship between the error range and the straight line. 10A and 10B show an XZ plan view when the measurement point 1001 is stereo-measured, and the region 1002 shows an error range when the measurement point 1001 is stereo-measured.

A point 1003, a point 1004, a point 1005, a point 1006, and a point 1007 shown in FIG. 10A indicate three-dimensional coordinates of measurement reference points for correcting the three-dimensional coordinates of the measurement point 1001. A straight line 1008 indicated by a dotted line is a straight line obtained by fitting the three-dimensional coordinates of the points 1003, 1004, 1005, 1006, and 1007 by the measurement correction unit 106. In the case shown in FIG. 10A, the measurement correction unit 106 determines that the straight line 1008 passes through the inside of the error range 1002 of the measurement point 1001.

10B, 1010, 1011, 1012, and 1013 shown in FIG. 10B are measurement reference points 3 for correcting the three-dimensional coordinates of the measurement point 1001 as in FIG. 10A. The dimensional coordinates are shown. A straight line 1014 indicated by a dotted line is a straight line obtained by fitting the three-dimensional coordinates of points 1009, 1010, 1011, 1012, and 1013 by the measurement correction unit 106. In the case shown in FIG. 10B, the measurement correction unit 106 determines that the straight line 1014 does not pass inside the error range 1002 of the measurement point 1001.

Therefore, in the case of FIG. 10A, the measurement point 1001 is highly likely to be a point on the fitted straight line 1008, and a measurement reference point suitable as information for correcting the three-dimensional coordinates of the measurement point 1001. Is selected. However, in the case of FIG. 10B, the measurement point 1001 is unlikely to be a point on the fitted straight line 1014, and a measurement reference point suitable as information for correcting the three-dimensional coordinates of the measurement point 1001. Is not selected.

Therefore, in the case shown in FIG. 10B, the measurement correction unit 106 re-searches for a more suitable measurement reference point to correct the three-dimensional coordinates of the more suitable measurement point. That is, the measurement correction unit 106 evaluates the distance in the error range between the fitted straight line and the measurement point, and when both are apart, re-searches for a more suitable measurement reference point. Also, when the plane is referred to as the information for correcting the three-dimensional coordinates, the measurement correction unit 106 evaluates the distance between the fitted plane and the error range of the measurement point, as in the case of the straight line, to perform the measurement. It is possible to determine whether or not the measurement reference point selected as the information for correcting the three-dimensional coordinates of the point is suitable.

When the measurement reference point is automatically selected for the first time and the measurement reference point is to be searched again, the measurement correction unit 106 selects (reselects) the measurement reference point for the second time and thereafter. ) May be done. With this configuration, the image processing apparatus 100 automatically selects the first measurement reference point, which is preferable because the burden on the user can be reduced. Further, in the case of a re-search, there is a possibility that it is difficult for the image processing apparatus 100 to select a suitable measurement reference point. Therefore, by allowing the user to select the measurement reference point, the image processing apparatus 100 A suitable measurement reference point can be selected, and the measurement accuracy can be improved.

Next, a method for selecting the measurement reference point more preferably will be described. In order to improve the measurement accuracy of the measurement point, the method of referring to the straight line or the plane around the measurement point was described.However, the measurement reference point located on the same straight line or the same plane as the measurement point should be closer to the measurement point. Has a high probability of existence. That is, a measurement reference point that is far from the measurement point is a point on a subject different from the measurement point, or even if it is a point on the same subject, it is not a point on the same straight line or the same plane as the measurement point Will be high. Therefore, the shorter the distance from the measurement point, the higher the probability of being a point on the same straight line or the same plane as the measurement point.

Thus, when searching for the measurement reference point, it is preferable to search for a point near the measurement point. Further, the measurement reference point may be selected in consideration of the color and brightness of the image. A point located near the measurement point and on the same straight line or on the same plane as the measurement point is likely to be a point on the same subject as the measurement point. Since they are close to each other, the color and brightness are close to the measurement point. Probability is high. Therefore, if the measurement reference point is selected in consideration of the similarity in color and brightness of the measurement point, it is highly likely that the measurement reference point is located on the same straight line or the same plane as the measurement point, which is preferable.

(Example implemented by software)
Each block of the image processing apparatus 100 described above (particularly, the measuring unit 102) may be realized by a computer. In that case, a program (image processing program) for realizing the above-described image processing apparatus 100 is recorded in a computer-readable recording medium, and the program recorded in this recording medium is read by a computer system and executed. May be realized by The "computer-readable recording medium" here means a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, and a storage device such as a hard disk built in a computer system. Further, the "computer-readable recording medium" means a program that dynamically holds a program for a short time, such as a communication line when transmitting the program through a network such as the Internet or a communication line such as a telephone line. In such a case, a volatile memory inside a computer system that serves as a server or a client, which holds a program for a certain period of time, may be included. Further, the program may be one for realizing some of the functions described above, or may be one that can realize the functions described above in combination with a program already recorded in the computer system.

Moreover, you may implement|achieve a part or all of the stereo measurement in the above-mentioned embodiment as integrated circuits, such as LSI(Large Scale Integration). Each functional block of the stereo measurement may be individually implemented as a processor, or a part or all of the functional blocks may be integrated and implemented as a processor. Further, the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. In addition, when a technique for forming an integrated circuit that replaces LSI appears with the progress of semiconductor technology, an integrated circuit according to the technique may be used.

Although one embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes and the like without departing from the scope of the present invention. It is possible to

[Summary]
An image processing apparatus (100) according to aspect 1 of the present invention is an image processing apparatus that calculates three-dimensional coordinates of measurement points commonly included in a plurality of images, and measures and references points included in the images. A selection unit (measurement correction unit 106) that is selected as a point, and a correction unit (measurement correction unit 106) that refers to the measurement reference point and corrects the coordinates of the measurement point are provided.

According to the above configuration, the image processing apparatus corrects the coordinates of the measurement points by referring to the measurement reference points when calculating the three-dimensional coordinates of the measurement points commonly included in the plurality of images. Therefore, for example, even when the user cannot specify a desired measurement point, the image processing apparatus can correct the measurement point to a position close to the desired measurement point, and thus the measurement accuracy of the three-dimensional coordinates of the measurement point can be improved. Can be improved.

In the image processing device according to the second aspect of the present invention, the correction unit in the first aspect may correct the three-dimensional coordinates of the measurement point with reference to the measurement reference point.

According to the above configuration, the image processing device refers to the measurement reference point and corrects the three-dimensional coordinates of the measurement point. Therefore, the image processing apparatus can correct the calculated three-dimensional coordinates of the measurement point to coordinates close to the three-dimensional coordinates of the desired measurement point, for example, even when the user cannot specify the desired measurement point. Therefore, the measurement accuracy of the three-dimensional coordinates of the measurement point can be improved.

In the image processing apparatus according to the third aspect of the present invention, the correction unit according to the first or second aspect includes an approximate plane obtained by fitting the three-dimensional coordinate after the correction of the measurement point to the measurement reference point, and the measurement. It may be calculated as an intersection with a straight line passing through the points.

According to the above configuration, the image processing apparatus sets the intersection of the approximate plane obtained by fitting the measurement reference point and the straight line passing through the measurement point as the corrected three-dimensional coordinates. Therefore, the image processing apparatus uses the three-dimensional coordinates of the intersection of the plane where the measurement point exists or the plane near the measurement point and the straight line through which the measurement point passes, as the three-dimensional coordinates of the measurement point. It is possible to improve the measurement accuracy of three-dimensional coordinates.

In the image processing apparatus according to Aspect 4 of the present invention, the correction unit according to Aspect 1 or 2 provides the corrected three-dimensional coordinates of the measurement point, among the points on the straight line passing through the measurement point, the measurement reference point. May be calculated as the closest point to the approximate straight line obtained by fitting.

According to the above configuration, the image processing apparatus, even if there is no intersection between the straight line through which the measurement point passes and the approximate straight line obtained by fitting the measurement reference point, corrects the closest point to 3 after correction. Use dimensional coordinates. Therefore, the image processing device can calculate three-dimensional coordinates even at measurement points on various images.

In the image processing device according to the fifth aspect of the present invention, the correction unit according to the first or second aspect calculates an error range corresponding to the three-dimensional coordinates of the measurement point before correction, and obtains the measurement reference point by fitting. The approximate plane obtained by fitting the approximate plane or the measurement reference point, and the positional relationship between the error range of the measurement point is referred to, and whether or not the selection unit reselects the measurement reference point is determined. You may judge.

According to the above configuration, the image processing apparatus reselects the measurement reference point when the approximate plane or the approximate straight line is not included in the error range corresponding to the three-dimensional coordinates of the measurement point before correction. Therefore, even when the appropriate measurement reference point cannot be selected, the image processing device can reselect until the appropriate measurement reference point can be selected.

In the image processing device according to the sixth aspect of the present invention, the correction unit in the first aspect may correct the two-dimensional coordinates on the image of the measurement point based on the measurement reference point.

According to the above configuration, the image processing device corrects the two-dimensional coordinates of the measurement point on the image. Therefore, the image processing apparatus calculates the three-dimensional coordinates from the corrected two-dimensional coordinates, so that the measurement accuracy of the three-dimensional coordinates of the measurement point can be improved.

In the image processing device according to the seventh aspect of the present invention, the correction unit in the sixth aspect may calculate the intersection point on the image of two approximate straight lines obtained by fitting the two-dimensional coordinates of the measurement reference point on the image as described above. It may be two-dimensional coordinates on the image of the measurement point.

According to the above configuration, the image processing device can correct the two-dimensional coordinates of the measurement point to the coordinates close to the coordinates of the desired measurement point. Therefore, the image processing apparatus can improve the measurement accuracy of the three-dimensional coordinates of the measurement point by calculating the three-dimensional coordinates from the corrected two-dimensional coordinates.

An image processing apparatus according to aspect 8 of the present invention is the image processing device according to any one of aspects 1 to 7, further including a measurement point acceptance unit (104), wherein the measurement point acceptance unit refers to a point selected by an operator as the measurement reference. You may accept as points.

According to the above configuration, the image processing apparatus causes the operator to select the measurement reference point. Therefore, the image processing device can select an appropriate measurement reference point.

An image processing method according to aspect 9 of the present invention is an image processing method for calculating three-dimensional coordinates of measurement points commonly included on a plurality of images, wherein points included in the images are measured reference points. And a correction step of correcting the coordinates of the measurement point with reference to the measurement reference point.

According to the above configuration, the image processing method has the same effects as the image processing device according to the first aspect.

An imaging apparatus according to aspect 10 of the present invention includes the image processing apparatus according to any one of aspects 1 to 8 above.

According to the above configuration, it is possible to realize the imaging device including the image processing device according to any one of the first to eighth aspects.

The image processing device according to each aspect of the present invention may be realized by a computer. In this case, the image processing device is implemented by operating the computer as each unit (software element) included in the image processing device. The program of the image processing apparatus realized by the above, and a computer-readable recording medium recording the program are also included in the scope of the present invention.

The present invention is not limited to the above-described embodiments, but various modifications can be made within the scope of the claims, and embodiments obtained by appropriately combining the technical means disclosed in the different embodiments Is also included in the technical scope of the present invention. Furthermore, a new technical feature can be formed by combining the technical means disclosed in each embodiment.

The present invention can be used for stereo measurement.

100 image processing device 101 image acquisition unit 102 measurement unit 103 storage unit 104 measurement point acceptance unit 105 measurement value calculation unit 106 measurement correction unit (selection unit, correction unit)
200a, 200b Imaging device 201a, 201b Imaging element 202a, 202b Lens

Claims (7)

An image processing apparatus for calculating three-dimensional coordinates of measurement points included in an image,
A receiving unit that receives input of four or more measurement reference points included in the image;
Referring to approximate plane that obtained by fitting all of the above four measurement reference points accepted by the accepting section, and a correcting unit for correcting the coordinates of the measurement point,
An image processing apparatus comprising: The receiving unit image processing apparatus according to claim 1, characterized in that accepting the input of four or more points of measurement reference points for specifying the pre-Symbol approximate plane. The image processing apparatus according to claim 2, wherein the correction unit calculates the corrected three-dimensional coordinates of the measurement point as a point on the approximate plane. The correction unit determines the three-dimensional coordinates before correction of the measurement points, depending on the positional relationship between the pre-Symbol approximate plane, whether to re-enter the measurement reference points of the four or more points on the receiving unit the image processing apparatus according to any one of claims 1 to 3, characterized in that. An image processing method for an image processing apparatus to calculate three-dimensional coordinates of measurement points included in an image,
Receiving a plurality of measurement reference points included in the image, the plurality of measurement reference points for designating two approximate straight lines ;
It is the two approximate straight lines obtained by fitting the plurality of measurement reference points, and the corrected three of the measurement points based on the intersection of the two approximate straight lines designated by the plurality of measurement reference points. An image processing method, wherein the coordinates of the measurement points are corrected by calculating dimensional coordinates . An image processing program for causing a computer to function as the image processing apparatus according to claim 1, any one of 4, an image processing program for causing a computer to function as the receiving unit and the correction unit. Imaging apparatus including an image processing apparatus according to any one of claims 1 to 4.

Images