WO2017122640A1

WO2017122640A1 - Measurement device and measurement method

Info

Publication number: WO2017122640A1
Application number: PCT/JP2017/000500
Authority: WO
Inventors: 野中　俊一郎; 正志藏之下
Original assignee: 富士フイルム株式会社
Priority date: 2016-01-15
Filing date: 2017-01-10
Publication date: 2017-07-20

Abstract

The purpose of the present invention is to provide a measurement device and a measurement method such that two-dimensional information or three-dimensional information about a measured object can be calculated quickly and with high precision. A measurement device pertaining to an embodiment of the present invention is equipped with: an image acquisition unit for acquiring multiple images obtained by photographing the same photographic subject from multiple viewpoints; a reduced image generation part for generating multiple reduced images by reducing the size of each of the multiple images; a geometric area extraction part for extracting a first geometric area, that is, a geometric area in the multiple reduced images, on the basis of parallax values obtained by means of block-matching of the multiple reduced images and the positions of pixels in the multiple reduced images; a geometric equation determination part for determining a geometric equation representing a second geometric area, that is, a geometric area in the multiple reduced images which corresponds to the first geometric area, on the basis of the extracted first geometric area; and a measurement part for calculating two-dimensional information or three-dimensional information about the measured object included in the photographic subject by using the determined geometric equation.

Description

Measuring device and measuring method

The present invention relates to an apparatus and method for measuring an object, and more particularly to an apparatus and method for measuring using an image.

In recent years, image measurement for acquiring two-dimensional information or three-dimensional information of an object based on an image acquired by an imaging apparatus has been performed. For example, Patent Document 1 describes that in a robot system, a plane such as a floor is extracted based on a point obtained by sampling three-dimensional data obtained from a stereo image, and an obstacle is detected based on the extracted plane. ing. Further, for example, in Patent Document 2, in a vehicle environment recognition device that recognizes an environment outside the host vehicle, a specific object is specified based on parallax obtained by pattern matching of a stereo image, and a tracking target is determined based on the specified result. The selection is described.

JP 2008-9999 A JP 2013-178664 A

However, in Patent Document 1, since the three-dimensional data is obtained by matching the left and right stereo images, it takes time to acquire the three-dimensional data for the entire image. Also, since the sampling data consists of one reference point and two other points existing within a predetermined distance from the reference point, if the image includes a plurality of planes, the points belonging to different planes are sampled. There is a fear. Further, in Patent Document 2, since blocks are grouped by relative distance in specifying a specific object, grouping can be performed only in units of “specific objects”, and it is difficult to specify a “surface” to which a measurement target belongs. As described above, the conventional technology cannot calculate the two-dimensional information or the three-dimensional information to be measured at high speed and with high accuracy.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide a measuring apparatus and a measuring method capable of calculating two-dimensional information or three-dimensional information to be measured at high speed and with high accuracy.

In order to achieve the above-described object, the measurement apparatus according to the first aspect of the present invention includes an image acquisition unit that acquires a plurality of images obtained by photographing the same subject from a plurality of viewpoints, and a plurality of images. A reduced image generation unit that generates a plurality of reduced images by reducing each, and a geometric region in the plurality of reduced images based on the parallax obtained by block matching the plurality of reduced images and the pixel positions of the plurality of reduced images. A second geometrical region extraction unit for extracting the first geometrical region and a second geometrical region corresponding to the first geometrical region in the plurality of images based on the extracted first geometrical region. A geometric equation deciding unit that decides a geometric equation representing the geometric region, and a measuring unit that calculates two-dimensional information or three-dimensional information of a measurement target included in the subject using the decided geometric equation.

In the measurement apparatus according to the first aspect, the first geometric region is extracted based on block matching of the reduced image, and the second geometric region in the image before reduction is represented based on the extracted first geometric region. Since a geometric equation is determined and measurement is performed using this geometric equation, the processing can be performed at high speed by narrowing the block matching range of the image before reduction, and different geometric regions can be separated and identified with high accuracy. Measurement can be performed. As described above, the measurement apparatus according to the first aspect can calculate the two-dimensional information or the three-dimensional information to be measured at high speed and with high accuracy.

In the first aspect, the reduction degree of the reduced image can be one type. In this case, it is not necessary to use a plurality of types of reduced images with different reduction degrees, and processing can be performed at high speed. In the first aspect, examples of the two-dimensional information and the three-dimensional information include the position, length, width, and area of the measurement target, but are not limited to these examples. In the first aspect, the geometric area means an area of a subject belonging to the same plane or curved surface and represented by the same geometric equation. One or more arbitrary geometric regions may exist for the same subject.

In the measurement device according to the second aspect, in the first aspect, the geometric region extraction unit determines a geometric equation representing the first geometric region, and the first geometric equation representing the first geometric region is determined based on the first geometric equation representing the first geometric region. Extract geometric regions. The second mode defines an example of a method for extracting the first geometric region from the reduced image.

In the measurement device according to the third aspect, in the first or second aspect, the geometric region extraction unit extracts a pixel whose distance from the first geometric region is equal to or less than a threshold as a pixel belonging to the first geometric region. . The third aspect shows a reference for extracting pixels belonging to the first geometric region.

In any one of the first to third aspects, the measurement device according to the fourth aspect calculates the parallax for a plurality of representative points extracted from the second geometric region, and the pixel positions and the plurality of representative points are calculated. A geometric equation representing the second geometric region is determined based on the parallax calculated for the representative point. According to the fourth aspect, the representative point is extracted and the parallax is calculated for the image before reduction, and the geometric equation is determined based on the extracted representative point. Therefore, the two-dimensional information or the three-dimensional information to be measured is calculated with high accuracy. can do. Note that the number of representative points to be extracted in the fourth mode may differ depending on the type of geometric region (a flat surface or a curved surface, or a curved surface if a curved surface).

In the measurement apparatus according to the fifth aspect, in the fourth aspect, the geometric equation determination unit sets a block matching range of a plurality of images based on a result of block matching of a plurality of reduced images, and a plurality of pieces within the set range The parallax of a plurality of representative points is obtained by performing block matching of the images. According to the fifth aspect, since the block matching range of a plurality of images (images before reduction) is set based on the block matching result of the reduced images, the parallax of the representative points can be accurately obtained using the images before reduction. Matching can be performed at high speed while obtaining, and the two-dimensional information or three-dimensional information of the measurement target can be calculated at high speed and with high accuracy.

In any one of the first to fifth aspects, the measurement device according to the sixth aspect determines whether the pixel indicating the measurement object belongs to which geometric area in the second geometric area, Two-dimensional information or three-dimensional information is calculated based on the geometric equation representing the determined geometric region and the pixel position of the pixel indicating the measurement target. As described above, since the measurement apparatus according to the present invention obtains the geometric equation for the second geometric region, in the sixth aspect, it is determined to which geometric region the measurement target belongs in the second geometric region. Above, 2D information or 3D information is calculated.

The measurement device according to a seventh aspect is the sixth aspect, wherein the measurement unit calculates the parallax of the pixel indicating the measurement target based on the geometric equation indicating the determined geometric area and the pixel position of the pixel indicating the measurement target, Two-dimensional information or three-dimensional information is calculated based on the calculated parallax. In the seventh aspect, two-dimensional information or three-dimensional information is calculated based on the parallax of the pixel indicating the measurement target, and such a parallax calculation method is specified.

The measurement device according to an eighth aspect is any one of the first to seventh aspects, wherein the geometric region extraction unit performs image processing for converting a plurality of images into a grayscale image and image processing for parallelizing the plurality of images. At least one image processing is performed, and a plurality of reduced images are generated for the image subjected to at least one image processing. The eighth mode prescribes the content of so-called “preprocessing”. By generating a reduced image for an image that has undergone such image processing, the two-dimensional information or the three-dimensional information to be measured can be processed at high speed. In addition, it can be calculated with high accuracy.

In the measurement apparatus according to the ninth aspect, in any one of the first to eighth aspects, the geometric equation determination unit determines a geometric equation representing one of a plane, a cylindrical surface, and a spherical surface. The ninth aspect prescribes that a geometric equation representing any one of a plane, a cylindrical surface, and a spherical surface, which is typical as a surface constituting the subject, is determined.

In any one of the first to ninth aspects, the measurement device according to the tenth aspect is a concrete structure and the measurement target is damage to the concrete structure. The concrete structure is damaged, and the shape and size of the generated damage change over time. The measuring device according to the tenth aspect is applied to measure the damage of such a concrete structure. By doing so, it is possible to calculate the two-dimensional information or three-dimensional information of the measurement target (ie, damage) at high speed and with high accuracy. Examples of concrete structures include bridges, tunnels, roads, and buildings. Examples of damage include cracks and free lime. Concrete structures to which the tenth aspect can be applied and Damage is not limited to these examples.

In any one of the first to tenth aspects, the measurement apparatus according to the eleventh aspect includes an optical system that acquires a plurality of images. The optical system included in the eleventh aspect can be a stereo optical system including a plurality of optical systems including a photographic lens and an imaging element corresponding to each of a plurality of viewpoints.

In order to achieve the above-described object, a measurement method according to a twelfth aspect of the present invention includes an image acquisition step of acquiring a plurality of images obtained by photographing the same subject from a plurality of viewpoints, and a plurality of images respectively. Based on the reduced image generation step of generating a plurality of reduced images by reduction, the parallax obtained by block matching the plurality of reduced images, and the pixel positions of the plurality of reduced images, the geometric region in the plurality of reduced images A geometric region extracting step for extracting a certain first geometric region, and a second geometric region corresponding to the first geometric region in the plurality of images based on the extracted first geometric region. A geometric equation determining step for determining a geometric equation representing a geometric region, and a measuring step for calculating two-dimensional information or three-dimensional information of a measurement target included in the subject using the determined geometric equation.In the measurement method according to the twelfth aspect, the two-dimensional information or the three-dimensional information to be measured can be calculated at high speed and with high accuracy, as in the first aspect.

As described above, according to the measurement apparatus and the measurement method of the present invention, it is possible to calculate the two-dimensional information or the three-dimensional information of the measurement target at high speed and with high accuracy.

FIG. 1 is a diagram showing a bridge which is an example of an application target of the measurement apparatus and measurement method of the present invention. FIG. 2 is a block diagram showing the configuration of the measuring apparatus according to one embodiment of the present invention. FIG. 3 is a flowchart showing processing of the measurement method according to the embodiment of the present invention. FIG. 4 is a conceptual diagram illustrating a state in which a vertical shift exists between the left and right images. FIG. 5 is a conceptual diagram showing how the left and right images are parallelized. FIG. 6 is a diagram showing left and right reduced images. FIG. 7 is a diagram illustrating a state in which the reduced image is subjected to block matching. FIG. 8 is a diagram illustrating a geometric region extracted by block matching. FIG. 9 is a diagram illustrating an example of representative points extracted in the geometric region of the reduced image. FIG. 10 is a diagram illustrating an example of representative points extracted from an image before reduction. FIG. 11 is a diagram illustrating how a crack is measured.

Hereinafter, embodiments of a measurement apparatus and a measurement method according to the present invention will be described with reference to the accompanying drawings.

FIG. 1 is a perspective view showing a structure of a bridge 1 (concrete structure) which is an example of an application target of a measuring apparatus and a measuring method according to the present invention. The bridge 1 shown in FIG. 1 has a main girder 3, and the main girder 3 is joined by a joint 3A. The main girder 3 is a member that is passed between the abutment and the pier and supports the load of the vehicle on the floor slab 2. In addition, a floor slab 2 for driving a vehicle or the like is placed on the main girder 3. The floor slab 2 is generally made of reinforced concrete. The bridge 1 has members such as a horizontal girder, a tilted frame, and a horizontal frame (not shown) in addition to the floor slab 2 and the main girder 3.

<Acquisition of image>
When measuring damage to the bridge 1, the inspector uses the digital camera 104 (see FIG. 2) to photograph the bridge 1 from below (direction C in FIG. 1) and obtain an image of the inspection range. The photographing is performed while appropriately moving in the extending direction of the bridge 1 (A direction in FIG. 1) and the orthogonal direction (B direction in FIG. 1). If it is difficult for the inspector to move due to the surrounding situation of the bridge 1, the digital camera 104 may be installed on a movable body that can move along the bridge 1 to perform imaging. Such a moving body may be provided with a lifting mechanism and / or a pan / tilt mechanism of the digital camera 104. Examples of the moving body include a vehicle, a robot, and a flying body, but are not limited to these.

<Configuration of measuring device>
FIG. 2 is a block diagram illustrating a schematic configuration of the measurement apparatus 100 according to the present embodiment. The measurement apparatus 100 includes an image acquisition unit 102, an image processing unit 108 (a reduced image generation unit, a geometric region extraction unit, a representative point extraction unit, a parallax calculation unit, a geometric equation determination unit, a determination unit, a measurement unit), a recording unit 110, A display unit 112 and an operation unit 114 are provided and connected to each other so that necessary information can be transmitted and received.

The function of each unit can be realized by a control device such as a CPU (Central Processing Unit) executing a program stored in the memory. The image input unit 106 includes a wireless communication antenna and an input / output interface circuit, and the recording unit 110 includes a non-temporary recording medium such as an HDD (Hard Disk Drive). The display unit 112 includes a display device such as a liquid crystal display, and the operation unit 114 includes an input device such as a keyboard. These show one example of the configuration of the measuring apparatus according to the present invention, and other configurations can be adopted as appropriate.

As described above, an image photographed using the digital camera 104 is input to the image input unit 106 by wireless communication, and measurement processing (described later) is performed by the image processing unit 108. The digital camera 104 and the image input unit 106 constitute an image acquisition unit 102. The digital camera 104 includes a left image optical system 104L for acquiring a left eye image and a right image optical system 104R for acquiring a right eye image. You can shoot from the viewpoint. The left image optical system 104L and the right image optical system 104R include a photographic lens and an image sensor (not shown). Examples of the image sensor include a CCD (Charge-Coupled Device) type image sensor and a CMOS (Complementary Metal-Oxide Semiconductor) type image sensor. An R (red), G (green), or B (blue) color filter is provided on the light receiving surface of the image sensor, and a color image of the subject can be acquired based on the signals of each color.

<Measurement procedure>
Next, measurement of a subject using the measurement apparatus 100 having the above-described configuration will be described. FIG. 3 is a flowchart showing the procedure of the measurement process according to this embodiment. In addition, this embodiment demonstrates the case where the crack which arose in the floor slab 2 of the bridge 1 which is a concrete structure is measured.

<Image acquisition>
First, the stereo image of the bridge 1 photographed by the digital camera 104 as described above is input to the image input unit 106 by wireless communication (step S100; image acquisition step). A plurality of images of the bridge 1 are input in accordance with the inspection range, and information on photographing date and time is added to the input image by the digital camera 104. Note that the shooting date and time of the input image does not necessarily have to be the same for all images, and may be for a plurality of days. As the image, a plurality of images may be input at a time, or one image may be input at a time. The image of the bridge 1 may be input via a non-temporary recording medium such as various memory cards instead of wireless communication, or image data that has already been captured and recorded is input via a network. Also good.

FIG. 4 shows examples of the left image iL0 and the right image iR0 input in this way. FIG. 4 shows an example of an image when a portion (corner portion) where three planes intersect in the bridge 1 is photographed. The three planes intersect at boundary lines E1, E2, and E3, and these boundary lines E1, E2, and E3 coincide at a point E0. In the left image iL0 and the right image iR0, the horizontal direction in FIG. 4 is the horizontal direction u, and the vertical direction is the vertical direction v. The number of channels and the image size of the left image iL0 and the right image iR0 are not particularly limited. For example, a color image (3 channels) of 4,800 pixels (horizontal direction u) × 3,200 pixels (vertical direction v) is used. ). The left image iL0 and the right image iR0 are images obtained by editing and / or combining a plurality of images (for example, an image showing the entire measurement range generated by combining images obtained by capturing a part of the measurement range). Also good.

<Pretreatment>
In the present embodiment, it is assumed that the left image iL0 and the right image iR0 are color images and are shifted by a distance δ in the vertical direction v as shown in FIG. Therefore, the image processing unit 108 converts the left image iL0 and the right image iR0 into grayscale images in step S110 prior to generation of a reduced image described later. In step S110, the image processing unit 108 shifts the left image iL0 and / or the right image iR0 in the vertical direction v to correct (parallelize) the shift of the distance δ described above. FIG. 5 shows examples of images (left image iL1 and right image iR1) obtained by performing these preprocessing. In the measurement apparatus 100 and the measurement method according to the present embodiment, highly accurate measurement can be performed by performing such preprocessing.

<Reduced image generation>
Next, the image processing unit 108 generates a reduced image based on the left image iL1 and the right image iR1 that are images after the preprocessing (step S120; reduced image generation step). The degree of reduction is not particularly limited. For example, the horizontal direction u and the vertical direction v of the left image iL1 and the right image iR1 are respectively reduced to 1/16 to obtain 300 pixels (horizontal direction u) × 200 pixels (vertical direction v ) Can be generated. Let this reduced image be the left image iL2 and the right image iR2 (see FIG. 6). Note that one type of reduced image may be generated in step S120. That is, it is not necessary to generate a plurality of images having different reduction degrees (for example, 1/8, 1/4, 1/2, etc.).

<Geometric region extraction>
Next, the image processing unit 108 extracts the geometric region (first geometric region) of the bridge 1 based on the left image iL2 and the right image iR2 (step S130; geometric region extraction step). The processing in step S130 includes calculation of parallax by block matching, determination of a geometric equation based on the pixel position and parallax, and extraction of a geometric region based on the determined geometric equation. Hereinafter, these processes will be described.

<Block matching>
Block matching sets a block including a plurality of pixels in one image (reference image) of the left image iL2 and the right image iR2, and sets a block having the same shape and size as the block in the other image (comparison image) Then, the block in the comparative image is moved in the horizontal direction u pixel by pixel, and the correlation value for the two blocks is calculated at each position. In the example of FIG. 7, the reference image is the left image iL2, the comparison image is the right image iR2, and the block AR having the same shape and size as the block AL set in the left image iL2 is moved in the horizontal direction u pixel by pixel. Since the left and right images are parallelized in the preprocessing in step S110, if the position of the block AL is determined, it is only necessary to move the block AR in the horizontal direction u at the time of block matching. When the left and right images are not parallelized in the preprocessing, in the block matching, the movement in the horizontal direction u is repeated while shifting the position in the vertical direction v.

In this way, the correlation value is calculated while moving the block AR, and when the position of the block AR having the lowest correlation value (that is, the highest correlation) with respect to the position of the block AL is specified, the target pixel in the block AL The distance between (for example, the center pixel) and the corresponding pixel (for example, the center pixel) of the block AR at the specified position is calculated as the parallax. Then, such processing is executed for all the pixels of the left image iL2, which is the reference image, and parallax is obtained for each pixel position to generate a parallax image. In the calculation of the parallax, examples of the correlation value calculation method include SAD (SumSof Absolute Difference), SSD (Sum of Squared intensity Difference), and NCC (Normalized Cross Correlation).

<Determining geometric equations and extracting geometric regions>
In the present embodiment, the image processing unit 108 performs plane extraction (extraction of the first geometric region) using a RANSAC (RANdom SAmple Consensus) algorithm. The RANSAC algorithm can obtain an optimum evaluation value by calculating a model parameter (a parameter representing a plane) using randomly sampled data (three if it is a plane) and evaluating the correctness of the calculated parameter. It is an algorithm that repeats until. A specific procedure will be described below.

(Step S1)
Three points are randomly extracted from the parallax image created by the block matching described above. For example, it is assumed that points f1 (u ₁ , v ₁ , w ₁ ), f ₂ (u ₂ , v ₂ , w ₂ ), and f 3 (u ₃ , v ₃ , w ₃ ) are extracted in FIG. The points to be extracted here are points for determining the geometric equation of each geometric region, and the number of points to be extracted is changed depending on the assumed geometric region type (plane, cylindrical surface, spherical surface, etc.). Good. For example, in the case of a plane, representative points of 3 points or more (assuming that they are not on the same straight line) are extracted. The horizontal coordinate of the image is represented by u _i , the vertical coordinate is represented by v _i , and the parallax (distance direction) is represented by w _i (i is an integer of 1 or more representing a point number).

(Step S2)
Next, a plane equation (geometric equation) is determined from the extracted points f1, f2, and f3. The plane equation F in the three-dimensional space (u, v, w) is generally expressed by the following (Expression 1) (a, b, c, d are constants).

F = a * u + b * v + c * w + d (Formula 1)
(Step S3)
For all the pixels (u _i , v _i , w _i ) of the parallax image, the distance to the plane represented by the plane equation F in (Expression 1) is calculated. If the distance is less than or equal to the threshold, it is determined that the pixel is on the plane represented by the plane equation F.

(Step S4)
If the number of pixels existing on the plane represented by the plane equation F is larger than the number of pixels for the current optimal solution, the plane equation F is determined as the optimal solution.

(Step S5)
Steps S1 to S4 are repeated a predetermined number of times.

(Step S6)
One plane is determined by using the obtained plane equation as a solution.

(Step S7)
The pixels on the plane determined up to step S6 are excluded from the processing target (plane extraction target).

(Step S8)
Steps S1 to S7 are repeated, and the process ends when the number of extracted planes exceeds a certain number or the number of remaining pixels is less than a specified number.

The geometric region (first geometric region) can be extracted from the reduced image by the above procedure. In the example of FIG. 8, it is assumed that three geometric regions (planes) G1, G2, and G3 have been extracted. In the measurement apparatus 100 and the measurement method according to the present embodiment, different geometric regions can be separated and identified as described above, and measurement can be performed with high accuracy.

Here, the geometric equation (plane equation) when the geometric region is a plane has been described, but a geometric equation representing another type of geometric region, such as a cylindrical surface (cylindrical surface) or a spherical surface, is determined according to the shape of the subject. You may make it do. This is because the shape of a structure such as a bridge pier or a tunnel is often expressed not only by a plane but also by a cylindrical surface or a spherical surface. Specifically, a cylinder whose center axis is the z-axis and whose radius is r is expressed by the following (formula 2) (z is an arbitrary value), and whose center is the origin of the coordinate system and whose radius is r is the following: (Equation 3).

x ² + y ² = r ² (Formula 2)
x ² + y ² + z ² = r ² (Formula 3)
<Extraction of representative points and parallax of representative points>
Although the reduced images are grouped for each geometric region by the processing up to step S130, in order to perform measurement with high accuracy, a plane equation is obtained for the image before reduction by the following procedure.

First, the image processing unit 108 extracts a plurality of points (representative points) belonging to the same geometric area from the image before reduction (step S140). In the example of FIG. 10, representative points f4, f5, and f6 are extracted from the geometric area G1A (second geometric area) corresponding to the above-described geometric area G1 in the left image iL1 that is an image before reduction. In FIG. 10, geometric regions G1A, G2A, and G3A (second geometric region) correspond to the above-described geometric regions G1, G2, and G3 (first geometric region), respectively.

Next, the parallax is calculated for the extracted representative points f4, f5, and f6 (step S150). Since the parallax is calculated by the block matching of the reduced image as described above, the block matching range between the left image iL1 and the right image iR1 that are the images before the reduction is set based on the result of the block matching of the reduced image. Then, block matching of the left image iL1 and the right image iR1 is performed within the set range. Note that the block matching in step S150 may be performed for the extracted representative points, and it is not necessary to target all the pixels. As a result, the parallax can be calculated with high accuracy for the image before reduction while performing high-speed processing by narrowing down the target and range of block matching.

<Determination of geometric equation>
Then, the image processing unit 108 estimates a plane equation based on the pixel position of the representative point and the parallax calculated in step S150 (step S160; geometric equation determination step). This process can be performed in the same manner as the plane equation estimation for a reduced image. When the plane equation is determined for one geometric region in this way, the processing of steps S140 to S160 is repeated for all geometric regions (planes) to obtain the plane equation. In steps S140 to S160, extraction of representative points and estimation of a plane equation may be repeated by the RANSAC algorithm as described for the reduced image processing.

<Damage extraction>
In this embodiment, since it is assumed that damage (cracking) of the bridge 1 is measured, cracks are extracted from an image (for example, the left image iL1 or the right image iR1) (step S170). Crack extraction can be performed by various methods. For example, a crack detection method described in Japanese Patent No. 4006007 can be used. This method calculates wavelet coefficients corresponding to the two concentrations to be compared, calculates each wavelet coefficient when each of the two concentrations is changed, creates a wavelet coefficient table, and detects the crack detection target. A wavelet image is created by wavelet transforming an input image of a concrete surface. In the wavelet coefficient table, the average density of neighboring pixels in the local area and the wavelet coefficient corresponding to the density of the target pixel are used as threshold values. A crack detection method comprising a step of determining a crack area and a non-crack area by comparing the wavelet coefficient of the target pixel and the threshold value.

Crack extraction can also be performed using the method described in Non-Patent Document 1 below. In the method described in Non-Patent Document 1, an area composed of pixels having a luminance value less than the threshold value is set as a percolated area (percolation area), and the threshold value is sequentially updated according to the shape of the percolation area. The crack is detected from the surface image. Note that the percolation method is a method in which the region is sequentially enlarged in general imitating water penetration (percolation) in nature.

[Non-Patent Document 1] Tomoyuki Yamaguchi, “A Study on Image Processing Method for Crack Inspection of Real Concreate Surfaces”, MAJOR IN PURE AND APPLIED PHYSICS, GRADUATE SCHOOL OF SCIENCE AND ENGINEERING, WASEDA UNIVERSITY, February 2008 Describes the mode of extracting cracks after the determination of the geometric equation. The extraction of cracks is performed in parallel with the processing from the preprocessing (step S110) to the determination of the geometric equation (step S160), or in these processes. It may be performed in advance. An example in the case where cracked Cr is extracted is shown in FIG. The crack Cr is a crack from the end point P1 to the end point P2.

<Geometry area determination>
In order to measure the crack Cr, first, a geometric region to which the crack Cr belongs is determined (step S180; measurement process). In this embodiment, since the geometric region is extracted by the processing up to step S160, the relationship between the pixel position and the geometric region is specified, and the image processing unit 108 determines the geometric region to which the crack Cr belongs based on this relationship. Can be determined. In the example of FIG. 11, the crack Cr is determined to belong to the geometric region G1A. Further, an image showing the crack Cr (for example, the left image iL1 or the right image iR1) may be displayed on the display unit 112, and the geometric region may be determined based on the result input by the user via the operation unit 114. Specifically, in this case, the user designates the end points P1 and P2 of the cracked Cr to be measured on the display screen of the display unit 112 with a pointing device such as a mouse, and the image processing unit 108 determines that “the end points P1 and P2 are geometric. It belongs to area | region G1A ".

<Measurement of damage>
When the geometric region to which the crack Cr belongs is determined in step S180, the two-dimensional information or three-dimensional information of the crack Cr is calculated using the geometric equation (plane equation) determined in the processing up to step S160 (step S190; measurement). Process). Examples of two-dimensional information or three-dimensional information can include position, length, width, and area, but items to be calculated are not limited to these examples, and depend on the nature of the subject and the measurement target. Thus, other items such as volume and cross-sectional area may be calculated.

When calculating the length of the crack Cr, the parallax at the end points P1 and P2 is calculated based on the pixel positions of the end points P1 and P2 and the plane equation of the geometric region G1A. Specifically, the parallax (corresponding to the w coordinate) of the end points P1 and P2 can be obtained from the (u, v) coordinates of the end points P1 and P2 and the plane equation of the geometric region G1A without performing block matching. Then, if necessary, the (u, v, w) coordinates of the end points P1, P2 are converted into (x, y, z) coordinates in the real space based on the position and shooting direction of the digital camera 104, and the following (formula 4 ) To obtain the length L (distance between the end points P1 and P2). If the crack Cr is a curved crack, the crack Cr is divided into a plurality of sections so that each section can be regarded as a straight line, and the length of each section is calculated to obtain the length of the crack Cr. Can do.

L = {(x ₁ −x ₂ ) ² + (y ₁ −y ₂ ) ² + (z ₁ −z ₂ ) ² } ^1/2 (Formula 4)
However, the coordinates in the real space of the end points P1 and P2 are (x ₁ , y ₁ , z ₁ ) and (x ₂ , y ₂ , z ₂ ), respectively.

As described above, in the measurement apparatus 100 and the measurement method according to the present embodiment, the same geometric region is extracted based on the block matching of the reduced image, so that the processing can be performed at a high speed, and different geometric regions can thereby be obtained. And can be measured with high accuracy. In addition, it is possible to calculate parallax by performing block matching in the range set based on block matching of the reduced image for the representative points extracted from the image before reduction, and to determine the geometric equation based on the calculated parallax, thereby achieving high speed In addition, processing can be performed with high accuracy. As described above, according to the measuring apparatus 100 and the measuring method according to the present embodiment, the two-dimensional information or the three-dimensional information of the measurement target can be calculated at high speed and with high accuracy.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 1 Bridge 2 Floor slab 3 Main girder 3A Joint part 100 Measuring apparatus 102 Image acquisition part 104 Digital camera 104L Left image optical system 104R Right image optical system 106 Image input part 108 Image processing part 110 Recording part 112 Display part 114 Operation part AL block AR block E1 boundary line E2 boundary line F plane equation G1 geometric region G1A geometric region G2 geometric region G2A geometric region G3 geometric region G3A geometric region u horizontal direction P1 end point P2 end point S1 to S8 perform geometric equation determination and geometric region extraction Steps S100 to S190 Steps constituting the measuring method v Vertical direction f4 Representative point f5 Representative point f6 Representative point iL0 Left image iL1 Left image iL2 Left image iR0 Right image iR1 Right image iR2 Right image δ Distance

Claims

An image acquisition unit for acquiring a plurality of images obtained by photographing the same subject from a plurality of viewpoints;
A reduced image generator that reduces the plurality of images to generate a plurality of reduced images;
A geometric region extraction unit that extracts a first geometric region that is a geometric region in the plurality of reduced images based on parallax obtained by block matching the plurality of reduced images and pixel positions of the plurality of reduced images. When,
Geometric equation determination that determines a geometric equation representing a second geometric region that is a geometric region in the plurality of images and that corresponds to the first geometric region based on the extracted first geometric region. And
A measurement unit that calculates two-dimensional information or three-dimensional information of a measurement object included in the subject using the determined geometric equation;
A measuring device comprising:
The geometric region extraction unit determines a geometric equation representing the first geometric region, and extracts the first geometric region based on the determined geometric equation representing the first geometric region. Measuring device.
The measurement device according to claim 1 or 2, wherein the geometric region extraction unit extracts pixels whose distance from the first geometric region is equal to or less than a threshold as pixels belonging to the first geometric region.
The geometric equation determination unit calculates parallax for a plurality of representative points extracted from the second geometric region, and based on the pixel positions of the plurality of representative points and the parallax calculated for the plurality of representative points. The measuring apparatus according to claim 1, wherein a geometric equation representing two geometric regions is determined.
The geometric equation determination unit sets a block matching range of the plurality of images based on a block matching result of the plurality of reduced images, and performs block matching of the plurality of images within the set range. The measurement apparatus according to claim 4, wherein parallaxes of a plurality of representative points are obtained.
The measurement unit determines which one of the second geometric regions the pixel indicating the measurement object belongs to, and a geometric equation representing the determined geometric region and a pixel position of the pixel indicating the measurement object The measurement apparatus according to claim 1, wherein the two-dimensional information or the three-dimensional information is calculated based on the information.
The measurement unit calculates a parallax of a pixel indicating the measurement target based on a geometric equation representing the determined geometric region and a pixel position of the pixel indicating the measurement target, and the two-dimensional information based on the calculated parallax Or the measuring apparatus of Claim 6 which calculates the said three-dimensional information.
The geometric region extraction unit performs at least one of image processing for converting the plurality of images into a grayscale image and image processing for parallelizing the plurality of images, and performs the at least one image processing. The measurement apparatus according to claim 1, wherein the plurality of reduced images are generated for an image.
The measurement apparatus according to any one of claims 1 to 8, wherein the geometric equation determination unit determines a geometric equation representing any one of a plane, a cylindrical surface, and a spherical surface.
10. The measuring apparatus according to claim 1, wherein the subject is a concrete structure, and the measurement target is damage to the concrete structure.
The measuring apparatus according to claim 1, further comprising an optical system that acquires the plurality of images.
An image acquisition step of acquiring a plurality of images obtained by photographing the same subject from a plurality of viewpoints;
A reduced image generating step of generating a plurality of reduced images by reducing the plurality of images respectively;
A geometric region extraction step of extracting a first geometric region, which is a geometric region in the plurality of reduced images, based on parallax obtained by block matching the plurality of reduced images and pixel positions of the plurality of reduced images. When,
Geometric equation determination that determines a geometric equation representing a second geometric region that is a geometric region in the plurality of images and that corresponds to the first geometric region based on the extracted first geometric region. Process,
A measurement step of calculating two-dimensional information or three-dimensional information of a measurement target included in the subject using the determined geometric equation;
A measurement method comprising: