JP2021139672A - Area measuring device, area measuring method, and program - Google Patents

Abstract

To provide an area measuring device with which it is possible to acquire the actual area of a region on a subject on the basis of a photographed image.SOLUTION: An area measuring device 100 comprises: feature point detection means 105 for detecting the coordinates of feature points of prescribed pattern light from a photographed image of a target subject 111 to which prescribed pattern light including a plurality of feature points is projected; distance acquisition means 103a for acquiring a subject distance at the time the target subject is photographed; coordinate holding means 106 for holding a previously prepared prescribed reference coordinate and the subject distance in association; image conversion means 107 for performing an image conversion process on the photographed image of the target subject on the basis of projection conversion in which the coordinates of feature points detected from the photographed image of the target subject and the reference coordinates corresponding to the target subject on that occasion are used; region detection means 108 for detecting a target region from the photographed image after the image conversion process; and area acquisition means 109 for acquiring the area of the target region from the number of pixels of the target region and the area per pixel that is based on the subject distance.SELECTED DRAWING: Figure 1

Description

The present invention relates to a technique for measuring the area of a subject area.

Patent Document 1 describes a technique in which a projector that projects projected light according to image information onto a screen captures a projected image, calculates the angle information of the screen, and corrects the keystone of the projected image based on the angle information. It is disclosed. In the technique of Patent Document 1, a plurality of regions in the image are used as distance information at the peak position where the contrast value of the predetermined region in the image in which the projected image is continuously captured while moving the optical system in the optical axis direction is maximized. And calculate the screen angle information based on the distance information.

Japanese Unexamined Patent Publication No. 2004-208089

By the way, in recent years, it has been desired to obtain the actual area of the area on the subject based on the captured image. In the case of the technique of Patent Document 1 described above, it is possible to calculate the distance to the area on the subject and the angle of the subject, but it is not assumed at all that the actual area of the area is obtained.

Therefore, an object of the present invention is to make it possible to acquire the actual area of the area on the subject based on the captured image.

The area measuring device of the present invention is a feature inspection that detects the coordinates of the feature points of the projected predetermined pattern light from a photographed image of a target subject on which a predetermined pattern light including a plurality of feature points is projected. An output means, a distance acquisition means for acquiring a subject distance when the target subject is photographed, a coordinate holding means for holding a predetermined reference coordinate prepared in advance and a subject distance in association with each other, and the target subject. Based on the projection conversion using the coordinates of the feature point detected from the captured image and the reference coordinates corresponding to the subject distance when the target subject is photographed, the image is converted into the captured image of the target subject. An image conversion means for performing processing, an area detecting means for detecting a target area for area measurement from a captured image of the target subject after performing the image conversion processing, and the target detected by the area detecting means. It is characterized by having an area acquisition means for acquiring the area of the target area from the number of pixels of the region and the area per pixel based on the subject distance when the target subject is photographed.

According to the present invention, it is possible to acquire the actual area of the area on the subject based on the captured image.

It is a figure which shows the system configuration example of the area measuring apparatus. It is a flowchart which shows the rough flow of area measurement processing. It is a flowchart of the feature point coordinate acquisition process. It is a flowchart of area measurement by simultaneous shooting of a projection pattern and a subject. It is a flowchart of area measurement by subject shooting after projection pattern shooting. It is explanatory drawing of area calculation per pixel. It is a figure which shows the example of the line projection pattern. It is a figure which shows the example of the point projection pattern. It is a figure which shows the example of the grid projection pattern. It is a figure which shows the example of the notification screen when the shooting angle is large. It is a figure which shows the example of the notification screen when a feature point touches a subject. It is a figure which shows the example of the notification screen when a feature point is away from a subject.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The configuration shown in the following embodiments is only an example, and the present invention is not limited to the illustrated configuration.
FIG. 1 is a diagram showing a configuration example of the entire system including the area measuring device 100 according to the present embodiment. The area measuring device 100 of the present embodiment is a device having a function of measuring the actual area of a desired area (hereinafter referred to as a target area 112) on a subject to be measured (hereinafter referred to as a target subject 111). be. The area measuring device 100 of the present embodiment includes an imaging unit 101, a projection unit 102, an imaging processing unit 103, a captured image storage unit 104, a feature point detection unit 105, a feature point coordinate holding unit 106, an image conversion unit 107, and an area detection unit. It includes 108, an area acquisition unit 109, and a display unit 110.

The projection unit 102 has a function of projecting a predetermined pattern light (hereinafter referred to as a projection pattern) including four or more optically decomposable feature points onto a desired projection area within the projectable range. It is a device.

In the present embodiment, as the feature points that can be optically decomposed, for example, a specific color (wavelength of light) of the pattern light, a specific illuminance, a specific shape, and the like can be mentioned. In the projection pattern in the present embodiment, at least one of these feature points is controlled (adjusted) for each region or each point region in the pattern. That is, under the control of the projection control unit 103b, the projection unit 102 projects feature points such as a specific color (wavelength of light), a specific illuminance, and a specific shape into each region or point region in the pattern to be projected. It is adjustable for each. Here, for example, when a specific color is used as the feature point, the color of the feature point may be, for example, a color in which the RGB values, which are the three primary colors of light, fall within a predetermined range. Further, when a specific shape is used as the feature point, for example, the specific shape may be, for example, the shape of an intersection portion where straight lines intersect. Of course, these are just examples, and may be other feature points such as color and shape. In the projection pattern projected from the projection unit 102 in the present embodiment, at least one of the feature points such as the specific color, the specific illuminance, and the specific shape described above is located at four or more different positions in the pattern. Each pattern is arranged. Specific examples of the projection pattern in this embodiment will be described in detail later.

Further, in the present embodiment, the projectable range is defined as a range within the projection angle of view of the projection unit 102 and within a distance within which the light of the projection pattern can effectively reach. The projection angle of view of the projection unit 102 may be fixed or variable. When the projected angle of view is variable, the projectable range also changes according to the change in the projected angle of view. In the case of the present embodiment, the target subject 111 is placed within the projectable range of the projection unit 102, and the projection pattern is projected from the projection unit 102 with the target subject 111 as a desired projection area.

The image pickup unit 101 is an image pickup element that photoelectrically converts an image pickup optical system (lens, lens unit) 101b including at least a focus lens and an optical image of a subject or the like imaged by the image pickup optical system 101b into an image pickup signal and outputs the light image. It is an image pickup apparatus having 101a. In the present embodiment, the shooting angle of view of the imaging unit 101 may be fixed or variable, and in any case, the target subject 111 in the state where the above-mentioned projection pattern is projected is projected. It is assumed that the image can be taken including the pattern. Of course, the imaging unit 101 can also image the target subject 111 in a state where the projection pattern is not projected. Then, the imaging unit 101 outputs to the imaging processing unit 103 an imaging signal that captures the target subject 111 in which the projection pattern is projected or an imaging signal that captures the target subject 111 in which the projection pattern is not projected. do.

Further, in the area measuring device 100 of the present embodiment, the imaging optical system 101b of the imaging unit 101 and the projection optical system of the projection unit 102 are installed so that their optical axes are parallel to each other, and also with the imaging direction of the imaging unit 101. It is desirable that the projection direction of the projection unit 102 coincides with that of the projection direction. In the following description, when only the "optical axis" is described, it is assumed that both the optical axis of the imaging optical system 101b and the optical axis of the projection optical system, which are parallel to each other, are included as described above. do.

The image pickup processing unit 103 converts the image pickup signal from the image pickup unit 101 into image data, and sends the image data to the photographed image storage unit 104. Hereinafter, the image data obtained by imaging the target subject 111 in the state where the projection pattern is projected or the projection pattern is not projected by the imaging unit 101 and converting the imaging signal by the imaging processing unit 103 is referred to as the target subject image. do. Further, in the following description, the target subject image obtained by photographing the target subject 111 in the state where the projection pattern is projected will be particularly referred to as “target subject image including the projection pattern”.

Further, in the present embodiment, the image pickup processing unit 103 includes a focus adjustment unit 103a and a projection control unit 103b.
The projection control unit 103b controls the designation, projection start, and projection stop of a desired projection area within the projectable range in the projection unit 102 described above. That is, in the case of the present embodiment, the projection control unit 103b instructs the projection unit 102 to start and stop the projection of the projection pattern described above by designating a desired projection area within the projectable range.

The focus adjustment unit 103a performs autofocus control (focus control) on the target subject 111 by driving and controlling the focus lens of the image pickup optical system 101b of the image pickup unit 101. In the case of the present embodiment, the focus adjustment unit 103a performs autofocus control by the so-called contrast AF method based on the image pickup signal from the image pickup unit 101. In contrast AF, the focus lens position when the contrast value of a predetermined detection region in the image continuously captured by the image sensor 101a is maximized while moving the focus lens of the image pickup optical system 101b in the optical axis direction is set. It is in focus position.

The focus adjusting unit 103a also has a function as a distance measuring unit that calculates the distance from the imaging unit 101 to the target subject 111 (referred to as the target subject distance) when the subject is focused on the target subject 111. In the case of the present embodiment, the distance measuring unit of the focus adjusting unit 103a calculates the target subject distance based on, for example, the focus adjustment amount or the focus lens movement amount in the contrast AF. In the present embodiment, contrast AF has been given as an example of the autofocus control method, but the present invention is not limited to this example, and so-called phase difference AF or other AF methods may be used. In these cases, the distance measuring unit of the focus adjusting unit 103a calculates the target subject distance based on the focus adjustment amount or the focus lens movement amount in the phase difference AF and other AF methods.

The captured image storage unit 104 stores the target subject image obtained by imaging the target subject 111 by the imaging unit 101 in association with the target subject distance information measured by the distance acquisition unit of the focus adjustment unit 103a at the time of imaging. do. Then, the captured image storage unit 104 sends out the saved target subject image and target subject distance information in response to a request from another functional unit of the area measuring device 100. In the present embodiment, the example in which the information of the target subject distance is stored in the captured image storage unit 104 has been given, but if the correspondence with the target subject image can be known, another storage unit (separately provided) may be provided. (Not shown) or may be stored in another functional unit or the like in the area measuring device 100.

The feature point detection unit 105 acquires a target subject image including a projection pattern from the captured image storage unit 104, and detects each of the four or more feature points described above from the projection pattern included in the target subject image. That is, as described above, the feature point detection unit 105 can detect at least one of a specific illuminance, a specific wavelength, and a specific shape controlled for each region in the projection pattern light as a feature point. .. For example, when the feature points have a specific color, the feature point detection unit 105 determines a region in the target subject image including the projection pattern in which a plurality of pixels whose RGB values fall within a predetermined range are adjacent to each other. Detected as a feature point area. Then, the feature point detection unit 105 detects the center coordinates of the detected feature point region as a feature point, and acquires the coordinates of the feature point. The method of obtaining the coordinates of the feature point is not limited to the method of obtaining the center coordinates of the feature point region as described above, and may be obtained by another method. Further, for example, when the feature point has a specific shape and the shape is an intersection portion of intersecting straight lines, the feature point detection unit 105 performs linear approximation from, for example, pixels in the same color region, and the straight line is The intersection part at the time of intersection is detected as a feature point, and the coordinates of the feature point are acquired. The approximation based on pixels in the same color region is not limited to the linear approximation, and may be a multidimensional approximation or the like. The method for detecting feature points is not limited to these examples, and any existing method may be used as long as it can detect feature points from other methods, colors, shapes, and the like. Hereinafter, the coordinates of the feature points detected from the target subject image including the projection pattern will be referred to as "target coordinates".

The feature point coordinate holding unit 106 holds the information of the reference coordinates and the information of the reference subject distance prepared in advance in association with each other. The reference coordinates are the same as those described above, for example, from a plurality of images (referred to as flat subject images) taken by frontal photography of a flat subject on which the same projection pattern as described above is projected while changing the subject distance. These are the coordinates of the feature points detected by the feature point detection process. Further, in the present embodiment, the reference subject distance is the subject distance when a flat subject on which the same projection pattern as described above is projected is photographed by frontal photography while changing the subject distance. Further, in the present embodiment, the reference coordinates may be obtained by a calculation formula of feature point coordinates in which the reference subject distance obtained by the design value and adjustment of the mechanic is a variable. Alternatively, the reference coordinates can be obtained by combining two methods: a method of obtaining the plane subject on which the projection pattern is projected from a photographed plane subject image, and a method of obtaining the reference coordinates by a calculation formula of feature point coordinates using the reference subject distance as a variable. You may be asked. As described above, in the present embodiment, the feature point coordinate holding unit 106 holds the reference coordinates prepared in advance and the reference subject distance in association with each other.

The flat subject used for acquiring the reference coordinates may be a subject having a uniform surface, but more preferably, the target subject has the same color and texture as the target subject 111 and has the same color and texture as the target subject 111. A subject having a uniform surface on which the target area 112 does not exist on 111 is preferable. Further, in the present embodiment, the frontal shooting means shooting in a state where the optical axis of the imaging optical system 101b of the imaging unit 101 is orthogonal to the subject surface. Hereinafter, in the present embodiment, the image obtained by frontal photography will be referred to as a "frontal image". On the other hand, an image taken in a state where the subject surface is not orthogonal to the optical axis of the imaging optical system 101b, that is, in a state where the frontal shooting is not performed, is referred to as an "inclined image".

In the present embodiment, the reference coordinates for each reference subject distance held by the feature point coordinate holding unit 106 are the front shots of the tilted image of the target subject 111 taken in a state other than the front shot, as described later. This is information required when converting to an image similar to an image.

That is, when the area measuring device 100 of the present embodiment measures the area of the target area 112 from the target subject image (tilted image) obtained by shooting the target subject 111 in a state other than the frontal shooting, the target subject image is used. , Converts to an image similar to the front image taken from the front. In the present embodiment, the image conversion process for converting the target subject image, which is an inclined image, into an image as if it was taken from the front is performed by the image conversion unit 107. At this time, the image conversion unit 107 transfers the coordinates of the feature points (target coordinates) detected from the target subject image including the projection pattern and the reference coordinates for each reference subject distance held by the feature point coordinate holding unit 106. , Used as a conversion parameter for image conversion processing.

Therefore, when the area of the target area 112 is measured, the image conversion unit 107 requests the feature point detection unit 105 for the target coordinates detected from the target subject image including the projection pattern. In response to this request, the feature point detection unit 105 acquires a target subject image including a projection pattern from the captured image storage unit 104, and detects the feature points from the target subject image as described above to obtain the target coordinates. Information is sent to the image conversion unit 107.

Further, the image conversion unit 107 at this time requests the feature point coordinate holding unit 106 for the reference coordinates associated with the reference subject distance of the same distance as the target subject distance. Then, the image conversion unit 107 acquires the information of the reference coordinates read from the feature point coordinate holding unit 106 in response to the request. At this time, it is also conceivable that the reference subject distance having the same distance as the target subject distance does not exist in the reference subject distance associated with the reference coordinates held by the feature point coordinate holding unit 106. In this case, the image conversion unit 107 acquires the reference coordinates corresponding to, for example, the reference subject distance closest to the target subject distance among the reference subject distances held by the feature point coordinate holding unit 106, and obtains the reference coordinates. , May be corrected according to the difference between the reference subject distance and the target subject distance.

Further, the image conversion unit 107 requests the captured image storage unit 104 of the target subject image to be converted into the front image by the image conversion process and the target subject distance associated with the image. Then, the image conversion unit 107 acquires the target subject image and the target subject distance information read from the captured image storage unit 104 in response to the request. At this time, the target subject image acquired by the image conversion unit 107 from the captured image storage unit 104 may be a target subject image including a projection pattern, or a target subject image obtained by photographing the target subject 111 in a state where the projection pattern is not projected. good. That is, the image to be image-converted to the front image in the image conversion unit 107 does not necessarily have to be an image including a projection pattern, and may be a target subject image that does not include a projection pattern.

Then, the image conversion unit 107 creates an image in which the target subject image, which is an inclined image, is taken in front based on the target coordinates and the reference coordinates acquired as described above as conversion parameters used in the image conversion process. Performs image conversion processing to be converted.

The image conversion process using the target coordinates and the reference coordinates as conversion parameters will be described in more detail. In the following description, an example will be given in which the coordinates of the feature points detected from the plane subject image obtained by photographing the plane subject in the state where the projection pattern is projected are used as the reference coordinates. Here, the X and Y coordinates (that is, reference coordinates) of the four feature points detected from the plane subject image obtained by photographing the plane subject in the state where the projection pattern is projected are set to (X ₁ , Y ₁ ) and (X, respectively). ₂ , Y ₂ ), (X ₃ , Y ₃ ), (X ₄ , Y ₄ ). On the other hand, the x, y coordinates (that is, the target coordinates) of the four feature points detected from the target subject image obtained by capturing the target subject 111 with the projection parameters projected are (x ₁ , y ₁ ) and (x, respectively). ₂ , y ₂ ), (x ₃ , y ₃ ), (x ₄ , y ₄ ). Further, the x and y coordinates of each pixel i of the target subject image are set to x _i and y _i, and the x and y coordinates of each pixel i of the converted subject image after the target subject image is converted to the front image are X _i. , Y _i .

The image conversion unit 107 associates the target coordinates corresponding to the four feature points with the reference coordinates corresponding to the four feature points by pairing the coordinates close to each other. Then, the image conversion unit 107 converts the target subject image into a front image by substituting the four pairs of coordinates, which are paired with each other, into the conversion formulas (1) and (2) below. Acquire the rear subject image. Equation (1) is a conversion equation for projective transformation, and a ₁ , a ₂ , a ₃ , ..., a ₇ and a ₈ in equations (1) and (2) are coefficients of the transformation equation. Is.

The image conversion unit 107 converts the target subject image, which is an inclined image, into an image as if it was taken in front by performing image conversion processing based on the conversion formulas of the formulas (1) and (2). Can be done. As the association between each target coordinate corresponding to each of the four feature points and each reference coordinate, for example, a method of associating each coordinate in the clockwise order of upper left → upper right → lower right → lower left may be used. Then, as described above, the image conversion unit 107 sends the converted subject image after the image conversion processing to the area detection unit 108 and the area acquisition unit 109.

The area detection unit 108 detects the target area 112 of the target subject 111 as the target area for area measurement (hereinafter referred to as the measurement target area) from the converted subject image by the image conversion unit 107. In the present embodiment, the area detection unit 108 performs semantic area division by, for example, deep learning when detecting the measurement target area from the converted subject image. That is, in the present embodiment, a learning computer (not shown) trains a neural network model using images of a plurality of subjects as teacher data in advance to generate a trained model. The area detection unit 108 estimates the measurement target area from the converted subject image by the image conversion unit 107 based on the generated learned model. As an example of the neural network model, a complete convolutional network FCN (Full Convolutional Network), which is a segmentation model using deep learning, can be applied. In addition, the region division may be realized by using another deep learning model. Further, the segmentation method is not limited to deep learning, and for example, graph cut, region growth, edge detection, governing division method, and the like may be used. The image used for region division is not limited to the image after being converted like the front image by the image conversion unit 107, and may be an inclined image taken in a state where the target subject 111 is not front shot. ..

The information of the measurement target area (for example, the coordinate information of each pixel included in the measurement target area) detected by the area detection unit 108 as described above is sent to the area acquisition unit 109.
The area acquisition unit 109 is obtained based on the coordinate information of each pixel in the measurement target area, the target subject distance of the target subject, the known sensor size of the image sensor 101a, and the known shooting angle of view of the image pickup optical system 101b. The area of the measurement target area is calculated from the actual area per pixel. When the imaging optical system 101b has a zoom optical system and the shooting angle of view is variable, the shooting angle of view can be obtained based on, for example, the zoom amount. A method for obtaining the actual area per pixel based on the sensor size and the shooting angle of view will be described with reference to FIG. 5 described later.

The display unit 110 will be described with reference to FIGS. 10, 11 and 12, which will be described later, based on each information of the target subject image, the target coordinates, the target subject distance, the reference coordinates corresponding to the target subject distance, and the measurement target area. Each screen is displayed on the display device. Details of each screen display shown in FIGS. 10, 11, 12 and the like will be described later. When the area of the measurement target area can be measured, the display unit 110 shifts to the measurement target area, the number of pixels of the measurement target area, the actual area per pixel number, the subject distance, the coordinates of the detection feature points, and the front image. It is also possible to display information that can be acquired, such as images before and after conversion of.

FIG. 2 is a flowchart showing a rough flow of the area measurement process in the area measurement device 100 of the present embodiment.
In step S201, the area measuring device 100 holds information on a plurality of reference coordinates and information on a plurality of subject distances (reference subject distances) associated with the reference coordinates in the feature point coordinate holding unit 106. Determine if it has been done. When it is determined that the information is not retained, the area measuring device 100 transitions to step S202, and when it is determined that the information is retained, the area measuring device 100 transitions to step S203.

In the transition to step S202, since the feature point coordinate holding unit 106 does not hold the information in which the reference information and the reference subject distance are associated with each other, the area measuring device 100 corresponds to the reference coordinates in step S202. Performs processing for acquiring the reference subject distance. The details of the process in step S202 will be described with reference to the flowchart of FIG. 3 described later.

When the transition to step S203 occurs, the feature point coordinate holding unit 106 holds the reference coordinates and the subject distance in association with each other. Therefore, the area measuring device 100 increases the area of the target area 112 of the target subject 111 in step S203. Perform measurement processing. The details of the process in step S203 will be described with reference to the flowchart of FIG. 4 or FIG. 5 described later.

FIG. 3 is a flowchart showing the flow of the reference coordinate acquisition process performed in step S202 of FIG. 2 when the information in which the reference coordinates and the reference subject distance are associated is not held in the feature point coordinate holding unit 106. be. In the flowchart of FIG. 3, steps S301 and S302 are operations performed in advance by a user or the like, but are included in the flowchart here for convenience of explanation.

In step S301, the flat subject is installed at a position within the shooting angle of view of the imaging unit 101 and within the projection angle of view of the projection unit 102. At that time, the plane subject is installed so that the plane portion is orthogonal to the optical axis.

Further, in step S302, the distance between the image pickup unit 101 and the flat subject is set within a predetermined range, and any one of a plurality of distances capable of acquiring an image having a predetermined resolution when the image pickup unit 101 captures the image. The distance is adjusted so that it is set to one. The distance setting in step S302 is performed a plurality of times after the processing after step S303, which will be described later, until it is determined that the measurement is completed in step S307, which will be described later. That is, the distance setting in step S302 is performed for each of a plurality of distances within a predetermined range and capable of acquiring an image having a predetermined resolution. In the case of the present embodiment, the plurality of distances between the imaging unit 101 and the flat subject set in step S302 correspond to the plurality of reference subject distances described above.

Next, in step S303, the projection control unit 103b of the image pickup processing unit 103 controls the projection unit 102 to project a predetermined pattern light (projection pattern) onto a flat subject.
Next, in step S304, the image pickup processing unit 103 drives and controls the image pickup optical system 101b of the image pickup unit 101 to focus on the plane subject by the focus adjustment unit 103a, and further focuses the plane subject on the image pickup unit 101 in the focused state. An image including a projection pattern is acquired by imaging the image. In the case of the present embodiment, the image acquired in step S304 corresponds to the plane subject image corresponding to the reference subject distance described above.

Next, in step S305, the feature point detection unit 105 detects feature points from the image acquired in step S304, that is, a flat subject image including a projection pattern, and obtains its coordinates (that is, reference coordinates).
Then, in the next step S306, the feature point coordinate holding unit 106 pairs the coordinates of the feature points detected in step S305 with the subject distance (reference subject distance) at the time of shooting the flat subject image acquired in step S304. Hold. In the case of the present embodiment, the coordinates held in association with the reference subject distance in step S306 correspond to the above-mentioned reference coordinates.

Next, in step S307, the area measuring device 100 has completed the processes from steps S303 to S306 for each of the plurality of distances within the predetermined range set in step S302 and capable of acquiring an image having a predetermined resolution. Is determined. Then, when it is determined in step S307 that the process is not completed, the process returns to step S302, the distance adjustment for setting the unset distance is performed, and the processes after step S303 are performed. On the other hand, when it is determined in step S307 that the process is completed, the process of the flowchart of FIG. 3 ends.

4 and 5 are flowcharts showing the flow of the area measurement process for the target area 112 of the target subject 111, which is performed in step S203 of FIG. FIG. 4 is a flowchart in the case where the target coordinates are acquired from the target subject image including the projection pattern and the area of the target area 112 is also measured. FIG. 5 is a flowchart in the case where the area of the target area 112 is measured from the target subject image in which the target subject 111 in the state where the projection pattern is not projected is measured after the target coordinates are acquired from the target subject image including the projection pattern. be.

First, the flow chart of FIG. 4 will be described in which the target coordinates are acquired from the target subject image including the projection pattern and the area of the target area 112 is also measured.
In step S401, the projection control unit 103b of the image pickup processing unit 103a controls the projection unit 102 to project a predetermined pattern light (projection pattern) onto the target subject 111.

In step S402, the image pickup processing unit 103 drives and controls the image pickup optical system 101b of the image pickup unit 101 by the focus adjustment unit 103a to focus on the target subject 111. Further, the image pickup processing unit 103 acquires the target subject image including the projection pattern by causing the image pickup unit 101 to image the target subject 111 in the focused state.

In step S403, the feature point detection unit 105 detects four feature points of the projection pattern from the image acquired in step S402, that is, the target subject image including the projection pattern, and obtains the coordinates thereof.

In step S404, the image conversion unit 107 acquires the information of the reference coordinates corresponding to the same reference subject distance as the target subject distance when the target subject image is acquired from the feature point coordinate holding unit 106. Then, the image conversion unit 107 has each of the target coordinates corresponding to the four feature points detected from the projection pattern included in the target subject image in step S403 and the four reference coordinates obtained from the feature point coordinate holding unit 106. Associate close coordinates with each other.

In step S405, the image conversion unit 107 has the same as the front image obtained by front-capturing the target subject image, which is an inclined image, by the conversion formula of the projection transformation obtained from the four coordinates associated in step S404. Convert to an image.

In step S406, the area detection unit 108 detects the measurement target area from the converted subject image after the image conversion process in step S405 is performed, and obtains the number of pixels included in the area.

In step S407, the area acquisition unit 109 determines the actual area per pixel from the coordinate information of each pixel in the measurement target area, the target subject distance, the sensor size of the image sensor 101a, and the shooting angle of view of the image pickup optical system 101b. Ask for. Then, the area acquisition unit 109 calculates the area of the measurement target area from the actual area per pixel and the number of pixels obtained in step S406.

Next, after acquiring the target coordinates from the target subject image including the projection pattern, the flow chart of FIG. 5 for measuring the area of the target area 112 from the target reference image obtained by photographing the target subject 111 on which the projection pattern is not projected is shown. explain. In the flowchart of FIG. 5, the difference from FIG. 4 is that the shooting of the target subject is divided into two times, one in which the projection pattern is projected and the other in which the projection pattern is not projected. The point is that a process for stopping the projection of the projection pattern is provided between the two. The processing of the flowchart shown in FIG. 5 may be performed as an alternative processing of the flowchart shown in FIG. 4, or may be switched in the form of mode selection.

In step S501, the projection control unit 103b of the image pickup processing unit 103a controls the projection unit 102 to project the projection pattern onto the target subject 111.

In step S502, the image pickup processing unit 103 drives and controls the image pickup optical system 101b of the image pickup unit 101 by the focus adjustment unit 103a to focus on the target subject 111. Further, the image pickup processing unit 103 acquires the target subject image including the projection pattern by causing the image pickup unit 101 to image the target subject 111 in the focused state.
After this step S502, in step S503, the projection control unit 103b of the imaging processing unit 103a controls the projection unit 102 to stop the projection of the projection pattern.

In step S504, the image pickup processing unit 103 drives and controls the image pickup optical system 101b of the image pickup unit 101 by the focus adjustment unit 103a to focus on the target subject 111. Further, the image pickup processing unit 103 acquires the target subject image in a state in which the projection pattern is not projected by causing the image pickup unit 101 to image the target subject 111 in the focused state. The target subject at the time of shooting in step S504 and the target subject at the time of shooting in step S502 differ only in the state in which the projection pattern is projected and the state in which the projection pattern is not projected. Therefore, the target subject distance at the time of shooting in step S504 and the target subject distance at the time of shooting in step S502 are the same.

In step S505, the feature point detection unit 105 detects four feature points of the projection pattern from the image acquired in step S502, that is, the target subject image including the projection pattern, and obtains the coordinates thereof.

In step S506, the image conversion unit 107 acquires the information of the reference coordinates corresponding to the same reference subject distance as the target subject distance when the target subject image was acquired in step S504 from the feature point coordinate holding unit 106. Then, the image conversion unit 107 associates close coordinates with each other with respect to the target coordinates corresponding to the four feature points detected in step S505 and the four reference coordinates obtained from the feature point coordinate holding unit 106.

In step S507, the image conversion unit 107 appears to have taken a frontal image of the target subject image (tilted image) acquired in step S504 by the conversion formula of the projective transformation obtained from the four coordinates associated in step S506. Convert to an image.

In step S508, the area detection unit 108 detects the measurement target area from the converted subject image after the image conversion process in step S507 is performed, and obtains the number of pixels included in the area.

In step S509, the area acquisition unit 109 per pixel is based on the coordinate information of each pixel in the measurement target area, the target subject distance when the target subject image is acquired in step S504, the sensor size, and the shooting angle of view. Find the actual area. Then, the area acquisition unit 109 calculates the area of the measurement target area from the actual area per pixel and the number of pixels obtained in step S508.

In the flowchart of FIG. 5, after shooting the target subject in the state where the projection pattern is projected in step S502, the target subject in the state where the projection pattern is not projected is shot in step S504, but the order of shooting is The reverse may be true. That is, for example, the target subject in which the projection pattern is not projected may be photographed in step S502, and then the target subject in which the projection pattern is projected may be photographed in step S504.

Next, the area calculation method in the area acquisition unit 109 will be described with reference to FIG.
An imaging device such as the imaging unit 101 can generally be treated as a pinhole model as shown in FIG. The incident light 607 passes through the principal point of the lens 602 and is received by the imaging surface 601 of the image sensor. The focal length F is the distance from the imaging surface 601 to the principal point of the lens 602. Here, when the thin lens approximation is used, it can be considered that the two principal points, the front principal point and the rear principal point, coincide with each other. Further, in the pinhole model, the lens 602 is apparently represented as a single lens having no thickness, but an actual lens such as the imaging optical system 101b has a plurality of thick lenses including a focus lens and a zoom lens. It is composed of the lenses of. Then, by adjusting the position of the focus lens of the lens 602 so as to move in the optical axis direction so that an optical image is formed on the image pickup surface 601 of the image sensor, focusing on the subject 603 is performed. When the focal length F is changed by moving the zoom lens in the optical axis direction, the angle of view θ changes.

At this time, the subject width W geometrically reflected on the focal plane is determined from the relationship between the angle of view θ and the subject distance D, and the subject width W is calculated using a trigonometric function. That is, the subject width W is determined by the relationship between the angle of view θ with the focus position and the zoom amount as parameters and the subject distance D. Then, by dividing the value of the subject width W by the number of pixels on the line of the imaging surface 601 of the corresponding image sensor, the length on the focal plane corresponding to one pixel on the image can be obtained. Then, the area calculation is obtained from the number of pixels of the extraction region (measurement target region) obtained from the above-mentioned region division result by the region detection unit 108 and the length on the focal plane corresponding to one pixel on the image. It can be calculated as the product of the area of one pixel. In the formula for obtaining the subject width W or the length on the focal plane corresponding to one pixel on the image, data is acquired by shooting a subject with a known subject width W by changing the subject distance D. It may be obtained recursively. When the subject distance D is single, it is premised that the subject is a flat surface in order to correctly obtain the area.

Next, the projection pattern by the projection unit 102 will be described with reference to FIGS. 7, 8 and 9.
7 (a) and 7 (b) are diagrams showing an example in the case where the projection pattern is, for example, a line projection pattern composed of quadrangular lines. In FIG. 7A, the line projection pattern 701a is projected from the projection unit 102 onto the front surface of the flat subject, and the line projection pattern 701a on the flat subject is photographed in the front angle of view 701 by the image pickup unit 101. It is a figure which showed the appearance. On the other hand, FIG. 7B shows a line projection pattern 702b within the shooting angle of view 702 of the imaging unit 101 when a quadrangular line projection pattern is projected onto the target subject in a state not orthogonal to the optical axis. It is a figure which showed the appearance of. When the target subject is not orthogonal to the optical axis as shown in FIG. 7B, the quadrangular line projection pattern looks like a trapezoidal line projection pattern 702b, and the circular target area on the target subject. 702b will look like an ellipse. In the feature point detection and coordinate acquisition process in step S305 of FIG. 3, for example, the four corners of the line projection pattern 701a shown in FIG. 7A are detected as feature points, and the coordinates of the feature points at those four corners are acquired as reference coordinates. NS. Further, in the feature point detection and coordinate acquisition process in step S403 of FIG. 4 or step S505 of FIG. 5, the four corners of the line projection pattern 702a shown in FIG. 7B are detected as feature points, and the coordinates of the feature points at those four corners are detected. Is acquired as the target coordinates. Further, in the image conversion process of step S405 of FIG. 4 or step S507 of FIG. 5, the reference coordinates of the four corners of the line projection pattern 701a of FIG. 7A and the target coordinates of the four corners of the line projection pattern 702a of FIG. Image conversion processing is performed based on the correspondence with. Then, in steps S406 to S407 of FIG. 4 or steps S508 to S509 of FIG. 5, the measurement target area is detected from the image after the image conversion process, and the area thereof is calculated.

8 (a) and 8 (b) are diagrams showing an example in the case where the projection pattern is, for example, a point projection pattern composed of four points. The four points of the point projection pattern illustrated in FIG. 8 correspond to, for example, the four corners of the quadrangular line projection pattern shown in FIG. 7, but are not limited to this example. In FIG. 8A, four point projection patterns 801a to 801d are projected from the projection unit 102 onto the front surface of the flat subject, and the image pickup unit 101 takes a frontal image of the point projection patterns 801a to 801d on the flat subject. It is a figure which showed the appearance in the shooting angle of view 801. FIG. 8B shows the point projection pattern within the shooting angle of view 802 of the imaging unit 101 when each point projection pattern is projected onto the tilted target subject, as in the example of FIG. 7B. It is a figure which showed the appearance of 802a to 802d. In the case of FIG. 8B as well as the example of FIG. 7B described above, when the target subject is in an inclined state, the four point projection patterns 802a to 802d appear to be positions corresponding to the four corners of the trapezoid. Further, the circular target area 803 on the target subject looks like an ellipse. In the case of the example of FIG. 8, in the feature point detection and coordinate acquisition process of step S305 of FIG. 3, the four point projection patterns 801a to 801d shown in FIG. 8A are detected as feature points, and their coordinates are obtained. Obtained as reference coordinates. Further, in the feature point detection and coordinate acquisition process of step S403 of FIG. 4 or step S505 of FIG. 5, the four point projection patterns 801a to 801d shown in FIG. 8B are detected as feature points, and their coordinates are targeted. Obtained as coordinates. Subsequent image conversion processing and area measurement processing are the same as described above. When the four point projection patterns illustrated in FIG. 8 are used, the processing of feature point detection becomes easier than in the case of the line projection pattern illustrated in FIG. 7.

9 (a) and 9 (b) are diagrams showing an example in which the projection pattern is, for example, a grid projection pattern in which a plurality of vertical lines and horizontal lines intersect. In FIG. 9A, the grid projection pattern 901 is projected from the projection unit 102 onto the front surface of the flat subject, and the grid projection pattern 901a on the flat subject is photographed in front of the plane subject within the shooting angle of view 901. It is a figure which showed the appearance. FIG. 9B shows the grid projection pattern 902a within the shooting angle of view 902 of the imaging unit 101 when the grid projection pattern is projected onto the tilted target subject, as in the example of FIG. 7B. It is a figure which showed the appearance of. In the case of FIG. 9B, similarly to the example of FIG. 7B described above, when the target subject is in an inclined state, the grid projection pattern looks like a trapezoidal grid projection pattern 902a, and a circle on the target subject. The target area 902b of the above looks like an ellipse. In the case of this example, in the feature point detection and coordinate acquisition process in step S305 of FIG. 3, each grid point where the vertical and horizontal lines of the grid projection pattern 901a shown in FIG. 9A intersect is detected as a feature point, and each of them is detected. The coordinates of the grid points are acquired as reference coordinates. Further, in the feature point detection and coordinate acquisition processing in step S403 of FIG. 4 or step S505 of FIG. 5, each grid point of the grid projection pattern 902a shown in FIG. 9B is detected as a feature point, and the grid points of those grid points are detected. The coordinates are acquired as the target coordinates. In the image conversion process of step S405 of FIG. 4 or step S507 of FIG. 5, the reference coordinates of each grid point of the grid projection pattern 901a of FIG. 9A and each grid point of the grid projection pattern 902a of FIG. 9B Image conversion processing is performed based on the correspondence with the target coordinates of. Subsequent area measurement processing is the same as described above. When the grid projection pattern illustrated in FIG. 9 is used, the image conversion process can be performed with the feature points near the target region 902b within the imaging angle of view of the imaging unit 101. That is, when the grid projection pattern is used, for example, even if the surface of the target subject is not flat but has curves or irregularities, it is possible to convert the image into a front image by using the feature points near the target area 902b. Therefore, the area measurement accuracy of the target area 902b can be improved.

Next, the details of the screen display by the display unit 110 will be described with reference to FIGS. 10, 11, and 12.
In FIG. 10, when it is difficult to measure the area of the target area 1004 because the target subject is not orthogonal to the optical axis and is greatly tilted, the display unit 110 displays a notification text 1002 to the user on the screen 1001. It is a figure which showed the displayed example. In the example of FIG. 10, the line projection pattern 1003 is projected on the target subject in a state of being tilted without being orthogonal to the optical axis, and the image captured by the imaging unit 101 of the line projection pattern 1003 on the target subject is displayed on the display unit 110. Is displayed on the screen 1001. That is, in the case of the example of FIG. 10, since the target subject is tilted without being orthogonal to the optical axis, the quadrangular line projection pattern is the trapezoidal line projection pattern 1003, and the circular target area is the elliptical target area 1004. The state of is displayed.

Here, if the target subject is not orthogonal to the optical axis and is greatly tilted, a part of the target area 1004 may be hidden when viewed from the imaging unit 101, and the entire area of the target area 1004 may not be photographed. .. Then, if a part of the target area 1004 is hidden and the entire area cannot be photographed, the area cannot be accurately measured. Therefore, in the area measuring device 100 of the present embodiment, the display unit 110 may not be able to photograph the entire area of the target area 1004 based on the information of the target coordinates, the target subject distance, and the reference coordinates. Determine if there is. When performing this determination process, the display unit 110 between the target coordinates detected from the line projection pattern 1003 included in the target subject image and the reference coordinates of the reference subject distance corresponding to the target subject distance at the time of shooting the target subject. Find the coordinate difference of. When the coordinate difference between the target coordinates and the reference coordinates exceeds a preset predetermined coordinate difference, the display unit 110 displays the target area 1004 because the target subject is greatly tilted with respect to the optical axis. It is determined that there is a possibility that the entire area of the image cannot be photographed.

When it is determined that the entire area of the target area 1004 may not be photographed as a result of the determination process as described above, the display unit 110 displays the notification text 1002 as shown in FIG. 10 on the screen. do. For example, the display unit 110 indicates that "the shooting angle of the subject is large", which indicates that the entire area of the target area may not be photographed, and "the subject is," which indicates a countermeasure for enabling the entire area of the target area to be photographed. Please adjust so that it comes to the front. ”Is displayed. By viewing this notification text 1002, the user using the area measuring device 100 can appropriately adjust the installation state of the target subject 111 or the shooting angle of the imaging unit 101 of the area measuring device 100. As a result, the area measurement accuracy of the target area can be improved.

In FIG. 11, when the target area 1103 overlaps with the line projection pattern 1103 or is outside the line projection pattern 1103 and it becomes difficult to measure the area of the target area 1104, the display unit 110 displays the notification text 1102 on the screen 1101. It is the figure which showed the example displayed above. In the example of FIG. 11, the target area 1104 of the target subject and a part of the line projection pattern 1103 overlap each other, and the image captured by the imaging unit 101 of the target subject including the line projection pattern 1103 in that state is displayed on the display unit 110. Is displayed on the screen 1101. In the example of FIG. 11, it is assumed that the target subject is tilted without being orthogonal to the optical axis, but it is determined that the entire area of the target region 1104 is not in a state where it cannot be photographed.

In the case of the example of FIG. 11, one of the four corners, which is the feature point of the line projection pattern 1103, hangs on the target area 1104. Here, when the target area 1104 and the feature point of the line projection pattern 1103 overlap, it may be difficult to detect the feature point of the line projection pattern 1103. As described above, when some feature points of the line projection pattern 1103 cannot be detected, the area cannot be accurately measured. Therefore, in the area measuring device 100 of the present embodiment, the display unit 110 fits the target area 1104 within the coordinate range based on the feature points of the line projection pattern 1103 based on the detection result of the target coordinates of the line projection pattern 1104. Determine if it is. That is, when the target area 1104 is within the coordinate range based on the feature points of the line projection pattern 1103, the target area 1104 and the feature points of the line projection pattern 1103 do not overlap. When performing this determination process, the display unit 110 measures the number of detected feature points based on the target coordinates detected from the line projection pattern 1103 included in the target subject image. Then, when the number of the measured feature points is less than the number of feature points to be detected from the line projection pattern 1103 (four corresponding to the four corners), the display unit 110 is based on those four feature points. It is determined that the target area 1104 does not fit within the coordinate range. In the case of the example of FIG. 11, it is determined that the target area 1104 and the feature points of the line projection pattern 1103 may overlap.

As a result of the determination process described above, when it is determined that the target area 1104 does not fit within the coordinate range based on the feature points of the line projection pattern 1103 (the target area and the feature points overlap), the display unit 110 displays the figure. The notification text 1102 shown in 11 is displayed on the screen. For example, the display unit 110 indicates that "the feature points hang on the subject" indicating that the target area and the feature points overlap, and "the feature points are the subject" indicating that the target area is within the coordinate range. Please adjust so that it does not get caught. ”Is displayed in the notification text 1102. By seeing this notification text 1102, the user using the area measuring device 100 can appropriately adjust the installation state of the target subject 111 or the projection position by the projection unit 102 of the area measuring device 100. As a result, the area measurement accuracy of the target area can be improved.

In FIG. 12, when the projection position of the line projection pattern 1203 is far from the target area 1204 and it becomes difficult to accurately measure the area of the target area 1204, the display unit 110 displays the notification text 1202 on the screen 1201. It is a figure which showed the example displayed in. In the example of FIG. 12, the line projection pattern 1203 is projected at a position somewhat distant from the target area 1204, and the image of the target subject including the line projection pattern 1203 in that state is displayed on the screen 1201 by the display unit 110. It is displayed above. In the example of FIG. 12, it is assumed that the target subject is tilted without being orthogonal to the optical axis, but it is determined that the entire area of the target area 1204 is not in a state where it cannot be photographed.

As shown in FIG. 12, when the line projection pattern 1203 is projected at a position some distance from the target area 1204, the coordinates of the target area 1204 on the target subject and the coordinates of the feature points of the line projection pattern 1203 There is a possibility that a deviation (error) may occur between the two. If there is a deviation (error) between the coordinates of the target area 1204 and the coordinates of the feature points of the line projection pattern 1203, accurate measurement of the area becomes impossible. Therefore, in the area measuring device 100 of the present embodiment, the display unit 110 determines whether or not the line projection pattern 1203 is too far from the target area 1204. When performing this determination process, the display unit 110 has a distance between the target coordinates detected from the line projection pattern 1204 and the coordinates of the target area 1204 detected by the area detection unit 108, which is larger than a predetermined distance. Determine if it is. Then, when the display unit 110 determines that the distance between the target coordinates and the coordinates of the target area 1204 is larger than a predetermined distance, the line projection pattern 1203 is too far from the target area 1204. Is determined.

When it is determined that the line projection pattern 1203 is too far from the target area 1204 as a result of the determination process described above, the display unit 110 displays the notification text 1202 as shown in FIG. 12 on the screen. For example, the display unit 110 requests that the line projection pattern be brought closer to the target area, saying "The feature point is too far from the subject", which indicates that the line projection pattern is too far from the target area. Display the notification text 1202 including "Please adjust so that the feature point is closer to the subject." By seeing this notification text 1202, the user using the area measuring device 100 can appropriately adjust the installation state of the target subject 111, and as a result, the area measurement accuracy of the target area can be improved. can.

As described above, in the area measuring device 100 of the present embodiment, the target subject image is converted into an image as if taken from the front based on the target coordinates and the reference coordinates, and the measurement target area is converted from the converted image. Detect and calculate the actual area per number of pixels in the measurement target area. Thereby, the area measuring device 100 of the present embodiment can measure the actual area of the measurement target area even when the target subject is not perpendicular to the optical axis.

The configuration according to the above-described embodiment or the processing of each flowchart may be realized by the hardware configuration, or may be realized by the software configuration by, for example, the CPU executing the program according to the present embodiment. Further, a part may be realized by a hardware configuration and the rest may be realized by a software configuration. The program for software configuration may be acquired not only when it is prepared in advance, but also from a recording medium such as an external memory (not shown) or via a network (not shown).

A program that realizes one or more functions in the control process according to the present invention can be supplied to a system or device via a network or storage medium, and is read and executed by one or more processors of the computer of the system or device. It is feasible by being done.
Each of the above-described embodiments is merely an example of embodiment in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from the technical idea or its main features.

100: Area measuring device, 101: Imaging unit, 102: Projection unit, 103: Imaging processing unit, 103a: Focus adjustment unit (distance measurement unit), 103b: Projection control unit, 104: Photographed image storage unit, 105: Feature inspection Output part, 106: Feature point coordinate holding part, 107: Image conversion part, 108: Area detection part, 109: Area acquisition part, 110: Display part

Claims (16)

A feature point detecting means for detecting the coordinates of the feature points of the projected predetermined pattern light from a captured image of a target subject on which a predetermined pattern light including a plurality of feature points is projected.
A distance acquisition means for acquiring the subject distance when the target subject is photographed, and
A coordinate holding means for holding a predetermined reference coordinate prepared in advance and a subject distance in association with each other,
A captured image of the target subject based on a projective transformation using the coordinates of the feature point detected from the captured image of the target subject and the reference coordinates corresponding to the subject distance when the target subject is photographed. Image conversion means that performs image conversion processing and
An area detecting means for detecting a target area for area measurement from a captured image of the target subject after performing the image conversion process, and
An area acquisition means for acquiring the area of the target area from the number of pixels of the target area detected by the area detection means and the area per pixel based on the subject distance when the target subject is photographed.
An area measuring device characterized by having. The reference coordinates are coordinates of feature points detected from an image of a flat subject in a state in which the predetermined pattern is projected, taken from the front at each of a plurality of subject distances.
The area measuring device according to claim 1, wherein the subject distance associated with the reference coordinates is the subject distance when the plane subject is photographed in front of the subject. The area measuring device according to claim 1, wherein the reference coordinates are coordinates of feature points obtained by a calculation formula using the subject distance as a variable. A projection device that projects a predetermined pattern of light including the plurality of feature points, and
An imaging device that acquires the captured image by imaging, and
The area measuring device according to any one of claims 1 to 3, wherein the area measuring device is characterized by having. The imaging device captures the target subject in a state in which the predetermined pattern light is projected.
The feature point detecting means detects the coordinates of the feature point of the projected predetermined pattern light from a captured image captured by the imaging device of the target subject in a state where the predetermined pattern light is projected. ,
The area according to claim 4, wherein the image conversion means performs the image conversion process on a captured image captured by the image pickup apparatus on the target subject in a state where the predetermined pattern light is projected. Measuring device. The imaging device takes a picture of the target subject in a state where the predetermined pattern light is projected and a picture of the target subject in a state where the predetermined pattern light is not projected, respectively, at the same subject distance.
The feature point detecting means detects the coordinates of the feature point of the projected predetermined pattern light from a captured image captured by the imaging device of the target subject in a state where the predetermined pattern light is projected. ,
The fourth aspect of claim 4, wherein the image conversion means performs the image conversion process on a captured image captured by the image pickup apparatus on the target subject in a state where the predetermined pattern light is not projected. Area measuring device. The claim is characterized in that the feature point detecting means detects at least one of a specific wavelength, a specific illuminance, and a specific shape controlled for each region in the predetermined pattern light as the feature point. The area measuring device according to any one of claims 1 to 6. When the feature point detecting means detects the specific wavelength of the predetermined pattern light as the feature point, a plurality of pixels in which the RGB values of the three primary colors fall within a predetermined range are adjacent to each other from the captured image. The area measuring device according to claim 7, wherein the area is detected, and the center coordinates in the area in which the plurality of pixels are adjacent to each other are detected as the coordinates of the feature points. When the specific shape of the predetermined pattern light is detected as the feature point, the feature point detecting means detects an intersection portion where straight lines intersect from the captured image, and the intersection portion is the coordinates of the feature point. The area measuring device according to claim 7, wherein the area measuring device is detected as. The area according to claim 9, wherein the feature point detecting means detects the intersection portion where straight lines approximated from pixels in the same color region intersect in the captured image as the coordinates of the feature point. Measuring device. It is characterized by having a display means for displaying a notification to a user on a display device based on the coordinates of the feature points detected from a captured image of the target subject in a state where the predetermined pattern light is projected. The area measuring device according to any one of claims 1 to 10. When the difference between the coordinates of the feature point detected from the captured image of the target subject in the state where the predetermined pattern light is projected and the reference coordinates is larger than the predetermined coordinate difference. The area measuring device according to claim 11, wherein a notification indicating an instruction for making the difference smaller than a predetermined coordinate difference is displayed. In the display means, the coordinates of the target area are contained in the coordinate range based on the coordinates of the feature points detected from the captured image of the target subject in the state where the predetermined pattern light is projected. The area measuring device according to claim 11 or 12, wherein if not, a notification indicating an instruction for accommodating the target area within the coordinate range is displayed. In the display means, the distance between the coordinates of the feature point detected from the captured image of the target subject in the state where the predetermined pattern light is projected and the coordinates of the target area is a predetermined distance. The area measuring device according to any one of claims 11 to 13, wherein when the size is larger, a notification indicating an instruction for bringing the feature point closer to the target area is displayed. It is an area measurement method executed by the area measurement device.
A feature point detection step of detecting the coordinates of the feature points of the projected predetermined pattern light from a captured image of a target subject on which a predetermined pattern light including a plurality of feature points is projected.
A distance acquisition process for acquiring the subject distance when the target subject is photographed, and
A coordinate holding process that holds a predetermined reference coordinate prepared in advance and a subject distance in association with each other,
A captured image of the target subject based on a projective transformation using the coordinates of the feature point detected from the captured image of the target subject and the reference coordinates corresponding to the subject distance when the target subject is photographed. Image conversion process that performs image conversion processing and
An area detection step of detecting a target area for area measurement from a captured image of the target subject after the image conversion process is performed.
An area acquisition step of acquiring the area of the target area from the number of pixels of the target area detected by the area detection step and the area per pixel based on the subject distance when the target subject is photographed.
An area measurement method characterized by having. A program for causing a computer to function as each means included in the area measuring device according to any one of claims 1 to 14.

Images