JP6176598B2 - Dimension measurement program, dimension measurement apparatus, and dimension measurement method - Google Patents

Description

  The present invention relates to a dimension measuring program, a dimension measuring apparatus, and a dimension measuring method used for measuring a dimension of a measurement target.

  There are many occasions where the actual size of an object is necessary in daily life. Therefore, in recent years, with the widespread use of mobile terminals such as smartphones and tablet terminals, many applications that can measure the dimensions of objects have been developed.

  For example, markers with known sizes are arranged on the same plane as the measurement object (hereinafter referred to as the arrangement surface), and are photographed almost perpendicularly to the arrangement surface from directly above the marker and the measurement object. A technique for measuring the size of an object to be measured by comparing the dimensions on an image has been developed.

  Further, in Patent Document 1 below, an image is taken in a state where a reference scale is arranged on the surface of a rectangular parallelepiped measurement object, and a parameter and a scale related to the posture of the measurement object are obtained from the captured image, and based on the parameter and the scale. A technique for measuring a dimension of a measurement object is disclosed.

JP 2000-78452 A

  However, in the above-described technique for comparing dimensions on the captured image, the lengths of the marker and the measurement object are compared unless the image is captured almost directly above the marker and the measurement object and substantially perpendicular to the arrangement surface. Cannot be accurately performed, and there is a problem that the measurement accuracy is lowered when photographing from an oblique direction.

  Moreover, since the technique of the said patent document 1 utilizes the parameter regarding the attitude | position of a rectangular parallelepiped-shaped measuring object, the measuring object was limited to the rectangular parallelepiped shape, and the problem that versatility was low has arisen.

  Therefore, the object of the present invention is that the measurement accuracy of the dimension does not decrease even when the image is taken from an oblique direction, and the measurement object is not limited to a rectangular parallelepiped shape, but a highly versatile dimension measurement program, a dimension measurement apparatus, And it is providing the dimension measuring method.

  As a result of repeated investigations to achieve the above object, the inventors have approximated a captured image from an oblique angle to a front image by coordinate conversion, and can obtain approximately the same measurement accuracy as when using the front image. Got the view. Therefore, the inventors have conducted further earnest research and can approximate the front image by performing perspective projection conversion of the captured image, and for that purpose, at least four points can be associated between the captured image and the front image. It is necessary, and the technical idea that this association can be realized using a marker whose front image and dimensions are known has been obtained. The present invention has been completed based on this technical idea.

In the dimension measuring program of the present invention, in the dimension measuring program used for measuring the dimension of the measurement object, when the dimension in the real space of the marker is input by the user, the dimension storing step of storing the dimension of the marker in the storage means when a photographed image acquisition step of acquiring a captured image including said marker with a measurement target, at least four points when the reference point included in the image upon shooting front image of the marker, the captured image From the captured image acquired in the acquisition step, the corresponding point specifying step for specifying the corresponding point corresponding to the reference point, the posture of the marker stored in the storage means, and the captured image acquisition step are acquired. The correspondence between the reference point and the corresponding point can be designated by the user so that the postures of the markers in the captured image are aligned, and the designated correspondence At engagement, corresponding points identified by the corresponding point specific step calculates conversion parameters for perspective projection conversion so as to be located on the reference point corresponding, the captured image based on the conversion parameter A perspective projection conversion step for converting the projection of the marker in real space from the storage means, the coordinates of the marker in the image after the perspective projection conversion, and the size of the marker in the real space, And a scale calculating step of calculating a scale of the image after the perspective projection conversion.
In the dimension measuring apparatus of the present invention, in the dimension measuring apparatus used for measuring the dimension of the measurement object, when the dimension of the marker in the real space is input by the user, the dimension of the marker is stored in the storage unit. When the storage means, the captured image acquisition means for acquiring the captured image including the marker and the measurement target, and at least four points included in the image when the front image of the marker is captured as the reference point, From the captured image acquired by the captured image acquisition means, the corresponding point specifying means for specifying the corresponding point corresponding to the reference point, the orientation of the marker stored in the storage means, and the captured image acquisition means and as orientation of the marker in the captured image are aligned, and can be designated by a user to the corresponding points corresponding relationship between the reference point at the designated relationship, the corresponding point Laid Perspective projection conversion means for calculating a transformation parameter for perspective projection transformation so that each corresponding point specified by the means is located at the corresponding reference point, and perspective projection transformation of the captured image based on the transformation parameter And the dimension of the marker in the real space is obtained from the storage means, and the scale of the image after the perspective projection conversion is obtained from the coordinate of the marker in the image after the perspective projection conversion and the dimension of the marker in the real space. And a scale calculating means for calculating.
Further, the dimension measuring method of the present invention is a dimension measuring method executed by a computer for measuring a dimension of a measurement object. When a dimension of a marker in a real space is input by a user, the dimension of the marker is stored. A dimension storing step for storing the image, a captured image acquiring step for acquiring a captured image including the marker and the measurement target, and at least four points included in the image when the front image of the marker is captured as reference points Sometimes, from the captured image acquired in the captured image acquisition step, a corresponding point specifying step for specifying a corresponding point corresponding to the reference point, the orientation of the marker stored in the storage means, and the captured image as orientation of the marker in the obtained captured image in acquiring step are aligned, and can be designated by a user to the corresponding points corresponding relationship between the reference point, A conversion parameter for perspective projection conversion is calculated so that each corresponding point specified in the corresponding point specifying step is positioned at the corresponding reference point with the specified correspondence relationship, and the conversion parameter is A perspective projection conversion step of performing a perspective projection conversion on the captured image based on the obtained image, acquiring the dimension of the marker in the real space from the storage unit, the coordinates of the marker in the image after the perspective projection conversion, and the marker in the real space And a scale calculating step for calculating a scale of the image after the perspective projection conversion from the dimensions.

  According to this invention, when at least four points included in the image when the front image of the marker is captured are used as reference points, the corresponding points corresponding to the reference points are identified from the captured image, and the corresponding points and the reference points are identified. By performing perspective projection conversion of the captured image using the conversion parameter calculated based on the above, it is possible to approximate the captured image captured obliquely to the front image. Since the scale is calculated using the image after the perspective projection conversion approximated to the front image, the same accuracy as the front image can be obtained in the dimension measurement. Accordingly, it is possible to prevent a decrease in measurement accuracy even when the image is taken from an oblique direction.

  According to the present invention, even if a captured image is taken obliquely, the perspective image is approximated to the image captured from the front by perspective projection conversion, and the scale is calculated in the image after the perspective projection conversion. Measurement accuracy equivalent to images can be obtained. Since the perspective projection conversion is performed based on the marker, the measurement target is not limited to the rectangular parallelepiped shape. Thereby, even if it is a case where it image | photographs from diagonal, a measurement precision does not fall, and a highly versatile dimension measuring program, a dimension measuring apparatus, and a dimension measuring method can be provided.

It is explanatory drawing explaining the outline | summary of 1st embodiment of this invention, FIG. 1 (a) is a figure which shows the picked-up image image | photographed from the diagonal, FIG.1 (b) is a figure which shows the front image of a marker, FIG. 1 (c) is a diagram illustrating an image after perspective projection conversion. It is explanatory drawing explaining the dimension measuring apparatus in the said embodiment. FIG. 3A is an explanatory diagram for explaining an example of a marker in the embodiment, FIG. 3A is a rectangular marker, FIGS. 3B and 3C are markers including a part of each side of the rectangle, FIG. ) Is a marker partially including each side of the rectangle, and FIG. 3E is an explanatory diagram for explaining the marker including a point corresponding to each vertex of the rectangle. It is a functional block diagram explaining the dimension measuring apparatus in the said embodiment. It is a figure which shows the flowchart of the process by the dimension measuring apparatus in the said embodiment. FIGS. 6A and 6B are explanatory diagrams for explaining marker contour extraction in the above embodiment, FIG. 6A is a diagram showing a photographed image obtained by photographing the marker from an oblique direction, and FIG. 6B is a diagram illustrating the marker contour from the photographed image. It is explanatory drawing explaining the extracted state. It is a figure which shows the flowchart of the process by the corresponding point specific | specification means in the said embodiment. FIG. 8A is an explanatory diagram for explaining identification of corresponding points in the embodiment, FIG. 8A is a diagram showing a state in which a straight line is detected from a contour line by straight line detection, and FIG. 8B is an extension of the straight line. It is a figure which shows a state. FIG. 9A is an explanatory diagram for explaining a straight line correction process in the above embodiment, and FIG. 9A is an explanatory diagram for explaining a region corresponding to a straight line detected by straight line detection in the contour line of the marker. ) Is an explanatory diagram for explaining a regression line in the region, and FIG. 9C is a diagram showing a state in which the straight line is corrected by the regression line. It is a figure which shows the flowchart of the straight line correction process in the said embodiment. It is explanatory drawing explaining the conversion process by the perspective projection conversion means in the said embodiment. It is a figure which shows the picked-up image used for description of the usage condition in the said embodiment. It is explanatory drawing explaining the area division | segmentation in the usage condition of the said embodiment. It is explanatory drawing explaining the quadrangle | tetragon and a corresponding point in the usage condition of the said embodiment. It is a figure which shows the image after perspective projection conversion in the usage condition of the said embodiment. It is a functional block diagram explaining the dimension measuring device in a second embodiment of the present invention. It is a figure which shows the flowchart of the process by the dimension measuring apparatus in the said embodiment. It is an example of the display image from which the scale bar in the said embodiment was output. It is a block diagram explaining the hardware resource of a computer.

[First embodiment]
The dimension measuring apparatus 1 according to the present embodiment is used when measuring the dimension of a measurement target. FIG. 1 is a schematic explanatory diagram of the present embodiment, in which FIG. 1 (a) is a view showing a captured image Pc1 taken from an oblique direction, FIG. 1 (b) is a view showing a front image of the marker Mk, and FIG. c) is a diagram showing a converted image Pc2 after perspective projection conversion. The marker Mk has a known shape feature and size of the front image, and the information is stored in the size measuring device 1.

  The dimension measuring apparatus 1 performs perspective projection conversion of the entire photographed image Pc1 so that the marker Mk of the photographed image Pc1 (FIG. 1A) becomes a front image (FIG. 1B). The resulting converted image Pc2 (FIG. 1 (c)) approximates the front image. Then, using the converted image Pc2, the dimension of the measurement object Ob is measured with the marker Mk as a reference. Thereby, even if the photographed image Pc1 is an image photographed from an oblique direction, measurement accuracy equivalent to that obtained when photographed from the front can be obtained. Moreover, since perspective projection conversion is performed based on the marker Mk, the measuring object Ob is not limited to a rectangular parallelepiped shape, and versatility is enhanced. Details will be described below.

  FIG. 2 is an explanatory diagram for explaining the dimension measuring apparatus 1. The dimension measuring apparatus 1 is a portable terminal, personal computer, or other computer, and by reading a dimension measuring program stored in a non-volatile storage means (not shown) such as a ROM, a CPU, a memory, etc. Hardware resources and software cooperate to realize each means described below and function as the dimension measuring apparatus 1.

  The dimension measuring apparatus 1 can input a photographed image photographed by the photographing means C, and can output a processing result to the output means Dp. The photographing means C is a digital camera or a digital video camera in which CCD (Charge Coupled Device) pixels are arranged, and the output means Dp is a liquid crystal display, a CRT (Cathode Ray Tube) display, or the like. Moreover, the dimension measuring apparatus 1 is provided with the input means In which receives the input from a user. The input means In is, for example, a keyboard, a mouse, a touch pad, or the like. The input means In and the output means Dp may be a touch panel in which both means are integrated. The photographing means C, output means Dp, and input means In may be integrated with the dimension measuring apparatus 1 or may be separate.

  The captured image Pc1 is an image including a marker Mk and a measurement object Ob as shown in FIG. In the present embodiment, the marker Mk and the measurement object Ob are located on substantially the same plane A. Here, the plane A may be a surface of a tangible object such as a table or a floor, or may be a virtual plane. For example, when the marker Mk is a part of a three-dimensional object as in the case of one side of the box, the three-dimensional object is preferably arranged so that a part thereof is positioned on the plane A. Similarly, when the measurement object Ob is a part of a three-dimensional object, it is preferable to arrange the three-dimensional object so that the measurement object Ob is positioned on a plane. In addition, when the marker Mk and the measurement object Ob are part of a solid object with a negligible thickness, the solid object may be arranged on the same plane.

  The marker Mk only needs to be able to identify at least four points (hereinafter referred to as reference points) from the components of the front image in the front image obtained by photographing the front image. Here, the front image means an image taken from a position substantially directly above (directly in front) and perpendicular to the marker Mk.

  In the following description, a marker Mk that can be identified with a point corresponding to a rectangular vertex as a reference point will be described as an example. In perspective projection conversion, at least four corresponding points are required between an image before conversion and an image after conversion. Therefore, by using the concept of a rectangle having the minimum four points as vertices, it becomes easy to identify a reference point and associate an image before conversion with an image after conversion. Further, the coordinates of each vertex can be obtained from the width and height dimensions of the marker Mk.

  FIG. 3 is an explanatory diagram illustrating an example of the marker Mk. Both will be described using a front image of the marker Mk. FIG. 3A is a front image of the rectangular marker Mk, and the reference point p1 is each vertex of the rectangle. Examples include a rectangular pattern, one side of a business card, one side of a notebook, the upper surface of a table, the display surface of a display, one side of a cigarette box, one side of a cardboard box, a window frame, a wall surface, and one side of a building. 3B and 3C are front images of the marker Mk including a part of each side of the rectangle, and the reference point p1 is an intersection obtained by extending the straight line. For example, as shown in FIG. 3 (b), a pattern corresponding to a vertex of a rectangle is curved, one surface of a card such as a credit card or a point card, one surface of a playing card, one surface of a mobile phone, etc. As shown in (c), a cross pattern or the like having a notch in each part corresponding to the vertex of a rectangle can be mentioned. FIG. 3D is a front image of the marker Mk partially including each side of the rectangle, and the reference point p1 is an intersection of the straight lines. For example, a lattice pattern, a lattice-shaped fence, etc. are mentioned. FIG. 3E is a front image of the marker Mk including points corresponding to the vertices of the rectangle, and the reference point p1 is each point. For example, an array of dots arranged in a matrix can be used. In the following description, a case where the marker Mk shown in FIG. 3B is used will be described as an example.

  FIG. 4 is a functional block diagram illustrating the dimension measuring apparatus 1 according to the above embodiment. The dimension measurement apparatus 1 includes a captured image acquisition unit 10, a marker recognition unit 20, a corresponding point identification unit 30, a perspective projection conversion unit 40, a scale calculation unit 50, a dimension calculation unit 60, an image storage unit 70, and a dimension storage unit 80. .

  The captured image acquisition unit 10 has a function of acquiring the captured image Pc1. The marker recognition unit 20 has a function of recognizing the marker Mk from the captured image Pc1 acquired by the captured image acquisition unit 10. The corresponding point specifying unit 30 has a function of specifying the corresponding points p2,, and p2 corresponding to the reference points p1,, and p1 based on the marker Mk recognized by the marker recognizing unit. The perspective projection conversion means 40 sets conversion parameters for perspective projection conversion so that the corresponding points p2,..., P2 specified by the corresponding point specifying means 30 are positioned at the corresponding reference points p1,. A function of calculating and perspective-projecting the captured image Pc1 based on the conversion parameter is provided. The scale calculating unit 50 calculates the scale of the converted image Pc2 based on the coordinates of the marker Mk of the converted image Pc2 after the perspective projection conversion by the perspective projection converting unit 40 and the dimension of the marker Mk acquired from the dimension storage unit 80 in the real space. The function to calculate is provided. The dimension calculating unit 60 has a function of calculating the dimension of the measurement object Ob based on the scale calculated by the scale calculating unit 50. The image storage unit 70 has a function of storing the captured image Pc1 acquired by the captured image acquisition unit 10, and the dimension storage unit 80 has a function of storing the dimension of the marker Mk in the real space. And a cache memory and a main memory.

  Hereinafter, the dimension measuring method by the dimension measuring apparatus 1 will be described with reference to the flowchart shown in FIG.

  In step S1, the captured image acquisition unit 10 acquires a captured image Pc1. The photographed image Pc1 is input, for example, from the photographing unit C integrated with the dimension measuring apparatus 1 via an internal wiring, or is input from a separate photographing unit C via an external wiring. It may be input from a network or a storage medium. The acquired captured image Pc1 is stored in the image storage means 70.

  In step S2, the marker recognizing unit 20 acquires the captured image Pc1 stored in the image storage unit 70, and recognizes the marker Mk from the captured image Pc1. For marker recognition, an object recognition technique based on image processing may be used. The marker recognizing means 20 of this embodiment is performed by contour extraction. The contour extraction can be performed by feature detection or feature extraction. In the present embodiment, a region division that divides the captured image Pc1 into the region of the marker Mk and the other region is used, and the boundary between the two regions is extracted as a contour.

  Hereinafter, it will be described in detail with reference to FIG. FIG. 6A is a diagram illustrating an example of the marker Mk in the captured image Pc1, and FIG. 6B is an explanatory diagram illustrating a state in which the contour of the marker Mk is extracted from the captured image Pc1. The marker recognizing unit 20 displays the captured image Pc1 on the display unit Dp, and allows the user to designate an arbitrary point r1 within the marker Mk region and an arbitrary point r2 outside the marker Mk region. A plurality of points may be designated in each region. When the marker recognizing unit 20 receives a user input from the input unit In, the marker recognizing unit 20 divides the captured image Pc1 into a region to which the point r1 belongs and a region to which the point r2 belongs, and a boundary line between the two regions (that is, the marker Mk (Outline) L0 is extracted. The processed image is a binary image (FIG. 6B) obtained by extracting the contour line L0 from the captured image Pc1. Note that open source can be used for the contour extraction processing by area division, and for example, a cvWatershed function using the Watershed algorithm published in the image processing library OpenCV may be used. However, the present invention is not limited to this, and division based on pixels using threshold processing, clustering, or the like, division based on an area using a local search type algorithm, division by edge detection, or the like may be used. Note that the processing by the marker recognizing means 20 is not limited to this, and for example, a method of recognizing a local region by SURF (Speeded Up Robust Feature) using a local feature amount, SIFT (Scale-invariant feature transform), or the like. Alternatively, other object recognition techniques such as comparison with a standard pattern may be used.

  Returning to FIG. 5, in step S <b> 3, based on the marker Mk recognized by the marker recognizing unit 20, the corresponding point specifying unit 30 sets the corresponding points p <b> 2,. Identify. Here, the reference points p2,..., P2 are points corresponding to the vertices of the rectangle in the front image (see FIG. 1B) of the marker Mk, but in the captured image Pc1 photographed obliquely, trapezoids and rhombuses. It becomes a point corresponding to the vertex of a rectangle such as. For this reason, the corresponding point specifying means 30 obtains four points corresponding to each vertex of the quadrangle as corresponding points p2,. FIG. 7 is a flowchart of the corresponding point specifying process in step S3.

  In step S31, the corresponding point specifying unit 30 detects a straight line from the extracted outline L0. For detection of a straight line, feature detection or feature extraction in image processing may be used. For example, Hough transform can be used. For line detection, for example, a cvHoughLines2 function disclosed in the image library OpenCV may be used. As a result of the processing, as shown in FIG. 8A, straight lines L1 to L4 are detected from the contour line L0.

  In step S32, the corresponding point specifying unit 30 performs a process of extending the detected straight lines L1 to L4. As shown in FIG. 8B, the straight lines L1 to L4 are extended to cross each other to form a quadrangle Sq1. Thereby, even if it is a case where the marker Mk whose corner is rounded is used, each vertex of the quadrangle Sq1 can be specified.

  Here, the straight lines L1 to L4 may be slightly shifted from the contour line L0. Therefore, in step S33, the corresponding point specifying unit 30 corrects the deviation of the straight lines L1 to L4. Specifically, the corresponding point specifying unit 30 includes straight lines L <b> 1 to L <b> 4 (FIG. 8B) among a plurality of pixels constituting the contour line L <b> 0 (see FIG. 6B) extracted by the marker recognizing unit 20. The regression lines RL1 to RL4 are calculated from the pixel distribution corresponding to ()), and the vertices of the quadrangle Sq2 formed by the regression lines RL1 to RL4 are extracted as corresponding points p2,. FIG. 9 is an explanatory diagram for explaining the straight line correction processing. FIG. 9A is an explanatory diagram for explaining a region a corresponding to the straight line L1 in the contour line L0 of the marker Mk, and FIG. FIG. 9C is a diagram illustrating a state in which the straight lines L1 to L4 are corrected by the regression lines RL1 to RL4. The correction of the straight line L1 shown in FIG. 8B will be described as an example. From the distribution of pixels corresponding to the straight line L1 (pixels px,..., Px distributed in the region a in FIG. 9A), the regression line RL1 is obtained. calculate.

  Details will be described below. FIG. 10 is a diagram showing a flowchart of straight line correction processing in step S33. The corresponding point specifying unit 30 selects the straight line Lx from the straight lines L1 to L4 in step S331, and specifies the pixel corresponding to the straight line Lx among the pixels constituting the contour line L0 in step S332. For example, among the pixels constituting the contour line L0, the pixels whose distance from the straight line Lx is within a predetermined threshold are pixels corresponding to the straight line Lx. Next, in step S333, the corresponding point specifying unit 30 calculates a regression line RLx from the specified pixel distribution. The regression line RLx can be calculated by the least square method or the like. In step S334, the corresponding point specifying unit 30 determines whether there is an unselected straight line among the straight lines L1 to L4. If there is, the process returns to step S331 to select the next straight line Lx from the unselected straight lines. Repeat the same process. If there are no unselected straight lines in the straight lines L1 to L4, the process is terminated. Thereby, as shown in FIG.9 (c), the regression lines RL1-RL4 which corrected each straight line L1-L4 are obtained.

  Returning to FIG. 7, the corresponding point specifying means 30 extracts the quadrangle Sq2 by extracting the line segment structure constituting the quadrilateral from the structure of the regression lines RL1 to RL4 in step S34, and each vertex of the extracted quadrilateral Sq2 Are identified as corresponding points p2,. For example, a cvApproxPoly function or a cvSeq structure disclosed in the image library OpenCV may be used for extracting the quadrangle Sq2.

  The straight line extension process (step S32) and the straight line correction process (step S33) are arbitrary and may be provided as necessary. For example, in the case of the rectangular marker shown in FIG. 3A or the grid-like marker shown in FIG. 3D, the straight line extension process (step S32) is unnecessary. Further, when the straight line correction process (step S33) is not performed, the quadrangle Sq1 is extracted from the structure of the straight lines L1 to L4, and the apexes thereof may be the corresponding points p2,.

  Returning to FIG. 5, in step S <b> 4, the perspective projection conversion unit 40 causes the corresponding points p <b> 2,... P <b> 2 specified by the corresponding point specifying unit 30 to be positioned at the corresponding reference points p <b> 1,. Conversion parameters for perspective projection conversion are calculated, and the captured image Pc1 is perspective projection converted based on the conversion parameters. FIG. 11 is a diagram for explaining the conversion process by the perspective projection conversion means 40. The perspective projection conversion means 40 includes the coordinates (x1, y1) (x2, y2) (x3, y3) (x4, y4) of the four corresponding points p2,, p2 in the image coordinate system of the converted image Pc2, and the front image. Conversion parameters in perspective projection conversion are calculated from the correspondence with the coordinates (X1, Y1) (X2, Y2) (X3, Y3) (X4, Y4) of the reference points p1,, p1 in the image coordinate system of The entire captured image Pc1 is subjected to perspective projection conversion using the conversion parameter.

  Data and rules for deriving the coordinates (X1, Y1) to (X4, Y4) of the reference points p1,..., P1 are stored in advance in the storage means, and are obtained based on these data and rules. . For example, the coordinates (X1, Y1) to (X4, Y4) of the reference points p1,..., P1 are stored in the storage means as the shape features of the marker Mk, and these may be read and used. Further, the coordinates (x1, y1) of any one of the corresponding points p2,..., P2 are set as the coordinates (X1, Y1) of the reference point, and the rectangle has a ratio of dimensions stored in the dimension storage means 80. As described above, the coordinates (X2, Y2) to (X4, Y4) of other reference points may be calculated. Further, as a shape feature of the marker Mk, arbitrary rectangular vertex coordinates are stored in the storage means in advance, and each side formed by each vertex coordinate is a ratio of the dimensions stored in the dimension storage means 80. As described above, the coordinates (X1, Y1) to (X4, Y4) of the reference points p1,, p1 may be calculated.

  The correspondence between the coordinates (X1, Y1) to (X4, Y4) of the reference points p1,, p1 and the coordinates (x1, y1) to (x4, y4) of the corresponding points p2,, p2 is determined by the user. It may be possible to specify, or may be automatically determined based on a predetermined rule. For example, the reference point and the corresponding point can be associated in four ways when the marker Mk is a square and two ways when the marker Mk is a rectangle. Therefore, the perspective projection conversion means 40 may present these plural candidates for association to the user so that they can be designated by the user, or may automatically select the candidates based on some rule. For example, in the present embodiment, the perspective projection conversion means 40 determines the coordinates of the reference points p1,, p1 so that the sides of the marker Mk are vertical and horizontal in the converted image Pc2, and the corresponding points p2,, p2 Is selected at the reference points p1,..., P1, and the candidate having the smallest rotation angle of the marker Mk is selected.

The perspective projection conversion specifically uses the following method. When an arbitrary coordinate (x, y) is projected to (X, Y) by perspective projection transformation, the perspective projection transformation is represented by the following equation 1, and the inverse transformation is represented by equation 2.

  The perspective projection conversion means 40 includes coordinates (x1, y1) to (x4, y4) of the four corresponding points p2,, and p2 and coordinates (X1, Y1) to (X4, Y4) of the reference points p1,, and p1. Then, the conversion parameters a to h or A to H in the above formula 1 or 2 are calculated, and the entire captured image Pc1 is subjected to perspective projection conversion using the calculated conversion parameters. For example, when the photographed image Pc1 shown in FIG. 1 (a) is converted by the perspective projection conversion means 40, a converted image Pc2 approximated to the front image is obtained as shown in FIG. 1 (c). Note that the converted image Pc2 may or may not be displayed on the display unit Dp.

  Here, since the conversion parameter is calculated from Equation 1 or Equation 2, perspective projection conversion is possible if there are at least four corresponding points between the captured image Pc1 and the converted image Pc2. That is, the marker Mk is not limited to a rectangle, and it is sufficient that at least four reference points p1,..., P1 can be specified and a reference dimension is known.

  Returning to FIG. 5, in step S5, the scale calculating means 50 acquires the dimension D of the marker Mk in the real space from the dimension storage unit 80, and coordinates of the corresponding points p2,..., P2 in the image coordinate system of the converted image Pc2. Then, the scale of the image coordinate system of the converted image Pc2 is calculated from the correspondence relationship with the dimension D of the marker in the real space. In the present embodiment, two dimensions of the marker Mk are stored as dimensions of the marker Mk (dimensions indicated by reference sign w and reference sign h in FIG. 3B), and the converted image is stored. A scale is calculated from the correspondence between two coordinates that specify the width of the marker Mk in Pc2 and two coordinates that specify the height. As these coordinates, the coordinates of the four corresponding points p2,..., P2 may be used. The calculated scale is stored in a storage means (not shown).

  As the dimension D in the real space of the marker Mk, a numerical value sequentially input by the user may be stored in the dimension storage unit 80. In this case, the degree of freedom in selecting a marker is increased. In addition, the dimension data of a plurality of markers may be stored in the dimension storage unit 80 in advance and the marker may be selected from the database by the user. In this case, it is possible to save the user from inputting numerical values of the dimension data. If the database is created using dimensions based on industrial standards such as Japanese Industrial Standards and International Standards, the work is easy and versatility is high. Moreover, when using a dedicated marker, the dimension data should just be memorize | stored.

  Returning to FIG. 5, in step S <b> 6, the dimension calculating unit 60 calculates the dimension of the measurement object Ob in the converted image Pc <b> 2 based on the scale calculated by the scale calculating unit 50. In the present embodiment, the captured image Pc1 or the converted image Pc2 is displayed on the display means Dp, and the user can specify the measurement location of the measurement object Ob on the display image. The dimension calculation means 60 acquires the coordinates of the measurement location in the converted image Pc2, and calculates the dimensions based on the scale. The calculated dimensions are output to the display means Dp.

  Hereinafter, an example of a usage mode of the dimension measuring apparatus 1 in the present embodiment will be described. FIG. 12 is a diagram showing a photographed image Pc1 used for describing the usage mode, and is a photographed image photographed at an angle of about 45 degrees in the horizontal direction with respect to the arrangement surface. The case where the marker Mk is an A4 size underlay defined by ISO 216, the measurement object Ob is one panel of a partition installed as a wall of the room, and the height H of the panel is measured will be described. In addition, about each process of the dimension measuring apparatus 1, since it demonstrated above, the detail is abbreviate | omitted.

  First, the user places the underlay in a vertically long posture so that the marker Mk is positioned substantially on the same plane as the measurement object Ob, and the imaging unit C includes both the marker Mk and the measurement object Ob in the captured image Pc1. Shoot with. At this time, the user may shoot from a free angle, and does not need to shoot so that the photographic image Pc1 becomes a front image. Further, the user can freely stand, for example, and does not need to stand on the same plane as the measurement object as in the technique using triangulation. Further, the shape of the measurement object Ob is not limited. Although a slight error occurs due to the thickness of the underlay, the error is negligible unless the image is taken extremely close.

  Hereinafter, a description will be given with reference to the flowchart shown in FIG. In step S1, the captured image acquisition unit 10 acquires the captured image Pc1. The captured image Pc1 is temporarily stored in the image storage means 70 and displayed on the display device Dp.

  As shown in FIG. 13, the user designates an arbitrary point r1 in the area of the marker Mk and an arbitrary point r2 outside the area of the marker Mk with the mouse or the like for the photographed image Pc1 displayed on the display means Dp. To do. The dimension measuring apparatus 1 stores the coordinates of these two points r1 and r2 in the image coordinate system in a storage means (not shown) as area division data. Further, the user inputs the dimension D of the marker Mk. Here, the dimensions are input assuming that the dimension of each side of the A4 size, that is, the dimension of the width w is 210 mm and the dimension of the height h is 297 mm. If it is possible to select from the database, “A4 size portrait” is selected from a plurality of dimension data. The dimension measuring apparatus 1 stores the input dimension D or the dimension selected from the database in the dimension storage unit 80.

  Next, when the user instructs the start of processing by the input means In, such as clicking the start button, the dimension measuring apparatus 1 starts the following processing. In step S2, the marker recognizing means 20 recognizes the marker Mk from the captured image Pc1. Specifically, the region is divided using the two points r1 and r2, and the contour line of the marker Mk is extracted.

  In step S3, the corresponding point specifying unit 30 extracts a quadrangle Sq2 based on the contour line of the marker Mk, and specifies four corresponding points p2,. FIG. 14 is an explanatory diagram illustrating the quadrangle Sq2 and the corresponding point p2. Note that the square Sq2 and the corresponding point p2 may or may not be displayed on the display means Dp. Thereafter, in step S4, the perspective projection conversion means 40 performs perspective projection conversion of the captured image Pc1 so that the four corresponding points p2,..., P2 are positioned at the reference points p1,. FIG. 15 is an explanatory diagram showing the converted image Pc2. The converted image Pc2 is an image photographed perpendicularly to the arrangement surface from directly above the marker Mk and the measurement object Ob, that is, an image approximated to a front image. In this example, the marker Mk has a shape close to a rectangle, and the photographing object Ob also has a shape close to a rectangle.

  Next, in step S5, the scale calculation means 50 acquires the dimension D in the real space of the marker Mk from the dimension storage means 80, and from the coordinates of the marker Mk in the image coordinate system of the converted image Pc2 and the dimensions in the real space, The scale of the image coordinate system of the converted image Pc2 is calculated.

  Next, the user designates the measurement location by clicking the points r3 and r4 at both ends of the measurement location with the mouse in the converted image Pc2 displayed on the display means Dp. The dimension measuring apparatus 1 stores the coordinates of the points r3 and r4 in the storage unit as information on the measurement location. Here, the case where the measurement location is specified in the converted image Pc2 has been described as an example, but it may be specified in the captured image Pc1 before conversion.

  Next, in step S6, the dimension calculation means 60 calculates the dimension between the point r3 and the point r4 in the image coordinate system of the converted image Pc2 based on the scale. The calculation result is displayed on the display means Dp and provided to the user. Since the converted image Pc2 approximates to the front image, even when the image is taken from an oblique direction, measurement accuracy equivalent to that obtained from the front can be obtained.

[Second Embodiment]
The dimension measuring apparatus 2 of the present embodiment is obtained by adding a plurality of other functions to the dimension measuring apparatus 1 of the above embodiment. FIG. 16 is a functional block diagram for explaining the dimension measuring apparatus 2. Note that the same elements as those in the above-described embodiment are denoted by the same reference numerals and description thereof is omitted. The dimension measuring apparatus 2 uses the coordinates (x1, y1) to (x4, y4) of the corresponding points p1,, p1 specified by the corresponding point specifying unit 30 and the conversion parameters calculated by the perspective projection converting unit 40. Meta information adding means 41 for adding as meta information to the photographed image Pc1, an instruction receiving means 101 for receiving an instruction from the user, and an actual size adjusting means for adjusting and outputting the converted image Pc2 so that the measurement object Ob becomes the actual size. 102, and a scale bar output unit 103 that outputs a scale bar representing the scale calculated by the scale calculation unit 50 on the converted image Pc2.

  FIG. 17 is a diagram showing a flowchart of processing by the dimension measuring apparatus 2. Steps S11 to S14 are the same as steps S1 to S4 in the above embodiment, and step S15 is the same as step S5 in the above embodiment, and thus description thereof is omitted.

  First, step S14-1 will be described. In step S14-1, the meta information adding unit 41 uses the coordinates (x1, y1) to (x4, y4) of the corresponding points p2,..., P2 specified by the corresponding point specifying unit 30, and the perspective projection converting unit 40. The calculated conversion parameter is added to the captured image Pc1 as meta information. Thus, in subsequent use of the captured image Pc1, the entire captured image Pc1 may be executed from the perspective projection conversion using the conversion parameter added as the meta information, and the corresponding point specifying step (step S3). Alternatively, the input of the points r1 and r2 in step S13) becomes unnecessary, and the operation becomes simple. For example, when the photographed image Pc1 is received by e-mail or the like, or when the photographed image Pc1 is used again, the user can immediately obtain the converted image Pc2 from the photographed image Pc1. Further, the coordinates (x1, y1) to (x4, y4) of the corresponding points p2,..., P2 can be used as coordinates corresponding to the dimension of the marker Mk.

  In this embodiment, the example of adding the coordinates of the corresponding point and the conversion parameter as meta information has been described. However, the meta information adding unit 41 corresponds to the corresponding points p2,..., P2 specified by the corresponding point specifying unit 30. At least one of the coordinates, the conversion parameter calculated by the perspective projection conversion means 40, the dimension stored in the dimension storage means 80, or the scale calculated by the scale calculation means 50 is recorded in the captured image Pc1 as meta information. Anything can be added. Thereby, a user's operation can be simplified. That is, when the coordinates of the corresponding points p2,..., P2 and the conversion parameters are added as meta information, the input of the points r1 and r2 in the corresponding point specifying step (step S3 or step S13) becomes unnecessary, and the dimension or scale Is added as meta information, it becomes unnecessary to input and select dimensions. In addition, since unnecessary processing can be omitted, an effect of speeding up the processing can be obtained.

  In addition, the meta information addition step (step S14-1) is a subsequent process of the perspective projection conversion unit 14 in the present embodiment, but may be appropriately changed depending on an element to be added as meta information. For example, the meta information adding step (step S14-1) can be executed for each element to be added, or can be executed in a lump after all the elements have been prepared.

  The meta information may be added only when an instruction is input from the input means In by the user, or may be automatically added to all the captured images Pc1. The captured image Pc1 is preferably stored in, for example, an Exif (Exchangeable image file format) format that can be stored by adding meta information to the image data.

  Next, returning to FIG. 17, the processing after step S16 will be described. In step S16, when the instruction receiving unit 101 receives an input for instructing any one of the dimension calculation, the actual size display, and the scale bar display from the input unit In, in step S17, the input instruction is the size calculation or the actual size. Judge whether the display or scale bar display. In the case of the dimension calculation, the process proceeds to step S18, and the dimension calculation process by the dimension calculation means 60 is activated. Step S18 is substantially the same as step S6 in the above embodiment, and a description thereof will be omitted.

  In step S17, if the input instruction is an actual size display, the process proceeds to step S19 to start processing by the actual size adjusting means 102.

  Hereinafter, the actual size adjustment process in step S19 will be described. Based on the scale calculated by the scale calculating unit 50, the actual size adjusting unit 102 adjusts the converted image Pc2 after the perspective projection conversion to the actual size, and outputs the actual size to the display unit Dp. Specifically, in the dimension measuring device 2, the data of the dot pitch (dimension between adjacent dots) of the display unit Dp is stored in the storage unit (not shown). The actual size adjusting means 102 refers to the dot pitch data, so that the scale of the image coordinate system of the converted image Pc2 is the same as the scale in the real space based on the relationship between the scale and the dot pitch. The converted image Pc2 is enlarged or reduced so that the dimension d of the marker Mk in Pc2 is the same as the dimension D of the marker in real space, and is output to the display means Dp. Note that the display image may be scrollable assuming that the display image does not fit on the screen of the display means Dp. Further, the data of the dot pitches of the plurality of display means Dp may be made into a database and stored in the storage means so as to be compatible with a plurality of display means Dp having different dot pitches. The dot pitch may be acquired from the product information of the output unit Dp, or may be acquired via an operating system.

  As a result, the actual measurement object Ob is displayed on the display means Dp, and sensory measurement is possible. The user can feel the size or compare the size visually.

  Returning to FIG. 17, in step S <b> 17, when the instruction from the user is scale bar display, the process proceeds to step S <b> 20 and the process by the scale bar output unit 103 is activated.

  Hereinafter, the process in step S20 will be described. The scale bar output means 103 superimposes the scale bar indicating the scale calculated by the scale calculation means 50 on the converted image Pc2 after the perspective projection conversion, and outputs it to the output means Dp. FIG. 18 is an example of a display image output to the output unit Dp. A scale bar Br is displayed together with the measurement object Ob. The scale bar Br can be freely moved and rotated in the image coordinate system of the converted image Pc2 in accordance with a user operation. The user measures the dimension by moving the scale bar Br so as to follow the measurement location of the measurement object Ob. Thereby, the dimension of the measuring object Ob can be freely measured as if measuring the dimension with a ruler. From the viewpoint of the operability of the scale bar Br, the display means Dp is preferably a multi-touch device.

  Returning to FIG. 17, when the process of step S18, step S19, or step S20 is completed, the process proceeds to step S21 to determine whether to end the measurement process. When a process end instruction is received from the user, the series of processes is ended. If there is no end instruction, the process returns to step S16.

  As described above, according to the present embodiment, measurement by size calculation, sensory measurement by actual size display, and measurement by a virtual ruler can be selectively performed.

  In the above embodiment, the marker Mk is described mainly using the marker in which each vertex of the rectangle shown in FIG. 3B exhibits a curve. However, for example, FIGS. 3A, 3C to 3E are used. The marker shown in FIG. In this case, a design change suitable for the shape of the marker may be performed so that the corresponding point specifying unit 3 can specify each corresponding point. For example, if the marker has the shape shown in FIG. 3A, each vertex of a square may be detected as a corresponding point. In the case of the shape shown in FIG. 3 (c), the straight line detected from the contour line is extended to extract the intersection, and the four intersections farthest from each other among the extracted intersections are specified as corresponding points. Just do it. In the case of the shape shown in FIG. 3D, the intersection of the grids may be specified as the corresponding point. Moreover, what is necessary is just to identify each point as a corresponding point if it is the shape shown in FIG.3 (e).

  Further, the marker Mk is not limited to the above, and it is sufficient that at least four reference points can be specified and the dimensions are known. For example, the shape of the marker may be a polygon such as a triangle or a pentagon, a circle, or any other shape. The marker may be a coin, a lighter, a pencil, an artifact such as a building or a car, a person or a tree. Natural objects, intangible objects such as patterns, and other arbitrary objects may be used. Even in this case, the corresponding point specifying unit 30 may be appropriately changed in design so that the corresponding point can be specified in accordance with the shape of the marker. Moreover, in the said embodiment, although the corresponding point specific | specification means 30 specified the corresponding point from the shape of the marker, you may specify from the color feature of a marker. For example, when the marker is four black points, the black point may be recognized by image recognition and specified as a corresponding point. Further, a point designated by the user with a mouse or the like for the captured image Pc1 displayed on the display screen Dp may be specified as a corresponding point.

  Further, in each of the above-described embodiments, it is possible to enlarge or reduce the image displayed on the display unit Dp based on the user's operation, and it is necessary to add a function for changing the scale bar to be displayed accordingly. It is optional.

  Note that the dimension measuring devices 1 and 2 are particularly effective when using a photographed image photographed from an oblique direction, but do not restrict the use of the photographed image photographed from the front.

  In the above-described embodiment, the marker and the measurement target are described as planes for the sake of simplification. However, for example, a solid object such as a pencil, a lighter, a ball, or a person may be used. By using the three-dimensional object, even if an error due to the thickness of the marker or the three-dimensional object may occur, the measurement accuracy becomes higher compared to the dimension measurement in the captured image.

  In addition, a program for causing a computer to perform the processing described above can be created. For example, the program is a flexible disk, a CD-ROM, a magneto-optical disk, a semiconductor memory (for example, ROM), a hard disk, or the like. Stored in a computer-readable storage medium or storage device. Note that data being processed is temporarily stored in a storage device such as a RAM.

  Further, the present technology is not limited to this. For example, each processing flow is an example, and as long as the processing result does not change, the order of the steps may be changed or may be performed in parallel.

  Moreover, the computer which implement | achieves each dimension measuring apparatus of the said embodiment may be a general purpose portable terminal, a dedicated portable terminal, a personal computer, or a server. In the case of a general-purpose portable terminal or personal computer, an application for performing the processing in this embodiment may be acquired by downloading or the like. In the case of a dedicated mobile terminal, the application may be stored in advance in a hard disk drive or the like. These may acquire a photographed image from a built-in digital camera and output the result to a built-in display. In the case of a server, a captured image may be acquired from the user terminal via the network, and the processing result may be output to the user terminal via the network.

  Each dimension measuring device described above is a computer device, and as shown in FIG. 19, a memory 2501, a CPU 2503, a hard disk drive (HDD) 2505, and a display control unit 2507 connected to the display device 2509. A drive device 2513 for the removable disk 2511, an input device 2515, and a communication control unit 2517 for connecting to a network are connected by a bus 2519. An operating system (OS) and an application program for executing the processing in this embodiment are stored in the HDD 2505, and are read from the HDD 2505 to the memory 2501 when executed by the CPU 2503. The CPU 2503 controls the display control unit 2507, the communication control unit 2517, and the drive device 2513 according to the processing content of the application program, and performs a predetermined operation. Further, data in the middle of processing is mainly stored in the memory 2501, but may be stored in the HDD 2505. In an embodiment of the present technology, an application program for performing the above-described processing is stored in a computer-readable removable disk 2511 and distributed, and installed from the drive device 2513 to the HDD 2505. In some cases, the HDD 2505 may be installed via a network such as the Internet and the communication control unit 2517. Such a computer apparatus realizes various functions as described above by organically cooperating hardware such as the CPU 2503 and the memory 2501 described above and programs such as the OS and application programs. .

  Further, the present invention is not limited to the above embodiment, and can be appropriately changed within the scope of the invention.

DESCRIPTION OF SYMBOLS 1, 2 Dimension measurement apparatus, 10 Captured image acquisition means, 20 Marker recognition means, 30 Corresponding point specification means, 40 Perspective projection conversion means, 50 Scale calculation means, 60 Dimension calculation means, 70 Image storage means, 80 Dimension storage means, 101 instruction accepting means, 102 actual size adjusting means, 103 scale bar output means, C photographing means, Dp display means, Mk marker, Ob measurement object, Pc1 photographed image taken obliquely, Pc2 front image, A plane, p1 reference point , P2 corresponding point, Sq1, Sq2 rectangle, Rc rectangle, L0 outline, L1-L4 line, RL1-RL4 regression line, Br scale bar

Claims (6)

In a dimension measurement program used to measure the dimensions of a measurement object,
When the dimension of the marker in the real space is input by the user, a dimension storing step for storing the dimension of the marker in the storage unit;
A photographed image acquisition step of acquiring a captured image including a measurement target and the marker,
When at least four points included in the image when the front image of the marker is captured are used as reference points, the corresponding points corresponding to the reference points are identified from the captured images acquired in the captured image acquisition step. Corresponding point identification step to
The correspondence between the reference point and the corresponding point can be specified by the user so that the posture of the marker stored in the storage unit and the posture of the marker in the captured image acquired in the captured image acquisition step are aligned. And calculating a conversion parameter for perspective projection conversion so that each corresponding point specified in the corresponding point specifying step is located at the corresponding reference point in the specified correspondence relationship, A perspective projection conversion step for perspective projection conversion of the captured image based on parameters;
The size of the marker in the real space is acquired from the storage unit, and the scale of the image after the perspective projection conversion is calculated from the coordinate of the marker in the image after the perspective projection conversion and the size of the marker in the real space. A scale calculation step;
A dimension measurement program for causing a computer to execute a process including: The dimension measurement program according to claim 1, further comprising a dimension calculation step of calculating a dimension of the measurement object based on the scale calculated in the scale calculation step. The dimension measurement program according to claim 1, further comprising: an actual size adjustment step for adjusting the image after the perspective projection conversion to an actual size based on the scale calculated by the scale calculation step. A scale bar output step of outputting a scale bar representing the scale calculated in the scale calculation step in an overlapping manner with the image after the perspective projection conversion;
The dimension measurement program according to claim 1, wherein the scale bar is movable in accordance with an operation by a user. In the dimension measuring device used for measuring the dimension of the measuring object,
When the dimension of the marker in the real space is input by the user, the dimension storage means for storing the dimension of the marker in the storage means;
A captured image acquiring means for acquiring a captured image including a measurement target and the marker,
The corresponding point corresponding to the reference point is identified from the captured image acquired by the captured image acquisition means when at least four points included in the image when the front image of the marker is captured is used as the reference point. Corresponding point identification means;
The correspondence between the reference point and the corresponding point can be designated by the user so that the posture of the marker stored in the storage unit and the posture of the marker in the captured image acquired by the captured image acquisition unit are aligned. Then, a conversion parameter for perspective projection conversion is calculated so that each corresponding point specified by the corresponding point specifying unit is positioned at the corresponding reference point in the specified correspondence relationship, and the conversion parameter is set as the conversion parameter. Perspective projection conversion means for perspective projection conversion of the captured image based on;
The size of the marker in the real space is acquired from the storage unit, and the scale of the image after the perspective projection conversion is calculated from the coordinate of the marker in the image after the perspective projection conversion and the size of the marker in the real space. A scale calculating means;
A dimension measuring apparatus comprising: In a dimension measuring method executed by a computer for measuring a dimension of a measuring object,
When the dimension of the marker in the real space is input by the user, a dimension storing step for storing the dimension of the marker in the storage means;
A photographed image acquisition step of acquiring a captured image including a measurement target and the marker,
When at least four points included in the image when the front image of the marker is captured are used as reference points, the corresponding points corresponding to the reference points are identified from the captured images acquired in the captured image acquisition step. Corresponding point identification step to
The correspondence between the reference point and the corresponding point can be specified by the user so that the posture of the marker stored in the storage unit and the posture of the marker in the captured image acquired in the captured image acquisition step are aligned. And calculating a conversion parameter for perspective projection conversion so that each corresponding point specified in the corresponding point specifying step is located at the corresponding reference point in the specified correspondence relationship, A perspective projection conversion step for perspective projection conversion of the captured image based on parameters;
The size of the marker in the real space is acquired from the storage unit, and the scale of the image after the perspective projection conversion is calculated from the coordinate of the marker in the image after the perspective projection conversion and the size of the marker in the real space. A scale calculation step;
A dimension measuring method comprising: