JP6218237B2 - Image conversion program, apparatus and method for parallelizing photographed image - Google Patents

Description

  The present invention relates to a technique for parallelizing a captured image.

  “Parallelization of photographed image” refers to a technique for converting a “captured image (camera image)” photographed from an oblique angle with respect to a plane into an “overhead image” viewed from vertically above. This is a kind of three-dimensional restoration technique and restores geometric information (parallel lines and orthogonal lines) of a two-dimensional plane of a subject from a captured image. According to this technique, it is possible to easily detect specific geometric information held only for the overhead image. Simultaneously with the generation of the overhead view image, the photographing posture with respect to the subject can be acquired, and the present invention can be applied to an AR (Augmented Reality) system that displays CG (Computer Graphics) superimposed on the real space.

  2. Description of the Related Art Conventionally, there is a technique for parallelizing a captured image using point correspondence between a camera image and a bird's-eye view image (for example, see Patent Document 1). Specifically, according to this technology, a plane projective transformation matrix for converting from a minimum of four sets of point correspondences to a bird's-eye view image is calculated using correspondence between known geometric shapes existing on the subject and shapes in the camera image. To do. For example, by arranging square markers in advance on the subject, parallelization is easily realized by associating the coordinates of the four vertexes of the square with the coordinates on the overhead image and the pixel coordinates on the camera image.

  There is also a technique for parallelizing a captured image using parallel lines detected from the subject (see, for example, Patent Document 2). Specifically, according to this technique, a minimum of two sets of parallel lines existing on a plane are detected, and using these parallel lines, the disappearance line in the camera image corresponding to the infinity line on the plan view is detected. Calculate the position. The normal direction of the subject is restored from the disappearance line in the camera image and the internal parameters of the camera, and an overhead image is created.

  According to the techniques described in Patent Documents 1 and 2 described above, there is a problem in that the technique is limited to those in which a specific geometric shape or parallel line exists on a captured image (subject).

  On the other hand, there is also a technique for parallelizing a captured image using a planar projection transformation matrix between camera images captured from multiple viewpoints (see, for example, Non-Patent Documents 1, 2, and 3). This technique is significantly different from the techniques described in Patent Documents 1 and 2 described above in that a plurality of camera images are used.

  FIG. 1 is a functional configuration diagram of a conventional image conversion program.

  Specifically, according to the technique described in Non-Patent Document 1, a cost function is defined in advance with respect to a parallelization parameter of a captured image and a planar projective transformation matrix between the camera images. Then, the point correspondence based on the local feature amount is calculated from a plurality of captured images input from the camera (corresponding to the point correspondence calculation unit in FIG. 1). Next, a plane projection transformation matrix is calculated from the point correspondence (corresponding to the plane projection transformation matrix calculation unit in FIG. 1). Then, a parallelization parameter is estimated by minimizing each planar projective transformation matrix as an input.

  This technique can be applied even when there is no geometric information or parallel lines on the subject because no prior information about the plane is used. For example, for a camera image obtained by photographing a subject from an oblique direction, it is difficult to detect a parallel line because two straight lines that are parallel to each other on a plane are not parallel. Even in such a case, the captured image can be parallelized.

  In the parallelization calculation, a plurality of parallelization parameter solutions including an optimal solution are obtained. The parallelization parameter corresponds to the planar projective transformation of the input image, and an overhead image taken from the front direction is obtained by the transformation based on the optimal solution.

  Further, according to the technique described in Non-Patent Document 2, the cost values calculated for each overhead image candidate are compared, and the one having the minimum cost value is selected as the finally obtained overhead image (FIG. 1). Corresponding to the overhead image calculation unit). Furthermore, according to the technique described in Non-Patent Document 3, the ratio of the second smallest cost value to the minimum cost value is used as a criterion for outputting an overhead image having the minimum cost value.

JP 2006-146760 A Japanese Patent No. 4617993

A. Ruiz, PEL de Teruel, and L. Fernandez, Practical planar metric rectification. In Proc. BMVC 2006, 2006, [online], [Search June 3, 2014], Internet <URL: http: // www .bmva.org / bmvc / 2006 / papers / 188.pdf> C. Pirchheim and G. Reitmayr, “Homography-based planar mapping and tracking for mobile phones,” in Proceedings of the 10th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR '11), 2011, pp. 27.36, [online ], [Search June 3, 2014], Internet <URL: http://www.researchgate.net/publication/221221458_Homography-based_planar_mapping_and_tracking_for_mobile_phones> Yuya Makibuchi, Haruhisa Kato, Hiromasa Yanagihara, “Examination of Camera Posture Estimation Method Using Multiple Planes for Markerless AR,” IPSJ SIG. 2014-AVM-84 (4) , 1-6, 2014-02-14, [online], [Search June 3, 2014], Internet <URL: https://ipsj.ixsq.nii.ac.jp/ej/?action=pages_view_main&active_action = repository_view_main_item_detail & item_id = 98561 & item_no = 1 & page_id = 13 & block_id = 8> “Calculation of homography”, [online], [Search June 3, 2014], Internet <URL: http://tessy.org/wiki/index.php?%A5%DB%A5%E2%A5 % B0% A5% E9% A5% D5% A5% A3% A4% CE% B7% D7% BB% BB> “Type Document OCR Library”, [online], [Search June 3, 2014], Internet <URL: http: //mediadrive.jp/products/library/katsuji_library/windows/index.html> "Four high-performance applications that make up FAX OCR / Scanner OCR" TeleForm "", [online], [Search June 3, 2014], Internet <URL: http://www.hammock.jp/ ocr / kinou / verifier.html>

  According to the techniques described in Non-Patent Documents 1, 2, and 3, the accuracy of parallelization largely depends on the number of camera images and the accuracy of point correspondence calculation between camera images, and an erroneous overhead image candidate may be obtained. is there. In particular, the techniques described in Non-Patent Documents 2 and 3 are insufficient to compare cost values in order to determine an accurate overhead image, and there is a problem that a high-accuracy overhead image cannot be obtained. According to such a conventional technique, the optimum solution is determined by comparison between the parallelization parameters, and an incorrect parallelization parameter may be determined as the optimum solution.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an image conversion program, apparatus, and method that can parallelize an image captured by a camera with high accuracy.

According to the present invention, in an image conversion program for causing a computer to function to parallelize a captured image,
The captured image shows the identification code given to the subject.
Point correspondence calculation means for calculating point correspondence of feature points between an arbitrary representative image and each subordinate image for a plurality of captured images;
A plane projection transformation matrix calculating means for calculating a plane projection transformation matrix between the representative image and each subordinate image for each point correspondence;
An overhead image candidate calculation means for estimating a parallelization parameter using each planar projection transformation matrix and outputting a plurality of overhead image candidates obtained by converting a representative image into an overhead image using each parallelization parameter;
The computer is caused to function as a code reading determination unit that outputs a bird's-eye view image that has been successfully image-reading an identification code from among a plurality of bird's-eye view image candidates.

According to another embodiment of the image conversion program of the present invention,
The overhead image candidate calculation means preferably causes the computer to function so as to estimate a plurality of parallelization parameters determined as the local minimum solution for the parallelization parameter space.

According to another embodiment of the image conversion program of the present invention,
It is also preferable that the overhead image candidate calculation means allows the computer to function so as to select only one of the parallel parameters when a plurality of parallel parameters existing within a certain distance in the parallel parameter space are estimated. .

According to another embodiment of the image conversion program of the present invention,
It is also preferable that the overhead image candidate calculation means calculates the similarity between the overhead image candidates and causes the computer to select only the overhead image candidates whose similarity is equal to or greater than a predetermined threshold.

According to another embodiment of the image conversion program of the present invention,
It is also preferable to make the computer function so that the identification code is a bar code, a QR (Quick Response) (registered trademark) code, an OCR (Optical Character Recognition) character, or a geometric feature image such as a line segment.

According to another embodiment of the image conversion program of the present invention,
The code reading determination means preferably causes the computer to function so as to output the bird's-eye view image having the highest reliability value among the bird's-eye view image images that have been successfully read in the image form of the identification code. The confidence value is a numerical value indicating the certainty of the reading result output from the code reading determining means.

According to another embodiment of the image conversion program of the present invention,
The code reading determination means preferably causes the computer to function so as to output the identification data, which has been successfully read in the image form of the identification code, to the application.

According to the present invention, in an image conversion apparatus for parallelizing a captured image,
The captured image shows the identification code given to the subject.
Point correspondence calculation means for calculating point correspondence of feature points between an arbitrary representative image and each subordinate image for a plurality of captured images;
A plane projection transformation matrix calculating means for calculating a plane projection transformation matrix between the representative image and each subordinate image for each point correspondence;
An overhead image candidate calculation means for estimating a parallelization parameter using each planar projection transformation matrix and outputting a plurality of overhead image candidates obtained by converting a representative image into an overhead image using each parallelization parameter;
Code reading determination means for outputting, as a paralleled bird's-eye view image, a bird's-eye view image that has succeeded in image-wise reading of an identification code from among a plurality of bird's-eye view image candidates.

According to the present invention, in an image conversion method for parallelizing a captured image using a user-operated information device,
The captured image shows the identification code given to the subject.
A first step of calculating a point correspondence of feature points between an arbitrary representative image and each subordinate image for a plurality of captured images;
A second step of calculating a planar projective transformation matrix between the representative image and each dependent image for each point correspondence;
A third step of estimating a parallelization parameter using each planar projective transformation matrix and outputting a plurality of overhead image candidates obtained by converting a representative image into an overhead image using each parallelization parameter;
And a fourth step of outputting a bird's-eye view image, which has been successfully read as an identification code, from among a plurality of bird's-eye view image candidates, as a parallel bird's-eye view image.

  According to the image conversion program, apparatus, and method of the present invention, it is possible to parallelize an image captured by a camera with high accuracy.

It is a functional block diagram of the image conversion program in a prior art. It is a functional block diagram of the image conversion program in this invention. It is the picked-up image from the diagonal direction with respect to the to-be-photographed object. It is the several picked-up image image | photographed while moving the camera. It is an image correspondence figure showing inlier and outlier based on a Homography matrix. It is a cost value matrix which expresses the discretized cost value with black and white shading. Three bird's-eye view image candidates represented in correspondence with each parallelization parameter. It is the screen display figure of the application with respect to the identification data which read the identification code.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 2 is a functional configuration diagram of an image conversion program according to the present invention.

  The image conversion program according to the present invention causes a computer to function in parallel with a captured image, and is installed and executed in a terminal such as a smartphone or a tablet. Such a terminal 1 generally includes a camera module (Web camera) 101 that captures a surrounding situation and a display 102. For example, the camera module 101 outputs a plurality of images obtained by photographing a subject by continuous shooting or a moving image to an image conversion program. For the camera module 101, it is assumed that internal parameters such as a focal length, an optical axis shift, and a distortion parameter are known by prior calibration.

  The plurality of images are not limited to those taken with one camera, and may be taken from different viewpoints with a plurality of different cameras. Further, the plurality of images may be taken at a user's desired instruction timing. These photographed images are sequentially buffered, but the memory usage can be saved by limiting to only a few (for example, about 5) inputted immediately before.

  The display 102 is not limited to a display incorporated in an information device, and may be a television connected to the outside, a display for a personal computer, or a head mounted display. Such a display can display a parallel bird's-eye view image and identification code identification data.

FIG. 3 is a photographed image from an oblique direction with respect to a subject to which a barcode is assigned.
FIG. 4 shows a plurality of captured images taken while moving the camera.

  According to the present invention, it is necessary that an “identification code” that can be visually recognized is added to the subject. Therefore, the identification code assigned to the subject is reflected in the photographed image. The identification code is a geometric feature image such as a barcode (one-dimensional code), a QR code (two-dimensional code), an OCR character or a line segment. According to the present invention, such an identification code is not mainly intended for reading the identification data, but mainly intended for parallelization of the photographed image. Of course, it may be applied to the use of reading the identification data of the identification code.

  The image conversion program includes a point correspondence calculation unit 11, a plane projection conversion matrix calculation unit 12, an overhead image candidate calculation unit 13, and a code reading determination unit 14. These functional components are realized by executing a program that causes a computer installed in the apparatus to function. These functional components can be understood as the internal structure of the image conversion apparatus, and the processing flow can also be understood as an image conversion method for information equipment.

[Point correspondence calculation unit 11]
The point correspondence calculation unit 11 extracts a local feature amount from each of a plurality of captured images obtained by capturing a subject (including an identification code), and between a feature point p of an arbitrary representative image and a feature point p ′ of each subordinate image. The point correspondence P = {(p, p ′)} is calculated.

  Here, the point correspondence calculation unit 11 first selects one representative image from a plurality of photographed images, and designates the others as “subordinate images”. For example, according to FIG. 4, the last input photographed image is selected as the “representative image”, but the present invention is not limited to this. The selection of the representative image may be determined under various criteria.

For example, SIFT (Scale-Invariant Feature Transform), SURF (Speeded Up Robust Features), or ORB (Oriented FAST and Rotated BRIEF) is used as an extraction algorithm for local feature points. These local feature points are described by the following elements.
Coordinate p = (x, y), direction θ, local feature vector f
It should be noted that the feature vectors have the same number of dimensions between captured images for which point correspondence is detected.

  For example, in the case of SIFT, a 128-dimensional feature point set is extracted from one image. SIFT is a technique for analyzing a characteristic local region using a scale space and describing a feature vector that is invariant to scale change and rotation. On the other hand, in the case of SURF, higher-speed processing is possible than in SIFT, and a 64-dimensional feature point set is extracted from one image. In addition, the ORB extracts a set of 256-bit binary feature vectors from one content by using BRIEF (Binary Robust Independent Elementary Features) as a feature description by a binary code. In particular, according to the ORB, it is possible to maintain an accuracy equal to or higher than that of SIFT or SURF and realize a speed increase of several hundred times.

  Next, the point correspondence calculation unit 11 matches the feature point set of the representative image with the feature point set of the subordinate image, and associates the local feature points with the points. The feature point p ′ of each subordinate image is associated with the feature point p of the representative image. The shorter the distance between the two feature points p ′ and p, the higher the similarity.

The point correspondence calculation unit 11 preferably uses the following method in order to improve the accuracy and processing speed of the point correspondence calculation.
(1) Sort point correspondences based on the distance between local feature points, and use only point correspondences whose distance is equal to or less than a predetermined threshold.
(2) Sort the point correspondences based on the distance between the local feature points, search for the closest distance and the second closest distance, and the ratio of those distances (the first and the second distance to the second distance) ) Is less than a predetermined threshold.
(3) In addition, binary feature quantities that are excellent in calculation cost, SSD (Sum of Squared Difference), normalized cross correlation (NCC), and the like can also be used.

  Then, the point correspondence calculation unit 11 outputs the point correspondence of the feature points between the representative image and each of the subordinate images to the planar projective transformation matrix calculation unit 12.

[Plane Projection Transformation Matrix Calculation Unit 12]
The plane projection transformation matrix calculation unit 12 calculates a plane projection transformation matrix between the representative image and each subordinate image for each point correspondence.

  Specifically, the plane projective transformation matrix calculation unit 12 uses the robust correspondence algorithm such as RANSAC (RAndom SAmple Consensus) from the point correspondence P = {(p, p ′)} set to represent the subordinate image as the representative image. A plane projective transformation matrix to be converted to is calculated. As a result, erroneous point correspondences can be removed. The transformation matrix is preferably a “Homography matrix” and requires at least four pairs of points. If all four pairs of points are not inlier, a correct Homography matrix cannot be obtained.

  FIG. 5 is an image correspondence diagram showing inliers and outliers based on the Homography matrix.

  The feature point set of the subordinate image is projected using the calculated Homography matrix, and the corresponding pair whose Euclidean distance from the feature point of the representative image is equal to or less than a predetermined threshold is determined as positive (inlier), and the other is out (outlier) Judge as. If the number of corresponding pairs determined to be inlier is equal to or greater than a predetermined threshold, it is determined that the representative image has been detected, and the Homography matrix is adopted. On the contrary, when the number of corresponding pairs determined to be inlier is smaller than a predetermined threshold, it is determined as not detected, and the homography matrix is removed.

A Homography matrix H that is a planar projective transformation matrix is expressed as follows (for example, see Non-Patent Document 4). This represents the relationship between the feature point p ′ = (x ₁ , y ₁ ) of the subordinate image and the feature point p = (x ₂ , y ₂ ) of the representative image.

  For the calculation of the Homography matrix, the feature point set of the representative image and the feature point set of the subordinate image are used. The number of unknown parameters in the Homography matrix H is 8 (h0 to h7, h8 = 1), and one set of corresponding points gives two constraint equations. Therefore, this matrix H can be calculated by the least square method if there are four or more pairs of corresponding points. Such a technique for estimating the camera posture is common in the AR system.

Then, when each captured image feature point is projected using the Homography matrix H, the determination is made as follows.
(1) If it is projected close to a predetermined threshold or less with respect to the feature point of the target image, it is determined as inlier.
(2) Conversely, if it is projected farther than the predetermined threshold, it is determined as outlier. According to FIG. 5, outlier is represented by a broken line.
After this process is executed a plurality of times, only the Homography matrix H in which the number of inliers is equal to or greater than a predetermined threshold is employed.

The planar projective transformation matrix H _j is represented by the following relational expression.
I _i (i = 1, 2, 3..., N): N shooting coordinates m _i (i = 1, 2, 3..., N): Pixel coordinate system of shooting coordinates I _i I ₁ : Representative image K: Camera internal parameters x _i : Pixel coordinate system normalized using K Relational expressions: x _{i to} K ⁻¹ m _i
(M _i and x _i are each homogeneous coordinate representation)
~: Scale indefinite And a plane projection transformation matrix H _j that satisfies the relational expressions x _{j to} H _j x ₁ (j = 2, 3,..., N) is calculated.

  Such processing is repeated for the representative image and each subordinate image. Each time it is detected as a representative image shown in the subordinate image, its planar projective transformation matrix H is output.

[Overhead Image Candidate Calculation Unit 13]
The overhead image candidate calculation unit 13 estimates a parallelization parameter using each planar projection transformation matrix, and outputs a plurality of overhead image candidates obtained by converting a representative image into an overhead image using each parallelization parameter.

I _0: overhead image m of the object _0: overhead image I pixel coordinate system x _{0 0:} normalized pixel coordinate system of the bird's-eye view image I ₀ and defines a collimating matrix C satisfying the relation x ₁ ~Cx _0.
Further, the representative image I ₁ is converted into the overhead image I ₀ by the inverse matrix C ⁻¹ of the parallelization matrix C.

The parallelization matrix C is constituted by a biaxial rotation matrix and is expressed as follows.
C (φ ₁ , φ ₂ ) = R _Z (φ ₁ ) R _Z (φ ₂ )
(Φ ₁ , φ ₂ ): A cost function is defined as follows for the parallelization parameter H _j representing the rotation amount of each axis.
e _j (φ ₁ , φ ₂ ) = (h ₁₂ / h ₁₁ ) ² + (h ₂₂ / h ₁₁ −1) ² where (C ^T H _j ^T ) (H _j ^T C ^T )
= {H _uv }

The parallelization parameters (φ ₁ , φ ₂ ) are searched for as minimizing the sum of the cost values calculated for all input planar projective transformation matrices.
_{argmin φ1, φ2 (E (φ} 1, φ 2)) = Σ j = 2 N e j (φ 1, φ 2)
Range of solution of parallelization parameter Φ = (φ ₁ , φ ₂ ): [(-π / 2, π / 2), (-π / 2, π / 2)]

  A global optimal solution is searched for this parameter space.

(S131) First, the cost value is sampled according to the above equation at regular intervals (for example, 10 °). As a result, a cost value matrix having cost values for each element is obtained.

  FIG. 6 is a cost value matrix representing the discretized cost values with black and white shading. Black represents E = 0 value.

(S132) Next, the overhead image candidate calculation unit 13 estimates a plurality of parallelization parameters determined to be the local minimum solution for the parallelization parameter space. The “local minimum solution” is detected by comparing the value of each element with the surrounding eight neighboring elements. In order to reduce the calculation cost, it may be compared with four adjacent elements on the top, bottom, left and right. For each parameter of the local minimum solution, the initial value is used to refine the cost value by an optimization method. For example, the downhill simplex method can be used. According to FIG. 6, three refined local minimum solutions Φ ₁ , Φ ₂ and Φ ₃ are represented.

(S133) Then, for each of the plurality of local minimum solutions, the bird's-eye view image candidate calculation unit 13 converts the representative image into a bird's-eye view image using each parallelization parameter and outputs a plurality of bird's-eye view image candidates.
m ₀ to (KC (φ ₁ , φ ₂ ) K ⁻¹ ) ⁻¹ m ₁
A feature of the present invention is that the overhead image candidate calculation unit 13 outputs a plurality of overhead image candidates.

  For example, according to the techniques described in Non-Patent Documents 1 to 3 (see FIG. 1), the overhead image candidate calculation unit attempts to derive one local minimum solution. However, the accuracy of parallelization largely depends on the number of captured images and the accuracy of point correspondence calculation between captured images, and an erroneous overhead image candidate may be obtained. This is because an incorrect parallelization parameter may be determined as an optimal solution. Therefore, the overhead image candidate calculation unit 13 of the present invention outputs an overhead image as a candidate for each of the plurality of calculated local minimum solutions.

Note that the overhead image candidate calculation unit 13 of the present invention does not necessarily need to output the overhead image candidates corresponding to all the parallelization parameters. For example, you may select as follows.
(Selection 1) The bird's-eye view image candidate calculation unit 13 selects only one parallel parameter when a plurality of parallelization parameters existing within a certain distance in the parallelization parameter space are estimated. Since the parallelization parameter is two-dimensional, the distance between the parallelization parameters may be determined by the Euclidean distance.
(Selection 2) The overhead image candidate calculation unit 13 calculates the similarity between the overhead image candidates, and selects only the overhead image candidates whose similarity is equal to or greater than a predetermined threshold. As the similarity, for example, SSD (Sum of Squared Difference) or normalized cross correlation (NCC) may be used.

[Code reading determination unit 14]
The code reading determination unit 14 outputs a bird's-eye view image that has succeeded in image-wise reading of the identification code from among a plurality of bird's-eye view image candidates as a parallelized bird's-eye view image. That is, the code reading determination unit 14 has a function of a code reader (bar code reader, QR code reader, etc.).

  FIG. 7 shows three bird's-eye view image candidates corresponding to each parallelization parameter.

The code reading determining unit 14 determines “reading success / failure” (and reliability value) for the identification code for each of the plurality of overhead image candidates. The code reading determination unit 14 outputs an overhead image that has been “read successfully” among a plurality of overhead image candidates. It is also preferable to output a bird's-eye view image that is “reading success” and has the highest reliability value. “Reliability value” refers to the certainty of the recognition result (read code data) output from the code reader (see, for example, Non-Patent Document 5 or 6). For example, according to the example of FIG. 7, Φ ₁ is output as an overhead image.

  FIG. 8 is a screen display of an application for the identification data obtained by reading the identification code.

  According to FIG. 8, the overhead image output by the code reading determination unit 14 is displayed on the display 102 of the terminal 1. In addition, the identification data on which the identification code has been successfully read by the code reading determination unit 14 is displayed by the application. When the identification data is, for example, a URL (Uniform Resource Locator), the URL can be accessed by a user's tap.

  As described above in detail, according to the image conversion program, apparatus, and method of the present invention, it is possible to parallelize a captured image by a camera with high accuracy.

  Various changes, modifications, and omissions of the above-described various embodiments of the present invention can be easily made by those skilled in the art. The above description is merely an example, and is not intended to be restrictive. The invention is limited only as defined in the following claims and the equivalents thereto.

DESCRIPTION OF SYMBOLS 1 Terminal 101 Camera 102 Display 11 Point correspondence calculation part 12 Planar projection transformation matrix calculation part 13 Overhead image candidate calculation part 14 Code reading determination part

Claims (9)

In an image conversion program that causes a computer to function to parallelize a captured image,
The photographed image includes an identification code given to the subject,
Point correspondence calculation means for calculating point correspondence of feature points between an arbitrary representative image and each subordinate image for a plurality of captured images;
A plane projection transformation matrix calculating means for calculating a plane projection transformation matrix between the representative image and each subordinate image for each point correspondence;
An overhead image candidate calculation means for estimating a parallelization parameter using each planar projection transformation matrix and outputting a plurality of overhead image candidates obtained by converting a representative image into an overhead image using each parallelization parameter;
An image conversion program that causes a computer to function as a code reading determination unit that outputs a bird's-eye view image that has been successfully image-reading an identification code from among a plurality of bird's-eye view image candidates. The image conversion program according to claim 1, wherein the overhead image candidate calculation unit causes the computer to function to estimate a plurality of parallelization parameters determined to be local minimum solutions for the parallelization parameter space. The overhead image candidate calculation means causes the computer to function so as to select only one parallel parameter when a plurality of parallel parameters existing within a certain distance in the parallel parameter space are estimated. The image conversion program according to claim 1 or 2, characterized in that The bird's-eye view image candidate calculation means calculates a similarity between the bird's-eye view image candidates, and causes the computer to function so as to select only the bird's-eye view image candidates whose similarity is equal to or greater than a predetermined threshold. The image conversion program according to any one of 1 to 3. 5. The computer according to claim 1, wherein the identification code is a bar code, a QR (Quick Response) code, an OCR (Optical Character Recognition) character, or a geometric feature image such as a line segment. The image conversion program according to any one of the above. 2. The code reading determination unit causes the computer to function so as to output an overhead image having the highest reliability value among the overhead image candidates for which the identification code has been successfully image-read. 6. The image conversion program according to any one of items 5. The image according to any one of claims 1 to 6, wherein the code reading determination unit causes the computer to function so as to output the identification data of which the identification code has been successfully read as an image to an application. Conversion program. In an image conversion device that parallelizes a captured image,
The photographed image includes an identification code given to the subject,
Point correspondence calculation means for calculating point correspondence of feature points between an arbitrary representative image and each subordinate image for a plurality of captured images;
A plane projection transformation matrix calculating means for calculating a plane projection transformation matrix between the representative image and each subordinate image for each point correspondence;
An overhead image candidate calculation means for estimating a parallelization parameter using each planar projection transformation matrix and outputting a plurality of overhead image candidates obtained by converting a representative image into an overhead image using each parallelization parameter;
An image conversion apparatus comprising: a code reading determination unit that outputs, from a plurality of bird's-eye view image candidates, a bird's-eye view image that has been successfully read as an identification code as a parallel bird's-eye view image. In an image conversion method for parallelizing a captured image using a user-operated information device,
The photographed image includes an identification code given to the subject,
A first step of calculating a point correspondence of feature points between an arbitrary representative image and each subordinate image for a plurality of captured images;
A second step of calculating a planar projective transformation matrix between the representative image and each dependent image for each point correspondence;
A third step of estimating a parallelization parameter using each planar projective transformation matrix and outputting a plurality of overhead image candidates obtained by converting a representative image into an overhead image using each parallelization parameter;
And a fourth step of outputting, from the plurality of bird's-eye view image candidates, the bird's-eye view image that has succeeded in image-wise reading of the identification code as a parallel bird's-eye view image, .