JP4970385B2 - Two-dimensional code reader and program thereof - Google Patents

Description

  The present invention relates to, for example, a two-dimensional code reading apparatus that captures an image of a printed two-dimensional code with a camera and decodes the two-dimensional code based on the image data, and a program thereof.

  In recent years, a two-dimensional code represented by a QR code displayed on a printed matter such as a magazine is imaged by a camera of a mobile phone, for example, and a URL (Uniform Resource Locator) included in the two-dimensional code is recognized from the image data. Then, a service that allows information to be acquired from a corresponding Web server by accessing the Internet based on this URL has become widespread. When this type of service is used, the user does not need to input the URL by key operation, which is very convenient.

  However, when the two-dimensional code is captured by the camera of the mobile phone, if the camera does not face the two-dimensional code, the captured image of the two-dimensional code is distorted into a trapezoidal shape and the two-dimensional code cannot be recognized correctly. Sometimes. Therefore, conventionally, a two-dimensional code reading device having a function of correcting the shape of the captured two-dimensional code has been proposed. For example, as shown in FIG. 19, a square ABCD is set around the captured two-dimensional code image PC, and AP: PB = CP ′ for s and t satisfying 0 ≦ s ≦ 1, 0 ≦ t ≦ 1. : PP ′ and QQ ′ are set such that P′D = s: 1−s, AQ: QC = BQ ′: Q′D = t: 1−t. As a result, the intersection R (s, t) between PP ′ and QQ ′ is obtained for any s and t. Here, a part of the two-dimensional code can be inspected by changing s from 0 to 1 with respect to a certain t. Furthermore, it becomes possible to inspect the entire two-dimensional code by setting the inspection line for t from 0 to 1, and the cell position of the data area included in the image is determined (for example, Patent Document 1). See).

JP 2006-155218 A

  However, although the devices proposed so far are effective for trapezoidal distortion, a captured image of a two-dimensional code is obtained, for example, due to the influence of the distortion of the optical system of the camera or the distortion of the surface of the printed material. As shown in FIGS. 18A and 18B, when the pincushion type or barrel type is distorted, such curvilinear distortion cannot be corrected, and as a result, highly accurate recognition can be performed. There wasn't.

  The present invention has been made paying attention to the above circumstances, and the purpose thereof is to effectively correct the curvilinear distortion even when the two-dimensional code image has curvilinear distortion, and decode the code with high accuracy. Is to provide a two-dimensional code reading apparatus and its program.

In order to achieve the above object, according to a first aspect of the present invention, image data of a two-dimensional code is fetched from an imaging means and stored in a memory. Then, a plurality of finder patterns included in the two-dimensional code are first detected from the image data read from the memory, and a boundary point of a side representing the outline of each of the detected plurality of finder patterns is detected. Next, on the basis of the detected boundary points of the plurality of finder patterns, an approximate line is generated by approximating at least one of a straight line and a curve of the shape of the four sides representing the contour of the image data of the two-dimensional code. Then, the distortion of the image data of the two-dimensional code is corrected based on the generated approximate line, and the image data of the two-dimensional code whose distortion is corrected is stored in the memory. Then, it configured to output image data of the two-dimensional code which the distortion is corrected by decoding the two-dimensional code based.
As means for approximating the shape of the four sides of the image data of the two-dimensional code and detecting the boundary lines of the four sides, the image data of the two-dimensional code is based on the boundary points of the sides of the plurality of finder patterns. The inclination of the side representing the contour is calculated, and pixel search is performed in a direction inferred from the calculated inclination of the side to detect a boundary point representing the side of the image data of the two-dimensional code. Then, for each of the detected boundary points, it is determined whether or not the boundary point is included within the range estimated from the calculated side inclination, and the boundary determined not to be included Means for deleting a point from the boundary point as an error point is used.

  Therefore, even if the captured image of the two-dimensional code has, for example, a pincushion-type or barrel-type curved distortion, the shape of the four sides representing the contour of the image data of the two-dimensional code is approximated by at least one of a straight line and a curve. The distortion of the image data of the two-dimensional code is corrected based on the approximate line. For this reason, it is possible to perform code decoding with high accuracy even for a two-dimensional code image having the above-described pincushion-type or barrel-type curved distortion.

In order to achieve the above object, a second aspect of the present invention provides a plurality of finder patterns when detecting the boundary lines of the four sides by approximating the shape of the four sides of the image data of the two-dimensional code. The inclination of the side representing the contour of the image data of the two-dimensional code is calculated based on the boundary point of the two sides, the pixel search is performed in the direction inferred from the calculated side inclination, and the two-dimensional code is calculated. A boundary point representing a side of the image data is detected, and error points included in the detected boundary point are removed. At this time, as means for removing the error point, the intersection of the first and second sides orthogonal to each other among the four sides of the image data of the two-dimensional code is calculated, and the boundary located in the range outside the intersection A means for deleting a point as an error point is used.
Further, as the intersection calculation means, first, a threshold line is drawn using one or more boundary points included in the boundary point set representing the first side as a calculation reference point, and the boundary point set representing the second side It is determined whether the boundary point included in is located on the intersection side of the threshold line. Then, a boundary point closest to the intersection point is detected as an out-threshold proximity point from among boundary points not located on the intersection side of the threshold line in the boundary point set representing the second side. Next, a straight line obtained by estimating the slope of the first side from the calculation reference point, and a straight line obtained by estimating the slope of the second side from the out-of-threshold proximity point of the second side The intersection point candidate point is calculated based on the intersection point of the boundary point, and the threshold value is selected from the boundary points located on the intersection side of the threshold line among the boundary points included in the boundary point set representing the first side. The boundary point closest to the straight line is detected as an in-threshold proximity point, and among the boundary points included in the boundary point set representing the second side, the boundary point located on the intersection side of the threshold line The boundary point closest to the threshold straight line is detected as a proximity point within the threshold. Of the detected first side in-threshold proximity point and the detected second side in-threshold proximity point, those outside the calculated intersection candidate point are A means for determining which one is used is used .
By doing in this way, it becomes possible to delete the error points included in the set of boundary points searched on the four sides of the image data of the two-dimensional code even more reliably.

Further, according to a third aspect of the present invention, the following processing is executed when correcting the distortion of the two-dimensional code. That is, first, a vertex frame connecting the vertices of the image data of the two-dimensional code is generated. Next, in order to calculate the pixel value of the target coordinate in the image data of the two-dimensional code to be corrected, the coordinate value of the point corresponding to the target coordinate on the upper side of the generated vertex frame and the target value on the lower side The coordinate value of the point corresponding to the coordinate, the coordinate value of the point corresponding to the target coordinate on the left side, and the coordinate value of the point corresponding to the target coordinate on the right side are calculated. At the same time, in the image data of the two-dimensional code, the coordinate value of the boundary point corresponding to the target coordinate on the upper side, the coordinate value of the boundary point corresponding to the target coordinate on the lower side, and the target coordinate on the left side The coordinate value of the boundary point to be calculated and the coordinate value of the boundary point corresponding to the target coordinate on the right side are calculated. Further, a target coordinate value corresponding to the target coordinate is calculated in the vertex frame. Then, from the point corresponding to the target coordinate calculated for each of the upper, lower, left and right sides of the vertex frame, to the boundary point corresponding to the target coordinate calculated for each of the upper, lower, left and right sides of the image data of the two-dimensional code. To calculate the two-dimensional code distortion vector, add the calculated two-dimensional code distortion vector to the target coordinate value calculated for the vertex frame, and obtain the image data of the two-dimensional code. A target coordinate value corresponding to the target coordinate at is calculated.
By performing the above processing, it is possible to correct the image data of the two-dimensional code having a curved distortion into a rectangle.

  That is, according to one aspect of the present invention, even when a two-dimensional code image has a curvilinear distortion, a two-dimensional code reading apparatus that effectively corrects the curvilinear distortion and enables high-accuracy code decoding, and its A program can be provided.

Embodiments according to the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing the hardware and software configurations of a two-dimensional code reader according to an embodiment of the present invention.
The two-dimensional code reader according to the present embodiment includes a central processing control unit 1, a camera 2, an input device 3, and a display device 4. When the two-dimensional code reader is provided in the mobile phone, the central processing control unit 1, the camera 2, the input device 3, and the display device 4 all use existing configurations provided in the mobile phone.

  The central processing control unit 1 includes a CPU (Central Processing Unit) 11. A program memory 13, a camera interface (camera I / F) 14, and an input / output interface (input I / F) are connected to the CPU 11 via a bus 12. 15) and the data memory 16 are connected.

  The camera 2 is connected to the camera I / F 14. The camera 2 uses a semiconductor image sensor such as a CCD or CMOS. The camera I / F 14 activates the camera 2 under the control of the CPU 11 and takes in the image data of the two-dimensional code captured by the camera 2.

  The input device 3 and the display device 4 are connected to the input / output I / F 15. The input device 3 includes a dial key and a plurality of function keys. The key configuration may be a mechanical key or a software key displayed on the display device 4. The display device 4 is composed of a display using, for example, an LCD or an EL. The input / output I / F 15 takes in the operation data of the input device 3 and displays the display data on the display device under the control of the CPU 11.

  The data memory 16 stores data necessary for carrying out the two-dimensional code reading process according to the present invention in addition to an area for storing various data such as a telephone directory, transmission / reception history, and transmission / reception mail necessary for transmission / reception of the mobile phone. An area for primary storage is provided. Data necessary for carrying out the two-dimensional code reading process includes two-dimensional code image data captured by the camera 2, its binary image data, and various data generated in the process.

  The program memory 13 includes, as application programs necessary for carrying out the two-dimensional code reading process according to the present invention, a binarized image generation program 131, a finder pattern detection program 132, an image rotation processing program 133, A finder pattern boundary detection program 134, a two-dimensional code boundary calculation program 135, and a two-dimensional code distortion correction program 136 are stored.

  The binarized image generation program 131 takes in the image data of the two-dimensional code captured by the camera 2 via the camera I / F 14 and temporarily stores it in the data memory 16, and reads out the image data from the data memory 16. The CPU 11 is caused to execute a process of converting into binarized image data and storing the converted binarized image data in the data memory 16.

  The finder pattern detection program 132 reads the binarized image data from the data memory 16, and from the binarized image data, a plurality of finder patterns (for example, three in the case of QR code) included in the two-dimensional code. The CPU 11 is caused to execute processing for detection.

  The image rotation processing program 133 rotates the binarized image data based on the position of the finder pattern detected by the finder pattern detection process so that the image of the two-dimensional code is in an upright state. The CPU 11 is caused to execute a process for storing the binarized image data after the rotation process in the data memory 16.

  The finder pattern boundary detection program 134 reads the binary image data after the rotation processing from the data memory 16 and detects boundary points representing the sides of the black frame of each finder pattern from the read binary image data. The CPU 11 is caused to execute processing for storing the coordinate values in the data memory 16.

  The two-dimensional code boundary calculation program 135 reads the coordinate value of the boundary point representing the side of the black frame of each detected finder pattern from the data memory 16, and based on the read coordinate value of the boundary point, the two-dimensional code An approximate line is generated by approximating the shape of each of the upper, lower, left and right sides representing the outline of the line by a straight line and a curve. Then, it is determined whether the generated approximate line or approximate curve is appropriate, and the approximate line determined to be valid is used as a boundary line representing the shape of each of the upper, lower, left and right sides of the two-dimensional code. The CPU 11 is caused to execute processing for storing the coordinate values in the data memory 16.

  The two-dimensional code distortion correction program 136 reads from the data memory 16 coordinate values representing the boundary lines of the upper, lower, left and right sides of the two-dimensional code image calculated by the two-dimensional code boundary calculation program 135. The CPU 11 is caused to execute processing for correcting the distortion of the binarized image data of the two-dimensional code image based on the boundary lines of the upper, lower, left and right sides.

Next, a two-dimensional code reading process operation by the apparatus configured as described above will be described. 2 to 5 are flowcharts showing the processing procedure and processing contents of the CPU 11.
(1) Capture of image data of two-dimensional code When the user performs an imaging operation on the input device 3, the CPU 11 detects the imaging operation via the input / output I / F 15 in step S11. Then, the process proceeds to step S12, the camera 2 is activated via the camera I / F 14, and the image data of the two-dimensional code captured by the camera 2 is captured via the camera I / F 14 and stored in the data memory 16. Let

The image data of the two-dimensional code is composed of a plurality of pixels. Here, if the width of the captured image data is width and the height is height, the captured image data is constituted by width × height pixels. Each pixel is data representing a color in the RGB color model, and is composed of an x-coordinate value, a y-coordinate value, and RGB values in the frame. The x-coordinate value is an integer value of 0 or more and less than width, and the y-coordinate value is It is an integer value of 0 or more and less than height, and the x-axis is directed from left to right and the y-axis is directed from top to bottom. Each of the RGB values is an integer value of 0 or more and 255 or less. Here, when the RGB values at the coordinates (x, y) are (r, g, b),
p (x, y) = (r, g, b)
It is expressed.

(2) Processing for generating binarized image data When the image data of the two-dimensional code is fetched, the CPU 11 activates the binarized image generating program 131 in step S2 to emphasize the contrast of the image data of the two-dimensional code. In order to do this, the process of converting the image data of the two-dimensional code into binarized image data is executed as follows.

That is, first, in step S21, the image data of the two-dimensional code is read from the data memory 16, and the luminance value of each pixel of the image data is calculated to make it gray scale. For example,
Grayscale (x, y) = RGBtoLuminance (r, g, b)
The luminance value is calculated by the above to generate a gray scale image. It is assumed that the luminance value has a value between 0 and 255.

Subsequently, in step S22, the maximum luminance value maxL and the reuse luminance value minL of the generated grayscale image are detected, respectively, and the detected maximum luminance value maxL and reuse luminance value minL are used as follows. A contrast-enlarged image is generated by the expression (1) shown below.

Furthermore, in step S23, for example, the threshold value is set to 128, and the processing shown in the following equation (2) is executed to generate binarized image data. Then, the generated binarized image data is stored in the data memory 16.

(3) Finder Pattern Detection Processing When the generation of the binarized image data is completed, the CPU 11 next activates the finder pattern detection program 132 in step S3, and the three included in the QR code from the binarized image data. The process for detecting the position pattern, i.e., the finder pattern is executed as follows.

  That is, first in step S31, the binarized image data is read from the data memory 16, and the y-axis coordinate value of the binarized image data is fixed and scanned in the x-axis direction. When this scanning detects a line pattern of “white”, “black”, “white”, “black”, “white” as shown in FIG. 6, for example, each “white” and each “black” Confirm each length, and the ratio of the length from the second “black” to the sixth “black”, B1: W1: B2: W2: B3 becomes 1: 1: 3: 1: 1 The detected value is stored in the data memory 16.

を満たす箇所を検出する。例えば、ｔ＝０．３等の値を用いる。 In particular,
L = B1 + W1 + B2 + W2 + B3
For a threshold t greater than or equal to 0

Detect locations that satisfy For example, a value such as t = 0.3 is used.

  Similarly, the line pattern detection processing in step S31 is executed for all coordinate values in the y-axis direction. When it is confirmed in step S32 that the detection values of all the line patterns have been obtained, the process proceeds to step S33, and the line patterns that are in contact with each other among the detected line patterns are segmented. Are recognized as QR code finder patterns, and their coordinate values are stored in the data memory 16.

(4) Image Rotation Processing When the detection of the three finder patterns is completed, the CPU 11 subsequently proceeds to step S4 to start the image line processing program 133, and based on the position of the detected finder pattern, the CPU 11 The binarized image data is rotated as follows.
That is, first, in step S41, the centers of gravity of the three detected finder patterns F1, F2, and F3 are calculated. Then, D1, D2, and D3 are calculated by the following equation (4), assuming that the calculated center of gravity is W1 = (u1, v1), W2 = (u2, v2), and W3 = (u3, v3).

  Based on the calculation results of D1, D2, and D3, the upper left, upper right, and lower left finder patterns are recognized in step S42. For example, F1 is recognized as the upper left finder pattern when D1 is the maximum, F2 when D2 is the maximum, and F3 when D3 is the maximum. This process utilizes the fact that the distance between the upper right and upper left viewfinder patterns is maximized. FIG. 7 shows an example in which D1 is maximized.

  Subsequently, the upper right finder pattern and the lower left finder pattern are recognized. When W1 is on the right side with respect to the straight line passing through W2 and W3, F2 is set as the upper right finder pattern, and F3 is set as the lower left finder pattern. Is the upper right finder pattern, and F2 is the lower left finder pattern.

  When the upper left, upper right, and lower left finder patterns are recognized, the CPU 11 determines in step S43 that the upper right finder pattern FPRT is positioned in the x-axis direction of the upper left finder pattern FPLT, for example, as shown in FIG. The binary image data of the two-dimensional code image is rotated.

(5) Finder Pattern Boundary Detection Processing Next, the CPU 11 activates the finder pattern boundary detection program 134 in step S5, and the boundary representing the sides of the black frames of the three finder patterns from the binary image data after the rotation processing. The point detection process is executed as follows.
That is, in step S51, the CPU 11 first extracts a black frame segment S as shown in FIG. 9, for example, and calculates the coordinates of the four corners of the black frame. Of the coordinates included in the segment S, the coordinates of the four corners are the upper left corner (cx1, cy1), the coordinate having the maximum −xy, and the upper right corner (cx2, cy2), the coordinate having the maximum xy. ), The coordinates where y-x is maximum are set to the lower left corner (cx3, cy3), and the coordinates where xy is maximum are set to be the lower right corner (cx4, cy4).

として検出される。 Next, in step S52, the CPU 11 detects the upper, lower, left and right sides of the black frame segment. For example, FinderTop on the upper side

Detected as

として検出される。 As shown in FIG. 10, this represents a set of points at which y is minimum for x between the upper left corner and the upper right corner in the segment S of the black frame. Similarly, the lower FinderBottom, the left FinderLeft, and the right FinderRight are

Detected as

(6) Calculation process of two-dimensional code boundary line Next, the CPU 11 proceeds to step S6 to start the two-dimensional code boundary line calculation program 135, and the upper, lower, left and right sides of the black frames of the three detected finder patterns. Based on these coordinate values, the coordinates included in each of the upper, lower, left and right sides representing the contour of the two-dimensional code image are detected, and the approximate line is calculated as the boundary line of the upper, lower, left and right sides.

  For example, when detecting the boundary point coordinates of the upper side of the two-dimensional code image, the CPU 11 performs the following processing in steps S61 to S68. That is, the upper left finder pattern FPLT is located on the left side of the upper side, and the upper right finder pattern FPRT is located on the right side. Since the boundary point coordinates of the upper sides of the respective finder patterns FPLT and FPRT have already been detected by the finder pattern boundary detection processing described above, the two-dimensional position between the finder patterns FPLT and FPRT is utilized using the detection result. A set of boundary points on the upper side of the code image is detected in step S62.

と推測できる。 Here, the set of boundary points on the upper side of the left finder pattern FPLT is LeftFinderEdge, the set of boundary points on the upper side of the right finder pattern FPRT is RightFinderEdge, the right end boundary point of LeftFinderEdge is (LeftEndX, LeftEndY), and the left end of RightFinderEdge Let the boundary point of be (RightStartX, RightStartY). Further, the center of gravity of the boundary point included in LeftFinderEdge is (LeftCenterX, LeftCenterY), and the center of gravity of the coordinate point included in RightFinderEdge is (RightCenterX, RightCenterY). Further, a straight line approximated from the LeftFinderEdge by the least square method is y = alx + bl, and a straight line approximated from the RightFinderEdge by the least square method is y = arx + br. From the straight line obtained by this least square method, the slope of the upper side edgeSlope (x) at a certain x coordinate is

Can be guessed.

として求める。なお、定数については、例えばcq=5、cr=0、dq=0.1、dr=0.1等の値を用いる。 Next, the CPU 11 detects the upper side of the two-dimensional code image between LeftEndX and RightStartX in the binarized image data read from the data memory 16. For example, as shown in FIG. 10, the detection processing is performed by calculating a point on the upper side where the x coordinate is px + k (k = 1 in the initial condition) with respect to the detected point P = (px, py) on the upper side. Set qy and ry to detect. Then, the first boundary point that changes from “white” to “black” in the downward direction from qy to ry is searched, and the found boundary point P ′ = (px + k, py ′) is detected as a boundary point on the new upper side. . Where qy and ry are constants cq, cr, dq, dr

Asking. For the constant, for example, values such as cq = 5, cr = 0, dq = 0.1, dr = 0.1 are used.

  On the other hand, if no new boundary point is found by the above processing, the search is performed by increasing the value of k by 1. If found, the same processing is performed on P ′ from k = 1. Search for the boundary point of. The search starts from P = (LeftEndX, LeftEndY), and the search ends when px + k = RightStartX. Through the above processing, a set of boundary points MiddleEdge on the upper side of the two-dimensional code located between the upper left finder pattern FPLT and the upper right finder pattern FPRT is detected.

  The boundary point set MiddleEdge on the upper side of the two-dimensional code detected by the process in step S62 includes points inside the two-dimensional code image from the original upper side. Therefore, the CPU 11 subsequently removes points in the two-dimensional code image. This error point removal method includes a first method for removing points that are not included in the range estimated from the slope of the upper side as an error, an intersection point of two sides orthogonal to each other, and a range outside the intersection point. There is a second method of deleting a boundary point located at a point as an error point. Here, the first method will be described first, and the second method will be described in detail later.

と表される。 The first method is performed as follows. That is, the CPU 11 performs error point removal processing from right to left. Here, with respect to the point P = (px, py), the first point included in the MiddleEdge and positioned to the left of P is P ′ = (px ′, py ′). At this time, the slope of the line segment PP ′ is

It is expressed.

と定義する。 Here, a function edgeMinSlope (x) that returns a slope smaller than the slope of the upper side at a certain point by a constant ct (for example, 0.1)

It is defined as

によって判定する。この式は、

とも表現される。 Here, whether P ′ is an error or not

Judgment by. This formula is

It is also expressed.

  In this determination process, as shown in FIG. 11, assuming that the slope of the line segment PQ is the slope of the estimated upper side, and the slope of the line segment PR is a slope smaller than the slope of PR by ct, P1 ′ and P2 ′ A point located above R such as is not an error, but a point located below R such as P3 ′ is detected as an error. However, in FIG. 11, the x coordinate values of Q, R, P1 ′, P2 ′, and P3 ′ are all px ′. By the above procedure, error points are removed from MiddleEdge.

と表される。 By the above procedure, the boundary point set on the upper side of the two-dimensional code image is detected by LeftFinderEdge, MiddleEdge, and RightFinderEdge. Where the top edge point set TopEdge is

It is expressed.

によって求まる。 Next, in steps S64 and S65, the CPU 11 calculates an approximate line based on the upper edge point set TopEdge. The approximate line of TopEdge is expressed by an approximate line or an approximate curve. Point sequence included in TopEdge
(x1, y1), (x2, y2), ..., (xn, yn)
Then, the approximate line y = ax + b by the least square method is

It is obtained by.

の最小化を行う。 On the other hand, the approximate curve is obtained by approximation to a circle by the least square method. In the approximation to the circle by the method of least squares, in order to find the circle (x−a) ² + (y−b) ² = R with radius R centered at (a, b)

Minimize.

とする。 Where z _i , B, C, and D are

And

となる。この連立方程式をCholesky分解により解き、B、C、Dを得て、

によりa、b、Rを得る。 When F (a, b, R) is partially differentiated with respect to B, C and D, respectively,

It becomes. Solve this simultaneous equation by Cholesky decomposition, get B, C, D,

To get a, b, R.

によって求まる整数であり、(x_M,y_M)は検出された上辺の点集合の中央の点である。以上の上辺の近似線算出の手順を図１２のステップＳ１２１〜Ｓ１２６に示す。 Next, in step S66, the CPU 11 compares the approximate line and the approximate curve to determine which is appropriate. This determination is made by comparing the sum of the distances from the approximate line to each (x _i , y _i ) and the sum of the distances from the approximate curve to each (x _i , y _i ). It can be done accurately by determining that the line is valid. If it is desired to reduce the amount of calculation, it may be determined by the sum of the distances to (x ₁ , y ₁ ), (x _M , y _M ), and (x _n , y _n ). Where m is

(X _M , y _M ) is the center point of the detected upper-side point set. The procedure for calculating the approximate line of the upper side is shown in steps S121 to S126 in FIG.

によって算出する。 The upper side boundary line is calculated by the above procedure. Similarly, as shown in FIG. 13, the left side is also considered by rotating the xy coordinate system 90 degrees to the left (the upper direction is the x-axis direction and the right direction is the y-axis direction). Is calculated. For the lower side, the point set on the lower side of the two-dimensional code image can be calculated by reversing the y direction of the xy coordinate system. However, since there is no finder pattern in the lower right of the QR code, there is no RightFinderEdge. Therefore, edgeSlope (x) is the slope of the line segment drawn from (LeftCenterX, LeftCenterY) to (px ′, py ′)

Calculated by

によって算出し、こちらも２次元コード画像の端まで探索を行うものとし、そうして検出した点の集合をMiddleRightEdgeとする。 Also, since there is no RightFinderEdge, the search is performed up to the end of the two-dimensional code image, and the set of points detected in this way is MiddleBottomEdge. Also for the right side, the point set on the lower side of the two-dimensional code image can be detected by considering the xy coordinate system rotated 90 degrees to the right (the lower direction is the x-axis direction and the left direction is the y-axis direction). However, in this case as well, there is no RightFinderEdge on the upper side, so edgeSlope (x) is the same as the lower side.

This is also used to search to the end of the two-dimensional code image, and the set of points detected in this way is referred to as MiddleRightEdge.

とし、同様に初期のxy座標系におけるMiddleRightEdgeを

とする。このときbx₁<bx₂<...<bx_M及びry₁<ry₂<...<ry_Nとなる。また、左下ファインダパターンＦＰLBは右上ファインダパターンＦＰRTより左下に位置すると考えられるので、bx₁<rx₁かつry₁<by₁となる。 Next, the coordinate RightBottom of the lower right corner of the two-dimensional code image is detected. MiddleBottomEdge in the initial xy coordinate system (the right direction is the x-axis direction and the down direction is the y-axis direction)

Similarly, MiddleRightEdge in the initial xy coordinate system is

And At this time, bx ₁ <bx ₂ <... <bx _M and ry ₁ <ry ₂ <... <ry _N. Further, since the lower left finder pattern FPLB is considered to be located at the lower left than the upper right finder pattern FPRT, bx ₁ <rx ₁ and ry ₁ <by ₁ .

  The CPU 11 also assumes that RightBottom is a point where f (x, y) = x + y is maximum in the two-dimensional code, and searches for a point located near RightBottom from MiddleBottomEdge and MiddleRightEdge. However, MiddleBottomEdge may contain points on the right side of RightBottom, and MiddleRightEdge may contain points below RightBottom, so these error points need to be removed.

とする。そして、j=f(i)としたとき(bxi, byi)および(rxj, ryj)からRightBottomを推定する関数InferRightBottom(i)を

とする。 The removal of this error point is performed as follows by the second method described above. FIG. 4 is a flowchart showing a procedure of intersection calculation processing which is a part of the processing.
That is, the CPU 11 in steps S631 to S635, for example, as shown in FIG. 14, from the points (bxi, byi) included in the MiddleBottomEdge, a threshold straight line having a slope of −1, that is,
y =-(x-bxi) + bxi
A function f (i) that calculates a value of j indicating a point in MiddleRightEdge that is located above each straight line and that has bxi ≦ rxj and the largest y coordinate.

And And when j = f (i), the function InferRightBottom (i) that estimates RightBottom from (bxi, byi) and (rxj, ryj)

And

となる。上記BottomEndを用いてRightBottomは、

によって検出される。 If the minimum i such that b _{i + 1} is on the right side or r _{j + 1} is down compared to InferRightBottom (i) is BottomEnd, BottomEnd is

It becomes. Using the above BottomEnd, RightBottom

Detected by.

  Next, the CPU 11 removes a point (a point having a large x coordinate value) located on the right side of the RightBottom from the MiddleBottomEdge. Also, a point (a point having a large y coordinate value) located below the RightBottom is removed from the MiddleRightEdge. Further, the y direction of the xy coordinate system is reversed again for MiddleBottomEdge, and error points in the two-dimensional code image as shown in FIG. 11 are removed in the same manner as the processing on the upper side. For MiddleRightEdge, the xy coordinate system is rotated 90 degrees to the right, and error points in the two-dimensional code image are similarly removed.

とする。 Further, the CPU 11 sets the bottom edge of the lower left finder pattern FPLB as LeftFinderBottomEdge and the right edge of the upper right finder pattern FPRT as TopFinderRightEdge, and sets the lower edge point set BottomEdge and the right edge point set RightEdge of the two-dimensional code image.

And

  The CPU 11 calculates an approximate line from the lower side point set BottomEdge obtained by the above procedure by the same procedure as the procedure for calculating the approximate line on the upper side, and returns the original approximate line of the lower side by returning to the original xy coordinate system. calculate. Similarly, an approximate line is calculated from the point set on the right side, and the approximate line on the right side is calculated by returning to the original xy coordinate system.

(7) Correcting the distortion of the two-dimensional code image When the boundary lines for the upper, lower, left and right sides of the two-dimensional code image are calculated by the above processing, the CPU 11 subsequently corrects the two-dimensional code distortion in step S7 shown in FIG. The program 136 is activated, and the distortion of the two-dimensional code image is corrected as follows using the calculated boundary lines of the upper, lower, left and right sides of the two-dimensional code image.

  That is, in correcting the distortion of the two-dimensional code image, the two-dimensional code image is corrected with respect to the width codeWidth and the height codeHeight, and the pixel value of the target coordinate (x, y) in the corrected two-dimensional code is determined. By calculating, a two-dimensional code after correction is generated.

  First, in step S71, the CPU 11 calculates the four-dimensional coordinates of the two-dimensional code. The four-dimensional coordinates of the two-dimensional code are: the top left corner point LeftTop from the top and left border, the top right corner point RightTop from the top and right border, and the bottom left corner point LeftBottom from the bottom and left border. From the boundary between the left and right sides, calculate the right bottom point RightBottom.

とした上で、上記作成されたBaseFrameにおいて、図１５に示すように、LeftTopからの距離とRightTopからの距離の比が(1-LeftWeight)LeftWeightとなる上辺上の点をBaseTop(LeftWeight)とし、LeftBottomからの距離とRightBottomからの距離の比が(1-LeftWeight)LeftWeightとなる点をBaseBottom(LeftWeight)とし、LeftrTopからの距離とLeftBottomからの距離の比が(1-TopWeight)TopWeightとなる点をBaseLeft(TopWeight)とし、RightTopからの距離とRightBottomからの距離の比が(1-TopWeight)TopWeightとなる点をBaseRight(TopWeight)とする。 Next, a quadrangle BaseFrame that connects four points of LeftTop, RightTop, RightBottom, and LeftBottom is created. And

Then, in the created BaseFrame, as shown in FIG. 15, the point on the upper side where the ratio of the distance from LeftTop to the distance from RightTop is (1-LeftWeight) LeftWeight is BaseTop (LeftWeight) The point where the ratio of the distance from the LeftBottom and the distance from the RightBottom is (1-LeftWeight) LeftWeight is BaseBottom (LeftWeight), and the ratio of the distance from the LeftrTop and the distance from the LeftBottom is (1-TopWeight) TopWeight BaseLeft (TopWeight) is set, and the point at which the ratio of the distance from RightTop to the distance from RightBottom is (1-TopWeight) TopWeight is set to BaseRight (TopWeight).

を満たす。 Subsequently, in steps S72 and S73, the CPU 11 obtains coordinates BaseCross (TopWeight, LeftWeight) corresponding to the target coordinates in BaseFrame. However, as shown in FIG. 15, an intersection of a line segment connecting BaseTop (LeftWeight) and BaseBottom (LeftWeight) and a line segment connecting BaseLeft (TopWeight) and BaseRight (TopWeight) is defined as BaseCross (TopWeight, LeftWeight).
BaseCross (TopWeight, LeftWeight)

Meet.

  In step S74, the CPU 11 sets a point on the upper side where the x coordinate value is the same as BaseTop as TopPoint (LeftWeight) and sets a point on the lower side where the x coordinate value is the same as BaseBottom as shown in FIG. LeftWeight), a point on the left side where the y-coordinate value is the same as LeftBase is called LeftPoint (TopWeight), and a point on the right side where the y-coordinate value is the same as RightBase is called RightPoint (TbpWeight).

とし、歪みベクトルDistortionVector(TopWeight,LeftWeight)を

とする。 In step S75, as shown in FIG. 16, the upper side distortion DistortionTop (LeftWeight), the lower side distortion DistortionBottom (LeftWeight), the left side distortion DistortionLeft (TopWeight), and the right side distortion DistortionRightRight (TopWieight) in the two-dimensional code image, respectively.

And distortion vector DistortionVector (TopWeight, LeftWeight)

And

とする。なお、CrossPoint(TopWeight,LeftWeight)は、

を満たし、DistortionVector(TopWeight, LeftWeight)を算出せずに求めることもできる。 Next, in step S76, the CPU 11 calculates a function CrossPoint (TopWeight, LeftWeight) for calculating a point obtained by adding DistortionVbctor (TopWeight, LeftWeight) to BaseCross (TopWeight, LeftWeight) as shown in FIG.

And CrossPoint (TopWeight, LeftWeight) is

Can be obtained without calculating DistortionVector (TopWeight, LeftWeight).

のピクセル値と同一の値を持たせる。 The CPU 11 corrects the distortion of the two-dimensional code image using the CrossPoint (TopWeight, LeftWeight) defined as described above. TopWeight and LeftWeight are as defined above. Therefore, the pixel value of the coordinates (x, y) in the two-dimensional code image after correction includes the pixel value in the two-dimensional code image before correction.

Have the same value as the pixel value of.

  Through the above processing, it becomes possible to correct a two-dimensional code image having a pincushion-type or barrel-type curve distortion as shown in FIGS. 18A and 18B into a rectangle. In addition, by setting codeWidth and codeHeight to the same value, correction to a square is possible.

  Then, the CPU 11 analyzes the code of the two-dimensional code image corrected by the above processing in step S <b> 8 and stores the code analyzed in the data memory 16. The code stored in the data memory 16 is read from the data memory 16 and transmitted to a Web server or the like when the user performs a two-dimensional code transmission operation on the input device 3, for example.

  As described above, in this embodiment, after the image data of the two-dimensional code captured by the camera 2 is converted into a binarized image, a finder pattern of the two-dimensional code is detected from the binarized image, and this detection is performed. The coordinate values of the upper, lower, left and right sides of the finder pattern are detected. Then, based on the detected coordinate values of the upper, lower, left and right sides of the finder pattern, an approximate line is generated by approximating the shape of the upper, lower, left and right sides of the image data of the two-dimensional code with a straight line and a curve. The distortion of the image data of the two-dimensional code is corrected based on the approximated line, and the two-dimensional code is decoded based on the image data of the two-dimensional code whose distortion has been corrected.

  Therefore, even if the captured image of the two-dimensional code has, for example, a pincushion-type or barrel-type curved distortion, the shapes of the upper, lower, left and right sides of the image data of the two-dimensional code are approximated by straight lines and curves. The distortion of the image data of the two-dimensional code is corrected based on the line. For this reason, it is possible to perform code decoding with high accuracy even for a two-dimensional code image having the above-described pincushion-type or barrel-type curved distortion.

  The present invention is not limited to the above embodiment. For example, in the above-described embodiment, the case where the two-dimensional code reading device is built in the mobile phone has been described as an example. However, the two-dimensional code reading device may be built in the remote controller of the television receiver or the video recording / reproducing device. Alternatively, it may be built in a personal computer. In the above embodiment, the case where the two-dimensional code reader includes a camera has been described as an example. However, a two-dimensional code is captured by a digital camera provided separately from the two-dimensional code reader, and the image data is signaled. You may make it take in to a two-dimensional code reader via a cable or a recording medium.

  In addition, the present invention also includes processing procedures and processing contents of binarized image generation processing, finder pattern detection processing, image rotation processing, finder pattern boundary detection processing, two-dimensional code boundary calculation processing, and two-dimensional code distortion correction processing. Various modifications can be made without departing from the scope of the present invention.

  In short, the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine suitably the component covering different embodiment.

It is a block diagram which shows the structure of the hardware and software of the two-dimensional code reader concerning one Embodiment of this invention. It is a flowchart which shows the process sequence and process content from imaging control to image rotation process among the two-dimensional code reading controls by the two-dimensional code reader shown in FIG. It is a flowchart which shows the process sequence and process content from a finder pattern boundary detection process to a two-dimensional code boundary line calculation process among the two-dimensional code reading controls by the two-dimensional code reader shown in FIG. FIG. 4 is a flowchart showing an intersection calculation process procedure for removing error points in the two-dimensional code boundary calculation process shown in FIG. 3 and its contents. It is a flowchart which shows the procedure and content of a two-dimensional code distortion correction process among the two-dimensional code reading controls by the two-dimensional code reader shown in FIG. It is a figure used for description of a finder pattern detection process. It is a figure used for description of a finder pattern detection process. It is a figure used for description of the rotation process of a two-dimensional code image. It is a figure used for the process which detects the coordinate value of each upper / lower / left / right side of a finder pattern. It is a figure used for the process which detects the coordinate value of each upper / lower / left / right side of a finder pattern. It is a figure for demonstrating the 1st process which removes an error point in the process which calculates a two-dimensional code boundary line. It is a figure for demonstrating the procedure when approximating the upper side of a two-dimensional code image with a straight line and a curve in the process which calculates a two-dimensional code boundary line. It is a figure for demonstrating the process which calculates a two-dimensional code boundary line. It is a figure for demonstrating the 2nd process which removes an error point in the process which calculates a two-dimensional code boundary line. It is a figure used for description of the distortion correction process of a two-dimensional code image. It is a figure used for description of the distortion correction process of a two-dimensional code image. It is a figure used for description of the distortion correction process of a two-dimensional code image. It is a figure which shows an example of the pincushion type | mold and barrel distortion which generate | occur | produce in the imaged two-dimensional code image. It is a figure for demonstrating the conventional two-dimensional code reader.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Central processing control unit, 2 ... Camera, 3 ... Input device, 4 ... Display device, 11 ... CPU, 12 ... Bus, 13 ... Program memory, 14 ... Camera I / F, 15 ... Input / output I / F, 16 ... Data memory 131 ... Binary image generation program 132 ... Finder pattern detection program 133 ... Image rotation processing program 134 ... Finder pattern boundary detection program 135 ... 2D code boundary line calculation program 136 ... 2D code A distortion correction program.

Claims (4)

Means for capturing image data of a two-dimensional code from an imaging means and storing it in a memory;
Finder pattern detection means for reading image data of a two-dimensional code stored in the memory and detecting a plurality of finder patterns included in the two-dimensional code from the read image data;
Finder pattern boundary detection means for detecting a boundary point of a side representing the outline of each of the detected plurality of finder patterns;
Two-dimensional that generates an approximate line that approximates at least one of a straight line and a curve of the shape of the four sides representing the contour of the image data of the two-dimensional code based on boundary points of the sides of the detected plurality of finder patterns. Code boundary detection means;
Two-dimensional code distortion correcting means for correcting distortion of the image data of the two-dimensional code based on the generated approximate line, and storing the image data of the two-dimensional code with corrected distortion in the memory;
Decoding means for decoding the two-dimensional code based on the image data of the two-dimensional code whose distortion has been corrected, and outputting the decoded data;
Comprising
The two-dimensional code boundary detection means includes
Means for calculating an inclination of a side representing an outline of the image data of the two-dimensional code based on boundary points of a plurality of finder patterns detected by the finder pattern boundary detection unit;
Means for detecting a boundary point representing a side of the two-dimensional code by performing a pixel search in a direction inferred from the calculated slope of the side;
For each of the detected boundary points, it is determined whether the boundary point is included in a range estimated from the calculated side inclination, and the boundary point determined not to be included is Means for deleting from the boundary points as error points ;
2-dimensional code reading apparatus you comprising: a. Means for capturing image data of a two-dimensional code from an imaging means and storing it in a memory;
Finder pattern detection means for reading image data of a two-dimensional code stored in the memory and detecting a plurality of finder patterns included in the two-dimensional code from the read image data;
Finder pattern boundary detection means for detecting a boundary point of a side representing the outline of each of the detected plurality of finder patterns;
Two-dimensional that generates an approximate line that approximates at least one of a straight line and a curve of the shape of the four sides representing the contour of the image data of the two-dimensional code based on boundary points of the sides of the detected plurality of finder patterns. Code boundary detection means;
Two-dimensional code distortion correcting means for correcting distortion of the image data of the two-dimensional code based on the generated approximate line, and storing the image data of the two-dimensional code with corrected distortion in the memory;
Decoding means for decoding the two-dimensional code based on the image data of the two-dimensional code whose distortion has been corrected, and outputting the decoded data;
Comprising
The two-dimensional code boundary detection means includes
Means for calculating an inclination of a side representing an outline of the image data of the two-dimensional code based on boundary points of a plurality of finder patterns detected by the finder pattern boundary detection unit;
Means for detecting a boundary point representing a side of the two-dimensional code by performing a pixel search in a direction inferred from the calculated slope of the side;
To remove a means for calculating the first and the intersection of the second side orthogonal among the four sides of the image data before Symbol 2-dimensional code, a boundary point located outside of the range from the calculated intersection as an error point An error point removing means having a means, and
Means for calculating the intersection of the first and second sides of the image data of the two-dimensional code,
A threshold line is drawn using one or more boundary points included in the set of boundary points representing the first side as a calculation reference point, and the boundary point included in the boundary point set representing the second side is the threshold value. Means for determining whether or not it is located on the intersection side of the line;
Means for detecting, as a non-threshold proximity point, a boundary point closest to the intersection point among boundary points not located on the intersection side of the threshold line in the set of boundary points representing the second side;
An intersection of a straight line obtained by estimating the slope of the first side from the calculation reference point and a straight line obtained by estimating the slope of the second side from the out-of-threshold proximity point of the second side Based on the above, a means for calculating intersection candidate points,
Among the boundary points included in the set of boundary points representing the first side, the boundary point closest to the threshold line among the boundary points located on the intersection side of the threshold line is close to the threshold value. And detecting a boundary point closest to the threshold straight line among the boundary points included in the boundary point set representing the second side and located on the intersection side of the threshold line. Means for detecting as a proximity point within the threshold;
Which of the detected first side in-threshold proximity point and the detected second side in-threshold proximity point is outside the calculated intersection candidate point 2-dimensional code that further comprising a means for determining a reading device. The two-dimensional code distortion correcting means includes
Vertex frame generating means for generating a vertex frame connecting the vertices of the image data of the two-dimensional code;
When calculating the pixel value of the target coordinate in the image data of the two-dimensional code to be corrected, the coordinate value of the point corresponding to the target coordinate on the upper side of the generated vertex frame and the target coordinate on the lower side A vertex frame corresponding point calculation means for calculating a coordinate value of a corresponding point, a coordinate value of a point corresponding to the target coordinate on the left side, and a coordinate value of a point corresponding to the target coordinate on the right side;
The coordinate value of the boundary point corresponding to the target coordinate on the upper side, the coordinate value of the boundary point corresponding to the target coordinate on the lower side, and the boundary point corresponding to the target coordinate on the left side of the image data of the two-dimensional code A boundary corresponding point calculating means for calculating a coordinate value and a coordinate value of a boundary point corresponding to the target coordinate on the right side;
In-vertex frame target coordinate value calculating means for calculating a target coordinate value corresponding to the target coordinate in the vertex frame;
A vector from a point corresponding to the target coordinate calculated for each of the upper, lower, left and right sides of the vertex frame to a boundary point corresponding to the target coordinate calculated for each of the upper, lower, left and right sides of the image data of the two-dimensional code 2D code distortion calculating means for calculating a two-dimensional code distortion vector by weighted addition,
In the two-dimensional code for calculating the target coordinate value corresponding to the target coordinate in the image data of the two-dimensional code by adding the calculated two-dimensional code distortion vector to the target coordinate value calculated for the vertex frame 2-dimensional code reading apparatus according to claim 1, wherein further comprising a target coordinate value calculating means. The program used for performing the process of each means in the two-dimensional code reader according to any one of claims 1 to 3 .

Images