KR101109510B1 - Method for recognising two-dimensional barcode - Google Patents

Abstract

According to the present invention, a finder pattern represented by a square line shape, a timing pattern positioned inside the finder pattern and having a black area and a white area repeated at regular intervals, and a direction for determining the direction of the barcode A method for recognizing a matrix barcode consisting of cells of a rectangular grid having a pattern and a code area for storing data in a cell except the patterns, by photographing a camera having a mobile device embedded therein:
(1) generating a grayscale image of the matrix barcode;
(2) detecting a boundary of the image;
(3) generating a candidate list as a result of step (2); And
And (4) performing error correction on each candidate generated in the candidate list.

Description

2D barcode recognition method {METHOD FOR RECOGNISING TWO-DIMENSIONAL BARCODE}

The present invention relates to a method for recognizing a two-dimensional code to be recognized by a mobile terminal such as a mobile phone.

Augmented reality is a kind of virtual reality. Unlike ordinary virtual reality, which excludes the real world from the user and provides only the perception of the complete virtual world, augmented reality simultaneously provides the real world and virtual objects added to or synthesized therefrom. In other words, if virtual reality completely replaces the real world, augmented reality is the concept of supplementing and augmenting the real world.

Augmented reality is used in a variety of ways and enhances the value of the real world. For example, <Sponsored Advertising in Sports Broadcasting Stadium>. When watching broadcasts of various sports, you can see advertisements from various sponsors throughout the stadium. Various formats of posters or display boards are used at specific places prepared for advertisements. In this case, the augmented reality application may leave only a specific place for the advertisement blank, and the advertisement content may be synthesized and broadcasted in real time. It is possible to display various types of advertisements clearly at low cost, no replacement cost, and different advertisements by region or country.

Another example is <Product Information Display>. The industrial product has a barcode so that the machine can easily recognize the type of product. When a product is read through a barcode embedded in a camera built into a mobile device such as a mobile phone having an augmented reality application system, the application system synthesizes and outputs various pieces of information on the image of the product. It is possible to output various types of additional information from basic information such as price, raw materials, and production place to multimedia information such as video recipes, advertisements, and related events.

In addition, in the case of <Three-dimensional illustrations>, if you look at ordinary flyers or books through the augmented reality system, you can make a video or a three-dimensional picture. Recognize this by inserting a specific mark or pattern, or recognizing ordinary illustrations and compositing virtual objects in real time at that location. When the user freely rotates the flyer or the book, or sees it near or far, it is possible to make the virtual object rotate or appear enlarged or reduced accordingly. In addition, augmented reality can be applied to <game>. For example, you can imagine a game where monsters appear in real spaces and users fight with virtual weapons. The user wears an HMD with a camera on his head and roams the real space with his computer attached to his back. Basically, the HMD shows the real world image taken by the camera, so it will feel as natural as wearing glasses. If the computer recognizes the user's location and the direction of the HMD, synthesizes the monster in the camera input image and transmits the monster to the HMD, it will feel as if the monster exists in the real world. When a button input is input from the weapon-shaped input device, the direction of the button may be calculated to damage the monster at the point indicated.

In the present invention, augmented reality systems in particular in a mobile environment are considered. Mobile augmented reality refers to augmented reality that is driven in a mobile device environment, such as mobile phones, smartphones, PDAs. Mobile devices have the advantage of being easy to carry anywhere, anytime. When this advantage is added to augmented reality, more diverse applications are possible.

In the above example, in the case of <information display of goods>, practical application is impossible except for a mobile device. And even in the case of stereoscopic illustrations that can be applied using a webcam and a fixed computer, using a mobile device can increase the immersion feeling because the camera and the display screen are integrated. But the mobile environment doesn't just offer advantages. Mobile devices have fundamental limitations due to miniaturization and portable battery-dependent power. Computing power is significantly lower than ordinary computers. The object of the present invention is to propose a tool for augmented reality application that overcomes these disadvantages and maximizes the advantages.

The essential element essential for all augmented reality applications is the position indicator. Location indicators provide a coherence between the real world and the virtual world trying to synthesize in augmented reality. Types of location indicators can be classified into active sensor methods and passive sensor methods. The former includes the position indicator actively transmitting signals such as RFID, LED, and GPS, and the latter includes a marker system having a special pattern, which is inserted in a form attached to the real world and recognized by a camera. . The present invention relates in particular to the latter.

Most markers used in augmented reality applications have very small amounts of information. A notable mobile but popular system is ARTag, which also provides only 1,024 IDs with markers. Thus, when there are many kinds of virtual objects to be augmented, two or more markers may be used in combination. While it is a choice for speed, this capacity can be a constraint when attempting various uses of augmented reality. This can be a problem when dealing with multiple situations or locations in a large augmented reality application system and trying to uniquely represent the kind of information you want to augment for each. For example, the number of merchandise displayed in a store far exceeds 1,024 or 4,096, as well as the number of items required for applications such as pages, illustrations, problem items in books, and various games.

Two-dimensional barcodes are widely used as markers for mobiles. Unlike conventional barcodes expressing information in one-dimensional linear form, information is represented in two-dimensional manner. Therefore, it is known that it is possible to express more information in the same area more efficiently. As a two-dimensional barcode, PDF-417 code, QR code, Data Matrix, MaxiCode are used.

1 shows an example of a QR code. When the minimum unit of information representation of a QR code represented by a dot is called a module, the QR code of FIG. 1 is 21 × 21 in width × length, and the largest version in which the width × length increases in stages as the version increases. 40 has a size of 177 × 177. And as the version increases, so does the amount of information that can be included. However, the QR code, which is an international standard (ISO / IEC18004), has a lot of operations to be processed in the recognition process and must be recognized as a dedicated reader. In addition, vision-based recognizers using ordinary cameras require a high-performance computer and are not suitable for use in mobile environments.

In order to solve this problem, a micro QR code designated by the Japanese Industrial Standard (JIS × 0510), which is miniaturized to be suitable for a mobile environment while using a code display system and an algorithm similar to the QR code, has been developed as shown in FIG. 2 is a Micro QR code compared to the QR code. As shown in FIG. 2, the main feature of the Micro QR code compared to the QR code is that only one finder pattern (position detection pattern, or finder pattern) is located at the upper left, and the margin area is composed of two module widths. Micro QR code is a maximum of 35 numbers of data that can be stored compared to QR code, and there are 4 versions of M1 ~ M4 as shown in Table 1 below.In the case of the largest M4, L, M, Q is present.

<Data capacity by version of Micro QR code>

The information capacity and error correction rate according to various versions of QR code or Micro QR code is a great advantage of this code, but from the point of view of recognition, it requires a separate recognizer for each version, which complicates the recognition process or is an important cause of unit price increase. Sometimes.

In order to support various information capacity and error correction rate in QR code, the length of codeword is defined variously according to each version and correction rate. Therefore, the recognizer must be able to process codewords of various lengths. In this case, there is a problem that a quick method using a conversion table cannot be utilized, and the complicated processing of Reed Solomon must be performed as it is. In addition, an increase in the amount of data to be processed due to an increase in storage capacity has a disadvantage in that it leads to an increase in direct calculations.

An object of the present invention is to provide a new two-dimensional code for mobile and at the same time propose a method for recognizing this new two-dimensional code. The novel two-dimensional code of the present invention can be recognized quickly enough to be recognized in real time in a mobile environment, instead of the variability of the barcode that the conventional QR code or Micro QR code focused on, and can express a sufficient quantity of IDs. Provide a new two-dimensional code. In addition, the novel two-dimensional code recognition method of the present invention uses a two-step hierarchical error processing method to improve the accuracy of recognition by finding and filtering out uncorrected or incorrect recognition cases.

If the storage information of the mobile barcode is limited only to the ID information, it does not need more than a certain size. In addition, the speed can be improved by simply fixing a sufficient capacity and simplifying the processing. The two-dimensional bar code of the present invention gives up the variability for this reason, instead it can be recognized quickly enough to be recognized in real time in a mobile environment, it is possible to represent a sufficient number of IDs.

On the other hand, other unspecified objects of the present invention will be further considered within the range that can be easily inferred from the following detailed description and effects.

Recognition method of the two-dimensional barcode of the present invention for achieving the above object, as a method of recognizing a matrix barcode consisting of cells of a rectangular grid:

A first step of correcting an error occurring when a barcode recognition device of a mobile device photographs the matrix barcode with a built-in camera; And

And detecting an error on the corrected image of the matrix barcode obtained through the first step, and acknowledging the error correction of the first step only when no error is detected.

In addition, the matrix barcode of the method,

A finder pattern represented by a square line shape;

A timing pattern positioned inside the finder pattern and having a black region and a white region repeated at regular intervals;

A direction pattern for determining a direction of the barcode; And

A code area for storing data is formed in a cell except for the patterns, and the data stored in the code area is read using the patterns.

In addition, the present invention is a finder pattern represented in the form of a square line, a timing pattern which is located inside the finder pattern, the black region and the white region are repeated at regular intervals, and determines the direction of the barcode A method of recognizing a matrix barcode consisting of cells of a rectangular grid having a direction pattern and a code area for storing data in a cell except the patterns, by photographing a camera having a built-in mobile device:

(1) generating a grayscale image of the matrix barcode;

(2) detecting a boundary of the image;

(3) generating a candidate list as a result of step (2); And

(4) performing error correction on each candidate created in the candidate list;

Characterized in that the two-dimensional matrix barcode recognition method comprising a.

Further, preferably, the step (4) of the method,

Finding four vertices of the candidate finder pattern; And

And using the four vertices to obtain a path of the portion where the finder pattern is located. The method may further include generating an array of grayscale images with respect to the inclined barcode image. ; And correcting the image information of the distorted array by using the information of the timing pattern.

In addition, according to an embodiment of the present invention, it is preferable to further include the step of excluding a candidate having a wrong direction pattern from the list by using the direction pattern.

The method may further include reading out and serializing a codeword recorded in the code area, wherein step (4) preferably includes performing error processing on the serialized bit string. Do.

Bar code of the present invention has the advantage that can be recognized quickly enough to be recognized in real time in a mobile environment instead of giving up the variability of the conventional QR code. In addition, as summarized in Table 2, like the QR code, it has a matrix-type symbol system. The symbol size is 13 × 13.

<Comparison between two-dimensional code and QR code of the present invention>

As will be described in detail below, the matrix barcode refers to a form in which cells in a grid having regular intervals are filled with black and white, and the size of 13 × 13 means that the grid is 13 squares and 13 squares. This means that it consists of a total of 169 black or white cells. In the case of QR codes, the size varies from version to version, and the amount of information depends on its size and error correction rate. In order to maintain the same conditions, we compared the Micro QR code M2 version with the same 13 × 13 size QR code. Two error correction levels of the M2 version, L and M, were compared. Each represents an error correction rate of 7% and 15%.

As described above, the symbol shape and the symbol size both have a matrix size of 13 × 13. The symbol component refers to patterns that are always inserted when representing a symbol to perform a special function and form a unique form of the symbol. The finder pattern of the QR code has a form in which the color is inverted at a unique ratio of 1: 1: 3: 1: 1 on any line of 360 degrees passing through its center. And it has a margin to distinguish this finder pattern and other symbols located on the upper left of the barcode. In addition, the timing pattern where white and black cross repeatedly is located at the upper and left sides except for the finder pattern and the margin. The barcode of the present invention has a finder pattern in the form of enclosing the outside with black cells, and also has a timing pattern in which white and black repeatedly cross on a cross form formed by horizontal and vertical lines passing through the center point of the barcode. In addition, since the direction cannot be determined using only a finder pattern and a timing pattern having a perfect symmetry structure, four cells at the center and four cells at the corners are used as the direction pattern defining the direction.

Information presentation shows information capacity. In the case of QR code, micro version does not support bit storage but it supports numbers and alphanumeric characters. The maximum number of digits that can be stored and the number of alphanumeric characters are shown in the table. In the case of the barcode of the present invention, an ID corresponding to 14 digits or numbers can be stored using a capacity of 6 bytes. The information efficiency is a ratio of the amount of information that can be expressed by the symbol size (13 x 13) of the bar code, and compares the degree of efficiency of the actual information capacity in the bar code of the same size. This difference in efficiency arises from the difference in space between each unique sign component and error handling.

In case of error handling, the QR code corrects 7% and 15%, respectively, using Reed Solomon. In the case of the barcode of the present invention, a two-stage hierarchical error processing method is used. In one step, an error correction is attempted using a hamming method that is different from a QR code, and an error detection is attempted in a second step. The reason for using the Hamming method is that it is not necessary to set the variability of the error correction rate, and it is possible to quickly correct the error by simply using the conversion table. Also, attempting to detect the error in the second stage is not corrected or wrongly recognized. This is to improve the accuracy rate by detecting the case. Basically, the Hamming code method or Reed Solomon has the potential to malfunction for the occurrence of an error that exceeds the error correction rate. To solve this problem, the barcode of the present invention is designed to have a hierarchical error processing method.

Through the above comparison, the advantages of the two-dimensional novel bar code of the present invention, (1) can be quickly recognized in a simple method, (2) has high information efficiency, (3) two-level hierarchical error processing The accuracy rate is improved through.

In addition, the present invention intends to contribute to the commercialization of the barcode of the present invention by simultaneously providing an accurate, fast and efficient recognition method for the novel two-dimensional barcode as described above.

Even if effects not specifically mentioned in the specification of the present invention are incorporated, the provisional effects expected by the technical features of the present invention are treated as described in the specification of the present invention.

1 is a view showing a conventional QR code.
2 is a view showing a conventional Micro QR code.
3 is a diagram illustrating a hierarchical error processing method according to the present invention.
4 is a diagram showing an example of the configuration of a finder pattern 10 of a two-dimensional barcode of the present invention.
5 is a diagram showing an example of the configuration of the timing pattern 20 of the two-dimensional barcode of the present invention.
Figure 6 (a) shows a conceptual example of the original image of the bar code of the present invention, (b) is an image image taken by tilting the bar code of (a) with a camera.
FIG. 7 is a diagram illustrating a state when the image of FIG. 6B is read in a grid having the same size.
Fig. 8 is a diagram showing an example of the configuration of the direction pattern of the two-dimensional barcode of the present invention.
Fig. 9 is a diagram showing an example of the basic template of the two-dimensional barcode of the present invention.
Fig. 10 is a diagram showing an example of arrangement of codewords in a code area of a two-dimensional bar code of the present invention.
Fig. 11 is a two-dimensional barcode of the present invention representing the character string "Hello!"
12 is a flowchart schematically showing the overall process of an embodiment of a method of recognizing a two-dimensional matrix barcode of the present invention.
FIG. 13 is a grayscale image obtained when photographing the printed barcode of FIG. 11.
FIG. 14 is an image obtained by performing the edge detection process of FIG. 13. FIG.
FIG. 15 is a diagram illustrating a result of generating candidate fragments with respect to FIG. 14.
FIG. 16 illustrates a four vertex finding for FIG. 15.
17 is a diagram illustrating a finder pattern inspection path on a grayscale image.
18 is a diagram exemplarily illustrating one candidate remaining as a result of a finder pattern inspection.
Fig. 19 shows the 39 × 39 grayscale alignment of barcodes.
FIG. 20 is a diagram illustrating an arrangement of distorted images when the printed barcode of FIG. 11 is photographed by tilting with a camera unlike FIG. 19.
FIG. 21 is a diagram illustrating a distorted arrangement in which the barcode image of FIG. 20 is divided into 3 × 3 size units.
22 is a diagram comparing a binary image of a distorted array and an original barcode.
FIG. 23 is a diagram illustrating an example of a structure of an asymmetric grating used for distortion correction.
24 is a diagram illustrating an example of the configuration of nine gratings used in the distortion correction step.
25 is a diagram illustrating distortion correction of a distorted array.
26 shows "Hello!" A diagram showing a bar code array and a grid.
27 is a view showing the positions of eight cells constituting the direction pattern.
Fig. 28 shows the binarization result of the arrangement and the respective codeword numbers.
29 shows serialization of codewords from an array.
<Description of the symbols for the main parts of the drawings>
1: two-dimensional barcode 10: finder pattern
20: timing pattern 25: center point of the two-dimensional barcode
31 ~ 38: direction pattern
The accompanying drawings show that they are illustrated as a reference for understanding the technical idea of the present invention, by which the scope of the present invention is not limited.

Hereinafter, with reference to the accompanying drawings will be described specific details for the practice of the invention. In the following description of the present invention, when it is determined that the subject matter of the present invention may be unnecessarily obscured as matters obvious to those skilled in the art, such as related well-known functions, the detailed description thereof will be omitted.

<Information capacity>

Information capacity is the amount of information that can be stored, usually defined in bits or bytes. The larger the dose, the better it is possible. But there are factors that make it impossible. It is closely related to the limitations of mobile devices. Complex processing is not possible due to the limitation of computing power. Increasing storage capacity can lead to simply increasing the amount of data to be processed and the need for additional processing. For example, an increase in general information capacity leads to an increase in the physical size of a barcode or, conversely, a decrease in the expression size, so that it is more easily exposed to error causes such as camera distortion and requires further processing. Tools such as a position correction pattern may be necessary, and calculations for distortion correction are increased. And the intensity of error handling must increase.

Of course it is not the best to make it small. The reason for proposing a new two-dimensional barcode is to solve the limitation of capacity of the existing mobile barcode. It is important to provide sufficient capacity for a variety of applications and to have the appropriate size considering the performance of the recognition process.

Barcodes applied in mobile environments do not need to contain additional information beyond ID information like general purpose 2D barcodes because it is not possible to store all the information necessary for various applications in 2D barcodes. Because you need to approach. Therefore, the mobile barcode only needs to store ID information. The remaining additional information is obtained from the storage space accessible by the mobile device in addition to the barcode.

Now, when there are a lot of objects in a single large system, think about how much capacity is needed to provide a unique ID for each of these. You can think of the Internet as such a representative system. Every computer connected to the Internet across the world is in principle given a unique address. The size of the Internet address (IPv4) used here is 4 bytes. Four bytes are 32 bits, and the number of IDs that can be represented by these 32 bits is 2 ³² , that is, 4,294,967,296. With more than 4 bytes of capacity, it is possible to uniquely represent at least the current Internet host. The barcode proposed by the present invention preferably has an information capacity of at least 4 bytes. The exact size is determined by considering the following error handling and determining the shape.

<Error handling>

Error handling can be classified into two categories. (1) error correction is possible and (2) detection only. Error correction is also possible in the detection of errors, but it is necessary to determine and use one of two purposes clearly. If an error is detected, either discard it or attempt to correct it, or do one of two things. There is no way to find out how many errors have occurred. As a result, it is impossible to correct one error and treat it as more than one. However, bar code applications also require error correction attempts, and it is important to know exactly if that fails. You should not accept bad code as it is and cause problems for your application.

The two-dimensional barcode of the present invention uses a hierarchical error processing method. The error handling phase is divided into two layers. As shown in FIG. 3, an attempt is made to correct the error in the first step S10 and a determination is made in the second step S20 as to whether it has succeeded or failed. So, if it is determined that the failure in the second step is considered as a barcode recognition failure (S22), if successful, error correction is approved (S21).

In order to enable the error processing step by step, an error detection code should be attached to the data and an error correction code should be attached to the error detection code and the data. In step 1 (S10), an error correction is attempted and its success is determined in step 2 (S20). Hamming codes are used as error correction methods and checksums or cyclic redundancy tests are used for error detection. This determined the capacity and type of the barcode information and the method of encoding the information. The conditions to be provided by the two-dimensional barcode of the present invention are shown in the following [Table 3].

Information capacity and coding method of bar code of the present invention Condition unit value Information capacity byte ≥4 Information type type ID Code method
Step 1: Error Correction Hamming code Step 2: determine the error Cyclic redundancy test or checksum

Considering the above conditions, the shape of the actual barcode of the present invention is designed.

<1. Shape of Barcode of the Present Invention>

Most existing two-dimensional barcodes have a finder pattern and a timing pattern in common. In addition, it may have special patterns to compensate for distortion caused by cameras or various causes. By placing the timing pattern in the center of the barcode, it can be used for correction without additional insertion of a special pattern for distortion correction. The role of the finder pattern and the timing pattern is defined as follows.

1. Finder pattern: A pattern used by a machine to find barcodes in the recognition process, and is compared to candidates that are not barcodes.

2. Timing pattern: A pattern that indicates the size of a dot, which is a representation unit of a barcode. Usually, black and white dots are alternately arranged and provide clues about the size of the barcode.

Bar code according to an embodiment of the present invention has a square matrix of 13 × 13 fixed size. Matrix-type bar code is composed of square cells, which can be easily handled by any output device, and has the advantage of easy printing. The reason for defining a fixed size is to eliminate the increased complexity of additional processing. Since only the ID information needs to be represented, if there is a certain level of information capacity, there is no need to have a variable size to store more information. The reason for the square is to ensure that the shape remains relatively constant in all directions because free recognition of 360 degrees in all applications may be required.

The bar code of the present invention has a basic finder pattern, a timing pattern, and a direction pattern. In particular, the direction pattern exists to provide a clue to determine the direction of the barcode when the finder pattern and the timing pattern are designed in a symmetrical structure. Details of the error handling to be considered in each pattern, code area, and shape of a barcode will be described below.

<2. Finder Pattern (10)>

As shown in FIG. 4, the finder pattern 10 of the two-dimensional code 1 of the present invention may be in the form of a closed square line. Because it is a matrix barcode, it can actually be seen as a series of black cells. In addition, various patterns will be arranged in the inner space 11 of the finder pattern.

An important point in forming the finder pattern 10 is that a white margin is necessary around the finder pattern 10. The white of this margin and the black of the finder pattern 10 make a great contrast to be recognized by the recognizer. That is, a strict limitation such as the definition in the QR code that the width of the white margin surrounding the finder pattern should be 2 cells is unnecessary, and a white margin of one cell width should surround the black of the finder pattern 10. However, in order to increase the barcode recognition rate in a severely distorted image, it may be set to 2 cells or more in width.

The finder pattern 10 has a disadvantage in that a large number of candidates are generated in the recognition process because the uniqueness of the pattern is relatively low. However, it has the advantage that it can be found by using a simple and fast algorithm in the recognition process targeting the boundary of the shape in the image. Rather, these simple patterns can help to increase the refresh rate regardless of image distortion or errors. Many candidates-non-barcodes-that are generated due to poor uniqueness are excluded first through timing and direction pattern checks, and finally through error detection, which is the second stage of the error handling process. This stepwise candidate selection process results in recall, accuracy, and computational simplicity.

Regarding the discovery method of the finder pattern 10 in detail, the recognition method in the design process of the recognizer will be described in detail. Now, how the timing pattern and the direction pattern are arranged in the inside 11 of the finder pattern 10 and how the code is represented will be described.

<3. Timing Patterns (20)>

The timing pattern 20 of the present invention may be configured in such a manner that the black region 22 and the white region 21 are repeated at regular intervals, as in a dotted line. The period of repetition will indicate the size of the cell. Since the bar code of the present invention does not have a position correction pattern, the timing pattern 20 should play its role. Therefore, it is desirable to keep the distance from cells in all areas of the code relatively constant so that they are evenly affected. In addition, the structure of the two-dimensional matrix is preferably in the form of a straight line in which the black region 22 and the white region 21 are repeated along the X and Y axes. To satisfy these conditions, it must be formed in the form of an intersection line passing through the center point 25 of the barcode.

In order to configure the timing pattern 20 to pass through the center, the matrix size of the present invention was determined to be an odd number 13 × 13. As shown in FIG. 5, the finder pattern except for the timing pattern 20 and the remaining areas are displayed in gray. White as well as black is a component of the timing pattern 20, and data or other patterns may not enter the portion. Since the timing pattern 20 passes through the center point 25 of the two-dimensional code 1, it is not biased in any one area. Cells in the code area in which the data inside the finder pattern are represented are not far from the timing pattern, so there is no problem in reading the data without the position correction pattern.

The timing pattern 20 corrects the distortion of the image information so that the data can be read correctly in the recognition process. Perspective phenomenon may be considered as a typical distortion phenomenon that the timing pattern 20 can correct. When photographing the camera with the bar code 1 tilted, the portion closer to the image on the image becomes larger and the portion farther away appears smaller. As a result, the sizes of the cells representing the data of the barcode 1 are uneven.

6 shows distortion caused by camera perspective. (a) is the original image of the barcode, and (b) is an image taken by tilting the camera by printing the barcode. It can be seen that the cells constituting the same thickness finder pattern are three times different from each other on the far side and the close side. 7 shows what happens if the image in this state is read as a grid of cells of the same size. .

As shown in Fig. 7, no cell properly enters the space within the line spacing. At the very bottom, the cells are larger than the intervals, so one cell spans one gap, affecting the gaps above it. In this case, if the interval is read at the bottom, data can be normally entered at the bottom, but there is a possibility that the gap at the top is incorrectly recognized by the influence of the bottom cell. Looking at the top, we can expect two or more cells to enter the gap and not be recognized correctly. The example of FIG. 7 considers only the Y axis. The problem is further complicated by considering the X axis. It is not possible to recognize internal patterns or data with these distortions intact.

It is the timing pattern that corrects the spacing of the lines to the actual cell spacing of the barcode.

<4. Direction Pattern (31 ~ 38)>

The two-dimensional barcode 1 of the present invention has a square shape in general, and only the finder pattern 10 and the timing pattern 20 described so far are completely identical in four directions so that the direction cannot be determined. This makes it impossible to read the data properly. Therefore, in the present invention, the direction patterns 31 to 38 which further determine the direction are defined.

As shown in FIG. 8, a total of eight cells may be used for the direction patterns 31 to 38 in the preferred embodiment of the present invention. The outer four cells 31, 32, 33 and 34 form one group, and the inner four cells 35, 36, 37 and 38 form one other group, which can be used together. The three cells have the same color and only one cell has a different color to indicate the direction. In the configuration of the direction pattern of the present embodiment, in the case of the cell group configured inside, the three cells 36, 37, and 38 have a white color, and only the cell 35 located at the upper left is black. In the case of the outer cell group, the three cells 32, 33, and 34 are black, and only the cells 31 located at the upper left are white.

The basic patterns of the present invention have been defined above. It was a finder pattern, a timing pattern, and a direction pattern, and the remaining area after displaying all of them can be used as a code area representing actual data. As shown in FIG. 9, the gray-colored part after displaying all the basic patterns is a part representing data and is composed of a total of 92 cells. This completes the basic barcode template. Next, the code area in which the actual data is represented will be described.

<5. Code Area>

As described above, except for the basic patterns constituting the template of the barcode of the present invention, the code area used to represent actual data may be composed of a total of 92 cells. If data is represented as it is, 92 bits can be represented. This is about 11 bytes. However, if Hamming code and error detection method are applied for error handling, the capacity to store the actual information is smaller than this.

First, since the codeword of the Hamming (7, 4) code is 7 bits long, a total of 13 codewords can be represented. If 13 codewords are represented, one cell remains, and a total of 52 bits of information can be represented. One remaining cell is left blank. Of the total 91 cells represented by the Hamming (7, 4) code, up to 13 cells are damaged or distorted, so that an error can be recognized successfully.

Now, secondly, a space for the check value for error detection is allocated to the 52-bit information capacity previously given. The maximum number of bytes that can be represented by 52 bits is 6 bytes, leaving 4 bits. Six bytes are used to represent information and four bits are used as check value storage. This determined most of the characteristics related to the appearance of the barcode. Finally, the last thing to consider in the shape of barcodes for error handling is explained.

<6. Error Handling>

In the above, it is possible to correct errors of up to 13 cells out of 91 cells. In some cases, however, only two errors can make the condition impossible to correct. One codeword is 7 bits long, and it is possible to correct one error for each codeword. If two or more errors occur in one codeword, the entire recognition may fail even if all of the other codewords are normal.

Errors in barcode recognition can be thought of as being divided into sporadic and local. If sporadic errors appear randomly, they can no longer be solved in any way. However, countermeasures can be taken for local errors. By differently arranging the bits of the codeword, it is possible to reduce the probability of intensive exposure of a codeword to a local error on a two-dimensional plane. In order to do so, it is necessary to reduce the bits placed in the codeword adjacent to each other. But on the other hand, the placement method should not be too complex to increase the computational complexity in the recognition process.

10 exemplarily shows how to arrange a codeword in a code region of a barcode. As shown in FIG. 10, seven bits of each codeword may be arranged in different places, if the code is divided into numbers from 1 to 13 of each of the 13 codewords. In particular, only cells corresponding to codeword 1 are grayed out. The number of cells in which codeword 1 is to be expressed is a total of seven. The seven bits of the Hamming (7, 4) code are filled sequentially from the first cell above to the first cell below. The expression <sequential> here means <distributed arrangement so as to be robust to locally concentrated errors>. In the case of FIG. 10, when 13 codewords are arranged in a code region, the 13 codewords are arranged in order from left to right, but in another embodiment, the 13 codewords may be arranged from top to bottom.

In this case, even if a local error is caused by any cause, the probability of damaging one codeword is reduced. If sporadic errors are evenly distributed, there is no special way to do this. But at least you don't get much worse performance than just lining up. With this, we have examined all the error handling to be considered in appearance. With this configuration, the bar code design of the present invention is now completed.

The procedure for actually recording information on the designed barcode is as follows.

First, a check value is calculated and attached to the 48-bit ID information to be expressed to generate a total of 52 bits of data. This data is generated using a Hamming (7, 4) code to generate a 7-bit codeword in 4-bit units. You can now fill the code area of the barcode template sequentially from the top left. At this time, 13 codewords are filled in order from the 1st bit to the 7th bit by alternating each codeword in the order of 1st bit of 1st codeword, 2nd bit of 2nd codeword, and 3rd bit of 3rd codeword. I'm going. Finally, the seventh bit of codeword 13 is filled. Fill each bit with 1 in black and 0 in white to represent the cell.

Hereinafter, a process of receiving and recognizing a novel two-dimensional barcode printed material having the above configuration through a camera will be described in detail.

12 is a schematic and exemplarily illustrating an entire process for a barcode recognition method of the present invention. The grayscale image obtained by photographing the barcode printed by the camera embedded in the mobile device is generated as input data (S101), and the boundary line of the shape of the image data is detected (S102) to generate one or more candidate pieces. Create a candidate group list (S103). Next, the vertices of the finder patterns are found with respect to each candidate included in the candidate list (S104).

Through the above process, the candidates including the outline of the finder pattern and the vertices of the candidates are obtained, and then the paths of the parts where the finder pattern is located are obtained using these vertices. Then, it is checked whether all of the finder pattern portions are black in the grayscale image which is the input data using the path (S106).

Next, the grayscale image array is generated with respect to the inclined barcode image (S107), and the image information of the distorted array is corrected using the timing pattern information (S108). Thereafter, candidates with wrong direction patterns are excluded from the list by using eight cells constituting the direction patterns (S109).

After performing the above steps, the information on the two-dimensional barcode is actually read to generate a bit string. That is, the codewords of 13 Hamming (7, 3) codes represented in the barcode are read out and serialized. Then, error processing is performed on the bit string obtained by serialization (S111), and the error is detected by the hierarchical method as described above (S112). Next, the threshold value is obtained (S113), and the ID information of the approved barcode is obtained (S114).

Each step of FIG. 12 as above will be described in more detail below.

<A. Input data generation step>

The video provided by the camera built into most mobile devices is represented in YV12 color space. Image data expressed in the YV12 color space can be used as the gray scale image of the Y plane in front of it. This grayscale image is used as input data to the recognizer.

In this embodiment, the barcode of Fig. 11 is printed and used. The two-dimensional bar code of FIG. 11 is a bar code for data in which all ASCII check values of the character string “Hello!” Are set to zero. The hexadecimal representation of this 52-bit long data is 0x48656C6C6F210.

Next, FIG. 13 is a 640x480 resolution grayscale image obtained by photographing the barcode of FIG. 11 with a camera. In the present embodiment, the image of FIG. 13 will be used to describe all the recognition processes from the boundary detection step, which is the beginning of the recognition process. Hereinafter, all steps from boundary detection, candidate fragment generation, four vertex finding, finder pattern inspection, alignment, distortion correction, direction determination, serialization, and error processing will be described.

<B. Boundary Detection Step>

The edge detection can grasp the shape of an object or a pattern in the image. It is a step of detecting the boundary of the shape, and an object thereof is to find the outline of the finder pattern of the barcode. One threshold is used for boundary detection. Scans in the X-axis and Y-axis directions of the image and examines the amount of change in the pixel values of the grayscale. Then, if the magnitude of the change amount is larger than the threshold, it is determined as a boundary. Using the position determined as the boundary, the display point is left on another resultant image area.

After all the scans are completed in both directions, the display points are connected to the left image area to form a boundary line. The results are shown in FIG. This boundary detection can be used to generate clues such as the finder pattern where white and black are contrasted to find the large difference.

Another method that can be considered simpler than the method described above as the boundary detection method is to obtain the boundary between the black and white dots in the binary image after performing binarization on the image. However, the method described above is faster and better finds finder patterns in many problems. First, the speed is increased because the number of times of accessing the image data is much reduced. And the problem peculiar to binarization is solved. The binarization method using a single boundary value has a problem that a barcode in the area cannot be found when the image is dark or light locally, and the binarization method that produces accurate and good results has a large complexity of calculation. And since the form of data required in the next step requires only the outline, not the binary image, it is better to use the method to find the result faster and better.

The key to boundary detection is to find the appropriate threshold. If the threshold value is too large, the candidates for the barcode decrease, increasing the likelihood that the actual barcode cannot be found. On the contrary, if the threshold value is too small, candidates other than the barcode will be included, thereby reducing efficiency due to unnecessary processing. In other words, the appropriateness of the threshold value can affect the recall and performance.

In the present invention, the method used in ARToolKitPlus is applied to a method for calculating an appropriate threshold. ARToolKitPlus is the method used to calculate the threshold for binarization, but can be similarly applied. It utilizes the image information of the barcode successfully recognized in the previous recognition process. Using the points in the area of the barcode successfully recognized on the previous grayscale image, since the barcode in the real environment is recognized through the image, various factors in the environment are reflected and smaller than the sudden change in the characteristics of the continuous image. Since most of the change is continuous, the previous success is high enough to support the next.

The threshold is calculated from the difference between the boundary between the black and white sections of the barcode that were successfully recognized. The purpose of the threshold is to find that boundary. For all points of the barcode area on the grayscale image, the median of the differences between the respective points and the surrounding points of the points is taken as the threshold. The minimum of the differences will be one of the differences between the same black or white points within the black or white area, and the maximum will be one of the differences between the black and white points located at the boundary of the black and white areas, respectively. Will be.

However, the boundary detection method using these thresholds has a problem in that if the previous recognition process is unsuccessful, it continuously affects the next process. If none of the barcodes have been successfully recognized to solve this problem, then an attempt to increase the recall is required in the next step, which gradually reduces the threshold to a predefined limit.

The result of the boundary detection through this process is shown in FIG. 14. In the figure, the boundary information consists of several pieces. Among the pieces are the boundaries of the barcode and the contents of the barcode. It also includes the outlines of letters that are not related to the recognition process. The candidate fragment generation step of separating and recognizing these candidate fragments is described in the next section.

<C. Candidate Fragmentation Steps>

Binary images containing several boundary lines obtained in the previous boundary detection step contain a lot of meaningless information in addition to barcodes. And even the outline of the barcode finder pattern has no meaning beyond that of a set of dots. The first step in identifying the meaning of the pieces, the set of points, is to create a list of candidate pieces. Here, the fragment means a set of point coordinates extracted in the boundary detection step. Some pieces are the outline of the actual finder pattern, and many others are meaningless pieces.

In the candidate fragment generation step, it is necessary to first separate the points in pieces. That is, a set of adjacent points on the boundary image is collected. This process starts with finding an arbitrary point in the image. Place the points you find in sets and stacks and delete them from the image. Then find the adjacent point again at that position and delete the point again in the set and stack. This process is repeated for the adjacent point at that location. When two or more points are adjacent, simply select any one of them and repeat the process. Then if there are no more neighbors, you can simply take one location off the stack and iterate over that location. If there are no neighbors, repeat the process of removing them from the stack. If there are no adjacent points and the stack is empty, a set is completed and this set can be a candidate piece.

The candidate piece simply calculates its size and discards it if it is less than the specified value. Because a piece that is too small is difficult to contain all the information that can be recognized, even if it is an image of the outline of the barcode finder pattern. This is caused by not shooting close enough, and such images are worthless because they are crushed in shape.

This simple calculation can effectively reduce the number of candidates without compromising recall. In this embodiment, the size is set to 39 once. The size is multiplied by 3 by 13 × 13, the size of the barcode finder pattern, taking into account that at least one pixel is affected by the next color when white and black are adjacent in the image acquired by the camera. .

15 shows a result of generating candidate pieces of pieces having lengths of 39 or more in both horizontal and vertical lengths. A total of two pieces are generated as candidates. In addition, the pieces formed by the connected points are each stored in a set unit so that logical processes can be performed more efficiently in later steps. The trailing steps leave the outline of the actual finder pattern, and the wrong candidate remaining inside the barcode goes through the trailing steps and is removed from the list. Now, in the next step, it is the whole process of the alignment step to read the tilted bar code image correctly, and it performs 4 vertex search which is a clue to determine posture and position.

<D. 4Find Vertex Steps>

In this step, we find the vertices of the finder pattern. Since the finder pattern is square, we find four vertices. Because this step is performed on the candidates created in the previous step, non-rectangular pieces are also targeted, which finds the vertex of the largest rectangle that the element points of the piece can achieve. First, all candidates are assumed to be rectangular, and after finding vertex 4, the candidates are reduced again by a simple finder pattern test in the next step.

Four vertices are clearly found in some form of outline. Therefore, the boundary line was detected in the previous step and the process was performed as a candidate. It's inefficient to consider all the stuff inside to find the vertices. Finding vertices proceeds sequentially with each of four vertices as follows.

Find the first vertex: First select any one of the points in the piece. The distance to the point is calculated for all the points in the piece, and the point where the distance is the maximum is the first vertex.

Find the second vertex: Similar to finding the first vertex, this time determining the distance from the first vertex. The distance from the first vertex is calculated for all the points in the piece, and the second vertex is the point at which the distance is the maximum.

Find the third vertex: This time, determine the distance between the first and second vertices. The third vertex is the point at which the sum of the distance between the first vertex and the second vertex becomes the maximum for all the points in the piece.

Find the fourth vertex: Unlike calculating the distance to find the first three vertices, the fourth vertex must calculate the area. The fourth vertex is a point that maximizes the area of the rectangle formed with the first vertex, the second vertex, and the third vertex for all the points in the piece. The equation used to find the area is shown in Equation 1 below. In Equation 1, x _n denotes the X coordinate of the nth vertex, and y _n denotes the Y coordinate of the nth vertex.

(식1)

(Eq. 1)

The results of finding the vertices in the above manner are shown in FIG. 16. In FIG. 16, the rectangles formed by the four vertices are indicated by a dark line, and the candidates generated in the previous step are indicated by a blurry line to confirm the shape of the candidates. You can see that the finder pattern finds a vertex of a rectangle that almost matches its outline.

The vertices found in this way finally find the center points of the four points and align them counterclockwise with respect to the angle that each point forms with the center point. The calculation of the angle used in this sort and the distance calculation used in the previous vertex finding process do not use any real arithmetic. This is because most mobile devices don't have floating-point units, which slows down real operations.

The vertices found in this way are aligned so that they are always regular in that location. Each candidate piece has a first vertex at the bottom left, a second vertex at the bottom right, a third vertex at the top right, and finally a fourth vertex at the top left from the center point. Now use these vertices to perform a finder pattern check in the next step.

<E. Finder Pattern Inspection Steps>

The finder pattern of the barcode recognized by the present invention has a closed square frame shape. Through the above steps, candidates including the outline of the finder pattern were generated, and vertices of the candidates were found. Now use the vertices to find the path to where the finder pattern is located. The path is used to check whether all the finder patterns are black on the grayscale image as input data.

와

지점에 각각 안쪽의 점들과 바깥쪽의 점들을 구한다. 이 지점의 거리는 바코드의 한 폭이 13셀의 길이임을 감안하여 중간점과 파인더패턴의 외곽선이 이루는 길이가 6.5셀이 된다는 계산에 의해 얻어진 것이다.If the original shape is not square, all of the vertices from the previous four vertices will not appear black. Here a threshold is needed to distinguish between black and white. If the value is lower than the threshold, it is black. Otherwise, it is white. To get this threshold, we need to look at the 4 vertices

Wow

Find the inner and outer points at each point. The distance of this point is obtained by calculating that the length of the intermediate point and the outline of the finder pattern is 6.5 cells, considering that the width of the barcode is 13 cells.

If the candidate matches the finder pattern, the line formed by the inner points is the path passing over the finder pattern, and the line formed by the outer points is the path passing over the margin. The finder pattern is black and the margin is white. The median of the values of this path on the grayscale image can be obtained and used as a threshold to check whether the values of the inner path are all below. In consideration of errors caused by various causes, when more than 80% is black, it is judged as barcode. If you're not a candidate, the outside can be black and the inside can be white, or else the inside finder pattern paths aren't all black.

지점의 안쪽 경로를 그레이스케일 상에 점선으로 표시한 것이다. 눈으로 확인해도 안쪽의 후보는 경로의 많은 부분이 검정색이 아님을 쉽게 알 수 있다. 이 파인더패턴 검사에서도 걸러지지 않는 후보들은 이후의 과정을 통해 몇 회 더 걸러지며 제외된다. 최후에까지 검사를 통과하더라도 결국 체크값 검사 과정에서 걸러지게 된다. 그러나 이번 경우에는 실제 파인더패턴의 외곽선만 남게 된다. 그 결과가 도 18에 나타난 바와 같다.17 shows the vertices of the actual finder pattern to be examined.

The inner path of the point is shown as a dotted line on grayscale. Visually, the inner candidate can easily see that much of the path is not black. Candidates that are not filtered out by the finder pattern check are filtered out a few times later. Even if the test passes to the end, it is filtered out during the check value check. In this case, however, only the outline of the actual finder pattern remains. The result is as shown in FIG.

In calculating thresholds or examining finder patterns, it is often necessary to calculate the path of a straight line between two points. Again, you need to use only integer arithmetic to improve computational performance. Therefore, the Bresenham algorithm is used to find the straight path between two points only by comparison operation and addition operation of integer except the multiplication of 3 integers.

This ended the inspection of the finder pattern and finally ended the role of the finder pattern. The finder pattern fulfilled the purpose of finding barcodes. In the following steps, the arrangement for subsequent steps for attempting to interpret the information of the barcode is described.

<F. Arrangement Steps>

In this step, a gray scale array of size 39 × 39 is generated by using a bar code inclined to the image. The grayscale image is used again because it contains the necessary information. The borders and vertices of the outline are information used to determine the location of the barcode, and grayscale information is needed in the process of determining codes and calculating thresholds.

The array created in this step is subsequently serialized in the serialization step considering the direction determined in the direction determination step after the correction work in the distortion correction step, and finally through the error processing step for the serialized data, Determine if it fails. Successful barcodes return position, posture, and code values as a result. In addition, the threshold value is used for the boundary detection in the next recognition process from the successful barcode.

The process of reading barcodes into a fixed size array causes a lot of information loss. However, this is also the best way to increase the efficiency of the calculation. The size of the barcode is 13x13 cells, so if you define an array to hold it with the minimum size, it will be a 13x13 size array. However, the size of the array actually used in the arranging step is larger than 39 × 39. The reason is to leave room for the distortion correction performed in the next step. If you try to interpret the information in a 13x13 size array, you will be exposed to the distortion problem described in the Timing Patterns section of the previous section, which suggests the shape of the barcode.

The size 39 is equal to 39, the size used as the minimum value of the candidate in the candidate fragment generation step. The reason is that there is a possibility of a candidate of that size, and then it is not worth reading into a larger array of sizes.

In Fig. 19, 39 x 39 points calculated using four vertices and an array read from the points are shown. Calculate by applying Equation 2 below to find the coordinates using four vertices.

(식2)

(Eq. 2)

과 점

사이의 비율r 지점의 점을 구할 수 있다. 여기서 비율을

,

,

, ... 등으로 바꾸면 13개 셀의 각 셀의 중심좌표를 구할 수 있다. 제안하는 바코드의 인식과정은 39× 39크기의 배열을 읽어 들여야 하기 때문에 비율을

,

,

, ... 의 비율을 사용한다. 이 수식을 이용해 우측상단과 좌측상단의 꼭지점 간에 이러한 39개 좌표들을 구하고, 좌측상단과 우측상단의 꼭지점 간에 이러한 39개 좌표들을 구하여 그 각각의 좌표들 쌍 39개에 대해 다시 이 수식을 적용하여 구하면 총 39× 39개의 점들의 좌표를 구할 수 있다. 이어지는 절에서 이 배열에 대한 왜곡보정을 설명한다.
This formula gives you a point

And dot

You can find the point at the ratio r between. Where the ratio

,

,

Replace with ..., to get the center coordinates of each cell in 13 cells. Since the proposed barcode recognition process has to read an array of 39 × 39 size,

,

,

Use the ratio of, ... Using these formulas, we get these 39 coordinates between the upper right and upper left vertices, and we get these 39 coordinates between the upper left and upper right vertices, and apply these formulas to each of the 39 pairs of coordinates. Coordinates of a total of 39 × 39 points can be obtained. The following section describes the distortion correction for this arrangement.

<G. Distortion Correction Step>

Distortion correction is an operation of correcting image information of a distorted array using information of a timing pattern. The bar code "Hello!" Used in this embodiment is in a good state so that the subsequent process can be performed normally without distortion correction, but in the case of the image shown in FIG. With a uniform size configuration of 3, it can be difficult to interpret 13x13 cells.

Unlike the case of FIG. 18, in the case of FIG. 19, the arranged images are slightly pushed to the left and upward. At first glance, it looks similar, but the distortion is more serious than you think. This may not be possible. Such a rolling phenomenon does not cause local error, but rather affects the whole, and thus a further solution is required. In order to confirm more accurately, the figure is shown in Fig. 21 by dividing the array by 3 × 3 size.

As shown in Fig. 21, the entire array is pushed out little by little to affect the next cell recognition range. The left and bottom finder patterns look almost normal. And the top and left finder patterns seem to be pushed back only one space, so the problem may not seem serious. But the distortion in the middle, which is really important, is serious. The cells and the timing pattern of the center point will be recognized by pushing almost one cell. The severity of the distortion can be easily understood by simply averaging the values in the divided units of FIG. 21 and comparing the original barcode with the result of binarization using the average values.

As shown in Figure 22, the averaged image looks quite confusing. The binarized image is on the right, and the original barcode is again shown on the right for comparison. Comparing the original barcode with the binarized array, the difference is seen in 69 cells. The total number of cells excluding the finder pattern is 121, resulting in a 57% error. In addition, it is impossible to recognize the direction pattern in the middle. Fortunately, the outer direction pattern can be recognized, but the middle direction pattern and the timing pattern can not find a trace. In fact, only in the averaged array, it can be seen that the outward direction pattern is not judged to be completely reliable. The problem is even worse if direction recognition fails. In the case of serialization, the data is read in a completely different order, in which case recognition will fail.

To solve this problem, the timing pattern was defined in the barcode design. In the previous step, the image was read in a 39 × 39 size array larger than 13 × 13 cells, leaving room for correction in this step. Now we actually try to correct the distortion using that pattern and arrangement.

Various methods can be considered for distortion correction, but we propose a simple and effective method rather than increasing the computational complexity. The method proposed by the present invention is a method using some predefined grids. First of all, one can think of a 13 × 13 grid composed of 3 × 3 size uniform units that can read normal barcodes without distortion. In addition, an asymmetrical grid is needed to recognize the image photographed by tilting to either side. The proposed method reduces the complexity and generates a total of nine gratings by simply combining one asymmetry ratio and a symmetry ratio for efficient processing. There may be many cases where the exact inconsistency does not exist, but some degree of inconsistency is likely to be recognized as normal, and even a small amount of errors may be corrected by a Hamming code.

It can be seen that the ratio of FIG. 23 is narrow in the left side and wide in the right side. Applying the ratio to the right on the side where the camera is tilted closer will help to read it properly. Nine grids use 3 types of vertically narrow, right-right and symmetrical ratios, and 3 types of horizontally narrower, narrower and lower symmetry ratios. It consists of branches. The actual configuration of the nine gratings is shown in FIG. 24 below.

All grids are narrow on one side except the middle one. 23 shows the position of the finder pattern. This shows how each grid is narrowly inclined. The arrangement shown in FIG. 19 uses the middle grating and the arrangement of FIG. 19 shows the best calibrated result when the grid is narrow in the upper left. Whether or not it is well calibrated can be determined by applying the grating and determining whether the timing pattern is properly matched. Apply the nine grids in order to find the best matching grid.

The degree of coincidence with each lattice arrangement is determined by the sum of the differences between the expected values of the timing pattern expected portions of each lattice and the actual array values. The smaller the sum, the greater the degree of agreement.

Although the grating for distortion correction of FIG. 25 does not exactly match the distorted arrangement, there is no problem in determining and reading the value of the cell. The direction can now be determined correctly and the subsequent serialization steps can be performed.

<H. Directional Step>

From this step, the process proceeds back to the arrangement obtained from the image which was not severely distorted. 26 is that. In fact, the lattice match is lower than the result after the distortion correction of the image, which was severely distorted in the previous step. But as explained there, some level of inconsistency doesn't matter. Positions of cells constituting the direction pattern for determining the direction are as shown in FIG. 27 below.

For convenience of description, each of the eight cells constituting the direction pattern is numbered. Determining the direction can be done very simply. It finds the darkest value among cells 1, 2, 3, and 4, and finds the brightest value among cells at positions 5, 6, 7, and 8. The darkest point and the innermost light point of the cells constituting the outer four direction patterns each represent the upper left side of the barcode. If the two directions differ in this process, the candidate is excluded from the list. In the case of the “Hello!” Barcode, the recognized direction coincides with the direction of the array, and the next process is performed without any further direction processing. If not, the direction information is passed to the next step to adjust the starting point and the serialization direction when serialization is performed in the next step.

<I. Serialization Steps>

We are now in the stage of actually reading the information of the two-dimensional barcode and generating the bit string. In this step, 13 Hamming (7, 3) codewords represented in the bar code are read and serialized. This serialized bit string is applied to the error handling method through the following steps to correct and check the error to obtain the final code. Codewords are arranged in a cyclical order, not sequentially. Therefore, in order to read one codeword, it is necessary to read at regular intervals rather than to generate bit strings through sequential access at a certain position. Alternatively, it may be possible to sequentially read the cells represented by sequentially accessing the information of the cells and recursively generate the information of the cells into a queue storing 13 codewords.

In the method proposed by the present invention, the latter is selected. Once this is done, binarization of the array is performed before generating the codeword. Of course, the actual implementation does not need to store the results of binarization separately. This is because the points of the array are sequentially approached by the grid units determined in the previous distortion correction step, and the binarization is performed by the grid units, and the results are alternately stored in 13 codewords.

First, the result of binarization is as shown in FIG. 28, and the number of the codeword queue to which the binarization result for each unit is to be stored is indicated.

In addition, since the barcode is in the correct direction, it reads from the upper right side to the left side and when it is read down, it goes down and repeats the process. If the direction is different, the starting position and the reading direction are different. For example, if the barcode is rotated 90 degrees clockwise and the direction pattern points to the upper right side of the barcode, the starting point is point 5 on the upper right side of Fig. 28, and 2, 12, Units marked with 7, ... will be read sequentially. Of course, after reading that line, you move one space to the left to continue.

The state of the 13 codeword queues storing the actual serialized results is shown in FIG. When the codewords are sequentially concatenated, a bit string containing an error correction code is generated. Error processing is performed in the next step using the value of the bit string.

<J. Error Handling Steps>

The codewords obtained in the previous step are serialized to obtain a bit string 1001100 1110000 1100110 0100101 1100110 0111100 1100110 0111100 1100110 1111111 0101010 1101001 0000000, and error processing thereof is performed. The first step in error handling is an attempt to correct using a Hamming code. Luckily, the code in the example does not contain errors, but there are a variety of possible causes that can be included and correcting these errors should be done in the first step. Error handling originally requires a computation of parity bits on the definition of the Hamming (7, 4) code and the resulting error location, but utilizes a conversion table for more efficient processing.

The conversion table method uses a table that defines the relationship between the codeword representation and the original code. You can use this table to immediately find out what code the original wordword representation represents and interpret the sign without any complicated calculations. In addition, all codewords of Hamming distance 1 that can be corrected are defined in a code-to-many relationship to ensure consistent and fast processing. Since the codeword is only seven, the number of table relationships is only 128, which is a total of 2 ⁷ .

As a result, the bit string 0100 1000 0110 0101 0110 1100 0110 1100 0110 1111 0010 0001 0000 is obtained through the conversion using a table. This is 0x48656C6C6F210 in hexadecimal notation, exactly like the sequence of ASCII code values in the message “Hello!” That was originally expressed when the barcode was created. In addition, a checksum or cyclic repetition test shall be conducted to check for errors, and candidates may be finally approved or discarded depending on whether or not the error is detected.

The approved barcode is now used to calculate the threshold for the first step of recognition, boundary detection. This threshold is used in the next recognition process. The method of calculating this threshold is explained in the following section.

<K. Threshold Calculation Steps>

The threshold is a value used to check for more than a certain difference in the magnitude of values between adjacent grayscale points during boundary detection, which is the first step of recognition process for grayscale image input data. If it is larger than this value, it is determined as a point forming a boundary, otherwise it is determined as a point forming a normal plane rather than a boundary. Since candidates are generated from this boundary and all recognition processes are performed, the appropriate threshold size is a very important factor affecting recognition performance, recall and accuracy. Too large a threshold may miss the actual barcode finder pattern outline, and too small a threshold may overproduce false candidates in addition to the barcode finder pattern.

As the process is repeated, changes in the environment will be subtle and continuous. Therefore, using the threshold obtained in the previous step in the next step is a good way to keep simple environmental requirements in mind. But a little change is obvious, so you need to calculate it with some margin.

The threshold should be able to extract the boundary of the outline of the finder pattern for that purpose. Thus, the threshold should be less than the difference between the white area of the barcode and the boundary of the white area. If possible, use the largest value from which all boundary lines can be extracted. That is, the minimum value of the difference values forming the boundary may be used. In the following procedure, we will set a slightly smaller amount to allow for some changes.

In order to find the threshold, first, the binarization is performed by using the grayscale array of barcodes. Then, the grayscale average value of the area determined to be white and the grayscale average value of the area determined to be black are obtained. The difference between each value is obtained, and a value corresponding to 3/4 size of the difference is used as a threshold.

So far, the process of recognizing the proposed barcode through the design of the recognizer of the present invention has been described step by step. Starting from receiving grayscale image, candidates are generated, various candidate tests are performed, and error processing is performed at the end to recognize the value represented in the barcode and the position or direction of the barcode. Using the information of the 4 vertices and the direction information obtained in the direction determination step, the exact position, direction, and posture of the bar code are indicated, and through the error processing, the ID information of the bar code is obtained with the approved code until the final error checking step. Passing this information to the application system makes it available to the application system.

On the other hand, the scope of protection of the present invention is not limited by the embodiments explicitly described above. Further, it should be noted that the protection scope of the present invention may not be limited due to obvious changes or substitutions in the technical field to which the present invention belongs.

Claims (7)

A method of recognizing a matrix barcode consisting of cells of a rectangular grid:
A first step of correcting an error occurring when a barcode recognition device of a mobile device photographs the matrix barcode with a built-in camera; And
Detecting an error with respect to the corrected image of the matrix barcode obtained through the first step, and approving the error correction of the first step only when no error is detected.
Recognition method of two-dimensional matrix barcode. The method of claim 1,
The matrix barcode is,
A finder pattern represented by a square line shape;
A timing pattern positioned inside the finder pattern and having a black region and a white region repeated at regular intervals;
A direction pattern for determining a direction of the barcode; And
A code region for storing data is formed in a cell excluding the patterns, and the data stored in the code region is read using the patterns. A finder pattern expressed in the form of a square line, a timing pattern positioned inside the finder pattern and having a black area and a white area repeated at regular intervals, a direction pattern for determining a direction of a barcode, and A method of recognizing a matrix barcode consisting of cells of a rectangular grid having a code area for storing data in a cell except for patterns by photographing the camera with a built-in mobile device:
(1) generating a grayscale image of the matrix barcode;
(2) detecting a boundary of the image;
(3) generating a candidate list as a result of step (2); And
(4) performing error correction on each candidate created in the candidate list;
Receiving a two-dimensional matrix barcode, characterized in that it comprises a. The method of claim 3,
Step (4) is,
Finding four vertices of the candidate finder pattern; And
And obtaining a path of a portion where the finder pattern is located by using the four vertices. The method of claim 4, wherein
Generating an array of grayscale images for the tilted barcode image; And
And correcting the distorted array of image information by using the information of the timing pattern. The method of claim 3,
And excluding candidates having an incorrect direction pattern from the list by using the direction pattern. The method of claim 3,
Reading and serializing a codeword recorded in the code region;
The step (4) includes performing error processing on the serialized bit strings.

Images