KR20200136239A - Method and apparatus for recognizing image code using machine learning - Google Patents

Abstract

Provided are a device for recognizing an image code using machine learning and a method thereof. The device for recognizing an image code comprises: a control unit extracting a pixel area corresponding to an image code in an image photographed by a camera to convert the pixel area based on a relative position between vertices on the boundary of the pixel area, and recognizing the image code in the converted pixel area; and an input/output unit obtaining the image.

Description

Device and method for recognizing image codes using machine learning {METHOD AND APPARATUS FOR RECOGNIZING IMAGE CODE USING MACHINE LEARNING}

The embodiments disclosed in this specification relate to a method and apparatus for recognizing an image code using machine learning, and more particularly, to identify and identify a region in which an image code is displayed in an image captured by a camera using machine learning. The present invention relates to a method and apparatus for increasing the recognition rate of an image code by converting an image code whose shape is distorted according to a point in time at which the image code is captured.

With the spread of simple payment and the like, a method of obtaining payment information using an image code is becoming more common.

When a user photographs an image code using a camera of a terminal, the image code is identified from the captured image, and information included in the image code is extracted to identify payment or a product to which the image code is attached.

In this case, the image code included in the captured image may be distorted according to the direction and angle at which the user shoots the image code, and a problem in that the image code cannot be identified from the captured image may occur.

Alternatively, as the image code is distorted according to the time of photographing, a problem of misrecognizing the image code may occur.

A related prior art document, Korean Patent Publication No. 10-2010-0034888, relates to a barcode scanner having an image capturing function, which can read not only one-dimensional barcodes but also two-dimensional barcodes, and a barcode scanner without a separate camera. It provides a barcode scanner with an image capture function that makes it easy to store image information, manage inventory, and manage documents as it can perform scanning operation and image capture. It does not accurately identify distorted image codes at the time of shooting. can not do it.

Therefore, there is a need for a technique to solve the above-described problem.

On the other hand, the above-described background technology is technical information that the inventor possessed for derivation of the present invention or acquired during the derivation process of the present invention, and is not necessarily known to be publicly known before filing the present invention. .

The embodiments disclosed in the present specification aim to provide an apparatus and method for recognizing an image code using machine learning.

The embodiments disclosed in the present specification aim to provide an apparatus and method for recognizing an image code that accurately identifies an image code from an image captured from a camera viewpoint through machine learning.

The embodiments disclosed in this specification are to provide an apparatus and method for acquiring a pixel area corresponding to an image code distorted according to a photographing point and recognizing a converted image code based on the vertex of the acquired pixel area. There is this.

Embodiments disclosed in the present specification aim to provide an apparatus and method for recognizing an image code for identifying whether or not the image code in a converted pixel region is normally recognized.

As a technical means for achieving the above-described technical problem, according to an embodiment, in an apparatus for recognizing an image code, a pixel area corresponding to an image code is extracted from an image photographed by a camera to determine the boundary of the pixel area. A control unit for converting the pixel area based on a relative position between the vertices of the image and recognizing an image code in the converted pixel area, and an input/output unit for obtaining the image.

According to another embodiment, in a method of recognizing an image code by an image code recognition apparatus, extracting a pixel area corresponding to an image code from an image captured by a camera, and a relative position between vertices on a boundary of the pixel area It may include converting the pixel region based on and recognizing an image code in the converted pixel region.

According to another embodiment, in a computer-readable recording medium in which a program for performing an image code recognition method is recorded, the image code recognition method extracts a pixel area corresponding to the image code in an image photographed by a camera. And converting the pixel region based on a relative position between vertices on the boundary of the pixel region and recognizing an image code within the converted pixel region.

According to another embodiment, in a computer program executed by an image code recognition device and stored in a medium to perform an image code recognition method, the image code recognition method corresponds to an image code within an image photographed by a camera. And extracting a pixel area to be converted, transforming the pixel area based on a relative position between vertices on a boundary of the pixel area, and recognizing an image code in the converted pixel area.

According to any one of the above-described problem solving means, an apparatus and method for recognizing an image code using machine learning can be provided.

According to any one of the above-described problem solving means, it is possible to provide an apparatus and method for recognizing an image code that accurately identifies an image code from an image captured from a camera viewpoint through machine learning.

According to any one of the above-described problem solving means, an apparatus and method for acquiring a pixel area corresponding to an image code distorted according to a photographing point and recognizing a converted image code based on the vertices of the acquired pixel area are presented. I can.

According to any one of the above-described problem solving means, it is possible to provide an apparatus and method for recognizing an image code for identifying whether or not the image code in the converted pixel region is normally recognized.

The effects that can be obtained in the disclosed embodiments are not limited to the above-mentioned effects, and other effects not mentioned are obvious to those of ordinary skill in the art to which the embodiments disclosed from the following description belong. Can be understood.

1 is a block diagram showing an image code recognition apparatus according to an embodiment.
2 is a flow chart for explaining an image code recognition method according to an embodiment.
3 to 5 are exemplary diagrams for explaining an image code recognition method according to an embodiment.

Hereinafter, various embodiments will be described in detail with reference to the accompanying drawings. The embodiments described below may be modified and implemented in various different forms. In order to more clearly describe the features of the embodiments, detailed descriptions of matters widely known to those of ordinary skill in the art to which the following embodiments belong are omitted. In addition, parts not related to the description of the embodiments are omitted in the drawings, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a component is said to be "connected" with another component, this includes not only the case where it is'directly connected', but also the case where it is'connected with another element in the middle.' In addition, when a certain configuration "includes" a certain configuration, this means that other configurations may be further included rather than excluding other configurations, unless otherwise specified.

Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings.

However, before describing this, the meaning of the terms used below is first defined.

The'image code' is a code that is optically recognized and transmits information, and may include, for example, a one-dimensional barcode or a QR code.

The'pixel area' is an area composed of pixels corresponding to the image code in the image in which the image code is captured.

'Machine learning' is a field of artificial intelligence (AI) that learns data and processes newly inputted data. For example, deep learning using artificial neural networks and Bayesian networks can be included.

At this time, the'artificial neural network' is an information processing technology that engineeringly mimics the advanced information processing mechanism of the biological nervous system and correlates inputs and outputs in detail with each other to perform complex control. Input layers that send signals from switches or sensors, etc. It is a network of three types of neuron (neuronal cell) models consisting of a hidden layer that adjusts the interrelationship while prioritizing the input and output based on and an output layer that calculates and outputs the required amount of control based on this.

In addition to the terms defined above, terms that require explanation will be separately described below.

1 is a block diagram illustrating an image code recognition apparatus 10 according to an exemplary embodiment.

The image code recognition apparatus 10 may be implemented as a computer, portable terminal, television, wearable device, etc. that connects to a server at a remote location through a network N, or connects to other terminals and servers. Here, the computer includes, for example, a notebook equipped with a web browser, a desktop, a laptop, and the like, and the portable terminal is, for example, a wireless communication device that ensures portability and mobility. , PCS (Personal Communication System), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), GSM (Global System for Mobile communications), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-Code Division Multiple Access (W-CDMA), Wireless Broadband Internet (Wibro), Smart Phone, Mobile Worldwide Interoperability for Microwave Access (WiMAX), etc. It may include a (Handheld) based wireless communication device. Further, the television may include Internet Protocol Television (IPTV), Internet Television (Internet Television), terrestrial TV, and cable TV. Furthermore, a wearable device is a type of information processing device that can be directly worn on the human body, such as watches, glasses, accessories, clothes, shoes, etc., and connects to a remote server or other terminal through a network directly or through another information processing device. Can be connected with.

Referring to FIG. 1, the image code recognition apparatus 10 according to an embodiment may include an input/output unit 110, a control unit 120, a communication unit 130, and a memory 140.

The input/output unit 110 may include an input unit for receiving an input from a user, and an output unit for displaying information such as a result of performing a task or a state of the image code recognition device 10. For example, the input/output unit 110 may include an operation panel for receiving a user input and a display panel for displaying a screen.

Specifically, the input unit may include devices capable of receiving various types of user input, such as a keyboard, a physical button, a touch screen, a camera, or a microphone. In addition, the output unit may include a display panel or a speaker. However, the present invention is not limited thereto, and the input/output unit 110 may include a configuration supporting various input/output.

The input/output unit 110 may acquire an image including an image code through a camera.

The controller 120 controls the overall operation of the image code recognition apparatus 10 and may include a processor such as a CPU. The controller 120 may control other components included in the image code recognition apparatus 10 to perform an operation corresponding to a user input received through the input/output unit 110.

For example, the control unit 120 may execute a program stored in the memory 140, read a file stored in the memory 140, or store a new file in the memory 140.

In this way, the controller 120 may extract a pixel area corresponding to the image code from the image captured by the camera.

To this end, the controller 120 may perform machine learning based on learning data including an image code, and may extract a pixel region corresponding to the image code in the image through segmentation.

For example, the controller 120 may learn a pattern for extracting a pixel area corresponding to an image code by a deep learning method, and may search for a pixel area corresponding to an image code in an image based on the learned pattern. have.

In this case, the controller 120 may identify a boundary of a pixel region corresponding to a previously learned pattern.

For example, the controller 120 may identify a pixel area corresponding to the pattern and a pixel adjacent to the pattern as a boundary while searching for a region corresponding to a previously learned pattern in the image.

Further, the controller 120 may identify vertices that are both ends of each corner, which is a boundary of the extracted pixel area, and may transform the pixel area based on a relative position between the identified vertices.

For example, the controller 120 may identify vertices that are both ends of an edge that is a boundary of a rhombic pixel area, and obtain a relative position of the remaining vertices based on any one of the four vertices.

At this time, for example, if the number of vertices of the pixel area identified by the controller 120 exceeds 4, the controller 120 may select 4 vertices that are the longest among the distances between vertices, and any of the 4 selected vertices The relative positions of the remaining vertices can be obtained based on one.

In addition, the controller 120 may generate a transformation matrix for converting the positions of the remaining vertices into a rectangular vertex position formed based on the vertex of the longest corner among the corners of the pixel area.

To this end, the controller 120 may calculate a correlation between a vertex of a rectangle formed on the basis of a corner having the longest length in the pixel area and the vertex of the pixel area.

For example, when the coordinates of each vertex of a corner constituting the pixel area are (3,2), (4,5), (5,5), (8,2), the controller 120 has the longest The remaining vertices (3,5) and (8,5) that create a rectangle based on the vertices of the corners (3,2) and (8, 2) are the coordinates of the pixel area (4,5), ( The transformation matrix can be generated by calculating the difference value for transforming 5,5).

Or, for example, the control unit 120 may generate a rectangle having the longest length of the corners constituting the pixel area, and when the center point of the generated rectangle matches the center point of the pixel area, the vertex of the rectangle and the pixel By calculating the correlation between the vertices of the area, a transformation matrix that moves the vertices of the pixel area to the rectangular vertices can be generated.

And, for example, the control unit 120 creates a rectangle based on the length of the longest corner, calculates the ratio between corners using the length of the corner constituting the pixel area, and calculates the length of the remaining corners according to the calculated ratio. It is possible to create a transformation matrix that moves the vertices of the pixel area to the vertices of the transformed rectangle.

In addition, the controller 120 may recognize a warping image code warped according to at least one of the length and ratio of the pixel region through the transformation matrix.

For example, the control unit 120 can obtain information from the warping image code by recognizing a barcode, which is a warping image code warped in a direct fire, by converting the pixel area into a rectangle having the length of the longest edge among the corners of the pixel area. have.

In this case, the controller 120 may calculate the recognition reliability of the warping image code, and may correct information on the acquired warping image code based on the calculated reliability.

For example, the controller 120 recognizes a barcode that is a warping image code, but can calculate the recognition reliability of the barcode corresponding to each digit (a probability calculated when recognizing a bar of each digit using deep learning). have. In addition, the control unit 120 may identify an error in barcode recognition by using a checksum capable of identifying whether a barcode recognition error has occurred.

And, for example, if the reliability of the checksum is lower than the reliability of the other digits by comparing the reliability of the checksum with the reliability of the other digits, the control unit 120 may correct the checksum to obtain information from the barcode. On the contrary, if the reliability of the checksum is higher than that of the other digits, the control unit 120 may obtain the barcode information by correcting the value of the remaining digits so as not to cause a recognition error using the checksum.

The communication unit 130 may perform wired or wireless communication with other devices or networks. To this end, the communication unit 130 may include a communication module supporting at least one of various wired and wireless communication methods. For example, the communication module may be implemented in the form of a chipset.

The wireless communication supported by the communication unit 130 may be, for example, Wireless Fidelity (Wi-Fi), Wi-Fi Direct, Bluetooth, Ultra Wide Band (UWB), or Near Field Communication (NFC). In addition, wired communication supported by the communication unit 130 may be, for example, USB or HDMI (High Definition Multimedia Interface).

Various types of data such as files, applications, and programs may be installed and stored in the memory 140. The controller 120 may access and use data stored in the memory 140, or may store new data in the memory 140. Also, the controller 120 may execute a program installed in the memory 140.

According to an embodiment, upon receiving an input requesting recognition of an image code from a user through the input/output unit 110, the control unit 120 executes a program that recognizes the image code stored in the memory 140 Carry out recognition.

2 is a flow chart for explaining an image code recognition method according to an embodiment.

The image code recognition method according to the embodiment shown in FIG. 2 includes steps processed in a time series by the image code recognition apparatus 10 shown in FIG. 1. Accordingly, even if the contents are omitted below, the contents described above with respect to the image code recognition apparatus 10 shown in FIG. 1 can also be applied to the image code recognition method according to the embodiment shown in FIG. 2.

3 to 5 will be described later with reference to FIG. 2. 3 to 5 are exemplary diagrams for explaining an image code recognition method according to an embodiment.

First, the image code recognition device 10 may extract a pixel area corresponding to the image code from the image captured by the camera (S2001).

To this end, the image code recognition apparatus 10 may perform machine learning to identify the image code, and may extract a pixel region corresponding to the image code in the image based on the learning result.

For example, the image code recognition apparatus 10 may learn a pattern capable of identifying an image code, and may search whether a pixel region corresponding to the learned pattern exists in the image. In addition, the image code recognition apparatus 10 may extract a pixel area in which a pattern corresponding to the image code is located.

3 is an exemplary diagram illustrating a process of identifying an image code from an image captured by a camera. Referring to FIG. 3, the image code recognition device 10 can identify the image code 303 attached to the box 302 from the image 301 acquired through the camera based on the previously learned image code pattern. In addition, the pixel area 304 corresponding to the identified image code 303 may be extracted.

In addition, the image code recognition apparatus 10 may identify the vertex of the pixel area (S2002).

For example, the image code recognition apparatus 10 may set the pixel positions at both ends of the edge constituting the boundary line of the extracted pixel area as a vertex.

4 is an exemplary view showing the identification of vertices of a pixel area corresponding to an image code. Referring to this, the image code recognition apparatus 10 includes edges 402, 403, and 404 on the boundary of the pixel area 401 corresponding to the image code. , 405) can be set as vertices 406, 407, 408, 409.

In addition, the image code recognition apparatus 10 may convert the pixel area based on the relative positions between vertices on the boundary of the cell area (S2003).

To this end, the image code recognition apparatus 10 may generate a transformation matrix for transforming the positions of the vertices to form a rectangle based on the correlation between vertices of the corners of the pixel area.

For example, the image code recognition device 10 may generate an arbitrary rectangle based on the longest edge among the corners of the pixel area corresponding to the identified image code, and the vertex of the arbitrary rectangle and the vertex of the pixel area A transformation matrix can be generated by analyzing the correlation between them.

Referring to FIG. 4, the image code recognition apparatus 10 may generate an arbitrary rectangle 410 based on the longest edge 404 among the corners of the pixel area, and the vertex 411 of the generated rectangle 410 A transformation matrix may be generated by calculating a correlation between the positions of each of the positions 412, 413, and 414 and the positions of the vertices 406, 407, 408, and 409 of the pixel area.

In addition, the image code recognition apparatus 10 may transform a pixel area using the generated transformation matrix.

5 is an exemplary diagram of warping an image code by transforming a pixel area. Referring to FIG. 5, the image code recognition apparatus 10 determines the positions of the vertices 502, 503, 504, and 505 of the pixel region 501 through the transformation matrix. By converting to 510), the image code 511 can be warped into a rectangular shape.

Thereafter, the image code recognition apparatus 10 may extract information by recognizing the image code in the pixel area converted in step S2003 (S2004).

For example, the image code recognition device 10 recognizes a barcode, which is a warping image code warped into a rectangle having a high recognition rate through steps S2002 to S2003, from the image acquired in step S2001, and then retrieves numeric information corresponding to the barcode. You can extract it right away.

At this time, the image code recognition apparatus 10 may calculate the reliability of the recognized warping image code, and correct the recognition error of the warping image code based on the calculated reliability.

For example, the image code recognition apparatus 10 may calculate the reliability of each digit obtained by recognizing the modified barcode. In addition, the image code recognition apparatus 10 may compare the reliability of the value of the checksum digit for confirming whether or not there is an error in barcode recognition with the reliability of the remaining digits.

In this case, if the reliability of the checksum digit is lower than the reliability of the remaining digits, the image code recognition apparatus 10 determines that there is an error in recognizing the checksum and may change the checksum value based on the remaining digits.

On the other hand, if the reliability of the checksum digit is higher than the reliability of the remaining digits, the image code recognition apparatus 10 may determine whether or not a barcode recognition error is based on the checksum, and may correct by changing the number of digits with low reliability.

In this way, by extracting the pixel area corresponding to the image code from the image, transforming the pixel area into a shape with high recognition rate, and extracting information from the warping image code rather than passing the warping image code in the transformed pixel area to a separate class. By providing it to other applications or devices, it is possible to accurately and quickly recognize image codes.

The term'~ unit' used in the above embodiments refers to software or hardware components such as field programmable gate array (FPGA) or ASIC, and the'~ unit' performs certain roles. However,'~ part' is not limited to software or hardware. The'~ unit' may be configured to be in an addressable storage medium or may be configured to reproduce one or more processors. Thus, as an example,'~ unit' refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, and procedures. , Subroutines, segments of program patent code, drivers, firmware, microcode, circuitry, data, database, data structures, tables, arrays, and variables.

The components and functions provided in the'~ units' may be combined into a smaller number of elements and'~ units' or separated from the additional elements and'~ units'. In addition, components and'~ units' may be implemented to play one or more CPUs in a device or a security multimedia card.

The image code recognition method according to the embodiment described with reference to FIG. 2 may be implemented in the form of a computer-readable medium that stores instructions and data executable by a computer. In this case, the instructions and data may be stored in the form of a program code, and when executed by a processor, a predetermined program module may be generated to perform a predetermined operation. Further, the computer-readable medium may be any available medium that can be accessed by a computer, and includes both volatile and nonvolatile media, and removable and non-removable media. Further, the computer-readable medium may be a computer recording medium, which is volatile and non-volatile implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. It may include both volatile, removable and non-removable media. For example, computer recording media may include magnetic storage media such as HDDs and SSDs, optical recording media such as CDs, DVDs, and Blu-ray discs, or accessible through a network. It may be a memory included in the server.

In addition, the image code recognition method according to the embodiment described with reference to FIG. 2 may be implemented as a computer program (or computer program product) including instructions executable by a computer. The computer program includes programmable machine instructions processed by a processor, and may be implemented in a high-level programming language, an object-oriented programming language, an assembly language, or a machine language. . Further, the computer program may be recorded on a tangible computer-readable recording medium (eg, memory, hard disk, magnetic/optical medium, solid-state drive (SSD), etc.).

Accordingly, the image code recognition method according to the embodiment described with reference to FIG. 2 may be implemented by executing the computer program as described above by the computing device. The computing device may include at least some of a processor, a memory, a storage device, a high speed interface connected to the memory and a high speed expansion port, and a low speed interface connected to the low speed bus and the storage device. Each of these components is connected to each other using a variety of buses, and can be mounted on a common motherboard or in other suitable manner.

Here, the processor can process commands within the computing device. Such commands include, for example, to display graphic information for providing a GUI (Graphic User Interface) on an external input or output device, such as a display connected to a high-speed interface. Examples are instructions stored in memory or storage devices. As another embodiment, multiple processors and/or multiple buses may be utilized with multiple memories and memory types as appropriate. In addition, the processor may be implemented as a chipset formed by chips including a plurality of independent analog and/or digital processors.

The memory also stores information within the computing device. As an example, the memory may be composed of volatile memory units or a set of them. As another example, the memory may be composed of a nonvolatile memory unit or a set of them. Also, the memory may be another type of computer-readable medium, such as a magnetic or optical disk.

In addition, the storage device may provide a large-capacity storage space to the computing device. The storage device may be a computer-readable medium or a configuration including such a medium, for example, devices in a storage area network (SAN) or other configurations, a floppy disk device, a hard disk device, an optical disk device, Or it may be a tape device, a flash memory, or another semiconductor memory device or device array similar thereto.

The above-described embodiments are for illustrative purposes only, and those of ordinary skill in the art to which the above-described embodiments belong can easily transform into other specific forms without changing the technical idea or essential features of the above-described embodiments. You can understand. Therefore, it should be understood that the above-described embodiments are illustrative and non-limiting in all respects. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope to be protected through the present specification is indicated by the claims to be described later rather than the detailed description, and should be interpreted as including all changes or modified forms derived from the meaning and scope of the claims and the concept of equivalents thereof. .

10: image code recognition device
110: input/output unit
120: control unit
130: communication department
140: memory

Claims (14)

In the device for recognizing image codes,
A control unit for extracting a pixel area corresponding to an image code from an image captured by a camera, converting the pixel area based on a relative position between vertices on a boundary of the pixel area, and recognizing an image code in the converted pixel area; And
An image code recognition device comprising an input/output unit for obtaining the image. The method of claim 1,
The control unit,
An image code recognition apparatus for recognizing a warping image code, which is a warped image code within the pixel area according to at least one transformation of a length and a ratio of the pixel area. The method of claim 2,
The control unit,
An image code recognition apparatus for generating a transformation matrix for converting the positions of the remaining vertices to the positions of a rectangular vertex formed based on a vertex of a corner having the longest length among corners of the pixel area. The method of claim 3,
The control unit,
An image code recognition apparatus for calculating a correlation between a vertex forming the rectangle and a vertex of the pixel area. The method of claim 2,
The control unit,
An image code recognition device that acquires information from the recognized warping image code. The method of claim 5,
The control unit,
The image code recognition apparatus for correcting the information based on the recognition reliability of the warping image code. In the method for the image code recognition device to recognize the image code,
Extracting a pixel area corresponding to the image code from the image captured by the camera; And
And converting the pixel region based on relative positions between vertices on the boundary of the pixel region and recognizing an image code in the converted pixel region. The method of claim 7,
Recognizing the image code in the converted pixel area,
Converting at least one of a length and a ratio of the pixel area; And
Recognizing a warping image code, which is an image code warped according to the converted pixel area. The method of claim 8,
The image code recognition method,
The image code recognition method further comprising the step of generating a transformation matrix for converting the positions of the remaining vertices into a position of a rectangular vertex formed based on a vertex of a corner having the longest length among the corners of the pixel area. The method of claim 9,
The step of generating the transformation matrix,
And calculating a correlation between a vertex forming the rectangle and a vertex of the pixel area. The method of claim 8,
The image code recognition method,
The image code recognition method further comprising the step of obtaining information from the recognized warping image code. The method of claim 11,
The step of extracting the information,
And correcting the information based on the recognition reliability of the warping image code. A computer-readable recording medium on which a program for performing the method according to claim 7 is recorded. A computer program executed by an image code recognition device and stored on a medium to perform the method according to claim 7.

Images