KR20210084386A - Method and device for recognizing visually coded patterns - Google Patents

Abstract

The present disclosure relates to a pattern recognition method operating in a user terminal. The pattern recognition method includes the steps of: receiving an image including a first pattern and a second pattern; determining the pattern type of the first pattern, determining the pattern type of the second pattern; comparing the pattern type of the first pattern and the pattern type of the second pattern for priority; and automatically executing processing of the first pattern when it is determined that the pattern type of the first pattern has a higher priority than the pattern type of the second pattern. It is possible to improve the accuracy of pattern recognition and provide a better user experience by easily providing the service desired by a user.

Description

VISUALLY CODED PATTERNS METHOD AND DEVICE FOR RECOGNIZING VISUALLY CODED PATTERNS

The present disclosure relates to a method and an apparatus for recognizing a visually coded pattern, and specifically, to recognize a plurality of visually coded patterns and automatically execute it or display a preview for a plurality of visually coded patterns. It relates to a method and apparatus.

BACKGROUND ART The spread of terminals such as smartphones, tablets, PCs, laptop computers, desktop computers, and the like equipped with a camera is widely expanding. In such an environment, cases of using a camera attached to a terminal are increasing. As a representative example, various services using barcodes photographed using a camera attached to a terminal are provided.

On the other hand, when a plurality of barcodes are included in an image photographed by a camera attached to a terminal, the barcode may not be recognized or a barcode unintended by the user may be recognized. In this case, there is an inconvenience in that the user needs to re-photograph the barcode to be recognized in order to receive the desired service. Accordingly, it is difficult to provide a smooth user experience because the time required to receive a service desired by the user increases.

The present disclosure provides a pattern recognition method, an apparatus, and a computer program stored in a recording medium for solving the above problems.

Receive an image including a plurality of visually coded patterns at the user terminal, and compare the priority of the pattern type by determining the pattern type of the patterns detected from the received image, thereby processing the pattern having a high priority A method, an apparatus, and a computer program stored in a recording medium are provided for automatically executing the

Provided are a method, an apparatus, and a computer program stored in a recording medium for calculating confidence scores of patterns having the same priority of visually coded pattern types, and automatically executing processing of patterns having a high confidence score accordingly.

Method, apparatus for receiving an image comprising a plurality of visually coded patterns at a user terminal, generating a preview for the pattern(s) detected from the received image, and displaying it in a preview area on a display of the user terminal and a computer program stored in the recording medium.

The present disclosure may be implemented in a variety of ways, including a method, an apparatus, or a computer program stored in a computer-readable storage medium.

A pattern recognition method operating in a user terminal according to an embodiment of the present disclosure includes the steps of receiving an image including a first visually coded pattern and a second visually coded pattern, determining a pattern type, determining a pattern type of a second visually coded pattern, comparing a priority of a pattern type of a first visually coded pattern with a pattern type of a second visually coded pattern and when it is determined that the pattern type of the first visually coded pattern has a higher priority than the pattern type of the second visually coded pattern, automatically executing the processing of the first visually coded pattern. do.

According to one embodiment, determining the pattern type of the first visually coded pattern comprises determining whether the first visually coded pattern is a pattern associated with a particular service and determining whether the first visually coded pattern is a particular pattern. and determining the pattern type of the first visually coded pattern as the first pattern type when it is determined that the pattern is associated with the service.

According to one embodiment, determining the pattern type of the first visually coded pattern comprises determining whether the first visually coded pattern is a pattern associated with a service of a particular company and the visually coded first pattern is The method further includes determining the pattern type of the first visually coded pattern as the second pattern type when it is determined that the pattern is associated with the service of the specific company.

According to an embodiment, the step of determining the pattern type of the first visually coded pattern may include: when it is determined that the visually coded first pattern is a pattern not associated with a service of a specific company, the visually coded second pattern if the first pattern is a two-dimensional pattern, determining the pattern type of the first visually coded pattern as a third pattern type; and when the first pattern visually coded is a one-dimensional pattern, the first visually coded pattern The method further includes recognizing the pattern type of ' as a fourth pattern type.

According to an embodiment, the first pattern type, the second pattern type, the third pattern type, and the fourth pattern type have a higher priority in the order.

According to an embodiment, when it is determined that the pattern type of the first visually coded pattern has the same priority as the pattern type of the second visually coded pattern, the visually coded first pattern in the received image calculating a confidence score of the first visually coded pattern based on at least one of the position or size of the pattern; the visually coded second pattern based on at least one of the position or size of the visually coded second pattern within the received image. calculating a confidence score of the second pattern coded as , and when it is determined that the confidence score of the visually coded first pattern is higher than the confidence score of the visually coded second pattern, The method further includes displaying a preview on the display of the user terminal.

According to an embodiment, calculating a confidence score of the visually coded first pattern based on at least one of a position or a size of the visually coded first pattern in the received image comprises: a center of the received image. and calculating a distance between the point and the center point of the first visually coded pattern.

According to an embodiment, calculating a confidence score of the visually coded first pattern based on at least one of a position or a size of the visually coded first pattern in the received image comprises: and calculating an area ratio occupied by an area corresponding to one pattern in the received image.

According to an embodiment, when it is determined that the pattern type of the first visually coded pattern has the same priority as the pattern type of the second visually coded pattern, the visually coded first pattern in the received image calculating a confidence score of the first visually coded pattern based on at least one of the position or size of the pattern; the visually coded second pattern based on at least one of the position or size of the visually coded second pattern within the received image. calculating a confidence score of the second visually coded pattern, and when it is determined that the confidence score of the visually coded first pattern is higher than the confidence score of the visually coded second pattern, The method further includes automatically executing the processing.

According to an embodiment, the method further includes the step of notifying the user of automatically executing the processing of the first visually coded pattern by displaying an indicator on the first area corresponding to the first visually coded pattern.

According to an embodiment, detecting a first region including a first visually coded pattern, detecting a second region including a second visually coded pattern, and the first region and the second region The method further includes highlighting with different colors.

According to an embodiment, when receiving an image including the step of sequentially receiving a plurality of images in the image capturing mode and the visually coded first pattern and the visually coded second pattern, whether to enter the pattern scan mode The method further includes displaying a notification for checking whether or not on the display of the user terminal.

A computer program stored in a computer-readable recording medium is provided for executing the pattern recognition method described above in a computer according to an embodiment of the present disclosure.

A pattern recognition apparatus according to an embodiment of the present disclosure includes an image sensor configured to receive an image including a first visually coded pattern and a second visually coded pattern, a pattern type of the first visually coded pattern, and a visual a pattern type determining unit configured to determine the pattern type of the second pattern coded as , and a pattern processing unit configured to compare the priority of the pattern type of the first visually coded pattern and the pattern type of the second visually coded pattern. and the pattern processing unit, when determining that the pattern type of the visually coded first pattern has a higher priority than the pattern type of the second visually coded pattern, automatically performs the processing of the visually coded first pattern further configured to run.

A pattern recognition method operating in a user terminal according to an embodiment of the present disclosure includes the steps of receiving an image including a visually coded first pattern and a visually coded second pattern; generating a preview, generating a preview for the visually coded second pattern, displaying a preview for the first pattern and a preview for the second pattern in a preview area on the display of the user terminal and receiving a selection for a preview for the first pattern from the user and executing processing on the visually coded first pattern in response to the selection.

According to an embodiment, the displaying of the preview for the first pattern and the preview for the second pattern on the preview area on the display of the user terminal may include: the visually coded first pattern in the received image. calculating a confidence score of the first visually coded pattern based on at least one of a location or size; coding visually based on at least one of a location or size of a second visually coded pattern within the received image. calculating the confidence score of the visually coded second pattern, and when it is determined that the confidence score of the visually coded first pattern is higher than the confidence score of the visually coded second pattern, the first visually coded pattern is evaluated in advance. and displaying the view at the top of the preview area on the display of the user terminal.

According to an embodiment, detecting a first region including a first visually coded pattern, detecting a second region including a second visually coded pattern, and the first region and the second region The method further includes highlighting with different colors.

According to an embodiment, generating a preview for the visually coded first pattern may include: when the visually coded first pattern corresponds to encoding of a URL (Uniform Resource Locator), the visually coded first pattern Transmitting URL information coded by the pattern to an external device, receiving at least one of a thumbnail, title, or descriptive phrase associated with the URL information from an external device, and receiving at least one of the received thumbnail, title, or descriptive phrase The method further includes displaying in a preview area on the display.

A computer program stored in a computer-readable recording medium is provided for executing the pattern recognition method described above in a computer according to an embodiment of the present disclosure.

A pattern recognition apparatus according to an embodiment of the present disclosure includes an input interface configured to receive an input from a user, an image sensor configured to receive an image including a visually coded first pattern and a visually coded second pattern, and visually a pattern preview generator configured to generate a preview for the first coded pattern and a preview for the second visually coded pattern, a preview for the generated visually coded first pattern and the second pattern in advance and a pattern processing unit configured to execute processing on the visually coded first pattern in response to receiving a selection for a preview for the first pattern through the input interface and the display configured to output to the viewing area.

According to embodiments of the present disclosure, by automatically executing pattern processing based on the priority of the pattern type and/or the confidence score of the pattern for a plurality of visually coded patterns, the accuracy of pattern recognition is improved, A better user experience can be provided by easily providing the services desired by the user.

According to embodiments of the present disclosure, previews of a plurality of patterns are displayed in a preview area on the display of the user terminal, so that the user can select a pattern associated with a desired service.

Effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, wherein like reference numerals denote like elements, but are not limited thereto.
1 is a diagram illustrating an example of a pattern recognition method operating in a user terminal according to an embodiment of the present disclosure.
2 is a diagram illustrating a configuration in which a plurality of user terminals are connected to communicate with a server in order to perform pattern recognition according to an embodiment of the present disclosure.
3 is a block diagram showing the configuration of a user terminal according to an embodiment of the present invention.
4 is a diagram illustrating a lookup table defining a priority for each pattern type according to an embodiment of the present invention.
5 is a diagram illustrating an example of automatically executing pattern processing without a user input according to an embodiment of the present invention.
6 is a flowchart illustrating a visually coded pattern recognition method according to an embodiment of the present disclosure.
7 is a flowchart illustrating a method of executing processing on one of patterns having the same priority according to an embodiment of the present disclosure.
8 is a flowchart illustrating a method of executing processing on one of patterns having the same priority according to an embodiment of the present disclosure.
9 is a flowchart illustrating a method of determining a pattern type of a visually coded pattern according to an embodiment of the present disclosure.
10 is a diagram illustrating an example of displaying a notification for checking whether to enter a pattern scan mode when a user terminal receives a plurality of visually coded patterns according to an embodiment of the present disclosure
11 is a diagram illustrating an example of displaying a preview of a pattern selected by a user on a display according to an embodiment of the present disclosure.
12 is a diagram illustrating an example of displaying a preview of each of a plurality of patterns included in an image on a display according to an embodiment of the present disclosure.
13 is a diagram illustrating an example of displaying a preview of a pattern selected by a user on a display according to an embodiment of the present disclosure;
14 is a flowchart illustrating a visually coded pattern recognition method according to an embodiment of the present disclosure.
15 is a flowchart illustrating a visually coded pattern recognition method according to an embodiment of the present disclosure.

Hereinafter, specific contents for carrying out the present disclosure will be described in detail with reference to the accompanying drawings. However, in the following description, if there is a risk of unnecessarily obscuring the subject matter of the present disclosure, detailed descriptions of well-known functions or configurations will be omitted.

In the accompanying drawings, identical or corresponding components may be assigned the same reference numerals. In addition, in the description of the embodiments below, overlapping description of the same or corresponding components may be omitted. However, even if descriptions regarding components are omitted, it is not intended that such components are not included in any embodiment.

Advantages and features of the disclosed embodiments, and methods of achieving them, will become apparent with reference to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the present disclosure to be complete, and those of ordinary skill in the art to which the present disclosure pertains. It is only provided to fully inform the person of the scope of the invention.

Terms used in this specification will be briefly described, and the disclosed embodiments will be described in detail.

The terms used in this specification are selected as currently widely used general terms as possible while considering the functions in the present disclosure, but may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in specific cases, there are also terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present disclosure should be defined based on the meaning of the term and the contents of the present disclosure, rather than the simple name of the term.

References in the singular herein include plural expressions unless the context clearly dictates that the singular is singular. Also, the plural expression includes the singular expression unless the context clearly dictates the plural.

When a part "includes" a certain element throughout the specification, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated.

In addition, the term 'module' or 'unit' used in the specification means a software or hardware component, and 'module' or 'unit' performs certain roles. However, 'module' or 'unit' is not meant to be limited to software or hardware. A 'module' or 'unit' may be configured to reside on an addressable storage medium or may be configured to refresh one or more processors. Thus, as an example, 'module' or 'unit' refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, may include at least one of procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, database, data structures, tables, arrays or variables. Components and 'modules' or 'units' are the functions provided therein that are combined into a smaller number of components and 'modules' or 'units' or additional components and 'modules' or 'units' can be further separated.

According to an embodiment of the present disclosure, a 'module' or a 'unit' may be implemented with a processor and a memory. 'Processor' should be construed broadly to include general purpose processors, central processing units (CPUs), microprocessors, digital signal processors (DSPs), controllers, microcontrollers, state machines, and the like. In some circumstances, a 'processor' may refer to an application specific semiconductor (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), or the like. 'Processor' refers to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of a plurality of microprocessors, a combination of one or more microprocessors in combination with a DSP core, or any other such configurations. You may. Also, 'memory' should be construed broadly to include any electronic component capable of storing electronic information. 'Memory' means random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erase-programmable read-only memory (EPROM); may refer to various types of processor-readable media, such as electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, and the like. A memory is said to be in electronic communication with the processor if the processor is capable of reading information from and/or writing information to the memory. A memory integrated in the processor is in electronic communication with the processor.

In the present disclosure, an 'image' may include an image captured by an image sensor attached to the user terminal, and an image captured by an image sensor configured to communicate with the user terminal. Also, the image may include some images extracted from the captured video image.

In the present disclosure, a 'visually coded pattern' refers to a code in which data or information is displayed in a machine-readable visual form. A pattern reader or an application program of a device with a built-in camera, such as a smartphone, can be used to scan/read the visually coded pattern. The visually coded pattern includes a visually coded one-dimensional pattern and a visually coded two-dimensional pattern. The visually coded one-dimensional pattern is made of parallel lines of different widths and spacings, and may include a Universal Product Code (UPC) or the like. The visually coded two-dimensional pattern consists of geometric patterns such as rectangles, dots, and hexagons, and may include Data Matrix, MaxiCode, and Quick Response (QR) codes. In this specification, a 'visually coded pattern' may also be referred to as a 'pattern'.

1 is a diagram illustrating an example of a pattern recognition method operating in a user terminal 120 according to an embodiment of the present disclosure. As shown in FIG. 1 , the user 110 scans one of the first pattern 132 , the second pattern 134 , and the third pattern 136 included in the object 130 , the user terminal 120 . ) may hold the user terminal 120 so that the image sensor faces the object 130 . For example, the object 130 may be a document on which a plurality of visually coded patterns are printed. As illustrated, the first pattern 132 and the second pattern 134 may be a two-dimensional pattern, and the third pattern 136 may be a one-dimensional pattern.

When the user terminal 120 receives an image including the first pattern 132 , the second pattern 134 , and the third pattern 136 , the user terminal 120 displays the visually coded pattern 132 in the image. , 134, 136) may be detected, and the pattern type of each pattern may be determined. The user terminal 120 may compare the priority of the pattern type of each pattern and automatically execute the processing of the pattern having the highest priority. For example, when the user terminal 120 determines that the priority of the first pattern 132 is higher than the priority of the second pattern 134 and the third pattern 136 , the user terminal 120 is the user It is possible to automatically execute the processing of the first pattern 132 without input.

To this end, the user terminal 120 may store a priority for each predefined pattern type. In an embodiment, the user terminal 120 determines that a pattern related to a specific company's financial service (eg, LINE PAY service) is the first pattern type, and the pattern related to a non-financial service (eg, LINE messenger service) of a specific company is the second pattern. It may be determined as a two-dimensional pattern type, other company service-related two-dimensional patterns may be determined as a third pattern type, and other one-dimensional patterns may be determined as a fourth pattern type. In this case, the user terminal 120 may be predefined as having a higher priority in the order of the first pattern type, the second pattern type, the third pattern type, and the fourth pattern type.

When a plurality of patterns having the same pattern type exist in one image and there is no pattern having a higher priority than the corresponding patterns, the user terminal 120 provides a preview for the corresponding patterns to the user. to select one pattern to be executed (or scanned). Alternatively, the user terminal may calculate the reliability scores of the corresponding patterns and automatically execute the processing of the pattern having the highest reliability score. In this case, the reliability score of each pattern may be calculated based on at least one of a position or a size of the pattern in the image.

In another embodiment, the user terminal 120 does not automatically execute processing on one of the plurality of patterns in the image, and the user 110 selects one of the plurality of patterns 132 , 134 , 136 included in the image. You can choose a pattern. In this case, the user terminal 120 may display the patterns 132 , 134 , and 146 detected in the image on the display as selectable options. Additionally or alternatively, the user terminal 120 may generate a preview for the patterns 132 , 134 , 146 detected in the image and display it on the display. In this case, the user 110 may check a preview of the patterns displayed on the display, select one pattern, and execute a process for the pattern.

2 is a diagram illustrating a configuration in which a plurality of user terminals 210 , 220 , and 230 are connected to communicate with a server 250 in order to perform pattern recognition according to an embodiment of the present disclosure. Although the mobile phone terminal 210, the tablet terminal 220, and the notebook terminal 230 are illustrated as examples of the user terminal in FIG. 2, the present invention is not limited thereto, and the user terminal is capable of wired and/or wireless communication and has an image sensor (eg, For example, it may be any computing device to which an application program capable of scanning a visually coded pattern can be installed and executed, capable of being attached to an image sensor or capable of communicating with an image sensor. For example, the user terminal includes a smart phone, a mobile phone, a navigation system, a computer, a notebook computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a tablet PC, a game console, It may include a wearable device, an Internet of things (IoT) device, a virtual reality (VR) device, an augmented reality (AR) device, and the like.

In addition, in FIG. 2 , three user terminals 210 , 220 , and 230 are illustrated as communicating with the server 250 through the network 240 , but the present invention is not limited thereto, and a different number of user terminals is connected to the network 240 . It may be configured to communicate with the server 250 through A communication method through the network 240 is not limited, and may include a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network, a satellite network, and short-range wireless communication between user terminals. For example, the network 240 may include a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), may include any one or more of the networks, such as the Internet. Further, the network 240 may include any one or more of a network topology including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, etc. not limited

According to an embodiment, the server 250 is one or more server devices and/or databases capable of storing, providing and executing computer executable programs and data, or one or more distributed computing devices and/or distributed databases based on cloud computing services. may include. In one embodiment, the server 250 may be a web server configured to store at least one of a thumbnail, a title, or a descriptive phrase for a Uniform Resource Locator (URL).

In one embodiment, when the pattern corresponds to the encoding of a Uniform Resource Locator (URL), the user terminals 210 , 220 , 230 transmit the corresponding pattern to the server 250 to generate a preview for the corresponding pattern, or URL information coded by the pattern may be transmitted to the server 250 . In this case, the server 250 may provide at least one of a thumbnail, a title, and a description associated with the URL to the user terminals 210 , 220 , and 230 . The user terminals 210 , 220 , and 230 may display at least one of the received thumbnail, title, and description on the display as a preview for the corresponding pattern.

3 is a block diagram showing the configuration of a user terminal 300 according to an embodiment of the present invention. The user terminal 300 may be any device that already has capture and image processing capabilities. As shown, the user terminal 300 may include an image sensor 310 , a display 320 , an input interface 330 , a communication module 340 , a processor 350 , and a storage unit 360 .

The image sensor 310 may be configured to receive one or more images. For example, the image sensor 310 may be a camera module. The image received by the image sensor 310 may be temporarily or permanently stored in the form of an electronic file in the storage unit 360 of the user terminal 300 . In one embodiment, the image received by the image sensor 310 may be provided to the processor 350 to determine whether the image includes a visually coded pattern. In another embodiment, image sensor 310 may be configured to store an image to storage 360 or provide to processor 350 in response to receiving an input command from a user via input interface 330 . .

The display 320 may be configured to output information and/or data generated by the user terminal 300 . According to an embodiment, the image received by the image sensor 310 may be displayed on the display 320 . In addition to the display 320 , devices such as a speaker, a haptic feedback device, etc. may be provided as an output device together with the display 320 .

The input interface 330 may be configured to allow a user to input information and/or data into the user terminal 300 . For example, the input interface 330 may include a device such as a touch screen, a button, a keypad, a touch pad, a keyboard, a microphone, and a mouse. Although the display 320 is illustrated as a separate component from the input interface 330 in FIG. 3 , the present invention is not limited thereto, and the display 320 and the input interface 330 may be provided as an integrated input/output device.

As shown in FIG. 3 , the user terminal 300 may be configured to communicate information and/or data with another device through a network 370 using the communication module 340 . For example, the request generated by the processor 350 of the user terminal 300 according to the program code stored in the storage unit 360 is controlled by the communication module 340 through the network 370 through another device (eg, a server). ) can be transferred. Conversely, a control signal, command, content, or file provided by another device may be received by the user terminal 300 through the communication module 340 . For example, a control signal, command, content, or file of another device received through the communication module 340 may be transmitted to the processor 350 or the storage unit 360 .

The processor 350 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. The command may be provided to the processor 350 by the storage 360 or the communication module 340 . For example, the processor 350 may be configured to execute a received instruction according to a program code stored in the storage unit 360 . In an embodiment, the processor 350 may include a pattern region detection unit 352 , a pattern type determination unit 354 , a pattern preview generation unit 356 , and a pattern processing unit 358 .

The storage unit 360 may include any computer-readable recording medium. According to an embodiment, the storage unit 360 may include a storage device such as random access memory (RAM), read only memory (ROM), disk drive, solid state drive (SSD), flash memory, and the like. have. At least one program code such as an operating system and an application program may be stored in the storage unit 360 .

In an embodiment, the image sensor 310 may receive an image including a plurality of patterns. In this case, the image received by the image sensor 310 may be output on the display 320 and provided to the processor 350 . The processor 350 provides the received image to the pattern area detection unit 352, and the pattern area detection unit 352 detects an area in the image including the visually coded pattern, so that the visually coded pattern in the image is detected. It can be determined whether or not

When the pattern region detection unit 352 determines that the visually coded pattern is not included in the received image, the processor 350 may cause the image sensor 310 to receive a subsequent image. When the pattern area detection unit 352 determines that one visually coded pattern is included in the received image, the pattern area detection unit 352 provides the pattern to the pattern processing unit 358, so that the pattern processing unit 358 receives the corresponding pattern. Pattern processing can be executed. On the other hand, when the pattern area detection unit 352 determines that a plurality of patterns are included in the received image, the pattern area detection unit 352 converts the detected plurality of patterns (or an image of an area in which the patterns are detected) to the pattern type determination unit. (354) may be provided.

The pattern type determination unit 354 may determine the pattern type of each of the plurality of received patterns. For example, a pattern associated with a specific company's financial service (eg, LINE Pay service) is determined as the first pattern type, and a pattern associated with a specific company's non-financial service (eg, LINE messenger service) is determined as the second pattern type In addition, the two-dimensional pattern associated with the service of the company may be determined as the third pattern type, and the one-dimensional pattern may be determined as the fourth pattern type. In one embodiment, the pattern type determination unit 354 separates the pattern or information coded by the pattern to determine whether each pattern is a pattern associated with a specific company's financial service (eg, LINE Pay service). It can be transmitted to a module (eg, LINE PAY application). In another embodiment, the pattern type determining unit 354 may include a module (eg, LINE Pay application) for determining whether each pattern is a pattern related to a specific company's financial service (eg, LINE Pay service). .

The pattern processing unit 358 may receive the pattern type of each of the plurality of patterns from the pattern type determination unit 354 , and compare the priorities of the patterns according to the priority of each pattern type stored in advance in the storage unit 360 . For example, the storage unit 360 may be predefined as having a higher priority in the order of the first pattern type, the second pattern type, the third pattern type, and the fourth pattern type. When it is determined that there is only one pattern having the highest priority, the pattern processing unit 358 may automatically execute the processing of the corresponding pattern without user input.

On the other hand, when it is determined that there are two or more patterns having the highest priority, the pattern processing unit 358 may calculate a reliability score of each pattern. In an embodiment, the confidence score of the visually coded pattern may be calculated based on at least one of a position or a size of the corresponding pattern. For example, the reliability score of the visually coded pattern may be calculated based on the distance between the center point of the image and the center point of the pattern, or may be calculated based on the ratio of the area occupied by the pattern in the image. Thereafter, the pattern processing unit 358 may automatically execute the processing of the pattern having the highest reliability score without user input.

In another embodiment, when it is determined that there are two or more patterns having the highest priority, the pattern processing unit 358 may provide the patterns having the highest priority to the pattern preview generation unit 356 . In this case, the pattern preview generator 356 may generate a preview for each pattern and display it on the display 320 so that the user can select one pattern. In this case, the pattern preview generation unit 356 may calculate a confidence score of each pattern and display a preview of the pattern having a higher confidence score at the top of the preview area on the display 320 . When the user selects one of the previews displayed on the display 320 through the input interface 330 , the pattern processing unit 358 may process a pattern associated with the preview selected by the user. Alternatively, the preview generator 356 may display only a preview of the pattern having the highest confidence score in the preview area on the display 320 .

In another embodiment, when the pattern area detection unit 352 detects a plurality of patterns in the received image, the detected plurality of patterns is sent to the pattern preview generation unit 356 instead of the pattern type determination unit 354 . may be configured to provide. In this case, the pattern preview generator 356 may generate a preview for each pattern and display it on the display 320 . For example, a preview of each pattern may be displayed in a preview area on the display 320 . In this case, the pattern preview generator 356 may determine the display order of the previews for the pattern based on the priority of the pattern and/or the reliability score of the pattern. When the user selects one of a plurality of previews displayed on the display 320 , the pattern processing unit 358 may process a pattern associated with the preview selected by the user.

Illustratively, when the visually coded pattern corresponds to encoding of a Uniform Resource Locator (URL), the pattern preview generator 356 transmits the pattern or URL information coded by the pattern to the communication module 340 . It can be transmitted to an external device (eg, a web server) through In this case, the user terminal 300 may receive at least one of a thumbnail, a title, and an explanatory text associated with the corresponding URL from the external device. The pattern preview generator 356 may generate a preview of the corresponding pattern based on at least one of a thumbnail, a title, and a description received from an external device.

As another example, when the visually coded pattern corresponds to encoding of text information, the pattern preview generator 356 may generate a visually coded pattern preview based on the text information. When the visually coded pattern corresponds to the encoding of information (eg, URL information) related to the financial service of a specific company or the non-financial service of a specific company, the pattern preview generation unit 356 is a specific company coded by the pattern. You can create as a preview a user interface that can run financial services or non-financial services of

The user terminal 300 may include more components than those shown in FIG. 3 . According to an embodiment, the user terminal 300 may further include other components such as a transceiver, a global positioning system (GPS) module, various sensors, and a database. For example, when the user terminal 300 is a smart phone, it may include components generally included in the smart phone, for example, using an acceleration sensor or a gyro sensor, various physical buttons, and a touch panel. Various components such as a button, an input/output port, and a vibrator for vibration may be implemented to be further included in the user terminal 300 . In this case, the user terminal 300 may be configured to provide feedback (eg, haptic feedback) to the user when a pattern is detected in the received image.

4 is a diagram illustrating a lookup table 400 defining a priority for each pattern type according to an embodiment of the present invention. In an embodiment, the visually coded pattern includes a pattern related to a specific company's financial service (eg, LINE Pay service), a pattern related to a non-financial service (eg, LINE messenger service) of a specific company, and a two-dimensional pattern related to other companies' services. and one-dimensional patterns. According to the lookup table 400, a financial service-related pattern of a specific company, a non-financial service-related pattern of a specific company, a 2D pattern related to a service of another company, and a 1D pattern have high priority in the order.

For example, a pattern related to a specific company's financial services may include a pattern for payment (eg, a QR code for a store placed in a store, a QR code that a user uses to pay for a store, etc.), a split bill ) pattern (eg, a QR code for users to divide and pay expenses), a pattern for cost remittance, and the like. Patterns related to a specific company's non-financial services include a pattern for adding friends (eg, a QR code to add a specific user as a friend), a chat room invitation pattern, an open chat invitation pattern, a pattern for logging in (eg QR code login), etc. may include. Other company service-related two-dimensional patterns may include two-dimensional patterns for encoding URLs, two-dimensional patterns for encoding text information, and the like.

5 is a diagram illustrating an example of automatically executing pattern processing without a user input according to an embodiment of the present invention. 5 illustrates a first operation step 510 and a second operation step 520 of the user terminal. The first operation step 510 represents that the first pattern 512 is automatically selected without user input from among the plurality of patterns 512 and 514 included in the image received by the image sensor of the user terminal. The first pattern 512 may be a 2D pattern related to a financial service of a specific company, and the second pattern 514 may be a 2D pattern related to a non-financial service of a specific company. In this case, it may be determined that the first pattern 512 has a higher priority than the second pattern 514 .

In the first operation step 510 , the user terminal may highlight the area including the first pattern 512 and the area including the second pattern 514 with different colors on the display. 5 illustrates that the detected first pattern 512 and the second pattern 514 are highlighted in different colors, but the present invention is not limited thereto. The pattern 512 and the second pattern 514 may be displayed in various ways such as shading. Also, the user terminal may display the indicator 516 on the first pattern 512 to inform the user that the processing of the first pattern 512 having a higher priority is automatically executed.

The second operation step 520 represents an example of a screen in which the processing of the first pattern 512 is automatically executed without a user input. Since the first pattern 512 is a financial service related pattern of a specific company, a financial service providing screen of a specific company related to the corresponding pattern may be displayed on the display of the user terminal. In this case, the user may proceed with payment by selecting the payment button 522 displayed on the display.

6 is a flowchart illustrating a method 600 for recognizing a visually coded pattern according to an embodiment of the present disclosure. In operation 610 , the method 600 may be initiated by the user terminal receiving an image including the first pattern and the second pattern. For example, an image including a plurality of patterns may be received while the user terminal is operating in a pattern scan mode for recognizing a visually coded pattern. Thereafter, the user terminal may detect a region including the visually coded patterns (the first pattern and the second pattern) in the image, and determine the pattern type of each pattern ( 620 ). In an embodiment, the user terminal determines that a pattern related to a financial service (eg, LINE PAY service) of a specific company is a first pattern type, and a pattern related to a non-financial service (eg, LINE messenger service) of a specific company is a second pattern type , and other company service-related two-dimensional patterns may be determined as the third pattern type, and the one-dimensional pattern may be determined as the fourth pattern type.

In step 630, the user terminal may compare the priority of the pattern type of the first pattern and the pattern type of the second pattern. For example, the user terminal may be predefined as having a high priority in the order of the first pattern type, the second pattern type, the third pattern type, and the fourth pattern type. Based on the determination result in step 630, the user terminal may determine whether the priority of the pattern type of the first pattern and the pattern type of the second pattern are the same (operation 640).

If it is determined in step 640 that the priority of the pattern type of the first pattern and the pattern type of the second pattern are not the same (ie, NO in 640), the user terminal has a higher priority without user input. Processing of the determined pattern may be automatically performed ( 650 ). With such a configuration, the pattern having the highest priority is automatically executed without the user pressing a camera shooting button or selecting one of a plurality of patterns, so that a better user experience can be provided. On the other hand, if it is determined in step 640 that the priority of the pattern type of the first pattern and the pattern type of the second pattern are the same (ie, YES in 640), the method 600 is the method 700 of FIG. 7 or The method 800 of FIG. 8 may proceed.

7 is a flowchart illustrating a method 700 of executing processing on one of patterns having the same priority according to an embodiment of the present disclosure. When it is determined that the priority of the pattern type of the first pattern and the pattern type of the second pattern is the same in step 640 of FIG. 6 (ie, YES in 640), the user terminal calculates the reliability score of the first pattern. may (710). For example, the reliability score of the first pattern may be calculated based on a result of calculating the distance between the center point of the first pattern and the center point of the received image. In this case, the reliability score of the first pattern may be inversely proportional to a distance between the center point of the first pattern and the center point of the received image. That is, the closer the central point of the visually coded pattern to the central point of the received image, the higher the reliability score of the pattern may be calculated. This is because there is a high possibility that the user holds the user terminal so that the pattern of interest is placed close to the center point of the image.

Alternatively, a reliability score of the first pattern may be calculated based on a ratio of an area occupied by an area corresponding to the first pattern in the received image. In this case, the higher the ratio of the area occupied by the region corresponding to the pattern in the received image, the higher the reliability score of the pattern may be calculated. This is because there is a high possibility that the user holds the user terminal so that the pattern of interest occupies a high area ratio in the image.

In step 720, the user terminal may calculate a confidence score of the second pattern. The confidence score of the second pattern may be calculated in the same way as the confidence score of the first pattern is calculated. That is, the reliability score of the second pattern is calculated based on the result of calculating the distance between the center point of the second pattern and the center point of the received image or the ratio of the area corresponding to the second pattern occupied in the received image. can In FIG. 7 , the step 710 of calculating the reliability score of the first pattern is illustrated as preceding the step 720 of calculating the reliability score of the second pattern, but the present invention is not limited thereto, and the reliability score of any pattern is calculated. It can be calculated first, or the reliability score can be calculated simultaneously.

Next, the user terminal may compare the calculated confidence score of the first pattern with the confidence score of the second pattern, and display a preview of the pattern having a higher confidence score on the display ( 730 ). For example, when the first pattern and the second pattern have the same priority and the first pattern is located closer to the center of the image than the second pattern, the user terminal determines that the confidence score of the first pattern is the confidence score of the second pattern. Determining that it is higher, a preview for the first pattern may be displayed on the display.

In one embodiment, in the case of a pattern associated with a specific company's financial service or a specific company's non-financial service, the user terminal previews a user interface capable of executing the corresponding service (a specific company's financial service or a specific company's non-financial service) as a preview. can be displayed on the display. When the pattern is a URL type pattern related to other company services, the user terminal receives at least one of a thumbnail, a title, or an explanatory text for a URL coded by the corresponding pattern from an external server and displays it on the display as a preview can do. In the case of a pattern in which text information is coded, the user terminal may generate a preview based on the text information and display it on the display.

In step 740, the user terminal may receive a user's selection of the pattern displayed as a preview. For example, the user's selection may be an input for executing processing of a pattern associated with a preview displayed on the display. In this case, the user terminal may process the pattern selected by the user ( 750 ). Although it is illustrated in FIG. 7 that a preview of a pattern having a higher reliability score is displayed on the display, the present invention is not limited thereto, and all previews of the pattern having the same priority may be displayed on the display. In this case, a preview for a pattern having a higher confidence score may be displayed at the top of the preview area on the display.

8 is a flowchart illustrating a method 800 of executing processing on one of patterns having the same priority according to an embodiment of the present disclosure. When it is determined that the priority of the pattern type of the first pattern and the pattern type of the second pattern is the same in step 640 of FIG. 6 (ie, YES in 640), the user terminal calculates the reliability score of the first pattern. may (810). For example, the reliability score of the first pattern may be calculated based on a result of calculating the distance between the center point of the first pattern and the center point of the received image. In this case, the reliability score of the first pattern may be inversely proportional to a distance between the center point of the first pattern and the center point of the received image. That is, the closer the central point of the visually coded pattern to the central point of the received image, the higher the reliability score of the pattern may be calculated. This is because there is a high possibility that the user holds the user terminal so that the pattern of interest is placed close to the center point of the image.

Alternatively, a reliability score of the first pattern may be calculated based on a ratio of an area occupied by an area corresponding to the first pattern in the received image. In this case, the higher the ratio of the area occupied by the region corresponding to the pattern in the received image, the higher the reliability score of the pattern may be calculated. This is because there is a high possibility that the user holds the user terminal so that the pattern of interest occupies a high area ratio in the image.

In step 820, the user terminal may calculate a confidence score of the second pattern. The confidence score of the second pattern may be calculated in the same way as the confidence score of the first pattern is calculated. That is, the reliability score of the second pattern is calculated based on the result of calculating the distance between the center point of the second pattern and the center point of the received image or the ratio of the area corresponding to the second pattern occupied in the received image. can In FIG. 8 , the step 810 of calculating the reliability score of the first pattern precedes the step 820 of calculating the reliability score of the second pattern, but the present invention is not limited thereto. It can be calculated first, or the reliability score can be calculated simultaneously.

Next, the user terminal may compare the calculated confidence score of the first pattern with the confidence score of the second pattern, and automatically execute the processing of the pattern having a higher confidence score without user input ( 830 ). For example, when the first pattern and the second pattern have the same priority and the first pattern is located closer to the center of the image than the second pattern, the user terminal determines that the confidence score of the first pattern is the confidence score of the second pattern. It is determined that it is higher, and the processing of the first pattern can be automatically executed without user input. With such a configuration, the pattern having the highest priority and the highest reliability score is automatically executed without the user pressing the camera shooting button or selecting one of a plurality of patterns, thereby providing a better user experience can do.

9 is a flowchart illustrating a method 900 of determining a pattern type of a visually coded pattern according to an embodiment of the present disclosure. For example, when a plurality of patterns are detected in an image while operating in a pattern scan mode, the user terminal may initiate pattern type determination for each pattern ( 910 ). The user terminal may first determine whether one pattern is a pattern related to a financial service (eg, LINE Pay service) of a specific company ( 920 ).

When it is determined that the corresponding pattern is a pattern related to a financial service of a specific company, the user terminal may determine that the corresponding pattern is the first pattern type ( 930 ). On the other hand, if it is determined that the pattern is not related to the financial service of the specific company, the user terminal proceeds to step 940 to determine whether the pattern is related to the non-financial service (eg, LINE messenger service) of the specific company. It can be determined whether Here, when it is determined that the corresponding pattern is a pattern related to a non-financial service of a specific company, the user terminal may determine that the corresponding pattern is the second pattern type ( 950 ). On the other hand, when it is determined that the corresponding pattern is a pattern not related to the non-financial service of a specific company, the user terminal may proceed to step 960 to determine whether the corresponding pattern is a two-dimensional pattern.

When it is determined that the corresponding pattern is a two-dimensional pattern, the user terminal may determine that the corresponding pattern is a third pattern type ( 970 ). On the other hand, when it is determined that the corresponding pattern is not a two-dimensional pattern (eg, a one-dimensional pattern), the user terminal may determine that the corresponding pattern is a fourth pattern type ( 980 ).

10 is a diagram illustrating an example of displaying a notification for confirming whether to enter a pattern scan mode when a user terminal receives a plurality of visually coded patterns according to an embodiment of the present disclosure; The user terminal may be configured to capture a picture by the user selecting the image capturing button 1016 through a touch input or the like while sequentially receiving a plurality of images through the image sensor in the image capturing mode. In addition, in the pattern scan mode, when a pattern is detected in a received image while sequentially receiving a plurality of images through the image sensor, the user terminal automatically executes the detected pattern without user input or previews the detected pattern. It may be configured to display on a display. Additionally, when a pattern is detected in an image received in the image capturing mode, the user terminal may confirm to the user whether to enter the pattern scan mode.

A first operation step 1010 represents that the user terminal receives an image including a plurality of patterns 1012 and 1014 through an image sensor in an image capturing mode. In this case, the user terminal may detect a plurality of patterns 1012 and 1014 in the received image. In the second operation step 1020, the user terminal detects a plurality of patterns 1012 and 1014 in the image received in the image capturing mode, and sends a notification 1022 for checking whether to enter the pattern scan mode of the user terminal. Indicates what is displayed on the display. When the user selects the notification 1022 through a touch input or the like, the user terminal may enter a pattern scan mode to process a plurality of patterns 1012 and 1014 .

11 is a diagram illustrating an example of displaying a preview of a pattern selected by a user on a display according to an embodiment of the present disclosure. 11 illustrates a first operation step 1110 , a second operation step 1120 , and a third operation step 1130 of the user terminal. In the first operation step 1110, when the user terminal receives an image including the first pattern 1112 and the second pattern 1114 visually coded through the image sensor in a state in which the user terminal is operating in the pattern scan mode, the second It indicates that the first pattern 1112 and the second pattern 1114 are detected, and the detected patterns 1112 and 1114 are displayed on the display as selectable options. In this case, the area including the first pattern 1112 and the area including the second pattern 1114 may be highlighted in different colors on the display.

In the second operation step 1120, the user terminal receives the user's selection for the first pattern 1112 from the user, and displays a preview 1124 for the first pattern 1112 in the preview area on the display. indicates that When the user selects the first pattern 1112 through a touch input, etc., the user terminal displays an indicator 1122 indicating that the first pattern 1112 has been selected on the first pattern 1112, and for this View 1124 may display in the preview area of the display. As illustrated, the first pattern 1112 may be a pattern for adding “James” as a friend in a non-financial service (ie, LINE messenger service) of a specific company.

A preview 1124 of the first pattern may include a LINE messenger logo, information about a user added as a friend (eg, “James”), an add friend button 1126 , and a share button 1128 . When the user selects the add friend button 1126 through a touch input or the like, the user terminal may perform a friend add function in the LINE messenger. When the user selects the share button 1128 through a touch input or the like, the user terminal allows the user to share the first pattern 1112 or information coded by the first pattern 1112 with other users. In the second operation step 1120 , a preview 1124 of the first pattern includes a LINE messenger logo, information about a user added as a friend (“James”), an add friend button 1126 , and a share button 1128 . Although illustrated as including, the present invention is not limited thereto, and other information may be further included.

The third operation step 1130 represents that the user terminal receives the user's selection for the second pattern 1114 from the user, and displays a preview 1134 for the second pattern in the preview area on the display. When the user selects the second pattern 1114 through a touch input, the user terminal displays an indicator 1132 indicating that the second pattern 1114 has been selected on the second pattern 1114, View 1134 may be displayed in the preview area of the display. As illustrated, the second pattern 1114 may be a pattern for coding URL information associated with a service of a company other than a specific company.

The user terminal may transmit the corresponding pattern to an external device (eg, a web server) to generate a preview of the second pattern 1114 coding URL information or transmit URL information coded by the corresponding pattern to the external device. . In this case, the user terminal may receive at least one of a thumbnail, a title, and an explanatory text associated with the corresponding URL from the external device. The preview 1134 for the second pattern may include a thumbnail, title, description, and share button 1136 received from an external device.

When the user selects the preview 1134 for the second pattern through a touch input or the like, the user terminal may process URL information coded by the second pattern 1114 in a web browser application or the like. When the user selects the share button 1136 through a touch input, etc., the user terminal allows the user to share the second pattern 1114 or URL information coded by the second pattern 1114 with other users. . In the third operation step 1130 , the preview 1134 for the second pattern is illustrated as including a thumbnail, a title, a description text, and a share button 1136 , but is not limited thereto, and may further include other information. can

In FIG. 11 , the first pattern 1112 and the second pattern 1114 are shown to be displayed on the display, but the present invention is not limited thereto, and when three or more patterns are included in the received image, three or more patterns are displayed. may be displayed as selectable options. Further, in FIG. 11 , when the user selects one of the selectable options (patterns) displayed on the display, a preview for the corresponding pattern is shown, but the present invention is not limited thereto. Alternatively, the priority of the patterns in the image It is also possible to automatically display a preview of a pattern having a high priority based on the pattern without user input. In this case, when the patterns in the image have the same priority, a preview of a pattern having a high reliability score may be automatically displayed without a user input by calculating the reliability score of each pattern.

12 is a diagram illustrating an example of displaying previews of a plurality of patterns included in an image on a display according to an embodiment of the present disclosure; 12 illustrates a first operation step 1210 and a second operation step 1220 of the user terminal. In the first operation step 1210, when the user terminal receives an image including the first pattern 1212 and the second pattern 1214 visually coded through the image sensor in a state in which the user terminal is operating in the pattern scan mode, the second The first pattern 1212 and the second pattern 1214 are detected, and the area including the detected first pattern 1112 and the area including the second pattern 1114 are highlighted in different colors on the display. indicates.

In the second operation step 1220 , in response to the user terminal detecting the first pattern 1212 and the second pattern 1214 in the received image, a preview 1222 of the first pattern and the second pattern are performed. to automatically display a preview 1228 of , in the preview area on the display. As shown in FIG. 12 , the first pattern 1212 is a two-dimensional pattern for adding “James” as a friend in a non-financial service (ie, LINE messenger service) of a specific company, and the second pattern 1214 is a text may be a coded two-dimensional pattern. A preview 1222 of the first pattern may include a LINE messenger logo, information about a user added as a friend (“James”), an add friend button 1224 , and a share button 1226 . When the user selects the add friend button 1246 in the preview 1222 for the first pattern through a touch input or the like, the user terminal may perform a friend add function in the LINE messenger. When the user selects the share button 1224 in the preview 1222 for the first pattern through a touch input or the like, the user terminal indicates that the user selects the first pattern 1212 or the coded by the first pattern 1212 . Allow information to be shared with other users.

The preview 1228 for the second pattern may include text information coded by the pattern, a text copy button 1230 , and a share button 1232 . When the user selects the text copy button 1230 by a touch input or the like, text information coded by the pattern may be copied to the memory area of the user terminal. When the user selects the share button 1232 by a touch input or the like, the user terminal allows the user to share the second pattern 1214 or text information coded by the second pattern 1214 with other users.

In FIG. 12 , a preview 1222 for the first pattern and a preview 1228 for the second pattern are shown to be displayed on the display, but the present invention is not limited thereto, and three or more patterns are included in the received image. In this case, a preview for each of the three or more patterns may be displayed on the display. In addition, although it is illustrated in FIG. 12 that previews are displayed in the order of patterns in the image, the present invention is not limited thereto, and alternatively, a preview of a pattern having a higher priority based on the priority of the patterns in the image is displayed in the preview area. It can also be displayed at the top of In this case, when the patterns in the image have the same priority, a preview of a pattern having a high reliability score may be displayed at the top of the preview area by calculating the reliability score of each pattern.

13 is a diagram illustrating an example of displaying a preview of a pattern selected by a user on a display according to an embodiment of the present disclosure; 13 illustrates a first operation step 1310 , a second operation step 1320 , and a third operation step 1330 of the user terminal. In the first operation step 1310, when the user terminal receives an image including the visually coded first pattern 1312 and the second pattern 1314 through the image sensor while operating in the pattern scan mode, the second It indicates that the first pattern 1312 and the second pattern 1314 are detected. In this case, the area including the first pattern 1312 and the area including the second pattern 1314 may be highlighted in different colors on the display.

The second operation step 1320 represents that the user terminal displays the detected patterns 1312 and 1314 as selectable options 1324 and 1326 on the option area 1322 on the display. For example, the selectable options 1324 and 1326 may be icons created by resizing the detected patterns 1312 and 1314 . All detected patterns may be displayed in the option area 1322 .

When the user selects the first option 1324 through a touch input, etc., a preview 1328 of the first pattern 1312 associated with the first option 1324 may be displayed in the preview area of the display. . As illustrated, the first pattern 1312 may be a two-dimensional pattern for adding “James” as a friend in a non-financial service (ie, LINE messenger service) of a specific company. In this case, the unselected option (ie, the second option 1326 ) may be shaded to indicate that the first option 1324 has been selected. When the user selects the add friend button in the preview 1328 for the first pattern 1312 associated with the first option 1324 through a touch input, the user terminal may perform a friend add function in the LINE messenger. have.

In the third operation step 1330 , the user selects the second option 1326 through a touch input or the like, so that a preview 1332 of the second pattern 1314 associated with the second option 1326 is displayed in advance of the display. Indicates what is displayed in the viewing area. As illustrated, the second pattern 1314 may be a two-dimensional pattern for coding URL information related to services of other companies. In this case, the user terminal receives at least one of thumbnails, titles, and descriptions associated with the URL from the external device, and adds at least one of the thumbnails, titles, and descriptions received from the external device to the second pattern 1314. It may be displayed in the preview area on the display as a preview 1332 of the . When the user selects the preview 1332 for the second pattern 1314 , the user terminal may process URL information coded by the second pattern 1314 in a web browser application or the like.

14 is a flowchart illustrating a visually coded pattern recognition method 1400 according to an embodiment of the present disclosure. In operation 1410 , the user terminal receives an image including the first pattern and the second pattern, and the operation 1400 may be initiated. For example, an image including a plurality of patterns may be received while the user terminal is operating in a pattern scan mode for recognizing a visually coded pattern.

After that, the user terminal may detect a region including the visually coded pattern (the first pattern and the second pattern) in the image, and display the first pattern and the second pattern as selectable options on the display. (1420). In an embodiment, the region including the first pattern and the region including the second pattern are highlighted in different colors on the display to be used as selectable options. In another embodiment, icons created by resizing the detected patterns may be displayed in an option area on the display as selectable options.

In operation 1430, the user terminal may receive a selection for the first pattern from the user. In this case, a preview of the selected first pattern may be displayed in a preview area on the display ( 1440 ). Upon receiving a selection for a preview for the first pattern from the user in step 1450 , the user terminal may execute processing on the first pattern ( 1460 ).

14 shows that the first pattern and the second pattern are displayed in the option area on the display as selectable options, but the present invention is not limited thereto, and when three or more patterns are included in the received image, three or more patterns These selectable options can be displayed in the options area on the display.

15 is a flowchart illustrating a method 1500 for visually coded pattern recognition according to an embodiment of the present disclosure. In operation 1510 , the user terminal receives an image including the first pattern and the second pattern, and the operation 1500 may be initiated. For example, an image including a plurality of patterns may be received while the user terminal is operating in a pattern scan mode for recognizing a visually coded pattern.

Thereafter, the user terminal may detect the first pattern and the second pattern from the received image, and display a preview of the first pattern and a preview of the second pattern in a preview area on the display ( 1520 ). . In an embodiment, the arrangement of the previews for the first pattern and the second pattern may be determined by comparing the priority of the pattern type of the first pattern and the pattern type of the second pattern. Alternatively or additionally, the placement of the preview for the first pattern and the second pattern may be determined by comparing the confidence score of the first pattern with the confidence score of the second pattern.

The user terminal may receive a selection for a preview of the first pattern displayed on the display from the user ( 1530 ). In this case, the user terminal may execute processing on the selected first pattern ( 1540 ). In FIG. 15 , previews of the first pattern and the second pattern are shown to be displayed in the preview area on the display, but the present invention is not limited thereto. A preview of the pattern may be displayed in a preview area on the display.

The above-described visually coded pattern recognition method may be implemented as a computer-readable code (eg, a computer program) on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data readable by a computer system is stored. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. In addition, the computer-readable recording medium is distributed in a computer system connected through a network, so that the computer-readable code can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the above-described embodiments can be easily inferred by programmers in the art to which the present invention pertains.

The method, operation, or techniques of this disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design requirements imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementations should not be interpreted as causing a departure from the scope of the present disclosure.

In a hardware implementation, the processing units used to perform the techniques include one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs). ), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described in this disclosure. , a computer, or a combination thereof.

Accordingly, the various illustrative logic blocks, modules, and circuits described in connection with this disclosure are suitable for use in general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or the present disclosure. It may be implemented or performed in any combination of those designed to perform the functions described in A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, eg, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other configuration.

In firmware and/or software implementations, the techniques may include random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), PROM ( on computer readable media such as programmable read-only memory), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage devices, etc. It may be implemented as stored instructions (eg, a computer program). The instructions may be executable by one or more processors and may cause the processor(s) to perform certain aspects of the functionality described in this disclosure.

If implemented in software, the techniques may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a computer. By way of non-limiting example, such computer readable medium may contain RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or desired program code in the form of instructions or data structures. may include any other medium that can be used for transport or storage to a computer and can be accessed by a computer. Also, any connection is properly termed a computer-readable medium.

For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, wireless, and microwave, the coaxial cable , fiber optic cable, twisted pair, digital subscriber line, or wireless technologies such as infrared, radio, and microwave are included within the definition of a medium. As used herein, disk and disk include CD, laser disk, optical disk, digital versatile disc (DVD), floppy disk, and Blu-ray disk, where disks are usually magnetic Data is reproduced optically, while discs reproduce data optically using a laser. Combinations of the above should also be included within the scope of computer-readable media.

A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium can be coupled to the processor such that the processor can read information from, or write information to, the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and storage medium may reside within the ASIC. The ASIC may exist in the user terminal. Alternatively, the processor and the storage medium may exist as separate components in the user terminal.

Although the embodiments described above have been described utilizing aspects of the presently disclosed subject matter in one or more standalone computer systems, the present disclosure is not so limited and may be implemented in connection with any computing environment, such as a network or distributed computing environment. . Still further, aspects of the subject matter in this disclosure may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across the plurality of devices. Such devices may include PCs, network servers, and portable devices.

Although the present disclosure has been described in connection with some embodiments herein, various modifications and changes may be made without departing from the scope of the present disclosure that can be understood by those skilled in the art to which the present disclosure pertains. Further, such modifications and variations are intended to fall within the scope of the claims appended hereto.

110: user 120: user terminal
130: object 132, 134, 136: visually coded pattern
210, 220, 230: user terminal
240: network 250: server 300: user terminal
310: image sensor 320: display
330: input interface 340: communication module
350: processor 352: pattern area detection unit
354: pattern type determination unit 356: pattern preview generation unit
358: pattern processing unit 360: storage unit

Claims (18)

As a pattern recognition method operating in a user terminal,
receiving an image comprising a first visually coded pattern and a second visually coded pattern;
generating a preview for the visually coded first pattern;
generating a preview for the visually coded second pattern; and
displaying at least one of a preview of the first pattern and a preview of the second pattern in a preview area on a display of the user terminal;
Including, a pattern recognition method.
According to claim 1,
The preview of the first pattern and the preview of the second pattern are simultaneously displayed in the preview area without a user input.
3. The method of claim 2,
Displaying at least one of a preview of the first pattern and a preview of the second pattern on a preview area on the display of the user terminal may include:
determining a pattern type of the first visually coded pattern based on a service associated with the first visually coded pattern;
determining a pattern type of the visually coded second pattern based on a service associated with the second visually coded pattern;
comparing the priority of the pattern type of the first visually coded pattern and the pattern type of the second visually coded pattern; and
When it is determined that the pattern type of the visually coded first pattern has a higher priority than the pattern type of the visually coded second pattern, a preview of the visually coded first pattern is displayed in the user terminal at the top of the preview area on the display of
Including, a pattern recognition method.
4. The method of claim 3,
Determining a pattern type of the first visually coded pattern based on a service associated with the first visually coded pattern comprises:
determining whether the visually coded first pattern is a pattern associated with a particular service; and
determining the pattern type of the first visually coded pattern as a first pattern type when it is determined that the first visually coded pattern is a pattern associated with a particular service;
Including, a pattern recognition method.
4. The method of claim 3,
Displaying at least one of a preview of the first pattern and a preview of the second pattern on a preview area on the display of the user terminal may include:
When it is determined that the pattern type of the first visually coded pattern has the same priority as the pattern type of the second visually coded pattern,
calculating a confidence score of the visually coded first pattern based on at least one of a location or a size of the visually coded first pattern within the received image;
calculating a confidence score of the visually coded second pattern based on at least one of a location or a size of the visually coded second pattern within the received image; and
When it is determined that the confidence score of the visually coded first pattern is higher than the confidence score of the visually coded second pattern, a preview of the visually coded first pattern is displayed on the display of the user terminal. step to display
Further comprising, a pattern recognition method.
3. The method of claim 2,
Displaying at least one of a preview of the first pattern and a preview of the second pattern on a preview area on the display of the user terminal may include:
calculating a confidence score of the visually coded first pattern based on at least one of a location or a size of the visually coded first pattern within the received image;
calculating a confidence score of the visually coded second pattern based on at least one of a location or a size of the visually coded second pattern within the received image; and
When it is determined that the confidence score of the visually coded first pattern is higher than the confidence score of the visually coded second pattern, a preview of the visually coded first pattern is previewed on the display of the user terminal. Steps to display at the top of the view area
Including, a pattern recognition method.
3. The method of claim 2,
receiving a selection for a preview for the first pattern from a user; and
executing processing on the visually coded first pattern in response to the selection;
Further comprising, a pattern recognition method.
According to claim 1,
detecting a first region including the visually coded first pattern;
detecting a second region including the visually coded second pattern; and
highlighting the first area and the second area with different colors;
Further comprising, a pattern recognition method.
9. The method of claim 8,
Displaying at least one of a preview of the first pattern and a preview of the second pattern on a preview area on the display of the user terminal may include:
displaying a preview of the first pattern in the preview area in response to the user selecting the first area
Including, a pattern recognition method.
According to claim 1,
The step of generating a preview for the visually coded first pattern comprises:
transmitting URL information coded by the visually coded first pattern to an external device when the visually coded first pattern corresponds to the encoding of a Uniform Resource Locator (URL);
receiving at least one of a thumbnail, a title, and a description associated with the URL information from the external device; and
displaying at least one of the received thumbnail, title, and description in a preview area on the display;
Including, a pattern recognition method.
A computer program stored in a computer-readable recording medium for executing the pattern recognition method according to any one of claims 1 to 10 in a computer.
A pattern recognition device comprising:
display;
Memory; and
at least one processor coupled to the memory and configured to execute at least one computer readable program contained in the memory
including,
the at least one program,
receive an image comprising a first visually coded pattern and a second visually coded pattern;
generate a preview for the visually coded first pattern;
generate a preview for the visually coded second pattern;
and instructions for displaying at least one of a preview of the first pattern and a preview of the second pattern on a preview area on the display.
13. The method of claim 12,
The preview of the first pattern and the preview of the second pattern are simultaneously displayed in the preview area without a user input.
14. The method of claim 13,
Displaying at least one of a preview of the first pattern and a preview of the second pattern in a preview area on the display comprises:
determining a pattern type of the first visually coded pattern based on a service associated with the first visually coded pattern;
determining a pattern type of the visually coded second pattern based on a service associated with the second visually coded pattern;
comparing the priority of the pattern type of the first visually coded pattern with the pattern type of the second visually coded pattern, and
When it is determined that the pattern type of the first visually coded pattern has a higher priority than the pattern type of the second visually coded pattern, a preview of the first visually coded pattern is displayed on the display. What is displayed at the top of the preview area
Including, a pattern recognition device.
14. The method of claim 13,
Displaying at least one of a preview of the first pattern and a preview of the second pattern in a preview area on the display comprises:
calculating a confidence score of the visually coded first pattern based on at least one of a location or a size of the visually coded first pattern within the received image;
calculating a confidence score of the visually coded second pattern based on at least one of a location or a size of the visually coded second pattern within the received image, and
If it is determined that the confidence score of the visually coded first pattern is higher than the confidence score of the visually coded second pattern, a preview of the visually coded first pattern is displayed in the preview area on the display. to show at the top
Including, a pattern recognition device.
14. The method of claim 13,
the at least one program,
receiving a selection for a preview for the first pattern from a user;
and instructions for executing processing on the visually coded first pattern in response to the selection.
13. The method of claim 12,
the at least one program,
detecting a first region including the visually coded first pattern,
detecting a second region including the visually coded second pattern,
and instructions for highlighting the first area and the second area with different colors.
13. The method of claim 12,
Generating a preview for the visually coded first pattern comprises:
When the visually coded first pattern corresponds to the encoding of a Uniform Resource Locator (URL), transmitting URL information coded by the visually coded first pattern to an external device;
Receiving at least one of a thumbnail, title, or description associated with the URL information from the external device, and
displaying at least one of the received thumbnail, title, or description in a preview area on the display;
Including, a pattern recognition device.

Images