KR20210047749A - Method and device for recognizing visually coded patterns - Google Patents

Abstract

The present disclosure relates to a method for recognizing a pattern operating in a user terminal. The method for recognizing a pattern includes the steps of: determining the pattern type of a first pattern; determining the pattern type of a second pattern; comparing the priority of the pattern type of the first pattern and the pattern type of the second pattern; and automatically executing processing of the first pattern, when determined that the pattern type of the first pattern has a higher priority than the pattern type of the second pattern.

Description

Visually coded pattern recognition method and device {METHOD AND DEVICE FOR RECOGNIZING VISUALLY CODED PATTERNS}

The present disclosure relates to a method and apparatus for visually recognizing a coded pattern, and specifically, to recognize and automatically execute a plurality of visually coded patterns, or to display a preview of a plurality of visually coded patterns. It relates to a method and apparatus.

The spread of terminals such as smartphones, tablets, PCs, laptop computers, and desktop computers with cameras has been widely expanded. In such an environment, the number of cases of using a camera attached to a terminal is increasing. As a representative example, various services using barcodes photographed using a camera attached to a terminal are provided.

On the other hand, when a plurality of barcodes are included in an image photographed by a camera attached to the terminal, the barcode may not be recognized, or a barcode not intended by the user may be recognized. In this case, in order to receive a desired service, the user is inconvenient in having to re-photograph the barcode to be recognized. Accordingly, the time required to receive the service desired by the user increases, and it is difficult to provide a smooth user experience.

The present disclosure provides a pattern recognition method, an apparatus, and a computer program stored in a recording medium for solving the above problems.

The user terminal receives an image including a plurality of visually coded patterns, compares the priority of the pattern type by determining the pattern type of the patterns detected from the received image, and processes the pattern having a high priority accordingly. A method, an apparatus and a computer program stored in a recording medium are provided for automatically executing the operation.

A method, an apparatus, and a computer program stored in a recording medium are provided for calculating the reliability scores of patterns having the same priority of visually coded pattern types, and thereby automatically processing a pattern having a high reliability score.

Method and apparatus for receiving an image including a plurality of visually coded patterns in a user terminal, generating a preview for the pattern(s) detected from the received image, and displaying it in a preview area on the display of the user terminal And a computer program stored in the recording medium.

The present disclosure may be implemented in a variety of ways, including a method, apparatus, or computer program stored on a computer-readable storage medium.

A pattern recognition method operating in a user terminal according to an embodiment of the present disclosure includes receiving an image including a visually coded first pattern and a visually coded second pattern, and Determining the pattern type, determining the pattern type of the visually coded second pattern, comparing the priority of the pattern type of the visually coded first pattern with the pattern type of the visually coded second pattern And when it is determined that the pattern type of the visually coded first pattern has a higher priority than the pattern type of the visually coded second pattern, automatically executing the processing of the visually coded first pattern. do.

According to an embodiment, the determining of the pattern type of the visually coded first pattern includes determining whether the visually coded first pattern is a pattern associated with a specific service, and the visually coded first pattern is If it is determined that it is a pattern associated with the service, determining the pattern type of the visually coded first pattern as the first pattern type.

According to an embodiment, the determining of the pattern type of the visually coded first pattern includes determining whether the visually coded first pattern is a pattern associated with a service of a specific company, and the visually coded first pattern When it is determined that it is a pattern associated with a service of a specific company, determining the visually coded pattern type of the first pattern as the second pattern type.

According to an embodiment, the determining of the pattern type of the visually coded first pattern is, when it is determined that the visually coded first pattern is a pattern that is not associated with a service of a specific company, the visually coded first pattern When the first pattern is a two-dimensional pattern, determining a visually coded pattern type of the first pattern as a third pattern type, and when the visually coded first pattern is a one-dimensional pattern, a visually coded first pattern And recognizing the pattern type of as the fourth pattern type.

According to an embodiment, the first pattern type, the second pattern type, the third pattern type, and the fourth pattern type have a high priority in order.

According to an embodiment, when it is determined that the pattern type of the visually coded first pattern has the same priority as the pattern type of the visually coded second pattern, the visually coded first pattern in the received image Calculating a reliability score of the visually coded first pattern based on at least one of the location or size of the pattern, visually based on at least one of the location or size of the visually coded second pattern in the received image Calculating a reliability score of the second pattern coded as and when it is determined that the reliability score of the visually coded first pattern is higher than the reliability score of the visually coded second pattern, the visually coded first pattern And displaying a preview of the user terminal on the display of the user terminal.

According to an embodiment, calculating the reliability score of the visually coded first pattern based on at least one of the position or size of the visually coded first pattern in the received image includes: And calculating a distance between the point and the central point of the visually coded first pattern.

According to an embodiment, calculating the reliability score of the visually coded first pattern based on at least one of the position or size of the visually coded first pattern in the received image includes: 1 and calculating an area ratio of the area corresponding to the pattern occupied in the received image.

According to an embodiment, when it is determined that the pattern type of the visually coded first pattern has the same priority as the pattern type of the visually coded second pattern, the visually coded first pattern in the received image Calculating a reliability score of the visually coded first pattern based on at least one of the location or size of the pattern, visually based on at least one of the location or size of the visually coded second pattern in the received image Calculating a reliability score of the second pattern coded as and when it is determined that the reliability score of the visually coded first pattern is higher than the reliability score of the visually coded second pattern, It further comprises the step of automatically executing the process.

According to an embodiment, the method further includes notifying the user to automatically execute the processing of the visually coded first pattern by displaying an indicator on the first area corresponding to the visually coded first pattern.

According to an embodiment, detecting a first area including a visually coded first pattern, detecting a second area including a visually coded second pattern, and determining the first area and the second area The method further includes displaying highlights in different colors.

According to an embodiment, in the case of sequentially receiving a plurality of images in an image capturing mode and receiving an image including a visually coded first pattern and a visually coded second pattern, whether to enter the pattern scan mode It further includes displaying a notification for confirming whether or not on the display of the user terminal.

In an embodiment of the present disclosure, a computer program stored in a computer-readable recording medium is provided in order to execute the pattern recognition method described above in a computer.

The pattern recognition apparatus according to an embodiment of the present disclosure includes an image sensor configured to receive an image including a visually coded first pattern and a visually coded second pattern, a pattern type and a visually coded first pattern. A pattern type determination unit configured to determine a pattern type of the second pattern coded with a pattern type and a pattern processing unit configured to compare the priority of the visually coded pattern type of the first pattern and the visually coded pattern type of the second pattern And, when it is determined that the pattern type of the visually coded first pattern has a higher priority than the pattern type of the visually coded second pattern, the pattern processing unit automatically processes the visually coded first pattern. It is further configured to run.

A pattern recognition method operating in a user terminal according to an embodiment of the present disclosure includes receiving an image including a visually coded first pattern and a visually coded second pattern. Generating a preview, generating a preview for a visually coded second pattern, displaying a preview for the first pattern and a preview for the second pattern in a preview area on the display of the user terminal A step, receiving a selection for a preview of the first pattern from the user, and executing processing on the visually coded first pattern in response to the selection.

According to an embodiment, the step of displaying the preview for the first pattern and the preview for the second pattern in the preview area on the display of the user terminal includes the visually coded first pattern in the received image. Calculating a reliability score of the visually coded first pattern based on at least one of the location or size, visually coding based on at least one of the location or size of the visually coded second pattern in the received image Calculating a reliability score of the visually coded second pattern, and when it is determined that the reliability score of the visually coded first pattern is higher than the reliability score of the visually coded second pattern, the visually coded first pattern is previously And displaying the view at the top of the preview area on the display of the user terminal.

According to an embodiment, detecting a first area including a visually coded first pattern, detecting a second area including a visually coded second pattern, and determining the first area and the second area The method further includes displaying highlights in different colors.

According to an embodiment, the step of generating a preview for the visually coded first pattern comprises: when the visually coded first pattern corresponds to encoding of a URL (Uniform Resource Locator), the visually coded first pattern Transmitting URL information coded by the pattern to an external device, receiving at least one of a thumbnail, title, or description phrase associated with the URL information from the external device, and at least one of the received thumbnail, title, or description phrase. The method further includes displaying in a preview area on the display.

In an embodiment of the present disclosure, a computer program stored in a computer-readable recording medium is provided in order to execute the pattern recognition method described above in a computer.

A pattern recognition apparatus according to an embodiment of the present disclosure includes an input interface configured to receive an input from a user, an image sensor configured to receive an image including a visually coded first pattern and a visually coded second pattern, and visually A pattern preview generator configured to generate a preview for the coded first pattern and a preview for the visually coded second pattern, and preview the generated visually coded first pattern and the second pattern in advance. A pattern processing unit configured to execute processing on the visually coded first pattern in response to receiving a selection for the preview for the first pattern through the display and input interface configured to output to the viewing area.

According to embodiments of the present disclosure, the accuracy of pattern recognition is improved by automatically executing pattern processing based on the priority of the pattern type and/or the reliability score of the pattern for a plurality of visually coded patterns, A better user experience can be provided by easily providing the service the user wants.

According to embodiments of the present disclosure, previews for a plurality of patterns are displayed in a preview area on a display of a user terminal, so that a user can select a pattern related to a desired service.

The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, in which like reference numerals denote like elements, but are not limited thereto.
1 is a diagram illustrating an example of a pattern recognition method operating in a user terminal according to an embodiment of the present disclosure.
2 is a diagram illustrating a configuration in which a plurality of user terminals are connected to enable communication with a server in order to perform pattern recognition according to an embodiment of the present disclosure.
3 is a block diagram showing the configuration of a user terminal according to an embodiment of the present invention.
4 is a diagram illustrating a lookup table defining priorities for each pattern type according to an embodiment of the present invention.
5 is a diagram illustrating an example of automatically executing pattern processing without a user input according to an embodiment of the present invention.
6 is a flowchart illustrating a visually coded pattern recognition method according to an embodiment of the present disclosure.
7 is a flowchart illustrating a method of executing processing for one of patterns having the same priority according to an embodiment of the present disclosure.
8 is a flowchart illustrating a method of executing processing for one of patterns having the same priority according to an embodiment of the present disclosure.
9 is a flowchart illustrating a method of determining a pattern type of a visually coded pattern according to an embodiment of the present disclosure.
FIG. 10 is a diagram illustrating an example of displaying a notification for confirming whether to enter a pattern scan mode when a user terminal receives a plurality of visually coded patterns according to an embodiment of the present disclosure.
11 is a diagram illustrating an example of displaying a preview of a pattern selected by a user on a display according to an embodiment of the present disclosure.
12 is a diagram illustrating an example of displaying a preview of each of a plurality of patterns included in an image on a display according to an embodiment of the present disclosure.
13 is a diagram illustrating an example of displaying a preview of a pattern selected by a user on a display according to an embodiment of the present disclosure.
14 is a flowchart illustrating a visually coded pattern recognition method according to an embodiment of the present disclosure.
15 is a flowchart illustrating a visually coded pattern recognition method according to an embodiment of the present disclosure.

Hereinafter, with reference to the accompanying drawings, specific details for the implementation of the present disclosure will be described in detail. However, in the following description, when there is a possibility that the subject matter of the present disclosure may be unnecessarily obscure, detailed descriptions of widely known functions or configurations will be omitted.

In the accompanying drawings, the same reference numerals may be assigned to the same or corresponding components. In addition, in the description of the following embodiments, overlapping descriptions of the same or corresponding components may be omitted. However, even if description of a component is omitted, it is not intended that such component is not included in any embodiment.

Advantages and features of the disclosed embodiments, and a method of achieving them will become apparent with reference to the embodiments described below together with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below, but may be implemented in a variety of different forms, and only these embodiments make the present disclosure complete, and those skilled in the art to which the present disclosure pertains. It is provided only to fully inform the person of the scope of the invention.

The terms used in the present specification will be briefly described, and the disclosed embodiments will be described in detail.

Terms used in the present specification have selected general terms that are currently widely used as possible while considering functions in the present disclosure, but this may vary according to the intention or precedent of a technician engaged in a related field, the emergence of new technologies, and the like. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning of the terms will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present disclosure should be defined based on the meaning of the term and the overall contents of the present disclosure, not a simple name of the term.

In the present specification, expressions in the singular include plural expressions unless the context clearly specifies that they are in the singular. In addition, plural expressions include expressions in the singular, unless the context clearly specifies that they are plural.

When a part of the specification is said to "include" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary.

In addition, the terms "module" or "unit" used in the specification mean software or hardware components, and "module" or "unit" performs certain roles. However,'module' or'unit' is not meant to be limited to software or hardware. The'module' or'unit' may be configured to be in an addressable storage medium, or may be configured to reproduce one or more processors. Thus, as an example, the'module' or'unit' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, It may include at least one of procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. Components and functions provided within the'module' or'unit' may be combined into a smaller number of components and'module' or'unit', or additional components and'module' or'unit'. Can be further separated.

According to an embodiment of the present disclosure, a'module' or'unit' may be implemented with a processor and a memory. 'Processor' should be interpreted broadly to include general purpose processors, central processing units (CPUs), microprocessors, digital signal processors (DSPs), controllers, microcontrollers, state machines, etc. In some circumstances, a'processor' may refer to an application specific semiconductor (ASIC), programmable logic device (PLD), field programmable gate array (FPGA), and the like. 'Processor' refers to a combination of processing devices such as, for example, a combination of a DSP and a microprocessor, a combination of a plurality of microprocessors, a combination of one or more microprocessors in combination with a DSP core, or any other such combination of configurations. You may. In addition, the'memory' should be broadly interpreted to include any electronic component capable of storing electronic information. 'Memory' refers to random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erase-programmable read-only memory (EPROM), It may refer to various types of processor-readable media such as electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, and the like. The memory is said to be in electronic communication with the processor if the processor can read information from and/or write information to the memory. The memory integrated in the processor is in electronic communication with the processor.

In the present disclosure, the'image' may include an image captured through an image sensor attached to a user terminal, and an image captured by an image sensor configured to communicate with the user terminal. Also, the image may include some images extracted from the captured video image.

In the present disclosure, a'visually coded pattern' refers to a code in which data or information is displayed in a machine-readable visual form. Visually coded patterns can be scanned/read using an application program of a device with a built-in camera, such as a pattern reader or a smartphone. The visually coded pattern includes a visually coded one-dimensional pattern and a visually coded two-dimensional pattern. The visually coded one-dimensional pattern is composed of parallel lines of different widths and intervals, and may include a Universal Product Code (UPC). The visually coded 2D pattern is composed of geometric patterns such as squares, dots, and hexagons, and may include Data Matrix, MaxiCode, and Quick Response (QR) codes. In this specification, the'visually coded pattern' may also be referred to as a'pattern'.

1 is a diagram illustrating an example of a pattern recognition method operating in a user terminal 120 according to an embodiment of the present disclosure. As shown in FIG. 1, the user 110 scans one of the first pattern 132, the second pattern 134, and the third pattern 136 included in the object 130. The user terminal 120 may be held so that the image sensor of) faces the object 130. For example, the object 130 may be a document on which a plurality of visually coded patterns are printed. As illustrated, the first pattern 132 and the second pattern 134 may be a two-dimensional pattern, and the third pattern 136 may be a one-dimensional pattern.

When the user terminal 120 receives an image including the first pattern 132, the second pattern 134, and the third pattern 136, the user terminal 120 displays a pattern 132 visually coded in the image. , 134, 136) may be detected, and the pattern type of each pattern may be determined. The user terminal 120 may compare the priorities of the pattern types of each pattern and automatically process the pattern having the highest priority. For example, when the user terminal 120 determines that the priority of the first pattern 132 is higher than that of the second pattern 134 and the third pattern 136, the user terminal 120 The processing of the first pattern 132 can be automatically executed without input.

To this end, the user terminal 120 may store priorities for each predefined pattern type. In one embodiment, the user terminal 120 determines a pattern related to a financial service (eg, LINE PAY service) of a specific company as a first pattern type, and a pattern related to a non-financial service (eg, LINE messenger service) of a specific company is determined. 2 The pattern type may be determined, other two-dimensional patterns related to the service of the company may be determined as the third pattern type, and other one-dimensional patterns may be determined as the fourth pattern type. In this case, the user terminal 120 may be predefined as having a high priority in the order of a first pattern type, a second pattern type, a third pattern type, and a fourth pattern type.

When a plurality of patterns having the same pattern type exist in one image, and there is no pattern having a higher priority than the corresponding patterns, the user terminal 120 provides a preview of the corresponding patterns to the user and You can choose a pattern that you want to run (or scan). Alternatively, the user terminal may calculate the reliability score of the corresponding patterns and automatically process the pattern having the highest reliability score. In this case, the reliability score of each pattern may be calculated based on at least one of the position or size of the pattern in the image.

In another embodiment, the user terminal 120 does not automatically execute processing on one of the plurality of patterns in the image, and the user 110 has one of the plurality of patterns 132, 134, and 136 included in the image. You can let them choose a pattern. In this case, the user terminal 120 may display the patterns 132, 134, and 146 detected in the image on the display as a selectable option. Additionally or alternatively, the user terminal 120 may generate a preview of the patterns 132, 134, and 146 detected in the image and display them on the display. In this case, the user 110 may check previews of patterns displayed on the display, and then select one pattern to process the corresponding pattern.

FIG. 2 is a diagram illustrating a configuration in which a plurality of user terminals 210, 220, and 230 are connected to communicate with the server 250 in order to perform pattern recognition according to an exemplary embodiment of the present disclosure. In FIG. 2, the mobile phone terminal 210, the tablet terminal 220, and the notebook terminal 230 are illustrated as examples of user terminals, but are not limited thereto, and the user terminal is capable of wired and/or wireless communication, and an image sensor (e.g. For example, a camera) may be attached or any computing device capable of communicating with an image sensor and installing and executing an application program capable of visually scanning a coded pattern. For example, the user terminal is a smart phone, mobile phone, navigation, computer, notebook, digital broadcasting terminal, PDA (Personal Digital Assistants), PMP (Portable Multimedia Player), tablet PC, game console, It may include a wearable device, an internet of things (IoT) device, a virtual reality (VR) device, an augmented reality (AR) device, and the like.

Further, in FIG. 2, three user terminals 210, 220, and 230 are shown to communicate with the server 250 through the network 240, but the present invention is not limited thereto, and a different number of user terminals may be connected to the network 240. It may be configured to communicate with the server 250 through. The communication method through the network 240 is not limited, and may include a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network, a satellite network, and short-range wireless communication between user terminals. For example, the network 240 includes a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), It may include any one or more of the networks, such as the Internet. In addition, the network 240 may include any one or more of a network topology including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, etc. Not limited.

According to an embodiment, the server 250 is one or more server devices and/or databases capable of storing, providing and executing computer-executable programs and data, or one or more distributed computing devices and/or distributed databases based on cloud computing services. It may include. In one embodiment, the server 250 may be a web server configured to store at least one of a thumbnail, a title, or a description phrase for a URL (Uniform Resource Locator).

In one embodiment, when the pattern corresponds to the encoding of a URL (Uniform Resource Locator), the user terminals 210, 220, 230 transmit the pattern to the server 250 to generate a preview for the pattern, or URL information coded by the pattern may be transmitted to the server 250. In this case, the server 250 may provide at least one of a thumbnail, a title, or a description phrase associated with the corresponding URL to the user terminals 210, 220, and 230. The user terminals 210, 220, 230 may display at least one of the received thumbnail, title, or description phrase on the display as a preview for the corresponding pattern.

3 is a block diagram showing the configuration of a user terminal 300 according to an embodiment of the present invention. The user terminal 300 may be any device that already has capturing and image processing capabilities. As shown, the user terminal 300 may include an image sensor 310, a display 320, an input interface 330, a communication module 340, a processor 350, and a storage unit 360.

The image sensor 310 may be configured to receive one or a plurality of images. For example, the image sensor 310 may be a camera module. The image received by the image sensor 310 may be temporarily or permanently stored in the storage unit 360 of the user terminal 300 in the form of an electronic file. In one embodiment, the image received by the image sensor 310 may be provided to the processor 350 to determine whether the image includes a visually coded pattern. In another embodiment, the image sensor 310 may be configured to store the image in the storage unit 360 or provide it to the processor 350 in response to receiving an input command from the user through the input interface 330. .

The display 320 may be configured to output information and/or data generated by the user terminal 300. According to an embodiment, an image received by the image sensor 310 may be displayed on the display 320. In addition to the display 320, devices such as a speaker and a haptic feedback device may be provided together with the display 320 as an output device.

The input interface 330 may be configured to allow a user to input information and/or data to the user terminal 300. For example, the input interface 330 may include devices such as a touch screen, a button, a keypad, a touch pad, a keyboard, a microphone, and a mouse. In FIG. 3, the display 320 is illustrated as a separate component from the input interface 330, but is not limited thereto, and the display 320 and the input interface 330 may be provided as an integrated input/output device.

As shown in FIG. 3, the user terminal 300 may be configured to communicate information and/or data with other devices through the network 370 using the communication module 340. As an example, the request generated by the processor 350 of the user terminal 300 according to the program code stored in the storage unit 360 is controlled by the communication module 340 through the network 370 through another device (e.g., a server). ) Can be delivered. Conversely, control signals, commands, contents, files, etc. provided by other devices may be received by the user terminal 300 through the communication module 340. For example, control signals, commands, contents, files, etc. of other devices received through the communication module 340 may be transmitted to the processor 350 or the storage unit 360.

The processor 350 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. The command may be provided to the processor 350 by the storage unit 360 or the communication module 340. For example, the processor 350 may be configured to execute a command received according to a program code stored in the storage unit 360. In one embodiment, the processor 350 may include a pattern area detection unit 352, a pattern type determination unit 354, a pattern preview generation unit 356, and a pattern processing unit 358.

The storage unit 360 may include any computer-readable recording medium. According to an embodiment, the storage unit 360 may include a storage device such as random access memory (RAM), read only memory (ROM), disk drive, solid state drive (SSD), flash memory, and the like. have. At least one program code such as an operating system and an application program may be stored in the storage unit 360.

In one embodiment, the image sensor 310 may receive an image including a plurality of patterns. In this case, the image received by the image sensor 310 may be output on the display 320 and provided to the processor 350. The processor 350 provides the received image to the pattern area detection unit 352, and the pattern area detection unit 352 detects an area including a visually coded pattern in the image, so that the visually coded pattern in the image is Whether or not it is included can be determined.

When the pattern area detection unit 352 determines that a visually coded pattern is not included in the received image, the processor 350 may cause the image sensor 310 to receive a subsequent image. When the pattern area detection unit 352 determines that a visually coded pattern is included in the received image, the pattern area detection unit 352 provides the pattern to the pattern processing unit 358, and the pattern processing unit 358 Pattern processing can be performed. On the other hand, when the pattern area detection unit 352 determines that a plurality of patterns are included in the received image, the pattern area detection unit 352 determines the plurality of detected patterns (or the image of the area where the pattern is detected) as a pattern type determination unit. Can be provided to (354).

The pattern type determination unit 354 may determine the pattern type of each of the plurality of received patterns. For example, a pattern associated with a specific company's financial service (e.g., LINE Pay service) is determined as the first pattern type, and a pattern associated with a specific company's non-financial service (e.g., LINE messenger service) is determined as a second pattern type. In addition, other two-dimensional patterns associated with the service of the company may be determined as a third pattern type, and a one-dimensional pattern may be determined as a fourth pattern type. In one embodiment, the pattern type determination unit 354 separates the pattern or information coded by the pattern in order to determine whether each pattern is a pattern associated with a financial service (eg, LINE Pay service) of a specific company. It can be transferred to a module (e.g., LINE PAY application). In another embodiment, the pattern type determination unit 354 may include a module (eg, LINE Pay application) for determining whether each pattern is a pattern related to a financial service (eg, LINE Pay service) of a specific company. .

The pattern processing unit 358 may receive pattern types of each of the plurality of patterns from the pattern type determination unit 354 and compare the priorities of the patterns according to the priorities of each pattern type previously stored in the storage unit 360. For example, the storage unit 360 may be predefined as having a high priority in the order of a first pattern type, a second pattern type, a third pattern type, and a fourth pattern type. When it is determined that the pattern having the highest priority is one, the pattern processing unit 358 may automatically execute the processing of the pattern without user input.

On the other hand, when it is determined that there are two or more patterns having the highest priority, the pattern processing unit 358 may calculate a reliability score of each pattern. In an embodiment, the reliability score of the visually coded pattern may be calculated based on at least one of the location or size of the corresponding pattern. For example, the reliability score of a visually coded pattern may be calculated based on a distance between a central point of an image and a central point of a corresponding pattern, or may be calculated based on a ratio of an area occupied by a corresponding pattern in the image. After that, the pattern processing unit 358 may automatically execute the processing of the pattern having the highest reliability score without user input.

In another embodiment, when it is determined that there are two or more patterns having the highest priority, the pattern processing unit 358 may provide the patterns having the highest priority to the pattern preview generator 356. In this case, the pattern preview generator 356 may generate a preview for each pattern and display it on the display 320 so that the user may select one pattern. In this case, the pattern preview generator 356 may calculate a reliability score of each pattern and display a preview of a pattern having a higher reliability score at the top of the preview area on the display 320. When the user selects one of the previews displayed on the display 320 through the input interface 330, the pattern processing unit 358 may process a pattern associated with the preview selected by the user. Alternatively, the preview generator 356 may display only a preview of the pattern having the highest reliability score in the preview area on the display 320.

In another embodiment, when the pattern area detection unit 352 detects a plurality of patterns in the received image, the detected plurality of patterns are transferred to the pattern preview generation unit 356 rather than the pattern type determination unit 354. It can be configured to be provided. In this case, the pattern preview generator 356 may generate a preview for each pattern and display it on the display 320. For example, a preview of each pattern may be displayed in a preview area on the display 320. In this case, the pattern preview generator 356 may determine the display order of previews for the pattern based on the priority of the pattern and/or the reliability score of the pattern. When the user selects one of a plurality of previews displayed on the display 320, the pattern processing unit 358 may process a pattern associated with the preview selected by the user.

For example, when the visually coded pattern corresponds to the encoding of a URL (Uniform Resource Locator), the pattern preview generation unit 356 transmits the corresponding pattern or URL information coded by the corresponding pattern to the communication module 340. It can be transmitted to an external device (for example, a web server) through. In this case, the user terminal 300 may receive at least one of a thumbnail, a title, or a description phrase associated with the URL from an external device. The pattern preview generator 356 may generate a preview for a corresponding pattern based on at least one of a thumbnail, a title, or a description phrase received from an external device.

As another example, when the visually coded pattern corresponds to encoding of text information, the pattern preview generator 356 may generate a visually coded pattern preview based on the text information. When the visually coded pattern corresponds to the encoding of information (eg, URL information) related to a financial service of a specific company or a non-financial service of a specific company, the pattern preview generation unit 356 is a specific company coded by the pattern. You can create a user interface as a preview that can execute financial services or non-financial services of.

The user terminal 300 may include more components than those shown in FIG. 3. According to an embodiment, the user terminal 300 may further include other components such as a transceiver, a Global Positioning System (GPS) module, various sensors, and a database. For example, when the user terminal 300 is a smartphone, it may include components generally included in the smartphone, for example, using an acceleration sensor or a gyro sensor, various physical buttons, and a touch panel. Various components such as a button, an input/output port, and a vibrator for vibration may be implemented to be further included in the user terminal 300. In this case, the user terminal 300 may be configured to provide feedback (eg, haptic feedback) to the user when a pattern is detected in the received image.

4 is a diagram illustrating a lookup table 400 for defining priorities for each pattern type according to an embodiment of the present invention. In one embodiment, the visually coded pattern is a pattern related to a specific company's financial service (e.g., LINE Pay service), a pattern related to a specific company's non-financial service (e.g., LINE messenger service), and other two-dimensional patterns related to the company's service. And a one-dimensional pattern. According to the look-up table 400, a financial service-related pattern of a specific company, a non-financial service-related pattern of a specific company, a service-related two-dimensional pattern of other companies, and a one-dimensional pattern have high priority in the order of.

For example, a certain company's financial service-related pattern is a pattern for payment (e.g., a QR code for a store in a store, a QR code that a user uses to pay at a store, etc.), and a split bill. ) For a pattern (eg, a QR code for users to divide and pay for payment), a pattern for remittance of expenses, and the like. Patterns related to non-financial services of specific companies include patterns for adding friends (e.g., QR code to add a specific user as a friend), chat room invitation pattern, open chat invitation pattern, and pattern for performing login (e.g., QR code login). It may include. In addition, the company's service-related two-dimensional pattern may include a two-dimensional pattern for encoding URL, a two-dimensional pattern for encoding text information, and the like.

5 is a diagram illustrating an example of automatically executing pattern processing without a user input according to an embodiment of the present invention. 5 illustrates a first operation step 510 and a second operation step 520 of the user terminal. The first operation step 510 represents that the first pattern 512 is automatically selected from among the plurality of patterns 512 and 514 included in the image received by the image sensor of the user terminal without a user input. The first pattern 512 may be a two-dimensional pattern related to a financial service of a specific company, and the second pattern 514 may be a two-dimensional pattern related to a non-financial service of a specific company. In this case, it may be determined that the first pattern 512 has a higher priority than the second pattern 514.

In the first operation step 510, the user terminal may highlight the area including the first pattern 512 and the area including the second pattern 514 in different colors on the display. In FIG. 5, the detected first pattern 512 and the second pattern 514 are shown to be highlighted in different colors. However, the present invention is not limited thereto, and the first pattern so that the user can easily check each pattern area on the display. The pattern 512 and the second pattern 514 may be displayed in various ways, such as a shade display. In addition, the user terminal may display the indicator 516 on the first pattern 512 to notify the user that the processing of the first pattern 512 having a higher priority is automatically executed.

The second operation step 520 shows an example of a screen in which the processing of the first pattern 512 is automatically executed without a user input. Since the first pattern 512 is a pattern related to a financial service of a specific company, a financial service providing screen of a specific company related to the pattern may be displayed on the display of the user terminal. In this case, the user can proceed with payment by selecting the payment button 522 displayed on the display.

6 is a flowchart illustrating a visually coded pattern recognition method 600 according to an embodiment of the present disclosure. In step 610, the method 600 may be initiated by the user terminal receiving an image including a first pattern and a second pattern. For example, while the user terminal is operating in a pattern scan mode for visually recognizing a coded pattern, an image including a plurality of patterns may be received. Thereafter, the user terminal may detect a region including a visually coded pattern (first pattern and second pattern) in the image and determine the pattern type of each pattern (620). In one embodiment, the user terminal determines a pattern related to a financial service (eg, LINE PAY service) of a specific company as a first pattern type, and a pattern related to a non-financial service (eg, LINE messenger service) of a specific company is a second pattern type. 2D patterns related to other services of the company may be determined as the third pattern type, and the 1D pattern may be determined as the fourth pattern type.

In step 630, the user terminal may compare the priority of the pattern type of the first pattern and the pattern type of the second pattern. For example, the user terminal may be predefined as having a high priority in the order of a first pattern type, a second pattern type, a third pattern type, and a fourth pattern type. Based on the determination result in step 630, the user terminal may determine whether the priority of the pattern type of the first pattern and the pattern type of the second pattern are the same (640 ).

In step 640, if it is determined that the priority of the pattern type of the first pattern and the pattern type of the second pattern are not the same (that is, NO at 640), the user terminal has a higher priority without user input. The pattern determined to be processed may be automatically performed (650). With such a configuration, the processing of the pattern having the highest priority is automatically executed without the user pressing the camera photographing button or selecting one of a plurality of patterns, thereby providing a better user experience. On the other hand, when it is determined in step 640 that the priority of the pattern type of the first pattern and the pattern type of the second pattern are the same (that is, YES in 640), the method 600 is the method 700 of FIG. 7 or It may proceed to method 800 of FIG. 8.

7 is a flowchart illustrating a method 700 of executing processing for one of patterns having the same priority according to an embodiment of the present disclosure. In step 640 of FIG. 6, if it is determined that the pattern type of the first pattern and the pattern type of the second pattern have the same priority (that is, YES at 640), the user terminal calculates the reliability score of the first pattern. Can (710). For example, the reliability score of the first pattern may be calculated based on a result of calculating the distance between the center point of the first pattern and the center point of the received image. In this case, the reliability score of the first pattern may be in inverse proportion to the distance between the center point of the first pattern and the center point of the received image. That is, as the central point of the visually coded pattern is closer to the central point of the received image, the higher the reliability score of the pattern may be calculated. This is because there is a high possibility of holding the user terminal so that the pattern of interest to the user is arranged close to the center point of the image.

Alternatively, the reliability score of the first pattern may be calculated based on the ratio of the area occupied by the area corresponding to the first pattern in the received image. In this case, as the ratio of the area that the area corresponding to the pattern occupies in the received image increases, the reliability score of the pattern may be calculated higher. This is because the user terminal is highly likely to be gripped so that the pattern of interest to the user occupies a high area ratio in the image.

In step 720, the user terminal may calculate a reliability score of the second pattern. The reliability score of the second pattern may be calculated in the same manner as that of calculating the reliability score of the first pattern. That is, the reliability score of the second pattern will be calculated based on the result of calculating the distance between the center point of the second pattern and the center point of the received image or the ratio of the area occupied by the area corresponding to the second pattern in the received image. I can. In FIG. 7, the step 710 of calculating the reliability score of the first pattern is shown to precede the step 720 of calculating the reliability score of the second pattern, but is not limited thereto, and the reliability score of an arbitrary pattern is It can be calculated first, or the reliability score can be calculated at the same time.

Next, the user terminal may compare the calculated reliability score of the first pattern with the reliability score of the second pattern, and display a preview of the pattern having a higher reliability score on the display (730 ). For example, if the first pattern and the second pattern have the same priority and the first pattern is located closer to the center of the image than the second pattern, the user terminal has the reliability score of the first pattern and the reliability score of the second pattern. Determined to be higher, a preview for the first pattern can be displayed on the display.

In one embodiment, in the case of a pattern associated with a specific company's financial service or a specific company's non-financial service, the user terminal previews a user interface capable of executing the corresponding service (a specific company's financial service or a specific company's non-financial service). Can be displayed on the display. If the pattern is a URL type pattern related to other company's service, the user terminal receives at least one of a thumbnail, title, or description of the URL coded by the pattern from an external server and displays it as a preview on the display. can do. In the case of a pattern in which text information is coded, the user terminal may generate a preview based on the text information and display it on the display.

In step 740, the user terminal may receive a user's selection for a pattern displayed as a preview. For example, the user's selection may be an input for executing processing of a pattern associated with the preview displayed on the display. In this case, the user terminal may perform processing of the pattern selected by the user (750 ). In FIG. 7, a preview of a pattern having a higher reliability score is displayed on the display, but the present invention is not limited thereto, and all previews of the pattern having the same priority may be displayed on the display. In this case, a preview of the pattern having a higher reliability score may be displayed at the top of the preview area on the display.

8 is a flowchart illustrating a method 800 of executing processing for one of patterns having the same priority according to an embodiment of the present disclosure. In step 640 of FIG. 6, if it is determined that the pattern type of the first pattern and the pattern type of the second pattern have the same priority (that is, YES at 640), the user terminal calculates the reliability score of the first pattern. Can (810). For example, the reliability score of the first pattern may be calculated based on a result of calculating the distance between the center point of the first pattern and the center point of the received image. In this case, the reliability score of the first pattern may be in inverse proportion to the distance between the center point of the first pattern and the center point of the received image. That is, as the central point of the visually coded pattern is closer to the central point of the received image, the higher the reliability score of the pattern may be calculated. This is because there is a high possibility of holding the user terminal so that the pattern of interest to the user is arranged close to the center point of the image.

Alternatively, the reliability score of the first pattern may be calculated based on the ratio of the area occupied by the area corresponding to the first pattern in the received image. In this case, as the ratio of the area that the area corresponding to the pattern occupies in the received image increases, the reliability score of the pattern may be calculated higher. This is because the user terminal is highly likely to be gripped so that the pattern of interest to the user occupies a high area ratio in the image.

In step 820, the user terminal may calculate a reliability score of the second pattern. The reliability score of the second pattern may be calculated in the same manner as that of calculating the reliability score of the first pattern. That is, the reliability score of the second pattern will be calculated based on the result of calculating the distance between the center point of the second pattern and the center point of the received image or the ratio of the area occupied by the area corresponding to the second pattern in the received image. I can. In FIG. 8, the step 810 of calculating the reliability score of the first pattern is shown to precede the step 820 of calculating the reliability score of the second pattern, but is not limited thereto, and the reliability score of an arbitrary pattern is It can be calculated first, or the reliability score can be calculated at the same time.

Next, the user terminal may compare the calculated reliability score of the first pattern with the reliability score of the second pattern, and automatically process a pattern having a higher reliability score without a user input (830 ). For example, if the first pattern and the second pattern have the same priority and the first pattern is located closer to the center of the image than the second pattern, the user terminal has the reliability score of the first pattern and the reliability score of the second pattern. It is determined to be higher, and the processing of the first pattern can be automatically executed without user input. With this configuration, the pattern with the highest priority and highest reliability score is automatically processed without the user pressing the camera shooting button or selecting one of a plurality of patterns, providing a better user experience. can do.

9 is a flow chart illustrating a method 900 of determining a pattern type of a visually coded pattern according to an embodiment of the present disclosure. When a plurality of patterns are detected in an image while operating in, for example, a pattern scan mode, the user terminal may start determining a pattern type for each pattern (910 ). The user terminal may first determine whether a pattern is a pattern associated with a financial service (eg, LINE Pay service) of a specific company (920).

If the pattern is determined to be a pattern associated with a financial service of a specific company, the user terminal may determine that the pattern is the first pattern type (930). On the other hand, if the pattern is determined to be a pattern that is not related to the financial service of a specific company, the user terminal proceeds to step 940, and whether the pattern is related to the non-financial service (eg, LINE messenger service) It can be determined whether or not. Here, when the pattern is determined to be a pattern associated with a non-financial service of a specific company, the user terminal may determine that the pattern is a second pattern type (950). On the other hand, if the pattern is determined to be a pattern that is not related to a non-financial service of a specific company, the user terminal may proceed to step 960 to determine whether the pattern is a two-dimensional pattern.

If the pattern is determined to be a two-dimensional pattern, the user terminal may determine that the pattern is a third pattern type (970). On the other hand, when it is determined that the pattern is not a two-dimensional pattern (eg, a one-dimensional pattern), the user terminal may determine that the pattern is a fourth pattern type (980).

FIG. 10 is a diagram illustrating an example of displaying a notification for confirming whether to enter a pattern scan mode when a user terminal receives a plurality of visually coded patterns according to an embodiment of the present disclosure. In the image capturing mode, the user terminal may be configured to capture a photo by the user selecting the image capturing button 1016 through a touch input or the like while sequentially receiving a plurality of images through an image sensor. In addition, in the pattern scan mode, if a pattern is detected in the received image while sequentially receiving a plurality of images through the image sensor, the user terminal automatically executes the detected pattern without user input or previews the detected pattern. It can be configured to display on a display. Additionally, when a pattern is detected in an image received in the image capturing mode, the user terminal may check with the user whether to enter the pattern scan mode.

The first operation step 1010 represents that the user terminal receives an image including a plurality of patterns 1012 and 1014 through an image sensor in an image capturing mode. In this case, the user terminal may detect a plurality of patterns 1012 and 1014 in the received image. In the second operation step 1020, the user terminal detects a plurality of patterns 1012 and 1014 in the image received in the image capturing mode, and sends a notification 1022 of the user terminal to confirm whether to enter the pattern scan mode. Indicates what is displayed on the display. When the user selects the notification 1022 through a touch input or the like, the user terminal may enter the pattern scan mode and process a plurality of patterns 1012 and 1014.

11 is a diagram illustrating an example of displaying a preview of a pattern selected by a user on a display according to an embodiment of the present disclosure. 11 illustrates a first operation step 1110, a second operation step 1120, and a third operation step 1130 of the user terminal. The first operation step 1110 is when the user terminal receives an image including the first pattern 1112 and the second pattern 1114 visually coded through an image sensor while the user terminal is operating in the pattern scan mode. This indicates that the first pattern 1112 and the second pattern 1114 are detected, and the detected patterns 1112 and 1114 are displayed on the display as a selectable option. In this case, the area including the first pattern 1112 and the area including the second pattern 1114 may be highlighted in different colors on the display.

In the second operation step 1120, the user terminal receives the user's selection for the first pattern 1112 from the user, and displays a preview 1124 for the first pattern 1112 in a preview area on the display. Indicates that. When the user selects the first pattern 1112 through a touch input, the user terminal displays an indicator 1122 indicating that the first pattern 1112 is selected on the first pattern 1112, and The view 1124 may be displayed in the preview area of the display. As illustrated, the first pattern 1112 may be a pattern for adding “James” as a friend in a non-financial service (ie, LINE messenger service) of a specific company.

The preview 1124 for the first pattern may include a LINE messenger logo, information on a user added as a friend (eg, "James"), an add friend button 1126 and a share button 1128. When the user selects the friend add button 1126 through a touch input or the like, the user terminal may perform a friend add function in LINE messenger. When the user selects the share button 1128 through a touch input or the like, the user terminal allows the user to share the first pattern 1112 or the information coded by the first pattern 1112 with other users. In the second operation step 1120, the preview 1124 for the first pattern includes the LINE messenger logo, information on the user added as a friend ("James"), the add friend button 1126, and the share button 1128. Although shown to include, the present invention is not limited thereto, and other information may be further included.

The third operation step 1130 represents that the user terminal receives a user's selection for the second pattern 1114 from the user and displays a preview 1134 for the second pattern in a preview area on the display. When the user selects the second pattern 1114 through a touch input, the user terminal displays an indicator 1132 indicating that the second pattern 1114 is selected on the second pattern 1114, and The view 1134 may be displayed in the preview area of the display. As illustrated, the second pattern 1114 may be a pattern for coding URL information related to a service of a company other than a specific company.

The user terminal may transmit the corresponding pattern to an external device (eg, a web server) or transmit URL information coded according to the pattern to an external device in order to generate a preview of the second pattern 1114 encoding URL information. . In this case, the user terminal may receive at least one of a thumbnail, a title, or a description phrase associated with the corresponding URL from the external device. The preview 1134 for the second pattern may include a thumbnail, a title, a description, and a share button 1136 received from an external device.

When the user selects the preview 1134 for the second pattern through a touch input or the like, the user terminal may process URL information coded by the second pattern 1114 in a web browser application or the like. When the user selects the share button 1136 through a touch input or the like, the user terminal allows the user to share the URL information coded by the second pattern 1114 or the second pattern 1114 with other users. . In the third operation step 1130, the preview 1134 for the second pattern is illustrated as including a thumbnail, a title, a description phrase, and a share button 1136, but is not limited thereto, and may further include other information. I can.

In FIG. 11, the first pattern 1112 and the second pattern 1114 are shown to be displayed on the display, but the present invention is not limited thereto. When three or more patterns are included in the received image, three or more patterns are displayed. It may be displayed as a selectable option on the screen. In addition, in FIG. 11, when the user selects one of the selectable options (patterns) displayed on the display, a preview of the corresponding pattern is displayed, but is not limited thereto, and alternatively, priority of patterns in the image A preview of a pattern with a high priority may be automatically displayed without a user's input based on. In this case, when the priorities of the patterns in the image are the same, the reliability score of each pattern is calculated, and a preview of the pattern having a high reliability score may be automatically displayed without a user's input.

12 is a diagram illustrating an example of displaying a preview of a plurality of patterns included in an image on a display according to an embodiment of the present disclosure. 12 illustrates a first operation step 1210 and a second operation step 1220 of the user terminal. The first operation step 1210 is when the user terminal receives an image including the first pattern 1212 and the second pattern 1214 visually coded through an image sensor while the user terminal is operating in the pattern scan mode. Detects the first pattern 1212 and the second pattern 1214, and indicates that the area including the detected first pattern 1112 and the area including the second pattern 1114 are highlighted in different colors on the display. Show.

The second operation step 1220 includes a preview 1222 and a second pattern for the first pattern in response to the user terminal detecting the first pattern 1212 and the second pattern 1214 in the received image. Indicates that a preview 1228 for is automatically displayed in the preview area on the display. As shown in FIG. 12, the first pattern 1212 is a two-dimensional pattern for adding “James” as a friend in a non-financial service (ie, LINE messenger service) of a specific company, and the second pattern 1214 is a text May be a coded two-dimensional pattern. The preview 1222 for the first pattern may include a LINE messenger logo, information on a user added as a friend ("James"), an add friend button 1224, and a share button 1226. When the user selects the friend add button 1246 in the preview 1222 for the first pattern through a touch input or the like, the user terminal may perform a friend add function in the LINE messenger. When the user selects the share button 1224 in the preview 1222 for the first pattern through a touch input, the user terminal is coded by the first pattern 1212 or the first pattern 1212. Allows information to be shared with other users.

The preview 1228 for the second pattern may include text information coded by the pattern, a text copy button 1230 and a share button 1232. When the user selects the text copy button 1230 by a touch input or the like, text information coded by a pattern may be copied to a memory area of the user terminal. When the user selects the share button 1232 by touch input or the like, the user terminal allows the user to share text information coded by the second pattern 1214 or the second pattern 1214 with other users.

In FIG. 12, the preview 1222 for the first pattern and the preview 1228 for the second pattern are shown to be displayed on the display, but the present invention is not limited thereto, and three or more patterns are included in the received image. In this case, a preview of each of three or more patterns may be displayed on the display. In addition, in FIG. 12, it is shown that previews are displayed in the order of patterns in the image, but the present invention is not limited thereto, and alternatively, a preview of a pattern having a high priority based on the priority of patterns in the image is displayed in the preview area. You can also mark it at the top of. In this case, when the priorities of the patterns in the image are the same, the reliability score of each pattern is calculated, and a preview of the pattern having a high reliability score may be displayed at the top of the preview area.

13 is a diagram illustrating an example of displaying a preview of a pattern selected by a user on a display according to an embodiment of the present disclosure. 13 illustrates a first operation step 1310, a second operation step 1320, and a third operation step 1330 of the user terminal. The first operation step 1310 is when the user terminal receives an image including the first pattern 1312 and the second pattern 1314 visually coded through an image sensor while the user terminal is operating in the pattern scan mode. This indicates the detection of the first pattern 1312 and the second pattern 1314. In this case, the area including the first pattern 1312 and the area including the second pattern 1314 may be highlighted in different colors on the display.

The second operation step 1320 indicates that the user terminal displays the detected patterns 1312 and 1314 as selectable options 1324 and 1326 on the option area 1322 on the display. For example, the selectable options 1324 and 1326 may be icons created by resizing the detected patterns 1312 and 1314. All of the detected patterns may be displayed in the option area 1322.

When the user selects the first option 1324 through a touch input or the like, a preview 1328 for the first pattern 1312 associated with the first option 1324 may be displayed in the preview area of the display. . As illustrated, the first pattern 1312 may be a two-dimensional pattern for adding “James” as a friend in a non-financial service (ie, LINE messenger service) of a specific company. In this case, an unselected option (that is, the second option 1326) may be shaded to indicate that the first option 1324 has been selected. When the user selects the add friend button in the preview 1328 for the first pattern 1312 associated with the first option 1324 through a touch input or the like, the user terminal can perform the add friend function in LINE messenger. have.

In the third operation step 1330, the user selects the second option 1326 through a touch input, etc., and a preview 1332 for the second pattern 1314 associated with the second option 1326 is displayed in advance of the display. Indicates what is displayed in the viewing area. As shown, the second pattern 1314 may be a two-dimensional pattern for coding URL information related to other company services. In this case, the user terminal receives at least one of a thumbnail, title, or description phrase associated with the corresponding URL from the external device, and sends at least one of the thumbnail, title, or description phrase received from the external device to the second pattern 1314. As a preview 1332 for, it may be displayed in a preview area on the display. When the user selects the preview 1332 for the second pattern 1314, the user terminal may process URL information coded by the second pattern 1314 in a web browser application or the like.

14 is a flowchart illustrating a visually coded pattern recognition method 1400 according to an embodiment of the present disclosure. In step 1410, step 1400 may be initiated by the user terminal receiving an image including a first pattern and a second pattern. For example, while the user terminal is operating in a pattern scan mode for visually recognizing a coded pattern, an image including a plurality of patterns may be received.

Thereafter, the user terminal may detect a region including a pattern (first pattern and second pattern) that is initially coded in the image, and display the first pattern and the second pattern on the display as selectable options. (1420). In an exemplary embodiment, an area including the first pattern and an area including the second pattern are highlighted in different colors on the display and may be used as selectable options. In another embodiment, icons created by resizing the detected patterns may be displayed in an option area on the display as a selectable option.

In step 1430, the user terminal may receive a selection for the first pattern from the user. In this case, a preview of the selected first pattern may be displayed in the preview area on the display (1440 ). When receiving a selection for the preview of the first pattern from the user in step 1450, the user terminal may execute processing for the first pattern (1460).

14 shows that the first pattern and the second pattern are displayed as selectable options in an option area on the display, but are not limited thereto. When three or more patterns are included in the received image, three or more patterns This selectable option can be displayed in an option area on the display.

15 is a flowchart illustrating a visually coded pattern recognition method 1500 according to an embodiment of the present disclosure. In step 1510, step 1500 may be initiated by the user terminal receiving an image including the first pattern and the second pattern. For example, while the user terminal is operating in a pattern scan mode for visually recognizing a coded pattern, an image including a plurality of patterns may be received.

Thereafter, the user terminal may detect the first pattern and the second pattern from the received image, and display a preview of the first pattern and a preview of the second pattern in a preview area on the display (1520). . In an embodiment, the arrangement of the previews for the first pattern and the second pattern may be determined by comparing the priority of the pattern type of the first pattern and the pattern type of the second pattern. Alternatively or additionally, the placement of the preview for the first pattern and the second pattern may be determined by comparing the reliability score of the first pattern and the reliability score of the second pattern.

The user terminal may receive a selection for a preview of the first pattern displayed on the display from the user (1530). In this case, the user terminal may execute processing on the selected first pattern (1540). In FIG. 15, a preview of the first pattern and the second pattern is shown to be displayed in the preview area on the display, but the present invention is not limited thereto. When three or more patterns are included in the received image, three or more A preview of the pattern may be displayed in a preview area on the display.

The visually coded pattern recognition method described above may be implemented as a computer-readable code (eg, a computer program) on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tapes, floppy disks, and optical data storage devices. Further, the computer-readable recording medium is distributed over a computer system connected through a network, so that computer-readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the above-described embodiments can be easily inferred by programmers in the technical field to which the present invention pertains.

The method, operation, or techniques of this disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will understand that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented in electronic hardware, computer software, or combinations of both. To clearly illustrate this interchange of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design requirements imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementations should not be interpreted as causing a departure from the scope of the present disclosure.

In a hardware implementation, the processing units used to perform the techniques include one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs). ), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described in this disclosure. , Computer, or a combination thereof.

Accordingly, various exemplary logic blocks, modules, and circuits described in connection with the present disclosure may include general purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or It may be implemented or performed in any combination of those designed to perform the functions described in. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, eg, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in connection with the DSP core, or any other configuration.

In firmware and/or software implementation, the techniques include random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), PROM ( on a computer-readable medium such as programmable read-only memory), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage device, etc. It may be implemented as stored instructions (eg, a computer program). The instructions may be executable by one or more processors, and may cause the processor(s) to perform certain aspects of the functionality described in this disclosure.

When implemented in software, the techniques may be stored on a computer readable medium as one or more instructions or codes or transmitted through a computer readable medium. Computer-readable media includes both computer storage media and communication media, including any medium that facilitates transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a computer. By way of non-limiting example, such computer readable medium may contain RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or desired program code in the form of instructions or data structures. It may include any other media that may be used for transfer or storage to and accessible by a computer. Also, any connection is properly termed a computer-readable medium.

For example, if the software is transmitted from a website, server, or other remote source using wireless technologies such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or infrared, wireless, and microwave, coaxial cable , Fiber optic cable, twisted pair, digital subscriber line, or wireless technologies such as infrared, wireless, and microwave are included within the definition of the medium. Disks and discs as used herein include CDs, laser disks, optical disks, digital versatile discs (DVDs), floppy disks, and Blu-ray disks, where disks are usually magnetic It reproduces data optically, whereas discs reproduce data optically using a laser. Combinations of the above should also be included within the scope of computer-readable media.

The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other known type of storage medium. An exemplary storage medium may be coupled to a processor such that the processor can read information from or write information to the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and storage medium may also reside within the ASIC. The ASIC may exist in the user terminal. Alternatively, the processor and storage medium may exist as separate components in the user terminal.

Although the above-described embodiments have been described as utilizing aspects of the presently disclosed subject matter in one or more standalone computer systems, the present disclosure is not limited thereto, and may be implemented in connection with any computing environment such as a network or a distributed computing environment. . Furthermore, aspects of the subject matter in this disclosure may be implemented in multiple processing chips or devices, and storage may be similarly affected across multiple devices. Such devices may include PCs, network servers, and portable devices.

In the present specification, the present disclosure has been described with reference to some embodiments, but various modifications and changes may be made without departing from the scope of the present disclosure that can be understood by those of ordinary skill in the art to which the present disclosure belongs. In addition, such modifications and changes should be considered to fall within the scope of the claims appended to this specification.

110: user 120: user terminal
130: objects 132, 134, 136: visually coded patterns
210, 220, 230: user terminal
240: network 250: server 300: user terminal
310: image sensor 320: display
330: input interface 340: communication module
350: processor 352: pattern area detection unit
354: pattern type determination unit 356: pattern preview generation unit
358: pattern processing unit 360: storage unit

Claims (20)

As a pattern recognition method operating in a user terminal,
Receiving an image comprising a visually coded first pattern and a visually coded second pattern;
Determining a pattern type of the visually coded first pattern;
Determining a pattern type of the visually coded second pattern;
Comparing the priority of the visually coded pattern type of the first pattern and the visually coded pattern type of the second pattern; And
When it is determined that the visually coded pattern type of the first pattern has a higher priority than the visually coded pattern type of the second pattern, automatically executing the processing of the visually coded first pattern
Containing, pattern recognition method.
The method of claim 1,
Determining the pattern type of the visually coded first pattern,
Determining whether the visually coded first pattern is a pattern associated with a specific service; And
When it is determined that the visually coded first pattern is a pattern associated with a specific service, determining a pattern type of the visually coded first pattern as a first pattern type
Containing, pattern recognition method.
The method of claim 2,
Determining the pattern type of the visually coded first pattern,
Determining whether the visually coded first pattern is a pattern associated with a service of a specific company; And
When it is determined that the visually coded first pattern is a pattern associated with a service of a specific company, determining a pattern type of the visually coded first pattern as a second pattern type
Further comprising a, pattern recognition method.
The method of claim 3,
Determining the pattern type of the visually coded first pattern,
When it is determined that the visually coded first pattern is not related to the service of a specific company,
If the visually coded first pattern is a two-dimensional pattern, determining a pattern type of the visually coded first pattern as a third pattern type; And
When the visually coded first pattern is a one-dimensional pattern, recognizing the visually coded pattern type of the first pattern as a fourth pattern type
Further comprising a, pattern recognition method.
The method of claim 4,
The first pattern type, the second pattern type, the third pattern type, and the fourth pattern type having a high priority in the order of the pattern recognition method.
The method of claim 1,
When it is determined that the pattern type of the visually coded first pattern has the same priority as the pattern type of the visually coded second pattern,
Calculating a reliability score of the visually coded first pattern based on at least one of a position or size of the visually coded first pattern in the received image;
Calculating a reliability score of the visually coded second pattern based on at least one of a location or size of the visually coded second pattern in the received image; And
When it is determined that the reliability score of the visually coded first pattern is higher than the reliability score of the visually coded second pattern, a preview of the visually coded first pattern is displayed on the display of the user terminal. Steps to mark
Further comprising a, pattern recognition method.
The method of claim 6,
Calculating a reliability score of the visually coded first pattern based on at least one of a position or size of the visually coded first pattern in the received image,
And calculating a distance between a central point of the received image and a central point of the visually coded first pattern.
The method of claim 6,
Calculating a reliability score of the visually coded first pattern based on at least one of a position or size of the visually coded first pattern in the received image,
And calculating an area ratio occupied by the area corresponding to the visually coded first pattern in the received image.
The method of claim 1,
When it is determined that the pattern type of the visually coded first pattern has the same priority as the pattern type of the visually coded second pattern,
Calculating a reliability score of the visually coded first pattern based on at least one of a position or size of the visually coded first pattern in the received image;
Calculating a reliability score of the visually coded second pattern based on at least one of a location or size of the visually coded second pattern in the received image; And
When it is determined that the reliability score of the visually coded first pattern is higher than the reliability score of the visually coded second pattern, automatically processing the visually coded first pattern
Further comprising a, pattern recognition method.
The method of claim 1,
Displaying an indicator on the first area corresponding to the visually coded first pattern to notify the user that the visually coded first pattern is automatically processed
Further comprising a, pattern recognition method.
The method of claim 1,
Detecting a first area including the visually coded first pattern;
Detecting a second area including the visually coded second pattern; And
Highlighting the first area and the second area in different colors
Further comprising a, pattern recognition method.
The method of claim 1,
Sequentially receiving a plurality of images in an image capturing mode; And
When receiving the image including the visually coded first pattern and the visually coded second pattern, displaying a notification for confirming whether to enter a pattern scan mode on the display of the user terminal
Further comprising a, pattern recognition method.
A computer program stored in a computer-readable recording medium to execute the pattern recognition method according to any one of claims 1 to 12 on a computer.
As a pattern recognition device,
An image sensor configured to receive an image comprising a visually coded first pattern and a visually coded second pattern;
A pattern type determination unit configured to determine a pattern type of the visually coded first pattern and a pattern type of the visually coded second pattern; And
A pattern processing unit configured to compare the priority of the visually coded pattern type of the first pattern and the visually coded pattern type of the second pattern
Including,
When it is determined that the pattern type of the visually coded first pattern has a higher priority than the pattern type of the visually coded second pattern, the pattern processing unit performs processing of the visually coded first pattern. A pattern recognition device, further configured to run automatically.
As a pattern recognition method operating in a user terminal,
Receiving an image comprising a visually coded first pattern and a visually coded second pattern;
Generating a preview for the visually coded first pattern;
Generating a preview for the visually coded second pattern;
Displaying a preview of the first pattern and a preview of the second pattern in a preview area on a display of the user terminal;
Receiving a selection for a preview of the first pattern from a user; And
Performing processing on the visually coded first pattern in response to the selection
Containing, pattern recognition method.
The method of claim 15,
Displaying the preview of the first pattern and the preview of the second pattern in a preview area on the display of the user terminal,
Calculating a reliability score of the visually coded first pattern based on at least one of a position or size of the visually coded first pattern in the received image;
Calculating a reliability score of the visually coded second pattern based on at least one of a location or size of the visually coded second pattern in the received image; And
When it is determined that the reliability score of the visually coded first pattern is higher than the reliability score of the visually coded second pattern, a preview of the visually coded first pattern is previewed on the display of the user terminal. Steps to display at the top of the viewing area
Containing, pattern recognition method.
The method of claim 15,
Detecting a first area including the visually coded first pattern;
Detecting a second area including the visually coded second pattern; And
Highlighting the first area and the second area in different colors
Further comprising a, pattern recognition method.
The method of claim 15,
Generating a preview for the visually coded first pattern,
If the visually coded first pattern corresponds to encoding of a URL (Uniform Resource Locator), transmitting URL information coded by the visually coded first pattern to an external device;
Receiving at least one of a thumbnail, a title, and a description phrase associated with the URL information from the external device; And
Displaying at least one of the received thumbnail, title, and description in a preview area on the display;
Further comprising a, pattern recognition method.
A computer program stored in a computer-readable recording medium to execute the pattern recognition method according to any one of claims 15 to 18 on a computer.
As a pattern recognition device,
An input interface configured to receive input from a user;
An image sensor configured to receive an image comprising a visually coded first pattern and a visually coded second pattern;
A pattern preview generator configured to generate a preview for the visually coded first pattern and a preview for the visually coded second pattern;
A display configured to output a preview of the generated visually coded first pattern and second pattern to a preview area; And
A pattern processing unit configured to execute processing on the visually coded first pattern in response to receiving a selection for the preview of the first pattern through the input interface
Containing, pattern recognition device.

Images