KR20200078880A - System for guiding store information using technology of image recognize - Google Patents

Abstract

The present invention relates to a system for providing store information using an image recognition technology and a method thereof. The system for providing store information using image recognition technology comprises: an extraction unit which extracts a sign image from a photographed image; a search unit which indexes for a sign image corresponding to the extracted sign image; and a guide unit which outputs the store information matching the indexed sign image. The search unit includes: a data extraction module which extracts a data value from the sign image; an index module which indexes the data equal to the data value extracted from a storage means; and a search module which indexes the stored contents, determines whether there are data equal to the extracted data value or not, and extracts the data equal to the extracted data value as a result of the determination. According to the present invention, the system is able to geometricize the sign image itself of a store, manage the same in a database, extract a sign image corresponding to the image photographed through a smart device, provide the store information matched with the extracted sign image, binarize the image, prevent the wrong recognition in accordance with letter extraction and translation in advance, and enable a user to rapidly check the information of a sale in the store even without entering the store.

Description

System for guiding store information using image recognition technology and its method {System for guiding store information using technology of image recognize}

The present invention relates to a store information guide system and its method using image recognition technology, and more specifically, to store the signboard image itself of the store and manage it in a database, but to display the signboard image corresponding to the image taken through the smart device. It relates to a technology for extracting, and providing store information matched with the extracted signboard image.

Recently, as smartphones have recently become an issue, various types of applications have appeared. Like most Apple's most commonly used iPhones or Samsung's smartphones, most of them have no keyboard and have touch functions to minimize user input.

Accordingly, the conventional search system, in which the search system provided information corresponding to the inputted text, has been transformed into a search system that provides information corresponding to an image captured through a camera of a smartphone.

In Korean Patent Publication No. 10-2012-0070795 (signboard image character recognition and post-processing system), it not only improves the recognition speed by lowering the recognition classification for Hangul through a database of registered names used and registered nationwide, but also various A technique for solving a form recognition error through post-processing is disclosed.

However, this prior art has a hassle that a separate binarization operation for extraction of characters must be preceded.

If the arrangement of the characters is constant in the forward direction, it is easy to binarize, but as the design of the signboard has recently diversified, the character array is often not in the forward direction, and there are also many signboards that image the characters themselves. There is a frequently occurring problem.

In addition, even after extracting characters through binarization, it is highly likely to be misrecognized because it is difficult to interpret them as accurate characters when the fonts of the characters are unique.

In addition, since various languages such as English, Japanese, and Hanja, as well as Korean, are used recently, an accurate and extensive translation function is essential. For this reason, it is very difficult to implement the prior art.

Republic of Korea Patent Publication No. 10-2010-0132253 (2012.07.02. published)

The object of the present invention is to image the signboard image of the store itself and manage it in the database, but extract the signboard image from the image taken through the smart device and provide the store information that is already matched with the extracted signboard image, thereby binarizing the image To prevent misrecognition due to text extraction and translation, and to quickly check the information sold in stores without entering the store.

The object of the present invention is to provide the information of the corresponding store as a live-action image at the same time as the image of the store sign is taken, so that even foreigners who cannot read the sign can check the store information matched with the sign image without a separate search procedure. And it is intended to quickly grasp the latter.

An embodiment of the present invention for achieving such a technical problem is a store information guide system using image recognition technology, comprising: an extraction unit for extracting a signboard image from a captured image; A search unit indexing the extracted signboard image and the corresponding signboard image; And a guide unit for outputting store information matched with the indexed signboard image, wherein the search unit comprises: a data extraction module for extracting data values from the signboard image; An index module that indexes data matching data values extracted from the repository; And a search module for indexing the storage to determine whether data matching the extracted data value exists, and extracting data matching the extracted data value as a result of the determination.

Preferably, the extraction unit controls the camera provided in the smart device to generate a shooting image that matches the input signal; And a signboard extraction module for extracting a signboard image included in the photographed image through outline extraction, wherein the signage extraction module extracts an image located within a preset outline box displayed on top of the output photographed image as a signboard image. Should be

The guide unit includes: a shop guide module for outputting shop information matching data authorized by the search unit; And it characterized in that it comprises an inquiry module for outputting the sub-information associated with the sub-category of store information to correspond to the input inquiry signal.

It characterized in that it further comprises a recommendation unit for sorting and outputting shop information adjacent to the current location based on the real-time location-based information received from the GPS.

The recommendation unit is characterized in that it outputs the adjacent shop information arranged in a distance order from a location corresponding to the input map search signal.

A store information guiding method using an image recognition technology according to an embodiment of the present invention based on the above-described system includes: (a) extracting a signboard image from an image captured by the extraction unit; (B) indexing a signboard image corresponding to the signboard image extracted by the search unit; And (c) outputting shop information matching the signboard image indexed by the guide portion, wherein step (c) comprises: (c-1) the guide portion outputting authorized shop information from the search portion; In the process (c-2) of determining whether the additional inquiry signal is input, and the determination result of the process (c-2), if an inquiry signal is input, the sub connected to the sub-category of store information so that the guide unit corresponds to the inquiry signal. And (c-3) outputting the information.

Step (b) comprises: (b-1) in which the search unit extracts the data value from the signboard image authorized by the extraction unit; (B-2) determining whether there is data matching the extracted data value by indexing the repository; (b-3) when there is data matching the extracted data value as a result of the determination in step (b-2), applying (b-3) the store information, which is the matching data, to the guide unit; And (b-4) outputting text or an image that the search unit does not have a matching data value when there is no data matching the extracted data value as a result of the determination in step (b-2). do.

After the step (b), (d) determining whether the recommendation unit receives any one of a search signal, a map search signal, or an industry search signal from the user; (d) when the inquiry signal is received as a result of the determination in step (d), the recommendation unit sorts and outputs shop information adjacent to the current location in order of distance based on real-time location-based information received from GPS; When the map search signal is input as a result of the determination in step (d), step (f) of arranging and displaying adjacent store information in a distance order from a location corresponding to the map search signal input by the recommendation unit; And (g) when the industry search signal is input as a result of the determination in step (d), sorting the store information of the industry corresponding to the industry search signal input by the recommendation unit in the order of distance and outputting it.

According to the present invention as described above, the signboard image itself of a store is graphically managed and managed in a database, but by extracting a signboard image corresponding to an image taken through a smart device, and providing store information matched with the extracted signboard image , Through image binarization, misrecognition due to character extraction and translation can be prevented in advance, and information sold in the store can be quickly checked without entering the store.

According to the present invention, by photographing a signboard image of a store and providing information of the corresponding store as a live-action image, even foreigners who cannot read the signage can check the store information matched with the signage image without a separate search procedure, and perform a live-action and You can quickly identify reviews.

1 is a block diagram showing a store information guide system using image recognition technology according to an embodiment of the present invention.
2 is a block diagram showing an extraction unit of a store information guide system using image recognition technology according to an embodiment of the present invention.
3 is a block diagram showing a search unit of a store information guide system using image recognition technology according to an embodiment of the present invention.
4 is a block diagram showing a guide portion of a store information guide system using image recognition technology according to an embodiment of the present invention.
5 is a block diagram showing a recommendation unit of a store information guide system using image recognition technology according to an embodiment of the present invention.
6 is a flowchart illustrating a method for guiding store information using image recognition technology according to an embodiment of the present invention.
7 is a flowchart illustrating a detailed process of step S100 of a method for guiding store information using image recognition technology according to an embodiment of the present invention.
8 is a flowchart illustrating a detailed process of step S200 of a store information guide method using image recognition technology according to an embodiment of the present invention.
9 is a flowchart illustrating a detailed process of step S300 of the store information guide method using the image recognition technology according to an embodiment of the present invention.
10 is a flowchart illustrating a process after step S208 of a store information guiding method using an image recognition technology according to an embodiment of the present invention.

Specific features and advantages of the present invention will become more apparent from the following detailed description based on the accompanying drawings. Prior to this, the terms or words used in the specification and claims are based on the principle that the inventor can appropriately define the concept of terms in order to best describe his or her invention in the technical spirit of the present invention. It should be interpreted as a matching meaning and concept. In addition, it should be noted that when it is determined that the detailed description of the known functions and configurations related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description is omitted.

Hereinafter, the detailed reference will be omitted, but the store information guide system using the image recognition technology according to an embodiment of the present invention is driven by an app (app) mounted on a smart device, and the smart device is a smart phone or a tablet PC. And a portable communication device including a notebook.

Referring to Figure 1, the store information guide system (S) using the image recognition technology according to an embodiment of the present invention, the extraction unit 100 for extracting the signage image from the captured image, and the signage corresponding to the extracted signage image It comprises a search unit 200 for indexing the image, and a guide unit 300 for outputting store information matched with the indexed signage image.

Referring to FIG. 2, the extracting unit 100 controls the camera provided in the smart device to generate a photographing image that matches the input signal, and a signboard image included in the photographed image through outline extraction. It comprises a sign extraction module 104 to extract.

At this time, the signboard extraction module 104 may extract an image located in a preset outline box displayed on the output photographed image as a signboard image.

In addition, the signboard extraction module 104 may extract an image located in a predetermined range from the center of the output photographed image as a signboard image.

Referring to FIG. 3, the search unit 200 includes a data extraction module 202 that extracts data values from the signboard image authorized by the extraction unit 100, and an index module that indexes data that matches the data values extracted from the storage. 204, and a search module 206 for determining whether there is data matching the extracted data value by indexing the storage, and extracting data matching the extracted data value as a result of the determination.

At this time, the data value may include graphic data or color data, and the storage is preferably understood as any one of a database of a server connected through a memory or an information communication network built in a smart device.

In addition, the search module 206 of the search unit 200 is configured to output a text or image indicating that there is no matching data value when there is no data matching the extracted data value as a result of the determination.

Referring to FIG. 4, the guide unit 300 includes a store guide module 302 that outputs store information matched with the authorized data from the search unit 200, that is, store information matched with a signboard image, and an input inquiry It comprises a query module 304 for outputting the sub-information associated with the sub-category of store information to correspond to the signal.

At this time, the sub-information can be changed in multiple languages, and may include any one of images, prices, origins, directions or contact information for products or menus sold in the store, and updated by the manager of each store Can.

Referring to FIG. 5, the store information guiding system S using an image recognition technology according to an embodiment of the present invention guides store information located within a preset range based on real-time location-based information received in conjunction with GPS. It is configured to further include a recommendation unit 400.

When there is no data in the storage that matches the extracted data value as a result of the determination of the search unit 200, the recommendation unit 400 sorts and stores adjacent shop information from the current location in the order of distance based on real-time location-based information received from GPS. do.

At this time, when the inquiry signal is input from the user after the store information is output, the recommendation unit 400 is configured to output the sub information associated with the sub-category of the corresponding store information under the control of the guide unit 300.

Also, when a map search signal is input from a user, the recommendation unit 400 sorts and outputs adjacent store information from a location corresponding to the input map search signal in the order of distance.

In addition, when the industry search signal is input from the user, the recommendation unit 400 sorts and outputs the store information of the industry corresponding to the industry search signal input in the order of distance.

Hereinafter, a method of guiding store information using an image recognition technology according to an embodiment of the present invention will be described with reference to FIG. 6 as follows.

First, the extraction unit 100 extracts the signboard image from the captured image (S100).

Subsequently, the search unit 200 indexes the extracted signboard image and the corresponding signboard image (S200).

Then, the store information matching the signboard image indexed by the guide unit 300 is output (S300 ).

Hereinafter, referring to FIG. 7, a detailed process of step S100 of a store information guide method using an image recognition technology according to an embodiment of the present invention will be described.

First, the extraction unit 100 controls the camera provided in the smart device to generate a captured image that matches the input signal (S102).

Then, the extraction unit 100 extracts the signboard image included in the captured image through the outline extraction (S104).

Hereinafter, referring to FIG. 8, a detailed process of step S200 of the store information guide method using the image recognition technology according to an embodiment of the present invention will be described.

After the step S100, the search unit 200 extracts the data value from the signboard image authorized by the extraction unit 100 (S202).

Subsequently, the search unit 200 indexes the storage to determine whether data matching the extracted data value exists (S204).

As a result of the determination in step S204, when there is data matching the extracted data value, the search unit 200 applies the store information, which is the matching data, to the guide unit 300 (S206).

As a result of the determination in step S204, when there is no data matching the extracted data value, the search unit 200 outputs a text or image indicating that there is no matching data value (S208).

Hereinafter, referring to FIG. 9, a detailed process of step S300 of the store information guiding method using the image recognition technology according to an embodiment of the present invention will be described.

After the step S200, the guide unit 300 outputs the authorized store information from the search unit 200 (S302).

Subsequently, it is determined whether the guide unit 300 receives the inquiry signal (S304).

As a result of the determination in step S304, when an inquiry signal is input, the guide unit 300 outputs sub information associated with a sub-category of store information to correspond to the inquiry signal (S306).

Hereinafter, a process after step S208 of the method for guiding store information using the image recognition technology according to an embodiment of the present invention will be described with reference to FIG. 10 as follows.

After the step S208, the recommender 400 determines whether any one of the search signal, the map search signal, or the industry search signal is input from the user (S400).

As a result of the determination in step S400, when an inquiry signal is input, the recommendation unit 400 sorts and outputs adjacent store information from the current location in the order of distance based on the real-time location-based information received from the GPS (S500).

As a result of the determination in step S400, when a map search signal is input, the recommender 400 sorts and outputs adjacent store information from a location corresponding to the input map search signal in the order of distance (S600).

As a result of the determination in step S400, when the industry search signal is input, the recommender 400 sorts and outputs the store information of the industry corresponding to the industry search signal input in order of distance (S700).

According to the present invention as described above, the signboard image of the store is figured out and managed in the database, but the signboard image corresponding to the image taken through the smart device is extracted, and the extracted signboard image and store information matched with the extracted signboard image are provided. By doing so, it is possible to prevent misrecognition due to character extraction and translation through binarization of the image, and quickly check information sold in the store without entering the store.

As described above, although it has been described and illustrated in connection with a preferred embodiment for illustrating the technical idea of the present invention, the present invention is not limited to the configuration and operation as illustrated and described, and deviates from the scope of the technical idea. It will be understood by those skilled in the art that many changes and modifications to the present invention are possible without. Accordingly, all such suitable modifications and modifications and equivalents should be considered as falling within the scope of the present invention.

S: Store information guidance system using image recognition technology
100: extraction unit
102: shooting module
104: signboard extraction module
200: search unit
202: data extraction module
204: index module
206: search module
300: guide
302: store information module
304: inquiry module
400: recommendation part

Claims (8)

An extraction unit for extracting a signboard image from the captured image;
A search unit indexing the extracted signboard image and the corresponding signboard image; And
Includes a guide for outputting store information matching the indexed signage image,
The search unit,
A data extraction module that extracts data values from the signboard image;
An index module that indexes data matching data values extracted from the repository; And
A search module for determining whether data matching the extracted data value exists by indexing the storage, and extracting data matching the extracted data value as a result of the determination.
Store information guidance system using image recognition technology, characterized in that it comprises. According to claim 1,
The extraction unit,
A photographing module that controls a camera provided in a smart device to generate a photographed image corresponding to an input signal; And
Includes a signage extraction module that extracts the signage image included in the captured image through outline extraction,
The signboard extraction module,
A store information guide system using image recognition technology, characterized in that an image located in a preset outline box displayed on the output image is extracted as a sign image. According to claim 1,
The guide portion,
A store guide module for outputting store information matched with the authorized data from the search unit; And
Inquiry module that outputs sub-information connected with sub-category of store information to correspond to the input inquiry signal
Store information guidance system using image recognition technology, characterized in that it comprises. According to claim 1,
Based on the real-time location-based information received from the GPS, a recommendation unit that sorts and displays adjacent shop information from the current location in order of distance
Store information guidance system using image recognition technology, characterized in that further comprising. According to claim 4,
The recommendation unit,
A store information guide system using image recognition technology, characterized in that the store information is arranged in the order of distance and output from the location corresponding to the input map search signal. (A) extracting a signboard image from the image taken by the extraction unit;
(b) indexing the signboard image corresponding to the signboard image extracted by the search unit; And
(c) outputting shop information matching the signboard image indexed by the guide,
Step (c) is,
The (c-1) process of the guide unit outputting the store information authorized by the search unit,
(C-2) determining whether the guide unit receives the inquiry signal, and
When the inquiry signal is input as a result of the determination in the process (c-2), the guide unit outputs (c-3) a process of outputting sub-information associated with the sub-category of the store information to correspond to the inquiry signal. A method for guiding store information using image recognition technology. The method of claim 6,
Step (b) is,
(b-1) the search unit extracts the data value from the signboard image authorized by the extraction unit;
(b-2) a step in which the search unit indexes the storage and determines whether data matching the extracted data value exists;
(b-3) when there is data matching the extracted data value as a result of the determination in step (b-2), applying store information, which is data matching the search unit, to the guide unit; And
(b-4) As a result of the determination in step (b-2), if there is no data matching the extracted data value, the search unit outputs a text or image indicating that there is no matching data value.
Store information guidance method using image recognition technology, characterized in that it comprises. The method of claim 6,
After step (b),
(d) determining whether the recommendation unit receives one of a search signal, a map search signal, or an industry search signal from the user;
(e) when the inquiry result is input as a result of the determination in step (d), the recommendation unit sorts and outputs shop information adjacent to the current location in the order of distance based on the real-time location-based information received from the GPS;
(f) as a result of the determination in step (d), when receiving a map search signal, sorting and displaying adjacent store information in a distance order from a location corresponding to the map search signal input by the recommender; And
(g) When the business search signal is received as a result of the determination in step (d), the step of sorting and displaying the store information of the business type corresponding to the business search signal input by the recommendation unit in the order of distance is displayed.
Store information guidance method using image recognition technology, characterized in that it comprises.

Images