KR20190093755A - Big data-based image text recognition and customized foodstuff recommendation methods and system - Google Patents

Abstract

The present invention relates to a method for recommending a product by analyzing the use pattern of a user and a product recommendation system thereof and, more specifically, to a method for recommending a product and a product recommendation system thereof to recommend products for each individual person through the analysis of big data collected based on the use information of user media by being formed to collect the use information of a user through various media, analyze the collected information as big data, and recommend optimal products in real time. The product recommendation method for each individual person through the analysis of big data collected based on the use information of user media and the product recommendation system thereof composes a matching system with a client which is any one medium among a smartphone, a smart TV, a desktop, a laptop, and a portable telecommunication device such as PDA and a server connecting the client to a wired or wireless network as a plurality of subscribers so as to capture and transmit the current view screen of a user in a client side to a matching system, and recommend matching products by analyzing the interest of a user with text words through one path of extracting text data from html sources included in the transmitted data and another path of recognizing an image as text from an image, from which html sources are excluded, recognizing an object included in an image, or recognizing the face of a person included in an image.

Description

Big data-based image text recognition and customized foodstuff recommendation methods and system}

The present invention relates to a method for recommending a product by analyzing a user's usage pattern and a recommendation system thereof. More particularly, the present invention collects user's usage information through various media and analyzes the collected information as big data. The present invention relates to a product recommendation method and a product recommendation system for each individual through big data analysis collected based on usage information of a user medium configured to recommend in real time.

In modern times, various information communication devices have been developed to form an infrastructure that can search and use various information anytime and anywhere.

Various trading behaviors derived from such information retrieval are growing as one axis of business, and these transactions may be referred to as electronic commerce.

E-commerce has existed in various forms, such as exchanging documents between companies electronically or even home shopping and home banking of PC communication, even before the Internet became popular. However, as the Internet becomes popular, e-commerce is thought to be related to transactions on the Internet. It became.

In a narrow sense, electronic commerce means that a user finds a specific item of interest through a web search and trades a product in real time through a store opened as a homepage on the Internet.

Products traded include not only real goods such as electronic components, but also services such as distance education, medical diagnostics and real estate transactions.

These products also include digital products such as news, audio, and software, and their proportion in electronic commerce is increasing.

In a broader sense, e-commerce includes not only business transactions with consumers, but also related activities with all institutions involved in the transaction, such as suppliers, financial institutions, government agencies, and transportation agencies.

At this time, in the case of e-commerce using a web browser, most e-commerce web servers require the user to use cookies.

Cookies are small files (less than 4kB) that contain information that mediates websites and netizens' computers.

In the shopping mall, you can put your choices in a cookie like a shopping cart, which means you can browse through the contents of this cookie again, even if you are disconnected while shopping. That means you don't have to.

However, such a shopping cart function can be used only for each website, and product information cannot be obtained from other shopping malls or price comparison sites using the same product identifier, and the user cannot obtain the same product identifier. Has the inconvenience of having to visit another website directly. In addition, such a shopping cart function has a problem that can only be used limited to the shopping cart function provided by the website containing the product.

In the case of Korean Patent Registration No. 882716, the information recommendation agent installed in the user terminal recognizes the product selected by the user terminal without restriction of the web site and solves the inconvenience of the limited cookie function. Recognizing at least one product selected through the user terminal to search for and analyze product information on the product in another web site using not only the web site but also the identifier of the same product and information input through the user terminal And receiving search data or condition data of the product through the user terminal, storing and maintaining the product in association with the product, retrieving product information of the product using the search data, and the condition. That corresponds to the data Group and a method for referring Products made known to the product information comprises the step of displaying through the user terminal. However, this method of recommending product information requires the user to select and input the product information through search, searching the web site through the network and searching for the product information in the connected site and continuously searching for the correct information. It can only function when the operation and management to boil the product information to achieve a more effective product search than the cookie method described above, but the inconvenience of forcing the conscious user's actions still remain.

That is, there is a problem in that it is limited to an internet web environment or a mobile app environment and at the same time, it cannot be applied to other devices used by users who are not familiar with the network environment.

According to another conventional technology, when collecting html data of an e-commerce client device, analyzing and selecting a preference product, and providing recommended product information to the client device through a web browser, the user of the client side loses interest or has already Even though the product has been purchased, there is a problem of giving rejection such as unnecessary spammy advertisements by providing and showing the product as an interest product.

The prior art and other known prior arts have a problem that they do not accurately reflect client's interests and issues by recommending products based on fragmentary data such as html data, search keyword, click view state, and the like.

The present invention has been made to solve the problem of the product recommendation of the prior art to solve the following problems.

The present invention derives the client's interest based on data extracted from OCR, image recognition technology, face recognition technology, etc. for product recommendation, and matches the product that matches the derived interest, and the client is interested in the present time. It is to provide a product recommendation method and a product recommendation system for each individual through big data analysis collected based on the usage information of a user medium configured to automatically recommend a product that is present.

The present invention provides an individual product recommendation method and a product recommendation system for each individual through big data analysis based on usage information of a user medium configured to include an advertisement in a recommendation product to satisfy both an advertiser and a client.

The present invention can be applied to a terminal that is not easy to track usage patterns by using html-based data such as an application of an IPTV smartphone or a mobile communication terminal in addition to html-based data such as OCR, image recognition, and face recognition. It is to provide a product recommendation method and a product recommendation system for each individual through big data analysis collected on the basis of the usage information.

The present invention recommends a product for sale through the collected big data, or collects on the basis of the use information of the user medium that can be used as a push-type personalized advertising or various analysis data by the user's interest analysis, etc. It is to provide individual product recommendation method and product recommendation system through big data analysis.

The present invention derives interest in all media by communication used by the client to make big data, and analyzes the client's propensity through the big data, thereby collecting the big data collected based on the usage information of the user medium that enables accurate product recommendation. It is to provide individual product recommendation method and product recommendation system through data analysis.

The present invention made to solve the above problems is made by the following means.

Matching a client consisting of a medium such as a smart phone, a smart TV, a desktop computer, a laptop computer, and a portable information communication device (PDA, etc.) with a server connecting the wired / wireless network using the client as a plurality of subscribers. The system captures the current viewing screen of the client-side user and sends it to the matching system, and recognizes the image as text from a path that extracts the text data from the html source included in the transmitted data, and the image from which the html source is excluded. Or another path for recognizing an object included in an image or a face of a person included in the image, and analyzing the user's interest as a text word to recommend a matching product.

Extracting HTML data from the screen viewed by the user on the client side, and transmitting the screen data captured by the image captured by the user to a matching system (server);

A viewing screen data recognition step of extracting html text data from which the source extraction is possible from the viewing screen data transmitted from the client;

Optical character recognition (OCR) recognition for extracting from the captured image that is not source extractable as text for analysis, image recognition for extracting the image as text to analyze objects contained in the image, and This includes image recognition for extracting the image as text to analyze the embedded face.

An image recognition step of recognizing data by extracting text in an unknown recognition step;

Data including a collection process of collecting the recognized text data from the html data and the image, an analysis process of analyzing the collected data to give meaning to each data, and a classification process of classifying the collected and analyzed data by field An analysis step;

A big data storage step of storing the analyzed data in a DB as various analysis data such as individual, medium, storage date, time, interest, gender, age, etc .;

An information matching step of deriving a product to be recommended to a client through a product DB which records products having various meta information;

A product recommendation step of transmitting the matched recommendation product information to the client side is performed by the recommendation product display step of displaying the recommended product on the client-side device.

The data analysis step may be configured to block the analysis of data that is unnecessary or contrary to public morals, further including an extraction exclusion setting process of specifying text and an image or a URL to be excluded from the analysis and excluding it from the analysis.

The big data storage step further includes a matching keyword input process of inputting additional information (significance, mood, era background, etc.) matching the extracted data to match the unstructured information (concept, mood, period background, etc.). Can be done.

The period of transmitting the client-side viewing screen data to the matching system is not specific but preferably repeated every predetermined period.

The matching keyword input process may be performed to be added by the server-side manager as a keyword that is input and stored in advance in the matching system to match the specific keyword with respect to extracted data that is difficult to be formatted.

The product information matched by the client-side viewing screen data analysis is not limited to products (food, home appliances, drugs, clothing, consumer goods, furniture, automobiles, etc.) that can be sold in the form of unit goods, and various services (schools, travel products, And a variety of intangible information, such as financial products, financial products, etc.) and various information (such as news in each field).

The present invention is to derive the degree of interest of the client based on the data extracted from the OCR, image recognition technology, face recognition technology, etc. for product recommendation, matching the product to the derived interest, the client is the current time Automatically recommends the products you are interested in, and if you include ads in the featured products, they will be delivered to clients who need the information and without spam processing, which can satisfy both advertisers who can increase your advertising effectiveness. .

In addition to using html-based data such as OCR, image recognition, and face recognition, it is applicable to terminals that are not easy to track usage patterns by using html-based information such as IPTV smartphones or mobile communication terminal applications. It can be applied to any device type without limitation, and it can effectively utilize big data collection, and recommend the product for sale through collected and analyzed data, or push-type customized advertisement or various analysis data based on user's interest analysis. It can be used to derive interests for all the media by the communication used by the client to make big data, and analyze the client's disposition through the big data, so it is an invention having various excellent effects such as accurate product recommendation.

1 is a schematic diagram illustrating a network concept made to implement a method for individual product recommendation through big data analysis of the present invention.
Figure 2 is a flow chart illustrating the processing of the client and matching system for explaining the individual product recommendation method through big data analysis of the present invention.
Figure 3 is a block diagram showing the configuration of individual goods system through big data analysis of the present invention.

Hereinafter, a detailed technical configuration of the present invention will be described with reference to the accompanying drawings.

When the following terms are defined in the process of describing the present invention, the client side is a user who forms a network through a wired / wireless connection to a specific server and uses a service. The matching system includes various information provided from a specific server to which the client is connected. It extracts the image, receives it, receives the information and the image as searchable text, stores the extracted data as big data for each client, and integrates the stored data to enable big data to be used for various analysis.

Here, the client and the user are agreed, and the matching system is used with the server.

Matching a client consisting of a medium such as a smart phone, a smart TV, a desktop computer, a laptop computer, and a portable information communication device (PDA, etc.) with a server connecting the wired / wireless network using the client as a plurality of subscribers. A path to form a system to capture the current viewing screen of the client-side user and send it to the matching system, extracting text data from the html source included in the transmitted data;

html source is analyzed and the user's interest is analyzed as a text word through another path that recognizes the image as text, recognizes the object in the image, or recognizes the face of the person included in the image. To be recommended.

Specifically, the viewing screen data transmission step (S100) of extracting HTML data from the screen of the user on the client side, and transmitting the screen data captured by the user viewing the extracted image to the matching system (server); ;

A viewing screen data recognition step (S200) of extracting html text data from which the source extraction is possible from the viewing screen data transmitted from the client;

Optical character recognition (OCR) recognition for extracting from the captured image that is not source extractable as text for analysis, image recognition for extracting the image as text to analyze objects contained in the image, and This includes image recognition for extracting the image as text to analyze the embedded face.

An image recognition step (S300) of recognizing data by extracting text in an unknown recognition step;

Data including a collection process of collecting the recognized text data from the html data and the image, an analysis process of analyzing the collected data to give meaning to each data, and a classification process of classifying the collected and analyzed data by field An analysis step (S400);

Big data storage step (S500) of storing the analyzed data in the DB as a variety of analysis data, such as individual, medium, storage date, time, interest, gender, age ...;

A recommendation product information matching step (S600) of deriving a product to be recommended to a client through a product DB recording products having various meta information in a DB;

Big data analysis based on the usage information of the user medium consisting of the product recommendation step (S700) of transmitting the matched recommendation product information to the client side and the recommended product display step (S800) of the recommended product displayed on the client-side device Achieve individual product recommendation through

The data analysis step (S400) further includes a block setting step of blocking data analysis that is unnecessary or contrary to public order, further including an extraction exclusion setting process of specifying text and images or URLs to be excluded from the analysis and excluding them from the analysis. The individual product recommendation method may be achieved by analyzing the collected big data based on the usage information of the user medium.

The blocking setting step may include a text word, an image, or a URL to be excluded from the analysis, which may include various adult-related data, immoral data, and disgusting data.

In the big data storage step (S500), a matching keyword input process for inputting additional information (signature, mood, era background, etc.) matching the extracted data in order to match the unstructured information (concept, mood, period background, etc.) It can be made to include more.

The period of transmitting the client-side viewing screen data to the matching system is not specific but preferably repeated every predetermined period.

The matching keyword input process may be performed to be added by the server-side manager as a keyword that is input and stored in advance in the matching system to match the specific keyword with respect to extracted data that is difficult to be formatted.

The product information matched by the client-side viewing screen data analysis includes products (food, home appliances, medicine, clothing, consumer goods, furniture, automobiles, etc.) that can be sold in the form of unit goods, and at the same time, various services are not limited to such products. The method may further include matching various intangible information such as a school, a travel product, a financial product, a financial product, and various information (such as news in each field).

A system configuration implementing the individual product recommendation method through big data analysis collected based on the usage information of the user medium as described above will be described.

A plurality of client media 100 made of any one of media such as a smart phone, a smart TV, a desktop computer, a laptop computer, and a portable information communication device (PDA, etc.), and a wired / wireless network using the client as a plurality of subscribers. It consists of a matching system 200 server connected to the server, the client medium 100 is provided with an image capture module 110 and the html extraction module 120 to the screen being played on the display 150 by these modules It is made by providing a data transmission module 130 for picking up and transmitting the pickup data to the matching system.

The image capturing module 110 captures the screen being played on the display of the client medium, and in the case of data including the html source, directly extracts the text data by the extraction module 120 and transmits the data. The data transmission module is a type of client medium (smartphone, smart TV, desktop computer, laptop computer and portable information communication device) and a module type (image capture) so that the captured image and the extracted html data can be identified by the matching system. Module or html extraction module) and additional information such as collection time, IP, channel number, SO (transmitter), operating system (OS), user information, etc. are compressed to be transmitted to the matching system.

Each module may be a memory resident application or a non-resident application depending on the service target.

On the other hand, the matching system 200 decompresses the compressed data received from the client side html, the recognition unit 210 for recognizing the captured image as a specific text word, and analyzes the data recognized as the specific text word A data analyzer 220 for collecting and analyzing data to give meaning to the DB, and a DB storage unit 230 for storing the analyzed data and providing data to recognize the image recognition of the recognition unit as a specific text word. A product DB 240 for recording a recommended product, an information matching unit 250 matching the analysis data of the data analysis unit 220 with the product of the product DB 240, and the matched product information to the client; Product recommendation unit 260 to form a personalized product recommendation system through the analysis of big data collected based on the usage information of the user medium.

The recognition unit 210 extracts html text data from which the source extraction is possible from the viewing screen data transmitted from the client, and an OCR for extracting the text for analysis from the captured image that cannot be extracted. (Optical character recognition (OCR) recognition unit 212, an image recognition unit 213 for extracting the image as a text to analyze the object included in the image, and the image to the text to analyze the face included in the image It includes a face recognition unit 214 for extracting.

The extracted and recognized data is made with reference to the above-described DB storage 230 data in order to convert the unstructured data or image into the data formatted into a specific text word.

The image recognizing unit 213 of the recognizing unit 210 is for recognizing at least an object, for example, a product (food, home appliance, medicine, clothing, consumer goods, furniture, automobile, etc.) that can be sold in the form of unit goods from an image. It is not limited to the product, and the image of various services (schools, travel products, financial products, financial products, etc.) and various information (news of each field, etc.) is recognized and extracted as text words.

The face recognition unit 214 analyzes at least a face included in an image and extracts the image as text. For example, the face recognition unit 214 is extracted based on a specific person such as a celebrity, an actor, a model, and a celebrity with reference to the DB storage unit. It is made to extract a text word by the name of the person.

As a result, image capture screens such as clothes, bags, and jewelry worn by a particular celebrity are converted into text words such as celebrity 000, 00 clothing, 00 bags, and 00 sunglasses.

The data analyzer 220 collects at least the text data recognized from the html data and the image, a collector 221, an analyzer 223 which analyzes the collected data and gives meaning to each data, and the analyzed data. It includes a data classifier 224 for classifying the data by field, it can be made by adding the setting unit 222 as a filter to exclude a predetermined keyword from the extracted data.

The analysis unit 223 analyzes the collected data to give meaning to each data, and extracts the data extracted from the users at set time intervals to derive a trend of personal interest and interests of other extracted users. Various analyzes can be performed, such as comparative analysis, which derives social interests compared to other matters, and common interests among all users, and this analysis can be understood by referring to big data analysis.

The classification unit 224 is configured to reclassify the collected and analyzed data into various fields and classifications and store the data in the DB storage unit 230.

In this case, the classification is performed to divide and store at least the analyzed data into various fields such as individual, medium, storage date, time, interest, gender, age, and the like.

The setting unit 222 is for excluding text words that match the keywords to be excluded from the analysis, and is provided to filter text, images, URLs, etc. for which adult authentication is not required or appropriate.

The DB storage unit 230 records, updates, stores, adds data through the recognition unit 210 and the data analysis unit 220, and provides reference data to the recognition unit and the analysis unit to provide uniform keywords for data analysis. It is configured to provide a keyword input unit 231 for inputting a specific keyword, such as synonyms to convert the recognition text into a specific text, and the analysis data DB 232 to record and store the analysis data of the data analysis unit 220 to form a big data Is done.

According to the individual product recommendation method and product recommendation system through big data analysis collected on the basis of the use information of the user medium of the present invention having the above configuration, the client may use any type of medium without restriction on his / her use information. Based on the best matching product information can be provided.

This matching is applicable everywhere, regardless of product or service type.

For example, a user who watches a lot of clothing sales broadcasts through home shopping can analyze the images and texts on the viewing screen to accurately analyze, match, and deliver the type of clothing of interest, and then play them on their displays. You can conveniently receive information on your area of interest.

Meanwhile, a user who mainly views a travel site or a travel broadcast may substitute and transmit information related to travel and goods of a travel company, and these recommended products are not only a unit product that is sold but also various service products and information. Is not particularly limited.

100: client medium 110: image capture module
120: html extraction module 130: data transmission unit
150: display 200: matching system
210: recognition unit 211: html extraction unit
212: OCR recognition unit 213: image recognition unit
214: face recognition unit 220: data analysis unit
221: collecting unit 222: setting unit
223 analysis unit 224 classification unit
230: DB storage unit 231: keyword input unit
232: analysis data DB 240: product DB
250: information matching unit 260: product recommendation unit

Claims (13)

Matching a client consisting of a medium such as a smart phone, a smart TV, a desktop computer, a laptop computer, and a portable information communication device (PDA, etc.) with a server connecting the wired / wireless network using the client as a plurality of subscribers. System to capture the current viewing screen of the client-side user and send it to the matching system,
A path for extracting text data from the html source included in the transmitted data,
html source is analyzed and the user's interest is analyzed as a text word through another path that recognizes the image as text, recognizes the object in the image, or recognizes the face of the person included in the image. Individual product recommendation method through big data analysis collected based on the usage information of the user medium, characterized in that made to recommend. The method according to claim 1,
The recommendation method may include extracting HTML data from a screen viewed by a user on the client side, and transmitting the screen data by capturing an image viewed by the user to a matching system (server);
A viewing screen data recognition step (S200) of extracting html text data from which the source extraction is possible from the viewing screen data transmitted from the client;
Optical character recognition (OCR) recognition for extracting from the captured image that is not source extractable as text for analysis, image recognition for extracting the image as text to analyze objects contained in the image, and An image recognition step (S300) of recognizing data by extracting text into an image recognition step including image recognition for extracting an image as text to analyze an included face;
Data including a collection process of collecting the recognized text data from the html data and the image, an analysis process of analyzing the collected data to give meaning to each data, and a classification process of classifying the collected and analyzed data by field An analysis step (S400);
Big data storage step (S500) of storing the analyzed data in the DB as a variety of analysis data, such as individual, medium, storage date, time, interest, gender, age ...;
A recommendation product information matching step (S600) of deriving a product to be recommended to a client through a product DB recording products having various meta information in a DB;
A product recommendation step (S700) of transmitting the matched recommendation product information to the client side, and the recommendation product collected on the basis of the usage information of the user medium, comprising a recommendation product display step (S800) displayed on the client-side device. Individual product recommendation method through big data analysis. The method according to claim 2,
The data analysis step (S400) further includes a block setting step of blocking data analysis that is unnecessary or contrary to public order, further including an extraction exclusion setting process of specifying text and images or URLs to be excluded from the analysis and excluding them from the analysis. Personal product recommendation method through big data analysis collected based on the usage information of the user medium, characterized in that made. The method according to claim 3,
The text word, image or URL to be excluded from the analysis in the blocking setting step is collected based on the usage information of the user medium, which includes various adult-related data, immoral data, and disgusting data as target text words. Individual product recommendation method through big data analysis. The method according to claim 2,
The method may further include a matching keyword input process of inputting additional information that matches the extracted data in order to match the unstructured information in the big data storage step (S500). Individual product recommendation method through big data analysis. The method according to claim 2,
The period of the transmitting step of transmitting the client-side viewing screen data to the matching system is repeated in an unspecified period, the individual product recommendation method through the big data analysis collected based on the usage information of the user medium. The method according to claim 2,
The product information matched by the client-side viewing screen data analysis may be sold in the form of unit goods (food, home appliances, drugs, clothing, consumer goods, furniture, automobiles, etc.), and various services (schools, travel products, financial products). , Jae Tech products, etc.) and a variety of information (such as news in each field) one or more of the information consisting of personal data recommendation method by analyzing the big data collected based on the usage information of the user medium. A plurality of client media 100 made of any one of media such as a smart phone, a smart TV, a desktop computer, a laptop computer, and a portable information communication device (PDA, etc.), and a wired / wireless network using the client as a plurality of subscribers. It consists of a matching system 200 server connected to the server, the client medium 100 is provided with an image capture module 110 and the html extraction module 120 to display the screen being played on the display 150 by these modules And a data transmission module 130 for picking up and transmitting the pickup data to the matching system. The matching system 200 decompresses the compressed data received from the client to recognize html and the captured image as a specific text word. Recognizing unit 210 and the data to give a meaning in order to analyze the data recognized as the specific text word A data analysis unit 220 for collecting and analyzing data, a DB storage unit 230 for storing the analyzed data and providing data to recognize the image recognition of the recognition unit as a specific text word, and a product DB 240 recording the recommended products. ), An information matching unit 250 for matching the analysis data of the data analysis unit 220 and the product of the product DB 240, and a product recommendation unit 260 transmitting the matched product information to the client. Individual product recommendation system through big data analysis collected based on the usage information of the user medium, characterized in that the individual product recommendation through analysis to the user use information. The method according to claim 8,
The image capturing module 110 captures the screen being played on the display of the client medium, and in the case of data including the html source, directly extracts the text data by the extraction module 120 and transmits the data. The data transmission module is a type of client medium (smartphone, smart TV, desktop computer, laptop computer and portable information communication device) and a module type (image capture) so that the captured image and the extracted html data can be identified by the matching system. Module or html extraction module) and additional information such as collection time, IP, channel number, SO (operating service provider), operating system (OS), and user information are compressed and transmitted to the matching system. Individual product recommendation system through big data analysis based on information. The method according to claim 8,
The recognition unit 210 extracts html text data from which the source extraction is possible from the viewing screen data transmitted from the client, and an OCR for extracting the text for analysis from the captured image that cannot be extracted. (Optical character recognition (OCR) recognition unit 212, an image recognition unit 213 for extracting the image as a text to analyze the object included in the image, and the image to the text to analyze the face included in the image Personal product recommendation system by analyzing the big data collected based on the usage information of the user medium, characterized in that it comprises a face recognition unit 214 for extracting. The method according to claim 10,
The extracted and recognized data is based on the use information of the user medium, characterized in that to refer to the above-described DB storage unit 230 data to convert the unstructured data or image into the data formatted into a specific text word. Individual product recommendation system through collected big data analysis. The method according to claim 8,
The data analyzer 220 collects at least the text data recognized from the html data and the image, a collector 221, an analyzer 223 which analyzes the collected data and gives meaning to each data, and the analyzed data. And a data classification unit 224 for classifying the data by field, and adding the setting unit 222 as a filter to exclude certain keywords from the extracted data. Individual product recommendation system through collected big data analysis. The method according to claim 8,
The DB storage unit 230 records, updates, stores, adds data through the recognition unit 210 and the data analysis unit 220, and provides reference data to the recognition unit and the analysis unit to provide uniform keywords for data analysis. And an analysis data DB 232 for recording and storing the analysis data of the keyword input unit 231 and the data analysis unit 220 for inputting a specific keyword such as a synonym to convert the recognized text into a specific text. Personal product recommendation system through big data analysis collected based on the usage information of the user media characterized in that.

Images