KR102363961B1 - Analysis method based on product image, apparatus and program - Google Patents

Abstract

Disclosed is an analysis device based on a product image. The analysis device includes a data collection unit, a text extraction unit, and one or more processors for pre-processing the extracted text. By providing the analysis device, market research for market entry can be effectively performed.

Description

Analysis method, device and program based on product image {ANALYSIS METHOD BASED ON PRODUCT IMAGE, APPARATUS AND PROGRAM}

The present invention relates to an apparatus for extracting text or images from product image data to analyze nutritional components, functions, and authentication-related information of products.

When developing or selling a product, market research related to the product is inevitable. The traditional method of market research is to visit distribution channels directly, which is time-consuming and costly.

Thanks to the development of Internet communication technology, the market research method that directly collects product information through an online shopping mall site takes less time and cost than the direct visit method, but requires a search/collection manpower, so the number of products to be analyzed There is a problem that the burden of time and cost increases as the number increases.

Accordingly, there is a need for a method for conducting market research related to a product more efficiently.

In order to derive the method, as part of the national R&D project under the Ministry of Agriculture, Food and R&D <Research on Common Technology to Support the Export Research Project Group>, a study on “Export Market Analysis based on Natural Language Processing and Natural Language Processing” was conducted.

Republic of Korea Patent Publication No. 10-2021-0007775 (published date: 2021.01.20)

The present invention for solving the problems described above is to extract and process text from product image data, and to provide analysis information related to nutritional components, functions, and authentication from the processed text.

In addition, the present invention is to provide a method of extracting and processing an image from product image data, and recognizing a certification mark in the processed image.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

A product image-based analysis device according to the present invention for solving the above-described problems includes a data collection unit for collecting one or more product image data, a text extraction unit for extracting text from product image data collected through the data collection unit, and extraction It may include one or more processors for pre-processing the text.

The processor may perform all of the commonly used basic text preprocessing, such as removing stopwords and extracting headwords.

The processor includes, for the preprocessed text, based on a pre-stored data pool, a nutritional component list including one or more nutritional component information, a function list including one or more function information, and one or more authentication information You can create a list of certifications that

The processor may be configured to determine a correlation between two or more pieces of information selected by allowing redundancy among the nutritional component information, the functional information, and the authentication information, based on the generated at least one list.

The text extractor may extract text from product image data based on OCR (Optical Character Recognition).

A processor performing pre-processing on the extracted text divides the extracted text into word or phrase units, checks whether the separated word or phrase is a word or phrase included in a pre-stored standard dictionary, and When the frequency of use is less than or equal to a predetermined frequency, the corresponding word or phrase may be excluded.

Also, the processor may perform preprocessing for classifying parts of speech (eg, verbs/adjectives/nouns, etc.) of the extracted text.

In addition, the processor may perform preprocessing for excluding meaningless special characters.

Even if the divided word is not included in the standard dictionary, the processor determines that the divided word or phrase is included in at least one of nutritional component information, functional information, and authentication information used by a predetermined expert group, the divided word or The phrase may be configured to be included in the corresponding list mapped to the corresponding information.

Product image-based analysis apparatus according to an embodiment of the present invention for solving the above-described problems is an image extraction unit for extracting an image from product image data collected through a data collection unit for collecting one or more product image data, and the extracted It may include one or more processors to pre-process the image.

The processor may recognize the authentication mark included in the pre-processed image based on a previously trained authentication mark recognition model to recognize the authentication mark included in the input image.

The processor performing pre-processing on the extracted image excludes advertisement data irrelevant to a predetermined product category from the extracted image, excludes duplicate data based on the product code, and returns an image having image quality of a predetermined or less can be excluded.

The authentication mark recognition model may be learned using training image data in which at least one of quality, color, photographing angle, tilt, partial loss, and partial distortion of the authentication mark is different from each other.

The processor may provide training image data to the authentication mark recognition model so that the configuration of the learning data is changed for each authentication mark based on the expression complexity of the authentication mark.

The product image-based analysis apparatus according to an embodiment of the present invention may further include a display.

The processor may integrate nutritional information, function information, authentication information, and certification mark for each product or product category and output the integrated information to the display.

In addition, the processor may output, on the display, at least one of nutritional information, function information, and authentication information required or recommended to be acquired for each specific product category.

When one or more pieces of authentication information are included, the processor may preferentially provide, in the product image data, an image of a region where an authentication mark corresponding to the authentication information is frequently disposed to the authentication mark recognition model.

The product image-based analysis method according to the present invention for solving the above-described problems includes the steps of collecting one or more product image data, extracting text from the collected product image data, pre-processing the extracted text, the For the preprocessed text, based on the pre-stored data pool, a nutritional component list including one or more nutritional component information, a function list including one or more function information, and an authentication list including one or more authentication information are generated. and determining a correlation between two or more pieces of information selected by allowing redundancy among the nutritional component information, the functional information, and the authentication information based on the generated at least one list.

The product image-based analysis method according to the present invention for solving the above-described problems includes the steps of collecting one or more product image data, extracting an image from the collected product image data, pre-processing the extracted image, and input The method may include recognizing the authentication mark included in the pre-processed image based on the previously trained authentication mark recognition model to recognize the authentication mark included in the image.

The computer program according to the present invention for solving the above problems is combined with a computer that is hardware, and stored in a medium to perform the product image-based analysis method, to provide a product image-based analysis method of an analysis device there is.

In addition to this, other methods, other devices, other systems for implementing the present invention, and a computer-readable recording medium for recording a computer program for executing the method may be further provided.

By providing the product image-based analysis apparatus according to the present invention as described above, time and cost for market research can be saved, and user convenience can be improved.

In addition, according to the present invention, product data can be automatically collected online, and reliable product-related information can be automatically extracted because the information described on product packaging is automatically extracted using a trusted public database. , analysis is performed based on the extracted data and can be used for new product development and marketing strategy derivation.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a view for schematically explaining an apparatus for extracting, processing and analyzing various information from an image of a product according to an embodiment of the present invention;
2 is a block diagram showing the configuration of a product image-based analysis device according to an embodiment of the present invention;
3 shows a result of text pre-processing performed by the product image-based analysis apparatus according to an embodiment of the present invention;
4 shows a product image-based analysis device showing the correlation between nutritional information and functional information according to an embodiment of the present invention;
5 shows a method for increasing the authentication mark recognition rate of the authentication mark recognition model using label information according to an embodiment of the present invention;
6 is a sequence diagram of an analysis method using text extracted from product image data according to an embodiment of the present invention;
7 is a sequence diagram of an analysis method using an image extracted from product image data according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Before the description, the meaning of the terms used in this specification will be briefly described. However, it should be noted that, since the description of the term is for the purpose of helping the understanding of the present specification, it is not used in the meaning of limiting the technical idea of the present invention unless explicitly described as limiting the present invention.

1 is a diagram schematically illustrating an apparatus for extracting, processing, and analyzing various information from an image of a product according to an embodiment of the present invention, and the apparatus will be referred to as a product image-based analysis apparatus 100 .

The product image-based analysis apparatus 100 may extract text or images from images such as a product surface and packaging displayed online, and may process (eg, pre-process, etc.) the extracted text or image and use it for analysis. .

Here, the product is not limited to the finished product, and the product image may include not only a product surface/package image, but also a product that represents the product on/offline and is expressed as an image. For example, product images include not only the image of the product itself, but also the web/mobile page itself related to the product or various information related to the product on the web/mobile page (review information, price information, rating information, information to describe the product, etc.) may include

In addition, product image data includes one or more product images, and may be built with big data and stored in a cloud, a server, a user terminal, and the like.

Referring to FIG. 1 , the product image-based analysis device 100 may display a product Ar through the display 140 , and extract text from the displayed product Ar based on Optical Character Recognition (OCR). can do. For example, the text may include product name 21 , nutritional information 22 , function information 23 , authentication information 24 , manufacturer information 26 , and the like. In an optional or additional embodiment, the text may include various information related to the product on the web/mobile page (eg, review information, price information, rating information, information for describing the product, etc.).

The product image-based analysis apparatus 100 may process the extracted text to perform an optimal analysis process.

The pre-processing of the extracted text may include all commonly used basic text pre-processing, such as removal of stopwords and extraction of headwords.

For example, the product image-based analysis apparatus 100 may perform a pre-processing process of classifying the extracted text into words or phrases. Here, the phrase may include a word or sentence including two or more words. Also, the product image-based analysis apparatus 100 may perform pre-processing for classifying parts of speech (eg, verbs/adjectives/nouns, etc.) of the extracted text.

In addition, the product image-based analysis apparatus 100 may perform pre-processing of excluding meaningless special characters.

The product image-based analysis apparatus 100 may perform an analysis process by generating a list of nutritional information, function information, and authentication information from the pre-processed text.

Here, the product image-based analysis apparatus 100 may determine a correlation between two or more pieces of information selected by allowing duplication among nutritional information, function information, and authentication information, based on the at least one generated list.

For example, the product image-based analysis apparatus 100 may include a correlation between nutritional information for a product, a correlation between functional information, a correlation between authentication information, a correlation between nutritional information and functional information, and functional information. and at least one of a correlation between the and authentication information and a correlation between the nutritional component information and the authentication information may be determined.

In addition, the product image-based analysis apparatus 100 may extract the authentication marks 25a and 25b of the product image in the form of an image, and recognize which authentication mark the extracted image is. For this, a model trained by neural network-based supervised learning may be used.

2 is a block diagram showing the configuration of the product image-based analysis apparatus 100 according to an embodiment of the present invention.

The product image-based analysis apparatus 100 may include a data collection unit 110 , a text extraction unit 120 , an image extraction unit 130 , a display 140 , a memory 150 , and a processor 190 . . The product image-based analysis apparatus 100 may include only some of the above-described components or may further include other components.

The data collection unit 110 may collect one or more product image data. The data collection unit 110 may have various interfaces to collect various data.

For example, the data collection unit 110 may collect surface images of front, back, left, and right (typically, before) of products belonging to a specific product category (eg, infant products, green tea, etc.) from the online shopping mall site as big data.

The text extraction unit 120 may extract text from the product image data collected through the data collection unit 110 . For example, the text extraction unit 120 may extract text from product image data based on OCR (Optical Character Recognition), but the embodiment is not limited thereto. The text extraction unit 120 may check whether the text has been correctly extracted.

The image extraction unit 130 may extract an image from the product image data collected through the data collection unit 110 that collects one or more product image data.

In an optional embodiment, the image extraction unit 130 may extract images of each of the entire region and the partial region of the product image data.

The display 140 may display various data on the screen under the control of the processor 190 , and may be implemented as a backlight-based display or a display equipped with a self-luminous device.

The memory 150 is a module that collectively refers to various types of storage, and the memory 150 may store information necessary to perform an operation using artificial intelligence, machine learning, and artificial neural networks. The memory 150 may store various learning models. The learning models may be used to infer a result value with respect to new input data other than the learning data, and the inferred value is used as a basis for a decision to perform a certain operation. can be used

The learning models may be trained based on label information, and a backpropagation algorithm may be used to increase learning accuracy. In this specification, the memory 150 may store the authentication mark recognition model (MM) for recognizing the authentication mark in the image extracted from the product image data.

The processor 190 may be implemented as one or more, and even if expressed in a singular number, it may be regarded as a plurality. The processor 190 is a module that controls the components of the product image-based analysis device 100, and the processor 190 has a circuit physically structured to perform a function expressed as a code or instruction included in a program, It may mean a data processing device built into hardware. As an example of the data processing apparatus embedded in the hardware as described above, a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated (ASIC) circuit) and a processing device such as a field programmable gate array (FPGA), but the scope of the present invention is not limited thereto. The processor 190 may separately include a learning processor for performing artificial intelligence operations, or may have a learning processor itself.

The processor 190 may pre-process the extracted text. Accordingly, the search speed and accuracy may be improved.

In addition, the processor 190 may perform all of the commonly used basic text preprocessing, such as removing stopwords and extracting headwords.

In addition, the processor 190 may classify the extracted text in units of words or phrases, and check whether the divided words or phrases are words (or phrases) included in a pre-stored standard dictionary. The standard dictionary may be provided for each of various product categories.

Also, when the frequency of use of the divided word or phrase is less than or equal to a predetermined frequency, the processor 190 may exclude the word or phrase. The predetermined frequency may be less than 1% of the frequency of use, but the embodiment is not limited thereto.

The processor 190 determines that even if the divided word or phrase is not included in the standard dictionary, when the divided word or phrase is included in at least one of nutritional information, function information, and authentication information used by a predetermined expert group, the corresponding The separated words or phrases may be included in the nutritional ingredient list, function list, and certification list to be described later.

The processor 190 includes a nutritional component list including one or more nutritional component information, a function list including one or more function information, and one or more authentication information for the pre-processed text, based on a pre-stored data pool. You can create a list of certifications that

In an optional embodiment, the processor 190 may utilize a public nutrition database (eg, USDA, etc.) to generate a nutritional component list included in the collected overall data. For example, the public nutrition database may include ingredients that can be labeled with nutrition provided by the Ministry of Agriculture, Food and Rural Affairs for each country in order to help with future commercialization strategies.

In addition, the processor 190 may generate a function list included in the collected total data by utilizing the public functionality database. For example, the processor 190 may use data such as MesH terms including function-related information and nutritional component information.

The processor 190 may generate an authentication list included in all data collected from various databases (eg, a standard dictionary used by a predetermined expert group or an authentication-related list organized by a researcher).

For example, the processor 190 may extract the corresponding word and not exclude it in order to determine the correlation between the nutritional component and the certification even if the certification-related word (eg, non-GMO, GMO, etc.) does not exist in the standard dictionary.

The processor 190 may determine a correlation between two or more pieces of information selected by allowing redundancy among nutritional component information, functional information, and authentication information based on the at least one generated list.

For example, the processor 190 may determine, based on the generated at least one list, a correlation between nutritional component information, a correlation between functional information, a correlation between authentication information, a correlation between nutritional component information and functional information, At least one of a correlation between the functional information and the authentication information and a correlation between the nutritional component information and the authentication information may be determined.

In an optional or additional embodiment, the processor 190 may determine a correlation between the nutritional component information, the functional information, and the authentication information.

When a specific nutritional component is emphasized, the processor 190 may identify other nutritional component information mentioned together. For example, since the processor 190 has a high probability that DHA and iron are mentioned together in the infant product category, and a probability that DHA is also mentioned when iron is used, it can map them together.

The term correlation used in this specification is a term for comprehensively expressing a relationship between information or a rule of association, and is not limited to correlation used in statistics.

When the correlation between the nutritional component information and the functional information is expressed, the processor 190 may determine which efficacy among the effects of a specific nutritional component is emphasized. For example, the processor 190 may recognize that lutein is more related to the brain than the eyes from among the product categories for infants and toddlers in the United States, and may map them together.

The processor 190 may determine which certification is referred to when emphasizing a particular nutritional component. For example, the processor 190 may map nutrients of a size such as carbohydrate, fat, protein, etc., compared to micronutrients, with the authentication-related words having a high rate of mention thereof.

That is, the processor 190 may determine that the related information (nutritional information, function information, authentication information, etc.) is written together from the product image data, and determine the correlation between the information by using various databases.

Meanwhile, the processor 190 may pre-process the image extracted by the image extraction unit 130 . The processor 190 may exclude advertisement data unrelated to a predetermined product category from the extracted image, exclude duplicate data based on a product code, and exclude images having image quality of a predetermined or less.

The processor 190 may recognize the authentication mark included in the pre-processed image based on the previously trained authentication mark recognition model (MM) to recognize the authentication mark included in the input image.

The authentication mark recognition model (MM) may include one or more neural network algorithms. For example, the authentication mark recognition model (MM) may include one or more Convolution Neural Networks (CNNs), Deep Neural Networks (DNNs), and the like. The authentication mark recognition model (MM) can collect various training data indicating the authentication mark for learning, and by providing label information together with the corresponding training data, it is possible to increase the recognition efficiency of the model.

The authentication mark recognition model (MM) may increase the recognition probability of the authentication mark by using training image data having different at least one of image quality, color, and shooting angle during training. In this case, a backpropagation algorithm may be used, and learning may be repeated until the value of the loss function reaches a predetermined target value.

As an optional embodiment, the authentication mark recognition model (MM) may be implemented based on Google's AutoML Vision model. The authentication mark recognition model (MM) may collect training data similar to image big data to be applied to the model later, and may be trained to balance the number of training data for each authentication mark.

The authentication mark recognition model (MM) may be learned using training image data in which at least one of image quality, color (including black and white), shooting angle, tilt, partial loss, and partial distortion of the authentication mark is different. The recognition model (MM) may consider a case where the product is configured in a cylindrical shape and the certification mark is curved. In addition, the authentication mark recognition model (MM) may be excluded from the training data when the product image is 50 KB or less.

The processor 190 may provide training image data to the authentication mark recognition model MM so that the configuration of the learning data is changed for each authentication mark based on the expression complexity of the authentication mark. Accordingly, even a simple authentication mark (eg, a combination of a simple character and a simple figure) can be easily recognized.

When learning is completed, when one or more authentication information is included, in the product image data, the processor 190 preferentially assigns an image of an area where an authentication mark corresponding to the authentication information is frequently disposed to the authentication mark recognition model. can provide Accordingly, it is possible to search only a partial area without searching the entire area of the image data, so that the authentication mark recognition speed can be improved.

Meanwhile, the processor 190 may integrate nutritional component information, function information, authentication information, and authentication mark for each product or product category and output it on the display. In this case, the processor 190 may express the results of the market research at a glance and provide it to the necessary users.

In addition, the processor 190 may output, on the display 140 , at least one of nutritional information required or recommended to be acquired, functional information, and authentication information for each specific product category for product strategy. Accordingly, it is possible to help individuals/companys aiming to launch a product.

Even if there is no authentication mark in the product image data, the processor 190 may compare and analyze the authentication-related text status by comparing it with the text extracted by OCR.

In an optional embodiment, the product image-based analysis apparatus 100 does not include all of the above-described components, but includes only the data collection unit, text extraction unit, memory, and processor 110, 120, 150, and 190, which are components related to text extraction. Alternatively, it may be implemented to include only the data collection unit, image extraction unit, memory, and processor 110 , 130 , 150 , 190 that are components related to image extraction. In this case, a plurality of product image-based analysis devices may be implemented, and various configurations including a display may be included as necessary.

In an optional or additional embodiment, the product image-based analysis apparatus 100 may further include a communication unit to collect product image data from various terminals or provide information related to an analysis result through the communication unit.

3 to 5 are diagrams for explaining a specific operation of the product image-based analysis apparatus 100 according to an embodiment of the present invention.

3 shows a result of text preprocessing performed by the product image-based analysis apparatus 100 according to an embodiment of the present invention.

Referring to FIG. 3 , the product image-based analysis apparatus 100 may classify text of product image data in units of words or phrases. The product image-based analysis apparatus 100 may classify text in units of words or phrases in a considerable amount of product image data, and may provide texts in units of words or phrases even in one product image data.

4 shows a product image-based analysis apparatus 100 showing a correlation between nutritional component information and functional information according to an embodiment of the present invention.

The product image-based analysis apparatus 100 may display the association rules derived from shopping cart analysis as in a table. The product image-based analysis apparatus 100 may provide information on which efficacy information is mainly associated with, whether there is a strong or weak correlation between nutritional components, and what kind of authentication information related thereto.

In an optional or additional embodiment, the product image-based analysis apparatus 100 may analyze how the content of the general nutritional component described in the nutritional component information is, but the embodiment is not limited thereto.

In an optional or additional embodiment, the product image-based analysis device 100 reports product name, food type, sales and production company name, expiration date, packaging unit and weight, raw material name, nutrient content, packaging material, and item in the case of food. Number, ingredient name, storage method, precautions, etc. can be analyzed, and in the case of cosmetics, product name, product information, usage method, ingredient name, precautions, capacity, manufacturing and sales company name, manufacturing number, expiration date, etc. can be analyzed. However, the embodiment is not limited to the above-described analysis.

5 shows a method for increasing the authentication mark recognition rate of the authentication mark recognition model (MM) using label information according to an embodiment of the present invention.

The processor 190 may generate a correct answer class by comparing the class information for the authentication mark recognized by the authentication mark recognition model (MM) with the verification data (correct answer).

6 is a sequence diagram of an analysis method using text extracted from product image data according to an embodiment of the present invention. Each step of the analysis method may be performed by the integrated control of the processor 190 .

First, the product image-based analysis apparatus 100 collects product image data (S610).

Here, the product image data may be implemented as big data, may be stored in the memory 150 of the product image-based analysis device 100 , and may be stored in various online storage locations (eg, cloud). In addition, the image-based analysis apparatus 100 may be implemented as a device, a server, a cloud system, etc., but the embodiment is not limited thereto.

After that, the product image-based analysis apparatus 100 extracts and pre-processes the text ( S620 ).

The image-based analysis apparatus 100 may perform all of the commonly used basic text preprocessing, such as removal of stopwords and extraction of headwords.

Next, the product image-based analysis apparatus 100 generates a nutritional component list, a function list, and a certification list (S630).

After that, the product image-based analysis apparatus 100 analyzes the correlation between two or more pieces of information selected by allowing duplication among nutritional information, functional information, and authentication information based on the at least one generated list (S640) .

7 is a sequence diagram of an analysis method using an image extracted from product image data according to an embodiment of the present invention.

First, the product image-based analysis apparatus 100 learns the authentication mark recognition model (S710).

Next, the product image-based analysis apparatus 100 collects product image data (S720).

After that, the product image-based analysis apparatus 100 performs image extraction and pre-processing (S730).

Next, the authentication mark is recognized by using the learned authentication mark recognition model (S740).

The method according to an embodiment of the present invention described above may be implemented as a program (or application) to be executed in combination with a computer, which is hardware, and stored in a medium. Here, the computer may be the product image-based analysis apparatus 100 described above.

The above-mentioned program, in order for the computer to read the program and execute the methods implemented as a program, C, C++, Python, JAVA, which the processor (CPU) of the computer can read through the device interface of the computer; It may include code coded in a computer language such as machine language. Such code may include functional code related to a function defining functions necessary for executing the methods, etc., and includes an execution procedure related control code necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do. In addition, the code may further include additional information necessary for the processor of the computer to execute the functions or code related to memory reference for which location (address address) in the internal or external memory of the computer to be referenced. there is. In addition, when the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the above functions, the code uses the communication module of the computer to determine how to communicate with any other computer or server remotely. It may further include a communication-related code for whether to communicate and what information or media to transmit and receive during communication.

The steps of a method or algorithm described in relation to an embodiment of the present invention may be implemented directly in hardware, as a software module executed by hardware, or by a combination thereof. A software module may contain random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.

In the above, embodiments of the present invention have been described with reference to the accompanying drawings, but those of ordinary skill in the art to which the present invention pertains can realize that the present invention can be embodied in other specific forms without changing the technical spirit or essential features thereof. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100: product image-based analysis device;
110: data collection unit;
120: text extraction unit;
130: image extraction unit
140: display
150 : memory
190: processor

Claims (14)

As a product image-based analysis device,
a data collection unit for collecting one or more product image data;
a text extraction unit for extracting text from the product image data collected through the data collection unit;
an image extraction unit for extracting an image from the product image data collected through the data collection unit; and
One or more processors for pre-processing the extracted text and images,
The processor is
For the pre-processed text, based on a pre-stored data pool including a predetermined nutritional database, a functional database, and an authentication-related database, a nutritional component list including one or more nutritional component information, one or more functions generating a function list including information and an authentication list including one or more authentication information;
Based on the generated at least one list, a correlation between two or more pieces of information selected by allowing redundancy among the nutritional information, the function information, and the authentication information is determined,
The processor is
Based on the previously trained authentication mark recognition model to recognize the authentication mark included in the input image, it is configured to recognize the authentication mark included in the pre-processed image,
The processor is
The extracted text is divided into words or phrases, and when the frequency of use of the divided word or phrase is less than or equal to a predetermined frequency, the word or phrase is excluded, and stopwords are removed;
Even if the divided word or phrase is not included in the standard dictionary, if the divided word or phrase is included in at least one of nutritional information, functional information, and authentication information used by a predetermined expert group, the divided word or include the phrase in that list that maps to that information;
and excluding advertisement data irrelevant to a predetermined product category from the extracted image, excluding duplicate data based on a product code, and excluding images with image quality below a predetermined value. According to claim 1,
The text extraction unit,
Text is extracted from product image data based on OCR (Optical Character Recognition),
A processor performing pre-processing on the extracted text,
An analysis device for confirming whether the divided word or phrase is a word or phrase included in a pre-stored standard dictionary. delete delete delete delete According to claim 1,
The authentication mark recognition model is,
An analysis device in which at least one of image quality, color, shooting angle, tilt, partial loss, and partial distortion of the authentication mark is learned using different training image data. 8. The method of claim 7,
The processor is
An analysis apparatus, configured to provide training image data to the authentication mark recognition model so that the configuration of the learning data is changed for each authentication mark based on the expression complexity of the authentication mark. According to claim 1,
further comprising a display;
The processor is
For each product or product category, the analysis device is configured to integrate nutritional information, function information, certification information and certification mark and output to the display. 10. The method of claim 9,
The processor is
The analysis device, configured to output, on the display, at least one of nutritional information, function information, and authentication information required or recommended to be acquired for each specific product category. According to claim 1,
The processor is
and, when one or more pieces of authentication information are included, in the product image data, an image of an area in which an authentication mark corresponding to the authentication information is frequently disposed is preferentially provided to the authentication mark recognition model. A product image-based analysis method performed by a processor, comprising:
collecting one or more product image data;
extracting text and images from the collected product image data;
pre-processing the extracted text and images;
For the pre-processed text, based on a pre-stored data pool including a predetermined nutritional database, a functional database, and an authentication-related database, a nutritional component list including one or more nutritional component information, one or more functions generating a function list including information and an authentication list including one or more pieces of authentication information; and
Based on the generated at least one list, allowing redundancy among the nutritional component information, the functional information, and the authentication information to determine a correlation between two or more pieces of information selected,
The processor is
Based on the previously trained authentication mark recognition model to recognize the authentication mark included in the input image, it is configured to recognize the authentication mark included in the pre-processed image,
The processor is
The extracted text is divided into words or phrases, and when the frequency of use of the divided word or phrase is less than or equal to a predetermined frequency, the word or phrase is excluded, and stopwords are removed;
Even if the divided word or phrase is not included in the standard dictionary, if the divided word or phrase is included in at least one of nutritional information, functional information, and authentication information used by a predetermined expert group, the divided word or include the phrase in that list that maps to that information;
and excluding advertisement data unrelated to a predetermined product category from the extracted image, excluding duplicate data based on a product code, and excluding images with image quality below a predetermined value. delete A computer program for providing a product image-based analysis method of an analysis device, which is stored in a computer-readable medium to perform the product image-based analysis method of claim 12 in combination with a computer that is hardware.

Images