KR20200045862A - Product management system and method based on image analysis - Google Patents

Abstract

The present invention provides a system for product management which can sense out-of-stock products and incorrectly displayed products in real time based on an image obtained by photographing a product display area. According to an embodiment of the present invention, the system for product management comprises: a database storing product information of each product; a photographing unit to photograph the product display area to acquire an image; an image analysis unit to analyze an image acquired by the photographing unit to acquire product information of a product existing in the image; and a determination unit to compare product information acquired by the image analysis unit with product information stored in the database to determine at least one of whether an out-of-stock product exists and whether an incorrectly displayed product exists. The product information includes major classification information based on the shape of a product and position information representing the position of a product in the product display area. The product information of each product stored in the database can be primarily classified into the same major classification group with products having the same major classification information to be stored.

Description

Product management system and method based on video analysis {PRODUCT MANAGEMENT SYSTEM AND METHOD BASED ON IMAGE ANALYSIS}

The present invention relates to a system and method for managing a displayed product based on an image of a product display area, and more particularly, to a product management system and method based on hierarchical image data analysis and management.

In a merchandise store where merchandise is displayed and sold at a store, such as a convenience store, it is common for store workers to check and manage the merchandise displayed directly. However, this method is difficult to quickly identify when a defect occurs. In addition, in the case of recent convenience stores, the number of single workers has increased, and it has become virtually impossible for workers to perform inventory management of products displayed at all stores while cleaning stores and responding to customers. Therefore, even though the product missing at the store has inventory in the store, there is a case where the customer cannot purchase it because it cannot be immediately replenished and displayed at the store.

In particular, in recent convenience stores, sales of fresh foods are increasing, and it is common that fresh foods do not have stock, so when shortages occur, quick recognition and ordering of goods are required.

In addition, in the case of franchise stores, strategic product display guidelines are provided to merchants to increase sales. However, if there are not enough workers, it is difficult to display products as instructed. Even when a customer inquires about the location of a specific product, there is a problem in that it is difficult for the worker to grasp the location of all the products, and thus it is difficult to provide information directly to the customer.

Korean Patent Publication No. 10-2014-0136089

An object of the present invention is to provide a product management system and method for real-time detection of defects generated and misaligned products in real time based on an image of a product display area for solving the aforementioned problems.

Further, an object of the present invention is to enable faster analysis by using a hierarchical analysis method in analyzing an image photographing a product display area and storing image data.

In addition, an object of the present invention is to shorten the time required for registering new product information in the product management system by using a hierarchical analysis method.

The problems to be solved by the present invention are not limited to the above-described problems, and the problems not mentioned will be clearly understood by those having ordinary knowledge in the technical field to which the present invention pertains from this specification and the accompanying drawings. .

A product management system according to an embodiment of the present invention includes a system for managing products based on an image of a product display area, comprising: a database in which product information of each product is stored; A photographing unit photographing the product display area to acquire an image; An image analysis unit that analyzes an image acquired by the photographing unit and obtains product information of a product existing in the image; And a determination unit that compares product information obtained by the image analysis unit with product information stored in the database, and determines at least one of whether there is a missing product and whether there is an incorrectly displayed product.

The product information includes large classification information based on the shape of the product and location information indicating the location of the product in the product display area, and the product information of each product stored in the database is the same large classification between products having the same large classification information It is first classified into groups and stored.

The product information may further include sub-classification information including at least one of a product name, a product code, and form information.

Product information of each product stored in the database may be stored in a second category based on the small classification information in a group classified based on the large classification information.

The image analysis unit may include: a first image analysis unit to analyze the image acquired by the photographing unit to obtain the large classification information and the location information; And it may include a pre-processing unit for performing a pre-processing for separating each area where the product is present in the image, based on the large classification information and the location information obtained from the first image analysis unit.

The image analysis unit includes at least one second image analysis unit corresponding to each of the primary classified large classification groups, and the second image analysis unit analyzes images of each region separated by pre-processing to obtain the sub-classification information. Can be obtained.

The photographing unit periodically acquires an image by photographing the product display area, and the image analysis unit periodically acquires product information based on the periodically acquired image, and the time information at which the corresponding product information is obtained is obtained. Can be stored in the database.

The determination unit may determine whether there is a missing product by comparing product information periodically acquired by the image analysis unit with product information stored in the database at a time earlier than the corresponding product information.

The reference location information of each product is stored in the database, and the determination unit may determine whether there is a misaligned product by comparing the reference location information with the location information included in the product information obtained from the image analysis unit. .

A display unit for displaying location information of each product based on the product information obtained from the image analysis unit may be further included.

Further comprising a new product registration unit for registering the product information of the new product in the database, the new product registration unit, an input unit for receiving a large category information of the new product; A new product photographing unit for acquiring an image by photographing the new product; A new product image analysis unit that analyzes the image acquired by the new product photographing unit to obtain small classification information of the new product; And an update unit that selects a group of the corresponding large category from the database based on the large classification information input through the input unit and adds the small classification information obtained by the new product image analysis unit to the selected group data.

The updating unit may update a second image analysis unit corresponding to the large classification information of the new product, from one or more second image analysis units corresponding to each of the primary classified large classification groups.

The pre-processing unit may further perform at least one of resizing and color regularization on an image of each separated area.

When the determination unit determines that there is a missing product or an incorrectly displayed product, the notification unit may further include a notification display unit that displays a notification.

A product management method according to an embodiment of the present invention is a method of managing a product using a database including product information of each product, and a photographing unit that acquires an image by capturing a product display area, wherein the product The information includes large classification information based on the shape of the product and location information indicating the location of the product in the product display area, and the product information of each product stored in the database is the same large classification group among products having the same large classification information. First classified and stored, the method includes: acquiring an image by photographing the product display area by the photographing unit; Analyzing the image obtained by the photographing unit and obtaining product information of a product existing in the image; And comparing product information of a product present in the image with product information stored in the database to determine at least one of a missing product and a misaligned product.

The product information may further include sub-classification information including at least one of a product name, product code, and form information, and the product information of each product stored in the database is within a group classified based on the large classification information, Secondly classified and stored based on the sub-classification information, and obtaining product information of a product existing in the image includes analyzing the image acquired by the photographing unit and obtaining the large classification information and the location information ; And performing pre-processing for separating each region in which the product exists in the image, based on the obtained large classification information and the location information.

The obtaining of the product information of the product existing in the image may further include obtaining the sub-classification information by analyzing an image of each region separated by pre-processing.

The method includes receiving information on a large category of a new product; Acquiring an image by photographing the new product; Analyzing the acquired image and obtaining sub-classification information of the new product; And selecting a group of the corresponding large category from the database based on the large classification information input through the input unit, and adding the small classification information obtained by the new product image analysis unit to the selected group data.

The method according to an embodiment of the present invention may be stored in a computer-readable recording medium with a series of instructions for causing the control device to perform the method.

According to an embodiment of the present invention, it is possible to provide a product management system capable of real-time detecting defects and erroneous products based on an image of a product display area.

Further, according to an embodiment of the present invention, in analyzing an image photographing a product display area and storing image data, a faster analysis may be possible by using a hierarchical analysis method.

In addition, according to an embodiment of the present invention, it is possible to shorten the time required for registering new product information in the product management system.

The effects of the present invention are not limited to the above-mentioned effects, and the effects not mentioned will be clearly understood by those skilled in the art from the present specification and the accompanying drawings.

1 is an exemplary block diagram for explaining the configuration of a product management system according to an embodiment of the present invention.
FIG. 2 is an exemplary diagram for explaining that product information is hierarchically stored in the database of FIG. 1.
3 is an exemplary block diagram for explaining the configuration of the image analysis unit of FIG. 1.
4 is an exemplary flowchart of a preprocessing process performed by the preprocessing unit of FIG. 3.
5 is an exemplary block diagram illustrating the configuration of the new product registration unit of FIG. 1.
6 is a view for explaining a method of analyzing the layout of a product display area, in order to obtain location information of a product according to an embodiment of the present invention.
7 is a view for illustratively explaining the photographing unit of the product management system of the present invention.
8 to 12 are diagrams for explaining a method of analyzing an image by a product management system according to an embodiment of the present invention.
13 is a view for illustratively explaining the display unit and the notification display unit of FIG. 1.
14 is a diagram illustrating a new product photographing unit of a new product registration unit according to an embodiment.

Other advantages and features of the present invention and methods for achieving them will be made clear by referring to embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the disclosure of the present invention to be complete, and the general knowledge in the technical field to which the present invention pertains. It is provided to fully inform the holder of the scope of the invention, and the invention is only defined by the scope of the claims.

If not defined, all terms (including technical or scientific terms) used herein have the same meaning as generally accepted by universal technology in the prior art to which this invention belongs. Terms defined by general dictionaries may be interpreted as having the same meaning as related to the related description and / or the text of the present application, and are not conceptually or excessively interpreted, even if not expressly defined herein. Will not.

The terminology used herein is for describing the embodiments and is not intended to limit the present invention. In this specification, the singular form also includes the plural form unless otherwise specified in the phrase. As used herein, 'includes' and / or various conjugations of this verb, such as 'includes', 'includes', 'includes', 'includes', etc., refer to the stated composition, ingredients, components, The steps, operations and / or elements do not exclude the presence or addition of one or more other compositions, ingredients, components, steps, operations and / or elements. The term 'and / or' used herein refers to each of the listed configurations or various combinations thereof.

Meanwhile, terms such as '~ unit', '~ group', '~ block', and '~ module' used throughout this specification may refer to a unit that processes at least one function or operation. For example, it can mean a hardware component such as software, FPGA or ASIC. However, '~ bu', '~ gi', '~ block', and '~ module' are not limited to software or hardware. The '~ unit', '~ gi', '~ block', and '~ module' may be configured to be in an addressable storage medium or may be configured to play one or more processors.

Accordingly, as examples, '~ unit', '~ gi', '~ block', and '~ module' are components such as software components, object-oriented software components, class components and task components. Fields, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, database, data structures, tables, arrays and Includes variables. Components and functions provided in '~ unit', '~ gi', '~ block', and '~ module' have a smaller number of components and '~ unit', '~ gi', '~ block It can be combined with ',' ~ modules', or can be further separated into additional components and '~ units',' ~ gi ',' ~ blocks', and '~ modules'.

The present invention relates to a product management system based on an image of a product display area, which can be applied when displaying and managing a product in a product display area such as a stand in a store such as a convenience store. According to an embodiment of the present invention, image data may be hierarchically analyzed and stored to improve processing speed. Accordingly, according to an embodiment of the present invention, it is possible to shorten the time required when recognizing a product by analyzing an image or when registering a new product in a product management system.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is an exemplary block diagram for explaining the configuration of a product management system 100 according to an embodiment of the present invention. The product management system 100 is a system for managing products based on an image of a product display area.

Referring to FIG. 1, the product management system 100 includes a database 110, a photographing unit 120, an image analysis unit 130, and a determination unit 140.

The database 110 stores product information of each product that can be displayed in the product display area. In one embodiment of the present invention, the product information includes large classification information based on the shape of the product and location information indicating the location of the product in the product display area.

To exemplarily explain the classification information, it is assumed that there are cider bottled drinks, cola bottled drinks, juice canned drinks, and coffee canned drinks as products that can be displayed. Cider bottled drinks and cola bottled drinks are different products, but the appearance will have a similar bottle shape. Accordingly, the cider bottled beverage and the coke bottled beverage may have the same large classification information, for example, large classification information called “bottle drink”. Likewise, juice cans and coffee cans are different products, but the appearance will have a similar can shape. Therefore, the juice can beverage and the coffee can beverage may have the same large classification information, for example, "can beverage".

The location information of the product indicates the location of the product in the product display area. For example, the location information of the corresponding product may be represented by "○ column ○ No.", etc., depending on the number of seats from the left of the number of columns in the stand arranged in the product display area. The display method of the location information such as "○ column ○ number" is exemplary, and the location of each product in the product display area may be changed and used in any way that can be relatively displayed.

According to an embodiment of the present invention, product information of each product, that is, large classification information and location information is stored in the database 110. Product information of each product stored in the database 110 is first classified and stored in the same large classification group among products having the same large classification information. For example, in the above example, the cider bottled beverage and the coke bottled beverage may be stored in the same “bottled beverage” group.

According to an embodiment, the product information may further include sub-classification information for classifying the product information again in the large classification group. If the large category information is based on the approximate appearance of the product, the small category information is information that can distinguish different products, such as product name, product code, and shape information. The product code may be information recognized as a barcode attached to the product. In addition, in the present invention, the shape information refers to information obtained through more precise analysis of product appearance than recognition of product shape when recognizing large classification information.

 For example, cider bottled beverages and coke bottled beverages have the same major classification information, but product names, product codes, and shape information are different from each other. Therefore, in the database 110, product information of cider beverages and coke beverages is stored in the same "beverage beverage" large classification group, but is secondly classified based on the small classification information in the large classification group, "Cider bottled drinks" and " Cola bottled beverages ”group respectively.

FIG. 2 is an exemplary diagram for explaining that product information is hierarchically stored in the database of FIG. 1.

In the example of FIG. 2, there are Z bottled beverages, K canned beverages, A cup ramen, and F cup ramen as products that can be displayed in the product display area. As illustrated in FIG. 2, product information of each product may be classified and stored in a large classification group of “bottle drink”, “canned drink”, and “cup ramen”. Further, even within the same "cup ramen" large classification group, A cup ramen and F cup ramen are divided into different products and stored according to their respective sub-classification information.

Referring back to FIG. 1, the product management system 100 according to an embodiment of the present invention includes a photographing unit 120 that acquires an image by photographing a product display area. According to an embodiment, the photographing unit 120 may include one or more cameras. As the camera included in the photographing unit 120, any type of camera capable of recognizing the product name, product code, and shape of the product in the product display area may be used. The camera may be placed in a location where a product display area can be photographed in the store. For example, as shown in FIG. 7, when there are facing poles, the photographing unit 120 may be configured to attach a camera to each stage. The camera attached to each stage can take pictures of products displayed on poles facing each other. According to an embodiment, as illustrated in FIG. 7, a price tag of each product may be attached to a stand, and the camera may photograph a product and a price tag together.

Referring back to FIG. 1, the product management system 100 according to an embodiment of the present invention analyzes an image acquired by the photographing unit 120 and acquires product information of a product existing in the image 130.

3 is an exemplary block diagram for explaining the configuration of the image analysis unit 130 of FIG. 1. According to an embodiment, the image analysis unit 130 may include a first image analysis unit 131, a pre-processing unit 132, and one or more second image analysis units 133-1 ,,, n. .

The first image analysis unit 131 may receive and analyze the image acquired by the imaging unit 120. According to an embodiment, the first image analysis unit 131 may obtain large classification information and location information of a product existing in the image. As described above, the large classification information is based on the shape of the product, and the first image analysis unit 131 may perform an analysis operation for distinguishing, for example, a cup beverage from a bottle beverage. Since the acquisition of the large classification information is possible only by grasping the approximate shape of the product, the first image analysis unit 131 according to an embodiment of the present invention can acquire the large classification information by relatively simple analysis.

Also, the first image analysis unit 131 may acquire location information of each product. Accordingly, the first image analysis unit 131 according to an embodiment of the present invention may acquire large classification information of each product and location information corresponding thereto.

Referring to FIG. 3, the image analysis unit 130 is a pre-processing unit that performs pre-processing for separating each region where a product exists in an image, based on the large classification information and the location information obtained from the first image analysis unit 131 It may include (132). That is, the pre-processing unit 132 according to an embodiment may perform a task of separating a region of interest based on an analysis result of the first image analysis unit 131 from the image acquired by the imaging unit 120. Through such a pre-processing process, it is possible to improve processing efficiency in image analysis.

According to an embodiment, the pre-processing unit 132 may perform other pre-processing operations to improve image processing efficiency in addition to the region of interest separation operation. For example, the pre-processor 132 may further perform at least one of resizing and color regularization. According to an embodiment, the pre-processing unit 132 may sequentially perform a pre-processing operation of separation of regions of interest, resizing, and color normalization as illustrated in FIG. 4.

Referring back to FIG. 3, the image analysis unit 130 may include one or more second image analysis units 133-1 to 133-n. The one or more second image analysis units 133-1 to 133-n may correspond to each of the large classification groups. For example, if there is a large category of "bottle drink" and "canned drink", "bottle drink-second image analysis unit" and "canned beverage-second image analysis unit" corresponding to each large category may be separately provided. have.

The second image analysis units 133-1 to 133-n may obtain pre-processing by the pre-processing unit 132 to analyze images of each separated area to obtain sub-classification information of each product. According to one embodiment, the second image analysis units 133-1 to 133-n perform a more precise analysis processing than the analysis processing of the first image analysis unit 131. Accordingly, the second image analysis units 133-1 to 133-n may acquire at least one of the product name, product code, and shape information of the product indicated in the image, and obtain small classification information. As described above, in the present invention, the shape information refers to information that analyzes the appearance of a product more precisely than the shape information of a product for obtaining large classification information.

For example, different products “A Cup Ramen” and “F Cup Ramen” are recognized by the first image analysis unit 131 as the same “Cup Ramen” category. However, by the precise image analysis of the second image analysis unit 133-n corresponding to the large category of "Cup Ramen", product name, product code, or shape information is recognized and corresponds to "A Cup Ramen" and "F Cup Ramen". Each of the sub-classification information may be obtained.

Referring back to FIG. 1, the image analysis unit 130 receives and analyzes an image of a product display area from the photographing unit 120. According to an embodiment, the photographing unit 120 may acquire an image by photographing a product display area at a predetermined period. Accordingly, the image analysis unit 130 may periodically receive a real-time image. The image analysis unit 130 may periodically acquire real-time product information based on an image that is periodically acquired. Also, the image analysis unit 130 may store the obtained product information in the database 110 together with time information on which the corresponding product information was obtained.

As shown in FIG. 1, the product management system 100 according to an embodiment of the present invention compares product information obtained from the image analysis unit 130 with product information stored in the database 110, and thus the product is sold out. It further includes a determination unit 140 for determining at least one of whether or not there is a misaligned product.

According to an embodiment, the determination unit 140 compares the real-time product information periodically obtained by the image analysis unit 130 with product information stored in the database 110 at a time earlier than the corresponding product information, and thus, the product that is shorted. It can be judged whether there is. Therefore, by using the product management system 100 according to an embodiment of the present invention, it is not necessary to check directly whether there is a missing product in the product display area in the store, and it is possible to quickly check if a defect occurs.

Further, according to an embodiment, the reference location information of each product may be stored in the database 110. For example, franchise convenience stores, etc., provide merchants with strategic product display guidelines to increase sales. The manager of the affiliate store may input reference location information of each product to the product management system 100 according to the product display guidelines. The reference location information input by the administrator may be stored in the database 110.

The determination unit 140 may compare the location information included in the product information periodically acquired by the image analysis unit 130 with the reference location information stored in the database 110 to determine whether there is a misaligned product. . Therefore, even if the store worker does not directly check whether the product is misaligned, it is possible to quickly check if the misaligned product deviates from the reference location information.

1, the product management system 100 according to an embodiment of the present invention, based on the product information obtained from the image analysis unit 130, the display unit 150 for displaying the location information of each product It may further include. For example, the display unit 150 may be provided as a display on one wall surface of the store as shown in FIG. 13. Customers who visit the store can quickly and conveniently find and purchase products through location information of each product displayed on the display unit 150.

In addition, referring to FIG. 1, the product management system 100 according to an embodiment of the present invention may further include a new product registration unit 160. The new product registration unit 160 may register product information of a new product in the database 110. Also, the new product registration unit 160 may selectively update the second image analysis unit corresponding to the large classification group to which the corresponding new product belongs, from one or more second image analysis units 133-1 to 133-n. .

5 is an exemplary block diagram for explaining the configuration of the new product registration unit 160 of FIG. 1. Referring to FIG. 5, the new product registration unit 160 according to an embodiment may include an input unit 161, a new product photographing unit 162, a new product image analysis unit 163, and an update unit 164. .

The input unit 161 may receive information on a large category of new products. That is, when a new product is entered, the manager grasps the major classification information of the new product and inputs it to the input unit 161.

The new product photographing unit 162 acquires an image by photographing a new product. For example, the new product photographing unit 162 may be provided in a form as shown in FIG. 14. Referring to FIG. 14, a small photographing booth is provided, a turntable capable of placing a new product is provided, and a plurality of lights and cameras can be provided. Accordingly, the new product photographing unit 162 according to an embodiment may photograph a new product from multiple angles and obtain a multi-angle image of the new product.

The new product image analysis unit 163 may acquire the small classification information of the new product by analyzing the image acquired by the new product photographing unit 162. As described above, since the new product photographing unit 162 according to an embodiment can acquire a precise image from multiple angles with respect to the new product, the new product image analysis unit 163 analyzes the corresponding image and analyzes the corresponding image. Subclass information including at least one of a product name, a product code, and form information may be obtained.

The update unit 164 selects the group of the corresponding large category from the database 110 based on the large classification information input through the input unit 161, and the small classification obtained by the new product image analysis unit 163 to the data of the selected large classification group Information can be added. Also, the update unit 164 may update the second image analysis unit corresponding to the large classification information of the new product among the one or more second image analysis units 133-1 to 133-n. For example, based on the image acquired by the new product photographing unit 162, the second image analysis unit may be learned and updated.

According to an embodiment of the present invention, when product information of a new product is registered in the product management system 100 by the new product registration unit 160, the system is updated by hierarchical data management based on large classification information and small classification information It can shorten the time required. That is, since the update unit 164 does not need to update the entire image analysis unit 130, only the second image analysis unit 133-n corresponding to the large category information of the new product can be learned and updated, thereby updating the system. You can proceed efficiently.

Referring back to FIG. 1, the product management system 100 according to an embodiment of the present invention, when the determination unit 140 determines that there is a missing product or an incorrectly displayed product, a notification display unit 170 that displays a notification ) May be further included. For example, the notification display unit 170 may be provided in the form of an indicator light attached to each stand, as shown in FIG. 13.

6 is a view for explaining a method of analyzing the layout of a product display area, in order to obtain location information of a product according to an embodiment of the present invention. As illustrated in FIG. 6, a two-dimensional data matrix may be attached to both ends of each stage of each stage, and a product display area may be photographed from the front side of each stage using one or more cameras. The two-dimensional data matrix can be a QR code. By photographing a stand with a two-dimensional data matrix, it is possible to grasp the position of each stage in the image, that is, the position information on which the product will be displayed.

Hereinafter, with reference to FIGS. 8 to 12, a process in which an image photographing a product display area is analyzed by the product management system 100 according to an embodiment of the present invention will be described in detail.

For example, it is assumed that a product display area and a product exist as shown in FIG. 8. By the photographing unit 120, an image as illustrated in FIG. 8 may be acquired. The first image analysis unit 131 may receive the image of FIG. 8 and recognize large category information and location information of each product, as shown in FIG. 9. For example, the “left and right” large category information and “first stage No. 1” location information may be acquired for the leftmost product at the top. The first image analysis unit 131 may divide each product shown in the image of FIG. 8 into three main categories: “bottle drink”, “canned drink”, and “cup ramen”.

The pre-processing unit 132 may perform a pre-processing operation for separating a region of interest as shown in FIG. 10 based on the analysis result of the first image analysis unit 131.

For each image pre-processed by the pre-processing unit 132, different second image analysis units 133-1 to 133-n may be applied according to a large classification. For example, a "beverage-second image analysis unit" may be applied to an image of a product having large classification information of "bottle and drink". Similarly, "canned beverage-second image analysis unit" is applied to the image of the product having the "canned beverage" large classification information, and "cup ramen-second image analysis unit" is applied to the image of the product having the "cup ramen" large classification information. Can be applied.

Each of the second image analysis units 133-1 to 133-n may analyze the image of each product to recognize sub-classification information. For example, the “cup ramen-second image analysis unit” may precisely analyze four products having “class ramen” major classification information and recognize at least one of a product name, product code, and form information. As a result, it can be recognized that two of them are "A cup ramen" and two are "F cup ramen".

The results analyzed according to the processes of the first image analysis unit 131, the pre-processing unit 132, and the second image analysis units 133-1 to 133-n as described above are shown in the table shown in FIG. According to an embodiment, the image analysis unit 130 periodically acquires result data as in the example of FIG. 12 to quickly recognize a missing product by detecting a product that exists in the previous result data but does not exist in the current result data. You can.

Referring to FIG. 12, in the result data table, recognition reliability based on the reliability of the first image analysis unit 131 and the second image analysis unit 133-n that analyze the images of each product may be displayed together. The store manager may receive the result data in the form of the table of FIG. 12, and may also receive recognition reliability of each data.

13 is a view for illustratively explaining the display unit 150 and the notification display unit 170 of FIG. 1. The display unit 150 and the notification display unit 170 may be provided as a display or an indicator shown in FIG. 13 as described above. However, the present invention is not limited thereto. For example, the notification display unit 170 may be implemented as a display of a POS terminal that is easily seen by a store manager.

Each component of the product management system 100 according to an embodiment of the present invention may communicate with a wireless or wired network and transmit data to each other. According to one embodiment, each component of the product management system 100 may transmit data to each other by a short-range wireless communication technology such as Bluetooth (Bluetooth).

It is to be understood that the above embodiments are provided to help the understanding of the present invention, and do not limit the scope of the present invention, and various modified examples may belong to the scope of the present invention. For example, each component shown in the embodiments of the present invention may be implemented in a distributed manner, or conversely, multiple components may be implemented in combination. Therefore, the technical protection scope of the present invention should be determined by the technical spirit of the claims, and the technical protection scope of the present invention is not limited to the literary description of the claims, but is substantially equal in technical value. It should be understood that it extends to one category of invention.

100: product management system
110: database
120: filming unit
130: image analysis unit
131: first image analysis unit
132: pre-processing unit
133-1 to 133-n: second image analysis unit
140: judgment unit
150: display unit
160: new product register
161: input
162: New product shooting department
163: new product video analysis unit
164: update unit
170: notification display

Claims (17)

As a system for managing products based on the image of the product display area,
A database in which product information of each product is stored;
A photographing unit photographing the product display area to acquire an image;
An image analysis unit that analyzes an image acquired by the photographing unit and obtains product information of a product existing in the image; And
And a judging unit that compares product information obtained from the image analysis unit with product information stored in the database, and determines at least one of whether there is a missing product or a misaligned product,
The product information includes large classification information based on the shape of the product and location information indicating the location of the product in the product display area,
A product management system in which product information of each product stored in the database is first classified and stored in the same large classification group among products having the same large classification information.
According to claim 1,
The product information further includes sub-classification information including at least one of a product name, a product code, and form information,
A product management system, wherein product information of each product stored in the database is second-classified and stored based on the small classification information in a group classified based on the large classification information.
According to claim 2,
The image analysis unit,
A first image analysis unit that analyzes the image acquired by the photographing unit to obtain the large classification information and the location information; And
And a pre-processing unit for performing pre-processing for separating each region of the product from the image, based on the large classification information and the location information obtained by the first image analysis unit.
The method of claim 3,
The image analysis unit includes at least one second image analysis unit corresponding to each of the primary classified large classification groups,
The second image analysis unit, the product management system to obtain the sub-classification information by analyzing the image of each region separated by pre-processing.
According to claim 2,
The photographing unit periodically acquires an image by photographing the product display area,
The image analysis unit, a product management system, periodically acquires product information based on the acquired image, and stores the product information together with the obtained time information in the database.
The method of claim 5,
The determination unit compares product information periodically obtained by the image analysis unit with product information stored in the database at a time earlier than the corresponding product information to determine whether there is a missing product.
According to claim 2,
Reference location information of each product is stored in the database,
The determination unit compares the location information included in the product information obtained by the image analysis unit with the reference location information to determine whether there is a misaligned product.
According to claim 2,
A product management system further comprising a display unit that displays location information of each product based on the product information obtained from the image analysis unit.
The method of claim 4,
Further comprising a new product registration unit for registering the product information of the new product in the database,
The new product registration unit,
An input unit that receives information on a large category of the new product;
A new product photographing unit for acquiring an image by photographing the new product;
A new product image analysis unit that analyzes the image acquired by the new product photographing unit to obtain small classification information of the new product; And
And an update unit that selects a group of the corresponding large category from the database based on the large classification information input through the input unit and adds the small classification information obtained by the new product image analysis unit to the selected group data.
The method of claim 9,
The update unit updates the second image analysis unit corresponding to the large classification information of the new product, among the one or more second image analysis units corresponding to each of the primary classified large classification groups.
The method of claim 3,
The pre-processing unit further performs at least one of resizing and color regularization on an image of each separated area.
According to claim 1,
When the determination unit determines that there is a missing product or an incorrectly displayed product, the product management system further includes a notification display unit that displays a notification.
A method of managing a product using a database including product information of each product and a photographing unit that acquires an image by photographing a product display area,
The product information includes large classification information based on the shape of the product and location information indicating the location of the product in the product display area, and the product information of each product stored in the database is the same large classification between products having the same large classification information First classified into groups and stored,
The above method,
Acquiring an image by photographing the product display area by the photographing unit;
Analyzing the image obtained by the photographing unit and obtaining product information of a product existing in the image; And
And comparing product information of a product existing in the image with product information stored in the database to determine at least one of a missing product and a misaligned product.
The method of claim 13,
The product information further includes sub-classification information including at least one of a product name, product code, and form information, and the product information of each product stored in the database is within the group classified based on the large classification information, and the sub-classification. Secondly classified and stored based on information,
The step of obtaining product information of a product existing in the image is
Analyzing the image acquired by the photographing unit to obtain the large classification information and the location information; And
And performing pre-processing for separating each region in which the product exists in the image, based on the acquired large classification information and the location information.
The method of claim 14,
The step of obtaining product information of a product existing in the image is
The pre-processing is performed, further comprising the step of acquiring the sub-classification information by analyzing the image of each region, the product management method.
The method of claim 15,
Receiving information on a large category of new products;
Acquiring an image by photographing the new product;
Analyzing the acquired image and obtaining sub-classification information of the new product; And
And selecting a group of the corresponding large classification from the database based on the large classification information input through the input unit and adding the obtained small classification information to data of the selected group.
A computer-readable medium including a sequence of instructions for managing a product using a device including a database in which product information of each product is stored, and a photographing unit that acquires an image by photographing a product display area,
The product information includes large classification information based on the shape of the product and location information indicating the location of the product in the product display area, and the product information of each product stored in the database is the same large classification between products having the same large classification information First classified into groups and stored,
When executed by a computing device, the computing device,
Acquiring an image by photographing the product display area by the photographing unit,
Analyze the image obtained by the photographing unit to obtain product information of products existing in the image,
A computer-readable medium comprising sequences of instructions for comparing product information of a product present in the image with product information stored in the database to determine whether there is a missing product or a misaligned product. .

Images