JP2021056992A - Registration method of back-end commodity of self-checkout system - Google Patents

Abstract

To provide a method for registering a back-end commodity of a self-checkout system capable of supporting a user to register a new commodity in a self-checkout system only with a small amount of commodity photograph.SOLUTION: A method for registering a back-end commodity of a self-checkout system includes following steps: taking a commodity photo of a commodity to be registered; performing a deformation process to the commodity photo on the basis of default parameters for generating a plurality of simulated photos; comparing the commodity photo as well as the multiple simulated photos with the plurality of photos in a database respectively by a well-trained convolutional neural networks (CNN) model and generating recognition rate; adding the commodity photo to an identification package for completing registration operation of the commodity if it is determined that the recognition rate satisfies a threshold value; and requesting a user to take again another commodity photo if it is determined that the recognition rate does not satisfy the threshold value.SELECTED DRAWING: Figure 4

Description

The present invention relates to a self-checkout system, and more particularly to a method of registering a back-end product used in the self-checkout system.

As the times progress, more and more stores are introducing self-checkout systems that do not require employee checkouts. In addition, some countries are now developing unmanned stores that do not require any employees to reduce operating costs.

In general, a self-checkout system stores a large number of product photos in a database in advance, acquires images of products that consumers purchase with a camera in the self-checkout system at checkout, and compares the products with each other. Obtain related information (for example, product name, selling price, etc.) and execute the checkout operation.

As mentioned above, the self-checkout system must store a large number of product photos to allow for efficient contrast. These product photographs were mainly taken by the product manufacturer before the product was registered and stored in the self-checkout system. In general, the larger the number of product photographs of one product, the higher the recognition accuracy of that product. However, too many product photos lead to a shortage of storage capacity and an increase in the hardware cost of the self-checkout system.

In addition, the above product photographs are mainly taken manually by the product supplier and annotated (that is, marking which product each photograph represents), and a large number of photographs (that is, marking which product each photograph represents) are registered for each product registered with the manufacturer. For example, requesting that 500 or 1000 images be taken takes a considerable amount of time. Thus, the registration time for a new product can take days to weeks, which is very difficult for the use of self-checkout systems.

Furthermore, since product photography and annotation rely on human resources, the cost required for product registration increases as the number of required product photographs increases.

A main object of the present invention is to provide a method for registering a back-end product of a self-checkout system that can assist a user in registering a new product in the self-checkout system with only a small amount of product photographs.

In order to achieve the above object, the method for registering a back-end product according to the present invention is a back-end having at least a processor and a database for storing a trained CNN model and an identification package composed of a plurality of photographs. A step of taking a product photo of a product used in a server and to be registered, and a step of the processor transforming the product photo based on default parameters to generate a plurality of simulation photos of the product photo. The processor compares the product photograph and the plurality of simulation photographs with the plurality of photographs in the database by the CNN model, and generates a recognition rate based on the comparison result. When it is determined that the recognition rate satisfies the threshold value, the step of adding the product photograph to the identification package and completing the registration operation of the product, and the processor, the recognition rate does not satisfy the threshold value. If determined, it includes a step of requesting the user to retake the product photo.

The technical effect that the present invention can achieve with respect to the related technology is to simulate a large number of product photographs taken by the user by executing a self-authentication program based on the simulation photographs automatically generated by the processor. It is in the point that it can be done. In this way, the user can register a product in the self-checkout system by giving a high recognition rate to one product by taking a small amount of product photographs. As a result, the present invention effectively eliminates the trouble of registering the product in the self-checkout system, and significantly reduces the time and cost required for registering the product.

This is the first specific example of the schematic diagram of photography of the present invention. This is the first specific example of the block diagram of the registration system of the present invention. It is the schematic of the 1st product registration of this invention. It is the schematic of the 2nd product registration of this invention. It is the schematic of the 3rd product registration of this invention. It is the first specific example of the flowchart of the product registration of this invention. It is the schematic of the product photograph of this invention. This is the first specific example of the schematic diagram of the simulation photograph of the present invention. It is a 2nd specific example of the schematic diagram of the simulation photograph of this invention. It is a 3rd specific example of the schematic diagram of the simulation photograph of this invention. It is a 4th specific example of the schematic diagram of the simulation photograph of this invention. This is the first specific example of the flowchart of foreground separation of the present invention. This is the first specific example of the schematic diagram of the image processing of the present invention. It is the first specific example of the schematic diagram of the self-checkout system of this invention. This is a first specific example of the self-checkout flowchart of the present invention.

Preferred embodiments of the present invention will be described in detail with reference to the drawings.

With reference to FIGS. 1 and 2, FIG. 1 is a first specific example of a schematic diagram of photography of the present invention, and FIG. 2 is a first specific example of a block diagram of a registration system of the present invention. The present invention discloses a method for registering a back-end product of a self-checkout system (hereinafter, simply referred to as a registration method in the present specification), and the above registration method is mainly used for the back-end registration system 1 of a self-checkout system. Used for.

As shown in FIGS. 1 and 2, the registration system 1 mainly includes a back-end server 2, an image acquisition unit 3, and a display unit 4. In one embodiment, the back-end server 2 is arranged at the same location as the image acquisition unit 3 and the display unit 4, and is electrically connected to the image acquisition unit 3 and the display unit 4 via a cable.

In another embodiment, the back-end server 2 can be wirelessly connected to the image acquisition unit 3 and the display unit 4 by various wireless transmission methods. For example, the back-end server 2 may be a cloud server provided to the system developer, and the image acquisition unit 3 and the display unit 4 may be arranged in the office of a user (for example, a product supplier). Good. As a result, the user can use the registration method of the present invention by remotely connecting the back-end server 2.

As shown in FIG. 2, the back-end server 2 has at least a processor 21 and a database 22 electrically connected to the processor 21. The database 22 stores a trained convolutional neural network (CNN) model 221 and an identification package 222 composed of a plurality of photographs. In the present invention, the processor 21 mainly executes a product registration program (details will be described later) by the CNN model 221 and the identification package 222.

The principle and learning method of the CNN model 221 are general technical means in the art, and description thereof will be omitted here.

FIG. 3A is a schematic view of the first product registration of the present invention. FIG. 3A shows a first registration page 41 that the registration system 1 can display on the display unit 4. As shown in FIG. 3A, on the first registration page 41, the user can clearly see the product information of all the currently registered products. In one embodiment, the product information includes, but is not limited to, at least a product number (ID), barcode, product name, price, number of stored photographs, and the like. When the user triggers the product addition button on the first registration page 41, the user can execute the product registration program by adding the related information of the product to be registered.

Specifically, in order to complete the registration program for one product, one or more product photographs must be taken, and the content and number of the product photographs are sufficiently recognizable by the CNN model 221 (for example). , In the embodiment of FIG. 3A, the number of photographs of potato chips needs to be 5, the number of photographs of bread needs to be 8, and the number of photographs of beverages needs to be 3. When the products are registered, these product photos are added to the identification package 111. In other words, the identification package 222 of the present invention is composed of product photographs of a plurality of registered products, and the larger the number of registered products, the larger the number of photographs in the identification package 222. There is a direct proportional relationship.

In one embodiment, the processor 21 compresses the photographs in the identification package 222 according to the CNN model 221 to generate a plurality of identifiable vector values. In other words, the identification package 222 may contain, but is not limited to, a plurality of photographs and / or a plurality of compressed vector values.

As shown in FIG. 1, when the user intends to register a new product 5, the image acquisition unit 3 can take a product photograph of the product 5. The processor 21 compares the product photo with the plurality of photos in the identification package 222, and whether or not the user can complete the registration program of the product 5 by the recognition rate of the comparison result (Confidient rate). It is possible to determine whether it is necessary to take another picture of the product 5. In the present invention, the recognition rate means the probability that the product photograph of the product 5 can be recognized from the identification package 222.

In one embodiment, the processor 21 compresses the product photograph taken by the image acquisition unit 3 by the CNN model 221 to generate the corresponding vector value. Then, the processor 21 mainly compares the vector value corresponding to the product 5 with the plurality of vector values stored in the identification package 222 to obtain the above recognition rate. However, the above-described embodiment is merely a specific example of the present invention, and the present invention is not limited to the above-mentioned embodiment.

Subsequently, FIG. 3B is also referred to. FIG. 3B is a schematic view of the second product registration of the present invention. When the user takes a picture of the product 5, the processor 21 displays the second registration page 42 by the display unit 4, and sets the recommended placement position 421 of the product 5 within the acquisition range of the image acquisition unit 3. It can be displayed on the registration page 42 of 2. In this embodiment, the user can arrange the product 5 at the recommended arrangement position 421 and take a photograph of the product 5 by the image acquisition unit 3 so that the photograph taken further satisfies the requirements of the registration system 1. ..

In one embodiment, the acquisition range of the image acquisition unit 3 may be set to the size of the tray 50 for currently loading the product 5, but is not limited thereto.

In one embodiment, the processor 21 may also set the required number of product photographs based on various parameters (eg, product shape, product category, recognition difficulty, etc.). When the user takes one product photo, the processor 21 can determine whether or not the current number of product photos satisfies a predetermined number (for example, five). If the number of product photos has not yet reached the predetermined number, the processor 21 displays the next recommended placement position 421 on the second registration page 42 according to the setting until the number of product photos satisfies the predetermined number. The user can be guided to take the next product photo that is different from the previous product photo.

Subsequently, FIG. 3C is also referred to. FIG. 3C is a schematic view of the third product registration of the present invention. When the user executes the registration program of the new product 5, the registration system 1 can display the third registration page 43 on the display unit 4. As shown in FIG. 3C, the user inputs product information such as the product number (ID), barcode, product name, and price of the product 5 in the left half column of the third registration page 43, and the third registration The product photograph 6 of the above product 5 can be imported into the right half column of the page 43.

After importing the product photo 6 of the product 5, the user can trigger the next button on the third registration page 43 to cause the registration system 1 to execute the self-authentication program for the product photo 6. After the self-authentication program is successful, the registration system 1 can add the product photo 6 to the identification package 222 of the database 22 to update the identification package 222 and complete the registration program for the new product 5.

Specifically, the self-authentication program described above mainly confirms whether or not the identification package 222 remains in a stable state after the new product photo 6 is added to the existing identification package 222, and also confirms whether or not the identification package 222 remains in a stable state, and also a CNN model. This is for confirming whether or not the recognition rate higher than the threshold value can still be obtained when the 221 performs the identification operation of each product based on the identification package 222 (details will be described later).

Subsequently, FIG. 4 is referred to. FIG. 4 is a first specific example of the flow chart for product registration of the present invention, and mainly shows specific execution steps of the registration method of the present invention.

As shown in FIG. 4, in order to execute the registration program for one product, the user first places the product 5 to be registered in one shooting position (for example, on the tray 50 shown in FIG. 1). At least one product photograph 6 of the product 5 is taken by the image acquisition unit 3 of the registration system 1 (step S10). Specifically, in step S10, the processor 21 of the back-end server 2 displays the recommended arrangement position 421 of the product 5 within the acquisition range of the image acquisition unit 3 on the display unit 4, and the product 5 is recommended as described above. By guiding the user to arrange the product at the arrangement position 421, it is possible to facilitate the taking of the product photograph 6 by the image acquisition unit 3.

In the product photo 6, product information such as the product number (ID), product name, barcode, and price of the corresponding product 5 is recorded by the setting of the third registration page 43 described above. Not limited to these.

In the present invention, the processor 21 can set a predetermined number of necessary product photographs according to parameters such as the shape of the product, the category of the product, and the difficulty of recognition. After step S10, the processor 21 determines whether or not the number of product photographs 6 currently taken satisfies a predetermined number (step S12). If the number of product photographs 6 does not meet the predetermined number, the processor 21 returns to step S10 and requests the user to take the next product photograph 6.

In the present embodiment, the purpose of increasing the number of product photographs 6 is to predict different appearances of the products 5, so that the processor 21 needs to ensure that the different product photographs 6 have different contents. is there. For example, after returning to step S10, the processor 21 displays the next recommended arrangement position 421 of the product 5 within the acquisition range of the image acquisition unit 3 by the display unit 4, so that the product photograph 6 having different contents can be displayed. The user can be guided to shoot.

After the user takes a plurality of product photographs 6 of the product 5 and the number of the product photographs 6 satisfies the above-mentioned predetermined number, the processor 21 uses the plurality of product photographs 6 to perform the transformation process and the step of the subsequent step S14. By simultaneously executing the self-authentication program of S16, the recognition rate obtained in step S20 can be improved.

In another embodiment, the predetermined number of the product photographs is one. In this embodiment, step S12 does not necessarily have to exist. In order to facilitate understanding, an example of taking only one product photograph will be described below.

Next, the back-end server 2 acquires the photographed product photograph 6 and transforms the product photograph 6 based on the default parameter 223 stored in the database 22 in advance to generate a plurality of simulation photographs (step). S14). In one embodiment, the default parameter 223 may include at least one of the size, placement position, rotation angle, light and shadow state, shadow position, reflection effect, product features, etc. of the product 5. , Not limited to these.

5A-5E are referenced at the same time. 5A is a schematic view of a product photograph of the present invention, and FIGS. 5B to 5E are first and fourth specific examples of the schematic diagram of the simulation photograph of the present invention, respectively.

FIG. 5A shows a product photograph 6 taken by the user in step S10 described above (in FIG. 5A, bread is taken as an example). In step S14 described above, the processor 21 deforms the product photograph 6 based on the default parameter 223 to automatically generate a plurality of simulation photographs 61 to 64 as shown in FIGS. 5B to 5E.

In the first embodiment, the processor 21 generates a simulation photograph 61 as shown in FIG. 5B by changing the number, distribution, and position of the product features (for example, sesame seeds) of bread. In the second embodiment, the processor 21 generates a simulation photograph 62 as shown in FIG. 5C by changing the rotation angle of the pan. In the third embodiment, the processor 21 generates the simulation photograph 63 as shown in FIG. 5D by changing the shading position of the pan. In the fourth embodiment, the processor 21 generates a simulation photograph 64 as shown in FIG. 5E by changing the reflection effect of the pan.

The above embodiment is merely a specific example of the present invention. In order to improve the identification accuracy, the processor 21 may generate about 500 to 1000 simulation photographs 61 to 64 for one product photograph 6 according to different parameters, but the present invention has the above-mentioned number. Not limited to.

One technical feature of the present invention is that the processor 21 automatically generates a plurality of simulation photographs 61 to 64 at the same time in addition to the product photograph 6 taken by the image acquisition unit 3 by the user, and executes the self-authentication program. It is to be. More specifically, the present invention simulates a large number of conventional product photographs 6 in which the product 5 is manually photographed by the automatically generated simulation photographs 61 to 64.

Due to the above-mentioned technical features, the user can give the new product 5 a recognition rate higher than the threshold in the identification package 222 by taking only a few product photographs 6, and the registration program of the new product 5 (Preferably, the registration program of the product 5 can be completed by taking only one product photograph 6). As a result, the registration cost of the product can be effectively saved, and the time required for registration can be significantly shortened.

Returning to FIG. 4, after step S14, the processor 21 executes the self-authentication program in order to compare the product photograph 6 and the plurality of simulation photographs 61 to 64 with the identification package 222 in the database 22 by the CNN model 221. Execute (step S16). In one embodiment, the CNN model 221 contrasts the product photograph 6 and the plurality of simulation photographs 61-64 with the plurality of photographs recorded on the identification package 222, respectively. In another embodiment, the CNN model 221 compresses the product photo 6 and the plurality of simulation photos 61-64 into identifiable vector values, and contrasts these vector values with the plurality of vector values in the identification package 222, respectively. To do.

For example, when 2000 photographs are stored in the identification package 222 and the processor 21 generates 500 simulation photographs 61 to 64 with respect to the product photograph 6, in the present invention, the processor 21 is the 501. The photographs 6, 61 to 64 are compared one by one with the 2000 photographs in the identification package 222, respectively. In the comparison process, the processor 21 continuously determines whether or not the comparison is completed (step S18), continuously executes step S16 until the comparison is completed, and when the comparison is completed, based on the comparison result. To generate a recognition rate (step S20).

In the present embodiment, the above-mentioned recognition rate means the probability that the above-mentioned product photographs 6 and simulation photographs 61 to 64 are normally recognized in the existing identification package 222 of the database 22. Specifically, the high recognition rate means that the product photograph 6 is not easily confused with another photograph (that is, another product) in the identification package 222. On the contrary, a low recognition rate means that the product photograph 6 is easily confused with another photograph (that is, another product) in the identification package 222.

Next, the processor 21 determines whether or not the recognition rate is larger than the preset threshold value (step S22). If the recognition rate is greater than the threshold (eg, 95%), it indicates that the product photo 6 is not confused with the other photos in the identification package 222, and even if the product photo 6 is added to the identification package 222, the identification package 222. Recognition rate remains stable. Therefore, the processor 21 can add the product photograph 6 to the identification package 222 in order to update the contents of the identification package 222 (step S24).

Conversely, if the recognition rate is not greater than the threshold, it indicates that the currently acquired product photo 6 may be confused with other photos in the identification package 222, and the product photo 6 is added to the identification package 222. Then, the performance (that is, recognition accuracy) of the identification package 222 may be affected. In this case, the processor 21 determines that the product photograph 6 cannot be used, deletes the photographed product photograph 6, and issues an instruction message requesting the user to re-photograph the product photograph 6 by the display unit 4. (Step S26).

If either the product photograph 6 or the simulation photographs 61 to 64 is not normally recognized, the processor 21 can determine that the CNN model 221 has a recognition problem. At this time, the processor 21 can report an error to the person in charge of the back-end server 2 and submit unrecognized product photographs 6 and / or simulation photographs 61 to 64. Thereby, the person in charge can further learn the CNN model 221 by the product photograph 6 and the simulation photographs 61 to 64 in order to fine-tune the CNN model 221.

As described above, in the present invention, one or more product photographs 6 of the new product 5 are added to the identification package 222 and do not affect the recognition rate of the identification package 222 (that is, the recognition rate is larger than the threshold value). ), The registration program of the product 5 is permitted and completed (step S28). Further, in the present invention, the self-authentication program before adding the product photo 6 to the identification package 222 is realized by a plurality of automatically generated simulation photos 61 to 64, so that the user can display a large number of product photos 6. It is possible to effectively save the time and labor cost required for taking a picture by oneself.

As described above, since the image acquisition unit 3 of the present invention mainly captures the entire acquisition range, the photographed photograph includes a foreground (for example, one or more products 5) and a background (for example, one or more products 5). The image of the tray 51) is included at the same time. In order to acquire the product photograph 6 that can be used for comparison by the CNN model 221, the processor 21 first executes a foreground separation program on the photograph taken by the image acquisition unit 3 to specify the product 5. After acquiring the image, this specific image may be referred to as the above-mentioned product photograph 6.

In one embodiment, the processor 21 can execute a foreground separation program primarily by means of a Salience Model and a Depth Segmentation Algorithm. The above-mentioned saliency model and depth division algorithm are general technical means in the art, and description thereof will be omitted here.

Subsequently, reference is made to FIGS. 6 and 7. FIG. 6 is a first specific example of the flowchart of the foreground separation of the present invention, and FIG. 7 is the first specific example of the schematic diagram of the image processing of the present invention. With reference to FIG. 6, how the processor 21 of the present invention acquires a specific image of the product 5 will be further described.

In the present embodiment, after the processor 21 acquires the product photograph 6 taken by the image acquisition unit 3, first, the outline of one object (that is, the product 5) is identified in the product photograph 6 (step S40), and this The product 5 is positioned by generating the mask 71 of the product 5 based on the contour (step S42). By generating the mask 71, the processor 21 can acquire the position of the product 5 in the product photograph 6.

Specifically, as shown in FIG. 7, the mask 71 is generated along the contour of the product 5 in the product photo 6 to describe the position of the contour of the product 5 in the entire product photo 6. .. The algorithm described above can obtain the position of the mask 71 and acquire the image in the mask 71 to generate a specific image of the product 5 (step S44).

In the present embodiment, the processor 21 updates the product photograph 6 with the specific image generated in step S44. In step S14 and step S16 of FIG. 4 described above, the processor 21 generates a plurality of simulation photographs 61 to 64 based on the updated product photograph 6 and compares them with the identification package 222. As a result, the processor 21 can obtain a more accurate comparison result.

In one embodiment, the processor 21 further generates and displays a mark frame 72 that can completely cover the product 5 based on the mask 71 (step S46). The mark frame 72 may be a polygonal frame (for example, a rectangle, a triangle, a rhombus, etc.). By generating and displaying the mark frame 72, the user can quickly and clearly confirm the position of the product 5 from the product photograph 6 on the display unit 4, and can perform the subsequent processing operation more efficiently.

As described above, when registering a new product 5, the manufacturer mainly uploads the product information of the product 5 and the product photo 6 to the back-end server 2, and finally adds the product photo 6 to the identification package 222. , Complete the registration program for product 5. On the other hand, after the store displays the product 5, the front-end self-checkout system (self-checkout system 1 shown in FIG. 8) is connected to the back-end server 2 by a network, and the CNN model 221 in the back-end server 2 is connected. And the identification package 222 can realize the identification operation required for the self-checkout program.

In another embodiment, the user can also pre-import the CNN model 221 and the identification package 222 into the self-checkout system 1. As a result, the self-checkout system 1 does not need to be connected to the back-end server 2 by the network, and can realize the self-checkout program by itself. After the new product 5 is registered and the backend server 2 updates the identification package 222, the user can use the updated identification package 222 so that the self-checkout system 1 can support the newly registered product 5. The self-checkout system 1 must be updated based on this.

Subsequently, reference is made to FIGS. 8 and 9. FIG. 8 is a first specific example of a schematic diagram of the self-checkout system of the present invention, and FIG. 9 is a first specific example of a flowchart of the self-checkout system of the present invention.

As shown in FIGS. 8 and 9, when the consumer purchases the purchased product 84 through the self-checkout system 8 in the store, the consumer purchases mainly below the camera 81 mounted on the self-checkout system 8. The product 84 is placed, and the recognition target image 85 of the purchased product 84 can be acquired by the camera 81 (step S50). Next, the self-checkout system 8 executes a foreground separation program on the acquired recognition target image 85 (step S52).

In the present embodiment, the self-checkout system 8 mainly performs a foreground separation program on the recognition target image 85 by a method similar to that of FIG. 6 described above. Specifically, the self-checkout system 8 first identifies the contour of the purchased product 84 in the recognition target image 85, and then wears a mask of the purchased product 84 (for example, the mask 71 shown in FIG. 7) based on the contour. Generate and position the purchased product 84. The self-checkout system 8 also acquires an image in the mask to generate a purchased product image that is completely focused on the purchased product 84. Finally, the self-checkout system 8 updates the recognition target image 85 based on the purchased product image so that subsequent identification actions can focus on the purchased product 84, resulting in improved recognition accuracy. Let me.

When the recognition target image 85 captured by the camera 81 includes a plurality of purchased products 84 at the same time, the self-checkout system 8 puts the above mask on each purchased product 84 in step S52. It is generated, positioning is performed for each purchased product 84, and a plurality of recognition target images 85 are generated after the update. As a result, the self-checkout system 8 can accelerate the progress of the self-checkout program by simultaneously recognizing the plurality of purchased products 84 by the camera 81.

As shown in FIG. 8, the self-checkout system 8 can display the updated recognition target image 85 on the screen 82 and show it to the consumer. Further, the self-checkout system 8 generates a mark frame 72 that can completely cover the purchased product 84 based on the mask when executing the foreground separation program, and displays the recognition target image 85 and the mark frame 72 on the screen 82. It can also be displayed on the top at the same time. This allows the consumer to see the purchased product 84 more quickly and clearly.

Returning to FIG. 9, after step S52, the self-checkout system 8 compares the image 85 to be recognized with the identification package 222 in the database 22 by the CNN model 221 and generates a comparison result (step S54). In one embodiment, the self-checkout system 8 directly contrasts the recognition target image 85 with a plurality of photographs in the identification package 222 by means of the CNN model 221. In another embodiment, the self-checkout system 8 compresses the recognition target image 85 into an identifiable vector value by the CNN model 221 and compares this vector value with a plurality of vector values in the identification package 222.

As shown in FIG. 3C described above, at least the product number (ID) of the corresponding product 5 is recorded in each product photograph 6 added to the identification package 222. After step S54, the self-checkout system 8 can obtain the contrast result. In this comparison result, one or more photographs in the identification package 222, which are the closest to the recognition target image 85, are recorded. Therefore, after step S54, the self-checkout system 8 completes the identification operation of the purchased product 84 by acquiring the product number corresponding to the matching photograph identified based on the comparison result (step S56). be able to.

After step S56, the self-checkout system 8 acquires product information such as the bar code, product name, and price of the purchased product 84 based on the product number (step S58), and manages the product information at the point-of-sale information of the front end. (Point of Sale, POS) The product is transferred to the register 83, the product information is displayed on the screen 82 by the POS register 83 to be confirmed by the consumer, and the checkout operation is executed by the POS register 83 (step S60). In this way, the consumer can complete the self-checkout task, which is quite convenient.

In another embodiment, the self-checkout system 8 obtains the product number corresponding to the purchased product 84 after step S56 and transmits the product number directly to the POS register 83 in a data format readable by the POS register 83. Thereby, the conventional POS register 83 can simulate the technical means for directly acquiring the product number and the code from the barcode scanner. In the present embodiment, the POS register 83 further searches the internal database based on the product number and the code after receiving the product number and the code, and acquires and displays the product information such as the product name and the price of the purchased product 84. can do.

According to the above-described embodiment, the store does not need to change the software and hardware architecture of the conventional POS register 83, and the cost can be effectively reduced. Further, in the processing method in which the self-checkout system 8 acquires the product information of the purchased product 84 from the comparison result of the product photo 6 and transmits it to the POS register 83, the conventional store uses a bar code scanner to display the product information of the purchased product 84. It is possible to simulate a processing method of acquiring and transmitting to the POS register 83, and as a result, the POS register 83 enables a checkout operation for a product (for example, pan) having no bar code, and the convenience is improved.

As described above, in the present invention, by automatically generating a large number of simulation photographs based on one product photograph, the user can take a small amount of product photographs and have a high recognition rate for one product. The product registration program can be completed normally. In this way, the time and labor cost required for product registration can be effectively saved.

The above description is merely a preferable specific example of the present invention and does not limit the scope of claims of the present invention. Therefore, any equivalent modification to which the content of the present invention is applied is the present invention. Included in the range.

1 ... Registration system 2 ... Backend server 21 ... Processor 22 ... Database 221 ... Convolutional neural network model 222 ... Identification package 223 ... Default parameter 3 ... Image acquisition unit 4 ... Display unit 41 ... First registration page 42 ... Second Registration page 421 ... Recommended placement position 43 ... Third registration page 50 ... Tray 5 ... Product 6 ... Product photos 61-64 ... Simulation photos 71 ... Mask 72 ... Mark frame 8 ... Self-checkout system 81 ... Camera 82 ... Screen 83 ... POS register 84 ... Purchased product 85 ... Recognition target image 86 ... Product information S10-S28 ... Registration step S10-S28 ... Foreground separation step S10-S28 ... Checkout step

Claims (10)

A back-end product of a self-checkout system used in a back-end server that has at least a processor and a database that stores a trained Convolutional Neural Networks (CNN) model and an identification package consisting of multiple photos. It is a registration method of
The image acquisition unit takes a product photograph in which the product number corresponding to the product to be registered is recorded, and the step (a).
The processor transforms the product photo based on the default parameters to generate a plurality of simulation photos, and the step (b).
The processor executes a self-authentication program that compares the product photograph, the plurality of simulation photographs, and the plurality of photographs in the database with the neural network model, and generates a recognition rate based on the comparison result. Step (c) and
When the processor determines that the recognition rate satisfies the threshold value, the step (d) of adding the product photograph to the identification package and completing the registration program of the product.
When the processor determines that the recognition rate does not satisfy the threshold value, the step (e) of issuing an instruction message for retaking the product photograph is performed.
How to register back-end products for self-checkout systems, including. After step (a)
A step (a11) of executing a foreground separation program on the product photo in order to acquire a specific image of the product from the product photo.
The method for registering a back-end product of the self-checkout system according to claim 1, further comprising a step (a12) of updating the product photo based on the specific image. The method for registering a back-end product of the self-checkout system according to claim 2, wherein in the step (a11), the foreground separation program is executed by a salience model and a depth segmentation algorithm. The step (a11) is
In the step (a111) of identifying the outline of the product in the product photograph,
In the step (a112) of generating a mask of the product based on the contour and positioning the product,
The method for registering a back-end product of a self-checkout system according to claim 2, comprising the step (a113) of acquiring an image in the mask and generating the specific image. The step (a11) is
The method for registering a back-end product of a self-checkout system according to claim 4, further comprising a step (a114) of generating a mark frame that can cover the product based on the mask. Before step (a)
The method for registering a back-end product of the self-checkout system according to claim 2, further comprising a step (a01) of displaying the recommended placement position of the product within the acquisition range of the image acquisition unit by the display unit. A step (f) of determining whether or not the number of product photographs satisfies a predetermined number by the processor, and
When the number of the product photographs does not satisfy the predetermined number by the display unit, the claim includes a step (g) of displaying the next recommended arrangement position of the product and executing the step (a) again. Item 6. The method of registering a back-end product of the self-checkout system according to item 6. The self-checkout system according to claim 1, wherein the default parameters include at least one of the product size, placement position, rotation angle, light and shadow conditions, shadow position, reflection effect, and product features. How to register backend products. Step (h), in which the front-end self-checkout system acquires the image to be recognized by the camera,
The step (i), in which the self-checkout system uses the neural network model to compare the image to be recognized with the plurality of photographs in the database and generate a comparison result.
A step (j) of acquiring the product number corresponding to one matching photograph identified based on the comparison result and completing the identification operation of the purchased product.
Step (k) of acquiring the product name and price of the corresponding product based on the product number, and
The method for registering a back-end product of the self-checkout system according to claim 1, further comprising a step (l) of transmitting the product name and the price to a POS register of the front end to perform a checkout operation. After the step (h),
The step (h1) of identifying the contour of the purchased product in the recognition target image, and
A step (h2) of generating a mask of the purchased product based on the contour and positioning the purchased product, and
In the step (h3) of acquiring the image of the mask and generating the image of the purchased product,
A step (h4) of generating and displaying a mark frame that can cover the purchased product based on the mask, and
The method for registering a back-end product of the self-checkout system according to claim 9, further comprising a step (h5) of updating the recognition target image based on the purchased product image.

Images