WO2019184646A1

WO2019184646A1 - Method and device for identifying merchandise, merchandise container

Info

Publication number: WO2019184646A1
Application number: PCT/CN2019/076408
Authority: WO
Inventors: 季涛; 张结龙; 吕秀凤; 戴江
Original assignee: 合肥美的智能科技有限公司; 合肥华凌股份有限公司
Priority date: 2018-03-27
Filing date: 2019-02-28
Publication date: 2019-10-03
Also published as: CN108549851A; CN108549851B

Abstract

Provided are a method and device for identifying merchandise, and a merchandise container. The method for identifying merchandise in a merchandise container comprises: receiving a merchandise identification instruction of a target merchandise container; collecting interior merchandise images of the target merchandise container and acquiring changes in the weights of various merchandise in the target merchandise container, where the interior merchandise images comprise all of the merchandise currently in the target merchandise container; and performing image identification with respect to the interior merchandise images to obtain the types of various merchandise of which the quantity changed in the target merchandise container; and determining, on the basis of the types of various merchandise that changed in the target merchandise container and the corresponding changes in the weights, the quantity of the various merchandise of which the quantity changed in the target merchandise container. The present application allows the accurate and quick identification of the merchandise in the merchandise container and effectively saves labor costs.

Description

Product identification method and device, container

cross reference

The present application is hereby incorporated by reference in its entirety in its entirety in its entirety in the entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire all

Technical field

The present application relates to the field of container technology, and in particular, to a method and device for identifying goods and a container.

Background technique

Intelligent containers generally refer to smart containers that sell goods in the form of self-service vending machines, usually covering an area of about 1-2 square meters, generally using cashless transactions and common currency transactions. For example: self-service orange machine, self-service coffee machine, self-service ice cream machine, self-service lunch machine, traditional unmanned self-service vending machine, etc.; due to the unmanned characteristics of smart containers, replenishment personnel are required to periodically check for replenishment, so that Intelligent containers can meet the user's purchase needs at any time.

At present, the existing methods for identifying goods in smart containers usually require the replenishing personnel to register the product information before replenishment, and place different types of goods in the corresponding positions according to the original mark during the replenishment process. Record the update of the goods, and enter the goods update status into the container commodity management system corresponding to the smart container by manual manual entry or external file import after the replenishment is completed.

However, the existing methods for identifying goods in smart containers have poor accuracy, and it is necessary to check the goods one by one when replenishing, which requires a lot of manpower and time cost.

Summary of the invention

In view of the problems in the prior art, the present application provides a method and device for identifying goods and a container, which can accurately and quickly identify goods in an intelligent container, and only need to classify the goods, without checking the quantity. Effectively saves labor costs.

To solve the above technical problem, the present application provides the following technical solutions:

In a first aspect, the present application provides a method for identifying a product, the method for identifying a product comprising:

Receiving a product identification instruction of the target container;

Collecting an internal product image of the target container and obtaining a weight change value of each item in the target container, wherein the internal product image includes all the items in the current target container;

And performing image recognition on the internal goods image to obtain types of various types of goods in the target container;

Determining the quantity of each item in the target container that changes in quantity according to the type of the various items in the target container and the corresponding weight change value.

Further, the method for identifying goods further includes:

Update the real-time information of the goods of the target container based on the type and quantity of various types of goods in the target container.

Further, the item identification instruction of the receiving target container includes:

Receiving user identification information or transaction information for the target container;

Sending a door opening instruction to the target container, so that the target container opens the door lock of the door according to the door opening instruction;

And receiving a goods identification instruction sent by the target container after the door is opened and closed.

Further, the receiving user identification information or transaction information for the target container includes:

Receiving user identification information or transaction information sent by the user in the application APP after connecting to the application APP corresponding to the target container by using the terminal device to scan the two-dimensional code;

Wherein the two-dimensional code is disposed on an outer wall of the target container.

Receiving user identification information or transaction information acquired by means of face image recognition;

The method for recognizing a face image includes: acquiring an image of a face of a user located outside the target container through an image collection device disposed on an outer wall of the target container, and performing a face image on the face image Identification.

Further, the collecting an internal product image of the target container and obtaining a weight change value of each item in the target container, including:

Controlling, by a plurality of image collection devices disposed in the target container, an internal product image of the target container, wherein different image collection devices are configured to collect images of goods on different shelves in the target container, and all shelves The image of the goods on the top constitutes the image of the internal goods;

And controlling the weight sensors respectively disposed on the respective shelves in the target container to collect the weight change values of the goods on the respective shelves, wherein the same type of goods are placed on the same shelf.

Further, the image recognition of the internal goods image is performed to obtain types of various types of goods in the target container, including:

Performing image recognition on the internal goods image to obtain image features of the quantity in the target container;

And determining, according to the image feature of the product and the preset product image feature library, the type of the item in the target container that changes in quantity;

The correspondence between the type of the product and the image feature is stored in the image image feature library.

Further, determining, according to the type of each type of goods and the corresponding weight change value of the quantity in the target container, determining the quantity of each item in the target container that changes in quantity, including:

Calculating the initial value of the quantity of each item in the target container according to the type of the goods in the target container and the corresponding weight change value, and the pre-acquired item weight library;

And determining, according to the initial value of the quantity of the goods of the various types, the quantity of each type of goods in the target container;

The cargo weight library stores a correspondence between the type of the product and the weight of the product.

Further, the determining, according to the initial value of the quantity of the goods of the various types, the quantity of each type of goods in the target container, including:

If the initial value of the quantity of the goods is an integer, the initial value of the quantity is directly determined to determine the quantity of each item in the target container;

If the initial value of the quantity of the goods is a non-integer, the quantity of each item in the target container is changed according to the value of the fractional part of the quantity initial value.

Further, the determining, according to the value of the fractional part of the initial value, the quantity of each item in the target container that changes in quantity, including:

Determining whether the value of the fractional part of the initial value is within an allowable error range;

If yes, the initial value of the quantity is rounded off, and the rounded value determines the quantity of each item in the target container that changes in quantity;

If not, the goods confirmation command is sent to the user's terminal device.

Further, the real-time information of the goods of the target container is updated, including:

Updating the real-time information of the goods of the target container in the application APP corresponding to the target container.

Further, the method for identifying goods further includes:

If the detection knows that the quantity of a certain type of goods in the target container is lower than the corresponding replenishment threshold, the replenishment notification information is sent to the replenishing terminal device.

In a second aspect, the present application provides a goods identification system, the goods identification system comprising:

a goods identification instruction receiving module configured to receive a goods identification instruction of the target container;

An image and weight collection module configured to collect an internal product image of the target container and obtain a weight change value of each item in the target container, wherein the internal product image includes all items in the current target container ;

An increase or decrease item type identification module configured to perform image recognition on the internal item image to obtain a type of the item in the target container that has changed in quantity;

The increase/decrease item quantity identification module is configured to perform image recognition on the internal goods image to obtain types of various types of goods in the target container.

In a third aspect, the present application provides an electronic device including a memory, a processor, and a computer program stored on the memory and operable on the processor, and the step of implementing the product identification method when the processor executes the program .

In a fourth aspect, the present application provides a computer readable storage medium having stored thereon a computer program that, when executed by a processor, implements the steps of the item identification method.

In a fifth aspect, the application provides a container that communicates with the electronic device.

According to the above technical solution, the method for identifying goods provided by the present application receives an item identification instruction of a target container; collects an internal product image of the target container, and obtains a weight change value of each item in the target container, wherein The internal goods image includes all the goods in the current target container; and image recognition of the internal goods image to obtain the types of the various types of goods in the target container; according to the change in the target container The type of each type of goods and the corresponding weight change value determine the quantity of each type of goods in the target container, which enables accurate and rapid identification of the goods in the container, and only needs to classify the goods. That is, without checking the quantity, the labor cost is effectively saved, and the intelligence degree and application reliability of the container are improved, so that the user experience is improved.

DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings to be used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description are Some embodiments of the present application can also obtain other drawings based on these drawings without departing from the prior art by those skilled in the art.

1 is a schematic flow chart of a method for identifying a product in the first embodiment of the present application;

2 is a schematic diagram of communication between a server, a weight sensor, a product identification instruction transmitting unit, and an image capturing device of the present application;

3 is a schematic flow chart of a method for identifying a product including step 500 of the present application;

4 is a schematic flow chart of step 100 in the method for identifying goods in the present application;

5 is a schematic flow chart of a method for identifying a product including step 600 in the present application;

6 is a schematic diagram of interaction between a server, a terminal device, and a container in the present application;

7 is a logic flow chart of a method for identifying a product in an application example of the present application;

8 is a schematic structural diagram of a product identification system in Embodiment 2 of the present application;

9 is a schematic structural diagram of an electronic device according to Embodiment 3 of the present application;

FIG. 10 is a schematic structural view of a container in Embodiment 5 of the present application.

detailed description

The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present application. It is a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

In view of the problems in the prior art, the present application provides a method and device for identifying goods and a container. It can be understood that the method for identifying the goods in the container described in the embodiments of the present application is applicable to various types of unmanned intelligent containers, such as a coin-operated vending machine installed in a subway station or a large shopping mall, and is also suitable for setting. Unmanned-type intelligent containers in office spaces and unmanned convenience stores, that is, the method for identifying goods in the smart containers is not only suitable for the automatic vending machine for falling objects, but also for opening the door. An unattended smart container, the goods identification method collects an internal product image of the target container by receiving a product identification instruction of the target container, and obtains a weight change value of each item in the target container, and The internal goods image is image-recognized, and the types of the goods in the target container are changed; the type of the goods in the target container and the corresponding weight change value are determined, and the target container is determined. The quantity of each type of goods whose quantity has changed is the type of each type of goods whose quantity changes in the target container, and the goods in the container are completed. Identification of update results. The product identification method provided by the present application can accurately and quickly identify the goods, and only needs to put the goods into categories, without checking the quantity, effectively saving the labor cost, and improving the intelligence degree and application reliability of the container. Sex, so the user experience is improved. The present application will be explained in detail below through the first to fifth embodiments.

Embodiment 1 of the present application provides a specific implementation manner of a method for identifying a product. Referring to FIG. 1, the method for identifying a product specifically includes the following contents:

Step 100: Receive a product identification instruction of the target container.

In this embodiment, referring to FIG. 2, the execution body of the goods identification method is a server, and the server may be disposed inside the container, or may remotely manage a plurality of containers, and further set in the container. The product identification instruction transmitting unit and the image collecting device are respectively connected to the server by communication, and the image capturing device may be a camera, and the product identification instruction is The transmitting unit is configured to transmit a goods identification instruction to the server after receiving the item identification instruction.

In a specific example, the item identification command sending unit may be a displacement sensor, and the two parts of the displacement sensor are respectively disposed on the cabinet door of the container and disposed on the inner wall of the container near the device. In the second device, when the door is opened, the displacement between the device 1 and the device 2 exceeds a preset threshold, the displacement sensor recording cabinet door is opened once, when the displacement between the device 1 and the device 2 exceeds the pre- After the threshold is set, after the displacement between the device 1 and the device 2 is less than or equal to the preset threshold, the displacement sensor recording cabinet door is closed once, and the displacement sensor continuously detects that the cabinet door of the container occurs once. After being turned on and off once, the displacement sensor transmits a goods identification instruction to the server.

Of course, in the case where the cabinet door of the container is opened once and closed once, the door of the container is opened, but the change of the goods is not sent. Therefore, the item identification instruction can be understood as follows: as long as the container is inside The situation in which the goods are suspected to have changed, that is, the goods identification instruction is sent to the server, so that the server can timely know the change of the goods of the container on the one hand, and on the other hand, the server receives the goods identification instruction every time. Only after learning about the changes in the goods of the container, effectively reducing the running loss of the image acquisition equipment and the server inside the container, thereby improving the service life of the image acquisition equipment and the server inside the container.

Step 200: Collect an internal product image of the target container, and obtain a weight change value of each item in the target container, wherein the internal product image includes all the items in the current target container.

In step 200, the server controls the image collection device in the target container to collect an internal product image of the target container, and obtains a weight change value of each item in the target container. Collecting an internal product image of the target container, and obtaining a weight change value of each item in the target container, specifically including the following: controlling a plurality of image collection devices disposed in the target container to collect the target container An internal product image and a weight change value of various items in the target container; wherein different image capture devices are configured to collect images of goods on different shelves in the target container.

It can be understood that the image collection device in each container can collect all the items in the current target container, and therefore, the image collection devices in each container can be set to multiple.

In a specific implementation, the step 200 specifically includes the following content:

Controlling, by a plurality of image collection devices disposed in the target container, an internal product image of the target container, wherein different image collection devices are configured to collect images of goods on different shelves in the target container, and all shelves The image of the goods on the top constitutes the image of the internal goods; and the weight sensors respectively disposed on the respective shelves in the target container collect the weight change values of the goods on the respective shelves, wherein the same type of goods are placed on the same shelf .

In a specific example, a container generally includes a plurality of shelves, and a plurality of image acquisition devices can be disposed on both sides of the shelves of each layer to ensure that the image collection devices in each container can All the items in the current target container are collected, and at the same time, in order to ensure the clarity of the acquired image, the corresponding lighting device can be corresponding to each image capturing device.

Step 300: Perform image recognition on the internal goods image to obtain types of various types of goods in the target container.

In step 300, the server receives an internal product image sent by the image collection device, and performs image recognition on the internal product image to obtain a type of various types of goods in the target container. The image recognition method makes it unnecessary for the replenishing personnel to place the merchandise in the designated position during the replenishment process, making the replenishment process faster and more convenient.

In a specific example, the step: 300 specifically includes the following content:

Step 301: Perform image recognition on the internal product image to obtain an image feature of the quantity in the target container.

Step 302: Determine, according to an image feature of the product and a preset product image feature database, a type of the item whose quantity is changed in the target container; wherein the item image feature library stores the type and image feature of the item Correspondence between them.

It can be understood that the image recognition method for the internal goods image is a technology for processing, analyzing and understanding images by using a computer to identify targets and objects of various modes, that is, using software. According to the gray scale difference of the picture, the image recognition can be performed based on the neural network, the image recognition based on the wavelet moment, or the image recognition based on the fractal feature. The process of image recognition based on neural network may be composed of preprocessing, feature extraction and neural network classifier. Preprocessing is to delete, smooth, binarize and normalize the useless information in the original data; The process of image recognition by wavelet moment can be identified by using the invariant matrix of the input two-dimensional binary image as the recognition feature, and the input image is normalized, polar coordinate, and rotation invariant wavelet moment feature extraction. After being sent to the BP network classifier for identification, the recognition result is obtained; the process of image recognition based on the fractal feature may include infrared image preprocessing, infrared image fractal feature extraction and neural network based infrared image recognition.

Step 400: Determine the quantity of each item in the target container that changes in quantity according to the type of the changed item in the target container and the corresponding weight change value.

In step 400, the server controls the weight sensor in the target container to collect the weight of each shelf, and each weight sensor sends a weight change value compared with the previous collection to the server, if the weight does not change. The shelf has a corresponding weight change value of zero.

Further, in the present embodiment, the order of execution of

steps

300 and 400 can be set to any order according to the actual application.

In a specific example, a container generally includes a plurality of shelves, and each of the shelves has at least one weight sensor at the bottom end, and each weight sensor communicates with the server. For example, when there is a cargo on the shelf provided with the weight sensor at the lower end, gravity is transmitted to the weight sensor through the shelf as the carrier under the action of gravity, and the elastic body of the weight sensor is deformed and strained on the elastic body. Jiqiao Road loses balance, outputs an electric signal proportional to the weight value, and amplifies the signal through a linear amplifier; then converts it into a digital signal by A/D, and the microprocessor's microprocessor (CPU) processes the weight signal and directly displays it. Weight data and send weight data to the server.

It can be seen from the above description that the method for identifying goods in a container provided by the embodiment of the present application can accurately and quickly identify the goods in the container, and only need to classify the goods, without checking the quantity, thereby effectively saving manpower. The cost, the replenishment process is faster and more convenient, and the intelligence of the container and the reliability of the application are improved, so that the user experience is improved.

In a specific embodiment, referring to FIG. 3, the present application further provides a step 500 performed after the step 400 in the item identification method, and the step 500 specifically includes the following content:

Step 500: Update real-time information of the goods of the target container based on the type and quantity of the various types of goods changed in the target container.

In step 500, the server updates the real-time information of the goods of the target container in a database corresponding to the target container based on the type and quantity of the various types of goods changed in the target container. In an application example, the updating the real-time information of the goods of the target container may include: updating real-time information of the goods of the target container in an application APP corresponding to the target container.

It can be seen from the above description that the method for identifying goods provided by the embodiment of the present application solves the corresponding data of the unmanned container product management system by updating the real-time information of the goods of the target container, and needs manual manual entry or external in the replenishment. File import is not conducive to the business growth of the system data rapid update iteration, and there is a problem of redundant redundant human resources.

In a specific embodiment, referring to FIG. 4, the present application further provides a first specific implementation manner of the step 100 in the method for identifying a product, where the step 100 specifically includes the following content:

Step 101: Receive user identification information or transaction information for the target container.

In a specific implementation, the step 101 specifically includes the following content:

Receiving, by the server, user identification information or transaction information sent by the user in the application APP after connecting to the application APP corresponding to the target container by scanning the two-dimensional code by using the terminal device; wherein the two-dimensional code It is disposed on the outer wall of the target container.

In another specific implementation, the step 101 specifically includes the following content:

Receiving user identification information or transaction information acquired by means of face image recognition; wherein the manner of face image recognition comprises: acquiring, by the image collection device disposed on an outer wall of the target container, the target container The face image of the user outside, and the face image is recognized by the face image.

Step 102: Send a door opening instruction to the target container, so that the target container opens the door lock of the door according to the door opening instruction.

Step 103: Receive a product identification instruction sent by the target container after the door is opened and closed.

It can be seen from the above description that the method for identifying goods in a container provided by the embodiment of the present application dynamically acquires product image information in real time by scanning a two-dimensional code method and a built-in camera photographing system, and automatically reports real-time images of the container, and realizes through image recognition. Fully intelligent and automated goods replenishment mechanism to achieve accurate commodity data management and control solutions; liberate the waste of human resources, time resources and other wastes brought by traditional retail unmanned cabinets in the process of replenishment; further provide efficient for the unmanned retail industry Convenient solution.

In a specific embodiment, referring to FIG. 5, the present application further provides a specific implementation manner of the step 600 in the product identification method. The step 600 is specifically performed after the step 400, and the step 600 specifically includes the following content. :

Step 600: If it is detected that the quantity of a certain type of goods in the target container is lower than a corresponding replenishment threshold, the replenishment notification information is sent to the terminal device of the replenishing personnel.

It can be seen from the above description that the method for identifying goods provided by the embodiment of the present application can realize timely replenishment of goods in the container more accurately and reliably, and can accurately verify the identification information of the user, and ensure the replenishment of the container. And the security and reliability of the transaction.

In a specific embodiment, the present application further provides a specific implementation manner of the step 400 in the method for identifying a product, where the step 400 specifically includes the following content:

Step 401: Calculate the quantity of each item in the target container according to the type of the goods in the target container and the corresponding weight change value, and the pre-acquired item weight library. An initial value; wherein the goods weight library stores a correspondence between a type of the goods and a weight of the goods.

Step 402: Determine, according to the initial value of the quantity of the various types of goods, the quantity of each type of goods whose quantity in the target container changes.

In step 402, if the initial value of the quantity of the goods is an integer, the server directly determines the quantity of the quantity of the goods in the target container in the initial value of the quantity; The initial value of the quantity is a non-integer, and the server determines the quantity of each item in the target container that changes in quantity in the target container according to the value of the fractional part of the quantity initial value. The determining, according to the value of the fractional part of the quantity initial value, the quantity of each type of goods in the target container, specifically including the following content: determining whether the value of the decimal part of the quantity initial value is allowed Within the error range; if yes, the initial value is rounded off, and the rounded value determines the quantity of each item in the target container that changes in quantity; if not, the item confirmation instruction is sent to the user's terminal device .

In addition, referring to FIG. 6, from the perspective of the interaction between the server and the container, the method for identifying the goods of the present application specifically includes the following contents:

(1) The user scans the two-dimensional code set on the container by using the terminal device, enters the APP corresponding to the two-dimensional code in the mobile terminal, and inputs the identity information in the APP, and the mobile terminal inputs the user identification information into the APP. Send to the server.

(2) The server verifies the user identification information, and sends a door opening instruction to the target container after the verification is successful.

(3) After the target container receives the door opening command, the door lock of the door is opened, and the item identification instruction sent to the server after the door is opened and closed.

(4) The server receives the item identification instruction, and sends a photographing instruction to the camera of the target container, and sends a weight collection instruction to the weight sensor of the target container.

(5) the camera of the target container performs image collection of the goods according to the photographing instruction, and sends the collected internal goods image to the server; and the weight sensor of the target container performs the weight collection of the goods according to the weight collection instruction. And send the collected weight change value to the server.

(6) The server performs image recognition on the internal goods image to obtain types of various types of goods in the target container, according to types and corresponding weights of various items in the target container. The change value determines the quantity of each item in the target container, and updates the real-time information of the target container in the database based on the type and quantity of the various items in the target container.

(7) The server sends the real-time information of the updated target container to the APP, so that the user can successfully check the successful user.

To further illustrate the solution, the present application further provides a specific application example of the product identification method. Referring to FIG. 7, the following specifically includes the following contents:

S1, replenishment personnel login replenishment APP, scan code to open the door;

S2. The replenishing personnel place the merchandise on the shelf, and only one commodity is placed in the shelf or tray corresponding to the same gravity sensor;

S3, close the door;

S4, start the camera work, take the picture of the shelf and send it to the server, and the server performs image analysis processing and comparison to obtain the product category of each shelf increase and decrease; at the same time, after closing the door, the main control board queries the weight change value of the weight sensor and sends it. To the server;

S5. After the server obtains the weight change, the commodity database is invoked, and the total weight change of the goods in the shelf is divided by the weight of the individual commodity, and the quantity of the goods in each shelf is further calculated;

S6. The server automatically sends the category and quantity of the replenished goods to the replenishment APP.

In order to deepen the understanding of this patent, the above scheme will be described in detail below.

The shelf or tray is configured to display goods, at least one weight sensor is built in each shelf or tray, and the shelves or trays are numbered sequentially, and only one item is placed in the shelf or tray corresponding to the same serial number. That is the same name and weight of the goods;

Further, the goods placed in the shelf or the tray may be roughly divided according to the shape or shape, or may be customized according to the shopping habits of the current regional population, for example, one or two layers of beverages and dairy products, three layers Bread, four layers of casual snacks; or, one side of the tray placed on the left side of the tray, one side of the tray placed on the right side of the dairy product; the second layer of the left side of the tray placed bread, the second layer of the right side of the tray placed Biscuit

The camera is distributed on at least one of the sides of each shelf, the front and rear of the shelf, or the top of the shelf;

The product database includes at least the name, quantity and unit price of each product, and may also include any one of a product category, a net content, a packaging method, a weight tolerance error, a minimum quantity of the product, a sufficient quantity of the product, a shelf life of the product, or the like. Multiple information;

In the S4, the image recognition method is as follows:

Through the method of image collection, the food is randomly placed on the container, the ingredients of each group of pictures are manually marked, the characteristics of the ingredients are extracted, and the database of the characteristics of the ingredients is established;

Characterizing the ingredients through repeated, large amounts of ingredients;

Processing the image to establish an image recognition algorithm model;

Identifying the ingredients in the picture sent by the terminal according to the characteristics of the ingredients;

The ingredients in each shelf before and after closing the door are compared to identify the name and quantity of the changed ingredients;

The calculation method in S5, when the result is not an integer, the number of commodities is obtained by rounding off;

The calculation method in S5, if the result is not an integer, further, when the value after the decimal point is within a certain range, further, the range can be selected as 4-6, indicating the actual weight of the commodity and the weight of the product. The error between the two is large, and the replenishing personnel can be prompted to confirm manually to avoid statistical errors in the quantity of replenished goods. Further, the example is as follows: If the packaging weight of a instant noodle is 110g, the replenisher is newly placed in the tray 1. The instant noodle has 10 barrels. If the actual test weight of the instant noodle has a large deviation from 110g, for example, the average weight of each barrel is 5g, the weight sensor detects that the weight change in the tray 1 may be 1150g, and the server obtains the inner swing of the tray 1 through image comparison. The product is a instant noodles, and the weight change transmitted by the weight sensor is 1150g. The server calls the product database to obtain the package weight of the instant noodles is 110g, and the calculation 1150 is divided by 110, and the number of instant noodles in the tray 1 is 10.46. Bucket, the value after the decimal point is between 4-6, the judgment error is large, please confirm the convenience of the tray 1 If the number of noodles is not processed, the replenisher will be placed in the tray 1 with 10 barrels of instant noodles according to rounding. The above is a treatment method in a relatively extreme case. Generally, when the error between the product package label weight and the actual measurement weight is small, for example, in the above example, if the actual test weight of the barrel instant noodles is 1 g more than the package label weight, The weight sensor detects that the weight change in the tray 1 may be 1110g. Using the same processing method as described above, the server calculates that the number of instant noodles in the tray 1 is 10.09 barrels, and the number after the decimal point is not between 4-6, then the rounding is determined to determine the replenisher. A tray of 10 instant noodles is placed in the tray 1 and no replenishment confirmation is required, and a replenishment list can be automatically generated.

The weight tolerance error is an acceptable error range between the actual measured weight of each commodity and the package label weight. The staff selects a certain number of commodity specimens for actual testing, and the weight allowance error may not be set. , if not set, the default is 0;

If the weight change is a negative value, it means that the replenisher takes out the merchandise from the shelf or the tray, and accordingly subtracts the inventory of the merchandise from the merchandise list;

The image recognition, if it is determined that the current placement position of the food does not coincide with the previous image result, and detects the last image recognition result, determines whether the product is a new product, and if so, adds the product data to the product inventory and the replenishment list. If not, replace the weight sensor value of the food placement position with the current position; further, for example, if a brand of carbonated beverage is placed in the tray 1 position during the last replenishment, the replenishment After the cloud image recognition finds that the branded carbonated beverage is placed in the tray 2 position, the last weight of the tray 1 is directly transplanted to the tray 2, and the current replenishment quantity is determined by the weight change in the tray 2; analogy. In addition, the application example of the application further includes a reminder method for replenishing goods:

Set the minimum quantity value Kn (Kn is a natural number) for each item in the container. After detecting the current quantity Qn of each item in the container through the above steps, compare the sizes of Qn and Kn:

If Qn=0, the product is out of stock; if 0<Qn<Kn, the product is insufficient; if Qn≥Kn, the product is sufficient;

Set when the goods are insufficient, the server sends a replenishment reminder to the replenishment app, and the replenisher can click to view the replenishment reminder;

If Kn=0 is set, it means that the quantity of the item is 0, then the shortage is reminded;

The method for determining the minimum quantity value Kn of the commodity is that the server collects the past purchase behavior data of the current regional population, and obtains the consumption habits of the current regional personnel according to the big data analysis, and the staff sets the minimum required quantity of each commodity accordingly. ;

The replenisher can check the merchandise inventory list and the current replenishment list through the replenishment APP. To further illustrate the patent, Table 1 is an example of a merchandise inventory in the application example.

Table 1

As shown in Table 1, the server sets the minimum required quantity of each item according to the past shopping data and habits of the current regional population. The demand for instant noodles and carbonated drinks is higher in the area, so the minimum number of instant noodles B and carbonated drinks C The requirement is 3, and the minimum quantity of the remaining products is 1. According to Fig. 2, the current inventory of instant noodles B is 2, the minimum quantity requirement of instant noodles is 3, and the display quantity is insufficient; the current inventory of carbonated beverage C is 0, indicating out of stock; At this point, the server will send a replenishment reminder to the client of the replenishment personnel. After receiving the replenishment list reminder, the replenishment personnel will view the replenishment reminder and all the stocks in the current container through the replenishment client, and then lead enough goods. Go for replenishment.

Table 2

As shown in Table 2, the replenishing personnel log in to the replenishment APP, and scan the container QR code to open the door, put the goods into the container, all the goods are placed, and the door is closed. At this time, the server determines, by image recognition, that the products placed on the shelves 1, 2, 3, and 4 are the small bread A, the instant noodles B, the carbonated beverage C, the yogurt D, and the weight change transmitted by the weight sensor built in the shelf, and judges the shelf 1 The weight changes of 2, 3, and 4 are 0, 345g, 2.64kg, and 1.4kg, respectively. The server calls the product database to obtain the unit weight of each item, and obtains the replenishment quantity of each shelf item by the following calculation: 0/100 =0;345/115=3;2640/330=8;1400/200=7; the server sends the above goods and weight information to the replenishment APP and the device (if there is a display, the replenisher can view after logging in) ), the replenishment personnel can log in to the replenishment app to view the replenishment details at any time even if they have left.

As shown in the example of Fig. 3, if the replenishing personnel finds that there is a large amount of small bread A in the container and it is placed on the shelf 1 in the corner position, the replenishing personnel temporarily place the small bread A on the shelf of the original carbonated beverage C. Then, the carbonated beverage C is placed on the original shelf 1 of the small bread A. At this time, the server determines, by image recognition, that the product placed on the shelf 1 is carbonated beverage C, and the product placed on the shelf 3 is small bread A. After the last weight values of the shelves 1 and 3 are swapped, the replenishment quantity of the two shelves is calculated according to the above manner.

In addition, the replenishment APP in this application example may also be installed on the door of the container, and displayed through the display of the container. Further, in addition to scanning the code, the refiller may open the door in addition to the method of opening the door. The product name is obtained by camera image recognition, and the quantity of the product is obtained by the weight sensor, and the recognition order can be adjusted before and after.

It can be seen from the above description that the product identification method in the application example of the present application acquires the weight change of the goods by installing the weight sensor in the shelf, acquires the product picture by installing the camera in the container, and performs image recognition to obtain the shelf. The category of goods increased or decreased; this method does not require the position of the goods to be fixed, you can place a certain product on any shelf or tray; according to the weight change of the goods and the category change, combined with the product database information of the server Quickly calculate the replenishment details of the goods, and automatically generate a replenishment list; when it is detected that the quantity of goods in the shelf is lower than a certain set value, the server automatically sends a replenishment reminder to the replenishing personnel; when the goods in the shelf are detected When the time is different from the last time, the weight value of the last shelf of the product is directly called, and the data is updated; the product identification method in the application example is compared with the traditional product identification method of the vending machine, and there is no need to scan one by one when replenishing the goods. Product barcode, directly into the product, close the door to leave; compared to the new unmanned retail machine In this way, the placement of goods in the shelf is no longer limited to a certain shelf, only a certain type of goods can be placed, and the replenishing personnel can place a certain product on any shelf or tray, and close the door. The replenishment category and quantity can be obtained automatically; for the replenishment staff, it is convenient and quick, and the replenishment efficiency is high.

Embodiment 2 of the present application provides a specific implementation manner of a product identification system capable of implementing the entire content of the above-mentioned goods identification method. Referring to FIG. 8, the item identification system specifically includes the following contents:

The item identification command receiving module 10 is configured to receive a product identification instruction of the target container.

The image and weight collection module 20 is configured to collect an internal product image of the target container and obtain a weight change value of each item in the target container, wherein the internal product image includes all of the current target containers Goods.

The increase or decrease item type identification module 30 is configured to perform image recognition on the internal item image to obtain a type of each item in which the quantity in the target container changes.

The increase/decrease item quantity identification module 40 is configured to perform image recognition on the internal goods image to obtain types of various types of goods in which the quantity in the target container changes.

The embodiment of the method for identifying the goods in the container provided by the present application may be specifically used to perform the processing flow of the embodiment of the method for identifying the goods in the above-mentioned container. The functions of the present invention are not described herein again. Reference may be made to the detailed description of the foregoing method embodiments.

It can be seen from the above description that the cargo identification system in the container provided by the embodiment of the present application can accurately and quickly identify the goods in the container, thereby effectively saving labor costs, making the replenishing process faster and more convenient, and improving the container. The degree of intelligence and application reliability make the user experience better.

Embodiment 3 of the present application provides a specific implementation manner of an electronic device capable of implementing all the steps in the foregoing method for identifying a product. Referring to FIG. 9, the electronic device specifically includes the following contents:

a processor 601, a memory 602, a communication interface 603, and a bus 604;

The processor 601, the memory 602, and the communication interface 603 complete communication with each other through the bus 604; the communication interface 603 is configured to implement information transmission between various devices in the server and the container;

The processor 601 is configured to invoke a computer program in the memory 602, and when the processor executes the computer program, all the steps in the first embodiment are implemented, for example, when the processor executes the computer program Implement the following steps:

Step 100: Receive a product identification instruction of the target container.

It can be seen from the above description that the electronic device provided by the embodiment of the present application can accurately and quickly identify the goods in the container, thereby effectively saving labor costs, making the replenishing process faster and more convenient, and improving the intelligence degree of the container. And application reliability, so the user experience is improved.

Embodiment 4 of the present application provides a computer readable storage medium capable of implementing all the steps in the above product identification method, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the above embodiment All of the steps, for example, when the processor executes the computer program, implement the following steps:

Step 100: Receive a product identification instruction of the target container.

It can be seen from the above description that the computer readable storage medium provided by the embodiment of the present application can accurately and quickly identify the goods, effectively save labor costs, make the replenishing process faster and more convenient, and improve the intelligence degree of the container. And application reliability, so the user experience is improved.

Embodiment 5 of the present application provides a specific implementation manner of a container. Referring to FIG. 10, the container specifically includes the following contents:

The container is in communication with the electronic device, the electronic device is configured to receive a product identification instruction of the target container, collect an internal product image of the target container, and obtain a weight change value of each item in the target container, Wherein, the internal goods image includes all the goods in the current target container, and image recognition of the internal goods image, and the types of various items in the target container are changed.

In a specific example, the container includes a displacement sensor, a plurality of weight sensors, and a plurality of cameras. A container generally includes a plurality of shelves, and each of the shelves can be provided on both sides and above. The camera ensures that all the products in the current target container can be collected by the camera in each container. At the same time, in order to ensure the clarity of the acquired image, each camera can have corresponding lighting, and the goods identification instruction is sent. The unit may be a displacement sensor, and the two parts of the displacement sensor are respectively a device 1 disposed on the cabinet door of the container and a device 2 disposed on the inner wall of the container close to the device, and the bottom ends of each shelf are At least one weight sensor is provided and each weight sensor is in communication with the server.

It can be seen from the above description that the container provided by the embodiment of the present application can accurately and quickly identify the goods in the container, effectively saving labor costs, making the replenishing process faster and more convenient, and improving the intelligence degree of the container and Application reliability, so the user experience is improved.

In the description of the present application, it is to be noted that the orientation or positional relationship of the terms "upper", "lower" and the like is based on the orientation or positional relationship shown in the drawings, for convenience of description of the present application and simplified description. Instead of indicating or implying that the device or component referred to must have a particular orientation, constructed and operated in a particular orientation, it is not to be construed as limiting the invention. Unless specifically stated and limited, the terms "mounted," "connected," and "connected" are used in a broad sense, and may be, for example, a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection, It can also be an electrical connection; it can be directly connected, or it can be connected indirectly through an intermediate medium, which can be the internal connection of two components. For those skilled in the art, the specific meanings of the above terms in the present application can be understood on a case-by-case basis.

It should also be noted that, in this context, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying such entities or operations. There is any such actual relationship or order between them. Furthermore, the term "comprises" or "comprises" or "comprises" or any other variations thereof is intended to encompass a non-exclusive inclusion, such that a process, method, article, or device that comprises a plurality of elements includes not only those elements but also Other elements, or elements that are inherent to such a process, method, item, or device. An element that is defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, item, or device that comprises the element.

The above embodiments are only used to explain the technical solutions of the present application, and are not limited thereto; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still The technical solutions are described as being modified, or equivalent to some of the technical features are replaced; and such modifications or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

A method for identifying goods, characterized in that the method for identifying goods comprises:

Receiving a product identification instruction of the target container;

Collecting an internal product image of the target container and obtaining a weight change value of each item in the target container, wherein the internal product image includes all the items in the current target container;

And performing image recognition on the internal goods image to obtain types of various types of goods in the target container;

Determining the quantity of each item in the target container that changes in quantity according to the type of the various items in the target container and the corresponding weight change value.
The method for identifying a product according to claim 1, wherein the method for identifying a product further comprises:

Update the real-time information of the goods of the target container based on the type and quantity of various types of goods in the target container.
The item identification method according to claim 1, wherein the item identification instruction of the receiving target container comprises:

Receiving user identification information or transaction information for the target container;

Sending a door opening instruction to the target container, so that the target container opens the door lock of the door according to the door opening instruction;

And receiving a goods identification instruction sent by the target container after the door is opened and closed.
The method for identifying a product according to claim 3, wherein the receiving user identification information or transaction information for the target container comprises:

Receiving user identification information or transaction information sent by the user in the application APP after connecting to the application APP corresponding to the target container by using the terminal device to scan the two-dimensional code;

Wherein the two-dimensional code is disposed on an outer wall of the target container.
The method for identifying a product according to claim 3, wherein the receiving user identification information or transaction information for the target container comprises:

Receiving user identification information or transaction information acquired by means of face image recognition;

The method for recognizing a face image includes: acquiring an image of a face of a user located outside the target container through an image collection device disposed on an outer wall of the target container, and performing a face image on the face image Identification.
The method for identifying a product according to claim 1, wherein the collecting an internal product image of the target container and obtaining a weight change value of each item in the target container comprises:

Controlling, by a plurality of image collection devices disposed in the target container, an internal product image of the target container, wherein different image collection devices are configured to collect images of goods on different shelves in the target container, and all shelves The image of the goods on the top constitutes the image of the internal goods;

And controlling the weight sensors respectively disposed on the respective shelves in the target container to collect the weight change values of the goods on the respective shelves, wherein the same type of goods are placed on the same shelf.
The method for identifying a product according to claim 1, wherein the image recognition of the internal product image is performed to obtain a type of each item in which the quantity in the target container changes, including:

Performing image recognition on the internal goods image to obtain image features of the quantity in the target container;

And determining, according to the image feature of the product and the preset product image feature library, the type of the item in the target container that changes in quantity;

The correspondence between the type of the product and the image feature is stored in the image image feature library.
The method for identifying a product according to claim 1, wherein said determining a quantity of said target container changes according to a type of each type of goods whose quantity is changed in said target container and a corresponding weight change value The number of goods, including:

Calculating the initial value of the quantity of each item in the target container according to the type of the goods in the target container and the corresponding weight change value, and the pre-acquired item weight library;

And determining, according to the initial value of the quantity of the goods of the various types, the quantity of each type of goods in the target container;

The cargo weight library stores a correspondence between the type of the product and the weight of the product.
The method for identifying a product according to claim 8, wherein the determining the quantity of each item in the target container according to the initial value of the quantity of the various types of goods comprises:

Determining, according to an integer initial value of the quantity of the goods of the various types, the quantity of each item in the target container in which the quantity is changed;

And determining, according to the initial value of the quantity of the goods of the non-integer number, the quantity of each item in the target container that changes in quantity in the target container according to the value of the decimal part of the quantity initial value.
The method for identifying a product according to claim 9, wherein the determining the quantity of each item of the quantity in the target container according to the value of the fractional part of the quantity initial value comprises:

And determining, according to the determination result that the value of the fractional part of the initial value is within the allowable error range, rounding off the initial value, and rounding the value to determine the quantity of each item in the target container;

The goods confirmation instruction is transmitted to the terminal device of the user according to the determination result that the value of the decimal portion of the initial value is not within the allowable error range.
The method for identifying goods according to claim 2, wherein the updating the real-time information of the goods of the target container comprises:

Updating the real-time information of the goods of the target container in the application APP corresponding to the target container.
The method for identifying a product according to claim 1, wherein the method for identifying a product further comprises:

Replenishing the replenishment notification information to the replenishing personnel terminal device according to the detected result that the quantity of the certain type of goods in the target container is lower than the corresponding replenishment threshold.
A goods identification system, characterized in that the goods identification system comprises:

a goods identification instruction receiving module configured to receive a goods identification instruction of the target container;

An image and weight collection module configured to collect an internal product image of the target container and obtain a weight change value of each item in the target container, wherein the internal product image includes all items in the current target container ;

An increase or decrease item type identification module configured to perform image recognition on the internal item image to obtain a type of the item in the target container that has changed in quantity;

The increase or decrease item quantity identification module is configured to perform image recognition on the internal goods image to obtain types of various items in the target container that change in quantity.
An electronic device comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein the processor executes the program as claimed in any one of claims 1 to 12 The steps of the method for identifying the goods.
A computer readable storage medium having stored thereon a computer program, characterized in that the computer program is executed by a processor to implement the steps of the article identification method according to any one of claims 1 to 12.
A container, characterized in that the container is in communication with the electronic device of claim 14.