WO2019232703A1

WO2019232703A1 - Smart vending cabinet, object recognition method, device, server, and storage medium

Info

Publication number: WO2019232703A1
Application number: PCT/CN2018/090041
Authority: WO
Inventors: 张站朝
Original assignee: 深圳前海达闼云端智能科技有限公司
Priority date: 2018-06-06
Filing date: 2018-06-06
Publication date: 2019-12-12
Also published as: CN108885813B; CN108885813A

Abstract

A smart vending cabinet, an object recognition method, a device, a server, and a storage medium, pertaining to the field of smart retailing. The smart vending cabinet (10) comprises: a cabinet body (102) provided with at least one accommodating space (101), a sensing light curtain (104) disposed on opposite surfaces within the cabinet body (102) and proximate to the position of an object collection opening in the cabinet body (102), a camera (105) disposed within each accommodating space (101), and a main control module (106) communicatively connected to the sensing light curtain (104) and the camera (105). Light generated by the sensing light curtain (104) covers the object collection opening. The sensing light curtain (104) acquires, under control of the main control module (106), dimension information of an object passing through the object collection opening. The camera (105) captures, under control of the main control module (106), an image of an object within the accommodating space (101). The main control module (106) transmits the dimension information and the image acquired by the camera (105) to an object identification device. The smart vending cabinet lowers the dependency on the camera thereof, and improves the accuracy of object identification.

Description

Intelligent sales container, method, device, server, and storage medium for item identification

Technical field

The present application relates to the field of intelligent retail, and in particular, to an intelligent sales container, a method, a device, a server, and a storage medium for identifying items.

Background technique

With the continuous development of technology, in order to facilitate people to buy items at any time, smart sales containers have appeared. The operation mode of the smart sales container is usually the three processes of scanning the code to open the door, selecting the items by the user, and closing the door automatically, which is convenient, fast, and requires no staff.

technical problem

The inventor discovered during the research of the existing technology that the current smart sales container usually installs a camera in the smart sales container, and the camera collects images and passes artificial intelligence (Artificial Intelligence (referred to as "AI") visually identifies the type and quantity of items in the intelligent sales container. However, only the images captured by the camera are identified. As a result, the identification of the type and amount of data in the smart container can only depend on the images collected by the camera. If the acquired images are unclear or incomplete (for example, occlusion between items) ), It will affect the identification of the items in the image, and reduce the recognition rate of the items purchased by the smart container; if the camera fails to take pictures, the items in the smart container cannot be identified, and the smart sale can only be terminated The use of containers, which will result in the waste of resources of smart sales containers, will seriously affect the user experience of using smart sales containers.

Technical solutions

The technical problem to be solved in some embodiments of the present application is to provide a smart container, a method, a device, a server, and a storage medium for identifying items, so as to reduce the dependence of the smart container on the camera, and improve the smart container's ability to identify items. accuracy.

An embodiment of the present application provides an intelligent sales container, which includes a cabinet provided with at least one storage space, and a measurement light curtain disposed on an opposite surface of the cabinet, and the measurement light curtain is close to the items in the cabinet to pick and place The location of the mouth, the camera set in each placement space, and the main control module that communicates with the measuring light curtain and the camera, respectively, in which the light generated by the measuring light curtain covers the access to the items; the measuring light curtain is in the main control module. Obtain the size information of the items through the item access port under the control of the camera; the camera shoots the items in the placement space under the control of the main control module; the main control module transmits the size information and the image obtained by the camera to the item recognition device.

An embodiment of the present application further provides a method for identifying an item, which is used to identify an item taken out or put in the above-mentioned smart vending container, including: receiving measurement information of the size of the item obtained through the item access port obtained by measuring the light curtain. According to the size information, identify the items and quantity through the item access port; receive the image obtained by the camera to capture the items in the placement space; based on the image, identify the items and number in the placement space; according to the items and number through the item access port And items in the storage space, determine the type and number of items taken out of or placed in the cabinet.

An embodiment of the present application further provides an article identification device. The article identification device is communicatively connected to the above-mentioned intelligent sales cabinet. The article identification device includes a communication module, a light curtain identification module, an image identification module, and a determination module. The communication module is used for: Receiving measurement of the size information of items passed through the item access port obtained by measuring the light curtain; the light curtain identification module is used to identify the items and quantity passing through the item access port according to the size information; the communication module is also used to receive the camera shooting space The image obtained by the item; the image recognition module is used to identify the items and the quantity in the placement space based on the image; the determination module is used to determine the removal from the cabinet according to the items and quantity through the item access port and the placement space. Or the type and number of items you put in.

An embodiment of the present application further provides a server, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor The processor executes to enable at least one processor to execute the method for item identification described above.

An embodiment of the present application further provides a computer-readable storage medium storing a computer program, and the computer program is executed by a processor to implement the foregoing method for item identification.

Beneficial effect

Compared with the prior art, in some embodiments of the present application, a measurement light curtain is added to the smart sales container. Since the measurement light curtain is set close to the item access port, it is possible to obtain the item access through the smart sales container. Mouth size information. Generally, the size information of different types of items is different, so that the type of the items can be identified by the size information of the items. Due to the increase of the measuring light curtain, the dependence of the intelligent sales container on the camera is reduced, and the recognition of the items in the cabinet due to the failure of the camera is reduced. In addition, the combination of the two identification methods performed by the measuring light curtain and the camera improves the accuracy of the identification of items taken out or put in the smart sales container.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments are exemplified by the pictures in the accompanying drawings. These exemplary descriptions do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the drawings in the drawings do not constitute a limitation on scale.

FIG. 1 is a schematic front structural diagram of a smart sales container in a first embodiment of the present application; FIG.

2 is a schematic structural diagram of a cabinet of a smart sales container in the first embodiment of the present application;

FIG. 3 is a schematic front structural diagram of a smart sales container in a second embodiment of the present application; FIG.

4a is a schematic structural diagram of a single-sided emission measurement light curtain in a smart sales container in a second embodiment of the present application;

4b is a schematic structural diagram of a multi-facet emission measurement light curtain in a smart sales container in a second embodiment of the present application;

FIG. 5 is a schematic front structural diagram of a smart sales container in a third embodiment of the present application; FIG.

6 is a schematic diagram of another front structure of a smart sales container in a third embodiment of the present application;

7 is a schematic front structural diagram of a smart sales container in a fourth embodiment of the present application;

8 is a schematic flowchart of a method for item identification in a fifth embodiment of the present application;

FIG. 9 is a schematic structural diagram of an article identification device in a sixth embodiment of the present application

FIG. 10 is a schematic structural diagram of a server in a seventh embodiment of the present application.

Embodiments of the invention

In order to make the purpose, technical solution, and advantages of the present application clearer, some embodiments of the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application. However, a person of ordinary skill in the art can understand that in the embodiments of the present application, many technical details are provided in order to make the reader better understand the application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solution claimed in this application can be implemented.

The first embodiment of the present application relates to a smart sales container, which can be placed in a place with a large number of people, such as a movie theater, a shopping mall, a train station, and the like. The intelligent sales cabinet 10 includes a cabinet body 102 provided with at least one storage space 101, and a measuring light curtain 104 provided on an opposite surface of the cabinet body 102. The measuring light curtain 104 is close to the article access opening in the cabinet body 102. (Not shown in FIG. 1), a camera 105 disposed in each placement space 102, and a main control module 106 communicatively connected to the measurement light curtain 104 and the camera 105 respectively, where the measurement light curtain 104 generates The light covers the access opening of the article; the specific structure is shown in Figure 1.

Specifically, the measurement light curtain 104 includes at least one pair of light emitters and light receivers, wherein each pair of light emitters and light receivers are separated from each other and are opposite to each other. As the light travels in a straight line, the positions of the pair of light emitters and light receivers in the measurement light curtain 104 are parallel to each other, as shown in FIG. 1. Of course, the logarithm of the light transmitter and the light receiver in the measuring light curtain 104 can be determined according to the size of the item access port in the smart sales container. If the area of the article access port is large, the number of pairs of the photometric receiver and the light transmitter in the measuring light curtain 104 is large. If the area of the article access port is small, the pair of the light transmitter and the light receiver in the light curtain 104 is measured. Few. In this embodiment, the light transmitter may be an infrared transmitter, and the light receiver may be an infrared receiver, but the types of the transmitter and the light receiver are not limited to those listed in this embodiment.

In a specific implementation, the measurement light curtain 104 is disposed on two opposite sides of the cabinet, and / or the measurement light curtain 104 is disposed on the bottom of the cabinet and the top of the cabinet.

Specifically, since the cabinet has one surface provided with an article access opening, and the light generated by the measurement light curtain 104 covers the article access opening, there are four opposing surfaces of the measurement light curtain 104 that can be set in the cabinet. , That is, the two opposite sides of the cabinet (such as 1021 and 1022 in FIG. 2), the bottom of the cabinet (such as 1023 in FIG. 2) and the top of the cabinet (such as 1024 in FIG. 2) Structure. It should be noted that, because it is necessary to measure the access opening of the light generated by the light curtain, the part where the light transmitter is installed can be set on one side of the cabinet, and the part where the light receiver is installed is set on the cabinet and light. The opposite side of the transmitter is shown in the structural diagram of the cabinet shown in Figure 2. The part where the light transmitter is installed can also be set on the top (or bottom) of the cabinet, and the part where the light receiver is installed can be set. At the bottom (or top) of the cabinet. It can be understood that if there are multiple measuring light curtains 104, the measuring light curtains 104 can be installed on two opposite sides of the cabinet and on the top and bottom of the cabinet.

In a specific implementation, the measurement light curtain 104 obtains the size information of the items through the item access port under the control of the main control module 106; the camera 105 shoots the items in the placement space under the control of the main control module 106; the main control module 106 transmits the size information and the image obtained by the camera 105 to the article recognition device.

Specifically, the measurement light curtain 104 and the camera 105 can be turned on for 24 hours. In order to save the power consumption of the measurement light curtain 104 and the camera 105, the main control module 106 can start the measurement after receiving a signal that satisfies a preset condition. For the light curtain 104 and / or the camera 105, for example, a proximity sensor is installed on the smart sales cabinet 10, and the proximity sensor is connected to the main control module 106. When the proximity sensor detects that a person is approaching the smart sales cabinet, it sends a signal to the main control module 106. After sending a signal, the main control module 106 receives the signal and turns on the measurement light curtain 104 and the camera 105. In this embodiment, a method in which the main control module 106 turns on the measurement light curtain 104 and the camera 105 after satisfying a preset condition is adopted. Of course, the trigger conditions of the measurement light curtain 104 and the camera 105 are not limited to the methods in this embodiment.

The main control module 106 turns on the measurement light curtain 104 when receiving a signal satisfying a preset condition. The measuring light curtain 104 generates a light curtain covering the access opening of the article. When an article passes through the light curtain, the light curtain appears to be blocked. According to the position of the blocked light emitter and the blocking time, the passing of the light can be determined. The size information of the light curtain items, for example, when a box of milk passes through the light curtain generated by the measurement light curtain, the carton of milk is scanned by the measurement light curtain 104 to obtain the shape information of the carton of milk and the length of the milk carton at the same time. , Width, and height information. The process of obtaining the size information is not described in this embodiment.

It should be noted that the measurement light curtain is provided with a control module. The measurement light curtain recognizes the positions of the light emitter and light receiver that are blocked by synchronous scanning, the blocking time, and the speed of the object when passing the light curtain generated by the measurement light curtain. The control module in the measuring light curtain can determine information such as the shape and volume of the item. Here, the size information of the item is information such as the shape and volume of the item.

It can be understood that the size information of an item can also be expressed only as information such as the position of the light emitter and light receiver where the blockage occurred, the block time, and the speed of the object when passing the light curtain generated by the measurement light curtain. The size information of the item is processed, but the size information is transmitted to the item identification device, and the item identification device converts the size information of the item into the corresponding shape, volume and other information.

The measuring light curtain 104 transmits the obtained size information of the article to the main control module 106, and the main control module 106 transmits the size information to the article identification device. The measurement light curtain 104 may implement a communication connection with the main control module 106 through a serial port.

In addition, the main control module 106 turns on the camera 105 at the same time when receiving a signal that satisfies a preset condition, and controls the camera 105 to take pictures of objects in the space where the camera 105 is located. Of course, after acquiring the N images, the main control module 106 may transmit the N images to the article identification device, where N is an integer greater than 0. Of course, the main control module 106 and the article identification device may be a wireless communication connection (such as 5G), or a wired communication connection, or a combination of wireless and wired communication.

A specific example will be used below to explain the use of the intelligent sales container.

It is assumed that the measurement light curtain is installed on two opposite sides of the smart sales container, and the measurement light curtain includes 5 pairs of light transmitters and light receivers. The five pairs of light transmitters and light receivers are evenly arranged on two sides of the cabinet in a vertical arrangement, and each pair of light transmitters and light receivers are opposite to each other. The camera is installed in the middle of the top of the cabinet, and a proximity sensor is installed near the measuring light curtain. When it is detected that someone is approaching the smart sales cabinet, the measuring light curtain and the camera are turned on. If the user takes out 1 can of milk from the smart container, the milk passes through the measuring light curtain, and the light curtain is blocked. The measuring light curtain scans synchronously to identify the position of the light emitter and light receiver that is blocked. The measuring light curtain will record the blocked light. The position of the light transmitter and the light receiver and the time of blocking, while obtaining the speed of the item passing through the light curtain, and measuring the data recorded by the light curtain, the shape of the box of milk and the length, width, and height of the box of milk can be obtained. The camera is turned on when someone approaches, and the inside of the cabinet is photographed. When someone is detected to be away, the camera is taken again and the camera is turned off after the picture is taken. Both the obtained item size information and the camera captured image are sent to the article identification device. Of course, when detecting that someone is far away from the smart sales container, turn off the measurement light curtain and / or the camera. The item identification device determines the type and quantity of the item taken out by the user from the smart sales cabinet through the size information of the item and the acquired image.

Compared with the prior art, in some embodiments of the present application, a measurement light curtain is added to the smart sales container. Since the measurement light curtain is disposed close to the item access opening, the item access opening through the smart sales container can be obtained. Size information for your item. Generally, the size information of different types of items is different, so that the type of the items can be identified by the size information of the items. Due to the increase of the measuring light curtain, the dependence of the intelligent sales container on the camera is reduced, and the recognition of the items in the cabinet due to the failure of the camera is reduced. In addition, the combination of the two identification methods performed by the measuring light curtain and the camera improves the accuracy of the identification of items taken out or put in the smart sales container.

The second embodiment of the present application relates to an intelligent sales container. This embodiment is further improved based on the first embodiment. The specific improvement is that in this embodiment, the intelligent sales container further includes a cabinet door. The cabinet door 107 is hinged with the cabinet body 102. The specific structure of the intelligent sales container 10 is shown in FIG. 3.

In a specific implementation, after detecting that the cabinet door is open, the main control module 106 turns on the measurement light curtain 104 and obtains the size information of each item that passes through the item access port; after detecting that the cabinet door 107 is closed, the measurement is closed The light curtain 104 controls the camera 105 to take pictures of objects in the placement space (not shown in FIG. 3).

Specifically, the cabinet door 107 is provided to ensure the safety of the items in the intelligent sales container. The closing of the cabinet door triggers the closing of the measuring light curtain and / or the camera to ensure that the information of the items in the smart sales cabinet can be obtained in time. For example, a sensor can be installed at the cabinet door 107 to accurately detect whether the cabinet door 107 is open. It can be understood that the measuring light curtain 104 can be installed on both sides of the cabinet door 107 or on the top of the cabinet body near the cabinet door 107 and on the bottom of the cabinet body 102. After the cabinet door 107 is closed, it is not possible to remove or put objects into the cabinet. Therefore, the main control module 106 closes the measurement light curtain 104 and controls the camera 105 to shoot the objects in the space. The article identification device determines the type and quantity of the articles taken out or put into the cabinet according to the image captured by the camera 105 and the size information of the articles acquired by the measuring light curtain 104.

It can be understood that the detection method of the cabinet door 107 by the main control module 106 is not limited to the methods listed in this embodiment, and can also be detected by other methods, which are not listed here one by one.

In a specific implementation, the measurement light curtain 104 includes a single-sided emission measurement light curtain or a multi-sided emission measurement light curtain; wherein all the light emitters in the single-sided emission measurement light curtain are located on the same side, and the corresponding light receivers are set On the other side opposite the position of the light emitter; at least one light emitter and at least one light receiver in the multi-facet emission measurement light curtain are on the same side.

Specifically, all the transmitters of the single-sided emission measurement light curtain are located on the same side, as shown in Figure 4a. In Figure 4a, the circle represents the light transmitter, the triangle represents the light receiver, and the dotted line represents the light generated by the light transmitter. . In the multi-facet emission measurement light curtain, two transmitters, three light receivers can be installed on one side, and three light transmitters and two light receivers can be installed on the corresponding other side, as shown in Figure 4b. Light transmitter, the triangle represents the light receiver, and the dotted line is the light generated by the light transmitter.

Both the light transmitter and the light receiver are arranged in a housing of the measuring light curtain, and the housing for mounting the light transmitter and the housing for mounting the light receiver are separated from each other. The measuring light curtain 104 has a rectangular shape as shown in Figs. 4a and 4b. The optical axis distance in the measuring light curtain 104 is determined by the size of the items placed in the cabinet, and the optical axis is formed by the light emitted by the light emitter.

The optical axis spacing generated by the measuring light curtain 104 is usually 1mm, 2mm, 2.5mm, 5mm, 7.5mm, and 10mm. You can choose a measuring light curtain with a suitable optical axis spacing for installation according to the size of the items placed in the smart sales cabinet, such as, For small items, you can choose a measuring light curtain with 1mm optical axis detection.

It should be noted that the light generated by the measurement light curtain 104 can cover the cabinet door. This embodiment does not limit the number of measurement light curtains, and multiple measurement light curtains can be installed.

Compared with the prior art, since the intelligent sales cabinet provided in this embodiment is provided with a cabinet door, the cabinet can effectively protect the contents of the cabinet through the cabinet door; the main control light module controls the opening of the measuring light curtain by closing the cabinet door. The close-up camera takes pictures of the items in the storage space, which reduces the power consumption of the smart container. At the same time, the measuring light curtain and the camera can obtain the types and quantities of the items in the cabinet in time, which is convenient for the item identification device to the items in the smart container. Identification.

The third embodiment of the present application relates to a smart sales container. This embodiment is further improved based on the second embodiment. The main improvement is that the storage space is formed by a layer partition installed in the cabinet 102. The 108 partition is formed, and the middle partition 108 is high in the middle and the height is gradually lowered to both ends. The multilayer partition 108 bridges between two opposite sides of the cabinet 102; the camera 105 is disposed on the top of the storage space. The structure is shown in Figure 5 (the main control module is not shown in Figure 5).

Specifically, the layer partition 108 is bridged between two opposite sides of the cabinet. The shape of the layer partition 108 may be triangular, trapezoidal, or arc. The camera 105 may be installed at the center of the top of the placement space, so that the viewing angle range of the camera 105 covers the entire placement space.

In a specific implementation, at least one partition bar 109 is provided on a side of the partition wall 108 facing the top of the cabinet 102, and the partition bar 109 is used to fix the items placed on the partition wall 102. The specific structure of this layer of partitions is shown in Figure 6 (the main control module is not shown in Figure 6).

Specifically, a plurality of partition bars 109 may be provided on the layer partition 108, for example, one partition bar 109 is provided at the center position of the layer partition 108, and other partition bars 109 are provided on both sides of the center position. Each of the partition bars 109 provided on both sides of the center position is rotated on the layer partition 108 with a fixed point (such as a solid black point in FIG. 6) as a center until the photographing intersection line coincides with the centerline of the placement, wherein the photographing intersection The line is the intersection between the surface where the viewing angle range of the camera 105 is located and the photographed surface of the placed object fixed by the partition 109. The height of the partition bars 109 is determined according to the height of the articles in the placement space, wherein the height of the partition bars 109 located at the center position is smaller than the height of each of the partition bars 109 provided on both sides of the center position.

The intelligent sales container provided in this embodiment, because the shape of each layer of the partition is high in the middle and the height gradually decreases to both ends, so that there is an inclination angle between the placement on the layer of the partition and the horizontal plane, reducing the phase. The blocking between adjacent objects increases the area of the camera to capture the object, increases the effective pixels of the object in the captured image, improves the accuracy of identifying the object in the image, and at the same time, the separator is convenient for fixed placement on the layer partition. Items.

The fourth embodiment of the present application relates to an intelligent sales container. This embodiment is further improved based on the third embodiment. The specific improvement is that in this embodiment, the intelligent sales container further includes: The mirror at the top of the space with the mirror surface facing the bottom of the space is shown in Figure 7, where the thick black solid line represents the mirror.

Specifically, a camera 105 is disposed at the bottom of the placement space, and the lens of the camera 105 faces the mirror 110 in the placement space where the camera is located. The angle range of the camera 105 covers the mirror surface of the mirror 110 in the placement space of the camera 105; The camera 105 is controlled to shoot the mirror 110 in the placement space.

In the smart sales container provided in this embodiment, by setting a mirror on the top of the storage space, and the camera photographs the mirror surface of the mirror, the distance between the camera and the objects in the storage space is increased, and the possibility of distortion of the objects in the captured image is reduced And improve the recognition ability of images.

The fifth embodiment of the present application relates to a method for identifying an item. The method for identifying an item is used to identify an item taken out or put in a smart vending container. The specific process is shown in FIG. 8.

Step 801: Receive the size information of the item obtained through the item access port obtained by the measurement light curtain.

Specifically, the smart sales container is communicatively connected to the article identification device, such as a wireless communication connection (4G / 5G communication). The main control module of the intelligent sales container controls the measurement light curtain to obtain the size information of the items passing through the item access port, and sends the obtained item size information to the item identification device.

In a specific implementation, before receiving the size information of the item through the item access port obtained by the measurement light curtain, the size information of the item is stored in advance, and the images of the item taken in different viewing angle ranges are stored.

Specifically, the size information of the stored items is obtained by measuring the light curtain by measuring the items. In order to increase the accuracy of determining the type and quantity of the items through the size information of the items in the subsequent steps, at the same time, in order to be more in line with actual life, different people's The size information obtained when the object is passed through the light curtain by holding the object, or the size information obtained when the object is passed through the light curtain by different gestures is stored. In short, it is necessary to enrich the size information of the item through the light curtain in different states as much as possible, which will not be listed here one by one.

Stores images of items captured in different viewing angle ranges, such as images of boxed milk when viewed from the top, images taken from the top, and images taken from the side.

Step 802: According to the size information, identify the items and the number of items through the item access port.

Specifically, deep learning is performed on the size information of the stored items, such as obtaining the characteristics of the size information (shape, volume, etc.) of the items, establishing a data model of the items, and bringing the size information of the received items into the established data model. , You can identify the type of the item. After identifying the type of each item, count the number of each item. Of course, it is also possible to directly match the obtained item size information with the pre-stored item size information, and determine the type of the item by matching the similarity. This embodiment no longer enumerates the methods for identifying articles by their size.

Step 803: Receive an image obtained by the camera taking an item in the placement space.

Specifically, this step is substantially the same as step 801, and will not be repeated here. Of course, for the accuracy of recognition, images obtained by multiple cameras capturing items in the placement space can be acquired.

Step 804: Identify the items and quantity in the placement space based on the image.

Specifically, the image can be identified in a deep learning manner to determine the type of the item and the position of the item in the image. The objects in the figure can also be identified by comparison. This method requires pre-storing images of the objects in different viewing angle ranges, matching the images with the pre-stored image of the objects, and determining the type of the objects in the image based on the similarity.

Of course, for the accuracy of recognition, the image can also be subjected to anti-interference processing to eliminate the images that are not related to the items in the smart container.

Step 805: Determine the type and quantity of the items to be taken out or put into the cabinet according to the number of items and the number of items in the storage space.

In a specific implementation, the identified items and quantity through the item access port are used as the first recognition result; the items in the currently recognized image are compared with the items in the last recognized image, and the currently recognized image is compared The number of items in the comparison is compared with the number of items in the last recognized image to determine the type and number of items taken out or put into the cabinet currently acquired by the image recognition method, and used as the second recognition result.

It is determined whether the first recognition result is the same as the second recognition result, and if they are the same, any recognition result is output as a shopping list.

If they are not the same, the monetary value corresponding to each recognition result is calculated, and the recognition result with the higher monetary value in the two ways is output as a shopping list.

Specifically, the first recognition result is obtained by recognizing the item size information obtained by measuring the light curtain, and the second recognition result is obtained by recognizing the image captured by the camera. If the first recognition result and the second recognition result are the same, it indicates that both the first recognition result and the second recognition result are accurate. If the first recognition result is different from the second recognition result, it indicates that there is an inaccuracy in the two recognition results. In order to reduce the loss of the smart container owner, at this time, the recognition result corresponding to the high monetary value of the recognition result is quasi. For example, the first recognition result is: 1 box of milk is taken out of the smart sales container, and the second recognition result is: two boxes of milk are taken out of the smart sales container. If the milk varieties are the same and the unit price is the same, the second recognition result is used for shopping Manifest output.

It is worth mentioning that before outputting the shopping list, check whether the current user corrects the abnormal items placed in the cabinet. If so, determine the corresponding processing list according to the user's corrective processing and add the processing list to the shopping list. .

Specifically, the items in the smart sales container are tilted, placed horizontally, and blocked, which are considered abnormal placement. If the user knows that the items in the smart sales container are abnormally placed, and accepts a processing request to correct the display abnormality, before outputting the shopping list, the user can determine whether the user is Anomalous items are placed for correction. If so, the type and quantity of the items after the user's correction is recorded, and the record content is added to the shopping list as a processing list.

In a specific implementation, after receiving the images obtained by the camera to capture the items in the placement space and before detecting whether the current user corrects the items placed abnormally in the cabinet, the method for identifying the items further includes: judging the intelligent sales container Are there any abnormal items in the cabinet?

Specifically, there are three methods for judging whether there are abnormal items in the cabinet of the smart sales container:

Method 1: According to the image, determine whether there is an abnormally placed item in the cabinet, and if so, send information to the user terminal about the abnormally placed item.

Specifically, the article recognition device can store images of various abnormally placed articles in advance, and compare the images obtained by the articles in the camera placement space with the pre-stored images of abnormally placed articles, according to the similarity after the comparison. Degree to determine whether there are items placed abnormally.

Method 2: According to the last transaction record of the smart sales container, determine whether the monetary value corresponding to the first recognition result in the previous transaction is smaller than the monetary value corresponding to the second recognition result. If so, send an abnormal item placement information to the user terminal.

Specifically, if the monetary value corresponding to the first recognition result in the previous transaction is smaller than the monetary value corresponding to the second recognition result, it indicates that there is a situation of obstruction, and it can be determined that there is an abnormal item placement.

Method 3: Combine method 1 and method 2 arbitrarily, that is, method 1 can be performed first, then method 2, or method 2 can be performed first, and then method 1 can be performed.

When an abnormal item is placed, the user is notified to correct the abnormally placed item, reducing the probability of misidentification caused by obstruction, and improving the quality of the camera's shooting of the item in the cabinet.

The method for item identification provided in this embodiment uses a measuring light curtain to identify the type, and determines the type and quantity of items taken from or placed in the smart container by combining the image recognition method, reducing the dependence of the smart container on the camera , Reducing the waste of resources caused by the camera failure; combined with two recognition methods to improve the accuracy of the identification of items taken out or put in the smart container.

The sixth embodiment of the present application relates to an article identification device, which is communicatively connected to a smart container. The article identification device 90 includes a communication module 901, a light curtain identification module 902, an image identification module 903, and a determination module 904. The specific structure of the article identification device 90 is shown in FIG. 9.

Specifically, the communication module 901 is used to receive the size information of the items obtained through the item access port obtained by measuring the light curtain; the light curtain identification module 902 is used to identify the items and the number of items passed through the item access port according to the size information; the communication module 901 is also used to receive images obtained by the camera shooting the items in the placement space; the image recognition module 903 is used to identify the items and quantity in the placement space based on the image; the determination module 904 is used to determine the items and quantity through the item access port. The items and quantity in the storage space determine the type and quantity of items taken out or put into the cabinet.

This embodiment is an embodiment of a virtual device corresponding to the foregoing method for identifying an item. The technical details in the foregoing embodiment of the method are still applicable in this embodiment, and details are not described herein again.

It should be noted that the above device embodiments are only schematic and do not limit the scope of protection of this application. In practical applications, those skilled in the art may select some or all of the modules to implement the present invention according to actual needs. The purpose of the embodiment scheme is not limited herein.

The seventh embodiment of the present application relates to a server 20 whose structure is shown in FIG. 10. It includes: at least one processor 201; and a memory 202 communicatively connected to the at least one processor 201. The memory 202 stores instructions executable by at least one processor 201. The instructions are executed by the at least one processor 201, so that the at least one processor 201 can execute the method for item identification described above.

The memory 202 and the processor 201 are connected in a bus manner. The bus may include any number of interconnected buses and bridges. The bus links one or more processors 201 and various circuits of the memory 902 together. The bus can also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art, so they are not described further herein. The bus interface provides an interface between the bus and the transceiver. A transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. The data processed by the processor 201 is transmitted on a wireless medium through an antenna. Further, the antenna also receives the data and transmits the data to the processor 201.

The processor 201 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. The memory 202 may be used to store data used by the processor when performing operations.

It should be noted that the processor in this embodiment can execute the implementation steps in the foregoing method embodiments, and the specific execution functions are not described in detail. For technical details in the method embodiments, details are not described herein again.

An eighth embodiment of the present application relates to a computer-readable storage medium. The readable storage medium is a computer-readable storage medium, and the computer-readable storage medium stores computer instructions that enable a computer to execute the first embodiment of the present application. The method for item identification involved in the five embodiments.

It should be noted that those skilled in the art can understand that the display method in the above embodiments is implemented by a program instructing related hardware. The program is stored in a storage medium and includes several instructions to make a device (may It is a single-chip microcomputer, a chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), and a random access memory (RAM, Random-Access Memory), magnetic disks or optical disks, and other media that can store program code.

Those of ordinary skill in the art can understand that the foregoing embodiments are specific embodiments for implementing the present application, and in practical applications, various changes can be made in form and details without departing from the spirit and range.

Claims

An intelligent sales container includes a cabinet provided with at least one storage space, and a measurement light curtain disposed on an opposite surface of the cabinet, and the measurement light curtain is close to an item access opening in the cabinet. Where it is located, a camera disposed in each placement space, and a main control module communicatively connected to the measurement light curtain and the camera, respectively, wherein the light generated by the measurement light curtain covers the item access port;

Obtaining, by the measurement light curtain, the size information of an item passing through the item access port under the control of the main control module;

The camera photographs items in the placement space under the control of the main control module;

The main control module transmits the size information and an image obtained by the camera to an article recognition device.
The smart vending container according to claim 1, wherein the measurement light curtain includes at least one pair of light transmitters and light receivers, and wherein each pair of light transmitters and light receivers are separated from each other and opposite to each other.
The smart vending container according to claim 2, wherein the measurement light curtain comprises a single-sided emission measurement light curtain or a multi-sided emission measurement light curtain;

All the light emitters in the single-sided emission measurement light curtain are located on the same side, and the corresponding light receivers are disposed on the other side opposite to the position of the light emitters;

At least one light emitter and at least one light receiver in the multi-facet emission measurement light curtain are located on the same side.
The intelligent sales container according to any one of claims 1 to 3, wherein the measurement light curtain is disposed on two opposite sides of the cabinet body,

and / or,

The measuring light curtain is disposed at the bottom of the cabinet and the top of the cabinet.
The intelligent sales container according to any one of claims 2 to 4, wherein a distance between optical axes in the measurement light curtain is determined by a size of an item placed in the cabinet, and the optical axis is determined by the light emitter The emitted light is formed.
The smart vending container according to any one of claims 1 to 5, further comprising a cabinet door, the cabinet door being hinged with the cabinet body;

The main control module is specifically configured to:

After detecting that the cabinet door is opened, the measurement light curtain is turned on, and size information of each item passing through the measurement light curtain is acquired;

After detecting that the cabinet door is closed, the measurement light curtain is closed, and the camera is controlled to shoot the objects in the placement space.
The smart sales container according to any one of claims 1 to 6, wherein the storage space is formed by partitions installed in the cabinet, and the partitions are high in the middle and the height is gradually lowered to both ends. The layer partition is bridged between two opposite sides of the cabinet body;

The camera is disposed on the top of the placement space.
The smart sales container according to claim 7, wherein at least one partition bar is provided on a side of the layer partition facing the top of the cabinet body, and the partition bar is used to be fixed on the layer partition. article.
The smart vending container according to any one of claims 1 to 6, wherein the smart vending container further comprises: a mirror covering a top of the placement space and a mirror surface facing the bottom of the placement space;

A camera is set at the bottom of the placement space, and the lens of the camera is directly facing the mirror in the placement space where the camera is located, and the angle of view of the camera covers the mirror surface of the mirror in the placement space where the camera is located;

The main control module controls the camera to shoot the mirror in the placement space.
A method for item identification, wherein the method for identifying an item taken out of or placed in a smart sales container according to any one of claims 1 to 9 includes:

Receive the size information of the items through the item access port obtained by the measurement light curtain;

Identify the items and quantity passing through the item access port according to the size information;

Receiving an image obtained by a camera taking an item in a placement space;

Identifying the items and quantity in the placement space according to the image;

The type and quantity of the items taken out or put into the cabinet are determined according to the items and quantities passing through the item taking and placing port and the items and quantities in the storage space.
The method for item identification according to claim 10, wherein before receiving the size information of the item obtained through the item access port obtained by the measurement light curtain, the method for item identification further comprises:

The size information of the article is stored in advance, and the images of the article taken in different viewing angle ranges are stored.
The method for identifying an article according to any one of claims 10 to 11, wherein determining from the cabinet is based on the items and quantity through the item access port and the items and quantity in the storage space. The types and quantity of items taken out of the body, including:

Using the identified items and quantities passing through the item access port as a first identification result;

Compare the items in the currently recognized image with the items in the last recognized image, and compare the number of items in the currently recognized image with the number of items in the last recognized image to determine the current image recognition method The type and quantity of the items taken out or put into the cabinet, and used as the second identification result;

Determine whether the first recognition result is the same as the second recognition result, and if they are the same, use any of the recognition results as a shopping list output;

If they are not the same, the monetary value corresponding to each recognition result is calculated, and the recognition result with the higher monetary value in the two ways is output as a shopping list.
The method for item identification according to claim 12, before outputting the shopping list, further comprising:

Detect whether the current user performs correction processing on the items placed abnormally in the cabinet, and if so, determine a corresponding processing list according to the user's correction processing, and add the processing list to the shopping list.
The method for item identification according to claim 13, wherein after receiving an image obtained by the camera taking an item in a placement space and before detecting whether a current user performs correction processing on an item placed abnormally in the cabinet, Also includes:

Judging whether there is an abnormally placed item in the cabinet according to the image, and if so, sending information about the abnormally placed item to the user terminal;

and / or,

According to the previous transaction record of the smart sales container, it is determined whether the monetary value corresponding to the first recognition result in the previous transaction is smaller than the monetary value corresponding to the second recognition result, and if it is, send an abnormal information to the user terminal.
An article recognition device, wherein the article recognition device is communicatively connected to the smart vending container according to any one of claims 1 to 9, and the article recognition device comprises: a communication module, a light curtain recognition module, and an image recognition module And identifying modules;

The communication module is configured to receive size information of an item obtained through the item access port obtained by a measurement light curtain;

The light curtain identification module is configured to identify items and quantities passing through the item access port according to the size information;

The communication module is further configured to receive an image obtained by a camera taking an item in the placement space;

The image recognition module is configured to identify items and quantities in the placement space according to the image;

The determining module is configured to determine the type and quantity of the items taken out or put into the cabinet according to the items and quantities passing through the item access port and the items and quantities in the storage space.
A server including:

At least one processor; and

A memory connected in communication with the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 10 to 14. Of item identification.
A computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the method for identifying an article according to any one of claims 10 to 14 is implemented.