WO2019237243A1 - 一种物品识别的方法、装置、服务器和可读存储介质 - Google Patents

一种物品识别的方法、装置、服务器和可读存储介质 Download PDF

Info

Publication number
WO2019237243A1
WO2019237243A1 PCT/CN2018/090778 CN2018090778W WO2019237243A1 WO 2019237243 A1 WO2019237243 A1 WO 2019237243A1 CN 2018090778 W CN2018090778 W CN 2018090778W WO 2019237243 A1 WO2019237243 A1 WO 2019237243A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
item
items
recognition
identification
Prior art date
Application number
PCT/CN2018/090778
Other languages
English (en)
French (fr)
Inventor
张站朝
廉士国
裘品浩
Original Assignee
深圳前海达闼云端智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳前海达闼云端智能科技有限公司 filed Critical 深圳前海达闼云端智能科技有限公司
Priority to PCT/CN2018/090778 priority Critical patent/WO2019237243A1/zh
Publication of WO2019237243A1 publication Critical patent/WO2019237243A1/zh

Links

Classifications

    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers

Definitions

  • the present application relates to the field of intelligent retail, and in particular, to a method, device, server, and readable storage medium for item identification.
  • the inventor has discovered in the process of studying the prior art that the key to the automatic settlement of the intelligent sales container is to identify the type and quantity of the items purchased by the user.
  • the smart sales container usually uses a camera to shoot the space inside the smart sales container to identify the objects in the image captured by the camera.
  • the appearance and color of various products on the market are very similar, which increases the difficulty of identifying items in the images taken by the smart container, and is prone to misidentification, which seriously affects the user experience.
  • a technical problem to be solved in some embodiments of the present application is to provide a method, device, server, and storage medium for item identification, reduce misidentification of items in a smart sales container, and improve accuracy of item identification.
  • An embodiment of the present application provides a method for item identification, including: receiving an image of an item in each placement space transmitted by a smart sales container, wherein the placement space is set inside the smart sales container; parsing the image to obtain the Identification information, the identification information includes the type and quantity of the identified items, and the identification information of the unidentified items; determining whether there are unidentified items in the image according to the identification information; if yes, obtaining the recognition result of manually identifying the items in the image, According to the identification result and the identification information, determine the type and quantity of the items in the intelligent sales container; otherwise, determine the type and quantity of the items in the intelligent sales container based on the identification information.
  • An embodiment of the present application further provides an article identification device, including: a communication module, an identification information acquisition module, a judgment module, a first determination module, and a second determination module; the communication module is configured to receive each transmission from the smart vending cabinet.
  • the identification information acquisition module is used to analyze the image to obtain the identification information of the items in the image, and the identification information includes the type and quantity of the identified items, and unidentified items
  • the identification information is used to determine whether there are unidentified items in the image according to the identification information;
  • the first determination module is used to obtain the recognition result of manually identifying the items in the image when it is determined that there are unidentified items, According to the identification result and the identification information, determine the type and quantity of the items in the smart container;
  • the second determination module is used to determine the type and number of the items in the intelligent container based on the identification information when it is determined that there are no unidentified items. Quantity.
  • An embodiment of the present application further provides a server, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor
  • the processor executes to enable at least one processor to execute the method for item identification described above.
  • An embodiment of the present application further provides a computer-readable storage medium storing a computer program, and the computer program is executed by a processor to implement the foregoing method for item identification.
  • the smart container after analyzing the image transmitted by the smart container and obtaining the identification information of the image, it is determined whether there are unidentified items in the current image according to the identification information. , Then obtain the recognition result of manual recognition of the article in the image. Due to the high accuracy of manual recognition, by obtaining the recognition result of the artificial recognition of the image, the probability of misrecognition due to similar or obstructed items in the image is reduced, and the accuracy of the recognition of the items in the image is greatly improved. And because only the unrecognized items in the image are manually identified, rather than identifying all the items in the image by manual recognition, the efficiency of identifying the items in the image is improved.
  • FIG. 1 is a specific flowchart of a method for identifying an article in a first embodiment of the present application
  • FIG. 2 is a schematic diagram of an image added with a marker in the method for identifying an article in the first embodiment of the present application
  • FIG. 3 is a detailed flowchart of a method for identifying an article in a second embodiment of the present application.
  • FIG. 4 is a schematic diagram of an image added with a mark in the method for identifying an article in a second embodiment of the present application
  • FIG. 5 is a schematic diagram of an image with added confidence in an article identification method in a second embodiment of the present application.
  • FIG. 6 is a schematic diagram of a specific structure of an article identification device in a third embodiment of the present application.
  • FIG. 7 is a schematic diagram of a specific structure of a server in a fourth embodiment of the present application.
  • the first embodiment of the present application relates to a method for item identification, and the specific process of the item identification method is shown in FIG. 1.
  • Step 101 Receive images of items in each placement space transmitted by the smart vending container. Wherein, the storage space is set inside the intelligent sales container.
  • the article identification method is applied to an article identification device, and the intelligent sales container is communicatively connected to the article identification device, such as a wireless communication connection (4G / 5G communication).
  • the intelligent sales container is communicatively connected to the article identification device, such as a wireless communication connection (4G / 5G communication).
  • At least one storage space is set in the intelligent sales container.
  • the main control module of the intelligent sales container controls the camera to acquire images of the objects in the storage space where the camera is located, and sends the acquired images of the objects in the storage space to the article identification device. It can be understood that the image captured by the camera should be comprehensive, or the occlusion between items in the captured image should be reduced as much as possible, so at least one wide-angle lens can be set in the smart sales container.
  • the object recognition device can capture images of items in the space in real time, and can also be obtained when the door is opened and closed, or it can be obtained when the smart container is closed. This embodiment does not limit the time for capturing images of items in the placement space.
  • taking images of the items in the placement space when the door is opened and closed can ensure that the images of the items in the smart container are obtained in time, avoiding the object recognition device identifying the Waste of resources caused by objects.
  • the storage space in the smart sales container is separated by layer partitions.
  • the layer partition can be curved, triangular or trapezoidal, and the camera can be set on the top of each storage space.
  • a mirror can also be installed on each layer of partitions. The mirror surface of the mirror faces the bottom of the placement space.
  • a height-adjustable bracket is set at the bottom of the placement space, and a camera is mounted on the bracket. It can be understood that any type of camera can be installed in the above smart sales container.
  • the partition plate is curved, triangular or trapezoidal, the camera's shooting angle is wider, so that the smart sales container can obtain a more comprehensive image of the items in the space.
  • Step 102 Parse the image to obtain the identification information of the items in the image.
  • the identification information includes the type and quantity of the identified items, and the identification information of the unidentified items.
  • the item recognition device obtains the feature information in the current image, and determines the recognition information of the item in the current image according to the feature information and the image recognition model.
  • the image recognition model is constructed based on the images stored in the pre-stored image recognition training set.
  • the image recognition training set includes images of the item and types of the item taken in different viewing angle ranges.
  • the item recognition device obtains feature information of the items in the image in the image recognition training set, and the feature information includes: color, light intensity value, etc., and performs deep learning based on the obtained feature information to establish an image recognition model.
  • the item recognition device obtains the feature information in the received image and brings the obtained feature information into the image recognition model.
  • the type of the article in the image can be identified by the image recognition model, and the type of each item identified can be counted. The type of each item is determined.
  • the image recognition model can quickly identify the image, and the type and number of items in the obtained image.
  • the item recognition device can also identify the items in the figure by means of comparison.
  • This method matches the obtained image with the sub-image containing the item in the image training set, and determines the type of the item in the obtained image according to the similarity.
  • the obtained identification information may be the type, location information of the identified items, and marking information of the unidentified items; counting the types of the items may determine the quantity of each item.
  • the identification information of the unidentified item may be the position information of the unidentified item in the image and / or the mark of the unidentified item.
  • the above identification information may be completely presented in the form of data. In the case that the item cannot be identified due to obstruction or the image is too bright or too dark, the identification information is marked with the identification information of the unidentified item.
  • the form of the identification information is not limited to the forms listed above, and may be other forms, which is not limited in this embodiment.
  • Step 103 Determine whether there are unidentified items in the image according to the identification information. If yes, go to step 104; otherwise, go to step 105.
  • the article recognition device can directly determine whether the identification information of the unrecognized article exists in the identification information of the image, and if it exists, it determines that the unrecognized article exists in the image; if it does not exist, it determines that the unrecognized article does not exist. Identify items.
  • the identification information also includes the confidence level of the identified items; the confidence level is the maximum value of the matching degree between the sub-images of the items contained in the image and each image in the image recognition training set. You can determine whether there is an unrecognized item in the image according to the confidence level of the item, that is, determine whether the confidence level of the item is less than a preset value; if it is, determine that the unrecognized item exists in the image; otherwise, determine that there is no unrecognized item in the image.
  • the article recognition device may cut the obtained image into a plurality of sub-images according to the identified features, and each sub-image includes an article.
  • the feature information of the items in the sub-image is matched with the feature information of the items in each image in the image training set to obtain the matching degree, and the maximum value of the matching degree is used as the confidence degree of the article in the sub-image.
  • a preset The value is set to 0.9, that is, when the confidence level of the article in the image is lower than 0.9, it is determined as an unidentified substance.
  • the preset value can be set according to actual needs, and is not limited to the value in this embodiment.
  • the difference between the identification information of the current analysis image and the identification information of the previous analysis image is too large, it is also determined that there are unrecognized items in the current image. For example, if the difference between the two identification information is 30%, the current information is determined. There are unidentified items in the image.
  • a high priority level is set for the image, where the order of manually identifying the image with the high priority level precedes the non-high priority level.
  • the images are manually identified in the order.
  • the article identification device may add a high-priority identifier to the image, and the identifier may be used to indicate the processing time of the image. For example, it is confirmed that there is an unidentified item in image A. If the mark "1A" indicates a high-priority, “1A” means that the processing time of the image is 5 minutes. When you manually identify the image, you can see the “1A” mark to know the priority level of the image and the length of time corresponding to the priority level.
  • Step 104 Obtain a recognition result of manual recognition of the items in the image, and determine the type and quantity of the items in the smart container according to the recognition results and the identification information.
  • an unrecognized mark is added to the unrecognized item in the image according to the mark information of the unrecognized item; the image after adding the unrecognized mark is transmitted to a manual recognition device, and the artificial recognition device obtains the artificial pair after adding the unrecognized mark The recognition result of the item indicated by the unrecognized mark in the image; obtain the recognition result transmitted by the artificial recognition device.
  • the article recognition device adds an unrecognized mark to an unrecognized article in the image.
  • Unrecognized marks can be special characters, such as adding "unrecognized” to unrecognized items, or adding a box to unrecognized items, such as the solid line frame in Figure 2, and Figure 2 is the image with the mark added Among them, triangular items are identified items, and cylindrical items with solid wire frames are unidentified items. This embodiment does not limit the form of the unidentified mark, and can be set as required.
  • the image after the unrecognized mark is added is transmitted to the artificial recognition device. Of course, a reminder to recognize the image can be sent to the artificial recognition device.
  • the recognition result is input to the manual recognition device.
  • the recognition result may be in the form of data, including the position of the unrecognized item in the image, and the type and quantity of the item; or the recognition result may be an image obtained by manually labeling the type of the unrecognized item in the image, according to the image
  • the label information can determine the quantity of each item in the image, and this embodiment does not limit the form of the recognition result.
  • the type and quantity of the items in the smart container determined this time and the items in the smart container determined last time Compare the type and quantity of the product to obtain the comparison result; determine the type and quantity of the items that are taken out or put into the smart container based on the comparison result.
  • the type and quantity of the items in the smart sales container determined this time are compared with the types and quantities of the items in the smart sales container determined last time, and the difference in the types of items in the smart sales container and the type of each are obtained twice.
  • the number of items is different, so that the type and quantity of items taken out or put into the smart container can be determined and output as a shopping list.
  • the shopping list can be output in the following form: the shopping list is sent to the user's terminal, or it is output in paper form, or it can be displayed on the display screen on the smart container.
  • the types and quantities of the items in the image obtained when the door is opened and the types of items in the image obtained when the door is closed are identified separately. And the number; compare the recognition result of the image obtained when the door is opened with the recognition result of the image obtained when the door is closed, so as to determine the type and quantity of the items taken out or put in the smart container.
  • Step 105 Determine the type and quantity of the items in the smart container according to the identification information.
  • the types and quantities of the identified items in the identification information can be directly obtained and used as the types and quantities of the items in the current smart sales container.
  • the types and quantities of the items in the smart container after determining the types and quantities of the items in the smart container based on the identification information, determine the types and quantities of the items that are taken out of or put into the smart container based on the determined types and quantity of the items in the smart container, and Output as a shopping list; receive feedback information transmitted by the user, the feedback information is used to indicate whether the shopping list is correct; based on the feedback information, determine whether the shopping list is correct; if it is incorrect, obtain the recognition result of manually identifying the items in the image; According to the identification result and the identification information, the type and quantity of the items in the current smart container are re-determined; the type and quantity of the items in the current smart container are re-determined; Kind and quantity are output.
  • the types and quantities of the items in the smart sales container determined this time are compared with the types and quantities of the items in the smart sales container determined last time, and the types of items in the smart sales container and the number of each item are obtained twice. The difference, so that the type and quantity of items taken out or put into the smart container this time can be determined and output as a shopping list.
  • the user may transmit feedback information to the article identification device through the terminal or directly input feedback information to the article identification device.
  • the article identification device receives feedback information transmitted by the user, and the feedback information includes an identifier for indicating whether the shopping list is correct, such as marking “correct” and “error”.
  • the object recognition device judges whether the shopping list is correct according to the feedback information.
  • the automatic recognition device If it is incorrect, it sends the image to the manual recognition device to obtain the recognition result of manual recognition of the items in the image, and re-determines the current intelligence based on the recognition result and the recognition information.
  • the type and quantity of items in the sales container According to the re-determined type and quantity of the items in the current intelligent sales container, the type and quantity of the items taken out or put into the intelligent sales container can be re-determined and output. If the feedback information is correct, it may not be processed.
  • the item recognition device sets a low priority level on the image, wherein the image with the low priority level is set
  • the order of manual recognition is later than the order of non-low priority images for manual recognition. Because the feedback information is sent by the user through the terminal, it may take a long time for the item identification device to receive the feedback information (for example, the feedback information is received after 24 hours). Therefore, the item identification device adds a low-priority response to the image.
  • the identifier is used to indicate that the order of manual recognition processing of the image is later. For example, it is determined that the shopping list is incorrect through feedback information.
  • the recognition result of manual recognition of the article in the image Due to the high accuracy of manual recognition, by obtaining the recognition result of the artificial recognition of the image, the probability of misrecognition due to similar or obstructed items in the image is reduced, and the accuracy of the recognition of the items in the image is greatly improved. degree.
  • the efficiency of identifying the items in the image is improved.
  • the second embodiment of the present application relates to a method for item identification. This embodiment is further improved based on the first embodiment.
  • the specific improvement is that the recognition result is that the type and After getting the number of images. After the item recognition device obtains the recognition result of manually identifying the items in the image, it adds the recognition result to the image recognition training set.
  • the specific process is shown in Figure 3:
  • Step 301 Receive images of items in each placement space transmitted by the smart vending container. Wherein, the storage space is set inside the intelligent sales container.
  • Step 302 Parse the image to obtain the identification information of the items in the image.
  • the identification information includes the type and quantity of the identified items, and the identification information of the unidentified items.
  • Step 303 Determine whether there are unidentified items in the image according to the identification information. If yes, go to step 304; otherwise, go to step 305.
  • Step 304 Obtain a recognition result of manual recognition of the items in the image, and determine the type and quantity of the items in the smart container according to the recognition results and the identification information. After this step is performed, step 306 is performed.
  • the item identification device adds a mark to each item in the image according to the identification information, adds an identified mark to the identified item, and adds an unidentified mark to the unidentified item.
  • the article recognition device transmits the tagged image to the artificial recognition device, and the artificial recognition device obtains the artificial identification mark of the article indicated by the unrecognized mark in the tagged image and the manual recognition of the article in the tagged image.
  • Correction information for the item indicated by the identified mark The article identification device obtains the identification label and the correction information transmitted by the manual identification device, and uses the identification label and the correction information as a recognition result.
  • an identified mark can be added to the identified items in the image, and an unrecognized mark can be added to the unrecognized items in the image; both the identified mark and the unrecognized mark can be as required Make settings.
  • the image to which the mark is added is shown in FIG. 4.
  • the recognized mark is a combination of the realization frame and the type name of the recognized item, and the unrecognized mark is a dotted frame.
  • the article recognition device transmits the tagged image to the manual recognition device. Of course, it is possible to send a reminder to recognize the image to the manual recognition device.
  • the artificial recognition device obtains the artificial identification mark of the item indicated by the unrecognized mark in the image after the mark is added, and the identification mark is the type of the unrecognized article; and the artificial correction of the article indicated by the recognized mark in the image after the mark is added
  • correction information is the result of manual identification of the incorrectly identified items among the identified items. For example, as shown in FIG. 4, A is an identified item, the type of A is cola, and the correction information of A is: the type of A is fruit juice.
  • the identification information also includes the confidence level of the identified items
  • the item identification device adds a mark to each item in the image according to the identification information, and adds to-be-corrected items that are identified and whose confidence level is lower than a preset value.
  • Mark add an identified mark to an item that is identified and has a confidence level higher than a preset value, and add an unidentified mark to an unidentified item.
  • the mark to be corrected indicates that the type of the item may be wrong.
  • the article recognition device transmits the tagged image to the artificial recognition device, and the artificial recognition device obtains the artificial identification mark of the article indicated by the unrecognized mark in the tagged image and the manual recognition of the article in the tagged image.
  • the article identification device obtains the identification label and the correction information transmitted by the manual identification device, and uses the identification label and the correction information as a recognition result. It can be understood that, in order to facilitate the manual viewing of the identification information in the image, the identification information can also be added to the image, such as adding confidence to the image, as shown in FIG. 5.
  • the preset value can be set as required, for example, the preset value is 0.9.
  • the manual identification device may not manually identify the items marked with the identified mark, and only manually identify the items marked with the mark to be corrected and the unrecognized mark, or the unmarked items (such as the missing-identified item). Manually identify and label identifying information.
  • Step 305 Determine the type and quantity of the items in the smart sales container according to the identification information.
  • Step 306 Add the recognition result to the image recognition training set.
  • the recognition result is an image obtained by manually labeling the type and number of articles in the image. That is, each item in the image is marked with a corresponding type, and the number corresponding to each item is marked in the image.
  • the article recognition device may add the recognition result obtained each time to the image recognition training set at a fixed time interval, and may also add the obtained recognition result to the image recognition training set each time. This embodiment adopts a manner of adding the recognition result obtained each time to the image recognition training set.
  • the recognition results after adding the recognition results to the image recognition training set, determine whether the number of recognition results added to the image recognition training set reaches a preset number; if so, according to the images in the image recognition training set obtained after adding the recognition results To update the image recognition model.
  • the number of added recognition results can be counted through a counter. Each time a recognition result is added, the counter is incremented by one. When the number of added recognition results reaches a preset number, the article recognition device is triggered to perform deep learning on the images in the image recognition training set obtained after adding the recognition results, reconstruct the image recognition model, and use the updated image recognition model. For the next image recognition.
  • the preset number can be 3000, 5000, etc. The preset number can be set according to actual needs, which is not limited in this embodiment.
  • steps 301 to 303 and step 305 are substantially the same as steps 101 to 103 and 105 in the first embodiment, and will not be repeated here.
  • the method for item recognition adds the recognition result to the image recognition training set, which enriches the image training set, and because the recognition result is an accurate result, it is convenient for subsequent learning based on the image training set, and improves the image recognition device for the image
  • the accuracy of the identification of the items in it Each item in the image is marked by the identification information, and the item that needs to be manually identified is determined according to the mark.
  • the accuracy of the recognition of the item in the image is improved by manual recognition.
  • the article identification device can also perform manual recognition only on the articles marked with the correction mark and the articles marked with the unrecognized mark, thereby accelerating the speed of the manual recognition, and thus the speed of obtaining the results of the manual recognition.
  • the third embodiment of the present application relates to an article identification device.
  • the article identification device 60 includes a communication module 601, an identification information acquisition module 602, a determination module 603, a first determination module 604, and a second determination module 605.
  • the specific structure As shown in Figure 6.
  • the communication module 601 is configured to receive an image of an item in each placement space transmitted by the smart sales container, where the placement space is disposed inside the smart sales container.
  • the identification information acquisition module 602 is configured to analyze the image to obtain the identification information of the items in the image.
  • the identification information includes the type and quantity of the identified items, and the identification information of the unidentified items.
  • the determining module 603 is configured to determine whether an unidentified item exists in the image according to the identification information.
  • the first determining module 604 is configured to obtain a recognition result of manual recognition of an item in the image when it is determined that there are unrecognized items, and determine the type and quantity of the items from the smart container according to the recognition result and the identification information.
  • the second determining module 605 is configured to determine the type and quantity of the items in the smart container according to the identification information when it is determined that there are no unidentified items.
  • This embodiment is an embodiment of a virtual device corresponding to the foregoing method for identifying an item.
  • the technical details in the foregoing embodiment of the method are still applicable in this embodiment, and details are not described herein again.
  • the fourth embodiment of the present application relates to a server 70, whose structure is shown in FIG. It includes: at least one processor 701; and a memory 702 communicatively connected to the at least one processor; wherein the memory 702 stores instructions executable by the at least one processor 701, and the instructions are executed by the at least one processor 701, so that at least one The processor 701 is capable of executing a method of item identification.
  • the memory 702 and the processor 701 are connected in a bus manner.
  • the bus may include any number of interconnected buses and bridges.
  • the bus links one or more processors 701 and various circuits of the memory 702 together.
  • the bus can also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art, so they are not described further herein.
  • the bus interface provides an interface between the bus and the transceiver.
  • a transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium.
  • the data processed by the processor 701 is transmitted on a wireless medium through an antenna. Further, the antenna also receives the data and transmits the data to the processor 701.
  • the processor 701 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions.
  • the memory 702 may be used to store data used by the processor when performing operations.
  • processor in this embodiment can execute the implementation steps in the foregoing method embodiments, and the specific execution functions are not described in detail. For technical details in the method embodiments, details are not described herein again.
  • a fifth embodiment of the present application relates to a computer-readable storage medium.
  • the readable storage medium is a computer-readable storage medium, and the computer-readable storage medium stores computer instructions that enable a computer to execute the first The method for item identification involved in the five embodiments.
  • the display method in the above embodiments is implemented by a program instructing related hardware.
  • the program is stored in a storage medium and includes several instructions to make a device (may It is a single-chip microcomputer, a chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application.
  • the aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), and a random access memory (RAM, Random-Access Memory), magnetic disks or optical disks, and other media that can store program code.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Cash Registers Or Receiving Machines (AREA)

Abstract

本申请涉及智能零售领域，尤其涉及一种物品识别的方法、装置、服务器和可读存储介质。本发明的实施例提供了一种物品识别的方法，包括：接收智能售货柜传输的每个放置空间内物品的图像；解析图像获得图像中物品的识别信息，识别信息包括识别出的物品的种类和数量，以及未识别物品的标记信息；根据识别信息判断图像中是否存在未识别物品；若是，获取对图像中的物品进行人工识别的识别结果，根据识别结果和识别信息，确定智能售货柜中物品的种类和数量；否则，根据识别信息，确定智能售货柜中物品的种类和数量。本实施例的物品识别的方法，减少对智能售货柜内物品的误识别，提高对物品识别的准确度。

Description

一种物品识别的方法、装置、服务器和可读存储介质 技术领域
本申请涉及智能零售领域，尤其涉及一种物品识别的方法、装置、服务器和可读存储介质。
 背景技术
随着科技的不断发展，移动支付的方式已经深入人们的生活，而为了便于人们可随时购买物品，出现了智能售货柜。智能售货柜无需人看守、结算，人们通过智能售货柜即可购买物品。一般的交易过程为：扫码打开柜门、用户选用物品、关门自动结算三个过程。
 技术问题
发明人在研究现有技术过程中发现，智能售货柜可自动结算的关键在于，识别用户购买的物品种类和数量。智能售货柜通常通过摄像头对智能售货柜的柜内空间进行拍摄，对摄像头拍摄的图像中的物品进行识别。但是，目前市场上各种商品的外形、颜色相似的情况非常多，增加了对智能售货柜拍摄的图像中物品的识别的难度，容易出现误识别的情况，严重影响用户的使用体验感。
 技术解决方案
本申请部分实施例所要解决的技术问题在于提供一种物品识别的方法、装置、服务器和存储介质，减少对智能售货柜内物品的误识别，提高对物品识别的准确度。
本申请的一个实施例提供了一种物品识别的方法，包括：接收智能售货柜传输的每个放置空间内物品的图像，其中，放置空间设置在智能售货柜内部；解析图像获得图像中物品的识别信息，识别信息包括识别出的物品的种类和数量，以及未识别物品的标记信息；根据识别信息判断图像中是否存在未识别物品；若是，获取对图像中的物品进行人工识别的识别结果，根据识别结果和识别信息，确定智能售货柜中物品的种类和数量；否则，根据识别信息，确定智能售货柜中物品的种类和数量。
本申请的一个实施例还提供了一种物品识别装置，包括：通信模块、识别信息获取模块、判断模块、第一确定模块和第二确定模块；通信模块用于接收智能售货柜传输的每个放置空间内物品的图像，其中，放置空间设置在智能售货柜内部；识别信息获取模块用于解析图像获得图像中物品的识别信息，识别信息包括识别出的物品的种类和数量，以及未识别物品的标记信息；判断模块用于根据识别信息判断图像中是否存在未识别物品；第一确定模块用于，在确定存在未识别物品的情况下，获取对图像中的物品进行人工识别的识别结果，根据识别结果和识别信息，确定从智能售货柜中物品的种类和数量；第二确定模块用于，在确定不存在未识别物品的情况下，根据识别信息，确定智能售货柜中物品的种类和数量。
本申请实施例还提供了一种服务器，包括：至少一个处理器；以及，与至少一个处理器通信连接的存储器；其中，存储器存储有可被至少一个处理器执行的指令，该指令被至少一个处理器执行，以使至少一个处理器能够执行上述的物品识别的方法。
本申请实施例还提供了一种计算机可读存储介质，存储有计算机程序，该计算机程序被处理器执行时实现上述的物品识别的方法。
 有益效果
相对于现有技术而言，本申请部分实施例中在解析智能售货柜传输的图像并获得该图像的识别信息后，根据该识别信息判断当前图像中是否存在未识别物品，若存在未识别物品，则获取对该图像中的物品进行人工识别的识别结果。由于人工识别的准确度高，通过获取对该图像的人工识别的识别结果，降低了因图像中存在相似物品或被遮挡的物品造成误识别的概率，大大提高了对图像中物品的识别的准确度；且由于仅对图像中的未识别物品采用人工识别，而非通过人工识别的方式识别图像中所有物品，提高了对图像中物品识别的效率。
 附图说明
一个或多个实施例通过与之对应的附图中的图片进行示例性说明，这些示例性说明并不构成对实施例的限定，附图中具有相同参考数字标号的元件表示为类似的元件，除非有特别申明，附图中的图不构成比例限制。
图1是本申请第一实施例中物品识别的方法的具体流程示意图；
图2是本申请第一实施例中物品识别的方法中添加了标记的图像示意图；
图3是本申请第二实施例中物品识别的方法的具体流程示意图；
图4是本申请第二实施例中物品识别的方法中添加了标记的图像示意图；
图5是本申请第二实施例中物品识别的方法中添加了置信度的图像示意图；
图6是本申请第三实施例中物品识别装置的具体结构示意图；
图7是本申请第四实施例中服务器的具体结构示意图。
 本发明的实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请部分实施例进行进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本申请，并不用于限定本申请。然而，本领域的普通技术人员可以理解，在本申请的各实施例中，为了使读者更好地理解本申请而提出了许多技术细节。但是，即使没有这些技术细节和基于以下各实施例的种种变化和修改，也可以实现本申请所要求保护的技术方案。
本申请的第一实施例涉及一种物品识别的方法，该物品识别方法的具体流程如图1所示。
步骤101：接收智能售货柜传输的每个放置空间内物品的图像。其中，放置空间设置在智能售货柜内部。
具体的说，该物品识别方法运用在物品识别装置上，智能售货柜与物品识别装置通信连接，如无线通信连接（4G/5G通信）的方式。智能售货柜内设置有至少一个放置空间，智能售货柜的主控模块控制摄像头获取摄像头所在放置空间内的物品的图像，并向物品识别装置发送获取到的放置空间内物品的图像。可以理解的是，摄像头拍摄的图像应当全面，或者尽可能减少拍摄的图像中物品之间的遮挡情况，因而，可以在智能售货柜内设置至少一个广角镜头。
需要说明的是，物体识别装置可以实时拍摄放置空间内物品的图像，也可以在开门和关门时拍摄获得，也可以是在智能售货柜关门时拍摄获得。本实施例不限制拍摄放置空间内物品的图像的时间。
值得一提的是，在开门和关门时拍摄放置空间内物品的图像，可以保证及时获取智能售货柜内的物品的图像，避免了在无人操作智能售货柜时，物体识别装置识别图像中的物体造成的资源浪费。
具体实现中，智能售货柜内的放置空间由层隔板分隔而成。为了提高智能售货柜中摄像头对放置空间内的物品的拍摄的质量，层隔板可以为弧形、三角形或梯形，摄像头可以设置在每个放置空间的顶部。当然，还可以在每个层隔板上安装镜子，镜子的镜面面向放置空间的底部，在放置空间的底部中间位置设置可以调节高度的支架，该支架上安装摄像头。可以理解的是，以上的智能售货柜中支持安装任意类型的摄像头。
值得一提的是，分隔板为弧形、三角形或梯形时，摄像头的拍摄视角更广，使得智能售货柜能够获得更全面的放置空间内物品的图像。
步骤102：解析图像获得图像中物品的识别信息。其中，识别信息包括识别出的物品的种类和数量，以及未识别物品的标记信息。
一个具体实现中，物品识别装置获取当前图像中的特征信息，并根据特征信息和图像识别模型，确定当前图像中物品的识别信息，图像识别模型是根据预先存储的图像识别训练集中的图像构建获得，图像识别训练集包括在不同视角范围内拍摄的所述物品的图像以及所述物品的种类。
具体的说，物品识别装置获取图像识别训练集中的图像中物品的特征信息，特征信息包括：颜色、光的强度值等，并根据获取的特征信息进行深度学习，建立图像识别模型。物品识别装置获取接收的图像中的特征信息，将获取的特征信息带入图像识别模型中，即可通过图像识别模型识别出该图像中物品的种类，统计识别出的每个物品的种类，可以确定出每种物品的种类，通过图像识别模型可以快速识别图像，获得的图像中的物品的种类以及数量。
当然，物品识别装置还可以采用比对的方式识别图中物品，该方法将获得的图像和图像训练集中包含物品的子图像进行匹配，根据相似度确定获得的图像中的物品的种类。
需要说明的是，获取的识别信息可以是识别出的物品的种类、位置信息以及未识别物品的标记信息；统计物品的种类即可确定出每种物品的数量。未识别物品的标记信息可以为记录未识别物品在图像中的位置信息和/或该未识别物品的标记，以上识别信息可以完全以数据的形式呈现，若图像中存在物品摆放倾斜、物品被遮挡或者图像过亮或过暗而导致物品无法识别的情况，则识别信息中标注未识别物品的标记信息。识别信息的形式不限于以上列举的形式，还可以为其他形式，本实施例不对此做限制。
步骤103：根据识别信息判断图像中是否存在未识别物品，若是，则执行步骤104，否则，执行步骤105。
具体的说，物品识别装置可直接判断该图像的识别信息中是否存在未识别物品的标记信息，若存在，则判定该图像中存在未识别物品；若不存在，则判定该图像中不存在未识别物品。
一个具体的实现中，识别信息中还包括识别出的物品的置信度；置信度为图像中包含的物品的子图像与图像识别训练集中各图像的匹配度的最大值。可以根据物品的置信度判断图像中是否存在未识别物品，即判断物品的置信度是否小于预设值；若是，则判定图像中存在未识别物品，否则，则判定图像中不存在未识别物品。
具体的说，物品识别装置可以将获得的图像根据识别的特征裁切为多个子图像，每个子图像中包含有物品。将子图像中物品的特征信息与图像训练集中各图像中物品的特征信息进行匹配，获取匹配度，将匹配度的最大值作为子图像中物品的置信度。可以理解的是，物品的置信度越高，表明对该物品的种类识别的准确度越高，物品置信度越低，表明该物品的种类被误识别的概率越高；因而，可以将预设值设置为0.9，即当图像中物品的置信度低于0.9，即判定为未识别物质。当然，预设值可以根据实际需求进行设置，不限于本实施例中的值。
一个具体实现中，若此次解析图像的识别信息与上一次解析图像的识别信息差别过大，也判定为当前图像中存在未识别的物品，如，两次识别信息相差30%，则判定当前图像中存在未识别物品。
需要说明的是，根据识别信息判断图像中存在未识别物品之后且在步骤104之前，对图像设置高优先等级，其中，对设置有高优先等级的图像进行人工识别的顺序先于非高优先等级的图像进行人工识别的顺序。物品识别装置可以在图像中添加高优先等级对应的标识，该标识可以用于表示处理该图像的时长，例如，图像A中确认存在未识别物品，若标记“1A”表示高优先等级，且“1A”表示处理图像的时长为5分钟，那么人工识别图像时，看到“1A”标识，即可获知该图像的优先等级以及该优先等级对应的时长。
步骤104：获取对图像中的物品进行人工识别的识别结果，根据识别结果和识别信息，确定智能售货柜中物品的种类和数量。
一个具体实现中，按照未识别物品的标记信息对图像中的未识别物品添加未识别标记；将添加未识别标记后的图像传输至人工识别装置，由人工识别装置获取人工对添加未识别标记后的图像中的未识别标记指示的物品的识别结果；获取人工识别装置传输的识别结果。
具体的说，物品识别装置对图像中未识别的物品添加未识别标记。未识别标记可以是特殊的字符，如在未识别的物品上添加“未识别”，或者在未识别的物品上添加一个框，如图2中的实线框，图2为添加了标记的图像，其中，三角形的物品为已识别物品，有实线框的柱形物品为未识别物品。本实施例不限制未识别标记的形式，可以根据需要进行设置。将添加未识别标记后的图像传输至人工识别装置，当然，可以向人工识别装置发送识别图像的提醒。人对未识别标记指示的物品进行识别后，将识别结果输入人工识别装置。获取人工识别装置传输的识别结果，将识别结果中未识别物品的种类和数量，以及识别信息中已识别物品的种类，作为智能货柜中物品的种类和数量。该识别结果可以是数据形式，包含未识别物品在图像中的位置，以及该物品的种类及数量；或者该识别结果可以是在图像中人工标注未识别物品的种类后得到的图像，根据图像中的标注信息可以确定出图像中每种物品的数量，本实施例不对识别结果的形式进行限制。
值得一提的是，根据识别结果和识别信息，确定智能售货柜中物品的种类和数量之后，将本次确定的智能售货柜中物品的种类和数量与上一次确定的智能售货柜中的物品的种类和数量进行比较，获取比较结果；根据比较结果，确定从智能售货柜中取出或放入的物品种类和数量。
具体的说，将本次确定的智能售货柜中物品的种类和数量与上一次确定的智能售货柜中的物品的种类和数量进行比较，获取两次智能售货柜内物品的种类差别以及每种物品的数量差别，从而可以确定出本次从智能售货柜内取出或放入的物品种类和数量，并作为购物清单输出。购物清单可以通过以下形式输出：将购物清单发送至用户的终端，或者以纸质形式输出，或者在智能售货柜上通过显示屏显示。
当然，若放置空间内物品的图像由智能售货柜在开门和关门时拍摄获得，那么分别识别开门时拍摄获得的图像中的物品的种类和数量，以及识别关门时拍摄获得的图像中物品的种类和数量；将开门时获得的图像的识别结果与关门时拍摄获得的图像的识别结果进行比对，从而可以确定出从智能售货柜中取出或放入的物品种类和数量。
步骤105：根据识别信息，确定智能售货柜中物品的种类和数量。
具体的说，由于图像中不存在未识别物品，则可直接获取识别信息中识别物品的种类和数量，并作为当前智能售货柜内的物品的种类和数量。
一个具体的实现中，根据识别信息，确定智能售货柜中物品种类和数量之后，根据确定的智能售货柜中物品种类和数量，确定从智能售货柜中取出或放入的物品种类和数量，并作为购物清单输出；接收用户传输的反馈信息，反馈信息用于指示购物清单是否正确；根据反馈信息，判断购物清单是否正确；若不正确，则获取对图像中的物品进行人工识别的识别结果；根据识别结果和识别信息，重新确定当前智能售货柜中的物品的种类和数量；根据重新确定的当前智能售货柜中的物品的种类和数量，重新确定从智能售货柜取出或放入的物品的种类和数量并输出。
具体的说，将本次确定的智能售货柜中物品种类和数量与上一次确定的智能售货柜中物品种类和数量进行比较，获取两次智能售货柜内物品的种类差别以及每种物品的数量差别，从而可以确定出本次从智能售货柜内取出或放入的物品种类和数量，并作为购物清单输出。用户在获取到购物清单之后，可以通过终端向物品识别装置传输反馈信息或者直接向物品识别装置输入反馈信息。物品识别装置接收用户传输的反馈信息，反馈信息中包括用于指示购物清单是否正确的标识，如标记“正确”“错误”等。物体识别装置根据反馈信息判断购物清单是否正确，若不正确，则将图像发送至人工识别装置，获取对该图像中的物品进行人工识别的识别结果，根据识别结果和识别信息，重新确定当前智能售货柜中的物品的种类和数量。根据重新确定的当前智能售货柜中的物品的种类和数量，可以重新确定从智能售货柜取出或放入的物品的种类和数量并输出。若反馈信息正确，可以不进行处理。
需要说明的是，根据反馈信息判断购物清单不正确之后，获取对图像中的物品进行人工识别的识别结果之前，物品识别装置对图像设置低优先等级，其中，对设置有低优先等级的图像进行人工识别的顺序晚于非低优先等级的图像进行人工识别的顺序。由于反馈信息是由用户通过终端发送，存在物品识别装置接收到反馈信息已经过了较长时间（如24小时后接收到反馈信息）的情况，因此，物品识别装置为图像中添加低优先等级对应的标识，用于表明该图像进行人工识别处理的顺序靠后。例如，通过反馈信息确定购物清单不正确，若标记“aa”表示低优先等级，且“aa”表示处理图像的时长为24小时，那么人工识别图像时，查看到“aa”标识，即可获知该图像的优先等级以及该优先等级对应的时长。
相对于现有技术而言，本申请部分实施例中在解析智能售货柜传输的图像并获得该图像的识别信息后，根据该识别信息判断当前图像中是否存在未识别物品，若存在未识别物品，则获取对该图像中的物品进行人工识别的识别结果。由于人工识别的准确度高，通过获取对该图像的人工识别的识别结果，降低了因图像中存在相似物品或被遮挡的物品造成误识别的概率，大大提高了对图像中物品的识别的准确度。并且，由于仅对图像中的未识别物品采用人工识别，而非通过人工识别的方式识别图像中所有物品，提高了对图像中物品识别的效率。
本申请的第二实施例涉及一种物品识别的方法，本实施例是在第一实施例的基础上做了进一步改进，具体改进之处为：识别结果为在图像中人工标注物品的种类和数量后得到的图像。物品识别装置在获取对图像中的物品进行人工识别的识别结果之后，将识别结果添加至图像识别训练集中，具体的流程如图3所示：
步骤301：接收智能售货柜传输的每个放置空间内物品的图像。其中，放置空间设置在智能售货柜内部。
步骤302：解析图像获得图像中物品的识别信息。其中，识别信息包括识别出的物品的种类和数量，以及未识别物品的标记信息。
步骤303：根据识别信息判断图像中是否存在未识别物品，若是，则执行步骤304，否则，执行步骤305。
步骤304：获取对图像中的物品进行人工识别的识别结果，根据识别结果和识别信息，确定智能售货柜中物品的种类和数量。该步骤执行完后，执行步骤306。
一个具体实现中，物品识别装置按照识别信息对图像中的每个物品添加标记，为识别出的物品添加已识别标记以及为未识别物品添加未识别标记。物品识别装置将添加标记后的图像传输至人工识别装置，由人工识别装置获取人工对添加标记后的图像中的未识别标记指示的物品的识别标注，以及获取人工对添加标记后的图像中的已识别标记指示的物品的矫正信息。物品识别装置获取人工识别装置传输的识别标注以及矫正信息，并将识别标注以及矫正信息作为识别结果。
具体的说，可以根据识别信息中的物品的位置信息，对图像中已识别的物品添加已识别标记，对图像中未识别的物品添加未识别标记；已识别标记和未识别标记均可以根据需要进行设置。例如，添加了标记的图像如图4所示，已识别标记为实现框和已识别物品的种类名称的组合，未识别标记为虚线框。物品识别装置将添加标记后的图像传输至人工识别装置，当然，可以向人工识别装置发送识别图像的提醒。人工识别装置获取人工对添加标记后的图像中未识别标记指示的物品的识别标注，该识别标注为未识别物品的种类；获取人工对添加标记后的图像中的已识别标记指示的物品的矫正信息，矫正信息为对已识别物品中识别错误物品的进行人工识别的结果。例如，如图4所示，A为已识别物品，A的种类为可乐，A的矫正信息为：A的种类为果汁。
可以理解的是，图像中已识别标识指示的物品都需要进行人工识别，可以根据置信度判断需要进行人工识别的物品。
一个具体实现中，识别信息中还包括识别出的物品的置信度，物品识别装置按照识别信息对图像中的每个物品添加标记，为识别出且置信度低于预设值的物品添加待修正标记，为识别出且置信度高于预设值的物品添加已识别标记，以及为未识别物品添加未识别标记。其中，待修正标记表明该物品的种类可能是错误的。物品识别装置将添加标记后的图像传输至人工识别装置，由人工识别装置获取人工对添加标记后的图像中的未识别标记指示的物品的识别标注，以及获取人工对添加标记后的图像中的待修正标记指示的物品的矫正信息。物品识别装置获取人工识别装置传输的识别标注以及矫正信息，并将识别标注以及矫正信息作为识别结果。可以理解的是，为了便于人工查看图像中的识别信息，还可以将识别信息添加在图像中，如将置信度添加在图像中，如图5所示。
需要说明的是，实际应用中，预设值可以根据需要设置，如预设值为0.9。
具体实现中，人工识别装置可以不对标有已识别标记的物品进行人工识别，仅对标有待修正标记和未识别标记的物品进行人工识别或者对没有标记的物品(如被漏识别的物品)进行人工识别并标注识别信息。
步骤305：根据识别信息，确定智能售货柜中物品的种类和数量。
步骤306：将识别结果添加至图像识别训练集中。
具体的说，识别结果为在图像中人工标注物品的种类和数量后得到的图像。即该图像中每一个物品标注有对应的种类，并且在该图像中标注有每种物品对应的数量。物品识别装置可以以固定时间间隔将每次获取到的识别结果添加至图像识别训练集中，也可以每次将获取到的识别结果添加至图像识别训练集中。本实施例采用了将每次获取到的识别结果添加至图像识别训练集中的方式。
一个具体的实现中，将识别结果添加至图像识别训练集之后，判断添加至图像识别训练集中的识别结果的数量是否达到预设数量；若是，根据添加识别结果后得到的图像识别训练集中的图像，更新图像识别模型。
具体的说，统计添加识别结果的数量，可以通过计数器进行统计，每添加一个识别结果，计数器加1。当添加的识别结果的数量达到预设数量时，触发物品识别装置对添加识别结果后得到的图像识别训练集中的图像进行重新深度学习，重新构建图像识别模型，并将更新后的图像识别模型用于进行下一次的图像识别。预设数量可以为3000，5000等，预设数量可以根据实际需要进行设置，本实施例不对此进行限制。
需要说明的是，本实施例中，步骤301至步骤303、以及步骤305与第一实施例中的步骤101至步骤103、以及步骤105大致相同，此处将不再进行赘述。
本实施例中提供的物品识别的方法，将识别结果添加至图像识别训练集中，丰富了图像训练集，且由于识别结果为准确结果，便于后续根据图像训练集进行学习，提高物品识别装置对图像中物品的识别准确性。通过识别信息对图像中的每一个物品进行标记，根据标记，确定需要进行人工识别的物品，通过人工识别提高了对该图像中物品的识别的准确度。另外，物品识别装置还可以只对标有修正标记的物品和标有未识别标记的物品进行人工识别，加快人工识别的速度，从而加快获取人工识别结果的速度。
本申请的第三实施例涉及一种物品识别装置，该物品识别装置60包括：通信模块601、识别信息获取模块602、判断模块603、第一确定模块604和第二确定模块605，具体的结构如图6所示。
通信模块601用于接收智能售货柜传输的每个放置空间内物品的图像，其中，放置空间设置在智能售货柜内部。识别信息获取模块602用于解析图像获得图像中物品的识别信息，识别信息包括识别出的物品的种类和数量，以及未识别物品的标记信息。判断模块603用于根据识别信息判断图像中是否存在未识别物品。第一确定模块604用于在确定存在未识别物品的情况下，获取对图像中的物品进行人工识别的识别结果，根据识别结果和识别信息，确定从智能售货柜中物品的种类和数量。第二确定模块605用于在确定不存在未识别物品的情况下，根据识别信息，确定智能售货柜中物品的种类和数量。
本实施例是与上述物品识别的方法对应的虚拟装置实施例，上述方法实施例中技术细节在本实施例中依然适用，此处不再赘述。
需要说明的是，上述装置实施例仅仅是示意性的，并不对本申请的保护范围构成限定，在实际应用中，本领域的技术人员可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的，此处不做限制。
本申请的第四实施例涉及一种服务器70，其结构如图7所示。包括：至少一个处理器701；以及与至少一个处理器通信连接的存储器702；其中，存储器702存储有可被至少一个处理器701执行的指令，指令被至少一个处理器701执行，以使至少一个处理器701能够执行物品识别的方法。
存储器702和处理器701采用总线方式连接，总线可以包括任意数量的互联的总线和桥，总线将一个或多个处理器701和存储器702的各种电路链接在一起。总线还可以将诸如外围设备、稳压器和功率管理电路等之类的各种其他电路链接在一起，这些都是本领域所公知的，因此，本文不再对其进行进一步描述。总线接口在总线和收发机之间提供接口。收发机可以是一个元件，也可以是多个元件，比如多个接收器和发送器，提供用于在传输介质上与各种其他装置通信的单元。经处理器701处理的数据通过天线在无线介质上进行传输，进一步，天线还接收数据并将数据传送给处理器701。
处理器701负责管理总线和通常的处理，还可以提供各种功能，包括定时，外围接口，电压调节、电源管理以及其他控制功能。而存储器702可以被用于存储处理器在执行操作时所使用的数据。
需要说明的是，本实施例中的处理器能够执行上述的方法实施例中实施步骤，具体的执行功能并未详细说明，可参见方法实施例中的技术细节，此处不再赘述。
本申请的第五实施例涉及一种计算机可读存储介质，该可读存储介质为计算机可读存储介质，该计算机可读存储介质中存储有计算机指令，该计算机指令使计算机能够执行本申请第五实施例中涉及的物品识别的方法。
需要说明的是，本领域的技术人员能够理解，上述实施例中显示方法是通过程序来指令相关的硬件来完成的，该程序存储在一个存储介质中，包括若干指令用以使得一个设备（可以是单片机，芯片等）或处理器（processor）执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器（ROM，Read-Only Memory）、随机存取存储器（RAM，Random-Access Memory）、磁碟或者光盘等各种可以存储程序代码的介质。
本领域的普通技术人员可以理解，上述各实施例是实现本申请的具体实施例，而在实际应用中，可以在形式上和细节上对其作各种改变，而不偏离本申请的精神和范围。

Claims (16)

  1. 一种物品识别的方法，其中，包括：
    接收智能售货柜传输的每个放置空间内物品的图像，其中，所述放置空间设置在所述智能售货柜内部；
    解析所述图像获得所述图像中物品的识别信息，所述识别信息包括识别出的物品的种类和数量，以及未识别物品的标记信息；
    根据所述识别信息判断所述图像中是否存在未识别物品；
    若是，获取对所述图像中的物品进行人工识别的识别结果，根据所述识别结果和所述识别信息，确定所述智能售货柜中物品的种类和数量；
    否则，根据所述识别信息，确定所述智能售货柜中物品的种类和数量。
  2. 根据权利要求1所述的物品识别的方法，其中，获取对所述图像中的物品进行人工识别的识别结果，具体包括：
    按照所述未识别物品的标记信息对所述图像中的未识别物品添加未识别标记；
    将添加未识别标记后的图像传输至人工识别装置，由所述人工识别装置获取人工对所述添加未识别标记后的图像中的未识别标记指示的物品的识别结果；
    获取所述人工识别装置传输的所述识别结果。
  3. 根据权利要求1所述的物品识别的方法，其中，获取对所述图像中的物品进行人工识别的识别结果，具体包括：
    按照所述识别信息对所述图像中的每个物品添加标记，为识别出的物品添加已识别标记以及为未识别物品添加未识别标记；
    将添加标记后的图像传输至人工识别装置，由所述人工识别装置获取人工对所述添加标记后的图像中的未识别标记指示的物品的识别标注，以及获取人工对所述添加标记后的图像中的已识别标记指示的物品的矫正信息；
    获取所述人工识别装置传输的所述识别标注以及所述矫正信息，并将所述识别标注以及所述矫正信息作为所述识别结果。
  4. 根据权利要求1所述的物品识别的方法，其中，所述识别信息中还包括识别出的物品的置信度，所述置信度为所述图像中包含的所述物品的子图像与图像识别训练集中各图像的匹配度的最大值，所述图像识别训练集包括在不同视角范围内拍摄的所述物品的图像以及所述物品的种类；
    获取对所述图像中的物品进行人工识别的识别结果，具体包括：
    按照所述识别信息对所述图像中的每个物品添加标记，为识别出且置信度低于预设值的物品添加待修正标记，为识别出且置信度高于预设值的物品添加已识别标记，以及为未识别物品添加未识别标记；
    将添加标记后的图像传输至人工识别装置，由所述人工识别装置获取人工对所述添加标记后的图像中的未识别标记指示的物品的识别标注，以及获取人工对所述添加标记后的图像中的待修正标记指示的物品的矫正信息；
    获取所述人工识别装置传输的所述识别标注以及所述矫正信息，并将所述识别标注以及所述矫正信息作为所述识别结果。
  5. 根据权利要求1至3中任一项所述的物品识别的方法，其中，所述识别信息中还包括识别出的物品的置信度；
    根据所述识别信息判断所述图像中是否存在未识别物品，具体包括：
    判断所述物品的置信度是否小于预设值；
    若是，则判定所述图像中存在未识别物品，否则，则判定所述图像中不存在未识别物品；
    所述置信度为所述图像中包含的所述物品的子图像与图像识别训练集中各图像的匹配度的最大值，所述图像识别训练集包括在不同视角范围内拍摄的所述物品的图像以及所述物品的种类。
  6. 根据权利要求1至3中任一项所述的物品识别的方法，其中，根据所述识别结果和所述识别信息，确定所述智能售货柜中物品的种类和数量之后，还包括：
    将本次确定的所述智能售货柜中物品的种类和数量与上一次确定的智能售货柜中的物品的种类和数量进行比较，获取比较结果；
    根据所述比较结果，确定从所述智能售货柜中取出或放入的物品种类和数量。
  7. 根据权利要求1所述的物品识别的方法，其中，根据所述识别信息，确定所述智能售货柜中物品种类和数量之后，还包括：
    根据确定的所述智能售货柜中物品种类和数量，确定从所述智能售货柜中取出或放入的物品种类和数量，并作为购物清单输出；
    接收用户传输的反馈信息，所述反馈信息用于指示所述购物清单是否正确；
    根据所述反馈信息，判断所述购物清单是否正确；
    若不正确，则获取对所述图像中的物品进行人工识别的识别结果；
    根据所述识别结果和所述识别信息，重新确定当前智能售货柜中的物品的种类和数量；
    根据重新确定的当前智能售货柜中的物品的种类和数量，重新确定从所述智能售货柜取出或放入的物品的种类和数量并输出。
  8. 根据权利要求1所述的物品识别的方法，其中，根据所述识别信息判断所述图像中存在未识别物品之后，获取对所述图像中的物品进行人工识别的识别结果之前，所述物品识别的方法还包括：对所述图像设置高优先等级，其中，对设置有所述高优先等级的图像进行人工识别的顺序先于非高优先等级的图像进行人工识别的顺序。
  9. 根据权利要求7所述的物品识别的方法，其中，根据所述反馈信息判断所述购物清单不正确之后，获取对所述图像中的物品进行人工识别的识别结果之前，所述物品识别的方法还包括：对所述图像设置低优先等级，其中，对设置有所述低优先等级的图像进行人工识别的顺序晚于非低优先等级的图像进行人工识别的顺序。
  10. 根据权利要求2或3所述的物品识别的方法，其中，所述解析所述图像获得所述图像中物品的识别信息，具体包括：
    获取所述图像中的特征信息，并根据所述特征信息和图像识别模型，确定所述图像中物品的识别信息，其中，所述图像识别模型是根据预先存储的图像识别训练集中的图像构建获得, 所述图像识别训练集包括在不同视角范围内拍摄的所述物品的图像以及所述物品的种类。
  11. 根据权利要求10所述的物品识别的方法，其中，所述识别结果为在所述图像中人工标注物品的种类和数量后得到的图像；
    所述获取对所述图像中的物品进行人工识别的识别结果之后，所述物品识别的方法还包括：
    将所述识别结果添加至所述图像识别训练集中。
  12. 根据权利要求11所述的物品识别的方法，其中，将所述识别结果添加至图像识别训练集中之后，所述物品识别的方法，还包括：
    判断添加至所述图像识别训练集中的所述识别结果的数量是否达到预设数量；
    若是，根据添加所述识别结果后得到的图像识别训练集中的图像，更新所述图像识别模型。
  13. 根据权利要求1所述的物品识别的方法，其中，所述放置空间内物品的图像由所述智能售货柜在开门和关门时拍摄获得，或者所述放置空间内物品的图像由所述智能售货柜在关门时拍摄获得。
  14. 一种物品识别装置，其中，包括：通信模块、识别信息获取模块、判断模块、第一确定模块和第二确定模块；
    所述通信模块用于接收智能售货柜传输的每个放置空间内物品的图像，其中，所述放置空间设置在所述智能售货柜内部；
    所述识别信息获取模块用于解析所述图像获得所述图像中物品的识别信息，所述识别信息包括识别出的物品的种类和数量，以及未识别物品的标记信息；
    所述判断模块用于根据所述识别信息判断所述图像中是否存在未识别物品；
    所述第一确定模块用于，在确定存在未识别物品的情况下，获取对所述图像中的物品进行人工识别的识别结果，根据所述识别结果和所述识别信息，确定从所述智能售货柜中物品的种类和数量；
    所述第二确定模块用于，在确定不存在未识别物品的情况下，根据所述识别信息，确定所述智能售货柜中物品的种类和数量。
  15. 一种服务器，包括：
    至少一个处理器；以及，
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行如权利要求1至13任一项所述的物品识别的方法。
  16. 一种计算机可读存储介质，存储有计算机程序，其中，所述计算机程序被处理器执行时实现权利要求1至13任一项所述的物品识别的方法。
PCT/CN2018/090778 2018-06-12 2018-06-12 一种物品识别的方法、装置、服务器和可读存储介质 WO2019237243A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/090778 WO2019237243A1 (zh) 2018-06-12 2018-06-12 一种物品识别的方法、装置、服务器和可读存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/090778 WO2019237243A1 (zh) 2018-06-12 2018-06-12 一种物品识别的方法、装置、服务器和可读存储介质

Publications (1)

Publication Number Publication Date
WO2019237243A1 true WO2019237243A1 (zh) 2019-12-19

Family

ID=68841813

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/090778 WO2019237243A1 (zh) 2018-06-12 2018-06-12 一种物品识别的方法、装置、服务器和可读存储介质

Country Status (1)

Country Link
WO (1) WO2019237243A1 (zh)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111444880A (zh) * 2020-04-10 2020-07-24 海信集团有限公司 一种食材识别方法及冰箱
CN111709262A (zh) * 2020-05-19 2020-09-25 李峰 三维码信息智能识别方法和装置
CN111832420A (zh) * 2020-06-17 2020-10-27 深圳达闼科技控股有限公司 物品处理方法、装置、介质及自动售货机
CN112084803A (zh) * 2020-08-27 2020-12-15 福建天甫电子材料有限公司 电子化学品包装出货方法、装置、计算机设备
CN112735030A (zh) * 2020-12-28 2021-04-30 深兰人工智能(深圳)有限公司 售货柜的视觉识别方法、装置、电子设备和可读存储介质
CN113095851A (zh) * 2021-04-02 2021-07-09 浙江玖重科技有限公司 烟草信息采集方法、装置、系统及可读存储介质
CN113111932A (zh) * 2021-04-02 2021-07-13 支付宝(杭州)信息技术有限公司 智能货柜的物品核对方法及系统
CN113569745A (zh) * 2021-07-29 2021-10-29 上海商汤智能科技有限公司 物品识别方法、装置、系统、电子设备及存储介质
CN114462932A (zh) * 2022-01-12 2022-05-10 合肥美的智能科技有限公司 库存统计方法、无人售货柜、终端设备及存储介质
CN115810193A (zh) * 2023-02-20 2023-03-17 深圳普菲特信息科技股份有限公司 一种投料视觉识别方法、系统和可读存储介质
CN115841620A (zh) * 2022-12-16 2023-03-24 深圳云天励飞技术股份有限公司 地摊活体商品识别方法、装置、电子设备及存储介质

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102855713A (zh) * 2011-05-27 2013-01-02 东芝泰格有限公司 信息处理装置及信息处理方法
CN106781014A (zh) * 2017-01-24 2017-05-31 广州市蚁道互联网有限公司 自动售货机及其运行方法

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102855713A (zh) * 2011-05-27 2013-01-02 东芝泰格有限公司 信息处理装置及信息处理方法
CN106781014A (zh) * 2017-01-24 2017-05-31 广州市蚁道互联网有限公司 自动售货机及其运行方法

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111444880B (zh) * 2020-04-10 2023-10-31 海信集团有限公司 一种食材识别方法及冰箱
CN111444880A (zh) * 2020-04-10 2020-07-24 海信集团有限公司 一种食材识别方法及冰箱
CN111709262A (zh) * 2020-05-19 2020-09-25 李峰 三维码信息智能识别方法和装置
CN111709262B (zh) * 2020-05-19 2024-01-09 海南天鉴防伪科技有限公司 三维码信息智能识别方法和装置
CN111832420A (zh) * 2020-06-17 2020-10-27 深圳达闼科技控股有限公司 物品处理方法、装置、介质及自动售货机
CN111832420B (zh) * 2020-06-17 2023-08-29 达闼机器人股份有限公司 物品处理方法、装置、介质及自动售货机
CN112084803A (zh) * 2020-08-27 2020-12-15 福建天甫电子材料有限公司 电子化学品包装出货方法、装置、计算机设备
CN112084803B (zh) * 2020-08-27 2022-08-30 福建天甫电子材料有限公司 电子化学品包装出货方法、装置、计算机设备
CN112735030B (zh) * 2020-12-28 2022-08-19 深兰人工智能(深圳)有限公司 售货柜的视觉识别方法、装置、电子设备和可读存储介质
CN112735030A (zh) * 2020-12-28 2021-04-30 深兰人工智能(深圳)有限公司 售货柜的视觉识别方法、装置、电子设备和可读存储介质
CN113111932A (zh) * 2021-04-02 2021-07-13 支付宝(杭州)信息技术有限公司 智能货柜的物品核对方法及系统
CN113111932B (zh) * 2021-04-02 2022-05-20 支付宝(杭州)信息技术有限公司 智能货柜的物品核对方法及系统
CN113095851A (zh) * 2021-04-02 2021-07-09 浙江玖重科技有限公司 烟草信息采集方法、装置、系统及可读存储介质
CN113569745A (zh) * 2021-07-29 2021-10-29 上海商汤智能科技有限公司 物品识别方法、装置、系统、电子设备及存储介质
CN114462932A (zh) * 2022-01-12 2022-05-10 合肥美的智能科技有限公司 库存统计方法、无人售货柜、终端设备及存储介质
CN115841620A (zh) * 2022-12-16 2023-03-24 深圳云天励飞技术股份有限公司 地摊活体商品识别方法、装置、电子设备及存储介质
CN115810193B (zh) * 2023-02-20 2023-04-21 深圳普菲特信息科技股份有限公司 一种投料视觉识别方法、系统和可读存储介质
CN115810193A (zh) * 2023-02-20 2023-03-17 深圳普菲特信息科技股份有限公司 一种投料视觉识别方法、系统和可读存储介质

Similar Documents

Publication Publication Date Title
WO2019237243A1 (zh) 一种物品识别的方法、装置、服务器和可读存储介质
US11151427B2 (en) Method and apparatus for checkout based on image identification technique of convolutional neural network
US20210056498A1 (en) Method and device for identifying product purchased by user and intelligent shelf system
CN108922026B (zh) 一种自动售货机的补货管理方法、装置和用户终端
CN109165645B (zh) 一种图像处理方法、装置以及相关设备
CN106462766B (zh) 在预览模式中进行图像捕捉参数调整
WO2019232703A1 (zh) 智能售货柜、物品识别的方法、装置、服务器和存储介质
WO2020107951A1 (zh) 一种基于图像的商品结算方法、装置、介质及电子设备
US11023908B2 (en) Information processing apparatus for performing customer gaze analysis
CN107862515A (zh) 一种售货方法及系统
CN108520409B (zh) 一种快速结账方法、装置及电子设备
CN108780505A (zh) 智能售货柜、物品识别方法、装置、服务器和存储介质
WO2019096222A1 (zh) 一种基于身份识别和商品识别的无人售货方法和设备
WO2021000418A1 (zh) 图像数据处理方法和图像数据处理设备
CN110119915B (zh) 对象入库处理方法、装置和系统
JP7036401B2 (ja) 学習用サーバ、不足学習用画像収集支援システム、及び不足学習用画像推定プログラム
CN111222870B (zh) 结算方法、装置和系统
CN112508132B (zh) 一种识别sku的训练方法及装置
CN109147174A (zh) 一种无人售货方法、服务器及无人售货柜
US11023712B2 (en) Suspiciousness degree estimation model generation device
WO2021179138A1 (zh) 商超货架上商品的分析方法和系统
TWI712903B (zh) 商品資訊查詢方法和系統
CN109213397A (zh) 数据处理方法、装置和用户端
CN104574087A (zh) 一种食堂定向小额支付方法及系统
CN108510673A (zh) 一种自动射频的快速结账方法、装置及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18922962

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 19.05.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 18922962

Country of ref document: EP

Kind code of ref document: A1