WO2019165894A1 - 物品识别方法、装置、系统及存储介质 - Google Patents

物品识别方法、装置、系统及存储介质 Download PDF

Info

Publication number
WO2019165894A1
WO2019165894A1 PCT/CN2019/075092 CN2019075092W WO2019165894A1 WO 2019165894 A1 WO2019165894 A1 WO 2019165894A1 CN 2019075092 W CN2019075092 W CN 2019075092W WO 2019165894 A1 WO2019165894 A1 WO 2019165894A1
Authority
WO
WIPO (PCT)
Prior art keywords
item
container
items
recognition result
weight
Prior art date
Application number
PCT/CN2019/075092
Other languages
English (en)
French (fr)
Inventor
张爱喜
冯亦军
殷向阳
谭志羽
王永杰
张屹峰
陈建业
刘巍
陈宇
翁志
Original Assignee
北京京东尚科信息技术有限公司
北京京东世纪贸易有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京京东尚科信息技术有限公司, 北京京东世纪贸易有限公司 filed Critical 北京京东尚科信息技术有限公司
Publication of WO2019165894A1 publication Critical patent/WO2019165894A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0036—Checkout procedures
    • G07G1/0045—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
    • G07G1/0054—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader with control of supplementary check-parameters, e.g. weight or number of articles
    • G07G1/0063—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader with control of supplementary check-parameters, e.g. weight or number of articles with means for detecting the geometric dimensions of the article of which the code is read, such as its size or height, for the verification of the registration
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07F—COIN-FREED OR LIKE APPARATUS
    • G07F9/00—Details other than those peculiar to special kinds or types of apparatus
    • G07F9/002—Vending machines being part of a centrally controlled network of vending machines
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01G—WEIGHING
    • G01G19/00—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups
    • G01G19/40—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups with provisions for indicating, recording, or computing price or other quantities dependent on the weight
    • G01G19/413—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups with provisions for indicating, recording, or computing price or other quantities dependent on the weight using electromechanical or electronic computing means
    • G01G19/414—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups with provisions for indicating, recording, or computing price or other quantities dependent on the weight using electromechanical or electronic computing means using electronic computing means only
    • G01G19/4144—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups with provisions for indicating, recording, or computing price or other quantities dependent on the weight using electromechanical or electronic computing means using electronic computing means only for controlling weight of goods in commercial establishments, e.g. supermarket, P.O.S. systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/08—Payment architectures
    • G06Q20/20—Point-of-sale [POS] network systems
    • G06Q20/203—Inventory monitoring
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/08—Payment architectures
    • G06Q20/20—Point-of-sale [POS] network systems
    • G06Q20/208—Input by product or record sensing, e.g. weighing or scanner processing

Definitions

  • the present disclosure relates to the field of data processing, and in particular, to an item identification method, apparatus, system, and storage medium.
  • the merchandise identification scheme of the vending machine is mainly an identification scheme based on RFID (Radio Frequency Identification) technology.
  • RFID Radio Frequency Identification
  • each item needs to be equipped with an RFID tag.
  • the merchandise is automatically perceived by an area equipped with an RFID reader to complete the merchandise identification and self-service payment.
  • Another solution is to determine a merchandise category based on image recognition. By deploying a camera on the container to photograph the container, and performing inter-frame contrast detection at a certain frequency, it can be determined whether the commodity area has changed to identify the product purchased by the consumer.
  • an item identification method comprising: identifying an acquired item image to obtain one or more recognition results, wherein each recognition result includes a user picking up and placing in a container One or more items; determining an item in a recognition result as an item picked up by the user according to the weight change value of the item in the container; and updating the set of items to be settled according to the item picked up by the user.
  • the image capture area is the area through which the user is picked up and dropped while the user is picking up the item.
  • determining, according to the weight change value of the item in the container, the item in the identification result as the item picked up by the user includes: calculating a weight change value corresponding to each recognition result according to the weight of each item; determining The recognition result corresponding to the weight change value matching the weight change value of the item in the container; determining the item in the matched recognition result as the item picked up by the user.
  • calculating the weight change value corresponding to each recognition result according to the weight of each item comprises: converting each recognition result according to the set of items in the container, the set of items to be settled, and the quantity of each item a result of one or more sub-identifications, wherein the items in each sub-identification result are the same as the items in the recognition result, and each item has a mark of the item in the container or a mark of the item to be settled; the container item in the sub-identification result The difference between the sum of the weights and the weight of the items to be settled is determined as the weight change value corresponding to the sub-identification result.
  • the captured image of the item is identified to obtain one or more recognition results and a confidence level for each of the recognition results; the number of recognition results corresponding to the weight change value that matches the weight change value of the item in the container In the case of the case, the item in the matching recognition result with the highest degree of confidence is determined as the item picked up by the user.
  • the item identification method further includes deleting the recognition result if the number of one of the items in the recognition result exceeds the number of the same item in the set of items in the container and the set of items to be settled.
  • the item identification method further includes: monitoring a load bearing weight of the container; and acquiring an image of the collected item in response to a change in a load weight of the container.
  • a video taken by the user when picking up the item on the container is acquired for a period of time, and an image of the item is obtained from the video, wherein the time period is from the carrying weight of the container.
  • an item identification apparatus comprising: an image recognition module configured to identify an acquired item image to obtain one or more recognition results, wherein each recognition result The item includes one or more items that the user picks up at the container; the user pick and place item determining module is configured to determine an item in a recognition result as the item picked up by the user according to the weight change value of the item in the container.
  • the image capture area is the area through which the user is picked up and dropped while the user is picking up the item.
  • the user pick and place item determining module may be further configured to calculate a weight corresponding to each of the recognition results according to a weight change value of each item; and determine a weight change value that matches a weight change value of the item in the container. Corresponding recognition result, the item in the matching recognition result is determined as the item picked up by the user.
  • the user pick and place item determination module can be further configured to convert each recognition result into one or more sub-recognition results based on the set of items in the container, the set of items to be settled, and the number of each item.
  • the items in each sub-identification result are the same as the items in the recognition result, and each item has a mark of the item in the container or a mark of the item to be settled; the sum of the weights of the container items in the sub-identification result and the item to be settled The difference result of the sum of the weights is determined as the weight change value corresponding to the sub-recognition result.
  • the image recognition module can be further configured to identify the captured item image to obtain one or more recognition results and a confidence level for each of the recognition results; the user pick and place item determination module can be further configured to When there are a plurality of recognition results corresponding to the weight change value matching the weight change value of the article in the container, the item in the matching recognition result having the highest degree of confidence is determined as the item picked up by the user.
  • the item identification device may further include: a recognition result screening module configured to: the number of one item in the recognition result exceeds the number of the same item in the set of items in the container and the set of items to be settled In the case of the deletion, the recognition result is deleted.
  • a recognition result screening module configured to: the number of one item in the recognition result exceeds the number of the same item in the set of items in the container and the set of items to be settled In the case of the deletion, the recognition result is deleted.
  • the item identification device can further include: a weight monitoring module configured to monitor a load weight of the container; and an image acquisition module configured to acquire an image of the collected item in response to a change in a load weight of the container.
  • the image acquisition module may be further configured to: in response to the change in the load weight of the container, acquire a video captured by the user when the user picks up the item on the container for a period of time, and obtain an image of the item from the video, wherein The period of time is the time from the preset time before the change in the carrying weight of the container to the change in the carrying weight of the container.
  • an item identification apparatus comprising: a memory; and a processor coupled to the memory, the processor being configured to execute the operation for operating based on an instruction stored in the memory Any item identification method.
  • an item identification system comprising any one of the foregoing item identification devices; an image pickup device configured to collect an item image; and a weighing device configured to acquire an item in the container The weight change value.
  • the item identification system further includes: a vending machine.
  • a computer readable storage medium having stored thereon a computer program that, when executed by a processor, implements any of the foregoing item identification methods.
  • FIG. 1 is an exemplary flow chart of an item identification method in accordance with some embodiments of the present disclosure.
  • FIG. 2 is an exemplary flow chart of a recognition result selection method in accordance with some embodiments of the present disclosure.
  • FIG. 3 is an exemplary flow diagram of a method of calculating a weight change value corresponding to a recognition result, in accordance with some embodiments of the present disclosure.
  • FIG. 4 is an exemplary flow chart of a method of selecting a recognition result in accordance with further embodiments of the present disclosure.
  • FIG. 5 is an exemplary flow chart of a recognition result screening method, in accordance with some embodiments of the present disclosure.
  • FIG. 6 is an exemplary flowchart of an item identification method in accordance with further embodiments of the present disclosure.
  • FIG. 7 is an exemplary flowchart of an item identification method in accordance with further embodiments of the present disclosure.
  • FIG. 8 is an exemplary structural diagram of an item identification device in accordance with some embodiments of the present disclosure.
  • FIG 9 is an exemplary structural diagram of an item identification system in accordance with some embodiments of the present disclosure.
  • FIG. 10 is an exemplary structural diagram of an article identification device according to further embodiments of the present disclosure.
  • FIG. 11 is an exemplary structural diagram of an article identification device in accordance with still further embodiments of the present disclosure.
  • the RFID tag encounters liquid and metal, which is easy to attenuate and shield, is sticky and easy to be torn, and the size and the sensing distance are difficult to coordinate, thereby reducing the recognition rate.
  • a disadvantage of the technical solution based on image recognition is that it is difficult to handle the case of taking or returning a plurality of articles, especially a plurality of different articles at the same time.
  • Image recognition is also difficult to cover in extreme cases of simultaneous pick and place. Therefore, the recognition rate is also low.
  • one technical problem to be solved by the embodiments of the present disclosure is how to improve the accuracy of item identification in an automatic sales scene.
  • FIG. 1 is an exemplary flow chart of an item identification method in accordance with some embodiments of the present disclosure. As shown in FIG. 1, the item identification method of this embodiment includes steps S102 to S104.
  • step S102 the collected item images are identified to obtain one or more recognition results, wherein each recognition result includes one or more items that the user picks up at the container.
  • an image of an item captured by a camera device such as a camera may be acquired.
  • the camera device can be mounted on a vending machine used in some embodiments of the present disclosure.
  • the camera device photographs the inside of the container.
  • the image capture area is the area through which the user is picked up and dropped while the user is picking up the item.
  • the camera can be mounted above the cabinet door, and the image capture area can be the area in front of the container door, so that the user can capture the picture when the user picks up the item or move the item with other equipment such as a tray or a jig.
  • a weighing area can be, for example, a cargo lane, or a row of containers in the container, an entire container, etc., which can be set by a person skilled in the art as needed.
  • a pre-trained neural network model can be employed for image recognition.
  • the neural network model can be trained using item image data labeled with the item category.
  • the camera on the container can also be used to take a picture of the user's pick and place items as the item image data.
  • Image recognition mainly recognizes which items the user has operated on, and therefore needs to further combine the results of gravity detection to determine which operations the user has performed on which items.
  • the items in each recognition result are unsettled items, and the unsettled items include container items, items to be settled.
  • the container item is the item located in the container before the load weight of the container changes; the item to be settled is the item to be settled before the user picks up the item, that is, the item that the user has taken away from the container but has not settled.
  • the item to be settled can be regarded as an item that the user has placed in the shopping cart, and the "shopping cart" can be a physical device or a virtual one.
  • the identification result may include the source of the item in addition to the name, the identifier, the quantity, and the like of the item, and is used to indicate whether the item is a container item or an item to be settled before being picked up.
  • step S104 the item in one recognition result is determined as the item picked up by the user based on the weight change value of the item in the container. That is, the recognition result that matches the weight change value of the item in the container is selected from one or more recognition results, and the item in the recognition result is taken as the item picked up by the user. For example, the corresponding weight in the recognition result can be selected to be closest to the weight change value of the item in the container.
  • the weight change value of the item in the container may be the weight change value of the item in a weighing area in the container. As described above, the weighing area may be a part of the item stored in the container or the entire container.
  • the set of items to be settled may be updated based on the item being picked up by the user. For example, the shelf item in the item picked up by the user is added to the set of items to be settled, and the item to be settled in the item combination picked up by the user is deleted from the set of items to be settled.
  • the user can perform multiple pick and place in one purchase.
  • the background may use the database corresponding to the purchase behavior of the user to record the items that the user has taken but not settled, and update the database according to the recognition result after each pick and place, so that the database stores the current settlement of the user. article.
  • the item to be settled may be emptied.
  • Some embodiments of the present disclosure may be applied, for example, to an application scenario in which a user picks up and unloads goods and settles after opening the door of the vending machine.
  • the goods can be picked up and released by different users after the door is opened, but the same user account is settled. For example, after user A authenticates, the cabinet door is opened, and after the cabinet door is opened, both user A and friend B can pick up and release the goods, and use A's account for settlement at the time of settlement.
  • gravity sensing and image recognition can be combined to determine the items that the user picks up, thereby improving the accuracy of item identification.
  • the user can pick up, release, pick up and release the goods at the same time, thereby increasing the flexibility of the user and improving the user experience.
  • the recognition result selection method of this embodiment includes steps S202 to S206.
  • step S202 the weight change value corresponding to each recognition result is calculated based on the weight of each item.
  • step S204 a recognition result corresponding to the weight change value matching the weight change value of the article in the container is determined.
  • step S206 the item in the matching recognition result is determined as the item picked up by the user.
  • the weight change value corresponding to the recognition result is the sum of the weights of all the items in the recognition result, and then the scene is distinguished by the positive and negative signs. And put the scene.
  • the user may pick up and drop items at the same time, more complicated calculations are required.
  • FIG. 3 is an exemplary flow diagram of a method of calculating a weight change value corresponding to a recognition result, in accordance with some embodiments of the present disclosure. As shown in FIG. 3, the method for calculating the weight corresponding to the recognition result of the embodiment includes steps S302 to S304.
  • each recognition result is converted into one or more sub-recognition results according to the set of items in the container, the set of items to be settled, and the quantity of each item, wherein the items and the recognition result in each sub-recognition result
  • the items in the same are the same, and each item has a mark of the item in the container or a mark of the item to be settled.
  • the number of items with the same item in the container does not exceed the number of the same item in the container item collection, and the number of items with the same unsettled item tag does not exceed the number of the same item in the unsettled item collection.
  • step S304 the difference between the sum of the weights of the container items in the sub-identification result and the weight of the items to be settled is determined as the weight change value corresponding to the sub-recognition result.
  • one of the recognition results is ⁇ A, A, B ⁇ , that is, the identification result includes two items A and one item B.
  • the item A from the container in the recognition result is A1
  • the item B is B1
  • the weights are a1 and b1 respectively
  • the recognition result the item A from the shopping cart is A2, the item B is B2, and the weight is a2 respectively.
  • Table 1 is a sub-recognition result obtained by converting the above-described recognition result, and a weight calculation method corresponding to each sub-recognition result.
  • the above method takes into consideration the case where the user takes an item, places an item, and simultaneously picks up an item.
  • the recognition result selection method of this embodiment includes steps S402 to S404.
  • step S402 the collected item images are identified to obtain one or more recognition results and a confidence level for each recognition result.
  • the articles among the matching recognition results having the highest degree of confidence are determined as the articles picked up by the user.
  • the selection can be made based on the confidence of the image recognition result.
  • the goal of image recognition is to identify which items are in the image based on the image features.
  • the identification process may not be aware of the current status of the current container and the item in the shopping cart. Therefore, there are odds in the image recognition results including combinations that are unlikely to occur.
  • Some embodiments of the present disclosure may screen the recognition results. An embodiment of the screening result screening method of the present disclosure will be described below with reference to FIG.
  • FIG. 5 is an exemplary flow chart of a recognition result screening method, in accordance with some embodiments of the present disclosure. As shown in FIG. 5, the recognition result screening method of this embodiment includes steps S502 to S504.
  • step S502 the collected item image is identified to obtain one or more recognition results.
  • step S504 if the number of one item in the recognition result exceeds the number of the same item in the set of items in the container and the set of items to be settled, the recognition result is deleted.
  • the total number of items A in the container and the items to be settled is 2, but the number of items A in a certain recognition result is 3, and the recognition result can be screened out at this time. Thereby the accuracy of item identification is further improved.
  • Embodiments of the present disclosure combine image recognition and gravity detection so that the recognition process can be triggered in response to changes in gravity.
  • An embodiment of the present article identification method will be described below with reference to FIG.
  • FIG. 6 is an exemplary flowchart of an item identification method in accordance with further embodiments of the present disclosure. As shown in FIG. 6, the item identification method of this embodiment includes steps S602 to S610.
  • step S602 the carrying weight of the container is monitored.
  • step S604 an image of the collected item is acquired in response to a change in the carrying weight of the container.
  • an image of the item may be acquired in response to a stable change in the weight of the container to prevent the container from causing a slight sway due to the action of external forces, causing a change in gravity.
  • a video taken by a user during picking up and dropping of items in the container may be acquired for a period of time, and an image of the item is obtained from the video, wherein the time period is a preset time before the change in the carrying weight of the container At the moment when the load weight of the container changes again.
  • the container produces the first weight change at time t1 and the second weight change at time t2, indicating that the user's action of holding the item through the camera coverage area occurs between time t1 and time t2, so for example, [ Video of time period t1- ⁇ , t2].
  • ⁇ is a positive number or 0, which can be, for example, a small value.
  • step S606 the collected item images are identified to obtain one or more recognition results, wherein each recognition result includes one or more items that the user picks up at the container.
  • step S608 the item in one recognition result is determined as the item picked up by the user based on the weight change value of the item in the container.
  • the confidence level of the recognition result of each item image in the plurality of item images may be positively correlated with the shooting time at which the item image is located. That is, the recognition result corresponding to the updated photographed item image has a greater degree of confidence.
  • step S610 the set of items to be settled is updated according to the items picked up by the user.
  • Embodiments of the present disclosure may partition the items on the shelf to allow for an optional range of items to be identified.
  • An embodiment of the disclosed article identification method will be described below with reference to FIG.
  • FIG. 7 is an exemplary flowchart of an item identification method in accordance with further embodiments of the present disclosure. As shown in FIG. 7, the item identification method of this embodiment includes steps S702 to S710.
  • step S702 in response to detecting that the load weight of the unit weighing area in the container changes, the weight change value of the unit weighing area is obtained.
  • a unit weighing area is the area in which a weighing device is responsible, such as a cargo lane in a container, or a pallet.
  • step S704 the collected item images are identified to obtain one or more recognition results, wherein each recognition result includes one or more items that the user picks up at the container.
  • step S706 the recognition result is screened based on the set of container items in the unit weighing area in which the weight change occurs and the set of items to be settled.
  • step S708 the item in a recognition result is determined as the item picked up by the user according to the weight change value of the item in the container, wherein the identified item picked up by the user includes the unit weighing area in which the weight change occurs. At least one of the items in the container and the items to be settled.
  • step S710 the set of items to be settled is updated according to the items picked up by the user.
  • a plurality of weighing devices can be disposed in the shelf, and the combination of the articles picked up by the user is determined according to the result of the item and image recognition in the unit weighing area where the weight change occurs, thereby further improving the item identification. Accuracy.
  • FIG. 8 is an exemplary structural diagram of an item identification device in accordance with some embodiments of the present disclosure.
  • the item identification device 80 of this embodiment includes: an image recognition module 810 configured to identify the collected item image to obtain one or more recognition results, wherein each recognition result includes the user One or more items picked up by the container; the user pick and place item determining module 820 is configured to determine an item in a recognition result as the item picked up by the user according to the weight change value of the item in the container.
  • the image capture area is the area through which the user is picked up and dropped while the user is picking up the item.
  • the user pick and place item determination module 820 can be further configured to calculate a weight corresponding to each recognition result based on the weight change value of each item; determine a weight change value that matches the weight change value of the item in the container. The corresponding recognition result determines the item in the matched recognition result as the item picked up by the user.
  • the user pick and place item determination module 820 can be further configured to convert each recognition result into one or more sub-recognition results based on the set of items within the container, the set of items to be settled, and the number of each item. , wherein the items in each of the sub-identification results are the same as the items in the recognition result, and each item has a mark of the item in the container or a mark of the item to be settled; the sum of the weights of the container items in the sub-identification result and the item to be settled The difference result of the sum of the weights is determined as the weight change value corresponding to the sub-identification result.
  • image recognition module 810 can be further configured to identify the captured item image to obtain one or more recognition results and a confidence in each of the recognition results; user pick and place item determination module 820 can be further configured In the case where there are a plurality of recognition results corresponding to the weight change values matching the weight change values of the articles in the container, the items in the matching recognition results having the highest degree of confidence are determined as the articles picked up by the user.
  • the item identification device 80 may further include: a recognition result screening module 830 configured to have the number of one item in the recognition result exceed the same item in the collection of items in the container and the set of items to be settled In the case of the number of deletions, the recognition result is deleted.
  • a recognition result screening module 830 configured to have the number of one item in the recognition result exceed the same item in the collection of items in the container and the set of items to be settled In the case of the number of deletions, the recognition result is deleted.
  • the item identification device 80 may further include: a weight monitoring module 840 configured to monitor a load weight of the container; and an image acquisition module 850 configured to acquire an image of the collected item in response to a change in the load weight of the container .
  • the image capture module 850 can be further configured to, in response to a change in the load weight of the container, acquire a video taken by the user during picking up and placing the item on the container for a period of time, and obtain an image of the item from the video, wherein For a period of time, the time from the preset time before the change of the carrying weight of the container to the load of the container changes again.
  • the item identification system 90 of this embodiment includes: an item identification device 91.
  • the specific embodiment may refer to the item identification device 80 in the embodiment of FIG. 8;
  • the imaging device 92 is configured to collect an image of the item;
  • the weight device 93 is configured to obtain a weight change value for the items in the container.
  • the item identification system can also include a vending machine 94.
  • One or more weighing devices 93, camera 92 may be installed in the vending machine 94.
  • FIG. 10 is an exemplary structural diagram of an article identification device according to further embodiments of the present disclosure.
  • the item identification apparatus 1000 of this embodiment includes a memory 1010 and a processor 1020 coupled to the memory 1010, the processor 1020 being configured to perform any of the foregoing implementations based on instructions stored in the memory 1010.
  • the item identification method in the example is an exemplary structural diagram of an article identification device according to further embodiments of the present disclosure.
  • the item identification apparatus 1000 of this embodiment includes a memory 1010 and a processor 1020 coupled to the memory 1010, the processor 1020 being configured to perform any of the foregoing implementations based on instructions stored in the memory 1010.
  • the item identification method in the example is an exemplary structural diagram of an article identification device according to further embodiments of the present disclosure.
  • the item identification apparatus 1000 of this embodiment includes a memory 1010 and a processor 1020 coupled to the memory 1010, the processor 1020 being configured to perform any of the foregoing implementations based on instructions stored in the memory 1010.
  • the memory 1010 may include, for example, a system memory, a fixed non-volatile storage medium, or the like.
  • the system memory stores, for example, an operating system, an application, a boot loader, and other programs.
  • the item identification apparatus 1100 of this embodiment includes a memory 1110 and a processor 1120, and may further include an input/output interface 1130, a network interface 1140, a storage interface 1150, and the like. These interfaces 1130, 1140, 1150 and the memory 1110 and the processor 1120 can be connected, for example, via a bus 1160.
  • the input/output interface 1130 provides a connection interface for input and output devices such as a display, a mouse, a keyboard, and a touch screen.
  • Network interface 1140 provides a connection interface for various networked devices.
  • the storage interface 1150 provides a connection interface for an external storage device such as an SD card or a USB flash drive.
  • An embodiment of the present disclosure further provides a computer readable storage medium having stored thereon a computer program, wherein the program is executed by a processor to implement any one of the foregoing item identification methods.
  • embodiments of the present disclosure can be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware aspects. Moreover, the present disclosure may take the form of a computer program product embodied on one or more computer-usable non-transitory storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code. .
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Business, Economics & Management (AREA)
  • Accounting & Taxation (AREA)
  • Finance (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Geometry (AREA)
  • Control Of Vending Devices And Auxiliary Devices For Vending Devices (AREA)

Abstract

一种物品识别方法、装置、系统及存储介质，涉及数据处理领域。物品识别方法包括：对采集的物品图像进行识别，获得一个或多个识别结果（S102，S704），其中，每个识别结果中包括用户在货柜取放的一个或多个物品；根据货柜内物品的重量变化值，将一个识别结果中的物品确定为用户取放的物品（S104，S708）。

Description

物品识别方法、装置、系统及存储介质
相关申请的交叉引用
本申请是以CN申请号为201810175167.X，申请日为2018年3月2日的申请为基础，并主张其优先权，该CN申请的公开内容在此作为整体引入本申请中。
技术领域
本公开涉及数据处理领域，特别涉及一种物品识别方法、装置、系统及存储介质。
背景技术
在相关技术中，自动售货机的商品识别方案主要为基于RFID(Radio Frequency Identification，无线射频)技术的识别方案。在这种方案中，需要为每个商品配备一个RFID标签。当消费者取走商品时，商品会经过一个配备了RFID阅读器的区域以自动地被感知，从而完成商品识别和自助付款。
另一种解决方案为基于图像识别确定商品品类的方案。通过在货柜上部署摄像头拍摄货柜的情况，并按一定频率进行帧间对比检测，可以判断商品区域是否发生了变化，以识别消费者购买的商品。
发明内容
根据本公开一些实施例的第一个方面，提供一种物品识别方法，包括：对采集的物品图像进行识别，获得一个或多个识别结果，其中，每个识别结果中包括用户在货柜取放的一个或多个物品；根据货柜内物品的重量变化值，将一个识别结果中的物品确定为用户取放的物品；根据用户取放的物品对待结算物品的集合进行更新。
在一些实施例中，图像采集区域为用户在取放物品时，被取放物品所经过的区域。
在一些实施例中，根据货柜内物品的重量变化值，将一个识别结果中的物品确定为用户取放的物品包括：根据每个物品的重量，计算每个识别结果对应的重量变化值；确定与货柜内物品的重量变化值匹配的重量变化值所对应的识别结果；将匹配的识别结果中的物品确定为用户取放的物品。
在一些实施例中，根据每个物品的重量，计算每个识别结果对应的重量变化值包括：根据货柜内物品的集合、待结算物品的集合以及每种物品的数量，将每个识别结 果转换为一个或多个子识别结果，其中，每个子识别结果中的物品与识别结果中的物品相同、并且每个物品具有货柜内物品的标记或者待结算物品的标记；将子识别结果中货柜物品的重量之和与待结算物品的重量之和的差值确定为子识别结果对应的重量变化值。
在一些实施例中，对采集的物品图像进行识别，获得一个或多个识别结果和每个识别结果的置信度；在与货柜内物品的重量变化值匹配的重量变化值对应的识别结果有多个的情况下，将置信度最高的匹配的识别结果中的物品确定为用户取放的物品。
在一些实施例中，物品识别方法还包括：在识别结果中的一种物品的数量超过在货柜内物品的集合和待结算物品的集合中同一种物品的数量的情况下，删除识别结果。
在一些实施例中，物品识别方法还包括：监测货柜的承载重量；响应于货柜的承载重量发生变化，获取采集的物品图像。
在一些实施例中，响应于货柜的承载重量发生变化，获取一段时间内的用户在货柜上取放物品时拍摄的视频，并从视频中获取物品图像，其中，一段时间是从货柜的承载重量发生变化之前的预设时刻、到货柜的承载重量再次发生变化的时刻。
根据本发明一些实施例的第二个方面，提供一种物品识别装置，包括：图像识别模块，被配置为对采集的物品图像进行识别，获得一个或多个识别结果，其中，每个识别结果中包括用户在货柜取放的一个或多个物品；用户取放物品确定模块，被配置为根据货柜内物品的重量变化值，将一个识别结果中的物品确定为用户取放的物品。
在一些实施例中，图像采集区域为用户在取放物品时，被取放物品所经过的区域。
在一些实施例中，用户取放物品确定模块可以进一步被配置为根据每个物品的重量变化值，计算每个识别结果对应的重量；确定与货柜内物品的重量变化值匹配的重量变化值所对应的识别结果，将匹配的识别结果中的物品确定为用户取放的物品。
在一些实施例中，用户取放物品确定模块可以进一步被配置为根据货柜内物品的集合、待结算物品的集合以及每种物品的数量，将每个识别结果转换为一个或多个子识别结果，其中，每个子识别结果中的物品与识别结果中的物品相同、并且每个物品具有货柜内物品的标记或者待结算物品的标记；将子识别结果中货柜物品的重量之和与待结算物品的重量之和的差值结果确定为子识别结果对应的重量变化值。
在一些实施例中，图像识别模块可以进一步被配置为对采集的物品图像进行识别，获得一个或多个识别结果和每个识别结果的置信度；用户取放物品确定模块可以进一步被配置为在与货柜内物品的重量变化值匹配的重量变化值所对应的识别结果有多 个的情况下，将置信度最高的匹配的识别结果中的物品确定为用户取放的物品。
在一些实施例中，物品识别装置还可以包括：识别结果筛选模块，被配置为在识别结果中的一种物品的数量超过在货柜内物品的集合和待结算物品的集合中同一种物品的数量的情况下，删除识别结果。
在一些实施例中，物品识别装置还可以包括：重量监测模块，被配置为监测货柜的承载重量；图像采集模块，被配置为响应于货柜的承载重量发生变化，获取采集的物品图像。
在一些实施例中，图像采集模块可以进一步被配置为响应于货柜的承载重量发生变化，获取一段时间内的用户在货柜上取放物品时拍摄的视频，并从视频中获取物品图像，其中，一段时间是从货柜的承载重量发生变化之前的预设时刻、到货柜的承载重量再次发生变化的时刻。
根据本发明一些实施例的第三个方面，提供一种物品识别装置，包括：存储器；以及耦接至存储器的处理器，处理器被配置为基于存储在存储器中的指令，执行用于运行前述任意一种物品识别方法。
根据本发明一些实施例的第四个方面，提供一种物品识别系统，包括前述任意一种物品识别装置；摄像装置，被配置为采集物品图像；和称重设备，被配置为获取货柜内物品的重量变化值。
在一些实施例中，物品识别系统还包括：自动售货机。
根据本发明一些实施例的第五个方面，提供一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现前述任意一种物品识别方法。
通过以下参照附图对本公开的示例性实施例的详细描述，本公开的其它特征及其优点将会变得清楚。
附图说明
为了更清楚地说明本公开实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为根据本公开一些实施例的物品识别方法的示例性流程图。
图2为根据本公开一些实施例的识别结果选择方法的示例性流程图。
图3为根据本公开一些实施例的计算识别结果对应的重量变化值的方法的示例性流程图。
图4为根据本公开另一些实施例的识别结果选择方法的示例性流程图。
图5为根据本公开一些实施例的识别结果筛选方法的示例性流程图。
图6为根据本公开另一些实施例的物品识别方法的示例性流程图。
图7为根据本公开另一些实施例的物品识别方法的示例性流程图。
图8为根据本公开一些实施例的物品识别装置的示例性结构图。
图9为根据本公开一些实施例的物品识别系统的示例性结构图。
图10为根据本公开另一些实施例的物品识别装置的示例性结构图。
图11为根据本公开又一些实施例的物品识别装置的示例性结构图。
具体实施方式
下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本公开及其应用或使用的任何限制。基于本公开中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本公开的范围。
同时，应当明白，为了便于描述，附图中所示出的各个部分的尺寸并不是按照实际的比例关系绘制的。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为授权说明书的一部分。
在这里示出和讨论的所有示例中，任何具体值应被解释为仅仅是示例性的，而不是作为限制。因此，示例性实施例的其它示例可以具有不同的值。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。
发明人经过分析后发现，基于RFID的技术方案的缺点在于，对于吞吐量巨大的自动售货机场景，RFID标签成本昂贵，不利于场景的复制和推广。同时，RFID标签遇到液体、金属易衰减屏蔽，黏贴麻烦、易被撕毁，并且尺寸和感应距离难协调，从 而降低了识别率。
基于图像识别的技术方案的缺点在于，对于同时拿取或者放回多件商品、尤其是多件不同品类的商品的情况难以处理。对于同时取放商品的极端情况，图像识别也难以覆盖。因此识别率也较低。
因此，本公开实施例所要解决的一个技术问题是：如何提高自动售卖场景中物品识别的准确率。
图1为根据本公开一些实施例的物品识别方法的示例性流程图。如图1所示，该实施例的物品识别方法包括步骤S102～S104。
在步骤S102中，对采集的物品图像进行识别，获得一个或多个识别结果，其中，每个识别结果中包括用户在货柜取放的一个或多个物品。
在一些实施例中，可以获取摄像机等摄像装置采集的物品图像。摄像装置可以安装在本公开的一些实施例所使用的自动售货机上。
在相关技术中，摄像装置拍摄货柜内部。然而，由于货柜内的物品较多，如果拍摄货柜内部，很难识别哪些是用户取放的物品，哪些是用户未取放的物品。因此，在本公开的一些实施例中，图像采集区域为用户在取放物品时，被取放物品所经过的区域。例如，摄像头可以安装在柜门上方，图像采集区域可以是货柜门前的区域，从而可以捕捉到用户取放物品时手持物品或者借助托盘、夹具等其他设备移动物品的画面。
货柜上有一个或多个用于承载物品的货道。货柜中设置有一个或多个称重设备，用于测量相应的称重区域的物品重量。一个称重区域例如可以是一个货道，也可以是货柜中的一行货柜、整个货柜等等，本领域技术人员可以根据需要设置。
在一些实施例中，可以采用预先训练的神经网络模型进行图像识别。例如，可以采用标注了物品类别的物品图像数据对神经网络模型进行训练。为了提高识别的精确度，可以同样采用货柜上的摄像头拍摄用户取放物品的画面以作为物品图像数据。
当用户从货柜中取走物品时，不一定意味着用户会购买该商品。用户可能对物品进行仔细查看再放回货柜。因此，用户可能进行取货、将货放回到货柜、同时取放货中的任意一种操作。而图像识别主要是识别用户对哪些物品进行了操作，因此还需要进一步地结合重力检测的结果，判断用户对哪些物品进行了何种操作。
在一些实施例中，每个识别结果中的物品均为未结算物品，未结算物品包括货柜物品、待结算物品。货柜物品为货柜的承载重量发生变化之前位于货柜的物品；待结算物品为用户取放物品之前的待结算的物品，即用户已从货柜上拿走、但是没有进行 结算的物品。待结算物品可以视为是用户已放入购物车的物品，该“购物车”可以是一个实体装置，也可以是虚拟的。识别结果中除了包括物品的名称、标识、数量等信息以外，还可以包括物品的来源，用于标明该物品在被取放之前是货柜物品还是待结算物品。
在步骤S104中，根据货柜内物品的重量变化值，将一个识别结果中的物品确定为用户取放的物品。即，从一个或多个识别结果中选择与货柜内物品的重量变化值匹配的识别结果，将识别结果中的物品作为用户取放的物品。例如，可以选择识别结果中对应的重量与货柜内物品的重量变化值最接近的。货柜内物品的重量变化值可以是货柜内某个称重区域内的物品的重量变化值，如前所述，称重区域可以是货柜内存放物品的部分区域，也可以是整个货柜。
在完成物品识别以后，在一些实施例中，可以根据用户取放的物品对待结算物品的集合进行更新。例如，将用户取放的物品中的货架物品添加到待结算物品的集合中，将用户取放的物品组合中的待结算物品从待结算物品的集合中删除。
用户在一次购买行为中可以进行多次取放。后台可以采用该用户的本次购买行为对应的数据库来记录用户已取走但是未结算的物品，并根据每次取放后的识别结果更新数据库，从而使得数据库中存储的是用户当前待结算的物品。
响应于用户对待结算物品进行支付，可以清空待结算物品。本公开的一些实施例例如可以应用于一个用户在打开自动售货机的柜门后进行取放货物并结算的应用场景。此外，在一些应用场景中，也可以在打开柜门后由不同的用户进行货物取放，但是针对同一个用户账号进行结算。例如，用户A进行身份验证后打开柜门，柜门打开后，用户A与其朋友B均可以取放货物，结算时使用A的账户进行结账。
通过上述实施例的方法，可以将重力感应和图像识别进行结合以判定用户取放的物品，从而提高了物品识别的准确率。并且，用户可以进行取货、放货、同时取货和放货，从而增加了用户使用的灵活性，提升了用户体验。
下面参考图2描述本公开计算每个识别结果对应的重量的方法的实施例。
图2为根据本公开一些实施例的识别结果选择方法的示例性流程图。如图2所示，该实施例的识别结果选择方法包括步骤S202～S206。
在步骤S202中，根据每个物品的重量，计算每个识别结果对应的重量变化值。
在步骤S204中，确定与货柜内物品的重量变化值匹配的重量变化值对应的识别结果。
在步骤S206中，将匹配的识别结果中的物品确定为用户取放的物品。
对于用户仅从货柜上取物品的场景、或者仅将物品放回货柜的场景，识别结果对应的重量变化值为识别结果中所有物品的重量之和，然后采用正号和负号区分取的场景和放的场景。然而考虑到用户可能同时取放物品，因此需要更复杂的计算。下面参考图3描述本公开一些实施例的计算识别结果对应的重量变化值的方法的实施例。
图3为根据本公开一些实施例的计算识别结果对应的重量变化值的方法的示例性流程图。如图3所示，该实施例的计算识别结果对应的重量的方法包括步骤S302～S304。
在步骤S302中，根据货柜内物品的集合、待结算物品的集合以及每种物品的数量，将每个识别结果转换为一个或多个子识别结果，其中，每个子识别结果中的物品与识别结果中的物品相同、并且每个物品具有货柜内物品的标记或者待结算物品的标记。每个子识别结果中，同一种具有货柜内物品标记的物品数量不超过货柜物品集合中同一种物品的数量，同一种具有未结算物品标记的物品数量不超过未结算物品集合中同一种物品的数量。
在步骤S304中，将子识别结果中货柜物品的重量之和与待结算物品的重量之和的差值确定为子识别结果对应的重量变化值。
例如，识别结果之一为{A,A,B}，即识别结果中包含两个物品A和一个物品B。在用户进行取放之前，货柜上有3个物品A、一个物品B，购物车中有一个物品A、一个物品B。此时，可以设识别结果中来自货柜的物品A为A1、物品B为B1，重量分别为a1、b1；设识别结果中来自购物车的物品A为A2、物品B为B2，重量分别为a2、b2。表1为将上述识别结果转换后获得的子识别结果，以及每个子识别结果对应的重量计算方法。
表1
子识别结果 重量计算方法
A1,A1,B1 a1+a1+b1
A1,A2,B1 a1-a2+b1
A1,A1,B2 a1+a1-b2
A1,A2,B2 a1-a2-b2
从而，上述方法考虑了用户取物品、放物品、同时取放物品的情况。
在部分情况下，可能有多个识别结果对应的重量变化值都与货柜内物品的重量变化值接近。此时可以参考识别结果的置信度进行判定。下面参考图4描述本公开识别 结果选择方法的实施例。
图4为根据本公开另一些实施例的识别结果选择方法的示例性流程图。如图4所示，该实施例的识别结果选择方法包括步骤S402～S404。
在步骤S402中，对采集的物品图像进行识别，获得一个或多个识别结果和每个识别结果的置信度。
在步骤S404中，在与货柜内物品的重量变化值匹配的重量变化值对应的识别结果有多个的情况下，将置信度最高的匹配的识别结果中的物品确定为用户取放的物品。
从而，在重量识别结果相同的情况下，可以根据图像识别结果的置信度进行选择。
图像识别的目标是根据图像特征识别出图像中有哪些物品，识别的过程中可能并不知晓当前货柜以及购物车中的物品现状。因此，图像识别结果中有几率包括不可能出现的组合。本公开的一些实施例可以对识别结果进行筛选。下面参考图5描述本公开识别结果筛选方法的实施例。
图5为根据本公开一些实施例的识别结果筛选方法的示例性流程图。如图5所示，该实施例的识别结果筛选方法包括步骤S502～S504。
在步骤S502中，对采集的物品图像进行识别，获得一个或多个识别结果。
在步骤S504中，在识别结果中的一种物品的数量超过在货柜内物品的集合和待结算物品的集合中同一种物品的数量的情况下，删除识别结果。
例如，货柜内和待结算物品中物品A的总数量为2，但是某个识别结果中物品A的数量为3，此时可以将该识别结果筛除。从而进一步提高了物品识别的准确性。
本公开的实施例将图像识别和重力检测相结合，因此可以响应于重力的变化触发识别流程。下面参考图6描述本公开物品识别方法的实施例。
图6为根据本公开另一些实施例的物品识别方法的示例性流程图。如图6所示，该实施例的物品识别方法包括步骤S602～S610。
在步骤S602中，监测货柜的承载重量。
在步骤S604中，响应于货柜的承载重量发生变化，获取采集的物品图像。
在一些实施例中，可以响应于货柜的承载重量发生稳定的变化再获取物品图像，以避免货柜由于外力的作用产生轻微晃动，而造成重力变化。
在一些实施例中，可以获取一段时间内的用户在货柜内取放物品时拍摄的视频，并从视频中获取物品图像，其中，一段时间的是从货柜的承载重量发生变化之前的预设时刻、到货柜的承载重量再次发生变化的时刻。例如，货柜在t1时刻产生第一次重 量变化，在t2时刻产生第二次重量变化，则说明用户拿着物品经过摄像头覆盖区域的动作发生在t1时刻和t2时刻之间，因此例如可以获取[t1-Δ,t2]时间段的视频。Δ为正数或0，例如可以是很小的数值。
在步骤S606中，对采集的物品图像进行识别，获得一个或多个识别结果，其中，每个识别结果中包括用户在货柜取放的一个或多个物品。
在步骤S608中，根据货柜内物品的重量变化值，将一个识别结果中的物品确定为用户取放的物品。
在一些实施例中，多张物品图像中的每张物品图像的识别结果的置信度可以与物品图像所处的拍摄时间成正相关关系。即，更新拍摄的物品图像对应的识别结果具有更大的置信度。
在步骤S610中，根据用户取放的物品对待结算物品的集合进行更新。
从而，可以准确地捕捉到用户进行物品取放的时间段的图像，提高了物品识别的准确性和识别效率。
本公开的实施例可以对货架上的物品进行分区维护，从而可以缩小待识别物品的可选范围。下面参考图7描述本公开物品识别方法的实施例。
图7为根据本公开另一些实施例的物品识别方法的示例性流程图。如图7所示，该实施例的物品识别方法包括步骤S702～S710。
在步骤S702中，响应于监测到货柜中的单位称重区域的承载重量发生变化，获取单位称重区域的重量变化值。一个单位称重区域为一个称重设备所负责的区域，例如可以为货柜中的一个货道、或者一个托盘。
在步骤S704中，对采集的物品图像进行识别，获得一个或多个识别结果，其中，每个识别结果中包括用户在货柜取放的一个或多个物品。
在步骤S706中，根据发生重量变化的单位称重区域中的货柜物品的集合和待结算物品的集合，对识别结果进行筛选。
此时，物品组合中无需包括其他单位称重区域中的物品，只需要关注该单位称重区域中的物品以及待结算物品。
在步骤S708中，根据货柜内物品的重量变化值，将一个识别结果中的物品确定为用户取放的物品，其中，识别出的用户取放的物品包括发生重量变化的单位称重区域中的货柜内物品、待结算物品中的至少一种。
在步骤S710中，根据用户取放的物品对待结算物品的集合进行更新。
通过上述实施例的方法，可以在货架中设置多个称重设备，并且根据发生重量变化的单位称重区域中的物品和图像识别的结果确定用户取放的物品组合，进一步提高了物品识别的精确度。
下面参考图8描述本公开物品识别装置的实施例。
图8为根据本公开一些实施例的物品识别装置的示例性结构图。如图8所示，该实施例的物品识别装置80包括：图像识别模块810，被配置为对采集的物品图像进行识别，获得一个或多个识别结果，其中，每个识别结果中包括用户在货柜取放的一个或多个物品；用户取放物品确定模块820，被配置为根据货柜内物品的重量变化值，将一个识别结果中的物品确定为用户取放的物品。
在一些实施例中，图像采集区域为用户在取放物品时，被取放物品所经过的区域。
在一些实施例中，用户取放物品确定模块820可以进一步被配置为根据每个物品的重量变化值，计算每个识别结果对应的重量；确定与货柜内物品的重量变化值匹配的重量变化值所对应的识别结果，将匹配的识别结果中的物品确定为用户取放的物品。
在一些实施例中，用户取放物品确定模块820可以进一步被配置为根据货柜内物品的集合、待结算物品的集合以及每种物品的数量，将每个识别结果转换为一个或多个子识别结果，其中，每个子识别结果中的物品与识别结果中的物品相同、并且每个物品具有货柜内物品的标记或者待结算物品的标记；将子识别结果中货柜物品的重量之和与待结算物品的重量之和的差值结果确定为子识别结果对应的重量变化值。
在一些实施例中，图像识别模块810可以进一步被配置为对采集的物品图像进行识别，获得一个或多个识别结果和每个识别结果的置信度；用户取放物品确定模块820可以进一步被配置为在与货柜内物品的重量变化值匹配的重量变化值所对应的识别结果有多个的情况下，将置信度最高的匹配的识别结果中的物品确定为用户取放的物品。
在一些实施例中，物品识别装置80还可以包括：识别结果筛选模块830，被配置为在识别结果中的一种物品的数量超过在货柜内物品的集合和待结算物品的集合中同一种物品的数量的情况下，删除识别结果。
在一些实施例中，物品识别装置80还可以包括：重量监测模块840，被配置为监测货柜的承载重量；图像采集模块850，被配置为响应于货柜的承载重量发生变化，获取采集的物品图像。
在一些实施例中，图像采集模块850可以进一步被配置为响应于货柜的承载重量 发生变化，获取一段时间内的用户在货柜上取放物品时拍摄的视频，并从视频中获取物品图像，其中，一段时间是从货柜的承载重量发生变化之前的预设时刻、到货柜的承载重量再次发生变化的时刻。
下面参考图9描述本公开物品识别系统的实施例。
图9为根据本公开一些实施例的物品识别系统的示例性结构图。如图9所示，该实施例的物品识别系统90包括：物品识别装置91，其具体实施方式可以参考图8实施例中的物品识别装置80；摄像装置92，被配置为采集物品图像；称重设备93，被配置为获取货柜内物品的重量变化值。
在一些实施例中，物品识别系统还可以包括自动售货机94。一个或多个称重设备93、摄像装置92可以安装在自动售货机94中。
图10为根据本公开另一些实施例的物品识别装置的示例性结构图。如图10所示，该实施例的物品识别装置1000包括：存储器1010以及耦接至该存储器1010的处理器1020，处理器1020被配置为基于存储在存储器1010中的指令，执行前述任意一个实施例中的物品识别方法。
其中，存储器1010例如可以包括系统存储器、固定非易失性存储介质等。系统存储器例如存储有操作系统、应用程序、引导装载程序(Boot Loader)以及其他程序等。
图11为根据本公开又一些实施例的物品识别装置的示例性结构图。如图11所示，该实施例的物品识别装置1100包括：存储器1110以及处理器1120，还可以包括输入输出接口1130、网络接口1140、存储接口1150等。这些接口1130，1140，1150以及存储器1110和处理器1120之间例如可以通过总线1160连接。其中，输入输出接口1130为显示器、鼠标、键盘、触摸屏等输入输出设备提供连接接口。网络接口1140为各种联网设备提供连接接口。存储接口1150为SD卡、U盘等外置存储设备提供连接接口。
本公开的实施例还提供一种计算机可读存储介质，其上存储有计算机程序，其特征在于，该程序被处理器执行时实现前述任意一种物品识别方法。
本领域内的技术人员应当明白，本公开的实施例可提供为方法、系统、或计算机程序产品。因此，本公开可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本公开可采用在一个或多个其中包含有计算机可用程序代码的计算机可用非瞬时性存储介质(包括但不限于磁盘存储器、CD-ROM、光学 存储器等)上实施的计算机程序产品的形式。
本公开是参照根据本公开实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解为可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
以上所述仅为本公开的较佳实施例，并不用以限制本公开，凡在本公开的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本公开的保护范围之内。

Claims (19)

  1. 一种物品识别方法，包括：
    对采集的物品图像进行识别，获得一个或多个识别结果，其中，每个识别结果中包括用户在货柜取放的一个或多个物品；
    根据货柜内物品的重量变化值，将一个识别结果中的物品确定为用户取放的物品。
  2. 根据权利要求1所述的物品识别方法，其中，图像采集区域为用户在取放物品时，被取放物品所经过的区域。
  3. 根据权利要求1所述的物品识别方法，其中，所述根据货柜内物品的重量变化值，将一个识别结果中的物品确定为用户取放的物品包括：
    根据每个物品的重量，计算每个识别结果对应的重量变化值；
    确定与所述货柜内物品的重量变化值匹配的重量变化值所对应的识别结果；
    将匹配的识别结果中的物品确定为用户取放的物品。
  4. 根据权利要求3所述的物品识别方法，其中，所述根据每个物品的重量，计算每个识别结果对应的重量变化值包括：
    根据货柜内物品的集合、待结算物品的集合以及每种物品的数量，将每个识别结果转换为一个或多个子识别结果，其中，每个子识别结果中的物品与识别结果中的物品相同、并且每个物品具有货柜内物品的标记或者待结算物品的标记；
    将子识别结果中货柜物品的重量之和与待结算物品的重量之和的差值确定为子识别结果对应的重量变化值。
  5. 根据权利要求3所述的物品识别方法，其中，
    对采集的物品图像进行识别，获得一个或多个识别结果和每个识别结果的置信度；
    在与所述货柜内物品的重量变化值匹配的重量变化值所对应的识别结果有多个的情况下，将置信度最高的匹配的识别结果中的物品确定为用户取放的物品。
  6. 根据权利要求1所述的物品识别方法，还包括：
    在识别结果中的一种物品的数量超过在货柜内物品的集合和待结算物品的集合中所述同一种物品的数量的情况下，删除所述识别结果。
  7. 根据权利要求1所述的物品识别方法，还包括：
    监测货柜的承载重量；
    响应于货柜的承载重量发生变化，获取采集的物品图像。
  8. 根据权利要求7所述的物品识别方法，其中，
    响应于货柜的承载重量发生变化，获取一段时间内的用户在货柜上取放物品时拍摄的视频，并从所述视频中获取物品图像，其中，所述一段时间是从所述货柜的承载重量发生变化之前的预设时刻、到所述货柜的承载重量再次发生变化的时刻。
  9. 一种物品识别装置，包括：
    图像识别模块，被配置为对采集的物品图像进行识别，获得一个或多个识别结果，其中，每个识别结果中包括用户在货柜取放的一个或多个物品；
    用户取放物品确定模块，被配置为根据货柜内物品的重量变化值，将一个识别结果中获取用户取放的物品。
  10. 根据权利要求9所述的物品识别装置，其中，图像采集区域为用户在取放物品时，被取放物品所经过的区域。
  11. 根据权利要求9所述的物品识别装置，其中，所述用户取放物品确定模块进一步被配置为根据每个物品的重量，计算每个识别结果对应的重量变化值；确定与所述货柜内物品重量变化值匹配的重量变化值所对应的识别结果，将匹配的识别结果中的物品确定为用户取放的物品。
  12. 根据权利要求11所述的物品识别装置，其中，所述用户取放物品确定模块进一步被配置为根据货柜内物品的集合、待结算物品的集合以及每种物品的数量，将每个识别结果转换为一个或多个子识别结果，其中，每个子识别结果中的物品与识别结果中的物品相同、并且每个物品具有货柜内物品的标记或者待结算物品的标记；将子识别结果中货柜物品的重量之和与待结算物品的重量之和的差值确定为子识别结果对应的重量变化值。
  13. 根据权利要求11所述的物品识别装置，其中，
    所述图像识别模块进一步被配置为对采集的物品图像进行识别，获得一个或多个识别结果和每个识别结果的置信度；
    所述用户取放物品确定模块进一步被配置为在与所述货柜内物品的重量变化值匹配的重量变化值所对应的识别结果有多个的情况下，将置信度最高的匹配的识别结果中的物品确定为用户取放的物品。
  14. 根据权利要求9所述的物品识别装置，还包括：
    识别结果筛选模块，被配置为在识别结果中的一种物品的数量超过在货柜内物品的集合和待结算物品的集合中所述同一种物品的数量的情况下，删除所述识别结果。
  15. 根据权利要求9所述的物品识别装置，还包括：
    重量监测模块，被配置为监测货柜的承载重量；
    图像采集模块，被配置为响应于货柜的承载重量发生变化，获取采集的物品图像。
  16. 根据权利要求15所述的物品识别装置，其中，所述图像采集模块进一步被配置为响应于货柜的承载重量发生变化，获取一段时间内的用户在货柜上取放物品时拍摄的视频，并从所述视频中获取物品图像，其中，所述一段时间的是从所述货柜的承载重量发生变化之前的预设时刻、到所述货柜的承载重量再次发生变化的时刻。
  17. 一种物品识别系统，包括：
    权利要求9～16中任一项所述的物品识别装置；
    摄像装置，被配置为采集物品图像；和
    称重设备，被配置为获取货柜内物品的重量变化值。
  18. 根据权利要求17所述的物品识别系统，还包括：
    自动售货机。
  19. 一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现权利要求1～8中任一项所述的物品识别方法。
PCT/CN2019/075092 2018-03-02 2019-02-14 物品识别方法、装置、系统及存储介质 WO2019165894A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810175167.XA CN108335408B (zh) 2018-03-02 2018-03-02 用于自动售货机的物品识别方法、装置、系统及存储介质
CN201810175167.X 2018-03-02

Publications (1)

Publication Number Publication Date
WO2019165894A1 true WO2019165894A1 (zh) 2019-09-06

Family

ID=62930255

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/075092 WO2019165894A1 (zh) 2018-03-02 2019-02-14 物品识别方法、装置、系统及存储介质

Country Status (2)

Country Link
CN (1) CN108335408B (zh)
WO (1) WO2019165894A1 (zh)

Families Citing this family (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108335408B (zh) * 2018-03-02 2020-11-03 北京京东尚科信息技术有限公司 用于自动售货机的物品识别方法、装置、系统及存储介质
CN108986357A (zh) * 2018-08-21 2018-12-11 深圳码隆科技有限公司 商品信息确定方法、系统和无人售货系统
CN109272647A (zh) * 2018-08-29 2019-01-25 北京华沁智联科技有限公司 自动售卖仓库物品状态的更新方法及装置
CN109696870A (zh) * 2018-09-12 2019-04-30 盈奇科技（深圳）有限公司 一种基于4g网络无人重力感应售柜电路系统
CN109727378A (zh) * 2018-09-12 2019-05-07 盈奇科技（深圳）有限公司 一种重力感应无人生鲜售柜系统
CN110926585B (zh) * 2018-09-20 2021-12-14 北京京东尚科信息技术有限公司 用于输出信息的方法和装置
CN110379070B (zh) * 2018-11-27 2023-05-02 北京京东乾石科技有限公司 商品检测方法和装置
CN109649915B (zh) * 2018-11-27 2021-01-19 上海京东到家元信信息技术有限公司 一种智能货柜货物识别方法和装置
CN109649916B (zh) * 2018-11-27 2021-01-26 上海京东到家元信信息技术有限公司 一种智能货柜货物识别方法和装置
CN111292466B (zh) * 2018-12-10 2022-06-07 北京京东乾石科技有限公司 售货柜及其订单生成方法和系统
CN109766962B (zh) * 2018-12-18 2021-01-19 创新奇智(南京)科技有限公司 一种商品识别方法、存储介质及商品识别系统
CN109389747A (zh) * 2018-12-29 2019-02-26 北京沃东天骏信息技术有限公司 售货装置和售货方法
CN109959974B (zh) * 2019-01-19 2020-10-09 创新奇智(重庆)科技有限公司 一种异物检测方法、计算机可读存储介质及检测系统
CN111523348B (zh) * 2019-02-01 2024-01-05 百度（美国）有限责任公司 信息生成方法和装置、用于人机交互的设备
CN109886169B (zh) * 2019-02-01 2022-11-22 腾讯科技（深圳）有限公司 应用于无人货柜的物品识别方法、装置、设备及存储介质
CN109840504B (zh) * 2019-02-01 2022-11-25 腾讯科技（深圳）有限公司 物品取放行为识别方法、装置、存储介质及设备
CN109934238B (zh) * 2019-03-06 2020-07-14 北京旷视科技有限公司 物品的识别方法、设备及存储介质
CN109977825B (zh) * 2019-03-15 2021-06-25 百度在线网络技术（北京）有限公司 物品识别方法及装置
CN110031074A (zh) * 2019-03-21 2019-07-19 钛深科技(深圳)有限公司 基于压力传感阵列的物品分类识别方法及其系统
CN110189466A (zh) * 2019-03-22 2019-08-30 上海小萌科技有限公司 一种智能售货柜商品监测方法、装置及系统
CN110147723B (zh) * 2019-04-11 2022-08-19 苏宁云计算有限公司 一种无人店中顾客异常行为的处理方法及系统
CN111860071A (zh) * 2019-04-30 2020-10-30 百度时代网络技术（北京）有限公司 用于识别物品的方法和装置
CN112052708B (zh) * 2019-06-06 2024-06-18 杭州海康威视数字技术股份有限公司 物品检测方法、装置及系统
CN112304410A (zh) * 2019-07-31 2021-02-02 梅特勒-托利多(常州)测量技术有限公司 具有物体识别的称重装置和称重方法
CN110533828A (zh) * 2019-08-30 2019-12-03 北京牧家科技有限公司 一种基于重力感应智能终端的售卖系统
CN112466035B (zh) * 2019-09-06 2022-08-12 图灵通诺（北京）科技有限公司 基于视觉和重力感应的商品识别方法、装置和系统
CN111127750A (zh) * 2019-12-24 2020-05-08 西安科技大学 一种基于重力传感器数据的商品移位识别方法
CN111126322A (zh) * 2019-12-27 2020-05-08 北京每日优鲜电子商务有限公司 应用于无人售货装置的物品识别方法、装置及设备
CN111178265A (zh) * 2019-12-30 2020-05-19 北京每日优鲜电子商务有限公司 商品识别方法及装置
CN113066225A (zh) * 2019-12-30 2021-07-02 深圳Tcl新技术有限公司 一种基于共享冰箱的购物方法及共享冰箱
CN111739091A (zh) * 2020-03-24 2020-10-02 北京京东乾石科技有限公司 一种物品放回检测方法、装置、设备及介质
CN111815852A (zh) * 2020-07-07 2020-10-23 武汉马克到家科技有限公司 一种用于开门自取式售货柜的图像和重力双模自动商品识别系统
CN113178032A (zh) * 2021-03-03 2021-07-27 北京迈格威科技有限公司 一种视频处理方法、系统及存储介质
CN116308327B (zh) * 2022-12-28 2023-10-27 深圳市销邦数据技术有限公司 一种基于rfid技术的自助收银系统及方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9412099B1 (en) * 2013-05-09 2016-08-09 Ca, Inc. Automated item recognition for retail checkout systems
CN106952402A (zh) * 2017-03-22 2017-07-14 帮团成都电子商务有限责任公司 一种数据处理方法及装置
CN107134053A (zh) * 2017-04-19 2017-09-05 石道松 智能售货门店
CN107330684A (zh) * 2017-07-06 2017-11-07 广州联业商用机器人科技股份有限公司 一种云端智能管控无人商店及其自动结算方法
US20180033066A1 (en) * 2016-08-01 2018-02-01 Microsoft Technology Licensing, Llc Multi-Signal Based Shopping Cart Content Recognition in Brick-and-Mortar Retail Stores
CN108335408A (zh) * 2018-03-02 2018-07-27 北京京东尚科信息技术有限公司 用于自动售货机的物品识别方法、装置、系统及存储介质

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104978585A (zh) * 2015-06-11 2015-10-14 陈建波 一种自动计价方法
CN105894362A (zh) * 2016-04-01 2016-08-24 天脉聚源(北京)传媒科技有限公司 一种推荐视频中的相关物品的方法及装置
CN105897916B (zh) * 2016-05-26 2019-05-07 北京精益理想科技有限公司 一种获得物品重量状态的方法和系统
CN106202316A (zh) * 2016-07-01 2016-12-07 传线网络科技（上海）有限公司 基于视频的商品信息获取方法及装置
CN105931371B (zh) * 2016-07-12 2018-12-18 帮团成都电子商务有限责任公司 自动售货机及自动售货方法
CN107393152A (zh) * 2017-08-14 2017-11-24 杭州纳戒科技有限公司 自助售货机及自助售货系统
CN109360331A (zh) * 2017-12-29 2019-02-19 广州Tcl智能家居科技有限公司 一种基于物品识别的自动售货方法及自动售货机

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9412099B1 (en) * 2013-05-09 2016-08-09 Ca, Inc. Automated item recognition for retail checkout systems
US20180033066A1 (en) * 2016-08-01 2018-02-01 Microsoft Technology Licensing, Llc Multi-Signal Based Shopping Cart Content Recognition in Brick-and-Mortar Retail Stores
CN106952402A (zh) * 2017-03-22 2017-07-14 帮团成都电子商务有限责任公司 一种数据处理方法及装置
CN107134053A (zh) * 2017-04-19 2017-09-05 石道松 智能售货门店
CN107330684A (zh) * 2017-07-06 2017-11-07 广州联业商用机器人科技股份有限公司 一种云端智能管控无人商店及其自动结算方法
CN108335408A (zh) * 2018-03-02 2018-07-27 北京京东尚科信息技术有限公司 用于自动售货机的物品识别方法、装置、系统及存储介质

Also Published As

Publication number Publication date
CN108335408A (zh) 2018-07-27
CN108335408B (zh) 2020-11-03

Similar Documents

Publication Publication Date Title
WO2019165894A1 (zh) 物品识别方法、装置、系统及存储介质
US20220270063A1 (en) Produce and bulk good management within an automated shopping environment
CN108875664B (zh) 选购商品的识别方法、装置以及售货机
WO2019165892A1 (zh) 自动售货方法、装置和计算机可读存储介质
US10861086B2 (en) Computer vision system and method for automatic checkout
CN106934692B (zh) 物品信息处理系统、方法及装置
CN108389315B (zh) 物品识别方法和装置以及计算机可读存储介质
CN108320404B (zh) 基于神经网络的商品识别方法、装置、自助收银台
CN108140209A (zh) 信息处理装置、信息处理方法和其中存储有程序的记录介质
WO2021042698A1 (zh) 基于视觉和重力感应的商品识别方法、装置和系统
CN108140282A (zh) 信息处理装置、信息处理方法和存储程序的记录介质
US20190327451A1 (en) Video image analysis apparatus and video image analysis method
JP2023076597A (ja) 店舗装置、店舗システム、店舗管理方法、プログラム
CN108805495A (zh) 物品存放管理方法和系统及计算机可读介质
JP2018206159A (ja) 商品を管理するためのシステム、方法、及びプログラム
CN111126990B (zh) 物品自动识别方法、结算方法、装置、终端及存储介质
CN111680654B (zh) 一种基于物品取放事件的人员信息获取方法、装置及设备
JP2018206372A (ja) 商品を管理するためのシステム、方法、及びプログラム
CN111339929B (zh) 一种无人超市的零售系统
JP2024040297A (ja) 物品推定装置、物品推定方法、及びプログラム
CN110443946B (zh) 售货机、物品种类的识别方法和装置
JP7500168B2 (ja) 物品管理装置及びプログラム
CN114358881A (zh) 自助结算方法、装置和系统
CN112950329A (zh) 商品动态信息生成方法、装置、设备及计算机可读介质
CN117690222A (zh) 自动感知补货明细的方法、装置、电子设备以及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19760875

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 01/12/2020)

122 Ep: pct application non-entry in european phase

Ref document number: 19760875

Country of ref document: EP

Kind code of ref document: A1