WO2020134100A1 - 售货机控制方法、装置、系统和售货机 - Google Patents

售货机控制方法、装置、系统和售货机 Download PDF

Info

Publication number
WO2020134100A1
WO2020134100A1 PCT/CN2019/099758 CN2019099758W WO2020134100A1 WO 2020134100 A1 WO2020134100 A1 WO 2020134100A1 CN 2019099758 W CN2019099758 W CN 2019099758W WO 2020134100 A1 WO2020134100 A1 WO 2020134100A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
vending machine
risk
user
goods
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/099758
Other languages
English (en)
French (fr)
Inventor
刘朋樟
刘巍
陈宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Wodong Tianjun Information Technology Co Ltd
Original Assignee
Beijing Wodong Tianjun Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Wodong Tianjun Information Technology Co Ltd filed Critical Beijing Wodong Tianjun Information Technology Co Ltd
Publication of WO2020134100A1 publication Critical patent/WO2020134100A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K17/00—Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07F—COIN-FREED OR LIKE APPARATUS
    • G07F11/00—Coin-freed apparatus for dispensing, or the like, discrete articles
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07F—COIN-FREED OR LIKE APPARATUS
    • G07F9/00—Details other than those peculiar to special kinds or types of apparatus
    • G07F9/02—Devices for alarm or indication, e.g. when empty; Advertising arrangements in coin-freed apparatus

Definitions

  • the present disclosure relates to the field of artificial intelligence technology, and particularly to a vending machine control method, device, system, and vending machine.
  • Unmanned vending machines are an effective expansion and extension of the online retail business to the offline market. Compared with unmanned shelves, unmanned vending machines use a closed space, which can reduce the risk of theft.
  • unmanned vending machines Similar to unmanned shelves, how to effectively prevent the theft of unmanned vending machine products and prevent harmful physical health products from being placed in unmanned vending machines is still a problem that needs to be solved. Since unmanned vending machines mainly sell food and daily necessities, if the above actions are not controlled, it will not only easily cause economic losses, but also may threaten the personal safety of the users of the goods.
  • a vending machine control method which includes: when the vending machine is open: collecting images within the effective range of the vending machine; performing a risk analysis on the collected images; if the collected images are determined If it is abnormal, it is determined that the collected image is a high-risk image; the high-risk image is reported to the vending machine control server in order to identify the user's risk behavior.
  • reporting the high-risk images to the vending machine control server includes: filtering the abnormal images to select some abnormal images; and reporting the selected abnormal images to the vending machine control server.
  • the image abnormality includes one or more of the following: according to image analysis, it is determined that one or more image acquisition devices are blocked; according to image analysis, it is determined that an abnormal object appears in the vending machine; the user performs abnormal operations on the goods, Abnormal operations include one or more of destroying the outer packaging of the goods, adding substances to the inside of the goods, taking substances from the goods or leaving the camera detection area with the goods and returning them; or, the size, color or transparency of the goods have changed.
  • reporting the high-risk image to the vending machine control server includes: judging whether there is a high-risk image if the time after the door of the sales office or keeping the door open state exceeds a predetermined first time; if there is a high-risk image , Then report high-risk images.
  • the vending machine control method further includes: determining the change of the goods based on the collected images; determining the change of the goods based on the gravity detection; and performing the determination and collection when the changes of the goods determined based on the images and the gravity detection are inconsistent
  • the image is a high-risk image operation.
  • the vending machine control method further includes: in the case where it is determined that the goods are taken from the vending machine based on the image or gravity detection, recording the unique identification of the taken out goods, wherein each item in the vending machine has a unique identification ; In the case of determining whether the goods are returned to the vending machine based on the image or gravity detection, determine whether the identification of the returned goods matches the identification of the removed goods; if they do not match, perform the operation of determining that the collected image is a high-risk image , If it matches, delete the recorded identification of the goods.
  • the vending machine control method further includes: receiving a door opening request from the user; if the user account is a restricted user, refusing to trigger the vending machine to open the door.
  • performing risk analysis on the collected images includes: performing risk analysis on the collected images based on local image features and image analysis models of the vending machine.
  • the vending machine control method further includes: when the vending machine control server receives the high-risk image, identify the high-risk image; if it is determined that the user performs the illegal operation by identifying the high-risk image, modify the user's illegal operation statistics Value; mark the user as a restricted user when the user’s violation operation statistical value reaches a predetermined threshold.
  • the vending machine control method further includes: if the vending machine control server determines that the user has not performed the illegal operation based on the high-risk image, updating the image analysis training data set using the high-risk image; analyzing the training data set based on the updated image Update the image features and image analysis models, and send them to the vending machine to update the image features and image analysis models of the vending machine.
  • a vending machine control device including: a high-risk image determination unit configured to perform a risk analysis on the collected images within the effective range of the vending machine when the vending machine is open If it is determined that the collected image is abnormal, the collected image is determined to be a high-risk image; the reporting unit is configured to report the high-risk image to the vending machine control server in order to identify user risk behavior.
  • the reporting unit is further configured to screen for abnormal images and select some abnormal images; report the selected abnormal images to the vending machine control server.
  • the image abnormality includes one or more of the following: according to image analysis, it is determined that one or more image acquisition devices are blocked; according to image analysis, it is determined that an abnormal object appears in the vending machine; the user performs abnormal operations on the goods, Abnormal operations include one or more of destroying the outer packaging of the goods, adding substances to the inside of the goods, taking substances from the goods or leaving the camera detection area with the goods and returning them; or, the size, color or transparency of the goods have changed.
  • the reporting unit is further configured to: determine whether there is a high-risk image if the time after the door of the sales office or keep the door open state exceeds a predetermined first time; if there is a high-risk image, report the high Risk image.
  • the vending machine control device further includes: an image processing unit configured to determine the change in goods based on the collected images; a gravity processing unit configured to determine the change in goods based on gravity detection; the high-risk image determination unit also It is configured to perform an operation of determining that the collected image is a high-risk image when the cargo change determined based on the image and the gravity detection is inconsistent.
  • the high-risk image determination unit is further configured to: in the case where it is determined based on the image or gravity detection that the goods are taken out of the vending machine, record the unique identification of the taken out goods, wherein each goods in the vending machine With a unique identification; in the case where the goods are returned to the vending machine based on the image or gravity detection, determine whether the identification of the returned goods matches the identification of the removed goods; if they do not match, the determination of the collected image is high risk If the operation of the image matches, the recorded cargo identification is deleted.
  • the vending machine control device further includes: a door opening control unit configured to: receive a door opening request from a user; if the user account is a restricted user, refuse to trigger the door opening of the vending machine.
  • a door opening control unit configured to: receive a door opening request from a user; if the user account is a restricted user, refuse to trigger the door opening of the vending machine.
  • the high-risk image determination unit is further configured to: perform risk analysis on the collected images based on the local image features and image analysis models of the vending machine; update the local area based on the image features and image analysis models from the vending machine control server Image features and image analysis models.
  • a vending machine control device including: a memory; and a processor coupled to the memory, the processor configured to execute any one of the above based on instructions stored in the memory Vending machine control method performed by a vending machine.
  • a computer-readable storage medium is provided on which computer program instructions are stored, which when executed by a processor implements the steps of any of the above vending machine control methods.
  • a vending machine control system including: any of the above vending machine control devices; and a vending machine control server configured to: in the case of receiving a high-risk image Next, identify high-risk images.
  • the vending machine is configured to perform risk analysis on the collected image based on the local image features and image analysis model of the vending machine; the vending machine control server is also configured to: if it is determined that the user has not performed the illegal operation based on the high-risk image , Use high-risk images to update the image analysis training data set; update the image features and image analysis model according to the updated image analysis training data set, and send them to the vending machine.
  • the vending machine control server is further configured to: if it is determined that the user performs a violation operation by identifying a high-risk image, modify the user violation operation statistical value; when the user violation operation statistical value reaches a predetermined threshold, the The user is marked as restricted user.
  • a vending machine including: a storage device configured to carry goods; an image acquisition device configured to collect images of the cargo area; and a gravity detection device configured In order to detect the change of the weight of the goods carried by the storage device; and, any one of the above vending machine control devices.
  • FIG. 1 is a flowchart of some embodiments of a vending machine control method of the present disclosure.
  • FIG. 2 is a flowchart of some other embodiments of the vending machine control method of the present disclosure.
  • FIG. 3 is a flowchart of still other embodiments of the vending machine control method of the present disclosure.
  • FIG. 4 is a flowchart of still other embodiments of the vending machine control method of the present disclosure.
  • FIG. 5 is a schematic diagram of some embodiments of a vending machine control device of the present disclosure.
  • FIG. 6 is a schematic diagram of other embodiments of the vending machine control device of the present disclosure.
  • FIG. 7 is a schematic diagram of some embodiments of a vending machine control system of the present disclosure.
  • FIG. 8 is a schematic diagram of some embodiments of the vending machine of the present disclosure.
  • FIG. 1 A flowchart of some embodiments of the vending machine control method of the present disclosure is shown in FIG. 1. Start the following operations when the vending machine is open:
  • step 101 the vending machine opens the door under the user's trigger.
  • the user can trigger the vending machine to open the door by scanning the QR code of the vending machine, or informing the identity of the vending machine by other means.
  • the vending machine collects images within the effective range of the vending machine through the image acquisition device.
  • the effective range may include one or more of the area directly in front of the vending machine, inside the vending machine, and the area between the door and the vending machine after the door is opened.
  • the vending machine includes more than two image acquisition devices, which can collect abnormal and non-abnormal images and cache them, and can also cache non-abnormal images at the time of abnormal images, so as to ensure the cache of high-risk images quality.
  • a five-channel camera may be used, which is located on the left, right, upper, and lower sides to detect the shelf, and to detect the front from the inside to the outside.
  • an image analysis algorithm is used to determine whether the collected image is abnormal.
  • the image abnormality can be determined using an image analysis algorithm.
  • image abnormality may refer to one or more image acquisition devices (such as cameras) being blocked. If it is determined that the collected image is abnormal, step 103 is performed.
  • the possibility of occurrence of an image abnormality may be analyzed based on the set image abnormality, and the image abnormality is determined when the probability exceeds a predetermined abnormality threshold.
  • step 103 it is determined that the currently acquired image is a high-risk image.
  • step 104 the high-risk image is reported to the vending machine control server, so that the server recognizes the user's risk behavior.
  • the image reporting operation may be performed immediately after it is determined that there is a high-risk image to improve the image reporting efficiency; in some embodiments, the door may be opened at the sales office, or the vending machine continues to open the door for more than a predetermined duration Perform the reporting process to reduce the frequency of image reporting and reduce the burden of communication and data processing.
  • FIG. 2 A flowchart of some other embodiments of the vending machine control method of the present disclosure is shown in FIG. 2.
  • step 201 the vending machine opens the door under the user's trigger.
  • step 202 image analysis is performed on the collected image, and it is determined whether the collected image is abnormal based on the image characteristics and the image analysis model stored locally at the vending machine.
  • the image abnormality may be that one or more cameras are blocked. If it is determined that the collected image is abnormal, step 203 is executed. If no abnormal image is found, step 204 is executed.
  • the image abnormality may include a variety of situations, such as: it is determined that an abnormal object has occurred in the vending machine according to the image analysis; the user performs abnormal operations on the goods, and the abnormal operations may include destroying the outer packaging of the goods and adding substances to the inside of the goods 1.
  • the abnormal operations may include destroying the outer packaging of the goods and adding substances to the inside of the goods 1.
  • One or more items taken from the goods or the user returns the goods after leaving the camera detection area with the goods; or, the size, color or transparency of the goods changes.
  • the image features and image analysis models locally stored in the vending machine may be used to perform feature matching analysis to determine whether the image is abnormal.
  • step 203 the images collected by each image collection device are cached as high-risk images, and then step 206 is executed.
  • the change in cargo is determined based on the image collected by the camera, and the change in cargo is determined based on gravity detection.
  • the only goods to be operated can be identified based on the image algorithm based on the collected image, and whether to put back or take out the operation can be identified based on the trajectory of the image.
  • gravity detection it is judged whether the goods are put back or taken out based on the weight increase or decrease.
  • the weight or the combination of goods that may be operated is determined based on the weight change in grams, and the weight reduction is determined, and which one or which goods are taken out is determined based on the weight of various pre-stored goods.
  • step 205 it is determined whether the cargo changes based on the image and the gravity detection are consistent.
  • the action determined according to the image trajectory is inconsistent with the action determined by the increase or decrease of gravity, or the operated product identified by the image does not coincide with the change in gram of gravity, it may indicate that the operation has occurred product occlusion or abnormality.
  • High-risk behaviors such as storage; when the image recognition and gravity recognition results are consistent, it is a normal shopping process. The shopping images are not cached, and the goods are added to or deleted from the shopping cart.
  • step 206 If the changes are consistent, go to step 206; if the changes are not consistent, go to step 203, and collect the image at the current moment as a high-risk image cache.
  • step 206 it is determined whether the vending machine closes the door, or whether the time for which the door remains open despite opening the door exceeds a predetermined first time. If the door of the sales office is triggered, or the opening time exceeds the predetermined first time, step 207 is executed.
  • step 207 may be performed only when the user closes the door to settle. In other embodiments, step 207 may be performed when the vending machine is idle.
  • step 207 it is determined whether high-risk images are cached in the current cache area. If high-risk images are cached, step 208 is executed.
  • image frames are filtered from the cached high-risk images.
  • the filtering operation may include selecting data frames with clear images that are not completely occluded, and retaining only one frame or a few frames for very similar data.
  • step 209 the filtered high-risk image frames are reported to the vending machine control server, and the high-risk images cached in the cache area are deleted.
  • the unmanned vending machine based on video images is more powerful than the unmanned vending machine based on pure gravity and RFID (Radio Frequency Identification), because of its wider selection space and better user experience. More and more attention.
  • the gravity-based unmanned vending machine has the pain point that it cannot monitor the user's shopping process.
  • the image-based unmanned vending machine needs to overcome the high-risk behaviors such as blocking the camera, placing dangerous goods, and covering the theft of goods with both hands. Will cause higher risks and higher cargo damage.
  • the related unmanned vending machine risk control mainly includes two conventional solutions: video image and gravity:
  • the risk control solution based on video images cannot provide effective real-time early warning. Only when the unmanned vending machine operation and maintenance personnel finds that the merchandise in the vending machine is abnormal (there are abnormal items) or the product is lost during verification, it can detect the risk. All videos need to be played back manually. In addition, the above solution first needs to save all the videos and occupy a lot of storage space of the unmanned vending machine. If the video is returned to the cloud through the network, it will occupy a lot of network bandwidth and the operation and maintenance cost will increase dramatically. At the same time, manually tracing video will increase labor costs and reduce work efficiency. The behavior recognition algorithm based on pure vision is not accurate enough at present.
  • the gravity sensor can monitor the weight of the goods taken or returned at any time, so as to realize the deduction.
  • the unmanned vending machine cannot promptly warn in time, resulting in high cargo damage and high risk.
  • there is no image monitoring function when it is found that there are abnormal goods in the unmanned vending machine, it cannot be traced back, and it cannot be located when and who has stolen the goods or placed dangerous goods.
  • abnormal behavior can be discovered based on the results of image analysis, and abnormal image reporting can be triggered in time when an abnormal situation is discovered.
  • the burden of data transmission is reduced, and manual interpretation and backtracking are reduced by orders of magnitude.
  • it can also improve the efficiency of abnormal situation detection, in addition, it can also reduce the probability of missed judgment and improve the safety of sales.
  • step 301 it is determined that the goods are taken out and sold according to the image or gravity detection.
  • each cargo may be configured with a unique identification, such as an RFID identification.
  • a unique identification such as an RFID identification.
  • step 303 is triggered to be executed.
  • step 303 the identification of the returned goods is collected, and it is determined whether it matches the identification of the removed goods. If the two match, go to step 305; if the two do not match, go to step 304.
  • step 304 the current time is buffered, or the collected image of the sales process is intercepted as a high-risk image.
  • step 305 it is determined whether the user has performed an abnormal operation on the goods.
  • the abnormal operation may include one or more of destroying the outer packaging of the goods, adding substances to the inside of the goods, taking substances from the goods, or returning from the camera detection area after the user holds the goods. If it is determined that the user makes an abnormal operation, step 304 is executed. If it is determined that the user has not performed an abnormal operation, step 306 is executed.
  • step 306 it is determined that the user returns the item normally, and the logo is deleted.
  • FIG. 4 A flowchart of still other embodiments of the vending machine control method of the present disclosure is shown in FIG. 4.
  • a door opening request from the user is received.
  • the user's door opening request may include the user's personal identity information, such as the user's account information for the vending machine, or the platform related to the vending machine, or ID number information.
  • step 402 the user identity information is uploaded to the platform to determine whether the user is a restricted user. If it is determined to be a restricted user, step 403 is performed; if it is determined to be a normal user, step 404 is performed.
  • step 403 refusal to trigger the vending machine to open the door.
  • step 404 the vending machine is triggered to open the door.
  • step 405 it is determined that a high-risk image was generated during the shopping process.
  • the method shown in the embodiments shown in FIGS. 2 and 3 may be used to determine whether to generate a high-risk image.
  • the high-risk image may be cached first for subsequent reporting. If it is determined that the high-risk image is generated (cached), step 406 is executed.
  • the high-risk image is reported to the server.
  • some frames may be extracted from the generated high-risk image first, and only the extracted image frames are reported, thereby reducing the data transmission burden.
  • step 407 the server issues an alarm so that the staff can manually identify it, or use a further image recognition method to interpret the high-risk image.
  • step 408 it is determined whether the user performs a violation operation based on the high-risk image. If it is determined that the user has performed the violation operation, step 409 is performed; if it is determined that the user has not performed the violation operation, it is considered that a misjudgment has occurred, and step 412 is performed.
  • step 409 it is determined whether the statistical value of the user's illegal operation has reached a predetermined threshold. If the predetermined threshold has been reached, step 411 is performed; if the predetermined threshold has not been reached, step 410 is performed.
  • the recorded user violation operation statistical value is increased.
  • the amount of increased statistical value may also be determined according to the type of the user's current violation operation, such as a common violation operation plus 1, a malicious violation manipulation plus 10, and so on.
  • step 411 the user right is modified to restrict the user.
  • step 412 the image analysis training data set is updated with high-risk images.
  • the image features and the image analysis model are updated according to the updated image analysis training data set, and sent to the vending machine to update the local image features and the image analysis model.
  • the image features and the image analysis model are updated according to the updated image analysis training data set, and sent to the vending machine to update the local image features and the image analysis model.
  • it can only be sent to the vending machine that provides the high-risk image, while avoiding excessive vending machine data, it can also target the vending machine's location.
  • the environment and the situation experienced are dynamically updated.
  • the user's authority can be dynamically changed according to the user's illegal operation, preventing the user from further illegal operation and improving security; it can also dynamically update the data foundation of image analysis, image extraction features and models, and improve the accuracy of vending machine image analysis To reduce the probability of wrong judgment.
  • the vending machine it is possible to mine difficult examples online through an effective online self-feedback mechanism, find images that are not easily recognized accurately and return them, so that the vending machine realizes self-iteration and optimization of the model, thus quickly adapting to deployment
  • the new environment and new product image recognition requirements save a lot of image acquisition costs of training images, and the improvement of image recognition accuracy will further improve user experience and reduce cargo losses.
  • FIG. 5 A schematic diagram of some embodiments of the vending machine control device of the present disclosure is shown in FIG. 5.
  • the high-risk image determining unit 501 can determine whether the collected image is abnormal when the door of the vending machine is opened, and determine that the collected image is a high-risk image if the abnormality is determined.
  • the vending machine includes more than two image acquisition devices, which can collect abnormal and non-abnormal images and cache them, and can also cache non-abnormal images at the time of abnormal images, so as to ensure the cache of high-risk images quality.
  • the image anomaly can be determined using an image analysis model and image features based on the local vending machine, and a second time using an image analysis algorithm.
  • image abnormality may refer to one or more image acquisition devices (such as cameras) being blocked.
  • the reporting unit 502 can report the high-risk image to the vending machine control server, so that the server recognizes the user's risk behavior.
  • the reporting process may be performed when the door of the vending machine or the vending machine continues to open for more than a predetermined period of time, reducing the frequency of image reporting and reducing the communication and data processing burden.
  • Such a vending machine control device can react in a timely manner when the image collection is abnormal, and upload the collected image to the server for the server to analyze, thereby realizing timely feedback of the image, improving the efficiency of abnormal discovery, and improving the safety of the vending machine.
  • the image abnormality may also include multiple situations: for example, according to image analysis, it is determined that an abnormal object has occurred in the vending machine; the user performs abnormal operations on the goods.
  • the abnormal operations include destroying the outer packaging of the goods, adding substances into the goods, One or more items taken from the goods or the user left the camera detection area with the goods and returned to the camera; or, the size, color or transparency of the goods were changed, thereby reducing the possibility of leaking violations and further improving safety Sex.
  • the reporting unit 502 may first select a portion of the high-risk image frames from the cached high-risk images, and then report the selected high-risk image frames to the vending machine control server, and delete the cached high-risk images. In some embodiments, some frames with low pixels can be deleted, or a few frames can be selected from very similar frames.
  • Such a vending machine control device can reduce the amount of data that needs to be reported to the server, on the one hand reduce the data transmission burden, on the other hand improve the data transmission efficiency, and facilitate the server side to quickly respond to illegal operations.
  • the vending machine control device may further include a gravity processing unit 503 and an image processing unit 504.
  • the gravity processing unit 503 can determine the change of the cargo based on the gravity detection. In some embodiments, the weight reduction can be determined according to the gravity detection, and which cargo or cargos are taken out can be determined based on the weight of various pre-stored cargoes.
  • the image processing unit 504 can determine the change of goods based on the image collected by the camera. In some embodiments, it can determine which goods or goods are taken out based on the collected image.
  • the high-risk image determination unit 501 can cache the high-risk image in the case where the change of the cargo determined based on the image and the gravity detection is inconsistent, so that the reporting unit can report.
  • Such a vending machine control device can determine whether the cached high-risk image is triggered and reported by the two ways of judging whether the image is abnormal and whether the result of the image and the change of gravity detection is consistent, which reduces the probability of missed event extraction and improves the vending machine The safety of selling goods.
  • the vending machine control device may further include a door opening control unit 505, which can determine whether the user is a restricted user when receiving the user's trigger of the door opening operation. If it is determined that the user is a restricted user, it refuses to trigger the vending machine to open the door; if the user is a normal user, the door opening operation is performed.
  • a door opening control unit 505 can determine whether the user is a restricted user when receiving the user's trigger of the door opening operation. If it is determined that the user is a restricted user, it refuses to trigger the vending machine to open the door; if the user is a normal user, the door opening operation is performed.
  • Such a vending machine control device can use the user credit system to determine whether to provide services to the user, which further improves security.
  • the vending machine control device includes a memory 601 and a processor 602.
  • the memory 601 may be a magnetic disk, flash memory, or any other non-volatile storage medium.
  • the memory is used to store the instructions in the corresponding embodiments of the vending machine control method above.
  • the processor 602 is coupled to the memory 601, and may be implemented as one or more integrated circuits, such as a microprocessor or a microcontroller.
  • the processor 602 is used to execute instructions stored in the memory, which can realize timely feedback of images, improve the efficiency of abnormal discovery, and improve the safety of the vending machine.
  • a computer-readable storage medium has stored thereon computer program instructions, which when executed by a processor, implements the steps of the method in the embodiment corresponding to the vending machine control method.
  • the embodiments of the present disclosure may be provided as methods, devices, or computer program products. Therefore, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer usable non-transitory storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code .
  • FIG. 7 A schematic diagram of some embodiments of the vending machine control system of the present disclosure is shown in FIG. 7.
  • the vending machine control device 71 may be any of the vending machine control devices mentioned above.
  • the vending machine control server 70 can interpret the high-risk image when receiving the high-risk image. In some embodiments, in order to improve the timeliness of risk assessment, an alarm may be issued.
  • the vending machine control device can react in a timely manner when the image acquisition is abnormal, upload the collected image to the server for the server to analyze, in order to determine the illegal operation in time, so as to achieve timely feedback through the image, Improve the efficiency of abnormal discovery and improve the security of vending machines; it can dynamically change the user's authority according to the user's illegal operation, prevent the user from further illegal operation, and improve safety.
  • the user authority is modified to restrict the user, so that when the user uses the vending machine again, the vending machine refuses to The user opens the door, so as to realize the statistics and effective application of user credit, and improve the safety of vending machine sales.
  • the vending machine control server 70 determines that the user has not performed the illegal operation after reading the high-risk image
  • the high-risk image may be used to update the image analysis training data set, and the image characteristics may be updated according to the updated image analysis training data set.
  • the image analysis model, and sent to the vending machine the vending machine control device updates the local image characteristics and image analysis model, and realizes the optimization of its own image analysis function.
  • Such a vending machine control system can dynamically update the data basis of image analysis, image extraction features and models, improve the accuracy of vending machine image analysis, and reduce the probability of erroneous judgment.
  • FIG. 8 A schematic diagram of some embodiments of the vending machine 80 of the present disclosure is shown in FIG. 8.
  • the storage device 81 can carry the goods for sale; the gravity detection device 82 can collect the weight change on the shelf; the image collection device 83 can collect images of the inside of the vending machine and the area where the user operates.
  • the gravity detection device and the image acquisition device may include multiple.
  • the vending machine control device 84 may be any of the vending machine control devices mentioned above.
  • Such a vending machine can react in a timely manner when the image acquisition is abnormal, and upload the collected image to the server for the server to analyze, thereby realizing timely feedback of the image, improving the efficiency of abnormal discovery, and improving the security of the vending machine.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Control Of Vending Devices And Auxiliary Devices For Vending Devices (AREA)

Abstract

一种售货机（80）控制方法、装置（71,84）、系统和售货机（80），其中售货机（80）控制方法，包括：售货机（80）开门（101）；若采集的图像异常（102），确定当前采集的图像为高风险图像（103）；将高风险图像上报售货机控制服务器（70），以便识别用户风险行为（104）。通过这样的方法，能够在图像采集异常的情况下及时做出反应，将采集的图像上传给服务器（70）供服务器（70）解析，从而实现图像的及时反馈，提高异常发现的效率，提高售货机（80）的安全性。

Description

售货机控制方法、装置、系统和售货机
相关申请的交叉引用
本申请是以CN申请号为201811630378.4，申请日为2018年12月29日的申请为基础，并主张其优先权，该CN申请的公开内容在此作为整体引入本申请中。
技术领域
本公开涉及人工智能技术领域，特别是一种售货机控制方法、装置、系统和售货机。
背景技术
随着零售业高速发展，越来越多的互联网企业开始探索新的零售模式，无人售货机是线上零售业务向线下市场的有效扩展与延伸。无人售货机相对于无人货架采用了封闭的空间，能够降低偷盗的风险。
类似于无人货架，如何有效防止无人售货机商品被偷盗，防止将有害身体健康的商品放入无人售货机仍然是需要解决的问题。由于无人售卖机主要销售食品与日常生活必需品，因此上述行为如果得不到控制，不仅仅容易引起经济损失，还可能会威胁到商品使用者的人身安全。
发明内容
根据本公开的一些实施例的一个方面，提出一种售货机控制方法，包括：在售货机开门状态下：采集售货机有效范围内的图像；对采集的图像进行风险分析；若确定采集的图像异常，则确定采集的图像为高风险图像；将高风险图像上报售货机控制服务器，以便识别用户风险行为。
在一些实施例中，将高风险图像上报售货机控制服务器包括：对异常图像进行筛选，挑选出部分异常图像；将挑选出的异常图像上报售货机控制服务器。
在一些实施例中，图像异常包括以下一项或多项：根据图像分析确定一个或多个图像采集设备被遮挡；根据图像分析确定售货机内出现了异常物体；用户对货物做出异常操作，异常操作包括破坏货物外包装、向货物内部加入物质、从货物中取物质或用户持货物离开摄像头探测区域后又返回中的一项或多项；或，货物的尺寸、颜色或 透明度发生改变。
在一些实施例中，将高风险图像上报售货机控制服务器包括：在售货机关门后或保持开门状态的时长超过预定第一时长的情况下，判断是否存在高风险图像；若存在高风险图像，则上报高风险图像。
在一些实施例中，售货机控制方法还包括：基于采集的图像确定货物变化情况；基于重力探测确定货物变化情况；在基于图像和基于重力探测确定的货物变化情况不一致的情况下，执行确定采集的图像为高风险图像的操作。
在一些实施例中，售货机控制方法还包括：在根据图像或重力探测确定货物被取出售货机的情况下，记录被取出的货物的唯一标识，其中，售货机中的每个货物具备唯一标识；在根据图像或重力探测确定货物被放回售货机的情况下，确定放回的货物的标识与取出的货物的标识是否匹配；若不匹配，则执行确定采集的图像为高风险图像的操作，若匹配，则删除记录的货物的标识。
在一些实施例中，售货机控制方法还包括：接收来自用户的开门请求；若用户账户为限制用户，则拒绝触发售货机开门。
在一些实施例中，对采集的图像进行风险分析包括：基于售货机本地的图像特征和图像分析模型对采集的图像进行风险分析。
在一些实施例中，售货机控制方法还包括：售货机控制服务器在收到高风险图像的情况下，识别高风险图像；若通过识别高风险图像确定用户执行违规操作，则修改用户违规操作统计值；在用户的违规操作统计值达到预定门限的情况下，将用户标记为限制用户。
在一些实施例中，售货机控制方法还包括：若售货机控制服务器根据高风险图像确定用户未执行违规操作，则利用高风险图像更新图像分析训练数据集；根据更新后的图像分析训练数据集更新图像特征和图像分析模型，并发送给售货机以便更新售货机的图像特征和图像分析模型。
根据本公开的另一些实施例的一个方面，提出一种售货机控制装置，包括：高风险图像确定单元，被配置为在售货机开门状态下对采集的售货机有效范围内的图像进行风险分析，若确定采集的图像异常，则确定采集的图像为高风险图像；上报单元，被配置为将高风险图像上报售货机控制服务器，以便识别用户风险行为。
在一些实施例中，上报单元还被配置为对异常图像进行筛选，挑选出部分异常图像；将挑选出的异常图像上报售货机控制服务器。
在一些实施例中，图像异常包括以下一项或多项：根据图像分析确定一个或多个图像采集设备被遮挡；根据图像分析确定售货机内出现了异常物体；用户对货物做出异常操作，异常操作包括破坏货物外包装、向货物内部加入物质、从货物中取物质或用户持货物离开摄像头探测区域后又返回中的一项或多项；或，货物的尺寸、颜色或透明度发生改变。
在一些实施例中，上报单元还被配置为：在售货机关门后或保持开门状态的时长超过预定第一时长的情况下，判断是否存在高风险图像；若存在高风险图像，则上报高风险图像。
在一些实施例中，售货机控制装置还包括：图像处理单元，被配置为基于采集的图像确定货物变化情况；重力处理单元，被配置为基于重力探测确定货物变化情况；高风险图像确定单元还被配置为在基于图像和基于重力探测确定的货物变化情况不一致的情况下，执行确定采集的图像为高风险图像的操作。
在一些实施例中，高风险图像确定单元还被配置为：在根据图像或重力探测确定货物被取出售货机的情况下，记录被取出的货物的唯一标识，其中，售货机中的每个货物具备唯一标识；在根据图像或重力探测确定货物被放回售货机的情况下，确定放回的货物的标识与取出的货物的标识是否匹配；若不匹配，则执行确定采集的图像为高风险图像的操作，若匹配，则删除记录的货物的标识。
在一些实施例中，售货机控制装置还包括：开门控制单元，被配置为：接收来自用户的开门请求；若用户账户为限制用户，则拒绝触发售货机开门。
在一些实施例中，高风险图像确定单元还被配置为：基于售货机本地的图像特征和图像分析模型对采集的图像进行风险分析；根据来自售货机控制服务器的图像特征和图像分析模型更新本地的图像特征和图像分析模型。
根据本公开的又一些实施例的一个方面，提出一种售货机控制装置，包括：存储器；以及耦接至存储器的处理器，处理器被配置为基于存储在存储器的指令执行如上文中任意一种由售货机执行的售货机控制方法。
根据本公开的再一些实施例的一个方面，提出一种计算机可读存储介质，其上存储有计算机程序指令，该指令被处理器执行时实现上文中任意一种售货机控制方法的步骤。
根据本公开的其中一些实施例的一个方面，提出一种售货机控制系统，包括：上文中任意一种售货机控制装置；和售货机控制服务器，被配置为：在收到高风险图像 的情况下，识别高风险图像。
在一些实施例中，售货机被配置为基于售货机本地的图像特征和图像分析模型对采集的图像进行风险分析；售货机控制服务器还被配置为：若根据高风险图像确定用户未执行违规操作，则利用高风险图像更新图像分析训练数据集；根据更新后的图像分析训练数据集更新图像特征和图像分析模型，并发送给售货机。
在一些实施例中，售货机控制服务器还被配置为：若通过识别高风险图像确定用户执行违规操作，则修改用户违规操作统计值；在用户的违规操作统计值达到预定门限的情况下，将用户标记为限制用户。
另外，根据本公开的一些实施例的一个方面，还提出一种售货机，包括：储物装置，被配置为承载货物；图像采集装置，被配置为采集货物区域图像；重力探测装置，被配置为探测储物装置承载的货物重量变化情况；和，上文中任意一种售货机控制装置。
附图说明
此处所说明的附图用来提供对本公开的进一步理解，构成本公开的一部分，本公开的示意性实施例及其说明用于解释本公开，并不构成对本公开的不当限定。在附图中：
图1为本公开的售货机控制方法的一些实施例的流程图。
图2为本公开的售货机控制方法的另一些实施例的流程图。
图3为本公开的售货机控制方法的又一些实施例的流程图。
图4为本公开的售货机控制方法的再一些实施例的流程图。
图5为本公开的售货机控制装置的一些实施例的示意图。
图6为本公开的售货机控制装置的另一些实施例的示意图。
图7为本公开的售货机控制系统的一些实施例的示意图。
图8为本公开的售货机的一些实施例的示意图。
具体实施方式
下面通过附图和实施例，对本公开的技术方案做进一步的详细描述。
本公开的售货机控制方法的一些实施例的流程图如图1所示。在售货机开门的状态下开始执行如下操作：
在步骤101中，售货机在用户的触发下开门。在一些实施例中，用户可以通过扫描售货机二维码，或通过其他方式告知售货机自身的身份以触发售货机开门。
当用户触发开门后，售货机通过图像采集设备采集售货机有效范围内的图像。在一些实施例中，有效范围可以包括售货机的正前方，售货机的里面，以及开门之后门与售货机之间的区域中的一种或多种。在一些实施例中，售货机包括2路以上图像采集设备，可以采集异常和非异常的图像且均缓存，还可以缓存存在异常图像的时刻的未异常的图像，从而保证缓存的高风险图像的质量。在一些实施例中，可以采用5路摄像头，分别位于左、右、上、下侧探测货架，以及由货架内向外探测前方。
在步骤102中，通过图像分析算法判断采集的图像是否异常。在一些实施例中，图像异常可以利用图像分析算法确定。在一些实施例中，图像异常可以指的是一个或多个图像采集设备(如摄像头)被遮挡。若确定采集的图像异常，则执行步骤103。
在一些实施例中，可以基于设定的图像异常情况分析出现图像异常的可能性，当可能性超过预定异常阈值时确定图像异常。
在步骤103中，确定当前采集的图像为高风险图像。
在步骤104中，将高风险图像上报售货机控制服务器，以便服务器识别用户风险行为。在一些实施例中，图像上报的操作可以在确定存在高风险图像后即时进行从而提高图像上报效率；在一些实施例中，可以在售货机关门，或售货机持续开门的时间超过预定时长时执行上报过程，降低图像上报的频率，降低通信和数据处理负担。
通过这样的方法，能够在图像采集异常的情况下及时做出反应，将采集的图像上传给服务器供服务器解析，从而实现及时反馈图像，提高异常发现的效率，提高售货机的安全性。
本公开的售货机控制方法的另一些实施例的流程图如图2所示。
在步骤201中，售货机在用户的触发下开门。
在步骤202中，对采集的图像进行图像分析，基于售货机本地存储的图像特征和图像分析模型判断采集的图像是否异常。在一些实施例中，图像异常可以为出现一个或多个摄像头被遮挡。若确定采集的图像异常，则执行步骤203。若未发现图像异常，则执行步骤204。
在一些实施例中，图像异常可以包括多种情况，如：根据图像分析确定售货机内出现了异常物体；用户对货物做出异常操作，异常操作可以包括破坏货物外包装、向货物内部加入物质、从货物中取物质或用户持货物离开摄像头探测区域后又返回中的 一项或多项；或，货物的尺寸、颜色或透明度发生改变等。
在一些实施例中，可以利用售货机本地存储的图像特征和图像分析模型进行特征匹配分析，确定图像是否异常。
在步骤203中，缓存各个图像采集设备采集的图像作为高风险图像，进而执行步骤206。
在步骤204中，基于摄像头采集的图像确定货物变化情况，基于重力探测确定货物变化情况。在一些实施例中，可以根据采集的图像，基于图像算法识别出被操作的唯一货物，基于图像的轨迹识别出是放回还是拿出操作。根据重力探测，基于重量增加或减少判断是货物放回还是拿出。基于重量变化克数确定可能被操作的货物或货物组合确定重量减少量，基于预存的各种货物的重量确定哪个或哪些货物被取出。
在步骤205中，基于图像和基于重力探测确定的货物变化情况是否一致。在一些实施例中，当根据图像轨迹判断出的动作与重力增减判断的动作不一致，或者图像识别的被操作商品与重力变化克数不吻合时，说明该次操作可能出现了商品遮挡、异常置物等高风险行为；当图像识别与重力识别结果一致时，说明这是一次正常的购物流程，不缓存购物图像，将商品加入购物车或从购物车删减。
若变化情况一致，则执行步骤206；若变化情况不一致，则执行步骤203，采集当前时刻的图像作为高风险图像缓存。
在步骤206中，判断售货机是否关门，或是否虽然开门但保持开门状态的时长超过预定第一时长。若触发售货机关门，或开门时长超过预定第一时长，则执行步骤207。
在一些实施例中，可以只在用户关门结算时，执行步骤207。在另一些实施例中，可以在售货机通信闲时执行步骤207。
在步骤207中，判断当前缓存区是否缓存有高风险图像。若缓存有高风险图像，则执行步骤208。
在步骤208中，从缓存的高风险图像中筛选图像帧。在一些实施例中，筛选操作可以包括挑选图像清晰的，未完全被遮挡的数据帧，以及对于极为相似的数据只保留一帧或几帧等。
在步骤209中，将筛选出的高风险图像帧上报售货机控制服务器，并删除缓存区缓存的高风险图像。
通过这样的方法，能够通过图像是否异常，以及图像与重力探测的变化情况判读结果是否一致这两种途径判断是否触发缓存高风险图像并上报，降低了事件漏提取的 概率，提高了售货机贩卖货物的安全性；同时，也无需持续存储图像，降低了对售货机存储空间的要求。
相关技术中，基于视频图像的无人售货机相对于基于纯重力、RFID(Radio Frequency Identification，射频识别)的无人售货机，由于其具备更广阔的选品空间，更优质的用户体验，引起越来越多的关注。但是基于重力的无人售货机存在无法监控用户购物流程的痛点，基于图像的无人售货机需要克服真实场景中容易出现的遮挡摄像头、危险品放置、双手盖住商品盗窃等高风险行为，仍然会造成较高的风险及较高的货损。
相关的无人售货机风险控制主要包含视频图像与重力两种常规解决方案：
(1)通过布置监控摄像头持续监控顾客消费过程，当补货人员或无人售货机维护人员发现售货机内出现异常商品或商品丢失时，可以通过查看录像，还原所有用户的完整的购物过程。
基于视频图像的风险控制解决方案无法提供有效的实时预警，只有当无人售货机运维人员发现售货机内的商品异常(有异常物品)或核对时发现商品丢失时，才能够发现风险，同时需要人工回放全部录像。此外，上述方案首先需要保存全部录像，占用无人售货机大量的存储空间，而如果通过网络回传视频到云端，则会占用大量网络带宽，运维成本急剧增加。同时，人工追溯视频会提高人力成本，且降低工作效率。基于纯视觉的行为识别算法目前尚不够准确，因此单独利用图像算法虽然能够节约存储空间，降低人力成本，但容易触发大量虚假报警，同时由于商品偷盗行为、异常置物行为与正常购物行为相似度高，很难利用算法区分，因此容易发生漏报，留下安全隐患。
(2)当无人售货机发现重力异常时，可以提供后台报警，运维人员可根据警情，到现场查看售货机情况，排除隐患。
基于重力的解决方案中，重力传感器能够时刻监控被取走或放回商品的重量，从而实现扣款。然而，当用户取走商品但是放回等重量的无用物品(比如沙子)或危险品时，无人售货机无法及时预警，从而导致高货损与高风险。同时由于不具备图像监控功能，因此当发现无人售货机出现异常商品时，无法进行追溯，无法定位何时、何人人偷盗了商品或置入了危险品。
通过本公开上述实施例中的方法，能够根据图像分析的结果发现异常行为，并在发现异常情况时及时触发上报异常图像，一方面降低了数据传输的负担，成数量级的 降低了人工判读、回溯的工作量；另一方面还能够提高异常情况发现的效率，另外，还能够降低漏判的概率，提高售货安全性。
在一些实施例中，为了应对违规用户将从售货机中取出的物品中掺入异常物质(如注射有毒液体)，或置换为其他的外观相同、相似的物品重新放回售货机的情况，本公开的售货机控制方法的又一些实施例的流程图如图3所示。
在步骤301中，根据图像或重力探测确定货物被取出售货机。
在步骤302中，记录被取出的货物的唯一标识。在一些实施例中，每个货物可以配置有唯一标识，如RFID标识。当用户结账关闭售货机后，删除记录的货物的唯一标识。
当根据图像或重力探测确定货物被放回售货机时，触发执行步骤303。
在步骤303中，采集放回的货物的标识，并判断与取出的货物的标识是否匹配。若两者匹配，则执行步骤305；若两者不匹配，则执行步骤304。
在步骤304中，缓存当前时刻，或截取采集的本次售货过程的图像作为高风险图像。
在步骤305中，判断用户是否对货物做出异常操作。异常操作可以包括破坏货物外包装、向货物内部加入物质、从货物中取物质或用户持货物离开摄像头探测区域后又返回中的一项或多项。若确定用户做出异常操作，则执行步骤304。若确定用户未做出异常操作，则执行步骤306。
在步骤306中，确定用户正常放回物品，删除标识。
通过这样的方法，能够应对将从售货机中取出的物品中掺入异常物质(如注射有毒液体)，或置换为其他的外观相同、相似的物品重新放回售货机的情况，进一步提高了售货机贩卖物品的安全性。
本公开的售货机控制方法的再一些实施例的流程图如图4所示。
在步骤401中，接收来自用户的开门请求。用户的开门请求中可以包括用户的个人身份信息，如用户对于售货机，或售货机相关的平台的账号信息，或身份证号码信息等。
在步骤402中，将用户身份信息上传平台判断用户是否为限制用户。若确定为限制用户，则执行步骤403；若确定为正常用户，则执行步骤404。
在步骤403中，拒绝触发售货机开门。
在步骤404中，触发售货机开门。
在步骤405中，判断在购物过程中产生了高风险图像。在一些实施例中，可以采用如图2、3所示实施例中的方式确定是否产生高风险图像。在一些实施例中，当确定产生高风险图像时，可以先缓存高风险图像以便后续上报。若确定产生(缓存)了高风险图像，则执行步骤406。
在步骤406中，向服务器上报高风险图像。在一些实施例中，可以先从产生的高风险图像中提取部分帧，只上报提取的图像帧，从而降低数据传输负担。
在步骤407中，服务器发出告警，以便工作人员人工识别，或采用进一步的图像识别方法判读高风险图像。
在步骤408中，根据高风险图像确定用户是否执行违规操作。若确定用户执行了违规操作，则执行步骤409；若确定用户未执行违规操作，则认为当前出现了误判，执行步骤412。
在步骤409中，判断该用户违规操作统计值是否达到了预定门限。若已达到预定门限，则执行步骤411；若未达到预定门限，则执行步骤410。
在步骤410中，增加记录的用户违规操作统计值。在一些实施例中，还可以根据用户当前违规操作的种类确定增加的统计值的量，如普通违规操作加1，恶意违规操纵加10等。
在步骤411中，修改用户权限为限制用户。
在步骤412中，利用高风险图像更新图像分析训练数据集。
在步骤413中，根据更新后的图像分析训练数据集更新图像特征和图像分析模型，并发送给提供给售货机更新售货机本地的图像特征和图像分析模型。在一些实施例中，考虑到售货机的存储空间和数据处理负担，可以只发送给提供了该高风险图像的售货机，在避免售货机数据量过多的同时，也能够针对售货机所处的环境、所经历的情况实现动态更新。
通过这样的方法，能够根据用户违规操作情况动态改变用户的权限，预防用户进一步违规操作，提高安全性；还能够动态更新图像分析的数据基础，图像提取特征和模型，提高售货机图像分析的准确性，降低错误判断的概率。由于在售货机使用过程中，能够通过有效的在线自反馈机制进行在线的难例挖掘，寻找不容易被准确识别的图像并回传，使售货机实现模型的自我迭代与优化，从而快速适应部署新环境、新商品的图像识别需求，节省大量训练图像的图像采集成本，同时图像识别准确度的提升也会进一步提高用户体验、减少货物损失。
本公开的售货机控制装置的一些实施例的示意图如图5所示。
高风险图像确定单元501能够在售货机开门的状态下，判断采集的图像是否异常，并在确定异常的情况下确定采集的图像为高风险图像。在一些实施例中，售货机包括2路以上图像采集设备，可以采集异常和非异常的图像且均缓存，还可以缓存存在异常图像的时刻的未异常的图像，从而保证缓存的高风险图像的质量。在一些实施例中，图像异常可以利用基于售货机本地的图像分析模型和图像特征，次用图像分析算法确定。在一些实施例中，图像异常可以指的是一个或多个图像采集设备(如摄像头)被遮挡。
上报单元502能够将高风险图像上报售货机控制服务器，以便服务器识别用户风险行为。在一些实施例中，可以在售货机关门，或售货机持续开门的时间超过预定时长时执行上报过程，降低图像上报的频率，降低通信和数据处理负担。
这样的售货机控制装置能够在图像采集异常的情况下及时做出反应，将采集的图像上传给服务器供服务器解析，从而实现图像的及时反馈，提高异常发现的效率，提高售货机的安全性。
在一些实施例中，图像异常还可以包括多种情况：如根据图像分析确定售货机内出现了异常物体；用户对货物做出异常操作，异常操作包括破坏货物外包装、向货物内部加入物质、从货物中取物质或用户持货物离开摄像头探测区域后又返回中的一项或多项；或，货物的尺寸、颜色或透明度发生改变，从而降低漏发现违规操作的可能性，进一步提高了安全性。
在一些实施例中，上报单元502可以先从缓存的高风险图像中筛选部分高风险图像帧，再将筛选出的高风险图像帧上报售货机控制服务器，并删除缓存区缓存的高风险图像。在一些实施例中，可以删除部分像素低的帧，或从极为相似的帧中选择几帧。这样的售货机控制装置能够降低需要上报给服务器的数据量，一方面降低数据传输负担，另一方面提高数据传输效率，便于服务器侧对于违规操作的迅速反应。
在一些实施例中，如图5所示，售货机控制装置还可以包括重力处理单元503和图像处理单元504。重力处理单元503能够基于重力探测确定货物变化情况，在一些实施例中，可以根据重力探测确定重量减少量，基于预存的各种货物的重量确定哪个或哪些货物被取出。图像处理单元504能够基于摄像头采集的图像确定货物变化情况，在一些实施例中，可以根据采集的图像确定哪个或哪些货物被取出。
高风险图像确定单元501能够在基于图像和基于重力探测确定的货物变化情况 不一致的情况下缓存高风险图像，以便上报单元上报。
这样的售货机控制装置能够通过图像是否异常，以及图像与重力探测的变化情况判读结果是否一致这两种途径判断是否触发缓存高风险图像并上报，降低了事件漏提取的概率，提高了售货机贩卖货物的安全性。
在一些实施例中，售货机控制装置还可以包括开门控制单元505，能够在收到用户触发开门操作时，判断用户是否为限制用户。若确定用户为限制用户，则拒绝触发售货机开门；若用户为正常状态的用户，则执行开门操作。
这样的售货机控制装置能够利用用户信用体系确定是否向用户提供服务，进一步提高了安全性。
本公开售货机控制装置的一些实施例的结构示意图如图6所示。售货机控制装置包括存储器601和处理器602。其中：存储器601可以是磁盘、闪存或其它任何非易失性存储介质。存储器用于存储上文中售货机控制方法的对应实施例中的指令。处理器602耦接至存储器601，可以作为一个或多个集成电路来实施，例如微处理器或微控制器。该处理器602用于执行存储器中存储的指令，能够实现及时反馈图像，提高异常发现的效率，提高售货机的安全性。
在另一些实施例中，一种计算机可读存储介质，其上存储有计算机程序指令，该指令被处理器执行时实现售货机控制方法对应实施例中的方法的步骤。本领域内的技术人员应明白，本公开的实施例可提供为方法、装置、或计算机程序产品。因此，本公开可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本公开可采用在一个或多个其中包含有计算机可用程序代码的计算机可用非瞬时性存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本公开的售货机控制系统的一些实施例的示意图如图7所示。售货机控制装置71可以为上文中提到的任意一种售货机控制装置。售货机控制服务器70能够在收到高风险图像的情况下，以便判读高风险图像。在一些实施例中，为了提高风险评估的及时性，可以发出告警。
这样的售货机控制系统中，售货机控制装置能够在图像采集异常的情况下及时做出反应，将采集的图像上传给服务器供服务器解析，以便及时确定违规操作，从而实现通过图像的及时反馈，提高异常发现的效率，提高售货机的安全性；能够根据用户违规操作情况动态改变用户的权限，预防用户进一步违规操作，提高安全性。
在一些实施例中，若根据高风险图像判读后确定用户执行违规操作，且违规操作统计值达到预定门限，则修改用户权限为限制用户，从而使该用户再次使用售货机时，售货机拒绝为用户开门，从而实现用户信用的统计和有效应用，提高售货机售货的安全性。
在一些实施例中，若售货机控制服务器70根据高风险图像判读后确定用户未执行违规操作，则可以利用高风险图像更新图像分析训练数据集，根据更新后的图像分析训练数据集更新图像特征和图像分析模型，并发送给售货机，售货机控制装置更新本地的图像特征和图像分析模型，实现自身图像分析功能的优化。
这样的售货机控制系统能够动态更新图像分析的数据基础，图像提取特征和模型，提高售货机图像分析的准确性，降低错误判断的概率。
本公开的售货机80的一些实施例的示意图如图8所示。储物装置81能够承载售卖的货物；重力探测装置82能够采集货架上的重量变化情况；图像采集装置83能够采集售货机内部，以及用户操作时所处区域的图像。在一些实施例中，重力探测装置和图像采集装置可以包括多个。售货机控制装置84可以为上文中提到的任意一种售货机控制装置。
这样的售货机能够在图像采集异常的情况下及时做出反应，将采集的图像上传给服务器供服务器解析，从而实现图像的及时反馈，提高异常发现的效率，提高售货机的安全性。
最后应当说明的是：以上实施例仅用以说明本公开的技术方案而非对其限制；尽管参照较佳实施例对本公开进行了详细的说明，所属领域的普通技术人员应当理解：依然可以对本公开的具体实施方式进行修改或者对部分技术特征进行等同替换；而不脱离本公开技术方案的精神，其均应涵盖在本公开请求保护的技术方案范围当中。

Claims (17)

  1. 一种售货机控制方法，包括：
    在售货机开门状态下：
    采集售货机有效范围内的图像；
    对采集的图像进行风险分析；
    若确定采集的图像异常，则确定采集的图像为高风险图像；
    将高风险图像上报售货机控制服务器，以便识别用户风险行为。
  2. 根据权利要求1所述的方法，其中，所述将高风险图像上报售货机控制服务器包括：
    对所述异常图像进行筛选，挑选出部分异常图像；
    将挑选出的异常图像上报所述售货机控制服务器。
  3. 根据权利要求1或2所述的方法，其中，所述图像异常包括以下一项或多项：
    根据图像分析确定一个或多个图像采集设备被遮挡；
    根据图像分析确定售货机内出现了异常物体；
    用户对货物做出异常操作，所述异常操作包括破坏货物外包装、向货物内部加入物质、从货物中取物质或用户持货物离开摄像头探测区域后又返回中的一项或多项；
    或，
    货物的尺寸、颜色或透明度发生改变。
  4. 根据权利要求1或2所述的方法，其中，所述将所述高风险图像上报售货机控制服务器包括：
    在售货机关门后或保持开门状态的时长超过预定第一时长的情况下，判断是否存在所述高风险图像；
    若存在所述高风险图像，则上报所述高风险图像。
  5. 根据权利要求1所述的方法，还包括：
    基于采集的图像确定货物变化情况；
    基于重力探测确定货物变化情况；
    在基于图像和基于重力探测确定的货物变化情况不一致的情况下，执行确定采集的图像为高风险图像的操作。
  6. 根据权利要求1所述的方法，还包括：
    在根据图像或重力探测确定货物被取出售货机的情况下，记录被取出的货物的唯一标识，其中，所述售货机中的每个货物具备唯一标识；
    在根据图像或重力探测确定货物被放回售货机的情况下，确定放回的货物的标识与取出的货物的标识是否匹配；
    若不匹配，则执行确定采集的图像为高风险图像的操作；
    若匹配，则删除记录的所述货物的标识。
  7. 根据权利要求1所述的方法，还包括：
    接收来自用户的开门请求；
    若用户账户为限制用户，则拒绝触发售货机开门。
  8. 根据权利要求1所述的方法，其中，对采集的图像进行风险分析包括：
    基于售货机本地的图像特征和图像分析模型对采集的图像进行风险分析。
  9. 根据权利要求1或7所述的方法，还包括：
    售货机控制服务器在收到所述高风险图像的情况下，识别所述高风险图像；
    若通过识别所述高风险图像确定用户执行违规操作，则修改用户违规操作统计值；
    在所述用户的违规操作统计值达到预定门限的情况下，将用户标记为限制用户。
  10. 根据权利要求8所述的方法，还包括：
    若所述售货机控制服务器根据所述高风险图像确定用户未执行违规操作，则利用所述高风险图像更新图像分析训练数据集；
    根据更新后的图像分析训练数据集更新图像特征和图像分析模型，并发送给售货机以便更新所述售货机的图像特征和图像分析模型。
  11. 一种售货机控制装置，包括：
    高风险图像确定单元，被配置为在售货机开门状态下对采集的售货机有效范围内的图像进行风险分析，若确定采集的图像异常，则确定采集的图像为高风险图像；
    上报单元，被配置为将高风险图像上报售货机控制服务器，以便识别用户风险行为。
  12. 一种售货机控制装置，包括：
    存储器；以及
    耦接至所述存储器的处理器，所述处理器被配置为基于存储在所述存储器的指令执行如权利要求1至8任一项所述的方法。
  13. 一种计算机可读存储介质，其上存储有计算机程序指令，该指令被处理器执行时实现权利要求1至8任意一项所述的方法的步骤。
  14. 一种售货机控制系统，包括：
    权利要求11或12所述的售货机控制装置；和
    售货机控制服务器，被配置为：在收到所述高风险图像的情况下，识别所述高风险图像。
  15. 根据权利要求14所述的系统，其中，
    所述售货机被配置为基于售货机本地的图像特征和图像分析模型对采集的图像进行风险分析；
    所述售货机控制服务器还被配置为：
    若根据所述高风险图像确定用户未执行违规操作，则利用所述高风险图像更新图像分析训练数据集；
    根据更新后的图像分析训练数据集更新图像特征和图像分析模型，并发送给售货机。
  16. 根据权利要求14或15所述的系统，其中，
    所述售货机控制服务器还被配置为：
    若通过识别所述高风险图像确定用户执行违规操作，则修改用户违规操作统计值；
    在所述用户的违规操作统计值达到预定门限的情况下，将用户标记为限制用户。
  17. 一种售货机，包括：
    储物装置，被配置为承载货物；
    图像采集装置，被配置为采集有效范围内图像；
    重力探测装置，被配置为探测所述储物装置承载的货物重量变化情况；和，
    权利要求11或12所述的售货机控制装置。
PCT/CN2019/099758 2018-12-29 2019-08-08 售货机控制方法、装置、系统和售货机 Ceased WO2020134100A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811630378.4A CN109711359B (zh) 2018-12-29 2018-12-29 售货机控制方法、装置和计算机可读存储介质
CN201811630378.4 2018-12-29

Publications (1)

Publication Number Publication Date
WO2020134100A1 true WO2020134100A1 (zh) 2020-07-02

Family

ID=66259311

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/099758 Ceased WO2020134100A1 (zh) 2018-12-29 2019-08-08 售货机控制方法、装置、系统和售货机

Country Status (2)

Country Link
CN (1) CN109711359B (zh)
WO (1) WO2020134100A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118447445A (zh) * 2024-05-13 2024-08-06 广东空港一号供应链科技有限公司 一种基于物联网技术的货物防盗方法及系统
CN120472369A (zh) * 2025-05-07 2025-08-12 河北盛马电子科技有限公司 一种智能视频识别方法及装置、设备、存储介质

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109711359B (zh) * 2018-12-29 2021-08-03 北京沃东天骏信息技术有限公司 售货机控制方法、装置和计算机可读存储介质
CN111080896A (zh) * 2019-11-30 2020-04-28 嘉兴聚变信息科技有限公司 一种自助售货机的防撬防盗方法
CN113837400B (zh) * 2020-06-08 2024-04-16 菜鸟智能物流控股有限公司 物流对象投递装置故障处理和装置
CN114419781A (zh) * 2022-01-04 2022-04-29 合肥美的智能科技有限公司 摄像设备被遮挡提醒方法、售货终端、存储介质及装置
CN116308675A (zh) * 2023-03-30 2023-06-23 北京沃东天骏信息技术有限公司 集成检测模型的确定方法及装置
CN116503994B (zh) * 2023-06-29 2023-09-26 深圳市乐科智控科技有限公司 一种商品防盗检测方法、装置、设备及存储介质
CN119672856A (zh) * 2024-11-28 2025-03-21 神楚科技(湖北)有限公司 用于智能无人售卖柜的交互行为识别方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101482063B1 (ko) * 2014-05-09 2015-01-13 주식회사 로탬 자동판매기의 판매정보 관리장치 및 방법
CN107680261A (zh) * 2017-09-25 2018-02-09 福州米鱼信息科技有限公司 一种自动售卖系统
CN108648334A (zh) * 2018-04-11 2018-10-12 合肥美的智能科技有限公司 无人售货柜及其异常上报控制方法、无人售货系统
CN109711359A (zh) * 2018-12-29 2019-05-03 北京沃东天骏信息技术有限公司 售货机控制方法、装置和计算机可读存储介质
CN109711360A (zh) * 2018-12-29 2019-05-03 北京沃东天骏信息技术有限公司 售货机风险控制方法、装置和控制系统

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106530529A (zh) * 2016-11-11 2017-03-22 贤石金 智能收银机
CN106781014B (zh) * 2017-01-24 2018-05-18 广州市蚁道互联网有限公司 自动售货机及其运行方法
CN106920152A (zh) * 2017-03-07 2017-07-04 深圳市楼通宝实业有限公司 自助售卖方法及系统
CN106934890A (zh) * 2017-03-09 2017-07-07 石道松 自助售货门店
CN107481414A (zh) * 2017-08-21 2017-12-15 文彬 一种开放式无人售卖装置的实时导购方法及系统
CN107622436A (zh) * 2017-09-06 2018-01-23 上海金荣翔企业发展有限公司 一种无人商店的售货系统及售货方法
CN107730782A (zh) * 2017-10-19 2018-02-23 安徽小豆网络科技有限公司 基于视觉图像识别的自助收银机的无人便利店运营系统
CN107742365A (zh) * 2017-12-01 2018-02-27 深圳市迪比科电子科技有限公司 无人智能售卖方法以及无人智能售卖系统
CN108171172A (zh) * 2017-12-27 2018-06-15 惠州Tcl家电集团有限公司 自助购物方法、自助售卖机及计算机可读存储介质
CN108154601A (zh) * 2018-01-09 2018-06-12 合肥美的智能科技有限公司 自动售货机及其控制方法
CN108335406A (zh) * 2018-02-08 2018-07-27 合肥美的智能科技有限公司 无人售货设备及其售货方法和无人售货系统
CN108492451A (zh) * 2018-03-12 2018-09-04 远瞳（上海）智能技术有限公司 自动售货方法
CN108416909B (zh) * 2018-04-23 2024-05-21 福建自贸试验区厦门片区益客联信息科技有限公司 一种智能售货柜及智能售货方法
CN108734862A (zh) * 2018-05-22 2018-11-02 孙海 组合式自动售货系统及其控制方法
CN108765702A (zh) * 2018-05-23 2018-11-06 济南每日优鲜便利购网络科技有限公司 自动售货机
CN108806074B (zh) * 2018-06-05 2021-08-03 腾讯科技（深圳）有限公司 购物信息的生成方法、装置及存储介质
CN108921048A (zh) * 2018-06-14 2018-11-30 深圳码隆科技有限公司 一种购物结算方法、装置和用户终端
CN108694779B (zh) * 2018-07-09 2020-04-03 深圳码隆科技有限公司 一种开放式自动售货机中商品的退货方法及其装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101482063B1 (ko) * 2014-05-09 2015-01-13 주식회사 로탬 자동판매기의 판매정보 관리장치 및 방법
CN107680261A (zh) * 2017-09-25 2018-02-09 福州米鱼信息科技有限公司 一种自动售卖系统
CN108648334A (zh) * 2018-04-11 2018-10-12 合肥美的智能科技有限公司 无人售货柜及其异常上报控制方法、无人售货系统
CN109711359A (zh) * 2018-12-29 2019-05-03 北京沃东天骏信息技术有限公司 售货机控制方法、装置和计算机可读存储介质
CN109711360A (zh) * 2018-12-29 2019-05-03 北京沃东天骏信息技术有限公司 售货机风险控制方法、装置和控制系统

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118447445A (zh) * 2024-05-13 2024-08-06 广东空港一号供应链科技有限公司 一种基于物联网技术的货物防盗方法及系统
CN120472369A (zh) * 2025-05-07 2025-08-12 河北盛马电子科技有限公司 一种智能视频识别方法及装置、设备、存储介质

Also Published As

Publication number Publication date
CN109711359B (zh) 2021-08-03
CN109711359A (zh) 2019-05-03

Similar Documents

Publication Publication Date Title
WO2020134100A1 (zh) 售货机控制方法、装置、系统和售货机
US12536859B2 (en) Investigation generation in an observation and surveillance system
JP7245321B2 (ja) スキャン漏れ識別方法および装置、セルフサービス・キャッシュ・レジスタ端末およびシステム
CN111263224B (zh) 视频处理方法、装置及电子设备
US9158975B2 (en) Video analytics for retail business process monitoring
CN104660979B (zh) 用于基于云的事件分析和模式检测的动态关联视图的系统和方法
US10460300B2 (en) Method of preventing fraud and theft during automated teller machine transactions and related system
US12340666B2 (en) Investigation generation in an observation and surveillance system
EP4075399A1 (en) Information processing system
US12333925B2 (en) Theft prediction and tracking system
JP7649934B2 (ja) 行動推定装置、行動推定方法、プログラム及び行動推定システム
JP6440327B2 (ja) 防犯システム、防犯方法、及びロボット
CN115546703B (zh) 自助收银的风险识别方法、装置、设备及存储介质
US20180157917A1 (en) Image auditing method and system
CN111260685B (zh) 视频处理方法、装置及电子设备
WO2018104999A1 (ja) 情報処理装置、制御方法、及びプログラム
US11928942B2 (en) Systems and methods for theft prevention and detection
CN112001349A (zh) 数据审核方法、系统及电子设备
JP6536643B2 (ja) 情報処理装置、制御方法、及びプログラム
CN116071784A (zh) 人员违规行为识别方法、装置、设备和存储介质
CN104052975B (zh) 一种带客流量计数的店铺联网视频报警器
CN108171286B (zh) 无人售货方法及其系统
CN109711360B (zh) 售货机风险控制方法、装置和控制系统
CA3095813A1 (en) Theft prediction and tracking system
CN111583521A (zh) 一种安全防护方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19904423

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19904423

Country of ref document: EP

Kind code of ref document: A1