WO2020020355A1 - 物品摆放状态获取方法和系统 - Google Patents

物品摆放状态获取方法和系统 Download PDF

Info

Publication number
WO2020020355A1
WO2020020355A1 PCT/CN2019/097936 CN2019097936W WO2020020355A1 WO 2020020355 A1 WO2020020355 A1 WO 2020020355A1 CN 2019097936 W CN2019097936 W CN 2019097936W WO 2020020355 A1 WO2020020355 A1 WO 2020020355A1
Authority
WO
WIPO (PCT)
Prior art keywords
hot zone
item
standard
image
current image
Prior art date
Application number
PCT/CN2019/097936
Other languages
English (en)
French (fr)
Inventor
马斌
范歆炜
周银华
李星毅
王彪
Original Assignee
北京京东尚科信息技术有限公司
北京京东世纪贸易有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京京东尚科信息技术有限公司, 北京京东世纪贸易有限公司 filed Critical 北京京东尚科信息技术有限公司
Priority to US17/263,847 priority Critical patent/US11861910B2/en
Publication of WO2020020355A1 publication Critical patent/WO2020020355A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/35—Categorising the entire scene, e.g. birthday party or wedding scene
    • G06V20/36—Indoor scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/18—Status alarms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30242—Counting objects in image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • the present disclosure relates to the field of electronic technology, and more particularly, to a method and system for acquiring an item placement state.
  • the inventors found that at least the following problems exist in the prior art: in the prior art, the tally work of retail stores is not supported by mature technology. The placement state of the items on the shelf is mainly obtained by manual inspection or by tally experience, so the placement state of the items on the shelf cannot be obtained in a timely manner. Therefore, the prior art not only has the defects of serious tally delay and poor timeliness, but also the defects of poor tally accuracy, low efficiency and high tally cost.
  • the present disclosure provides a method and system for acquiring an item placement state capable of improving tally efficiency, and a processing method and system.
  • An aspect of the present disclosure provides a method for acquiring an item placement state, including: sending an acquisition request, the acquisition request including one or more association relationships, each association relationship including an association relationship between a hot zone and a camera device , One hot zone corresponds to one kind of article; and receives the placement state of at least one hot zone corresponding article, the placement state of the item corresponding to the at least one hot zone is in response to the acquisition request, according to the current image of the at least one hot zone and Determined by the standard image, the camera device is used to obtain the current image of the hot zone with which the camera device is associated.
  • the above method further includes: setting the one or more association relationships; and / or displaying a placement state of items corresponding to one or more hot zones in at least one hot zone; and / or according to
  • the filtering conditions entered by the user are selected from the placed states of the articles corresponding to the at least one hot zone to obtain the placed states of the articles corresponding to one or more hot zones that match the filtering conditions, wherein the filtering conditions include at least one of the following : Camera equipment number, hot zone number, item name and / or item placement status.
  • the above method further includes: setting a period in which the imaging device acquires a current image of a hot zone having an associated relationship with it; and / or setting a period in which a placement state of an item corresponding to at least one hot zone is received, The period for receiving the placed state of the item corresponding to the at least one hot zone is not less than the period for the camera device to obtain the current image of the hot zone with an associated relationship with it.
  • the state of placing an item corresponding to one hot zone includes a normal state and an abnormal state.
  • the above-mentioned method for obtaining an article placing state further includes: corresponding to one or more hot zones in at least one hot zone.
  • an alarm signal is sent to the terminal device.
  • the acquiring of the current image of the hot zone having an association relationship with the camera device includes: acquiring multiple real-time images of the hot zone having an association relationship with the camera device, and the multiple real-time images are continuous at time intervals. Acquired; identifying multiple real-time images to obtain the number of items included in the multiple real-time images; and determining the current image of the hot zone that has an associated relationship with the camera device, the current image is the largest number of items included in the multiple real-time images A real-time image of the camera, where the camera device is associated with one or more hot zones.
  • Another aspect of the present disclosure provides a processing method, including: receiving an acquisition request, the acquisition request including one or more association relationships, each association relationship including an association relationship between a hot zone and a camera device, An area corresponds to an item; in response to an acquisition request, obtaining a current image and a standard image of at least one hot area; and determining a placement state of the item corresponding to the at least one hot area according to the current image and the standard image of the at least one hot area, wherein , The camera device is used to obtain the current image of the hot zone with which the camera device is associated.
  • determining a placement state of an item corresponding to the at least one hot zone according to the current image and the standard image of the at least one hot zone includes: identifying the current image of the at least one hot zone and obtaining the current image of the at least one hot zone Included image features; determining item features corresponding to at least one hot zone based on image features included in the current image of at least one hot zone; and comparing item features corresponding to at least one hot zone with standard article features corresponding to at least one hot zone to determine The placement state of the articles corresponding to the at least one hot zone, wherein the standard item characteristics corresponding to the at least one hot zone are obtained based on the standard image of the at least one hot zone.
  • a state of placing an article corresponding to a hot zone includes a normal state and an abnormal state, and the abnormal state includes at least one of the following situations: among the corresponding article characteristics, there are corresponding standard article characteristics In the case of an item name with a different standard item name in the corresponding item feature; the number of items corresponding to the item name in the corresponding item feature that is the same as the standard item name of the corresponding standard item feature, and the ratio of the number of standard items corresponding to the standard item name is less than the first The ratio of the items; or among the corresponding item characteristics, the item mark corresponding to the item name corresponding to the standard item name of the corresponding standard item feature, the item with the standard item name corresponding to the standard item name having a similarity lower than the preset similarity The number of marks, where the ratio of the total number of item marks corresponding to the item name is greater than the second ratio, where the item characteristics include at least one of the following: item name, item number corresponding to item name, item sign corresponding to
  • the acquiring of the current image of the hot zone having an association relationship with the camera device includes: acquiring multiple real-time images of the hot zone having an association relationship with the camera device, and the multiple real-time images are continuous at time intervals. Acquired; identifying multiple real-time images to obtain the number of items included in the multiple real-time images; and determining the current image of the hot zone that has an associated relationship with the camera device, the current image is the largest number of items included in the multiple real-time images A real-time image of the camera, where the camera device is associated with one or more hot zones.
  • the article placement status acquisition system can interact with a processing system, and includes an acquisition request sending module and a placement status receiving module.
  • the acquisition request sending module is used for sending.
  • An acquisition request the acquisition request includes one or more association relationships, each association relationship includes an association relationship between a hot zone and a camera device, and each hot zone corresponds to an item, and the state receiving module is configured to receive at least one The placement state of the items corresponding to the hot zone.
  • the placement state of the items corresponding to the at least one hot zone is determined by the processing system in response to the acquisition request according to the current image and the standard image of the at least one hot zone. The current image of the hot zone of the association.
  • the system further includes a parameter setting module, a display module, and / or a screening module.
  • the parameter setting module is used to set one or more association relationships
  • the display module is used to display one of the at least one hot zone.
  • the screening module is configured to filter one or more heats that match the screening conditions from the placement status of the items corresponding to at least one hot zone according to the filtering conditions input by the user.
  • the display state of the articles corresponding to the zone is for display, wherein the screening conditions include at least one of the following: a camera device number, a hot zone number, an article name, and / or an article placement state.
  • the parameter setting module is further configured to set a period for the imaging device to acquire a current image of a hot zone with an associated relationship with the imaging device; and / or, the placement state acquisition module is further configured to set to receive at least The period of the placed state of the article corresponding to one hot zone, wherein the period of receiving the placed state of the article corresponding to the at least one hot zone is not less than the period of the camera device acquiring the current image of the hot zone associated with it.
  • the state of placing the articles corresponding to the at least one hot zone includes a normal state and an abnormal state
  • the above-mentioned article placing state acquiring system further includes an alarm module for When the placing state of the articles corresponding to the multiple hot zones is abnormal, an alarm signal is sent to the terminal device.
  • the acquiring of the current image of the hot zone having an association relationship with the camera device includes: acquiring multiple real-time images of the hot zone having an association relationship with the camera device, and the multiple real-time images are continuous at time intervals. Acquired; identifying multiple real-time images to obtain the number of items included in the multiple real-time images; and determining the current image of the hot zone that has an associated relationship with the camera device, the current image is the largest number of items included in the multiple real-time images Real-time images, where the camera device is associated with one or more hot zones.
  • Another aspect of the present disclosure provides a processing system capable of interacting with an item placement status acquisition system, including an acquisition request receiving module, an image acquisition module, and a placement status determination module, and the acquisition request receiving module is configured to receive an item.
  • the acquisition request sent by the display status acquisition system the acquisition request includes one or more association relationships, each association relationship includes an association relationship between a hot zone and a camera device, and one hot zone corresponds to an item; an image acquisition module Responding to the acquisition request, acquiring the current image and the standard image of the at least one hot zone; the placement state determination module is configured to determine the placing state of the items corresponding to the at least one hot zone according to the current image and the standard image of the at least one hot zone ,
  • the camera device is configured to obtain a current image of a hot zone with which the camera device is associated.
  • the placement state determination module includes an image feature identification sub-module, an item feature determination sub-module, and a placement status determination sub-module.
  • the image feature identification sub-module is configured to identify a current image of at least one hot zone, and obtain Image features included in the current image of the at least one hot zone;
  • the item feature determination submodule is used to determine the item features corresponding to the at least one hot zone based on the image features included in the current image of the at least one hot zone;
  • the placement status determination submodule is used to Compare the item characteristics corresponding to at least one hot zone with the standard item characteristics corresponding to at least one hot zone to determine the placement state of the items corresponding to at least one hot zone, wherein the standard item characteristics corresponding to at least one hot zone are based on at least one hot zone Get the standard image.
  • the above-mentioned items are placed in a normal state and an abnormal state
  • the abnormal state includes at least one of the following situations: among the corresponding article characteristics, there is a standard article in the corresponding standard article characteristics Cases of item names with different names; the number of items corresponding to the item names in the corresponding item characteristics that are the same as the standard item names of the corresponding standard item characteristics; and the case where the ratio of the number of standard items corresponding to the standard item names is less than the first ratio; Or, among the corresponding item characteristics, the item mark corresponding to the item name corresponding to the standard item name of the corresponding standard item feature, the similarity of the standard item mark corresponding to the standard item name is lower than the preset number of item marks, In the case where the ratio of the total number of item marks corresponding to the item name is greater than the second ratio, the item characteristics include at least one of the following: item name, number of items corresponding to item name, item logo corresponding to item name, and / or item name The corresponding
  • the processing system is a cloud system; and / or the system further includes an image storage module for storing the current image of the hot zone and the at least one hot zone standard image acquired by the camera device and associated with the hot storage zone. For acquisition by the image acquisition module.
  • the acquiring of the current image of the hot zone having an association relationship with the camera device includes: acquiring multiple real-time images of the hot zone having an association relationship with the camera device, and the multiple real-time images are continuous at time intervals. Acquired; identifying the multiple real-time images to obtain the number of items included in the multiple real-time images; and determining the current image of the hot zone that is associated with the camera device, the current image is the largest number of items included in the multiple real-time images A real-time image of, wherein the camera device has an association relationship with one or more hot zones.
  • Another aspect of the present disclosure provides an electronic device including: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are used by the one or more programs When the two processors execute, the one or more processors are caused to execute the above-mentioned method for acquiring an item placement state.
  • Another aspect of the present disclosure provides an electronic device including: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are used by the one or more programs When the two processors execute, the one or more processors are caused to execute the foregoing processing method.
  • Another aspect of the present disclosure provides a non-volatile storage medium storing computer-executable instructions that, when executed, are used to implement the method for acquiring an item placement state as described above.
  • Another aspect of the present disclosure provides a non-volatile storage medium storing computer-executable instructions that, when executed, are used to implement a processing method as described above.
  • the embodiments of the present disclosure it is possible to at least partially avoid the defects of high tallying cost caused by manual inspection to obtain the item placement status, and the inability to obtain the item placement status due to the delay in tallying in real time. And therefore, the real-time automatic acquisition of the item placement status can be achieved to improve the efficiency of tally, improve the timeliness of tally and reduce the technical effect of tally labor cost.
  • FIG. 1 schematically illustrates an application scenario of a method and system for acquiring an item placement state according to an embodiment of the present disclosure
  • FIG. 2A-2B schematically illustrate a flowchart of a method for acquiring an item placement state according to an embodiment of the present disclosure
  • 3A-3B schematically illustrate a flowchart of a method for acquiring an article placement state according to another embodiment of the present disclosure
  • FIG. 4 schematically illustrates a display screen obtained according to the method for acquiring an item placement state described with reference to FIG. 3A;
  • FIG. 5 schematically illustrates a flowchart of acquiring a current image of a hot zone according to an embodiment of the present disclosure
  • FIG. 6 schematically illustrates an operation flowchart of a processing method according to an embodiment of the present disclosure
  • FIG. 7 schematically illustrates a flowchart of determining an article placement state according to an embodiment of the present disclosure
  • FIGS. 8A to 8C are schematic diagrams schematically showing that an article is placed in an abnormal state according to an embodiment of the present disclosure
  • FIG. 9A to FIG. 9E schematically illustrate a structure block diagram of an article placement state acquisition system according to an embodiment of the present disclosure
  • FIG. 10A-10C schematically illustrate a structural block diagram of a processing system according to an embodiment of the present disclosure
  • FIG. 11 schematically illustrates a block diagram of a computer system suitable for implementing an item placement state acquisition method or processing method according to an embodiment of the present disclosure.
  • FIG. 1 schematically illustrates an application scenario of a method and system for acquiring an article placement state according to an embodiment of the present disclosure. It should be noted that FIG. 1 is only an example of an application scenario to which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used for other applications.
  • Device, system, environment, or scenario
  • the application scenario 100 includes a terminal device 101, a cloud system 102, a camera device 103, and a shelf 104.
  • Media connection between the camera device 103 and the cloud system 102, between the cloud system 102 and the terminal device 101, or between the camera device 103 and the terminal device 101 through a communication link, and the connection may include various connection types, such as wired, wireless Communication link or fiber optic cable, etc.
  • the terminal device 101 can interact with the cloud system 102 to receive the item placement status and the like.
  • Various operation applications can be installed on the terminal device 101 to send an acquisition request to the cloud system 102; the terminal device 101 can also interact with the camera device 103 to adjust the shooting frequency of the camera device 103.
  • the terminal device 101 may be various electronic devices with a display screen to show the display state of the items to the user, including but not limited to a desktop computer, a laptop portable computer, a tablet charger, a smart phone, and the like.
  • the terminal device 101 may also obtain, through interaction with the cloud device 102, the current image 200 (including the current image of the hot zone) of the shelf captured by the camera device 103 stored in the cloud system 102 and the shelf.
  • the standard image (including the standard image of the hot zone) is displayed to better display the placement state of the items corresponding to each hot zone on the shelf 104.
  • one hot zone corresponds to an area where the same product is placed on the shelf 104.
  • the one hot zone may, for example, place "Coca-Cola" on the bottom of the shelf 104.
  • the area of the item It can be understood that the above description of the hot zone is merely an example to facilitate understanding of the present disclosure, and the present disclosure does not limit this.
  • the cloud system 102 may be a cloud virtual terminal device (such as a virtual personal computer) with storage and computing functions to store the current image 200 of the shelf 104 and the standard image of the shelf captured by the camera device 103 and respond to the acquisition of the terminal device 101
  • the request is to process and analyze the current image and the standard image to obtain the placement state of the articles in the one or more hot zones on the shelf 104 (only examples).
  • the cloud system 102 also has, for example, an image processing function, so as to mark the corresponding area of the hot zone on the current image of the shelf captured by the imaging device 103 according to the coordinate range of the area corresponding to the hot zone on the standard image.
  • the camera device 103 may communicate with the cloud system 102 through a set communication protocol, which may be a terminal device with a shooting function, which is used to capture a real-time image of the shelf 104 and send the captured current image 200 to the cloud system 102 to For cloud system 102 storage.
  • a set communication protocol which may be a terminal device with a shooting function, which is used to capture a real-time image of the shelf 104 and send the captured current image 200 to the cloud system 102 to For cloud system 102 storage.
  • the imaging device 103 may also be, for example, an imaging device 103 having a calculation processing function.
  • the imaging device 103 may continuously capture the rack 104 at intervals to obtain multiple real-time images, and then the imaging device performs calculation processing through the imaging device 103.
  • the function can perform recognition processing on the multiple real-time images to filter the best image from the multiple real-time images as the current image (including the current image of the shelf in the hot zone).
  • the camera device 103 may further have, for example, a function of image processing similar to a cloud system.
  • the camera device 103 may be, for example, a camera device such as a surveillance camera with a real-time shooting function or a regular shooting function, so as to acquire the image of the shelf 104 in real time or periodically, thereby obtaining the image of the shelf 104 in real time or periodically (including An image of one or more hot zones included in the shelf 104).
  • a camera device such as a surveillance camera with a real-time shooting function or a regular shooting function, so as to acquire the image of the shelf 104 in real time or periodically, thereby obtaining the image of the shelf 104 in real time or periodically (including An image of one or more hot zones included in the shelf 104).
  • the method for acquiring an item placement state provided by the embodiment of the present disclosure may generally be executed by the terminal device 101.
  • the article placement status acquisition system provided by the embodiment of the present disclosure may generally be provided in the terminal device 101.
  • the cloud system 102 may not be included, and the camera device 103 communicates with the terminal device 101.
  • the terminal device 101 also has storage and calculation functions to store the camera device 103 for shooting.
  • the current image 200 of the shelf 104 (including the current image of the hot zone) and the standard image of the shelf 104 (including the standard image of the hot zone). After analyzing and identifying the current image 200 and the standard image, one or more of the shelves 104 are included. The placement of items in each hot zone.
  • terminal device the number and type of the terminal device, cloud system, camera device, shelf, and items placed on the shelf in FIG. 1 are only schematic. According to implementation needs, there can be any number and type of terminal equipment, cloud systems, camera equipment, shelves and items placed.
  • FIG. 2A-2B schematically illustrate a flowchart of a method for acquiring an article placement state according to an embodiment of the present disclosure.
  • the method for acquiring an item placement state includes operations S210 to S220.
  • an acquisition request is sent, where the acquisition request includes one or more association relationships, each association relationship includes an association relationship between a hot zone and an imaging device, and one hot zone corresponds to an item.
  • association relationships may be preset, for example, as shown in FIG. 2B, the method may further include operation S230 to set one or more association relationships.
  • This operation S230 may be performed before operation S210, for example.
  • the hot zone may be, for example, an area where the same article is placed on the shelf 104
  • the imaging device may be, for example, an imaging device that acquires a current image of the hot zone that has an associated relationship with it.
  • the imaging device 103 may be located in the same space as the shelf 104 preset with a hot zone, for example, the imaging device 103 may be specifically disposed on a wall opposite to the shelf 104 and another opposite the shelf 104.
  • the shelf, or the top wall of the space where the shelf 104 is located, is used to capture a real-time image of the shelf 104.
  • the captured real-time image of the shelf 104 includes a real-time image of a hot zone of the shelf 104, and therefore also includes the shelf.
  • 104 has a current image of the hot zone.
  • the imaging device may obtain a current image of a hot zone that is associated with the terminal device 101 when the terminal device 101 responds to a user ’s operation and sends a shooting instruction to the imaging device (specifically, for example, the shelf where the hot zone is captured) Current image), or the camera device may also periodically obtain the current image of the hot zone associated with it according to a set period.
  • the period may be set by the terminal device 101, for example, and the item is placed.
  • the acquiring method may further include a corresponding operation: setting a period for the imaging device to acquire the current image of the hot zone having an associated relationship with the imaging device, where the set acquiring period may be, for example, 5 min.
  • the acquisition request may be sent, for example, by the terminal device 101 in response to a user operation, or may be sent according to a preset period so as to periodically receive at least one hot zone corresponding
  • the method for obtaining the item placement state may further include a corresponding operation: setting a period for receiving the placement state of the item corresponding to at least one hot zone, where the set receiving period may be, for example, 5min same as the acquisition cycle.
  • a period for receiving a placement state of an item corresponding to at least one hot zone is not less than a period for an imaging device to acquire a current image of a hot zone with an associated relationship with it, thereby ensuring that the article placement state acquisition method can be ensured
  • the placed state of the articles corresponding to the received hot zone is updated in real time according to the state of the articles placed in the hot zone.
  • the setting of one or more association relationships may be set by the user through the terminal device 101, and a process of setting the association relationship between the hot zone and the camera device may be specifically performed by the terminal device 101, for example.
  • the process of binding camera device information and hot zone information may include, for example, an imaging device number, model, and / or manufacturer
  • the hot zone information may include, for example, a hot zone number and item information (such as an article name, an article number, and the like) placed in the hot zone.
  • the association relationship between the set hot zone and the camera device may specifically be, for example, that the camera device, the shelf, and the hot zone are sequentially registered through the terminal device 101, where:
  • the process of registering the camera device may be, for example, a process of filling in the camera device information (such as the camera device number, model, and / or manufacturer, etc.);
  • the process of registering a shelf may be, for example, first filling out shelf information (for example, it may include a shelf number and a camera equipment number, and may also include the name of the store where the shelf is located, shelf length, shelf width, etc.), and then determine whether the cloud system or the terminal device 101 is local. There is a standard image of the shelf. If there is a standard image of the shelf, the registration of the shelf is completed. If there is no standard image of the shelf, then the placement of the items on the shelf is readjusted or the shooting angle of the camera device is readjusted. Confirm whether the standard image is based on the real-time image of the shelf re-captured by the camera device, and complete the registration of the shelf until the standard image of the shelf exists;
  • the registration process of the hot zone may be, for example: first filling in the shelf information, and then judging whether the shelf corresponding to the shelf information is registered. If the registration of the shelf is completed, the standard image of the shelf is read and the hot zone is marked on the standard image. Correspond to the area range and fill in the hot area information (for example, it can include the hot area number, the name and / or number of the item placed in the hot area, user information (such as the name of the person in charge of the hot area), etc.); Registration, return to the shelf registration process, and perform the hot zone registration process again.
  • the binding of the camera device, the shelf, and the hot zone can be achieved, thereby setting the association relationship between the hot zone and the camera device, and it can be understood that the above setting temperature
  • the method of associating a region with an imaging device is merely an example to facilitate understanding of the present disclosure, and the present disclosure does not limit this.
  • the standard image of the shelf may be, for example, an image that meets requirements taken by a camera at a specific time, and the standard image may be, for example, that the shelf is just filled with items, and the placement of the items meets the standard The conditions (the placement of the article is for the imaging device at an angle where the article mark is fully exposed).
  • the standard image of the shelf may also be, for example, that the terminal device 101 acquires an image of the shelf taken from the cloud system 102 / or the camera device 103 in multiple time periods and displays it to the user.
  • the selected images are uploaded to the cloud system 102; for example, the standard images of the shelves can also be obtained by screening the images of the shelves in multiple time periods according to the preset standards by the cloud system 102.
  • the one or more association relationships included in the acquisition request and the current image of the hot zone with which the acquisition device 103 has an association relationship can be obtained to obtain the placement state of the articles corresponding to the at least one hot zone. provide conditions.
  • the placement state of the articles corresponding to the at least one hot zone is received.
  • the placement state of the articles corresponding to the at least one hot zone is determined in response to the acquisition request according to the current image and the standard image of the at least one hot zone.
  • the above-mentioned standard image of the hot zone may be, for example, an image of a region corresponding to the hot zone in the standard image of the shelf, and the items placed in the hot zone in the standard image of the hot zone are in compliance with the standard conditions: It is full and the item identification is fully exposed to the camera.
  • the current image of the hot zone may be, for example, a corresponding area of the hot zone on the current image of the shelf, and the placement state of the item may be specifically determined based on the current image of the shelf and the standard image of the shelf, for example.
  • the current image of the shelf may be obtained by filtering from real-time images of multiple shelves.
  • the current image of the hot zone may also be specifically obtained by: according to the coordinate range of the corresponding zone of the hot zone marked on the standard image of the shelf 104 when registering in the hot zone, the current image on the shelf 104 The image of the area corresponding to the coordinate range is divided in the middle, then the divided image is the current image of the hot zone, and the placement state of the item may be, for example, the current image of the hot zone obtained by the division. And the standard image of the hot zone is determined. It can be understood that the current image of the hot zone can be obtained, for example, through the camera device 103 having an image processing function, or through the cloud system 102, and also through the terminal device 101, which is not limited in this disclosure. According to an embodiment of the present disclosure, the current image of the hot zone may also be obtained by filtering from multiple real-time images, for example.
  • the placed state of the articles corresponding to the received hot zone may include, for example, a normal state and an abnormal state.
  • determining the placement state of the item according to the current image and the standard image of at least one hot zone may include, for example, first identifying the current image and the standard image of the hot zone, and obtaining the current image characteristics included in the hot zone. And standard image features, and then according to the current image features and standard image features, the current corresponding item features and standard item features of the hot zone are obtained, and the item characteristics are determined by comparing the currently corresponding item features and standard item features of the hot zone.
  • the placed state where the normal state may be, for example, a situation in which the currently corresponding item characteristics of the hot zone satisfy a plurality of preset conditions compared with the standard article characteristics, and the abnormal state may be, for example, the current zone corresponding to the hot zone.
  • the feature of the item does not satisfy any one of the preset multiple conditions compared with the feature of the standard item.
  • the article characteristics and standard article characteristics corresponding to the hot zone may also be obtained, for example, in the following manner: a standard image feature obtained from a standard image of the hot zone and an image feature obtained from a current image of the hot zone
  • the item characteristics and standard item characteristics corresponding to the hot zone are obtained through a pre-trained model.
  • the item characteristics corresponding to the hot zone may include, for example, the item name, the number of items corresponding to the item name, the item logo corresponding to the item name, and / or Item position (ie, the coordinate position in the current image of the hot zone).
  • the standard item characteristics may include, for example, the standard item name, the number of items corresponding to the standard item name, the item logo corresponding to the standard item name, and / or the standard item position. (That is, the coordinate position in the standard image of the hot zone).
  • the feature of the article corresponding to the hot zone can also be obtained, for example, in the following manner: the image feature obtained from the current image of the shelf to which the hot zone belongs and the coordinates of the hot zone on the standard image of the shelf to which it belongs The range is used as an input, and the feature of the item in the coordinate range on the real-time image of the shelf obtained through the pre-trained model is the feature of the item corresponding to the hot zone.
  • the characteristics of the standard article corresponding to the hot zone can be obtained, for example, by: marking one or more zones corresponding to the hot zone on the standard image of the shelf to which the hot zone belongs in advance, and using the labeled standard image As input, the characteristics of the item corresponding to one or more hot zones on the standard image of the shelf are obtained through a pre-trained model.
  • the method for acquiring an item state may further include, for example, operation S240 to display a display state of items corresponding to one or more hot regions in at least one hot region.
  • the terminal device 101 may have a display screen, for example, and the method for acquiring the item placement status may be executed by the terminal device 101.
  • the display of the terminal device 101 is used to display the placement of items corresponding to one or more hot zones. Status so that users can know the status of items placed in each hot zone on the shelf in real time through the display content.
  • the display screen of the terminal device 101 may further display, for example, at least one of the following: the store name of the store where the shelf where the hot zone is located, the shelf number of the shelf to which the hot zone belongs, and the hot zone number , Item name, item number, and standard image of the hot zone, real-time image of the hot zone, and user information (such as the name of the person in charge of the hot zone).
  • the placement status of an item corresponding to a hot zone can be received, so as to obtain the placement status of the item periodically or in real time for users. Determine whether tally is required, as this method does not require users to visit the shelf area where the hot zone is located, which can effectively improve the efficiency of tally, avoid the serious delays and poor timeliness of the tally, and reduce the cost of tally; Furthermore, because the placement state of the item is obtained based on the standard image of the hot zone and the current image, and does not require the user to judge through tally experience, the accuracy of tally can be improved to a certain extent.
  • FIG. 3A to 3B schematically illustrate a flowchart of a method for acquiring an article placement state according to another embodiment of the present disclosure
  • FIG. 4 schematically illustrates a display obtained according to the method for acquiring an article placement state described with reference to FIG. 3A. Screen.
  • the method for acquiring an item placement state may include operations S350 and S360 in addition to operations S210 to S220 described with reference to FIG. 2A.
  • operation S350 may be performed, for example, after operation S220, or may be performed when a user inputs a filtering condition.
  • operation S350 according to the filtering conditions input by the user, the placement status of the articles corresponding to the one or more hot zones that matches the screening conditions is obtained from the placement status of the articles corresponding to the at least one hot zone for display; then Correspondingly, in operation S360, the display state of the items corresponding to the one or more hot zones obtained by the screening and matching the screening conditions is displayed.
  • a terminal device with a display screen also displays a filter condition input window for a user to input a filter condition
  • the filter condition may include at least one of the following: a camera device number, a hot zone number, an item name, and / Or the state of the item.
  • the operation obtained by filtering in S350 is the placement state of the articles corresponding to all the hot zones that have an associated relationship with the camera device corresponding to the camera device number; when the user enters the article name , And the result obtained by the operation S350 screening is the placement state of all the articles in the hot zone where the articles corresponding to the article names are located.
  • the placement status of the items obtained at various time periods may be stored in a local storage space (for example, referring to the terminal device shown in FIG. 1), and the filtering conditions input by the user may further include, for example, Time period, obtained by filtering in operation S350, is the placement status of all items stored in the local storage space during the time period; if the association relationship between the hot zone and the camera device is set through the above registration method, the filtering conditions entered by the user For example, it can also include the store name, shelf number, and so on. What is obtained by filtering in operation S350 is the storage state of the items stored in the local storage space and corresponding to the hot zone included in the store / shelf corresponding to the store name / shelf number.
  • the method for acquiring the placement state of the article may, for example, periodically clean the placement state stored in the local storage space, so that the local storage space stores only the placement state of the most recent period, which reduces the Load of local storage space.
  • the user information and the multiple hot zones and the multiple hot zones can also be set.
  • the filtering conditions entered by the user may also include the user information, for example, the filtering conditions obtained in operation S350 are items corresponding to multiple hot areas that the user corresponding to the user information is responsible for.
  • Placement status where user information can include, for example, the name of the person in charge, the position of the person in charge, and / or the number of the person in charge.
  • the input screening conditions are merely examples to facilitate understanding of the present disclosure, and this disclosure is not limited thereto.
  • the input screening conditions may also be, for example, two or more of the above listed screening conditions, such as the input screening conditions.
  • the display page obtained by the method for acquiring an item placement state described with reference to FIG. 3A is shown in FIG. 4.
  • the user can enter the filtering conditions through the display page of FIG. 4.
  • the display interface displays There are options for filtering conditions, and there are also display results of the placement status of the items obtained by filtering after the user clicks the "query" control after entering the filtering conditions.
  • the placing state of the items may include, for example, a "normal state", a "out of stock state", an "irregular state", and a "misplaced state". The specific meanings of these states will be described in the subsequent content.
  • a user can input filtering conditions according to requirements to flexibly retrieve the placement state of the article of interest, thereby making the displayed result more in line with the needs of the user and improving the user. See how efficiently items are placed.
  • the method for acquiring an item placement status may include operation S370, for example, operation S370, which may be performed after operation S220, in addition to operations S210 to S220 described with reference to FIG. 2A.
  • the terminal device herein may be, for example, the terminal device 101 described with reference to FIG. 1, or may be a portable electronic device such as a smart phone or a tablet computer, and the alarm information may be, for example, a push message to the terminal device 101. , Sending an instruction to play specific music or ringtone to the terminal device, or sending an instruction to pop up a specific window to the terminal device, so that the holder of the terminal device can know that there is a situation in which the item is placed in an abnormal state , And carry out the corresponding tally operation according to the abnormal state.
  • the user can know in time that the item placement does not meet the requirements, or that the item is out of stock, so that the tally operation can be performed in a timely manner, thus avoiding the existing technology.
  • the shortcomings of poor tally timeliness result in poor customer shopping experience and reduced store sales.
  • FIG. 5 schematically illustrates a flowchart of acquiring a current image of a hot zone according to an embodiment of the present disclosure.
  • the acquisition of the current image of the hot zone having an association relationship with the imaging device includes operations S510 to S530.
  • a plurality of real-time images of a hot zone having an association relationship with the camera device are acquired, and the plurality of real-time images are continuously acquired at time intervals.
  • the operation may specifically be, for example, that the camera device continuously acquires multiple real-time images of a shelf in which a hot zone is associated with the camera device at a set time interval, and uses the multiple real-time images of the shelf as the hot zone.
  • the subsequent operations S520 to S530 are performed on the multiple real-time images.
  • the operation may specifically be, for example, after acquiring multiple real-time images of the shelf, and then according to the corresponding coordinate range in the standard image of the shelf where the hot zone has an association relationship with the camera device, from the shelf An image corresponding to the coordinate range is obtained from each of the multiple real-time images of, that is, multiple real-time images of the hot zone associated with the camera device are obtained, and the multiple real-time images obtained from the multiple real-time images on the shelf are obtained. Based on the multiple real-time images of the hot zone, subsequent operations S520 to S530 are performed.
  • the value of the time interval may be any value from 1 to 30 seconds, for example, and the time interval may be set by referring to the terminal device 101 in FIG. 1, for example.
  • the multiple real-time images of the identified hot zone may be detected by, for example, an object detection algorithm to obtain the number of items included in each of the multiple real-time images.
  • an article included in the multiple real-time images may be detected by a Faster-RCNN algorithm.
  • a current image of the hot zone having an associated relationship with the camera is determined, and the current image is a real-time image with the largest number of items included in the plurality of real-time images.
  • the above-mentioned acquisition operation of the current image may be performed by, for example, an imaging device with a data processing function, and then the output of the imaging device is directly the current image of the hot zone with an associated relationship with it.
  • the above-mentioned current image obtaining operation may also be performed by, for example, the cloud system 102 or the terminal device 101 described with reference to FIG. 1, where multiple real-time images on the shelf Obtained by a camera device, the camera device interacts with the cloud system 102 or the terminal device 101 to transmit multiple real-time images of the racks taken by the camera device to the cloud system 102 or the terminal device 101, thereby reducing the item placement state to a certain extent Get the execution cost of the method.
  • An imaging device may have an association relationship with one or more hot zones, and specifically, for example, may establish an association relationship with all preset hot zones captured by the camera.
  • the embodiments of the present disclosure acquire multiple real-time images continuously captured by the camera device and select from the multiple real-time images The real-time image with the largest number of items included is used as the current image to determine the item placement status, which can improve the accuracy of the obtained item placement status to a certain extent.
  • FIG. 6 schematically illustrates an operation flowchart of a processing method according to an embodiment of the present disclosure.
  • the processing method includes operations S610 to S630.
  • an acquisition request is received, where the acquisition request includes one or more association relationships, each of which includes an association relationship between a hot zone and a camera device, and one hot zone corresponds to an item.
  • the imaging device is configured to acquire a current image of a hot zone having an associated relationship with the imaging device.
  • the current image of the hot zone here may specifically be the current image of the shelf where the hot zone is located, and the current image may be obtained by filtering from multiple real-time images; or the current of the hot zone here The image may specifically be obtained by dividing the current image of the shelf where the hot zone is located.
  • the acquisition request may be, for example, an acquisition request sent with reference to operation S210 in FIG. 2A, and details are not described herein again.
  • the current image and the standard image may be determined, for example, from real-time images of a shelf captured by the camera device at various time points.
  • the current image of the hot zone can be obtained, for example, by using the current image acquisition method of the hot zone described with reference to FIG. 5, and the standard image of the hot zone can be, for example, a real-time image that meets the standard conditions obtained by filtering.
  • the standard conditions can be Yes, the number of items placed in the hot zone is in a saturated (full) state, and the items are placed in such a manner that the item's logo is fully exposed relative to the camera device.
  • the current image and the standard image of the at least one hot zone may be stored in a cloud storage space that interacts with a camera device, and the cloud storage space may store, for example, all images taken by a plurality of camera devices.
  • the operation S620 is to obtain the current image and the standard image of the at least one hot zone from the cloud storage space.
  • the cloud storage space may, for example, periodically clean up the current image of the at least one hot zone that it stores, so that the cloud storage space stores only the recent certain period of time (for example, within a week). The current image of at least one hot zone.
  • a placement state of an item corresponding to the at least one hot zone is determined.
  • the operation S630 may be implemented by, for example, an image recognition and comparison algorithm, and specifically, by identifying and comparing the current image and the standard image of the at least one hot zone, and combining with the logic for judging the state of the article, to obtain The state of the articles corresponding to the at least one hot zone.
  • image recognition is mainly implemented by, for example, using an object detection algorithm (specifically, a Faster-RCNN algorithm) to obtain item features included in the current image of the at least one hot zone, and the item features may include at least one of the following 1: Item name, the number of items corresponding to the item name, the item logo corresponding to the item name, and the position of the item corresponding to the item name.
  • object detection algorithm specifically, a Faster-RCNN algorithm
  • the standard image of the at least one hot zone can be obtained by identifying the standard article characteristic through the image recognition in advance, and the standard article characteristic can include at least one of the following: the standard article name, the number of standard articles corresponding to the standard article name, the standard article The standard article mark corresponding to the name and the standard article position corresponding to the standard article name, etc.
  • the determination logic may specifically determine, for example, in the current image of at least one hot zone obtained through image recognition: whether the item name is the same as the standard item name, and the item corresponding to the item name is the same as the standard item name. Whether the ratio of the quantity of the standard article quantity corresponding to the standard article name is greater than the first ratio, in the same article name as the standard article name, the similarity between the corresponding article mark and the standard article mark corresponding to the standard article name is less than the preset similarity Whether the ratio of the number of item marks, the total number of item marks corresponding to the item name is less than the second ratio, and so on. It can be understood that the above judgment logic is merely an example to facilitate understanding of the present disclosure, and the present disclosure does not limit this.
  • FIG. 7 schematically illustrates a flowchart of determining an article placement state according to an embodiment of the present disclosure.
  • the operation S630 for determining the item placement state may specifically include operations S631 to S633.
  • operation S631 the current image of the at least one hot zone is identified, and image features included in the current image of the at least one hot zone are obtained.
  • operation S632 at least one heat is determined based on the image features included in the current image of the at least one hot zone.
  • Article characteristics corresponding to the zone in operation S633, comparing the article characteristics corresponding to the at least one hot zone with the standard article characteristics corresponding to the at least one hot zone to determine the placement state of the articles corresponding to the at least one hot zone.
  • the operations S631 to S632 may be specifically performed using an image recognition algorithm (for example, Faster-RCNN algorithm), or the image recognition algorithm is used as a basic algorithm, and a large number of images including different items are used as training data.
  • the deep learning model trained by using the real-time image of the shelf as test data is executed.
  • the input of the model may be, for example, the current image of the shelf obtained by dividing the current image of the shelf by the operation described with reference to FIG. 5, or by referring to the figure.
  • the output of this model is the feature of the article corresponding to the one hot zone obtained in operation S632, or the feature of the article corresponding to one or more hot zones included in the shelf.
  • the item position corresponding to the item name in the feature can group the item features into hot zones.
  • the above operation S633 may specifically obtain, for example, the placement state of the articles corresponding to the at least one hot zone through the foregoing judgment logic. I will not repeat them here.
  • 8A to 8C are schematic diagrams schematically showing that an article is placed in an abnormal state according to an embodiment of the present disclosure.
  • the state of placing the articles corresponding to the hot zone determined by referring to FIG. 6 to FIG. 7 includes a normal state and an abnormal state.
  • the abnormal state may include at least one of the following situations, for example:
  • the article name is different from the standard article name in the corresponding standard article feature.
  • the standard article name corresponding to a hot zone is, for example, “Coca-Cola”.
  • the item names corresponding to the hot zone obtained by referring to operation S632 in FIG. 7 include "Coca-Cola” and “Wang Lo Kat”.
  • "Wang Lao Ji" is different from the standard item name "Coca-Cola”, so the items in the hot zone are placed in an abnormally misplaced state.
  • the above-mentioned standard image, real-time image, standard item information, and item information of the hot zone are merely examples to facilitate understanding of the present disclosure, which is not limited in the present disclosure.
  • the ratio of the number of standard articles corresponding to the standard article name is less than the first ratio.
  • the first ratio is, for example, 0.5.
  • the standard article name corresponding to a hot zone (corresponding to the area surrounded by a dashed box) is “Coca-Cola ",
  • the number of standard items corresponding to the standard item name is 6.
  • the real-time image (current image) of the shelf as shown in the right image of Fig. 8B the corresponding hot zone obtained by referring to operation S632 in Fig.
  • the corresponding article name corresponding to the standard article name of the corresponding standard article name corresponds to the article name corresponding to the article name
  • the standard article name corresponding to the standard article name has a degree of similarity lower than the preset similarity number of article marks, and In the case where the ratio of the total number of item marks corresponding to the item name is greater than the second ratio.
  • the preset similarity is set to 50%
  • the second ratio is 0.5.
  • the corresponding standard article name is "Coca-Cola”
  • the standard article name corresponding to the standard article name is all exposed "CocaCola” patterns arranged vertically on the surface of the tank of the article in the figure.
  • the number of items corresponding to the “Coca-Cola” with the same item name as the standard item in the hot zone is 6, and the corresponding item mark That is to say, it includes all the displayed patterns of "CocaCola”, and also includes only the patterns of "CocaCola”, and the pattern of the exposed part of "CocaCola” is similar to the pattern of "CocaCola” that all standard article marks are exposed.
  • the degree of exposure is only 30%, that is, the exposure ratio of the word "CocaCola” in the pattern of the exposed part "CocaCola” is less than 30%, and only part of it is exposed
  • Canonical status It can be understood that the standard image, real-time image, standard item information, item information, preset similarity value, and second ratio value of the above hot zone are only examples to facilitate understanding of the present disclosure, and this disclosure does not make any reference to this. It is limited that according to the embodiment of the present disclosure, if the ratio of the quantity of the foregoing items is not an integer, for example, the ratio may be compared with the second ratio after rounding the ratio.
  • the state of placing the articles corresponding to the hot zone is normal.
  • FIG. 9A to FIG. 9E schematically illustrate a structure block diagram of an article placement state acquisition system according to an embodiment of the present disclosure.
  • the article placement status acquisition system 900 can interact with a processing system (such as the cloud system 102 described with reference to FIG. 1), and includes an acquisition request sending module 910 and a placement status receiving module 920.
  • a processing system such as the cloud system 102 described with reference to FIG. 1
  • the acquisition request sending module 910 is configured to send an acquisition request.
  • the acquisition request includes one or more association relationships, and each association relationship includes an association relationship between a hot zone and a camera device, and one hot zone corresponds to an item.
  • the camera device is used to obtain a current image of a hot zone with which the camera device is associated.
  • the acquisition request sending module 810 may perform, for example, operation S210 described with reference to FIG. 2A, and details are not described herein again.
  • the placement status receiving module 920 is configured to receive the placement status of the articles corresponding to the at least one hot zone.
  • the placement status of the articles corresponding to the at least one hot zone is responded to the acquisition request by the processing system according to the current status of the at least one hot zone.
  • the image and standard image are determined.
  • the placed-state receiving module 920 may perform, for example, operation S220 described with reference to FIG. 2A, and details are not described herein again.
  • the item placement status acquisition system 900 may further include, for example, a parameter setting module 930, which is configured to set one or more associations included in the acquisition request. relationship.
  • the parameter setting module 930 may perform, for example, operation S230 described with reference to FIG. 2B, and details are not described herein again.
  • the parameter setting module 930 may also be used to set a period for which the imaging device acquires the current image of the hot zone with which the imaging device is associated, and / or the placement state receiving module 920 may also be used, for example, For setting a period for receiving a display state of an article corresponding to at least one hot zone, wherein a period for receiving a display state of an article corresponding to at least one hot zone is not less than The period of the current image.
  • the item placement status acquisition system 900 may further include, for example, a display module 940 for displaying items corresponding to one or more hot zones in at least one hot zone. Placement.
  • the display module 940 may perform, for example, operation S240 described with reference to FIG. 2B, and details are not described herein again.
  • the item placement status acquisition system 900 may further include, for example, a screening module 950, which is configured to correspond to the at least one hot zone according to a screening condition input by a user.
  • a screening module 950 In the display state of the items, the display state of the items corresponding to one or more hot zones matching the selected conditions is obtained for display.
  • the screening module 950 may be used to perform the operation S350 described with reference to FIG. 3A, and accordingly, the display module 940 may also be used to perform the operation S360 described with reference to FIG. 3A, which is not described herein again.
  • the state of placing the articles corresponding to the at least one hot zone includes a normal state and an abnormal state.
  • the article placing state acquiring system 900 may further include an alarm module 960, for example.
  • the alarm module 960 is configured to send an alarm signal to a terminal device in a case where an item corresponding to one or more of the at least one thermal zone is placed in an abnormal state.
  • the alarm module 960 may perform, for example, operation S370 described with reference to FIG. 3B, and details are not described herein again.
  • the acquiring of the current image of the hot zone having an association relationship with the camera device includes: acquiring multiple real-time images of the hot zone having an association relationship with the camera device, and the multiple real-time images are continuous at time intervals. Obtained; identifying multiple real-time images to obtain the number of items included in the multiple real-time images; and determining a current image of a hot zone having an associated relationship with the camera device, the current image being the number of items included in the multiple real-time images The most real-time images, wherein the camera device has an associated relationship with one or more hot zones.
  • the acquisition of the current image of the hot zone associated with the imaging device may be obtained, for example, through operations S510 to S530 described with reference to FIG. 5, and details are not described herein again.
  • the article placement status acquisition system 900 may be provided in the terminal device 101 described with reference to FIG. 1, for example.
  • At least some of the modules, sub-modules, units, sub-units, or at least a part of any of the functions according to the embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be implemented by being split into multiple modules.
  • any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), System-on-chip, system-on-substrate, system-on-package, application-specific integrated circuit (ASIC), or any other reasonable means of hardware or firmware that can integrate or package the circuit, or software, hardware, and firmware Any one of the implementation manners or an appropriate combination of any of them may be implemented.
  • FPGA field programmable gate array
  • PLA programmable logic array
  • ASIC application-specific integrated circuit
  • any one of the implementation manners or an appropriate combination of any of them may be implemented.
  • one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be at least partially implemented as computer program modules, and when the computer program modules are executed, corresponding functions may be performed.
  • any number of the acquisition request sending module 910, the placement status receiving module 920, the parameter setting module 930, the display module 940, the filtering module 950, and the alarm module 960 can be combined into one module, or any one of them Modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module.
  • At least one of the acquisition request sending module 910, the placement status receiving module 920, the parameter setting module 930, the display module 940, the screening module 950, and the alarm module 960 may be implemented at least partially as a hardware circuit , Such as Field Programmable Gate Array (FPGA), Programmable Logic Array (PLA), system on chip, system on substrate, system on package, application specific integrated circuit (ASIC), or any Other reasonable ways are implemented by hardware or firmware, or by any one of software, hardware, and firmware, or any appropriate combination thereof.
  • FPGA Field Programmable Gate Array
  • PLA Programmable Logic Array
  • ASIC application specific integrated circuit
  • At least one of the acquisition request sending module 910, the placement status receiving module 920, the parameter setting module 930, the display module 940, the screening module 950, and the alarm module 960 may be at least partially implemented as a computer program module. When a program module is run, it can perform the corresponding function.
  • FIG. 10A to 10C schematically illustrate a structural block diagram of a processing system according to an embodiment of the present disclosure.
  • the processing system 1000 can interact with, for example, the article placement status acquisition system described with reference to FIGS. 9A to 9E.
  • the processing system 1000 includes an acquisition request receiving module 1010, an image acquisition module 1020, and a placement status determination. Module 1030.
  • the acquisition request receiving module 1010 is configured to receive an acquisition request sent by an item placement status acquisition system.
  • the acquisition request includes one or more associations, and each association includes an association between a hot zone and a camera device.
  • One hot zone corresponds to one item, and the camera device is used to obtain a current image of the hot zone that has an associated relationship with it.
  • the acquisition request receiving module 1010 may be used to perform operation S610 described with reference to FIG. 6, which is not described herein again.
  • the image acquisition module 1020 is configured to acquire a current image and a standard image of at least one hot zone in response to the acquisition request. According to the embodiment of the present disclosure, the image acquisition module 1020 may be used, for example, to perform operation S620 described with reference to FIG. 6, and details are not described herein again.
  • acquiring the current image of a hot zone having an associated relationship with an imaging device includes acquiring multiple real-time images of a hot zone having an associated relationship with the imaging device, and the multiple real-time images are at time intervals. Continuously acquired; identifying the multiple real-time images to obtain the number of items included in the multiple real-time images; and determining a current image of a hot zone having an association relationship with the camera device, the current image being the multiple images.
  • the acquisition of the current image may be obtained by, for example, operations described with reference to FIG. 5, and details are not described herein again.
  • the placing state determining module 1030 is configured to determine a placing state of an item corresponding to the at least one hot zone according to the current image and the standard image of the at least one hot zone. According to the embodiment of the present disclosure, the placement state determination module 1030 may be used to perform operation S630 described with reference to FIG. 6, for example, and details are not described herein again.
  • the placement state determination module 1030 may include, for example, an image feature recognition sub-module 1031, an item feature determination sub-module 1032, and a placement state determination sub-module 1033.
  • the image feature identification sub-module 1031 is configured to identify the current image of the at least one hot zone, and obtain image features included in the current image of the at least one hot zone; the item feature determination sub-module 1032 is configured to be based on the at least one hot zone.
  • the image feature included in the current image determines the feature of the item corresponding to the at least one hot zone; the placement state determination submodule 1033 is used to compare the feature of the item corresponding to the at least one hot zone with a standard item corresponding to the at least one hot zone Characteristics to determine the placement state of the articles corresponding to the at least one hot zone, wherein the standard article characteristics corresponding to the at least one hot zone may use the above-mentioned image feature identification sub-module 1031 and article feature determination sub-module 1032, based on the Standard image of at least one hot zone.
  • the image feature recognition sub-module 1031, the item feature determination sub-module 1032, and the placement state determination sub-module 1033 may be respectively used to perform operations S631 to S633 described with reference to FIG. 7, which are not repeated here. .
  • the placed state of the article includes a normal state and an abnormal state
  • the abnormal state includes at least one of the following situations: among the corresponding article characteristics, there is a standard article among the corresponding standard article characteristics Cases of item names with different names; the number of items corresponding to the item names in the corresponding item characteristics that are the same as the standard item names of the corresponding standard item characteristics, and the ratio of the number of standard items corresponding to the standard item names is less than the first ratio Situation; or among the corresponding article characteristics, the number of article marks corresponding to the standard article mark corresponding to the standard article name is lower than the preset similarity among the article marks corresponding to the article name corresponding to the standard article name of the corresponding standard article feature If the ratio of the total number of item marks corresponding to the item name is greater than the second ratio, wherein the item characteristics include at least one of the following: item name, number of items corresponding to item name, item logo corresponding to item name, and / Or the item position corresponding to the item name.
  • the item characteristics include at least one of
  • the processing system may be, for example, the cloud system 102 described with reference to FIG. 1, and the cloud system 102 can interact with a camera device to receive a real-time image of a hot zone uploaded by the camera device.
  • the processing system may further include, for example, an image storage module 1040, which is configured to store a current image of a hot zone that is acquired by the imaging device and has an associated relationship with the image storage module 1040.
  • a standard image of the at least one hot zone is provided for the image acquisition module 1020 to acquire.
  • At least some of the modules, sub-modules, units, sub-units, or at least a part of any of the functions according to the embodiments of the present disclosure may be implemented in one module. Any one or more of a module, a submodule, a unit, and a subunit according to an embodiment of the present disclosure may be implemented by being divided into a plurality of modules.
  • any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), System-on-chip, system-on-substrate, system-on-package, application-specific integrated circuit (ASIC), or any other reasonable means of hardware or firmware that can integrate or package the circuit, or software, hardware, and firmware Any one of the implementation manners or an appropriate combination of any of them may be implemented.
  • FPGA field programmable gate array
  • PLA programmable logic array
  • ASIC application-specific integrated circuit
  • any one of the implementation manners or an appropriate combination of any of them may be implemented.
  • one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be at least partially implemented as computer program modules, and when the computer program modules are executed, corresponding functions may be performed.
  • any of the acquisition request receiving module 1010, the image acquisition module 1020, the placement status determination module 1030, the image storage module 1040, the image feature identification submodule 1031, the item feature determination submodule 1032, and the placement status determination submodule 1033 Multiple can be combined in one module, or any one of them can be split into multiple modules.
  • at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module.
  • At least one of the modules 1033 may be implemented at least partially as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or it can be implemented by hardware or firmware in any other reasonable way to integrate or package the circuit, or in any of the three implementations of software, hardware, and firmware or in any appropriate combination of any of these to realise.
  • FPGA field programmable gate array
  • PLA programmable logic array
  • ASIC application specific integrated circuit
  • At least one of the acquisition request receiving module 1010, the image acquisition module 1020, the placement status determination module 1030, the image storage module 1040, the image feature identification submodule 1031, the item feature determination submodule 1032, and the placement status determination submodule 1033 One can be implemented at least partially as a computer program module, and when the computer program module is executed, it can perform a corresponding function.
  • FIG. 11 schematically illustrates a block diagram of a computer system suitable for implementing the item placement state acquisition method or processing method described above according to an embodiment of the present disclosure.
  • the computer system shown in FIG. 11 is only an example, and should not impose any limitation on the functions and scope of use of the embodiments of the present disclosure.
  • a computer system 1100 includes a processor 1101 that can be loaded into a random access memory (RAM) 1103 according to a program stored in a read-only memory (ROM) 1102 or from a storage portion 1108.
  • the program performs various appropriate actions and processes.
  • the processor 1101 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (such as an application-specific integrated circuit (ASIC)), and so on.
  • the processor 1101 may also include on-board memory for cache use.
  • the processor 1101 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present disclosure.
  • the processor 1101, the ROM 1102, and the RAM 1103 are connected to each other through a bus 1104.
  • the processor 1101 executes various operations of the method flow according to the embodiment of the present disclosure by executing programs in the ROM 1102 and / or the RAM 1103. It should be noted that the program may also be stored in one or more memories other than the ROM 1102 and the RAM 1103.
  • the processor 1101 may also execute various operations of the method flow according to the embodiment of the present disclosure by executing a program stored in the one or more memories.
  • the system 1100 may further include an input / output (I / O) interface 1105, and the input / output (I / O) interface 1105 is also connected to the bus 1104.
  • the system 1100 may also include one or more of the following components connected to the I / O interface 1105: an input portion 1106 including a keyboard, a mouse, and the like; including, such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker Output section 1107, etc .; a storage section 1108 including a hard disk, etc .; and a communication section 1109, including a network interface card such as a LAN card, a modem, and the like.
  • CTR cathode ray tube
  • LCD liquid crystal display
  • speaker Output section 1107 etc .
  • storage section 1108 including a hard disk, etc .
  • a communication section 1109 including a network interface card such as a LAN card, a modem, and the like.
  • the communication section 1109 performs communication processing via a network such as the Internet.
  • the driver 1110 is also connected to the I / O interface 1105 as needed.
  • a removable medium 1111 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as needed, so that a computer program read therefrom is installed into the storage section 1108 as needed.
  • the method flow according to the embodiment of the present disclosure may be implemented as a computer software program.
  • embodiments of the present disclosure include a computer program product including a computer program carried on a computer-readable medium, the computer program containing program code for performing a method shown in a flowchart.
  • the computer program may be downloaded and installed from a network through the communication section 1109, and / or installed from a removable medium 1111.
  • the processor 1101 When the computer program is executed by the processor 1101, the above functions defined in the system of the embodiment of the present disclosure are executed.
  • the systems, devices, devices, modules, units, and the like described above may be implemented by computer program modules.
  • the present disclosure also provides a computer-readable medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist alone without being assembled into the device / apparatus / System.
  • the computer-readable medium described above carries one or more programs, and when the one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.
  • the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination of the foregoing.
  • the computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programming read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in combination with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal that is included in baseband or propagated as part of a carrier wave, and which carries computer-readable program code. Such a propagated data signal may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable medium may send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device .
  • the program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, optical fiber cable, radio frequency signals, etc., or any suitable combination of the foregoing.
  • the computer-readable medium may include one or more memories other than the ROM 1102 and / or the RAM 1103 and / or the ROM 1102 and the RAM 1103 described above.
  • each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, which contains one or more of the logic functions used to implement the specified logic.
  • Executable instructions may also occur in a different order than those marked in the drawings. For example, two successively represented boxes may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved.
  • each block in the block diagram or flowchart, and combinations of blocks in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with A combination of dedicated hardware and computer instructions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Studio Devices (AREA)
  • Alarm Systems (AREA)

Abstract

本公开提供了一种物品摆放状态获取方法，包括：发送获取请求，该获取请求包括一个或多个关联关系，每一个关联关系包括一个热区与一个摄像设备之间的关联关系，一个热区对应一种物品；以及接收至少一个热区对应的物品的摆放状态，该至少一个热区对应的物品的摆放状态是响应于获取请求根据至少一个热区的当前图像及标准图像确定的，其中，摄像设备用于获取与其具有关联关系的热区的当前图像。本公开还提供了一种处理方法，一种物品摆放状态获取系统和一种处理系统。

Description

物品摆放状态获取方法和系统 技术领域
本公开涉及电子技术领域，更具体地，涉及一种物品摆放状态获取方法和系统。
背景技术
随着消费与技术两股力量的绞合，零售基础设施发生变化，引发了第四次零售革命，将人们带入无界零售的时代。在无界零售时代，最重要的基础设施之一是货架，关键的环节之一是理货。
在实现本公开构思的过程中，发明人发现现有技术中至少存在如下问题：现有技术中，零售门店的理货工作，没有成熟的技术做支撑。货架上物品的摆放状态主要还是依靠人工巡检得到，或者依赖于理货经验得到，因此无法及时的获取该货架上物品的摆放状态。因此，现有技术不仅存在理货延时严重、时效性差的缺陷，还存在理货精准度差、效率低、理货成本高的缺陷。
发明内容
有鉴于此，本公开提供了一种能够提高理货效率的物品摆放状态获取方法和系统，以及一种处理方法和系统。
本公开的一个方面提供了一种物品摆放状态获取方法，包括：发送获取请求，该获取请求包括一个或多个关联关系，每一个关联关系包括一个热区与一个摄像设备之间的关联关系，一个热区对应一种物品；以及接收至少一个热区对应的物品的摆放状态，该至少一个热区对应的物品的摆放状态是响应于获取请求，根据至少一个热区的当前图像及标准图像确定的，摄像设备用于获取与其具有关联关系的热区的当前图像。
根据本公开的实施例，上述方法还包括：设定所述一个或多个关联关系；并且/或者展示至少一个热区中一个或多个热区对应的物品的摆放状态；并且/或者根据用户输入的筛选条件，自至少一个热区对应的物品的摆放状态中筛选得到与筛选条件匹配的、一个或多个热区对应的物品的摆放状态，其中，筛选条件包括以下至少之一：摄像设备编号、热区编号、物品名称和/或物品的摆放状态。
根据本公开的实施例，上述方法还包括：设定摄像设备获取与其具有关联关系的热区的当前图像的周期；并且/或者设定接收至少一个热区对应的物品的摆放状态的周期，其中接收至少一个热区对应的物品的摆放状态的周期不小于摄像设备获取与其具有关联关系的热区的当前图像的周期。
根据本公开的实施例，一个热区对应的物品的摆放状态包括正常状态和非正常状态，上述物品摆放状态获取方法还包括：在至少一个热区中的一个或多个热区对应的物品的摆放状态为非正常状态的情况下，向终端设备发送报警信号。
根据本公开的实施例，上述与摄像设备具有关联关系的热区的当前图像的获取包括：获取与摄像设备具有关联关系的热区的多张实时图像，该多张实时图像是按时间间隔连续获取的；识别多张实时图像，得到该多张实时图像中包括的物品数量；以及确定与摄像设备具有关联关系的热区的当前图像，该当前图像是多张实时图像中包括的物品数量最多的实时图像，其中，摄像设备与一个或多个热区具有关联关系。
本公开的另一个方面提供了一种处理方法，包括：接收获取请求，该获取请求包括一个或多个关联关系，每一个关联关系包括一个热区与一个摄像设备之间的关联关系，一个热区对应一种物品；响应于获取请求，获取至少一个热区的当前图像及标准图像；以及根据至少一个热区的当前图像及标准图像，确定至少一个热区对应的物品的摆放状态，其中，摄像设备用于获取与其具有关联关系的热区的当前图像。
根据本公开的实施例，根据至少一个热区的当前图像及标准图像，确定至少一个热区对应的物品的摆放状态包括：识别至少一个热区的当前图像，得到至少一个热区的当前图像包括的图像特征；基于至少一个热区的当前图像包括的图像特征，确定至少一个热区对应的物品特征；以及对比至少一个热区对应的物品特征与至少一个热区对应的标准物品特征，确定至少一个热区对应的物品的摆放状态，其中，至少一个热区对应的标准物品特征是基于至少一个热区的标准图像得到的。
根据本公开的实施例，一个热区对应的物品的摆放状态包括正常状态和非正常状态，该非正常状态包括以下情况的至少之一：对应的物品特征中，存在与对应的标准物品特征中的标准物品名称不同的物品名称的情况； 对应的物品特征中与对应的标准物品特征的标准物品名称相同的物品名称对应的物品数量，与标准物品名称对应的标准物品数量的比值小于第一比值的情况；或对应的物品特征中与对应的标准物品特征的标准物品名称相同的物品名称对应的物品标志中，与标准物品名称对应的标准物品标志的相似度低于预设相似度的物品标志数量，与该物品名称对应的物品标志总数量的比值大于第二比值的情况，其中，物品特征包括以下至少之一：物品名称、物品名称对应的物品数量、物品名称对应的物品标志和/或物品名称对应的物品位置。
根据本公开的实施例，上述与摄像设备具有关联关系的热区的当前图像的获取包括：获取与摄像设备具有关联关系的热区的多张实时图像，该多张实时图像是按时间间隔连续获取的；识别多张实时图像，得到该多张实时图像中包括的物品数量；以及确定与摄像设备具有关联关系的热区的当前图像，该当前图像是多张实时图像中包括的物品数量最多的实时图像，其中，摄像设备与一个或多个热区具有关联关系。
本公开的另一个方面还提供了一种物品摆放状态获取系统，该物品摆放状态获取系统能够与处理系统交互，包括获取请求发送模块和摆放状态接收模块，获取请求发送模块用于发送获取请求，该获取请求包括一个或多个关联关系，每一个关联关系包括一个热区与一个摄像设备之间的关联关系，一个热区对应一种物品，摆放状态接收模块用于接收至少一个热区对应的物品的摆放状态，该至少一个热区对应的物品的摆放状态由处理系统响应于获取请求，根据至少一个热区的当前图像及标准图像确定，摄像设备用于获取与其具有关联关系的热区的当前图像。
根据本公开的实施例，上述系统还包括参数设定模块、展示模块和/或筛选模块，参数设定模块用于设定一个或多个关联关系，展示模块用于展示至少一个热区中一个或多个热区对应的物品的摆放状态，筛选模块用于根据用户输入的筛选条件，自至少一个热区对应的物品的摆放状态中筛选得到与筛选条件匹配的、一个或多个热区对应的物品的摆放状态以供展示，其中，筛选条件包括以下至少之一：摄像设备编号、热区编号、物品名称和/或物品的摆放状态。
根据本公开的实施例，上述参数设定模块还用于设定摄像设备获取与 其具有关联关系的热区的当前图像的周期；并且/或者，上述摆放状态获取模块还用于设定接收至少一个热区对应的物品的摆放状态的周期，其中，接收至少一个热区对应的物品的摆放状态的周期，不小于摄像设备获取与其具有关联关系的热区的当前图像的周期。
根据本公开的实施例，上述至少一个热区对应的物品的摆放状态包括正常状态和非正常状态，上述物品摆放状态获取系统还包括报警模块，用于在至少一个热区中的一个或多个热区对应的物品的摆放状态为非正常状态的情况下，向终端设备发送报警信号。
根据本公开的实施例，上述与摄像设备具有关联关系的热区的当前图像的获取包括：获取与摄像设备具有关联关系的热区的多张实时图像，该多张实时图像是按时间间隔连续获取的；识别多张实时图像，得到多张实时图像中包括的物品数量；以及确定与摄像设备具有关联关系的热区的当前图像，该当前图像是多张实时图像中包括的物品数量最多的实时图像，其中，摄像设备与一个或多个热区具有关联关系。
本公开的另一个方面提供了一种处理系统，该处理系统能够与物品摆放状态获取系统交互，包括获取请求接收模块、图像获取模块以及摆放状态确定模块，获取请求接收模块用于接收物品摆放状态获取系统发送的获取请求，该获取请求包括一个或多个关联关系，每一个关联关系包括一个热区与一个摄像设备之间的关联关系，一个热区对应一种物品；图像获取模块用于响应于获取请求，获取至少一个热区的当前图像及标准图像；摆放状态确定模块用于根据至少一个热区的当前图像及标准图像，确定至少一个热区对应的物品的摆放状态，其中，摄像设备用于获取与其具有关联关系的热区的当前图像。
根据本公开的实施例，上述摆放状态确定模块包括图像特征识别子模块、物品特征确定子模块以及摆放状态确定子模块，图像特征识别子模块用于识别至少一个热区的当前图像，得到至少一个热区的当前图像包括的图像特征；物品特征确定子模块用于基于至少一个热区的当前图像包括的图像特征，确定至少一个热区对应的物品特征；摆放状态确定子模块用于对比至少一个热区对应的物品特征与至少一个热区对应的标准物品特征，确定至少一个热区对应的物品的摆放状态，其中，至少一个热区对应的标 准物品特征是基于至少一个热区的标准图像得到的。
根据本公开的实施例，上述物品的摆放状态包括正常状态和非正常状态，该非正常状态包括以下情况的至少之一：对应的物品特征中，存在与对应的标准物品特征中的标准物品名称不同的物品名称的情况；对应的物品特征中与对应的标准物品特征的标准物品名称相同的物品名称对应的物品数量，与标准物品名称对应的标准物品数量的比值小于第一比值的情况；或者，对应的物品特征中与对应的标准物品特征的标准物品名称相同的物品名称对应的物品标志中，与标准物品名称对应的标准物品标志的相似度低于预设相似度的物品标志数量，与该物品名称对应的物品标志总数量的比值大于第二比值的情况，其中，物品特征包括以下至少之一：物品名称、物品名称对应的物品数量、物品名称对应的物品标志和/或物品名称对应的物品位置。
根据本公开的实施例，上述处理系统为云端系统；并且/或者上述系统还包括图像存储模块，用于存储摄像设备获取的与其具有关联关系的热区的当前图像及至少一个热区的标准图像，以供图像获取模块获取。
根据本公开的实施例，上述与摄像设备具有关联关系的热区的当前图像的获取包括：获取与摄像设备具有关联关系的热区的多张实时图像，该多张实时图像是按时间间隔连续获取的；识别该多张实时图像，得到多张实时图像中包括的物品数量；以及确定与摄像设备具有关联关系的热区的当前图像，该当前图像是多张实时图像中包括的物品数量最多的实时图像，其中，所述摄像设备与一个或多个热区具有关联关系。
本公开的另一个方面提供了一种电子设备，包括：一个或多个处理器；存储装置，用于存储一个或多个程序，其中，当所述一个或多个程序被所述一个或多个处理器执行时，使得所述一个或多个处理器执行上述的物品摆放状态获取方法。
本公开的另一个方面提供了一种电子设备，包括：一个或多个处理器；存储装置，用于存储一个或多个程序，其中，当所述一个或多个程序被所述一个或多个处理器执行时，使得所述一个或多个处理器执行上述的处理方法。
本公开的另一个方面提供了一种非易失性存储介质，存储有计算机可 执行指令，所述指令在被执行时用于实现如上所述的物品摆放状态获取方法。
本公开的另一个方面提供了一种非易失性存储介质，存储有计算机可执行指令，所述指令在被执行时用于实现如上所述的处理方法。
根据本公开的实施例，可以至少部分地避免依靠人工巡检获取物品摆放状态导致的理货成本高，且无法实时获取物品摆放状态导致的理货延时的缺陷。并因此可以实现实时的自动获取物品摆放状态，以提高理货效率，提高理货时效性以及降低理货人力成本的技术效果。
附图说明
附图是用来提供对本发明的进一步理解，并且构成说明书的一部分，与下面的具体实施方式一起用于解释本发明，但并不构成对本发明的限制。在附图中：
图1示意性示出了根据本公开实施例的物品摆放状态获取方法和系统的应用场景；
图2A～图2B示意性示出了根据本公开实施例的物品摆放状态获取方法的流程图；
图3A～图3B示意性示出了根据本公开另一实施例的物品摆放状态获取方法的流程图；
图4示意性示出了根据参考图3A描述的物品摆放状态获取方法得到的展示画面；
图5示意性示出了根据本公开实施例的热区的当前图像的获取流程图；
图6示意性示出了根据本公开实施例的处理方法的操作流程图；
图7示意性示出了根据本公开实施例的确定物品摆放状态的流程图；
图8A～图8C示意性示出了根据本公开实施例的物品摆放状态为非正常状态的示意图；
图9A～图9E示意性示出了根据本公开实施例的物品摆放状态获取系统的结构框图；
图10A～图10C示意性示出了根据本公开实施例的处理系统的结构框图；
图11示意性示出了根据本公开实施例的适于实现物品摆放状态获取 方法或处理方法的计算机系统的方框图。
具体实施方式
为使本发明的目的、技术方案和优点更加清楚明白，以下结合具体实施例，并参照附图，对本发明进一步详细说明。
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
图1示意性示出了根据本公开实施例的物品摆放状态获取方法和系统的应用场景。需要注意的是，图1所示仅为可以应用本公开实施例的应用场景的示例，以帮助本领域技术人员理解本公开的技术内容，但并不意味着本公开实施例不可以用于其他设备、系统、环境或场景。
如图1所示，根据该实施例的应用场景100包括终端设备101、云端系统102、摄像设备103、货架104。摄像设备103与云端系统102之间、云端系统102与终端设备101之间或者摄像设备103与终端设备101之间通过通信链路的介质连接，该连接可以包括各种连接类型，例如有线、无线通信链路或者光纤电缆等等。
终端设备101可以与云端系统102进行交互，以接收物品摆放状态等。终端设备101上可以安装有各种操作应用，以向云端系统102发送获取请求；终端设备101还可以与摄像设备103进行交互，以调节摄像设备103的拍摄频率。
终端设备101可以是具有显示屏的各种电子设备，以向用户展示物品的摆放状态，包括但不限于台式计算机、膝上型便携计算机、平板电荷和智能手机等。
根据本公开的实施例，该终端设备101例如还可以通过与云端设备102的交互，获取云端系统102存储的摄像设备103拍摄的货架的当前图像200(包括热区的当前图像)以及该货架的标准图像(包括热区的标准图像)进行展示，以更好的展示货架104上各个热区对应的物品的摆放状态。
根据本公开的实施例，一个热区对应于货架104上摆放有同一商品的区域，对于参考图1中的货架104，该一个热区例如可以为该货架104的最底层放置有“可口可乐”商品的区域。可以理解的是，上述对热区的描述仅作为示例以利于理解本公开，本公开对此不作限定。
云端系统102可以是具有存储及计算功能的云端虚拟终端设备(例如虚拟个人计算机等)，以存储摄像设备103拍摄的货架104的当前图像200及货架的标准图像，并响应于终端设备101的获取请求，对当前图像与标准图像进行处理分析得到该货架104上一个或多个热区的物品的摆放状态(仅为示例)。
云端系统102例如还具有图像处理功能，以根据标准图像上与热区对应区域的坐标范围，在摄像设备103拍摄的货架的当前图像上标注得到热区的对应区域。
摄像设备103例如可以通过设定的通信协议与云端系统102通信，其可以是具有拍摄功能的终端设备，用于拍摄货架104的实时图像，并将拍摄的当前图像200发送至云端系统102，以供云端系统102存储。
根据本公开的实施例，该摄像设备103例如还可以是具有计算处理功能的摄像设备103，该摄像设备103可以以间隔时间连续抓拍货架104得到多张实时图像，则该摄像设备通过其计算处理功能可以对该多张实时图像进行识别处理，以从该多张实时图像中筛选到最佳图像作为当前图像(包括热区的货架的当前图像)。根据本公开的实施例，该摄像设备103例如还可以具有类似于云端系统的图像处理的功能。
根据本公开的实施例，该摄像设备103例如可以是具有实时拍摄或定期拍摄功能的监控摄像头等摄像装置，以实时或定期获取货架104的图像，从而实时或定期得到该货架104的图像(包括该货架104包括的一个或多个热区的图像)。
需要说明的是，本公开实施例所提供的物品摆放状态获取方法一般可以由终端设备101执行。相应地，本公开实施例所提供的物品摆放状态获取系统一般可以设置于终端设备101中。
根据本公开的实施例，参考图1中的应用场景例如也可以不包括云端系统102，则摄像设备103与终端设备101通信，该终端设备101还具有 存储及计算功能，以存储摄像设备103拍摄的货架104的当前图像200(包括热区的当前图像)及该货架104的标准图像(包括热区的标准图像)，并分析识别当前图像200及标准图像后得到该货架104包括的一个或多个热区的物品的摆放状态。
可以理解的是，图1中的终端设备、云端系统、摄像设备、货架及货架上摆放物品的数目及类型仅仅是示意性的。根据实现需要，可以具有任意数目及类型的终端设备、云端系统、摄像设备、货架及摆放物品。
图2A～图2B示意性示出了根据本公开实施例的物品摆放状态获取方法的流程图。
如图2A所示，该物品摆放状态获取方法包括操作S210～操作S220。
在操作S210，发送获取请求，该获取请求包括一个或多个关联关系，每一个关联关系包括一个热区与一个摄像设备之间的关联关系，一个热区对应一种物品。
其中，一个或多个关联关系例如可以是预先设定好的，则如图2B所示，该方法例如还可以包括操作S230，设定一个或多个关联关系。该操作S230例如可以在操作S210之前执行。
根据本公开的实施例，热区例如可以为货架104上摆放有同一种物品的区域，摄像设备例如可以是获取与其具有关联关系的热区的当前图像的摄像设备，具体例如可以是获取与其具有关联关系的热区所在货架104的当前图像的摄像设备103。
根据本公开的实施例，摄像设备103例如可与预设有热区的货架104位于同一空间中，摄像设备103具体例如可以设置于与货架104相对的墙壁上、与该货架104相对的另一货架上、或者该货架104所在空间的顶壁上，以用于拍摄该货架104的实时图像，该拍摄的货架104的实时图像包括该货架104具有的热区的实时图像，因此也包括该货架104具有的热区的当前图像。
根据本公开的实施例，摄像设备可以在终端设备101响应于用户的操作，向该摄像设备发送拍摄指令时获取与其具有关联关系的热区的当前图像(具体例如可以为拍摄该热区所在货架的当前图像)，或者，该摄像设备还可以根据设定的周期，周期性地获取与其具有关联关系的热区的当前 图像，该周期例如可以通过终端设备101设定，则该物品摆放状态获取方法还可以包括相应的操作：设定摄像设备获取与其具有关联关系的热区的当前图像的周期，其中，该设定的获取周期例如可以是5min。
根据本公开的实施例，该获取请求例如可以是响应于用户的操作由终端设备101发送的，还可以是按照预先设定好的周期发送的，从而以周期性的接收至少一个热区对应的物品的摆放状态，则该物品摆放状态获取方法例如还可以包括相应的操作：设定接收至少一个热区对应的物品的摆放状态的周期，其中，该设定的接收周期例如可以是与获取周期相同的5min。
根据本公开的实施例，接收至少一个热区对应的物品的摆放状态的周期不小于摄像设备获取与其具有关联关系的热区的当前图像的周期，从而可保证通过该物品摆放状态获取方法接收到的热区对应的物品的摆放状态是根据该热区摆放的物品的状态实时更新的。
根据本公开的实施例，该设定一个或多个关联关系例如可以由用户通过终端设备101设定，设定热区与摄像设备之间的关联关系的过程具体例如可以是通过终端设备101将摄像设备信息与热区信息绑定的过程。其中的摄像设备信息例如可以包括摄像设备编号、型号和/或厂商等，热区信息例如可以包括热区编号、该热区摆放的物品信息(例如物品名称、物品编号等)。
根据本公开的实施例，该设定热区与摄像设备之间的关联关系具体例如还可以为：通过终端设备101依次注册摄像设备、货架及热区，其中：
注册摄像设备的过程例如可以是填写摄像设备信息(如摄像设备编号、型号和/或厂商等)的过程；
注册货架的过程例如可以是先填写货架信息(例如可以包括货架编号和摄像设备编号，还可以包括该货架所在门店名称、货架长度、货架宽度等)，然后确定云端系统或该终端设备101本地是否存在该货架的标准图像，若存在该货架的标准图像，则完成该货架的注册，若不存在货架的标准图像，则重新调整该货架上物品的摆放或重新调节摄像设备的拍摄角度后，根据摄像设备重新拍摄的货架的实时图像确认是否为标准图像，直至存在该货架的标准图像后，完成该货架的注册；
热区的注册过程例如可以为：首先填写货架信息，然后判断该货架信 息对应的货架是否完成注册，若完成该货架注册则读取该货架的标准图像并在该标准图像上标注出热区的对应区域范围，并填写热区信息(例如可以包括热区编号、该热区摆放的物品的名称和/或编号、用户信息(例如该热区的负责人名称)等)；若未完成货架注册，则返回进入货架注册流程，并再次进行热区注册过程。
综上可知，通过上述实施例的注册过程，可以实现摄像设备、货架与热区的绑定，从而实现了热区与摄像设备的关联关系的设定，且可以理解的是，上述设定热区与摄像设备的关联关系的方法仅作为示例以利于理解本公开，本公开对此不作限定。
根据本公开的实施例，上述货架的标准图像例如可以是摄像设备在特定时刻拍摄的、符合要求的图像，该标准图像例如可以是该货架刚被摆满物品，且该物品的摆放满足标准条件(物品的摆放对于摄像设备而言，处于物品标志完全显露的角度)时拍摄得到的。具体地，该货架的标准图像例如还可以是终端设备101从云端系统102/或摄像设备103获取多个时间段拍摄得到的货架的图像展示给用户，由用户自该多个时间段的货架的图像中挑选后，上传至云端系统102的；该货架的标准图像例如还可以由云端系统102根据预设的标准，自多个时间段的货架的图像中筛选得到。
根据本公开的实施例，该获取请求包括的一个或多个关联关系，及摄像设备103获取的与其具有关联关系的热区的当前图像，能够为得到至少一个热区对应的物品的摆放状态提供条件。
在操作S220，接收至少一个热区对应的物品的摆放状态，至少一个热区对应的物品的摆放状态是响应于获取请求，根据至少一个热区的当前图像及标准图像确定的。
根据本公开的实施例，上述热区的标准图像例如可以是该货架的标准图像中与热区对应区域的图像，该热区的标准图像中该热区摆放的物品处于满足标准条件：摆满且物品标识相对于摄像设备完全显露的状态。
根据本公开的实施例，该热区的当前图像例如可以是所述货架的当前图像上热区的对应区域，物品的摆放状态具体例如可以是根据货架的当前图像及该货架的标准图像确定的，其中，货架的当前图像可以是自多张货架的实时图像中筛选得到的。
根据本公开的实施例，热区的当前图像具体还可以通过以下方式得到：根据在热区注册时，在货架104的标准图像上标注的热区对应区域的坐标范围，在货架104的当前图像中划分得到与该坐标范围相对应的区域的图像，则该划分得到的图像即为该热区的当前图像，则，物品的摆放状态具体例如可以是通过划分得到的该热区的当前图像及热区的标准图像确定的。可以理解的是，该热区的当前图像例如可以通过具有图像处理功能的摄像设备103得到，也可以通过云端系统102得到，还可以通过终端设备101得到，本公开对此不作限定。根据本公开的实施例，该热区的当前图像例如还可以是自多张实时图像中筛选得到的。
根据本公开的实施例，该接收得到的热区对应的物品的摆放状态例如可以包括正常状态与非正常状态。其中，响应于获取请求，根据至少一个热区的当前图像及标准图像确定物品的摆放状态例如可包括：首先识别该热区的当前图像和标准图像，得到该热区包括的当前的图像特征及标准图像特征，再根据该当前的图像特征与标准图像特征，得到该热区当前对应的物品特征及标准物品特征，通过比对该热区当前对应的物品特征与标准物品特征来确定物品的摆放状态，其中的正常状态例如可以是：该热区当前对应的物品特征与标准物品特征相比满足预设的多个条件的情况，而其中的非正常状态例如可以是该热区当前对应的物品特征与标准物品特征相比不满足预设的多个条件中任一个条件的情况。
根据本公开的实施例，热区对应的物品特征及标准物品特征例如还可以通过以下方式得到：以根据该热区的标准图像得到的标准图像特征及根据该热区的当前图像得到的图像特征作为输入，通过预先训练好的模型得到热区对应的物品特征及标准物品特征，该热区对应的物品特征例如可以包括物品名称、物品名称对应的物品数量、物品名称对应的物品标志和/或物品位置(即在热区的当前图像中的坐标位置)，相应地，标准物品特征例如可以包括标准物品名称、标准物品名称对应的物品数量、标准物品名称对应的物品标志和/或标准物品位置(即在热区的标准图像中的坐标位置)。
根据本公开的实施例，热区对应的物品特征例如还可以通过以下方式得到：以该热区所属的货架的当前图像得到的图像特征及该热区在其所属 的货架的标准图像上的坐标范围作为输入，通过预先训练好的模型得到的货架的实时图像上该坐标范围内的物品特征，即为该热区对应的物品特征。相应地，该热区对应的标准物品特征例如可通过以下方式得到：预先在该热区所属的货架的标准图像上标注出一个或多个热区对应的区域，并以该标注后的标准图像作为输入，通过预先训练好的模型得到货架的标准图像上一个或多个热区对应的物品特征。
根据本公开的实施例，如图2B所示，该物品状态获取方法例如还可以包括操作S240，展示至少一个热区中一个或多个热区对应的物品的摆放状态。
根据本公开的实施例，终端设备101例如可以具有显示屏，该物品摆放状态获取方法可由终端设备101执行，通过该终端设备101的显示屏展示一个或多个热区对应的物品的摆放状态，以使用户能够通过展示内容实时的获知货架上各热区摆放物品的状态。
根据本公开的实施例，该终端设备101的显示屏例如还可以显示有以下至少之一：摆放有该热区所在货架的门店的门店名称、该热区所属货架的货架编号、热区编号、物品名称、物品编号、及该热区的标准图像、热区的实时图像及用户信息(例如该热区的负责人名称)等。
综上可知，通过本公开实施例的物品摆放状态获取方法，通过发送获取请求，可以接收热区对应的物品的摆放状态，以定期或实时的得知该物品的摆放状态，供用户判断是否需要理货，由于该方法无需用户到热区所在货架区域巡检，从而可以有效地提高理货效率，避免通常理货存在的延时严重，时效性差的缺陷，同时降低理货成本；再者，由于该物品的摆放状态是根据热区的标准图像及当前图像得到的，而无需用户通过理货经验判断得到，因此在一定程度上能够提高理货的精准度。
图3A～图3B示意性示出了根据本公开另一实施例的物品摆放状态获取方法的流程图，图4示意性示出了根据参考图3A描述的物品摆放状态获取方法得到的展示画面。
如图3A所示，该物品摆放状态获取方法除了参考图2A描述的操作S210～操作S220，还可以包括操作S350和操作S360。
根据本公开的实施例，操作S350例如可以在操作S220之后执行，或 者在用户输入筛选条件时执行。
在操作S350，根据用户输入的筛选条件，自至少一个热区对应的物品的摆放状态中筛选得到与筛选条件匹配的、一个或多个热区对应的物品的摆放状态以供展示；则相应地，在操作S360，展示该筛选得到的与筛选条件匹配的、一个或多个热区对应的物品的摆放状态。
根据本公开的实施例，具有显示屏的终端设备例如还展示有筛选条件输入窗口，以便用户输入筛选条件，该筛选条件例如可以包括以下至少之一：摄像设备编号、热区编号、物品名称和/或物品的摆放状态。则在用户输入摄像设备编号的情况下，操作S350筛选得到的即为与该摄像设备编号对应的摄像设备具有关联关系的所有热区对应的物品的摆放状态；在用户输入物品名称的情况下，操作S350筛选得到的即为所有摆放有与对应于该物品名称的物品所在热区该物品的摆放状态。
根据本公开的实施例，在各个时间段获取得到的物品的摆放状态例如可以存储于本地存储空间(例如参考图1所示的终端设备)中，则用户输入的筛选条件例如还可以包括有时间段，操作S350筛选得到的即为本地存储空间中存储的该时间段内所有的物品的摆放状态；若通过上述注册方式设定热区与摄像设备的关联关系，则用户输入的筛选条件例如还可以包括门店名称、货架编号等，操作S350筛选得到的即为本地存储空间中存储的、与该门店名称/货架编号对应的门店/货架包括的热区对应的物品的摆放状态。根据本公开的实施例，本物品摆放状态获取方法例如还可以对本地存储空间中存储的摆放状态进行定期清理，以使得该本地存储空间仅存储有最近一段时间的摆放状态，降低该本地存储空间的负荷。
根据本公开的实施例，对于实际的应用场景，例如一个用户负责多个热区的物品的摆放状态的情况，则还可以设定该用户信息与该多个热区、与该多个热区具有关联关系的摄像设备等的关联关系，则用户输入的筛选条件例如还可以包括该用户信息，操作S350筛选得到的即为与该用户信息对应的用户负责的多个热区对应的物品的摆放状态，此处用户信息例如可以包括负责人名称、负责人职位和/或负责人工号等。
可以理解的是，上述筛选条件仅作为示例以利于理解本公开，本公开对此不作限定，输入的筛选条件例如还可以为上述列举的筛选条件中的两 个或多个，例如输入的筛选条件还可以为门店名称和用户信息等。
根据本公开的实施例，通过参考图3A描述的物品摆放状态获取方法得到的展示页面如图4所示，用户例如可以通过该图4的展示页面输入筛选条件，具体地，该展示界面展示有筛选条件选项，还展示有通过输入筛选条件后用户点击“查询”控件后筛选得到的物品的摆放状态的展示结果。其中物品的摆放状态例如可以包括“正常状态”、“缺货状态”、“不规范状态”及“错放状态”等，此些状态的具体含义将在后续内容中描述。
综上可知，本公开实施例的物品摆放状态获取方法，用户可根据需求输入筛选条件，以灵活的调取其关注物品的摆放状态，从而使得展示的结果更符合用户的需求，提高用户查看物品摆放状态的效率。
如图3B所示，本公开实施例的物品摆放状态获取方法除了参考图2A描述的操作S210～操作S220，例如还可以包括操作S370，该操作S370例如可以在操作S220之后执行。
在操作S370，在至少一个热区中的一个或多个热区对应的物品的摆放状态为非正常状态的情况下，向终端设备发送报警信号。
根据本公开的实施例，此处的终端设备例如可以是参考图1描述的终端设备101，还可以是智能手机、平板电脑等便携式电子设备，该报警信息例如可以是向该终端设备101推送消息、向该终端设备发送播放特定音乐或铃声的指令、或者向终端设备发送弹出特定窗口的指令等，以使该终端设备的持有者能够得知存在物品的摆放状态处于不正常状态的情况，并根据该不正常状态进行相应的理货操作。
因此，通过上述物品摆放状态获取方法，用户可以及时的得知物品的摆放不符合要求，或者该物品存在缺货的情况，从而可以及时的进行理货操作，因此能够避免现有技术中理货时效性差导致的顾客购物体验不高、店铺销量降低的缺陷。
图5示意性示出了根据本公开实施例的热区的当前图像的获取流程图。
如图5所示，该与摄像设备具有关联关系的热区的当前图像的获取包括操作S510～操作S530。
在操作S510，获取与摄像设备具有关联关系的热区的多张实时图像，该多张实时图像是按时间间隔连续获取的。
根据本公开的实施例，该操作具体例如可以是摄像设备按设定的时间间隔连续获取与其具有关联关系的热区所在货架的多张实时图像，并以该货架的多张实时图像作为热区的多张实时图像执行后续的操作S520～操作S530。
根据本公开的实施例，该操作具体例如还可以是在获取到货架的多张实时图像后，再根据与摄像设备具有关联关系的热区所在货架的标准图像中对应的坐标范围，自该货架的多张实时图像中分别获取到与该坐标范围相对应的图像，即得到与该摄像设备具有关联关系的热区的多张实时图像，并以该从货架的多张实时图像中获取得到的热区的多张实时图像为基础执行后续的操作S520～操作S530。
根据本公开的实施例，其中时间间隔的取值例如可以是1～30秒中的任一值，该时间间隔例如可以通过参考图1中的终端设备101来设定。
在操作S520，识别该热区的多张实时图像，得到该多张实时图像中包括的物品数量。
根据本公开的实施例，该识别热区的多张实时图像例如可通过物体检测算法来检测，以得到该多张实时图像中每一张包括的物品数量。具体地例如可通过Faster-RCNN算法来对该多张实时图像包括的物品进行检测。可以理解的是，上述识别热区的多张实时图像的方法仅作为示例以利于理解本公开，本公开对此不作限定。
在操作S530，确定与摄像设备具有关联关系的热区的当前图像，该当前图像是多张实时图像中包括的物品数量最多的实时图像。
根据本公开的实施例，上述当前图像的获取操作例如可以是具有数据处理功能的摄像设备执行，则摄像设备的输出直接为与其具有关联关系的热区的当前图像。
根据本公开的实施例，由于具有处理功能的摄像设备价格昂贵，上述当前图像的获取操作例如还可以由参考图1描述的云端系统102或终端设备101来执行，其中，货架的多张实时图像由摄像设备拍摄得到，该摄像设备与云端系统102或终端设备101交互，以将其拍摄的货架的多张实时图像传给云端系统102或终端设备101，从而在一定程度上降低物品摆放状态获取方法的执行成本。
根据本公开的实施例，由于一个货架上可以摆放有多种物品，因此可具有多个热区，且摄像设备可拍摄的空间范围不局限于一个热区，甚至不局限于一个货架，因此一个摄像设备可以与一个或多个热区具有关联关系，具体例如可以与其拍摄到的所有预设的热区建立关联关系。
根据本公开的实施例，由于摄像设备获取货架的实时图像时，可能存在货架前站有顾客的情况，此种情况拍摄的货架的实时图像中，货架上部分物品会被顾客遮挡，因此根据此种情况下得到的实时图像确定的物品的摆放状态必定是不准确的，本公开实施例为了避免此种情况，获取摄像设备连续拍摄的多张实时图像，并从该多张实时图像中选择包括的物品数量最多的实时图像作为当前图像进行物品摆放状态的确定，可以在一定程度上提高获取的物品的摆放状态的精度。
图6示意性示出了根据本公开实施例的处理方法的操作流程图。
如图6所示，该处理方法包括操作S610～操作S630。
在操作S610，接收获取请求，所述获取请求包括一个或多个关联关系，每一个该关联关系包括一个热区与一个摄像设备之间的关联关系，一个热区对应一种物品。
根据本公开的实施例，摄像设备用于获取与其具有关联关系的热区的当前图像。根据本公开的实施例，此处的热区的当前图像具体可以是该热区所在货架的当前图像，该当前图像可以是自多个实时图像筛选得到的；或者该此处的热区的当前图像具体可以是自该热区所在货架的当前图像分割得到的。根据本公开的实施例，该获取请求具体例如可以是参考图2A中操作S210发送的获取请求，在此不再赘述。
在操作S620，响应于获取请求，获取至少一个热区的当前图像及标准图像。
根据本公开的实施例，该当前图像及标准图像例如可以是自摄像设备在各个时间点拍摄得到的货架的实时图像中确定的。其中热区的当前图像例如可以是通过参考图5描述的热区的当前图像获取方法获取得到的，而热区的标准图像例如可以是通过筛选得到的满足标准条件的实时图像，标准条件例如可以是，该热区摆放的物品数量处于饱和(摆满)状态，且物品的摆放方式为物品标志相对于摄像设备完全显露。
根据本公开的实施例，该至少一个热区的当前图像及标准图像例如可存储于与摄像设备交互的云端存储空间中，该云端存储空间例如可以存储有多个摄像设备拍摄的所有图像，则该操作S620即为从该云端存储空间中获取该至少一个热区的当前图像及标准图像。根据本公开的实施例，该云端存储空间例如还可以定期清理其存储的该至少一个热区的当前图像，以使该云端存储空间仅存储有近期的一定时间段内(例如一周内)所述至少一个热区的当前图像。
在操作S630，根据所述至少一个热区的当前图像及标准图像，确定该至少一个热区对应的物品的摆放状态。
根据本公开的实施例，该操作S630例如可以通过图像识别和对比算法来实现，具体为通过对该至少一个热区的当前图像及标准图像进行识别和对比，结合物品摆放状态判断逻辑，得到该至少一个热区对应的物品的摆放状态。
根据本公开的实施例，图像识别例如主要采用物体检测算法(具体可以是Faster-RCNN算法)来实现，得到该至少一个热区的当前图像中包括的物品特征，该物品特征可以包括以下至少之一：物品名称、该物品名称对应的物品数量、该物品名称对应的物品标志及该物品名称对应的物品位置等。其中，该至少一个热区的标准图像例如可预先通过该图像识别得到标准物品特征，该标准物品特征可以包括以下至少之一：标准物品名称、该标准物品名称对应的标准物品数量、该标准物品名称对应的标准物品标志及该标准物品名称对应的标准物品位置等。
根据本公开的实施例，该判断逻辑具体例如可以是分别判断通过图像识别得到的至少一个热区的当前图像中：物品名称是否与标准物品名称相同、与标准物品名称相同的物品名称对应的物品数量与标准物品名称对应的标准物品数量的比值是否大于第一比值、与标准物品名称相同的物品名称中，对应的物品标志与标准物品名称对应的标准物品标志的相似度小于预设相似度的物品标志数量，与该物品名称对应的物品标志总数量的比值是否小于第二比值等。可以理解的是，上述判断逻辑仅作为示例以利于理解本公开，本公开对此不作限定。
图7示意性示出了根据本公开实施例的确定物品摆放状态的流程图。
如图7所示，参考图6中确定物品摆放状态的操作S630具体可包括操作S631～操作S633。
在操作S631，识别所述至少一个热区的当前图像，得到该至少一个热区的当前图像包括的图像特征；在操作S632，基于至少一个热区的当前图像包括的图像特征，确定至少一个热区对应的物品特征；在操作S633，对比至少一个热区对应的物品特征与至少一个热区对应的标准物品特征，确定至少一个热区对应的物品的摆放状态。
根据本公开的实施例，上述操作S631～操作S632具体可以采用图像识别算法(具体例如Faster-RCNN算法)执行，或者以该图像识别算法为基本算法、以包括有不同物品的大量图像为训练数据、以货架的实时图像为测试数据训练得到的深度学习模型执行，该模型的输入例如可以是通过参考图5描述的操作得到的自货架的当前图像划分得到的货架的当前图像，或者通过参考图5描述的操作得到的货架的当前图像和该货架的标准图像中一个或多个热区的对应区域的坐标范围。该模型的输出即为操作S632得到的该一个热区对应的物品特征，或者该货架包括的一个或多个热区对应的物品特征，对于得到多个热区对应的物品特征的情况，通过物品特征中物品名称对应的物品位置即可将物品特征按热区分组。
根据本公开的实施例，上述操作S633具体例如可以通过前述的判断逻辑得到至少一个热区对应的物品的摆放状态。在此不再赘述。
图8A～图8C示意性示出了根据本公开实施例的物品摆放状态为非正常状态的示意图。
通过参考图6～参考图7确定的热区对应的物品的摆放状态包括正常状态和非正常状态。其中，非正常状态例如可以包括以下情况的至少之一：
对应的物品特征中，存在与对应的标准物品特征中的标准物品名称不同的物品名称的情况。具体地，对于如图8A的左侧图所示的货架的标准图像，其包括的一个热区(对应虚线方框围成的区域)对应的标准物品名称例如为“可口可乐”，对于如图8A的右侧图所示的该货架的实时图像(当前图像)，通过参考图7中操作S632得到的该热区对应的物品名称包括“可口可乐”和“王老吉”，则由于此处得到的物品名称“王老吉”与标准物品名称“可口可乐”不同，因此该热区对应的物品的摆放状态为非正常的 错放状态。可以理解的是，上述热区的标准图像、实时图像、标准物品信息及物品信息仅作为示例以利于理解本公开，本公开对此不作限定。
对应的物品特征中与对应的标准物品特征的标准物品名称相同的物品名称对应的物品数量，与标准物品名称对应的标准物品数量的比值小于第一比值的情况。具体地，第一比值例如为0.5，对于如图8B的左侧图所示的货架的标准图像，其包括的一个热区(对应虚线方框围成的区域)对应的标准物品名称为“可口可乐”，该标准物品名称对应的标准物品数量为6个，对于如图8B的右侧图所示的该货架的实时图像(当前图像)，通过参考图7中操作S632得到的该热区对应的物品名称与标准物品名称相同的“可口可乐”对应的物品数量为2个，则该数量与标准物品名称对应的标准物品数量6的比值为2/6＝1/3，小于第一比值0.5，则此时该热区对应的物品的摆放状态为非正常的缺货状态。可以理解的是，上述热区的标准图像、实时图像、标准物品信息、物品信息及第一比值的取值仅作为示例以利于理解本公开，本公开对此不作限定，根据本公开的实施例，对于物品数量与标准物品名称对应的标准物品数量比值不为整数的情况下，可以将该比值四舍五入后与第一比值进行比对。
对应的物品特征中与对应的标准物品特征的标准物品名称相同的物品名称对应的物品标志中，与标准物品名称对应的标准物品标志的相似度低于预设相似度的物品标志数量，与该物品名称对应的物品标志总数量的比值大于第二比值的情况。具体地，设定预设相似度为50％，第二比值为0.5，则对于如图8C的左侧图所示的货架的标准图像，其包括的一个热区(对应虚线方框围成的区域)对应的标准物品名称为“可口可乐”，该标准物品名称对应的标准物品标志为图中该物品罐体表面竖直排列的全部显露的“CocaCola”字样图样，对于如图8C的右侧图所示的该货架的实时图像(当前图像)，通过参考图7中操作S632得到的该热区对应的物品名称与标准物品名称相同的“可口可乐”对应的物品数量为6个，对应的物品标志即包括全部显露的“CocaCola”字样图样，也包括仅显露有部分“CocaCola”字样的图样，且该显露出的部分“CocaCola”字样的图样与标准物品标志全部显露的“CocaCola”字样图样的相似度仅为30％，即该显露出的部分“CocaCola”字样的图样中“CocaCola”字样的显露比例小 于30％，且仅显露有部分“CocaCola”字样的图样的数量为4个，则其与物品数量6的比值为4/6＝2/3，大于第二比值0.5，因此该热区对应的物品的摆放状态为非正常的不规范状态。可以理解的是，上述热区的标准图像、实时图像、标准物品信息、物品信息、预设相似度的取值及第二比值的取值仅作为示例以利于理解本公开，本公开对此不作限定，根据本公开的实施例，上述物品数量的比值在不为整数的情况下，例如还可以在将该比值四舍五入后与第二比值进行比对。
对于不属于非正常状态包括的任一情况时，该热区对应的物品的摆放状态即为正常状态。
图9A～图9E示意性示出了根据本公开实施例的物品摆放状态获取系统的结构框图。
如图9A所示，该物品摆放状态获取系统900能够与处理系统(如参考图1描述的云端系统102)交互，包括获取请求发送模块910与摆放状态接收模块920。
其中，获取请求发送模块910用于发送获取请求，该获取请求包括一个或多个关联关系，每一个关联关系包括一个热区与一个摄像设备之间的关联关系，一个热区对应一种物品，其中摄像设备用于获取与其具有关联关系的热区的当前图像。根据本公开的实施例，该获取请求发送模块810例如可以执行参考图2A描述的操作S210，在此不再赘述。
其中，摆放状态接收模块920用于接收至少一个热区对应的物品的摆放状态，该至少一个热区对应的物品的摆放状态由处理系统响应于获取请求，根据至少一个热区的当前图像及标准图像确定。其中，根据本公开的实施例，该摆放状态接收模块920例如可以执行参考图2A描述的操作S220，在此不再赘述。
根据本公开的实施例，如图9B所示，该物品摆放状态获取系统900例如还可以包括参数设定模块930，该参数设定模块930用于设定获取请求包括的一个或多个关联关系。根据本公开的实施例，该参数设定模块930例如可以执行参考图2B描述的操作S230，在此不再赘述。
根据本公开的实施例，该参数设定模块930例如还可以用于设定摄像设备获取与其具有关联关系的热区的当前图像的周期，并且/或者，摆放状 态接收模块920例如还可以用于设定接收至少一个热区对应的物品的摆放状态的周期，其中，接收至少一个热区对应的物品的摆放状态的周期，不小于所述摄像设备获取与其具有关联关系的热区的当前图像的周期。
根据本公开的实施例，如图9C所示，该物品摆放状态获取系统900例如还可以包括展示模块940，该展示模块940用于展示至少一个热区中一个或多个热区对应的物品的摆放状态。根据本公开的实施例，该展示模块940例如可以执行参考图2B描述的操作S240，在此不再赘述。
根据本公开的实施例，如图9D所示，该物品摆放状态获取系统900例如还可以包括筛选模块950，该筛选模块950用于根据用户输入的筛选条件，自所述至少一个热区对应的物品的摆放状态中筛选得到与所筛选条件匹配的、一个或多个热区对应的物品的摆放状态以供展示。根据本公开的实施例，该筛选模块950例如可以用于执行参考图3A描述的操作S350，相应地，展示模块940例如还可以用于执行参考图3A描述的操作S360，在此不再赘述。
根据本公开的实施例，所述至少一个热区对应的物品的摆放状态包括正常状态和非正常状态，如图9E所示，物品摆放状态获取系统900例如还可以包括报警模块960，该报警模块960用于在所述至少一个热区中的一个或多个热区对应的物品的摆放状态为非正常状态的情况下，向终端设备发送报警信号。根据本公开的实施例，该报警模块960例如可以执行参考图3B描述的操作S370，在此不再赘述。
根据本公开的实施例，上述与摄像设备具有关联关系的热区的当前图像的获取包括：获取与摄像设备具有关联关系的热区的多张实时图像，该多张实时图像是按时间间隔连续获取的；识别多张实时图像，得到多张实时图像中包括的物品数量；以及确定与所述摄像设备具有关联关系的热区的当前图像，该当前图像是多张实时图像中包括的物品数量最多的实时图像，其中，所述摄像设备与一个或多个热区具有关联关系。根据本公开的实施例，与摄像设备具有关联关系的热区的当前图像的获取例如可以通过参考图5描述的操作S510～操作S530得到，在此不再赘述。
根据本公开的实施例，该物品摆放状态获取系统900例如可以设置于参考图1描述的终端设备101中。
根据本公开的实施例的模块、子模块、单元、子单元中的任意多个、或其中任意多个的至少部分功能可以在一个模块中实现。根据本公开实施例的模块、子模块、单元、子单元中的任意一个或多个可以被拆分成多个模块来实现。根据本公开实施例的模块、子模块、单元、子单元中的任意一个或多个可以至少被部分地实现为硬件电路，例如现场可编程门阵列(FPGA)、可编程逻辑阵列(PLA)、片上系统、基板上的系统、封装上的系统、专用集成电路(ASIC)，或可以通过对电路进行集成或封装的任何其他的合理方式的硬件或固件来实现，或以软件、硬件以及固件三种实现方式中任意一种或以其中任意几种的适当组合来实现。或者，根据本公开实施例的模块、子模块、单元、子单元中的一个或多个可以至少被部分地实现为计算机程序模块，当该计算机程序模块被运行时，可以执行相应的功能。
例如，获取请求发送模块910、摆放状态接收模块920、参数设定模块930、展示模块940、筛选模块950以及报警模块960中的任意多个可以合并在一个模块中实现，或者其中的任意一个模块可以被拆分成多个模块。或者，这些模块中的一个或多个模块的至少部分功能可以与其他模块的至少部分功能相结合，并在一个模块中实现。根据本公开的实施例，获取请求发送模块910、摆放状态接收模块920、参数设定模块930、展示模块940、筛选模块950以及报警模块960中的至少一个可以至少被部分地实现为硬件电路，例如现场可编程门阵列(FPGA)、可编程逻辑阵列(PLA)、片上系统、基板上的系统、封装上的系统、专用集成电路(ASIC)，或可以通过对电路进行集成或封装的任何其他的合理方式等硬件或固件来实现，或以软件、硬件以及固件三种实现方式中任意一种或以其中任意几种的适当组合来实现。或者，获取请求发送模块910、摆放状态接收模块920、参数设定模块930、展示模块940、筛选模块950以及报警模块960中的至少一个可以至少被部分地实现为计算机程序模块，当该计算机程序模块被运行时，可以执行相应的功能。
图10A～图10C示意性示出了根据本公开实施例的处理系统的结构框图。
如图10A所示，该处理系统1000例如可以与参考图9A～图9E中描述 的物品摆放状态获取系统交互，该处理系统1000包括获取请求接收模块1010、图像获取模块1020和摆放状态确定模块1030。
其中，获取请求接收模块1010用于接收物品摆放状态获取系统发送的获取请求，该获取请求包括一个或多个关联关系，每一个关联关系包括一个热区与一个摄像设备之间的关联关系，一个热区对应一种物品，摄像设备用于获取与其具有关联关系的热区的当前图像。根据本公开的实施例，该获取请求接收模块1010例如可以用于执行参考图6描述的操作S610，在此不再赘述。
其中，图像获取模块1020用于响应于获取请求，获取至少一个热区的当前图像及标准图像。根据本公开的实施例，该图像获取模块1020例如可以用于执行参考图6描述的操作S620，在此不再赘述。
根据本公开的实施例，与摄像设备具有关联关系的热区的当前图像的获取包括：获取与所述摄像设备具有关联关系的热区的多张实时图像，该多张实时图像是按时间间隔连续获取的；识别所述多张实时图像，得到所述多张实时图像中包括的物品数量；以及确定与所述摄像设备具有关联关系的热区的当前图像，该当前图像是所述多张实时图像中包括的物品数量最多的实时图像，其中，所述摄像设备与一个或多个热区具有关联关系。根据本公开的实施例，该当前图像的获取例如可以通过参考图5描述的操作得到，在此不再赘述。
其中，摆放状态确定模块1030用于根据所述至少一个热区的当前图像及标准图像，确定至少一个热区对应的物品的摆放状态。根据本公开的实施例，该摆放状态确定模块1030例如可以用于执行参考图6描述的操作S630，在此不再赘述。
根据本公开的实施例，如图10B所示，该摆放状态确定模块1030例如可以包括图像特征识别子模块1031、物品特征确定子模块1032和摆放状态确定子模块1033。
图像特征识别子模块1031用于识别所述至少一个热区的当前图像，得到所述至少一个热区的当前图像包括的图像特征；物品特征确定子模块1032用于基于所述至少一个热区的当前图像包括的图像特征，确定所述至少一个热区对应的物品特征；摆放状态确定子模块1033用于对比所述至 少一个热区对应的物品特征与所述至少一个热区对应的标准物品特征，确定所述至少一个热区对应的物品的摆放状态，其中，所述至少一个热区对应的标准物品特征可以采用上述图像特征识别子模块1031和物品特征确定子模块1032，基于所述至少一个热区的标准图像得到的。根据本公开的实施例，图像特征识别子模块1031、物品特征确定子模块1032和摆放状态确定子模块1033例如可以分别用于执行参考图7描述的操作S631～操作S633，在此不再赘述。
根据本公开的实施例，该物品的摆放状态包括正常状态和非正常状态，该非正常状态包括以下情况的至少之一：对应的物品特征中，存在与对应的标准物品特征中的标准物品名称不同的物品名称的情况；对应的物品特征中与对应的标准物品特征的标准物品名称相同的物品名称对应的物品数量，与所述标准物品名称对应的标准物品数量的比值小于第一比值的情况；或对应的物品特征中与对应的标准物品特征的标准物品名称相同的物品名称对应的物品标志中，与标准物品名称对应的标准物品标志的相似度低于预设相似度的物品标志数量，与该物品名称对应的物品标志总数量的比值大于第二比值的情况，其中，所述物品特征包括以下至少之一：物品名称、物品名称对应的物品数量、物品名称对应的物品标志和/或物品名称对应的物品位置。根据本公开的实施例，该非正常状态例如可以包括参考图8A～参考图8C描述的非正常的错放状态、非正常的缺货状态或非正常的不规范问题，在此不再赘述。
根据本公开的实施例，该处理系统例如可以是参考图1描述的云端系统102，该云端系统102与摄像设备能够交互，以能够接收摄像设备上传的热区的实时图像。
根据本公开的实施例，如图10C所示，该处理系统例如还可以包括图像存储模块1040，该图像存储模块1040用于存储所述摄像设备获取的与其具有关联关系的热区的当前图像及所述至少一个热区的标准图像，以供所述图像获取模块1020获取。
根据本公开的实施例的模块、子模块、单元、子单元中的任意多个、或其中任意多个的至少部分功能可以在一个模块中实现。根据本公开实施例的模块、子模块、单元、子单元中的任意一个或多个可以被拆分成多个 模块来实现。根据本公开实施例的模块、子模块、单元、子单元中的任意一个或多个可以至少被部分地实现为硬件电路，例如现场可编程门阵列(FPGA)、可编程逻辑阵列(PLA)、片上系统、基板上的系统、封装上的系统、专用集成电路(ASIC)，或可以通过对电路进行集成或封装的任何其他的合理方式的硬件或固件来实现，或以软件、硬件以及固件三种实现方式中任意一种或以其中任意几种的适当组合来实现。或者，根据本公开实施例的模块、子模块、单元、子单元中的一个或多个可以至少被部分地实现为计算机程序模块，当该计算机程序模块被运行时，可以执行相应的功能。
例如，获取请求接收模块1010、图像获取模块1020、摆放状态确定模块1030、图像存储模块1040、图像特征识别子模块1031、物品特征确定子模块1032、以及摆放状态确定子模块1033中的任意多个可以合并在一个模块中实现，或者其中的任意一个模块可以被拆分成多个模块。或者，这些模块中的一个或多个模块的至少部分功能可以与其他模块的至少部分功能相结合，并在一个模块中实现。根据本公开的实施例，获取请求接收模块1010、图像获取模块1020、摆放状态确定模块1030、图像存储模块1040、图像特征识别子模块1031、物品特征确定子模块1032、以及摆放状态确定子模块1033中的至少一个可以至少被部分地实现为硬件电路，例如现场可编程门阵列(FPGA)、可编程逻辑阵列(PLA)、片上系统、基板上的系统、封装上的系统、专用集成电路(ASIC)，或可以通过对电路进行集成或封装的任何其他的合理方式等硬件或固件来实现，或以软件、硬件以及固件三种实现方式中任意一种或以其中任意几种的适当组合来实现。或者，获取请求接收模块1010、图像获取模块1020、摆放状态确定模块1030、图像存储模块1040、图像特征识别子模块1031、物品特征确定子模块1032、以及摆放状态确定子模块1033中的至少一个可以至少被部分地实现为计算机程序模块，当该计算机程序模块被运行时，可以执行相应的功能。
图11示意性示出了根据本公开实施例的适于实现上文描述的物品摆放状态获取方法或处理方法的计算机系统的方框图。图11示出的计算机系统仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限 制。
如图11所示，根据本公开实施例的计算机系统1100包括处理器1101，其可以根据存储在只读存储器(ROM)1102中的程序或者从存储部分1108加载到随机访问存储器(RAM)1103中的程序而执行各种适当的动作和处理。处理器1101例如可以包括通用微处理器(例如CPU)、指令集处理器和/或相关芯片组和/或专用微处理器(例如，专用集成电路(ASIC))，等等。处理器1101还可以包括用于缓存用途的板载存储器。处理器1101可以包括用于执行根据本公开实施例的方法流程的不同动作的单一处理单元或者是多个处理单元。
在RAM 1103中，存储有系统1100操作所需的各种程序和数据。处理器1101、ROM 1102以及RAM 1103通过总线1104彼此相连。处理器1101通过执行ROM 1102和/或RAM 1103中的程序来执行根据本公开实施例的方法流程的各种操作。需要注意，所述程序也可以存储在除ROM1102和RAM 1103以外的一个或多个存储器中。处理器1101也可以通过执行存储在所述一个或多个存储器中的程序来执行根据本公开实施例的方法流程的各种操作。
根据本公开的实施例，系统1100还可以包括输入/输出(I/O)接口1105，输入/输出(I/O)接口1105也连接至总线1104。系统1100还可以包括连接至I/O接口1105的以下部件中的一项或多项：包括键盘、鼠标等的输入部分1106；包括诸如阴极射线管(CRT)、液晶显示器(LCD)等以及扬声器等的输出部分1107；包括硬盘等的存储部分1108；以及包括诸如LAN卡、调制解调器等的网络接口卡的通信部分1109。通信部分1109经由诸如因特网的网络执行通信处理。驱动器1110也根据需要连接至I/O接口1105。可拆卸介质1111，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器1110上，以便于从其上读出的计算机程序根据需要被安装入存储部分1108。
根据本公开的实施例，根据本公开实施例的方法流程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通 信部分1109从网络上被下载和安装，和/或从可拆卸介质1111被安装。在该计算机程序被处理器1101执行时，执行本公开实施例的系统中限定的上述功能。根据本公开的实施例，上文描述的系统、设备、装置、模块、单元等可以通过计算机程序模块来实现。
本公开还提供了一种计算机可读介质，该计算机可读介质可以是上述实施例中描述的设备/装置/系统中所包含的；也可以是单独存在，而未装配入该设备/装置/系统中。上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被执行时，实现根据本公开实施例的方法。
根据本公开的实施例，计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：无线、有线、光缆、射频信号等等，或者上述的任意合适的组合。
例如，根据本公开的实施例，计算机可读介质可以包括上文描述的ROM 1102和/或RAM 1103和/或ROM 1102和RAM 1103以外的一个或多个存储器。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法 和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，上述模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图或流程图中的每个方框、以及框图或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
本领域技术人员可以理解，本公开的各个实施例和/或权利要求中记载的特征可以进行多种组合或/或结合，即使这样的组合或结合没有明确记载于本公开中。特别地，在不脱离本公开精神和教导的情况下，本公开的各个实施例和/或权利要求中记载的特征可以进行多种组合和/或结合。所有这些组合和/或结合均落入本公开的范围。
以上对本公开的实施例进行了描述。但是，这些实施例仅仅是为了说明的目的，而并非为了限制本公开的范围。尽管在以上分别描述了各实施例，但是这并不意味着各个实施例中的措施不能有利地结合使用。本公开的范围由所附权利要求及其等同物限定。不脱离本公开的范围，本领域技术人员可以做出多种替代和修改，这些替代和修改都应落在本公开的范围之内。

Claims (21)

  1. 一种物品摆放状态获取方法，包括：
    发送获取请求，所述获取请求包括一个或多个关联关系，每一个所述关联关系包括一个热区与一个摄像设备之间的关联关系，一个热区对应一种物品；以及
    接收至少一个热区对应的物品的摆放状态，所述至少一个热区对应的物品的摆放状态是响应于所述获取请求，根据所述至少一个热区的当前图像及标准图像确定的，
    其中，所述摄像设备用于获取与其具有关联关系的热区的当前图像。
  2. 根据权利要求1所述的方法，还包括：
    设定所述一个或多个关联关系；并且/或者
    展示所述至少一个热区中一个或多个热区对应的物品的摆放状态；并且/或者
    根据用户输入的筛选条件，自所述至少一个热区对应的物品的摆放状态中筛选得到与所述筛选条件匹配的、一个或多个热区对应的物品的摆放状态，
    其中，所述筛选条件包括以下至少之一：摄像设备编号、热区编号、物品名称和/或物品的摆放状态。
  3. 根据权利要求1所述的方法，还包括：
    设定所述摄像设备获取与其具有关联关系的热区的当前图像的周期；并且/或者
    设定接收所述至少一个热区对应的物品的摆放状态的周期，
    其中，接收所述至少一个热区对应的物品的摆放状态的周期，不小于所述摄像设备获取与其具有关联关系的热区的当前图像的周期。
  4. 根据权利要求1所述的方法，其中，一个热区对应的物品的摆放状态包括正常状态和非正常状态，所述方法还包括：
    在所述至少一个热区中的一个或多个热区对应的物品的摆放状 态为非正常状态的情况下，向终端设备发送报警信号。
  5. 根据权利要求1所述的方法，其中，与摄像设备具有关联关系的热区的当前图像的获取包括：
    获取与所述摄像设备具有关联关系的热区的多张实时图像，该多张实时图像是按时间间隔连续获取的；
    识别所述多张实时图像，得到所述多张实时图像中包括的物品数量；以及
    确定与所述摄像设备具有关联关系的热区的当前图像，该当前图像是所述多张实时图像中包括的物品数量最多的实时图像，
    其中，所述摄像设备与一个或多个热区具有关联关系。
  6. 一种处理方法，包括：
    接收获取请求，所述获取请求包括一个或多个关联关系，每一个所述关联关系包括一个热区与一个摄像设备之间的关联关系，一个热区对应一种物品；
    响应于所述获取请求，获取至少一个热区的当前图像及标准图像；以及
    根据所述至少一个热区的当前图像及标准图像，确定所述至少一个热区对应的物品的摆放状态，
    其中，所述摄像设备用于获取与其具有关联关系的热区的当前图像。
  7. 根据权利要求6所述的方法，其中，根据所述至少一个热区的当前图像及标准图像，确定所述至少一个热区对应的物品的摆放状态包括：
    识别所述至少一个热区的当前图像，得到所述至少一个热区的当前图像包括的图像特征；
    基于所述至少一个热区的当前图像包括的图像特征，确定所述至少一个热区对应的物品特征；以及
    对比所述至少一个热区对应的物品特征与所述至少一个热区对应的标准物品特征，确定所述至少一个热区对应的物品的摆放状态，
    其中，所述至少一个热区对应的标准物品特征是基于所述至少一 个热区的标准图像得到的。
  8. 根据权利要求7所述的方法，其中，一个热区对应的物品的摆放状态包括正常状态和非正常状态，所述非正常状态包括以下情况的至少之一：
    对应的物品特征中，存在与对应的标准物品特征中的标准物品名称不同的物品名称的情况；
    对应的物品特征中与对应的标准物品特征的标准物品名称相同的物品名称对应的物品数量，与所述标准物品名称对应的标准物品数量的比值小于第一比值的情况；或
    对应的物品特征中与对应的标准物品特征的标准物品名称相同的物品名称对应的物品标志中，与标准物品名称对应的标准物品标志的相似度低于预设相似度的物品标志数量，与该物品名称对应的物品标志总数量的比值大于第二比值的情况，
    其中，所述物品特征包括以下至少之一：物品名称、物品名称对应的物品数量、物品名称对应的物品标志和/或物品名称对应的物品位置。
  9. 根据权利要求6所述的方法，其中，与摄像设备具有关联关系的热区的当前图像的获取包括：
    获取与所述摄像设备具有关联关系的热区的多张实时图像，该多张实时图像是按时间间隔连续获取的；
    识别所述多张实时图像，得到所述多张实时图像中包括的物品数量；以及
    确定与所述摄像设备具有关联关系的热区的当前图像，该当前图像是所述多张实时图像中包括的物品数量最多的实时图像，
    其中，所述摄像设备与一个或多个热区具有关联关系。
  10. 一种物品摆放状态获取系统，能够与处理系统交互，该物品摆放状态获取系统包括：
    获取请求发送模块，发送获取请求，所述获取请求包括一个或多个关联关系，每一个所述关联关系包括一个热区与一个摄像设备之间的关联关系，一个热区对应一种物品；以及
    摆放状态接收模块，接收至少一个热区对应的物品的摆放状态，所述至少一个热区对应的物品的摆放状态由所述处理系统响应于所述获取请求，根据所述至少一个热区的当前图像及标准图像确定，
    其中，所述摄像设备用于获取与其具有关联关系的热区的当前图像。
  11. 根据权利要求10所述的系统，还包括：
    参数设定模块，设定所述一个或多个关联关系；并且/或者
    展示模块，展示所述至少一个热区中一个或多个热区对应的物品的摆放状态；并且/或者
    筛选模块，根据用户输入的筛选条件，自所述至少一个热区对应的物品的摆放状态中筛选得到与所述筛选条件匹配的、一个或多个热区对应的物品的摆放状态以供展示，
    其中，所述筛选条件包括以下至少之一：摄像设备编号、热区编号、物品名称和/或物品的摆放状态。
  12. 根据权利要求11所述的系统，其中：
    所述参数设定模块，还用于设定所述摄像设备获取与其具有关联关系的热区的当前图像的周期；并且/或者
    所述摆放状态获取模块，还用于设定接收所述至少一个热区对应的物品的摆放状态的周期，
    其中，接收所述至少一个热区对应的物品的摆放状态的周期，不小于所述摄像设备获取与其具有关联关系的热区的当前图像的周期。
  13. 根据权利要求10所述的系统，其中，所述至少一个热区对应的物品的摆放状态包括正常状态和非正常状态，所述系统还包括：
    报警模块，在所述至少一个热区中的一个或多个热区对应的物品的摆放状态为非正常状态的情况下，向终端设备发送报警信号。
  14. 根据权利要求10所述的系统，其中，与摄像设备具有关联关系的热区的当前图像的获取包括：
    获取与所述摄像设备具有关联关系的热区的多张实时图像，该多张实时图像是按时间间隔连续获取的；
    识别所述多张实时图像，得到所述多张实时图像中包括的物品数 量；以及
    确定与所述摄像设备具有关联关系的热区的当前图像，该当前图像是所述多张实时图像中包括的物品数量最多的实时图像，
    其中，所述摄像设备与一个或多个热区具有关联关系。
  15. 一种处理系统，能够与物品摆放状态获取系统交互，该处理系统包括：
    获取请求接收模块，接收所述物品摆放状态获取系统发送的获取请求，所述获取请求包括一个或多个关联关系，每一个所述关联关系包括一个热区与一个摄像设备之间的关联关系，一个热区对应一种物品；
    图像获取模块，响应于所述获取请求，获取至少一个热区的当前图像及标准图像；以及
    摆放状态确定模块，根据所述至少一个热区的当前图像及标准图像，确定所述至少一个热区对应的物品的摆放状态，
    其中，所述摄像设备用于获取与其具有关联关系的热区的当前图像。
  16. 根据权利要求15所述的系统，其中，所述摆放状态确定模块包括：
    图像特征识别子模块，识别所述至少一个热区的当前图像，得到所述至少一个热区的当前图像包括的图像特征；
    物品特征确定子模块，基于所述至少一个热区的当前图像包括的图像特征，确定所述至少一个热区对应的物品特征；以及
    摆放状态确定子模块，对比所述至少一个热区对应的物品特征与所述至少一个热区对应的标准物品特征，确定所述至少一个热区对应的物品的摆放状态，
    其中，所述至少一个热区对应的标准物品特征是基于所述至少一个热区的标准图像得到的。
  17. 根据权利要求16所述的系统，其中，所述物品的摆放状态包括正常状态和非正常状态，所述非正常状态包括以下情况的至少之一：
    对应的物品特征中，存在与对应的标准物品特征中的标准物品名称不同的物品名称的情况；
    对应的物品特征中与对应的标准物品特征的标准物品名称相同的物品名称对应的物品数量，与所述标准物品名称对应的标准物品数量的比值小于第一比值的情况；或
    对应的物品特征中与对应的标准物品特征的标准物品名称相同的物品名称对应的物品标志中，与标准物品名称对应的标准物品标志的相似度低于预设相似度的物品标志数量，与该物品名称对应的物品标志总数量的比值大于第二比值的情况，
    其中，所述物品特征包括以下至少之一：物品名称、物品名称对应的物品数量、物品名称对应的物品标志和/或物品名称对应的物品位置。
  18. 根据权利要求15所述的系统，其中：
    所述处理系统为云端系统；并且/或者
    所述系统还包括：
    图像存储模块，存储所述摄像设备获取的与其具有关联关系的热区的当前图像及所述至少一个热区的标准图像，以供所述图像获取模块获取。
  19. 根据权利要求15所述的系统，其中，与摄像设备具有关联关系的热区的当前图像的获取包括：
    获取与所述摄像设备具有关联关系的热区的多张实时图像，该多张实时图像是按时间间隔连续获取的；
    识别所述多张实时图像，得到所述多张实时图像中包括的物品数量；以及
    确定与所述摄像设备具有关联关系的热区的当前图像，该当前图像是所述多张实时图像中包括的物品数量最多的实时图像，
    其中，所述摄像设备与一个或多个热区具有关联关系。
  20. 一种电子设备，包括：
    一个或多个处理器；
    存储装置，用于存储一个或多个程序，
    其中，当所述一个或多个程序被所述一个或多个处理器执行时，使得所述一个或多个处理器执行根据权利要求1～9任意一项所述的方法。
  21. 一种计算机可读介质，其上存储有可执行指令，该指令被处理器执行时使处理器执行权利要求1～9任意一项所述的方法。
PCT/CN2019/097936 2018-07-27 2019-07-26 物品摆放状态获取方法和系统 WO2020020355A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/263,847 US11861910B2 (en) 2018-07-27 2019-07-26 Method and system for acquiring item placing state

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810851381.2A CN110765825A (zh) 2018-07-27 2018-07-27 物品摆放状态获取方法和系统
CN201810851381.2 2018-07-27

Publications (1)

Publication Number Publication Date
WO2020020355A1 true WO2020020355A1 (zh) 2020-01-30

Family

ID=69180840

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/097936 WO2020020355A1 (zh) 2018-07-27 2019-07-26 物品摆放状态获取方法和系统

Country Status (3)

Country Link
US (1) US11861910B2 (zh)
CN (1) CN110765825A (zh)
WO (1) WO2020020355A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111736740A (zh) * 2020-03-24 2020-10-02 北京沃东天骏信息技术有限公司 图片热区生成方法、装置、设备和存储介质

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11915192B2 (en) 2019-08-12 2024-02-27 Walmart Apollo, Llc Systems, devices, and methods for scanning a shopping space
CN115493343B (zh) * 2021-06-18 2023-10-27 海信集团控股股份有限公司 一种冰箱及其食材管理方法及装置

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107077659A (zh) * 2016-09-26 2017-08-18 达闼科技（北京）有限公司 一种智能库存管理系统、服务器、方法、终端和程序产品
CN107393152A (zh) * 2017-08-14 2017-11-24 杭州纳戒科技有限公司 自助售货机及自助售货系统
CN107730168A (zh) * 2017-09-28 2018-02-23 中南大学 一种基于图像识别的自动售货机自动库存管理系统及方法

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11126259A (ja) * 1997-06-02 1999-05-11 Matsushita Electric Ind Co Ltd 画像検出方法、画像検出装置、画像処理方法、画像処理装置、及び媒体
CN102830958B (zh) * 2011-06-16 2017-11-24 奇智软件（北京）有限公司 一种获取界面控件信息的方法及系统
US9020189B2 (en) * 2012-12-07 2015-04-28 The Nielsen Company (Us), Llc Methods and apparatus to monitor environments
CN103150631B (zh) * 2013-03-11 2016-11-09 苏州斯普锐智能系统有限公司 基于图像匹配的自动物品管理方法及其系统
US20150310601A1 (en) * 2014-03-07 2015-10-29 Digimarc Corporation Methods and arrangements for identifying objects
US20160012453A1 (en) * 2014-07-11 2016-01-14 Shamim A. Naqvi System and Method for Inferring the Intent of a User While Receiving Signals On a Mobile Communication Device From a Broadcasting Device
US9571985B2 (en) * 2015-04-09 2017-02-14 Datalogic Ip Tech S.R.L. High accuracy indoor locating system
CN105374031A (zh) * 2015-10-14 2016-03-02 江苏美的清洁电器股份有限公司 基于机器人的家庭安防数据处理方法及系统
US20170262783A1 (en) * 2016-03-08 2017-09-14 International Business Machines Corporation Team Formation
CN105868772A (zh) * 2016-03-23 2016-08-17 百度在线网络技术（北京）有限公司 图像识别方法及装置
CN105872595A (zh) * 2016-03-31 2016-08-17 乐视控股（北京）有限公司 转码任务分配方法及装置
CN107862247B (zh) * 2017-10-13 2018-09-11 平安科技（深圳）有限公司 一种人脸活体检测方法及终端设备

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107077659A (zh) * 2016-09-26 2017-08-18 达闼科技（北京）有限公司 一种智能库存管理系统、服务器、方法、终端和程序产品
CN107393152A (zh) * 2017-08-14 2017-11-24 杭州纳戒科技有限公司 自助售货机及自助售货系统
CN107730168A (zh) * 2017-09-28 2018-02-23 中南大学 一种基于图像识别的自动售货机自动库存管理系统及方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111736740A (zh) * 2020-03-24 2020-10-02 北京沃东天骏信息技术有限公司 图片热区生成方法、装置、设备和存储介质

Also Published As

Publication number Publication date
US11861910B2 (en) 2024-01-02
US20210319684A1 (en) 2021-10-14
CN110765825A (zh) 2020-02-07

Similar Documents

Publication Publication Date Title
WO2020020355A1 (zh) 物品摆放状态获取方法和系统
US11282129B2 (en) Order processing and order-picking task processing method, server, order-picking terminal, and electronic price tag
WO2020010827A1 (zh) 一种基于开放式自动售货机的购物结算方法和装置
WO2019057168A1 (zh) 一种货物订单处理方法、装置、服务器、购物终端及系统
WO2018184301A1 (zh) 网点销售服务系统、方法、服务器和存储介质
GB2544871A (en) Intelligent display system and method
WO2019233098A1 (zh) 一种商品购买判定方法、装置和用户终端
WO2016134569A1 (zh) 信息展示方法和装置
WO2019101088A1 (zh) 一种无人售货方法和设备
WO2019192231A1 (zh) 商品定位方法及装置、设备和存储介质
CN104268770A (zh) 超市到访顾客特征分析系统
CN106909365A (zh) 资讯信息的展示方法和装置
WO2020107951A1 (zh) 一种基于图像的商品结算方法、装置、介质及电子设备
CN112200631B (zh) 一种行业分类模型的训练方法及装置
JP2021517315A (ja) データ処理方法、装置、電子機器、プログラム、及び記憶媒体
US20210049663A1 (en) Product information query method and system
CN112488602A (zh) 补货方法、装置和系统
WO2021179138A1 (zh) 商超货架上商品的分析方法和系统
CN110263190B (zh) 一种数据处理方法、装置、设备和机器可读介质
CN109146576A (zh) 店铺货物摆放策略的确定方法、装置和电子设备
WO2021233058A1 (zh) 监控货架上的物品的方法、计算机和系统
WO2021147950A1 (zh) 货品识别方法、货品识别系统及电子设备
CN108154139B (zh) 商品热区探测系统及方法、电子设备、存储介质
CN110503226B (zh) 确定待放置商品的方法、装置和系统
CN111753614A (zh) 一种商品货架的监控方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19840875

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19840875

Country of ref document: EP

Kind code of ref document: A1