CN109671227B - Intelligent container consumption behavior identification method and device, storage medium and electronic equipment - Google Patents
Intelligent container consumption behavior identification method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN109671227B CN109671227B CN201811347542.0A CN201811347542A CN109671227B CN 109671227 B CN109671227 B CN 109671227B CN 201811347542 A CN201811347542 A CN 201811347542A CN 109671227 B CN109671227 B CN 109671227B
- Authority
- CN
- China
- Prior art keywords
- commodity
- change information
- intelligent container
- image
- consumption behavior
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 230000006399 behavior Effects 0.000 claims abstract description 123
- 230000002159 abnormal effect Effects 0.000 claims abstract description 72
- 230000004927 fusion Effects 0.000 claims abstract description 21
- 238000004590 computer program Methods 0.000 claims description 9
- 230000001066 destructive effect Effects 0.000 claims description 7
- 238000004891 communication Methods 0.000 description 11
- 230000005484 gravity Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000003384 imaging method Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 238000005259 measurement Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000035622 drinking Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- KLDZYURQCUYZBL-UHFFFAOYSA-N 2-[3-[(2-hydroxyphenyl)methylideneamino]propyliminomethyl]phenol Chemical compound OC1=CC=CC=C1C=NCCCN=CC1=CC=CC=C1O KLDZYURQCUYZBL-UHFFFAOYSA-N 0.000 description 1
- 235000013361 beverage Nutrition 0.000 description 1
- 201000001098 delayed sleep phase syndrome Diseases 0.000 description 1
- 208000033921 delayed sleep phase type circadian rhythm sleep disease Diseases 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
- G07G1/00—Cash registers
- G07G1/0036—Checkout procedures
- G07G1/0045—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/35—Categorising the entire scene, e.g. birthday party or wedding scene
- G06V20/36—Indoor scenes
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The disclosure relates to a method, a device, a storage medium and electronic equipment for identifying consumption behaviors of an intelligent container, belongs to the field of intelligent containers, and can detect abnormal consumption behaviors such as consumption fraud or misoperation in actual use of the intelligent container. The method comprises the following steps: acquiring weight change information and image change information of commodities of an intelligent container; and identifying the consumption behavior of the commodity of the intelligent container based on the fusion analysis of the weight change information and the image change information.
Description
Technical Field
The disclosure relates to the field of intelligent containers, in particular to a method and a device for identifying consumption behaviors of an intelligent container.
Background
Currently, when judging the consumption behavior of intelligent containers, either gravity scales are used to detect weight changes to estimate the removed goods, or visual recognition techniques are used to detect the types and numbers of the removed goods through image recognition. However, both of these solutions cannot solve the problems of consumer fraud or misoperation in actual use, such as replacing a genuine commodity with a fake commodity (e.g., a model), replacing a bottle empty after drinking, replacing a commodity with an equivalent weight, replacing the commodity in a reverse position, placing the commodity in a blocked position (the commodity can be extracted from the blocked position later), and tipping the commodity.
Disclosure of Invention
The invention aims to provide a method, a device, a storage medium and electronic equipment for identifying consumption behaviors of an intelligent container, which can detect abnormal consumption behaviors such as consumption fraud or misoperation in actual use of the intelligent container.
According to a first embodiment of the present disclosure, there is provided an intelligent container consumption behavior recognition method, the method including: acquiring weight change information and image change information of commodities of an intelligent container; and identifying the consumption behavior of the commodity of the intelligent container based on the fusion analysis of the weight change information and the image change information.
Optionally, the image variation information includes built-in image variation information obtained based on an image captured by a camera disposed inside the intelligent container, and the identifying the consumption behavior of the commodity of the intelligent container based on the fusion analysis of the weight variation information and the image variation information includes: and identifying the consumption behavior of the commodity in the intelligent container as abnormal consumption under the condition that the weight change information indicates that the commodity in the intelligent container is changed and the built-in image change information indicates that the commodity in the intelligent container is not changed.
Optionally, the image variation information includes built-in image variation information obtained based on an image captured by a camera disposed inside the intelligent container, and the identifying the consumption behavior of the commodity of the intelligent container based on the fusion analysis of the weight variation information and the image variation information includes: and identifying the consumption behavior of the commodity in the intelligent container as abnormal consumption under the condition that the weight change information indicates that the commodity in the intelligent container is unchanged and the built-in image change information indicates that the commodity in the intelligent container is changed.
Optionally, the image variation information includes built-in image variation information obtained based on an image captured by a camera disposed inside the intelligent container and external image variation information obtained based on an image captured by a camera disposed outside the intelligent container, and the identifying the consumption behavior of the commodity of the intelligent container based on the fusion analysis of the weight variation information and the image variation information includes: and identifying whether the consumption behavior of the commodity in the intelligent container is abnormal consumption or normal consumption based on the external image change information under the condition that the weight change information indicates that the commodity in the intelligent container is unchanged and the built-in image change information also indicates that the commodity in the intelligent container is unchanged.
Optionally, the image change information further includes external image change information obtained based on an image captured by a camera disposed outside the intelligent container, and the method further includes: the types of abnormal consumption are further distinguished based on the external image change information, wherein the types of abnormal consumption include destructive abnormal consumption and unintentional abnormal consumption.
Optionally, the image variation information includes built-in image variation information obtained based on an image captured by a camera disposed inside the intelligent container, and the identifying the consumption behavior of the commodity of the intelligent container based on the fusion analysis of the weight variation information and the image variation information includes: in case the weight change information indicates a change in the commodity in the intelligent container and the built-in image change information also indicates a change in the commodity in the intelligent container,
If the commodity consumption behavior obtained based on the weight change information is consistent with the commodity consumption behavior obtained based on the built-in image change information, recognizing that the commodity consumption behavior in the intelligent container is normal consumption;
And if the commodity consumption behavior obtained based on the weight change information is inconsistent with the commodity consumption behavior obtained based on the built-in image change information, identifying the consumption behavior of the commodity in the intelligent container as abnormal consumption.
According to a second embodiment of the present disclosure, there is provided an intelligent container consumption behavior recognition apparatus, the apparatus including: the acquisition module is used for acquiring the weight change information and the image change information of the commodity of the intelligent container; and the identification module is used for identifying the consumption behavior of the commodity of the intelligent container based on the fusion analysis of the weight change information and the image change information.
Optionally, the image change information includes built-in image change information obtained based on an image captured by a camera disposed inside the intelligent container, and the identification module is further configured to identify the consumption behavior of the commodity in the intelligent container as abnormal consumption if the weight change information indicates that the commodity in the intelligent container is changed and the built-in image change information indicates that the commodity in the intelligent container is not changed.
Optionally, the image change information includes built-in image change information obtained based on an image captured by a camera disposed inside the intelligent container, and the identification module is further configured to identify the consumption behavior of the commodity inside the intelligent container as abnormal consumption if the weight change information indicates that the commodity inside the intelligent container is unchanged and the built-in image change information indicates that the commodity inside the intelligent container is changed.
Optionally, the image change information further includes external image change information obtained based on an image captured by a camera disposed outside the intelligent container, and the identification module is further configured to further distinguish the type of abnormal consumption based on the external image change information, wherein the type of abnormal consumption includes destructive abnormal consumption and unintentional abnormal consumption.
Optionally, the image change information includes built-in image change information obtained based on an image captured by a camera disposed inside the intelligent container and external image change information obtained based on an image captured by a camera disposed outside the intelligent container, and the identification module is further configured to identify whether the consumption behavior of the commodity in the intelligent container is abnormal consumption or normal consumption based on the external image change information in a case where the weight change information indicates that the commodity in the intelligent container is unchanged and the built-in image change information also indicates that the commodity in the intelligent container is unchanged.
Optionally, the image change information includes built-in image change information obtained based on an image captured by a camera disposed inside the intelligent container, and the identification module is further configured to, in a case where the weight change information indicates that the commodity inside the intelligent container is changed and the built-in image change information also indicates that the commodity inside the intelligent container is changed:
if the commodity consumption behavior obtained based on the weight change information is consistent with the commodity consumption behavior obtained based on the built-in image change information, recognizing that the commodity consumption behavior in the intelligent container is normal consumption;
And if the commodity consumption behavior obtained based on the weight change information is inconsistent with the commodity consumption behavior obtained based on the built-in image change information, identifying the consumption behavior of the commodity in the intelligent container as abnormal consumption.
Optionally, the device is located at the cloud or terminal.
According to a third embodiment of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method according to the first embodiment of the present disclosure.
According to a fourth embodiment of the present disclosure, there is provided an electronic apparatus including: a memory having a computer program stored thereon; a processor for executing the computer program in the memory to implement the steps of the method according to the first embodiment of the present disclosure.
By utilizing the technical scheme, the weight change information can be used for knowing which commodity or commodities on which layer of goods shelf of the intelligent container is taken away, and the image change information can be used for knowing which commodity or commodities on which layer of goods shelf of the intelligent container is taken away, so that the consumption behavior of the commodity in the intelligent container, such as whether the commodity is taken away normally or replaced by a dummy model or the like, can be identified through fusion analysis of the weight change information and the image change information, and further can be used for detecting abnormal consumption behaviors such as consumption cheating or misoperation and the like in actual use of the intelligent container.
Additional features and advantages of the present disclosure will be set forth in the detailed description which follows.
Drawings
The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification, illustrate the disclosure and together with the description serve to explain, but do not limit the disclosure. In the drawings:
FIG. 1 is a flow chart of a method of intelligent container consumption behavior identification according to one embodiment of the present disclosure.
FIG. 2 is a schematic block diagram of an intelligent container consumption behavior identification apparatus according to one embodiment of the disclosure.
Fig. 3 is a block diagram of an electronic device, according to an example embodiment.
Fig. 4 is a block diagram of an electronic device, according to an example embodiment.
Detailed Description
Specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating and illustrating the disclosure, are not intended to limit the disclosure.
FIG. 1 is a flow chart of a method of intelligent container consumption behavior identification, as shown in FIG. 1, according to one embodiment of the present disclosure, the method including the following steps S11 and S12.
In step S11, weight change information and image change information of the commodity of the intelligent cabinet are acquired.
Typically, a gravity scale is installed on each layer of shelves of the intelligent container. These gravity scales may be arranged to detect the weight of the shelf in which they are located, for example, each time the door of the intelligent container is opened, and each time the door of the intelligent container is closed. Therefore, the weight of each layer of goods shelf can be obtained by acquiring the weight of each layer of goods shelf detected by the gravity scale when the cabinet door is opened and closed and calculating the weight difference of each layer of goods shelf when the cabinet door is opened and closed.
In addition, an internal camera is arranged inside the intelligent container (for example, above each layer of shelf), and an external camera is arranged outside the intelligent container. The built-in camera may be arranged to take an image of the commodity, for example, each time the door of the intelligent container is opened, and also each time the door of the intelligent container is closed, so that by comparing the commodity images taken by the built-in camera when the door is opened and when the door is closed (also including detecting and identifying the type, number, etc. of the commodity from the commodity images, respectively, and comparing them), the image change information can be obtained. The external camera may be arranged to take a picture of the process of taking/putting back a commodity by a user, for example, during each opening of a cabinet door of the intelligent container, such that by comparing the images taken by the external camera (including detecting the type, number, etc. of the commodity in the plurality of images and comparing them), image change information can be obtained and whether a commodity is taken out of the intelligent container can be analyzed.
Wherein, whether an external camera or an internal camera, it may be, for example, a fish-eye camera or other type of camera.
In step S12, the consumption behavior of the commodity of the intelligent container is identified based on the fusion analysis of the weight change information and the image change information.
By utilizing the technical scheme, the weight change information can be used for knowing which commodity or commodities on which layer of goods shelf of the intelligent container is taken away, and the image change information can be used for knowing which commodity or commodities on which layer of goods shelf of the intelligent container is taken away, so that the consumption behavior of the commodity of the intelligent container, such as whether the commodity is taken away normally or replaced by a dummy model or the like, can be identified through fusion analysis of the weight change information and the image change information, and further can be used for detecting abnormal consumption behaviors such as consumption fraud or misoperation in actual use of the intelligent container.
For the recognition of the consumption behavior of the commodity by weight change/image change, the following description is given:
1. The weight change indicates that commodity consumption behavior is likely to occur, namely if the weight change exceeds the weight measurement error (for example, 5 g) of the gravity sensor, the measurement error comprises the error of the gravity sensor and the error caused by icing/frost and the like in the use process of the intelligent container, the type and the number of the changed commodity can be detected and identified by comparing the weight change with the commodity weight. Otherwise, if the weight change does not exceed the weight measurement error of the gravity sensor, it indicates that there is no commodity change from the weight perspective, and no further calculation or discrimination is necessary (i.e., no further identification of specific commodity or commodities is necessary).
2. The image change indicates that commodity consumption behavior may occur, that is, if the difference between the front and rear images exceeds an imaging error, where the imaging error may be, for example, an average variation amplitude of a pixel gray value (for example, 3, a value between 0 and 255), an average variation pixel ratio (that is, a ratio of a number of pixels to a total number of pixels of the image, for example, 2%), or a combination of the foregoing, and the imaging error includes an imaging error of the image sensor itself and also includes an imaging error caused by environmental changes such as illumination, frost, etc. during use of the smart container, the commodity type and number in the image may be detected by the commodity detection and identification method, and compared with the detected and identified result in the previous image to give a changed commodity type and number. Otherwise, if the image change does not exceed the imaging error, the image change is no commodity change from the image perspective, and subsequent calculation and discrimination are not needed.
In one possible implementation manner, the image variation information includes built-in image variation information obtained based on an image captured by a camera disposed inside the intelligent container, and the identifying the consumption behavior of the commodity of the intelligent container based on the fusion analysis of the weight variation information and the image variation information in step S12 includes: and identifying the consumption behavior of the commodity of the intelligent container as abnormal consumption under the condition that the weight change information indicates that the commodity of the intelligent container is changed and the built-in image change information indicates that the commodity of the intelligent container is not changed.
For example, assuming that the weight change information indicates that the weight on the second shelf, which is the intelligent container, has changed by 500g, and the image change information indicates that none of the goods on the respective shelves of the intelligent container has changed, in this case, it is recognized that an abnormal consumption behavior of the intelligent container has occurred, such as that a drunk empty bottle is put back in place, a genuine commodity is replaced with a fake commodity model, and so on.
In still another possible implementation manner, the image variation information includes built-in image variation information obtained based on an image captured by a camera disposed inside the intelligent container, and the identifying the consumption behavior of the intelligent container based on the fusion analysis of the weight variation information and the image variation information in step S12 includes: and identifying the consumption behavior of the commodity of the intelligent container as abnormal consumption under the condition that the weight change information indicates that the commodity of the intelligent container is unchanged and the built-in image change information indicates that the commodity of the intelligent container is changed. In this case, the kind of abnormal consumption may be further distinguished based on external image change information obtained from an image photographed by a camera provided outside the smart container, wherein the kind of abnormal consumption includes destructive abnormal consumption and unintentional abnormal consumption.
For example, assuming that the weight change information indicates that none of the items on the shelves of the intelligent container have changed, and the image change information indicates that none of the items on the shelves of the second tier of the intelligent container have changed, then in this case, it is recognized that an abnormal consumption behavior of the intelligent container has occurred, such as replacing one of the items on the second tier with another item of equal weight, one or more of the items on the second tier being blocked or knocked down (which may result in missing the identification of the item), and so on. Therefore, at this time, it is possible to further determine based on external image change information obtained from an image captured by a camera disposed outside the intelligent container: if the external image change information indicates that the commodity is taken out from the intelligent container, the abnormal consumption behavior is likely to be that other commodities with the same weight are used for replacing one commodity on the second-layer goods shelf, and the abnormal consumption behavior is destructive abnormal consumption; if the external image change information indicates that the commodity is not taken out of the intelligent container, the abnormal consumption behavior may be that a certain commodity or a certain commodity on the second layer of goods shelf is blocked or the commodity is knocked down in the process of consumption by a user, and the abnormal consumption behavior is unintentional abnormal consumption.
In one possible implementation manner, the image change information includes built-in image change information obtained based on an image captured by a camera disposed inside the intelligent container and external image change information obtained based on an image captured by a camera disposed outside the intelligent container, and the identifying the consumption behavior of the intelligent container based on the fusion analysis of the weight change information and the image change information in step S12 includes: and identifying whether the consumption behavior of the commodity in the intelligent container is abnormal consumption or normal consumption based on the external image change information under the condition that the weight change information indicates that the commodity in the intelligent container is unchanged and the built-in image change information also indicates that the commodity in the intelligent container is unchanged.
For example, if the weight change information indicates that the commodity on each layer of the shelf of the intelligent container is not changed, and the built-in image change information also indicates that the commodity on each layer of the shelf of the intelligent container is not changed, then in this case, whether the intelligent container has abnormal consumption behavior cannot be judged only according to the fusion analysis of the weight change information and the built-in image change information, for example, if the empty bottle after drinking is put back to the original position after having been filled with the dummy beverage with equal weight, then it cannot be judged that the intelligent container has abnormal consumption behavior only according to the fusion analysis of the weight change information and the built-in image change information. Therefore, in this case, it is necessary to further identify whether the intelligent container is consumed normally or abnormally by means of the external image change information. For example, if the external image change information indicates that the commodity is taken out of the intelligent container, the abnormal consumption behavior of the intelligent container can be identified; if the external image change information indicates that no commodity is taken out of the intelligent container, the intelligent container can be identified to have normal consumption behavior (namely, the user does not take the commodity finally).
In a possible implementation manner, the image variation information includes built-in image variation information obtained based on an image captured by a camera disposed inside the intelligent container, and the identifying the consumption behavior of the intelligent container based on the fusion analysis of the weight variation information and the image variation information in step S12 includes: in case the weight change information indicates a change in the commodity in the intelligent container and the built-in image change information also indicates a change in the commodity in the intelligent container,
If the commodity consumption behavior obtained based on the weight change information is consistent with the commodity consumption behavior obtained based on the built-in image change information, recognizing that the commodity consumption behavior in the intelligent container is normal consumption;
And if the commodity consumption behavior obtained based on the weight change information is inconsistent with the commodity consumption behavior obtained based on the built-in image change information, identifying the consumption behavior of the commodity in the intelligent container as abnormal consumption.
For example, assuming that the weight change information indicates that the first commodity on the second shelf of the intelligent container is taken away, resulting in a weight change of 500g on the second shelf, the built-in image change information also indicates that the first commodity on the second shelf of the intelligent container is taken away, that is, the commodity consumption behavior obtained based on the weight change information and the commodity consumption behavior obtained based on the built-in image change information are identical, in this case, it can be recognized that the consumption behavior of the commodity in the intelligent container is normal consumption.
For another example, assuming that the weight change information indicates that the weight on the second-layer shelf of the intelligent container is changed but it cannot be determined which article/articles are changed specifically (for example, the change position of the first article/articles on the second-layer shelf and the second article/articles on the third-layer shelf causes the weight change of the second-layer shelf, but it cannot be determined which article/articles are changed specifically by the weight difference), the built-in image change information indicates that the second-layer shelf of the intelligent container has fewer first articles (the type and number of articles in the image are detected by visual recognition first, and then the conclusion is obtained by comparing the recognition result of the image when the door is opened and the recognition result of the image when the door is closed), the consumption behavior of the articles in the intelligent container can be recognized as abnormal consumption, for example, the articles are changed in position (between the second-layer shelf and the third-layer shelf).
In addition, in the embodiment according to the disclosure, if the abnormal consumption behavior is identified, the identified abnormal consumption behavior can be timely notified to the intelligent container operation and maintenance system and/or personnel so as to process the abnormal consumption behavior in time.
FIG. 2 is a schematic block diagram of an intelligent container consumption behavior identification apparatus according to one embodiment of the disclosure, as shown in FIG. 2, the apparatus comprising: an acquisition module 21 for acquiring weight change information and image change information of the commodity of the intelligent container; an identification module 22 for identifying the consumption behavior of the commodity of the intelligent container based on a fusion analysis of the weight change information and the image change information.
By utilizing the technical scheme, the weight change information can be used for knowing which commodity or commodities on which layer of goods shelf of the intelligent container is taken away, and the image change information can be used for knowing which commodity or commodities on which layer of goods shelf of the intelligent container is taken away, so that the consumption behavior of the commodity in the intelligent container, such as whether the commodity is taken away normally or replaced by a dummy model or the like, can be identified through fusion analysis of the weight change information and the image change information, and further can be used for detecting abnormal consumption behaviors such as consumption cheating or misoperation and the like in actual use of the intelligent container.
Optionally, the image change information includes built-in image change information obtained based on an image captured by a camera disposed inside the intelligent container. The identification module 22 is further configured to identify the consumption behavior of the commodity in the intelligent container as abnormal consumption in a case where the weight change information indicates a change in the commodity in the intelligent container and the built-in image change information indicates no change in the commodity in the intelligent container.
Optionally, the image change information includes built-in image change information obtained based on an image captured by a camera disposed inside the intelligent container. The identification module 22 is further configured to identify the consumption behavior of the commodity in the intelligent container as abnormal consumption in a case where the weight change information indicates that the commodity in the intelligent container has not changed and the built-in image change information indicates that the commodity in the intelligent container has changed. In this case, the identification module 22 may further distinguish the types of abnormal consumption, including destructive abnormal consumption and unintentional abnormal consumption, based on external image change information obtained from an image captured by a camera disposed outside the intelligent container.
Optionally, the image change information includes built-in image change information obtained based on an image photographed by a camera disposed inside the intelligent container and external image change information obtained based on an image photographed by a camera disposed outside the intelligent container. The identification module 22 is further configured to identify whether the consumption behavior of the commodity in the intelligent container is abnormal consumption or normal consumption based on the external image change information if the weight change information indicates that the commodity in the intelligent container is unchanged and the internal image change information also indicates that the commodity in the intelligent container is unchanged.
Optionally, the image change information includes built-in image change information obtained based on an image captured by a camera disposed inside the intelligent container. The identification module 22 is further configured to, in case the weight change information indicates a change in the commodity in the smart container and the built-in image change information also indicates a change in the commodity in the smart container: if the commodity consumption behavior obtained based on the weight change information is consistent with the commodity consumption behavior obtained based on the built-in image change information, recognizing that the commodity consumption behavior in the intelligent container is normal consumption; and if the commodity consumption behavior obtained based on the weight change information is inconsistent with the commodity consumption behavior obtained based on the built-in image change information, identifying the consumption behavior of the commodity in the intelligent container as abnormal consumption.
Optionally, the device is located on a cloud or terminal. Thus, the requirement on the local processing capacity of the intelligent container can be reduced.
Additionally, in embodiments according to the present disclosure, if the identification module 22 identifies an abnormal consumption behavior, the identified abnormal consumption behavior may be timely notified to the intelligent container operation and maintenance system and/or personnel to facilitate timely processing of the abnormal consumption behavior.
The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.
Fig. 3 is a block diagram of an electronic device 700, according to an example embodiment. As shown in fig. 3, the electronic device 700 may include: a processor 701, a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input/output (I/O) interface 704, and a communication component 705.
The processor 701 is configured to control the overall operation of the electronic device 700 to perform all or part of the steps in the intelligent container consumption behavior recognition method described above. The memory 702 is used to store various types of data to support operation on the electronic device 700, which may include, for example, instructions for any application or method operating on the electronic device 700, as well as application-related data, such as contact data, messages sent and received, pictures, audio, video, and so forth. The Memory 702 may be implemented by any type or combination of volatile or non-volatile Memory devices, such as static random access Memory (Static Random Access Memory, SRAM for short), electrically erasable programmable Read-Only Memory (ELECTRICALLY ERASABLE PROGRAMMABLE READ-Only Memory, EEPROM for short), erasable programmable Read-Only Memory (Erasable Programmable Read-Only Memory, EPROM for short), programmable Read-Only Memory (Programmable Read-Only Memory, PROM for short), read-Only Memory (ROM for short), magnetic Memory, flash Memory, magnetic disk, or optical disk. The multimedia component 703 can include a screen and an audio component. Wherein the screen may be, for example, a touch screen, the audio component being for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 702 or transmitted through the communication component 705. The audio assembly further comprises at least one speaker for outputting audio signals. The I/O interface 704 provides an interface between the processor 701 and other interface modules, which may be a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is for wired or wireless communication between the electronic device 700 and other devices. Wireless Communication, such as Wi-Fi, bluetooth, near field Communication (NFC for short), 2G, 3G or 4G, or a combination of one or more thereof, so the corresponding Communication component 705 may comprise: wi-Fi module, bluetooth module, NFC module.
In an exemplary embodiment, the electronic device 700 may be implemented by one or more Application Specific Integrated Circuits (ASIC), digital signal Processor (DIGITAL SIGNAL Processor, DSP), digital signal processing device (DIGITAL SIGNAL Processing Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field Programmable GATE ARRAY, FPGA), controller, microcontroller, microprocessor, or other electronic element for performing the intelligent container consumption behavior recognition method described above.
In another exemplary embodiment, a computer readable storage medium is also provided comprising program instructions which, when executed by a processor, implement the steps of the intelligent container consumption behavior identification method described above. For example, the computer readable storage medium may be the memory 702 comprising program instructions described above, which are executable by the processor 701 of the electronic device 700 to perform the intelligent container consumption behavior identification method described above.
Fig. 4 is a block diagram illustrating an electronic device 1900 according to an example embodiment. For example, electronic device 1900 may be provided as a server. Referring to fig. 4, the electronic device 1900 includes a processor 1922, which may be one or more in number, and a memory 1932 for storing computer programs executable by the processor 1922. The computer program stored in memory 1932 may include one or more modules each corresponding to a set of instructions. Further, the processor 1922 may be configured to execute the computer program to perform the intelligent container consumption behavior identification method described above.
In addition, the electronic device 1900 may further include a power component 1926 and a communication component 1950, the power component 1926 may be configured to perform power management of the electronic device 1900, and the communication component 1950 may be configured to enable communication of the electronic device 1900, e.g., wired or wireless communication. In addition, the electronic device 1900 may also include an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system stored in memory 1932, such as Windows Server, mac OS XTM, unixTM, linuxTM, and the like.
In another exemplary embodiment, a computer readable storage medium is also provided comprising program instructions which, when executed by a processor, implement the steps of the intelligent container consumption behavior identification method described above. For example, the computer readable storage medium can be the memory 1932 described above including program instructions that are executable by the processor 1922 of the electronic device 1900 to perform the intelligent container consumption behavior identification method described above.
The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the present disclosure is not limited to the specific details of the embodiments described above, and various simple modifications may be made to the technical solutions of the present disclosure within the scope of the technical concept of the present disclosure, and all the simple modifications belong to the protection scope of the present disclosure.
In addition, the specific features described in the above embodiments may be combined in any suitable manner without contradiction. The various possible combinations are not described further in this disclosure in order to avoid unnecessary repetition.
Moreover, any combination between the various embodiments of the present disclosure is possible as long as it does not depart from the spirit of the present disclosure, which should also be construed as the disclosure of the present disclosure.
Claims (7)
1. An intelligent container consumption behavior recognition method is characterized by comprising the following steps:
Acquiring weight change information and image change information of commodities of an intelligent container;
Identifying abnormal consumption behavior of the commodity of the intelligent container based on a fusion analysis of the weight change information and the image change information;
The image change information includes built-in image change information obtained based on an image shot by a camera arranged inside the intelligent container and external image change information obtained based on an image shot by a camera arranged outside the intelligent container, the image shot by the camera arranged inside the intelligent container is a commodity image, and the image shot by the camera arranged outside the intelligent container is an image of a commodity picking/placing back process of a user, and if the abnormal consumption behavior of the commodity of the intelligent container is identified based on fusion analysis of the weight change information and the image change information, the method comprises the following steps:
identifying the consumption behavior of the commodity in the intelligent container as abnormal consumption under the condition that the weight change information indicates that the commodity in the intelligent container is changed and the built-in image change information indicates that the commodity in the intelligent container is not changed;
identifying the consumption behavior of the commodity in the intelligent container as abnormal consumption under the condition that the weight change information indicates that the commodity in the intelligent container is unchanged and the built-in image change information indicates that the commodity in the intelligent container is changed;
in case the weight change information indicates a change in the commodity in the intelligent container and the built-in image change information also indicates a change in the commodity in the intelligent container,
If the commodity consumption behavior obtained based on the weight change information is inconsistent with the commodity consumption behavior obtained based on the built-in image change information, recognizing that the consumption behavior of the commodity in the intelligent container is abnormal consumption;
And identifying abnormal consumption behaviors of the commodity in the intelligent container based on the external image change information under the condition that the weight change information indicates that the commodity in the intelligent container is unchanged and the built-in image change information also indicates that the commodity in the intelligent container is unchanged.
2. The method according to claim 1, wherein the method further comprises: the types of abnormal consumption behaviors are further distinguished based on the external image change information, wherein the types of abnormal consumption behaviors include destructive abnormal consumption behaviors and unintentional abnormal consumption behaviors.
3. An intelligent container consumption behavior recognition device, characterized in that the device comprises:
the acquisition module is used for acquiring the weight change information and the image change information of the commodity of the intelligent container;
the identification module is used for identifying the consumption behavior of the commodity of the intelligent container based on fusion analysis of the weight change information and the image change information;
The image change information comprises built-in image change information obtained based on an image shot by a camera arranged inside the intelligent container and external image change information obtained based on an image shot by a camera arranged outside the intelligent container, the image shot by the camera arranged inside the intelligent container is a commodity image, the image shot by the camera arranged outside the intelligent container is an image of a commodity picking/placing process of a user, and the image is that: the identification module is also used for:
identifying the consumption behavior of the commodity in the intelligent container as abnormal consumption under the condition that the weight change information indicates that the commodity in the intelligent container is changed and the built-in image change information indicates that the commodity in the intelligent container is not changed;
identifying the consumption behavior of the commodity in the intelligent container as abnormal consumption under the condition that the weight change information indicates that the commodity in the intelligent container is unchanged and the built-in image change information indicates that the commodity in the intelligent container is changed;
In case the weight change information indicates a change in the commodity in the intelligent container and the built-in image change information also indicates a change in the commodity in the intelligent container:
if the commodity consumption behavior obtained based on the weight change information is inconsistent with the commodity consumption behavior obtained based on the built-in image change information, recognizing that the consumption behavior of the commodity in the intelligent container is abnormal consumption;
And identifying abnormal consumption behaviors of the commodity in the intelligent container based on the external image change information under the condition that the weight change information indicates that the commodity in the intelligent container is unchanged and the built-in image change information also indicates that the commodity in the intelligent container is unchanged.
4. The apparatus of claim 3, wherein the identification module is further configured to further distinguish a category of the abnormal consumption behavior based on the external image change information, wherein the category of abnormal consumption behavior comprises destructive abnormal consumption behavior and unintentional abnormal consumption behavior.
5. The apparatus of any one of claims 3 to 4, wherein the apparatus is located at a cloud or terminal.
6. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the steps of the method according to any of claims 1-2.
7. An electronic device, comprising:
A memory having a computer program stored thereon;
A processor for executing the computer program in the memory to implement the steps of the method of any of claims 1-2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811347542.0A CN109671227B (en) | 2018-11-13 | 2018-11-13 | Intelligent container consumption behavior identification method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811347542.0A CN109671227B (en) | 2018-11-13 | 2018-11-13 | Intelligent container consumption behavior identification method and device, storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109671227A CN109671227A (en) | 2019-04-23 |
| CN109671227B true CN109671227B (en) | 2024-04-26 |
Family
ID=66142102
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811347542.0A Active CN109671227B (en) | 2018-11-13 | 2018-11-13 | Intelligent container consumption behavior identification method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109671227B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111445638B (en) * | 2020-05-14 | 2022-10-11 | 深圳街电科技有限公司 | Mobile power supply leasing equipment and mobile power supply management method |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012167945A (en) * | 2011-02-10 | 2012-09-06 | Seiko Instruments Inc | Particle counter |
| CN103699099A (en) * | 2013-12-27 | 2014-04-02 | 苏州市职业大学 | Intelligent shelf controller |
| WO2017142191A1 (en) * | 2016-02-16 | 2017-08-24 | 제이씨스퀘어주식회사 | System and method for remotely monitoring product trading in store by using pos terminal and camera |
| CN107123006A (en) * | 2017-07-12 | 2017-09-01 | 杨智勇 | A kind of smart shopper system |
| CN207337465U (en) * | 2017-07-06 | 2018-05-08 | 广州联业商用机器人科技股份有限公司 | A kind of unmanned shop of high in the clouds intelligence management and control |
| CN108416901A (en) * | 2018-03-27 | 2018-08-17 | 合肥美的智能科技有限公司 | Method and device for identifying goods in intelligent container and intelligent container |
| CN108549851A (en) * | 2018-03-27 | 2018-09-18 | 合肥美的智能科技有限公司 | Method and device for identifying goods in intelligent container and intelligent container |
| CN207895558U (en) * | 2018-02-06 | 2018-09-21 | 诚悦（大连）科技有限公司 | A kind of vending machine with dual identification function |
| CN108648334A (en) * | 2018-04-11 | 2018-10-12 | 合肥美的智能科技有限公司 | Self-service cabinet and its abnormal method for controlling reporting, self-service system |
| CN108765702A (en) * | 2018-05-23 | 2018-11-06 | 济南每日优鲜便利购网络科技有限公司 | Automatic vending machine |
| CN108780505A (en) * | 2018-05-17 | 2018-11-09 | 深圳前海达闼云端智能科技有限公司 | Intelligent sales counter, item identification method, device, server and storage medium |
-
2018
- 2018-11-13 CN CN201811347542.0A patent/CN109671227B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012167945A (en) * | 2011-02-10 | 2012-09-06 | Seiko Instruments Inc | Particle counter |
| CN103699099A (en) * | 2013-12-27 | 2014-04-02 | 苏州市职业大学 | Intelligent shelf controller |
| WO2017142191A1 (en) * | 2016-02-16 | 2017-08-24 | 제이씨스퀘어주식회사 | System and method for remotely monitoring product trading in store by using pos terminal and camera |
| CN207337465U (en) * | 2017-07-06 | 2018-05-08 | 广州联业商用机器人科技股份有限公司 | A kind of unmanned shop of high in the clouds intelligence management and control |
| CN107123006A (en) * | 2017-07-12 | 2017-09-01 | 杨智勇 | A kind of smart shopper system |
| CN207895558U (en) * | 2018-02-06 | 2018-09-21 | 诚悦（大连）科技有限公司 | A kind of vending machine with dual identification function |
| CN108416901A (en) * | 2018-03-27 | 2018-08-17 | 合肥美的智能科技有限公司 | Method and device for identifying goods in intelligent container and intelligent container |
| CN108549851A (en) * | 2018-03-27 | 2018-09-18 | 合肥美的智能科技有限公司 | Method and device for identifying goods in intelligent container and intelligent container |
| CN108648334A (en) * | 2018-04-11 | 2018-10-12 | 合肥美的智能科技有限公司 | Self-service cabinet and its abnormal method for controlling reporting, self-service system |
| CN108780505A (en) * | 2018-05-17 | 2018-11-09 | 深圳前海达闼云端智能科技有限公司 | Intelligent sales counter, item identification method, device, server and storage medium |
| CN108765702A (en) * | 2018-05-23 | 2018-11-06 | 济南每日优鲜便利购网络科技有限公司 | Automatic vending machine |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109671227A (en) | 2019-04-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108875664B (en) | Method and device for identifying purchased goods and vending machine | |
| CN108922026B (en) | Replenishment management method and device for vending machine and user terminal | |
| CN108335408B (en) | Article identification method, device and system for vending machine and storage medium | |
| CN108985359B (en) | Commodity identification method, unmanned vending machine and computer-readable storage medium | |
| CN108335406A (en) | Self-service equipment and its good selling method and self-service system | |
| CN110866429B (en) | Missing scanning identification method, device, self-service cashing terminal and system | |
| WO2019184646A1 (en) | Method and device for identifying merchandise, merchandise container | |
| WO2019154087A1 (en) | Vending machine and vending method and vending system therefor | |
| CN108171172A (en) | Self-help shopping method, self-service sale device and computer readable storage medium | |
| US20160088262A1 (en) | Method For Managing Storage Product In Refrigerator Using Image Recognition, And Refrigerator For Same | |
| CN109241883B (en) | Goods returning control method and device based on vending machine | |
| CN107818631A (en) | A kind of source of early warning and method for early warning based on automatic vending machine | |
| JP2021511599A (en) | Unmanned sales equipment, sales methods and unmanned sales systems | |
| CN111079478B (en) | Unmanned goods shelf monitoring method and device, electronic equipment and system | |
| CN114387735A (en) | Method, device and system for picking up goods | |
| CN110648468A (en) | Express cabinet, management method thereof and computer-readable storage medium | |
| CN111126990A (en) | Automatic article identification method, settlement method, device, terminal and storage medium | |
| CN113468914B (en) | Method, device and equipment for determining purity of commodity | |
| CN110837824B (en) | Commodity identification method for vending device, vending device and storage medium | |
| CN109671227B (en) | Intelligent container consumption behavior identification method and device, storage medium and electronic equipment | |
| CN114708441A (en) | Target detection and state determination method, device, equipment and storage medium | |
| CN203873422U (en) | Special-type article deposit box | |
| CN112184751A (en) | Object identification method and system and electronic equipment | |
| CN112364702A (en) | Article verification method and device | |
| CN110910567A (en) | Deduction method, device, electronic equipment, computer readable storage medium and container |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210303 Address after: 201111 2nd floor, building 2, no.1508, Kunyang Road, Minhang District, Shanghai Applicant after: Dalu Robot Co.,Ltd. Address before: 518000 Room 201, building A, No. 1, Qian Wan Road, Qianhai Shenzhen Hong Kong cooperation zone, Shenzhen, Guangdong (Shenzhen Qianhai business secretary Co., Ltd.) Applicant before: CLOUDMINDS (SHENZHEN) ROBOTICS SYSTEMS Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 201111 Building 8, No. 207, Zhongqing Road, Minhang District, Shanghai Applicant after: Dayu robot Co.,Ltd. Address before: 201111 2nd floor, building 2, no.1508, Kunyang Road, Minhang District, Shanghai Applicant before: Dalu Robot Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |