CN109711360B - Vending machine risk control method, vending machine risk control device and vending machine risk control system - Google Patents
Vending machine risk control method, vending machine risk control device and vending machine risk control system Download PDFInfo
- Publication number
- CN109711360B CN109711360B CN201811630397.7A CN201811630397A CN109711360B CN 109711360 B CN109711360 B CN 109711360B CN 201811630397 A CN201811630397 A CN 201811630397A CN 109711360 B CN109711360 B CN 109711360B
- Authority
- CN
- China
- Prior art keywords
- image
- vending machine
- images
- risk
- misrecognized
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The disclosure provides a vending machine risk control method, a vending machine risk control device and a vending machine risk control system, and relates to the technical field of artificial intelligence. The vending machine risk control method disclosed by the invention comprises the following steps: adding the misrecognized images from the vending machine to a training data set; updating the image model and the cargo feature information based on the training data set; and sending the updated image model and the goods characteristic information to the vending machine. By the method, the misidentification image of the vending machine can be obtained, and the image model and the goods characteristic information are updated based on the misidentification image, so that the image identification capability of the vending machine is improved, and the accuracy of risk control of the vending machine is improved.
Description
Technical Field
The disclosure relates to the technical field of artificial intelligence, in particular to a vending machine risk control method, a vending machine risk control device and a vending machine risk control system.
Background
With the rapid development of the retail industry, more and more internet enterprises begin to explore new retail models, and the vending machine is an effective expansion and extension of online retail business to offline markets. Compared with a purely gravity and RFID (Radio Frequency Identification) based unmanned vending machine, the video image based unmanned vending machine has a wider selection space and higher user experience, and therefore more and more attention is paid to the video image based unmanned vending machine.
Because the vending machine mainly sells food and daily necessities, how to effectively prevent the goods of the vending machine from being stolen is still a problem to be solved for preventing the goods harmful to the health from being put into the vending machine.
Disclosure of Invention
One object of the present disclosure is to improve accuracy of vending machine risk control.
According to an aspect of some embodiments of the present disclosure, there is provided a vending machine risk control method, including: adding the misrecognized images from the vending machine to a training data set; updating the image model and the cargo feature information based on the training data set; and sending the updated image model and the goods characteristic information to the vending machine.
In some embodiments, adding the misrecognized images from the vending machine to the training data set includes: adding misrecognized images from a single vending machine to a training data set for the vending machine providing the misrecognized images; updating the image model and the cargo feature information based on the training dataset includes: generating personalized image models and goods characteristic information for vending machines providing misrecognized images; sending the updated image model and the cargo characteristic information to the vending machine comprises: the personalized image model and the cargo characteristic information are sent to the vending machine providing the misrecognized image.
In some embodiments, the vending machine risk control method further comprises: receiving a high-risk image from a vending machine, wherein when the vending machine determines that a user carries out risk operation, the image acquired by an image acquisition device is reported as the high-risk image; and extracting the high-risk image under the condition of reporting caused by image misrecognition to serve as a misrecognized image.
In some embodiments, extracting a high-risk image in the case of a report due to image misrecognition as a misrecognized image comprises: checking whether the user executes the illegal operation according to the high-risk image; and if the user does not execute the illegal operation, determining the high-risk image as the misrecognized image.
In some embodiments, the vending machine risk control method further comprises: and if the user is determined to execute the illegal operation and the number of the illegal operations reaches a preset threshold, modifying the user authority to be the limited user.
In some embodiments, the vending machine risk control method further comprises: acquiring an initial training data set according to the operation of simulating a customer shopping flow by an image acquisition person; acquiring an initial image model and cargo characteristic information based on an initial training data set; the initial image model and the cargo characteristic information are sent to the vending machine so that the vending machine performs image analysis and provides misrecognized images based on the initial image model and the cargo characteristic information.
In some embodiments, the vending machine risk control method further comprises: the vending machine is in the state of opening the door: if the acquired images are determined to be abnormal, caching the images acquired by each image acquisition device as high-risk images and reporting the high-risk images, wherein the vending machine comprises more than 2 paths of image acquisition devices.
In some embodiments, the image anomalies include one or more of: determining from the image analysis that one or more image capture devices are occluded; determining that an abnormal object exists in the vending machine according to the image analysis; the user carries out abnormal operation on the commodity, wherein the abnormal operation comprises one or more of destroying external packages of the commodity, adding substances into the commodity, taking the substances from the commodity or returning after the commodity leaves the camera detection area; or, the size, color or transparency of the goods changes.
In some embodiments, the vending machine risk control method further comprises: determining the goods change condition based on the image acquired by the camera; determining cargo change conditions based on gravity detection; and under the condition that the cargo change conditions determined based on the images and the gravity detection are inconsistent, caching the images collected by the cameras as high-risk images.
By the method, the mistaken identification image of the vending machine can be obtained, and the image model and the goods characteristic information are updated based on the mistaken identification image, so that the image identification capability of the vending machine is improved in time, and the accuracy of risk control of the vending machine is improved.
According to an aspect of other embodiments of the present disclosure, there is provided a vending machine risk control device, including: a training set update unit configured to add the misrecognized images from the vending machine to a training data set; a model and feature update unit configured to update the image model and the cargo feature information based on a training data set; and the issuing unit is configured to send the updated image model and the goods characteristic information to the vending machine.
In some embodiments, the training set update unit is configured to: adding misrecognized images from a single vending machine to a training data set for the vending machine providing the misrecognized images; the model and feature update unit is configured to: generating personalized image models and goods characteristic information for vending machines providing misrecognized images; the issuing unit is configured to: the personalized image model and the cargo characteristic information are sent to the vending machine providing the misrecognized image.
In some embodiments, the vending machine risk control device further comprises: the image receiving unit is configured to receive a high-risk image from the vending machine, wherein the vending machine reports the image acquired by the image acquisition device as the high-risk image when determining that a user carries out risk operation; and the false recognition image extraction unit is configured to extract a high-risk image in the case of reporting due to image false recognition as a false recognition image.
In some embodiments, the misrecognized image extraction unit is configured to: checking whether the user executes the illegal operation according to the high-risk image; and if the user does not execute the illegal operation, determining the high-risk image as the misrecognized image.
In some embodiments, the vending machine risk control device further comprises: and the user permission modification unit is configured to modify the user permission to be the limited user if the user is determined to execute the illegal operation and the number of times of the illegal operation reaches a preset threshold.
In some embodiments, the training set update unit is further configured to obtain an initial training data set according to an operation of the image capture personnel to mimic a customer shopping procedure; the model and feature updating unit is further configured to obtain an initial image model and cargo feature information based on the initial training dataset; the issuing unit is also configured to send the initial image model and the goods characteristic information to the vending machine so that the vending machine performs image analysis and provides a misrecognized image based on the initial image model and the goods characteristic information.
According to an aspect of further embodiments of the present disclosure, there is provided a vending machine risk control device, including: a memory; and a processor coupled to the memory, the processor configured to perform any of the vending machine risk control methods above based on instructions stored in the memory.
The risk control device for the vending machine can acquire the mistaken identification image of the vending machine, and updates the image model and the goods characteristic information based on the mistaken identification image, so that the image identification capability of the vending machine is improved in time, and the accuracy of risk control of the vending machine is improved.
According to an aspect of still further embodiments of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, which instructions, when executed by a processor, implement the steps of any of the above vending machine risk control methods.
By executing the instructions on the computer-readable storage medium, the misidentification image of the vending machine can be acquired, and the image model and the goods characteristic information are updated based on the misidentification image, so that the image identification capability of the vending machine is improved in time, and the accuracy of risk control of the vending machine is improved.
Further, according to an aspect of some embodiments of the present disclosure, there is provided a vending machine control system including: any of the vending machine risk control devices described above; and the vending machine is configured to cache images acquired by each image acquisition device as high-risk images and report the high-risk images to the vending machine risk control device if the acquired images are determined to be abnormal in the door opening state, wherein the vending machine comprises more than 2 paths of image acquisition devices; the image analysis algorithm is updated based on the updated image model and the cargo characteristic information from the vending machine risk control device.
In some embodiments, the image anomalies include one or more of: determining from the image analysis that one or more image capture devices are occluded; determining that an abnormal object exists in the vending machine according to the image analysis; the user carries out abnormal operation on the commodity, wherein the abnormal operation comprises one or more of destroying external packages of the commodity, adding substances into the commodity, taking the substances from the commodity or returning after the commodity leaves the camera detection area; or, the size, color or transparency of the goods changes.
In some embodiments, the vending machine is further configured to: determining the goods change condition based on the image acquired by the camera; determining cargo change conditions based on gravity detection; and under the condition that the cargo change conditions determined based on the images and the gravity detection are inconsistent, caching the images collected by the cameras as high-risk images.
In the vending machine control system, the vending machine can find abnormal conditions and upload images in time, and the vending machine risk control device can update the image model and the goods characteristic information based on the mistaken identification images, so that the image identification capability of the vending machine is improved in time, and the accuracy of risk control of the vending machine is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this disclosure, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure and not to limit the disclosure. In the drawings:
fig. 1 is a flow chart of some embodiments of a vending machine risk control method of the present disclosure.
FIG. 2 is a flow chart of further embodiments of the vending machine risk control method of the present disclosure.
Fig. 3 is a flow chart of still further embodiments of the vending machine risk control method of the present disclosure.
Fig. 4 is a schematic diagram of some embodiments of vending machine risk control devices of the present disclosure.
Fig. 5 is a schematic diagram of further embodiments of vending machine risk control devices of the present disclosure.
Fig. 6 is a schematic diagram of some embodiments of vending machine risk control systems of the present disclosure.
Fig. 7 is a schematic diagram of further embodiments of vending machine risk control systems of the present disclosure.
Detailed Description
The technical solution of the present disclosure is further described in detail by the accompanying drawings and examples.
The inventor finds that the image algorithm in the vending machine uses an object detection and object recognition algorithm such as a deep learning framework, which requires a large number of scene images for model training. In general, the previous image acquisition can be performed by artificially simulating the behavior of purchasing a commodity by a customer, and model training can be performed using the data. However, in the model training, it is necessary to find an image in which a commodity is difficult to detect, an image in which a background is easy to detect as a commodity, an image including an unidentifiable commodity, and an image training model which is easy to be erroneously recognized as another commodity from a large number of images, so as to improve the accuracy of the model. Such image acquisition operations are cumbersome and consume a lot of manpower.
After the image algorithm is trained through the image collected by the testing machine, a certain detection and recognition accuracy can be achieved, however, how to further improve the accuracy of the image algorithm becomes a technical bottleneck, on one hand, the algorithm design is basically optimal and difficult to break through, on the other hand, the image algorithm collected manually by the testing machine is fully trained, so that the accuracy of the image algorithm is limited, and the generalization is not high.
A flow chart of some embodiments of the vending machine risk control method of the present disclosure is shown in fig. 1.
In step 101, misrecognized images from the vending machine are added to the training data set. In some embodiments, the false recognition image can be an image which is triggered in real time during vending of the vending machine and has difficulty in recognition, occlusion and abnormality suspicion, is fed back by the vending machine in real time, and is determined to be an image which is fed back due to false recognition through screening.
In step 102, the image model and cargo feature information are updated based on the training dataset. In some embodiments, the updated training data set may be utilized to derive updated image models and cargo feature information based on an algorithm, such as a depth mining algorithm, that forms the initial updated image models and cargo feature information.
In step 103, the updated image model and the cargo characteristic information are sent to the vending machine.
By the method, the misidentification image of the vending machine can be obtained, and the image model and the goods characteristic information are updated based on the misidentification image, so that the image identification capability of the vending machine is improved, and the accuracy of risk control of the vending machine is improved.
In some embodiments, the image model and the cargo characteristics may be different because each vending machine may be subjected to different environmental conditions or may carry different cargo. The method can update the training data set of each vending machine on the basis of the unified training data set, add the misrecognized images from a single vending machine into the personalized training data set of the vending machine providing the misrecognized images, generate personalized image models and goods characteristic information of the vending machine providing the misrecognized images, and send the personalized image models and the goods characteristic information to the vending machine providing the misrecognized images. In the using process of the vending machine, the individualized training data set is continuously updated, so that the continuous updating of the image model and the goods characteristic information is realized, on one hand, the burden of image recognition operation of the vending machine caused by overlarge data amount of the universal image model and the goods characteristic information is avoided, and on the other hand, the image recognition accuracy of each vending machine can be pertinently improved.
A flow chart of further embodiments of the vending machine risk control method of the present disclosure is shown in fig. 2.
In step 201, a high risk image is received from a vending machine. In some embodiments, the high-risk image may be an image which is triggered in real time during vending of the vending machine and has difficulty in identification, occlusion and abnormality suspicion, and is fed back by the vending machine in real time, and the image fed back due to false identification is determined through screening.
In some embodiments, the vending machine may provide high-risk images in one or more of the following situations: determining from the image analysis that one or more image capture devices are occluded; determining that an abnormal object exists in the vending machine according to the image analysis; the user carries out abnormal operation on the commodity, wherein the abnormal operation comprises one or more of destroying external packages of the commodity, adding substances into the commodity, taking the substances from the commodity or returning after the commodity leaves the camera detection area; or, the size, color or transparency of the goods changes.
In some embodiments, the vending machine may also provide images captured by the respective cameras as high-risk images in the event that the cargo change conditions determined based on the images and based on gravity detection are inconsistent.
In some embodiments, the vending machine can intercept and screen the acquired high-risk images, and reduce the number of image frames to be uploaded, thereby reducing the data transmission burden.
In step 202, whether the user performs the violation operation is checked according to the high-risk image. In some embodiments, a manual interpretation may be used to determine whether the user performed the violation. In the case where it is determined that the user does not perform the illegal operation, it is determined that the image misrecognition occurs in the vending machine, and step 203 is performed.
In step 203, the high risk image provided by the vending machine is determined to be a misrecognized image.
In step 204, the misrecognized images from the vending machine are added to the training data set.
In step 205, the image model and cargo feature information are updated based on the training dataset.
In step 206, the updated image model and the cargo characteristic information are sent to the vending machine.
By the method, the high-risk images provided by the vending machine can be acquired in real time, and the images provided by the vending machine triggered by the false identification are confirmed, so that on one hand, the image identification function of the vending machine can be updated timely, the image identification capability of the vending machine is improved, on the other hand, the situation that the abnormal behaviors of the user are ignored due to the fact that the images provided by the vending machine triggered by the abnormal behaviors of the user are mistakenly identified as the false identification images can be avoided, the risk of missed alarm of the vending machine is reduced, and the safety of the vending.
In some embodiments, in the event that it is determined in step 202 above that the user performed an illegal operation, step 207 may be performed, as shown in FIG. 2.
In step 207, it is determined whether the number of user violations to perform the violations reaches a predetermined threshold. In some embodiments, the user may be identified by an account number used when the user triggers the vending machine to open the door, and a history of violations of the user may be read. If the number of illegal operations of the user reaches a preset threshold, executing step 208; if the predetermined threshold is not reached, step 209 is performed.
In step 208, the user permissions are modified to restrict the user. In some embodiments, user levels of different permissions may be set, gradually decreasing user permissions as the number of violations increases, e.g., from opening all vending machines to the user, to allowing only some vending machines to shop, to disallowing the user to use any vending machine, etc.
In step 209, the recorded number of user violations is increased. In some embodiments, the increased number of times value may be determined based on the type of illegal manipulation by the user, such as more heavily increased number of times value, less lightly increased number of times value, etc.
By the method, the efficiency of discovering the abnormal vending machine can be improved, whether the service is provided for the user or not is determined by utilizing a user credit system provided by the vending machine, and the safety is further improved.
A flow chart of still further embodiments of the vending machine risk control method of the present disclosure is shown in fig. 3.
In step 301, before the vending machine is operating normally, the staff member imitates the shopping process of the customer, collects images of the operation of the staff member, and acquires an initial training data set.
In step 302, an initial image model and cargo feature information are obtained based on an initial training data set.
In step 303, the initial image model and the cargo characteristic information are sent to the vending machine.
In some embodiments, because of the large number of vendors, a unified image model and cargo feature information may be generated and provided to each vendor based on the standard usage environment, thereby reducing the amount of data collected for the initial training data set.
In other embodiments, in consideration of the problems that the using environments of various vending machines are complex and changeable, and the types of loaded goods are different, the vending machines can be classified, initial training set acquisition is performed on each type of vending machine, initial image models and goods characteristic information for the type of vending machine are generated, and the self-adaption capability and accuracy of image recognition are improved.
In step 304, the vending machine determines whether the captured image is abnormal based on the image recognition algorithm, and in the case where it is determined to be abnormal, step 305 is performed. In some embodiments, the vending machine comprises more than 2 paths of image acquisition devices, can acquire abnormal images and non-abnormal images and cache the abnormal images, and can cache the images which are not abnormal at the moment when the abnormal images exist, so that the quality of the cached high-risk images is ensured. In some embodiments, 5-way cameras may be used, located at the left, right, top, and bottom side of the detection rack, respectively, and directly in front of the detection rack from the inside to the outside.
In some embodiments, when it is determined that the user performs an abnormal operation on the goods or the results of the image recognition and the gravity detection do not match, the image may be buffered first and fed back when the vending machine is closed (i.e., the current shopping process is finished), so as to reduce the uploading frequency of the image and reduce the burden of data transmission and processing.
In step 305, the vending machine reports a high risk image.
In step 306, in the event that the high-risk image is determined to be a misrecognized image, the image model and the cargo characteristic information are updated.
In step 307, the updated image model and the cargo feature information are sent to the vending machine, and then the process continues to step 304, and high-risk images reported by the vending machine are waited for.
In the related technology, a large amount of labor cost and time cost are consumed to carry out image acquisition training on the background and the loaded goods of the vending machine, an algorithm model is slow in iteration, an online feedback mechanism is lacked, and the optimization and promotion effect is limited. When an unmanned vending machine is newly deployed or new commodities are sold, the carried target detection and identification model cannot be adapted in time, and the accuracy of detection and identification is reduced to a certain extent. If training picture collection is performed again in a new background before a vending machine is deployed or a new commodity is loaded, a large amount of labor and material cost is consumed.
By the method in the embodiment, online difficult mining can be performed through an effective online self-feedback mechanism, images which are not easy to accurately recognize are searched and transmitted back, self-iteration and optimization of the model of the vending machine are achieved, image recognition requirements of new environments and new commodities are rapidly adapted, image acquisition cost of a large number of training images is saved, and meanwhile user experience and goods loss are further improved and reduced due to improvement of image recognition accuracy.
A schematic diagram of some embodiments of the vending machine risk control device of the present disclosure is shown in fig. 4. The training set update unit 401 can add misrecognized images from the vending machine to the training data set. In some embodiments, the false recognition image can be an image which is triggered in real time during vending of the vending machine and has difficulty in recognition, occlusion and abnormality suspicion, is fed back by the vending machine in real time, and is determined to be an image which is fed back due to false recognition through screening.
The model and feature update unit 402 can update the image model and the cargo feature information based on the training data set. In some embodiments, the updated training data set may be utilized to derive updated image models and cargo feature information based on an algorithm, such as a depth mining algorithm, that forms the initial updated image models and cargo feature information.
The issuing unit 403 can transmit the updated image model and the goods characteristic information to the vending machine.
The risk control device for the vending machine can acquire the mistaken identification image of the vending machine, and updates the image model and the goods characteristic information based on the mistaken identification image, so that the image identification capability of the vending machine is improved in time, and the accuracy of risk control of the vending machine is improved.
In some embodiments, the image model and the cargo characteristics may be different because each vending machine may be subjected to different environmental conditions or may carry different cargo. The training set updating unit 401 can update the training data sets of the vending machines respectively on the basis of the unified training data sets, and add the misrecognized images from the single vending machine to the training data sets of the vending machines providing the misrecognized images; the model and feature update unit 402 can generate personalized image models and goods feature information for vending machines that provide misrecognized images; the issuing unit 403 can transmit the personalized image model of the vending machine and the characteristic information of the goods to the vending machine that provides the misrecognized image.
The vending machine risk control device avoids the burden of image recognition operation of the vending machine caused by overlarge data quantity of universal image models and goods characteristic information on the one hand, and can also improve the image recognition accuracy of each vending machine in a targeted manner on the other hand.
In some embodiments, as shown in fig. 4, the vending machine risk control device may further include an image receiving unit 404 and a misrecognized image extraction unit 405. The image receiving unit 404 can receive a high risk image from the vending machine. In some embodiments, the high-risk image may be an image which is triggered in real time during vending of the vending machine and has difficulty in identification, occlusion and abnormality suspicion, and is fed back by the vending machine in real time, and the image fed back due to false identification is determined through screening.
The misrecognized image extraction unit 405 can check whether the user performs an illegal operation based on the high-risk image. In some embodiments, a manual interpretation may be used to determine whether the user performed the violation. In the case where it is determined that the user has not performed the illegal operation, it is determined that the image misrecognition has occurred in the vending machine, and the training set updating unit 401 is activated.
The vending machine risk control device can acquire the high-risk images provided by the vending machine in real time and confirm the images provided by the vending machine triggered by the false identification, on one hand, the image identification function of the vending machine can be updated timely, the image identification capability of the vending machine is improved, on the other hand, the situation that the abnormal behaviors of users are ignored due to the fact that the images provided by the vending machine triggered by the abnormal behaviors of the users are mistakenly identified as the false identification images can be avoided, the risk that the vending machine generates missed alarms is reduced, and the safety of the vending machine is improved.
In some embodiments, as shown in fig. 4, the vending machine risk control device may further include a user authority modification unit 406 capable of determining whether the number of times of the user's illegal operation for performing the illegal operation reaches a predetermined threshold in a case where the misrecognized image extraction unit 405 determines that the user performs the illegal operation. If the number of illegal operations of the user reaches a preset threshold, modifying the user authority to limit the user; and if the number of the illegal operations of the user is not up to the preset threshold, increasing the recorded number of the illegal operations of the user.
The vending machine risk control device can improve the efficiency of abnormal discovery of the vending machine, and whether the service is provided for the user or not is determined by utilizing a user credit system provided by the vending machine, so that the safety is further improved.
A schematic structural view of some embodiments of the vending machine risk control device of the present disclosure is shown in fig. 5. The vending machine risk control device includes a memory 501 and a processor 502. Wherein: the memory 501 may be a magnetic disk, flash memory, or any other non-volatile storage medium. The memory is for storing instructions in the corresponding embodiments of the vending machine risk control method above. The processor 502 is coupled to the memory 501 and may be implemented as one or more integrated circuits, such as a microprocessor or microcontroller. The processor 502 is used for executing the instructions stored in the memory, so that the image recognition capability of the vending machine can be improved, and the accuracy of risk control of the vending machine can be improved.
Some embodiments of the present disclosure propose a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the article identification method in any of the foregoing embodiments.
A schematic diagram of some embodiments of the vending machine risk control system of the present disclosure is shown in fig. 6. The vending machine risk control device 61 may be any of the vending machine risk control devices mentioned above. The vending machine risk control system further comprises vending machines 621-62 n, wherein n is a positive integer. The vending machine can cache images collected by each image collecting device as high-risk images and report the high-risk images to the vending machine risk control device if the collected images are determined to be abnormal in the door opening state; the image analysis algorithm is updated based on the updated image model and the cargo characteristic information from the vending machine risk control device.
In the vending machine control system, the vending machine can find abnormal conditions and upload images in time, and the vending machine risk control device can update the image model and the goods characteristic information based on the mistaken identification images, so that the image identification capability of the vending machine is improved in time, and the accuracy of risk control of the vending machine is improved.
In some embodiments, the vending machine may be opened by a user, such as by scanning a two-dimensional code of the vending machine, or by otherwise informing the vending machine of its own identity to trigger the vending machine to open the door. The vending machine utilizes an image analysis algorithm to determine whether the image is anomalous. And if the image is determined to be abnormal, caching the images acquired by each image acquisition device as high-risk images.
In some embodiments, the vending machine comprises more than 2 paths of image acquisition devices, can acquire abnormal images and non-abnormal images and cache the abnormal images, and can cache the images which are not abnormal at the moment when the abnormal images exist, so that the quality of the cached high-risk images is ensured. In some embodiments, 5-way cameras may be used, located at the left, right, top, and bottom side of the detection rack, respectively, and directly in front of the detection rack from the inside to the outside.
In some embodiments, an image anomaly may be one or more image capture devices (e.g., cameras) being occluded. And if the acquired images are determined to be abnormal, reporting the high-risk images to the vending machine risk control device.
In some embodiments, the image anomalies include one or more of: determining from the image analysis that one or more image capture devices are occluded; determining that an abnormal object exists in the vending machine according to the image analysis; the user carries out abnormal operation on the commodity, wherein the abnormal operation comprises one or more of destroying external packages of the commodity, adding substances into the commodity, taking the substances from the commodity or returning after the commodity leaves the camera detection area; or, the size, color or transparency of the goods changes.
In some embodiments, the vending machine may further determine a cargo change condition based on the images captured by the cameras, determine the cargo change condition based on gravity detection, and perform an operation of caching the images captured by the respective cameras as the high-risk images in a case where the cargo change condition determined based on the images and the gravity detection is inconsistent.
In the vending machine control system, the vending machine can judge whether to trigger the cache of the high-risk image and report the high-risk image through two ways of judging whether the image is abnormal or not and whether the change condition interpretation results of the image and the gravity detection are consistent or not, so that the probability of missed extraction of an event is reduced, and the safety of vending goods by the vending machine is improved; meanwhile, images do not need to be stored continuously, and the requirement on the storage space of the vending machine is reduced.
In some embodiments, as shown in fig. 7, the initial model is first trained by the images collected by the tester, clustering the goods initial features DB (DataBase). And then, returning the abnormal image by using a risk control mechanism, screening, and taking the misrecognized image as a sample difficult to sample. Then, the mined difficult cases are added into a training data set, and the model is retrained. Meanwhile, the cargo DB is updated by using the data returned by the wind control, so that the DB is closer to a real data scene, and a new model and a new characteristic DB are obtained. And finally deploying the iterated model and the feature DB to the vending machine. Closed loop feedback has been realized to whole flow, can effectively promote algorithm rate of accuracy and generalization nature, restores fast and lays the new scene, go up the low problem of rate of identification of new goods, utilizes new identification model to draw the characteristic DB who sells the goods, adapts to new scene and new goods fast, and then promotes the unmanned user experience who sells the machine to reduce the loss of goods cost. The closed-loop feedback is formed in the process, the algorithm and the characteristic DB are periodically updated, the iterative system forms the closed-loop feedback, the capability of the image algorithm classifier for distinguishing the false positive samples is effectively enhanced, meanwhile, the generalization capability is fully improved, the algorithm model and the characteristic DB are quickly updated in an iterative mode, and the accuracy is continuously improved.
As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable non-transitory storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The above description is only exemplary of the present disclosure and is not intended to limit the present disclosure, so that any modification, equivalent replacement, or improvement made within the spirit and principle of the present disclosure should be included in the scope of the present disclosure.
Claims (18)
1. A vending machine risk control method comprising:
receiving a high-risk image from a vending machine, wherein when the vending machine determines that a user carries out risk operation, the image acquired by an image acquisition device is reported as the high-risk image;
extracting the high-risk image under the condition that the report is caused by image misrecognition to serve as a misrecognition image;
adding the misrecognized images from the vending machine to a training data set;
updating an image model and cargo feature information based on the training dataset;
and sending the updated image model and the goods characteristic information to the vending machine.
2. The method of claim 1, wherein,
the adding of the misrecognized images from the vending machine to the training data set comprises: adding misrecognized images from a single vending machine to a training data set for the vending machine providing the misrecognized images;
the updating the image model and the cargo feature information based on the training dataset comprises: generating an individualized image model and item characteristic information for the vending machine providing the misrecognized image;
the sending the updated image model and the goods characteristic information to the vending machine comprises: and sending the personalized image model and the goods characteristic information to a vending machine providing the misrecognized image.
3. The method of claim 1 or 2, wherein extracting the high-risk image in the case of a report due to image misrecognition as the misrecognized image comprises:
checking whether the user executes illegal operation according to the high-risk image;
and if the user does not execute the illegal operation, determining the high-risk image as the misrecognized image.
4. The method of claim 3, further comprising:
and if the user is determined to execute the illegal operation and the number of the illegal operations reaches a preset threshold, modifying the user authority to be the limited user.
5. The method of claim 1, further comprising:
acquiring an initial training data set according to the operation of simulating a customer shopping flow by an image acquisition person;
acquiring an initial image model and cargo feature information based on the initial training data set;
and sending the initial image model and the goods characteristic information to the vending machine so that the vending machine can perform image analysis based on the initial image model and the goods characteristic information and provide the misrecognized image.
6. The method of claim 1, further comprising: the vending machine is in the state of opening the door:
and if the acquired images are determined to be abnormal, caching the images acquired by each image acquisition device as high-risk images and reporting the high-risk images, wherein the vending machine comprises more than 2 paths of image acquisition devices.
7. The method of claim 6, wherein the image anomalies include one or more of:
determining from the image analysis that one or more image capture devices are occluded;
determining that an abnormal object exists in the vending machine according to the image analysis;
the method comprises the following steps that a user conducts abnormal operation on a commodity, wherein the abnormal operation comprises one or more of destroying external packages of the commodity, adding substances into the commodity, taking the substances from the commodity or returning after the commodity leaves a camera detection area; or the like, or, alternatively,
the size, color or transparency of the goods changes.
8. The method of claim 6 or 7, further comprising:
determining the goods change condition based on the image acquired by the camera;
determining cargo change conditions based on gravity detection;
and under the condition that the cargo change conditions determined based on the images and the gravity detection are inconsistent, caching the images collected by the cameras as high-risk images.
9. A vending machine risk control device comprising:
the image receiving unit is configured to receive a high-risk image from a vending machine, wherein the vending machine reports an image acquired by an image acquisition device as the high-risk image when determining that a user carries out risk operation;
the false recognition image extraction unit is configured to extract the high-risk image under the condition that the report is caused by image false recognition to serve as a false recognition image;
a training set update unit configured to add the misrecognized images from the vending machine to a training data set;
a model and feature update unit configured to update an image model and cargo feature information based on the training dataset;
and the issuing unit is configured to send the updated image model and the updated cargo characteristic information to the vending machine.
10. The apparatus of claim 9, wherein,
the training set update unit is configured to: adding misrecognized images from a single vending machine to a training data set for the vending machine providing the misrecognized images;
the model and feature update unit is configured to: generating an individualized image model and item characteristic information for the vending machine providing the misrecognized image;
the issuing unit is configured to: and sending the personalized image model and the goods characteristic information to a vending machine providing the misrecognized image.
11. The apparatus according to claim 9 or 10, wherein the misrecognized image extraction unit is configured to:
checking whether the user executes illegal operation according to the high-risk image;
and if the user does not execute the illegal operation, determining the high-risk image as the misrecognized image.
12. The apparatus of claim 11, further comprising:
and the user permission modification unit is configured to modify the user permission to be the limited user if the user is determined to execute the illegal operation and the number of times of the illegal operation reaches a preset threshold.
13. The apparatus of claim 9, wherein the training set update unit is further configured to obtain an initial training data set according to an operation of an image capturing person mimicking a customer shopping procedure;
the model and feature updating unit is further configured to obtain an initial image model and cargo feature information based on the initial training dataset;
the issuing unit is further configured to send the initial image model and the goods characteristic information to the vending machine so that the vending machine performs image analysis based on the initial image model and the goods characteristic information and provides the misrecognized image.
14. A vending machine risk control device comprising:
a memory; and
a processor coupled to the memory, the processor configured to perform the method of any of claims 1-5 based on instructions stored in the memory.
15. A computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the steps of the method of any one of claims 1 to 5.
16. A vending machine control system comprising:
the vending machine risk control device of any one of claims 9 to 14; and the combination of (a) and (b),
the vending machine is configured to cache images acquired by each image acquisition device as high-risk images and report the high-risk images to the vending machine risk control device if the acquired images are determined to be abnormal in a door opening state, wherein the vending machine comprises more than 2 paths of image acquisition devices; the image analysis algorithm is updated based on the updated image model and the cargo characteristic information from the vending machine risk control device.
17. The system of claim 16, wherein the image anomalies include one or more of:
determining from the image analysis that one or more image capture devices are occluded;
determining that an abnormal object exists in the vending machine according to the image analysis;
the method comprises the following steps that a user conducts abnormal operation on a commodity, wherein the abnormal operation comprises one or more of destroying external packages of the commodity, adding substances into the commodity, taking the substances from the commodity or returning after the commodity leaves a camera detection area; or the like, or, alternatively,
the size, color or transparency of the goods changes.
18. The system of claim 16 or 17, wherein the vending machine is further configured to:
determining the goods change condition based on the image acquired by the camera;
determining cargo change conditions based on gravity detection;
and under the condition that the cargo change conditions determined based on the images and the gravity detection are inconsistent, caching the images collected by the cameras as high-risk images.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811630397.7A CN109711360B (en) | 2018-12-29 | 2018-12-29 | Vending machine risk control method, vending machine risk control device and vending machine risk control system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811630397.7A CN109711360B (en) | 2018-12-29 | 2018-12-29 | Vending machine risk control method, vending machine risk control device and vending machine risk control system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109711360A CN109711360A (en) | 2019-05-03 |
| CN109711360B true CN109711360B (en) | 2021-03-30 |
Family
ID=66259409
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811630397.7A Active CN109711360B (en) | 2018-12-29 | 2018-12-29 | Vending machine risk control method, vending machine risk control device and vending machine risk control system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109711360B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109711359B (en) * | 2018-12-29 | 2021-08-03 | 北京沃东天骏信息技术有限公司 | Vending machine control method, vending machine control device and computer-readable storage medium |
| CN110826481A (en) * | 2019-11-01 | 2020-02-21 | 北京云迹科技有限公司 | Data processing method, commodity identification method, server and storage medium |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170032092A1 (en) * | 2016-06-16 | 2017-02-02 | Benjamin Franklin Mink | Real Time Multispecialty Telehealth Interactive Patient Wellness Portal (IPWP) |
| CN108520194A (en) * | 2017-12-18 | 2018-09-11 | 上海云拿智能科技有限公司 | Kinds of goods sensory perceptual system based on imaging monitor and kinds of goods cognitive method |
| CN108171172A (en) * | 2017-12-27 | 2018-06-15 | 惠州Tcl家电集团有限公司 | Self-help shopping method, self-service sale device and computer readable storage medium |
| CN108229965A (en) * | 2018-01-03 | 2018-06-29 | 深圳正品创想科技有限公司 | A kind of commodity anti-theft settlement method, device and electronic equipment |
| CN108564442A (en) * | 2018-04-11 | 2018-09-21 | 合肥美的智能科技有限公司 | Unmanned retail trade system and its control method, goods selling equipment and Cloud Server |
| CN108806074B (en) * | 2018-06-05 | 2021-08-03 | 腾讯科技（深圳）有限公司 | Shopping information generation method and device and storage medium |
| CN109002772A (en) * | 2018-06-27 | 2018-12-14 | 深圳市赛亿科技开发有限公司 | A kind of measure of supervision and system of unmanned supermarket's customer behavior |
| CN108875664B (en) * | 2018-06-27 | 2022-07-05 | 北京京东尚科信息技术有限公司 | Method and device for identifying purchased goods and vending machine |
-
2018
- 2018-12-29 CN CN201811630397.7A patent/CN109711360B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109711360A (en) | 2019-05-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106104555B (en) | For protecting the behavioural analysis of peripheral equipment | |
| US20230316762A1 (en) | Object detection in edge devices for barrier operation and parcel delivery | |
| US7617167B2 (en) | Machine vision system for enterprise management | |
| US8478048B2 (en) | Optimization of human activity determination from video | |
| CN101933058A (en) | Video sensor and alarm system and method with object and event classification | |
| CN107958435A (en) | Safe examination system and the method for configuring rays safety detection apparatus | |
| CN113366487A (en) | Operation determination method and device based on expression group and electronic equipment | |
| US20180232904A1 (en) | Detection of Risky Objects in Image Frames | |
| CN103093187A (en) | Authentication system | |
| CN109711359B (en) | Vending machine control method, vending machine control device and computer-readable storage medium | |
| CN108269333A (en) | Face identification method, application server and computer readable storage medium | |
| CN103324912A (en) | Face recognition system and method for ATM | |
| CN109711360B (en) | Vending machine risk control method, vending machine risk control device and vending machine risk control system | |
| CN104205111A (en) | Computing device to detect malware | |
| US20200226360A1 (en) | System and method for automatically detecting and classifying an animal in an image | |
| CN111626201A (en) | Commodity detection method and device and readable storage medium | |
| US20100002142A1 (en) | System and method for video-processing algorithm improvement | |
| WO2020024414A1 (en) | User authentication method and device based on slider verification code | |
| CN109712324B (en) | Vending machine image identification method, vending method and vending equipment | |
| CN108922029A (en) | Vending equipment and its control method | |
| CN107122743A (en) | Security-protecting and monitoring method, device and electronic equipment | |
| CN105303442A (en) | Online bank account number detection method and apparatus | |
| CN114022945A (en) | Real-name receiving, sending and vision checking equipment and real-name receiving, sending and vision checking method | |
| CN109671211B (en) | Intelligent container tearing and signing behavior monitoring method, intelligent container and computer storage medium | |
| EP2193435A2 (en) | Machine vision system for enterprise management |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |