CN109166257B - Shopping cart commodity verification method and device thereof - Google Patents
Shopping cart commodity verification method and device thereof Download PDFInfo
- Publication number
- CN109166257B CN109166257B CN201810964593.1A CN201810964593A CN109166257B CN 109166257 B CN109166257 B CN 109166257B CN 201810964593 A CN201810964593 A CN 201810964593A CN 109166257 B CN109166257 B CN 109166257B
- Authority
- CN
- China
- Prior art keywords
- shopping cart
- shopping
- commodity
- image
- image acquisition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
- G07G1/00—Cash registers
- G07G1/0036—Checkout procedures
- G07G1/0045—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
- G07G1/0054—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader with control of supplementary check-parameters, e.g. weight or number of articles
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
- G07G1/00—Cash registers
- G07G1/12—Cash registers electronically operated
Abstract
The invention provides a shopping cart commodity verification method and a device thereof, wherein the method comprises the following steps: acquiring a 3D image in the vehicle; identifying and obtaining the total volume of the current commodity, calculating the total volume, determining the predicted stacking volume, and constructing a predicted stacking volume database; judging whether a predicted stacking volume which is the same as the total volume of the current commodity exists in the predicted stacking volume database; if yes, judging that no abnormal condition occurs; if not, judging that an abnormal condition occurs so as to be convenient for checking. The invention realizes the direct check of goods in the shopping cart without damage and transportation, shortens the check time, improves the shopping settlement efficiency, avoids the condition of little calculation or more calculation easily caused by manual piece-by-piece check, improves the user experience and brings convenience to check settlement personnel and consumers.
Description
Technical Field
The invention relates to the technical field of shopping settlement, in particular to a shopping cart commodity verification method and a device thereof.
Background
In an open shopping environment, for example, a supermarket generally refers to a store mainly for opening and displaying commodities, self-purchasing by customers, queuing for cash settlement, and managing fresh foods and miscellaneous goods. The system is a retail enterprise for self-service purchase and unified cash register settlement of consumers.
At present, in an open shopping environment, a shopping settlement method is to manually check the commodities in a shopping cart of a consumer one by one, and the general process is as follows: the method comprises the steps of commodity taking out and checking, code scanning one by one, checking whether commodities are consistent with a list, finally settling, manually checking the commodities, taking the commodities out and checking the commodities one by one, long checking time, low shopping settlement efficiency, low calculation or more calculation easily caused by manual one by one checking, poor user experience and inconvenience for checking and settling personnel and consumers.
Disclosure of Invention
In view of the above, the present invention provides a method and a device for verifying a commodity in a shopping cart to solve the deficiencies of the prior art.
In order to solve the above problems, the present invention provides a shopping cart commodity verification method, comprising:
3D image acquisition is carried out on the shopping cart in the image acquisition area, and a 3D image in the cart is obtained;
identifying the in-vehicle 3D image to obtain the total volume of the current commodity corresponding to the in-vehicle 3D image;
acquiring the monomer volume of each commodity in the shopping list according to the shopping list corresponding to the commodity in the shopping cart, calculating to obtain the total volume of all the commodities in the shopping list, determining the predicted stacking volume of all the commodities in the shopping cart stacked in different forms according to the total volume and the monomer volume, and constructing a predicted stacking volume database corresponding to the shopping list;
judging whether a predicted stacking volume in a combination form same as the total volume of the current commodity exists in the predicted stacking volume database or not;
if yes, judging that no abnormal condition occurs;
if not, judging that an abnormal condition occurs, and generating a prompt message so as to prompt a manager to check according to the prompt message.
Preferably, before the "acquiring 3D images of shopping carts in the image acquisition area to obtain 3D images in carts", the method further comprises:
receiving a shopping list corresponding to the commodities in the shopping cart, and generating shopping cart movement prompt information to prompt a user to push the shopping cart into the image acquisition area;
receiving a shopping cart verification instruction generated by triggering an infrared sensor when the shopping cart completely enters the image acquisition area; and according to the shopping cart verification instruction, starting a multi-angle 3D image acquisition device so as to acquire 3D images of the shopping cart in the image acquisition area.
Preferably, after the step of turning on the multi-angle 3D image capturing device according to the shopping cart verification instruction, the method further comprises the steps of:
focusing and photometry are carried out on the shopping cart through a light ray detection device, and shooting parameters are obtained;
the step of acquiring the 3D image of the shopping cart in the image acquisition area to obtain the 3D image in the cart comprises the following steps:
sending a flash instruction to an auxiliary LED flash lamp connected with the multi-angle 3D image acquisition device through the multi-angle 3D image acquisition device according to the shooting parameters;
3D image acquisition based on the shooting parameters is carried out on the shopping cart in the image acquisition area through the multi-angle 3D image acquisition device, meanwhile, the auxiliary LED flash lamp synchronously flashes according to the flash instruction, and the 3D image in the cart after flash and light supplement is obtained.
Preferably, before the "acquiring 3D images of shopping carts in the image acquisition area to obtain 3D images in carts", the method further comprises:
acquiring a current light brightness value in the shopping cart through a light sensor;
comparing the current light brightness value with a preset light threshold range;
and if the current light brightness value reaches the preset light threshold range, turning on an LED illuminating lamp to provide an illuminating light source.
Preferably, after the "if, determining that no abnormal condition occurs" further includes:
identifying the 3D image in the shopping cart through a pre-trained commodity confirmation analysis model so as to determine the current commodity information in the shopping cart; the current commodity information comprises commodity varieties and commodity quantity corresponding to the commodity varieties; calculating to obtain the total weight of the current commodity according to the commodity variety, the commodity quantity corresponding to the commodity variety and the commodity weight corresponding to the commodity variety in the current commodity information;
calculating the total weight of the list of the commodities according to the shopping list;
acquiring the net weight of the commodities in the shopping cart in the image acquisition area through a weight sensor, and taking the net weight as the actual total weight of the commodities in the cart;
and if the total weight of the current commodities, the actual total weight of the commodities in the shopping cart and the total weight of the list are the same, judging that the commodities in the shopping cart are consistent with the commodities in the shopping list, and finishing the verification.
Preferably, before the "acquiring 3D images of shopping carts in the image acquisition area to obtain 3D images in carts", the method further comprises:
acquiring the azimuth image of the shopping cart in the image acquisition area through an azimuth image acquisition device to obtain an azimuth height pre-judgment image;
identifying the azimuth height pre-judgment image to obtain azimuth information and height information corresponding to the articles in the shopping cart;
and controlling the multi-angle 3D image acquisition device to move to an image acquisition position which is adaptive to the azimuth information and the height information so as to acquire 3D images of the shopping cart in the image acquisition area at the image acquisition position.
In addition, in order to solve the above problems, the present invention provides a shopping cart commodity verification apparatus, comprising: the device comprises an acquisition module, an identification module, a construction module, a judgment module and a prompt module;
the acquisition module is used for acquiring a 3D image of the shopping cart in the image acquisition area to obtain a 3D image in the cart;
the identification module is used for identifying the in-vehicle 3D image to obtain the total volume of the current commodity corresponding to the in-vehicle 3D image;
the building module is used for obtaining the monomer volume of each commodity in the shopping list according to the shopping list corresponding to the commodity in the shopping cart, calculating to obtain the total volume of all the commodities in the shopping list, determining the predicted stacking volume of all the commodities in the shopping cart stacked in different forms according to the total volume and the monomer volume, and building a predicted stacking volume database corresponding to the shopping list;
the judging module is used for judging whether the predicted stacking volume in the combined form which is the same as the total volume of the current commodity exists in the predicted stacking volume database or not;
and the prompt module is used for generating prompt information when the abnormal condition is judged to occur so as to prompt a manager to check according to the prompt information.
In addition, in order to solve the above problems, the present invention further provides a user terminal, including a memory and a processor, where the memory is used for storing a shopping cart commodity verification program, and the processor runs the shopping cart commodity verification program to make the user terminal execute the shopping cart commodity verification method.
In addition, to solve the above problem, the present invention further provides a computer readable storage medium, wherein a shopping cart commodity verification program is stored on the computer readable storage medium, and when being executed by a processor, the shopping cart commodity verification program realizes the shopping cart commodity verification method.
The invention provides a shopping cart commodity verification method and a device thereof. The method comprises the steps that 3D images of shopping carts in an image acquisition area are acquired, so that 3D images in the carts are obtained; then, obtaining the total volume of the current commodity through identification; constructing a predicted stacking volume database containing predicted stacking volumes in various different combination forms according to the shopping list; judging whether a predicted stacking volume in a combined form which is the same as the total volume of the current commodity exists in the predicted stacking volume database or not; if so, judging that no abnormity exists; if not, judging that the abnormity occurs, and prompting the staff to carry out further inspection. The invention realizes the direct check of goods in the shopping cart without damage and transportation, checks whether the goods are consistent with the shopping list based on the total volume of the goods in the shopping cart through 3D image recognition, thereby realizing the check, shortening the check time, improving the shopping settlement efficiency, avoiding the condition of little or more calculation easily caused by manual piece-by-piece check, improving the user experience and bringing convenience to check settlement personnel and consumers.
Drawings
FIG. 1 is a schematic structural diagram of a hardware operating environment according to an embodiment of a shopping cart commodity verification method of the present invention;
FIG. 2 is a schematic flow chart illustrating a first embodiment of a shopping cart merchandise verification method according to the present invention;
FIG. 3 is a flowchart illustrating a second embodiment of the shopping cart merchandise verification method of the present invention;
FIG. 4 is a flowchart illustrating a shopping cart merchandise verification method according to a third embodiment of the present invention;
FIG. 5 is a flowchart illustrating a fourth embodiment of the shopping cart merchandise verification method of the present invention;
FIG. 6 is a schematic flow chart illustrating a shopping cart merchandise verification method according to a fifth embodiment of the present invention;
FIG. 7 is a functional block diagram of the merchandise verification device of the shopping cart according to the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
Reference will now be made in detail to the embodiments of the present invention, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
In the present invention, unless otherwise expressly stated or limited, the terms "mounted," "connected," "secured," and the like are to be construed broadly and can, for example, be fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, fig. 1 is a schematic structural diagram of a hardware operating environment of a terminal according to an embodiment of the present invention.
The terminal of the embodiment of the invention can be a PC, and can also be a mobile terminal device such as a tablet computer, a portable computer and the like.
As shown in fig. 1, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may comprise a display screen, an input unit such as a keyboard, a remote control, and the optional user interface 1003 may also comprise a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high speed RAM memory or a stable memory such as a disk memory. The memory 1005 may alternatively be a storage device separate from the processor 1001.
In addition, the terminal comprises image acquisition equipment which can be a camera, a camera and the like, and the terminal is a multi-angle 3D image acquisition device and can be an Intel 3D perception camera which acquires images by an image acquisition device based on an Intel realistic stereo depth technology.
In addition, the terminal also comprises a gravity sensing device used for acquiring the quality of the commodities in the shopping cart.
Optionally, the terminal may further include RF (Radio Frequency) circuits, sensors, audio circuits, WiFi modules, and the like. In addition, the mobile terminal may further be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which are not described herein again.
Those skilled in the art will appreciate that the terminal shown in fig. 1 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer-readable storage medium, may include therein an operating system, a data interface control program, a network connection program, and a shopping cart goods check program.
The invention provides a shopping cart commodity verification method and a device thereof. The method realizes the nondestructive and non-transport direct check of the commodities in the shopping cart, and checks whether the commodities are consistent with a shopping list or not based on the total volume of the commodities in the shopping cart through 3D image recognition, thereby realizing the check, shortening the check time, improving the shopping settlement efficiency, avoiding the condition of little or more calculation due to manual piece-by-piece check, improving the user experience, and bringing convenience to check settlement personnel and consumers.
Example 1:
referring to fig. 2, a first embodiment of the present invention provides a shopping cart commodity verification method, including:
step S10, 3D image acquisition is carried out on the shopping cart in the image acquisition area, and 3D images in the cart are obtained;
as described above, the shopping cart commodity verification method provided in this embodiment may be applicable to an open shopping environment, such as a shopping mall, a supermarket, and other places, that is, unattended, and a user may freely take a commodity. In this embodiment, the shopping device may be an open shopping supermarket, after a user starts shopping, the user places the purchased goods in a shopping cart, and can perform self-settlement in a self-help code scanning manner, after the settlement, obtain an electronic or paper shopping list, then push the shopping cart into an image acquisition area for verification, and after the verification passes, end the shopping settlement process.
The checking step may be a step of checking the commodities in the shopping cart of the user before and after the payment is made for the user. In addition, the user shopping cart may be checked before or during shopping.
The image acquisition area can be a special area for user verification, and when the user needs to enter a verification process, the shopping cart is pushed to enter the area so as to perform a verification step.
The 3D image acquisition can be realized by the relevant 3D image acquisition device.
The 3D image acquisition device, in this embodiment, can be for multi-angle 3D image acquisition device, utilize the image acquisition device based on the stereoscopic depth technique of intel's sense to gather.
The intel real feeling technology enables the computing device to sense human intentions through multiple sensory modes such as vision, hearing, touch, voice, even emotion and situation, and enables interaction between people and the device to be more natural. In this embodiment, the "active stereo imaging principle" may be adopted, which simulates the "parallax principle" of human eyes, and by emitting a beam of infrared light, the left infrared sensor and the right infrared sensor track the position of the beam of light, and then the "depth" information in the 3D image is calculated by using the triangulation principle.
Above-mentioned, multi-angle 3D image acquisition device can be for Intel's 3D perception camera, and it can provide accurate depth information to also compromise ordinary red green blue camera function. Through the 3D perception camera, functions such as gesture control, augmented reality, virtual reality, face scanning, three-dimensional scanning can be realized, simultaneously can also do speech input. In the embodiment, the 3D perception camera is utilized to realize three-dimensional scanning on the commodities in the shopping cart so as to obtain the 3D image in the cart.
Step S20, recognizing the in-vehicle 3D image to obtain the total volume of the current commodity corresponding to the in-vehicle 3D image;
in the above, the 3D image modeling is used for identifying the commodities in the shopping cart, so that the corresponding total volume of the current commodities is obtained.
Step S30, obtaining the monomer volume of each commodity in the shopping list according to the shopping list corresponding to the commodity in the shopping cart, calculating to obtain the total volume of all the commodities in the shopping list, determining the predicted stacking volume of all the commodities in the shopping cart stacked in different forms according to the total volume and the monomer volume, and constructing a predicted stacking volume database corresponding to the shopping list;
the shopping list is a shopping list generated by the user after payment is carried out by the user or according to the instruction of a manager, or generated in other ways; the shopping list can be generated piece by piece in a code scanning mode, and two-dimension codes can be arranged in the shopping list, and the shopping list can be obtained by relevant devices for verification through scanning the two-dimension codes.
In the above, the shopping list may include the variety of the goods purchased by the user and the quantity corresponding to the variety. In some cases, the size, shape, hardness of the packaging material, angle, occupied space, elasticity, and other parameters of each commodity are different, and when the commodities are stacked together, the total volume is sometimes not equal to the sum of the monomer volumes of all the commodities, and may be larger than or smaller than the sum of the monomer volumes. Through computer modeling, the possibility of the volume of all commodities in the shopping list when stacked together (predicted stacking volume) can be predicted by referring to the single volume, so that the predicted stacking volume corresponding to the possibility is constructed into a database, namely the predicted stacking volume database.
Step S40, judging whether the predicted stacking volume database has the same combined form predicted stacking volume as the total volume of the current commodity;
step S50, if yes, it is determined that no abnormal condition occurs;
and step S60, if not, judging that an abnormal condition occurs, and generating a prompt message so as to prompt a manager to check according to the prompt message.
The constructed prediction stacking volume database comprises a plurality of different prediction combination forms, and the different combination forms may have certain gaps after stacking due to different stacking orders, orientations, angles and the like, so that the final stacked appearance total volumes are different. Therefore, a database containing all stacking conditions predicted by a computer is constructed, data in the database is compared with the total volume of the current commodity, and if the database contains the predicted stacking volume matched with or the same as the database, the abnormal condition is judged not to occur; if not, then the abnormal situation is determined, and a prompt message is generated, such as a voice speaker prompt "volume parameter abnormal! "to prompt the manager to troubleshoot or to see if the items in the shopping cart are consistent with the list.
According to the method provided by the embodiment, 3D images of the shopping cart in the image acquisition area are acquired, so that 3D images in the shopping cart are acquired; then, obtaining the total volume of the current commodity through identification; constructing a predicted stacking volume database containing predicted stacking volumes in various different combination forms according to the shopping list; judging whether a predicted stacking volume in a combined form which is the same as the total volume of the current commodity exists in the predicted stacking volume database or not; if so, judging that no abnormity exists; if not, judging that the abnormity occurs, and prompting the staff to carry out further inspection. The embodiment realizes the direct check of commodities in the shopping cart without damage and transport, and checks whether the commodities are consistent with a shopping list or not based on the total volume of the commodities in the shopping cart through 3D image recognition, thereby realizing the check, shortening the check time, improving the shopping settlement efficiency, avoiding the condition of little or more calculation due to manual one-by-one check, improving the user experience, and bringing convenience to check settlement personnel and consumers.
Example 2:
referring to fig. 3, a second embodiment of the present invention provides a shopping cart commodity verification method, based on the first embodiment shown in fig. 2, before the step S10, "acquiring 3D images of shopping carts in an image acquisition area, and obtaining 3D images in carts", the method further includes:
step S70, receiving a shopping list corresponding to the commodity in the shopping cart, and generating a shopping cart movement prompting message to prompt a user to push the shopping cart into the image acquisition area;
the shopping list can be received by the user, wherein the input mode can be that the two-dimensional code of the user is scanned by a scanner on the verification device, and the verification device receives the shopping list.
The shopping movement prompt may be a prompt generated according to the received shopping list and played by voice or image, for example, "please push shopping cart into the collection area! ", to prompt the user to push the cart into the image capture area, and the image capture area may make a placard line with a distinct indication.
Above-mentioned, the image acquisition region, for corresponding with multi-angle 3D image acquisition device, multi-angle 3D image acquisition device can carry out the region that whole all goods carry out image acquisition in the shopping cart.
Step S80, receiving a shopping cart verification instruction generated by triggering an infrared sensor when the shopping cart completely enters the image acquisition area; and according to the shopping cart verification instruction, starting a multi-angle 3D image acquisition device so as to acquire 3D images of the shopping cart in the image acquisition area.
In this embodiment, the multi-angle 3D image acquisition device acquires based on the image acquisition device of the Intel realistic stereo depth technology, and may be an Intel 3D perception camera.
The above-mentioned, infrared sensor for being used for the perception shopping cart to get into the sensor in image acquisition region completely, after the user pushed the shopping cart into image acquisition region completely, triggered infrared sensor generates a shopping cart verification instruction, and then opens multi-angle 3D image acquisition device.
Step S90, focusing and photometry are carried out on the shopping cart through a light ray detection device, and shooting parameters are obtained;
as described above, when image acquisition is performed, there is a possibility that a captured light is dim, and a high-quality recognizable image cannot be acquired, so the light and shade of the light are one of important conditions for image acquisition. In this embodiment, through light detection device, carry out image acquisition's region to multi-angle 3D image acquisition device's needs, the region in the shopping cart carries out light detection promptly, focuses photometry to obtain the shooting parameter that more is fit for current scene light. The shooting parameters may include parameters such as focal length, aperture, exposure, shutter speed, and the like.
The step S10, "acquiring 3D images of shopping carts in the image acquisition area to obtain 3D images in the carts" includes:
step S11, sending a flash instruction to an auxiliary LED flash lamp connected with the multi-angle 3D image acquisition device according to the shooting parameters through the multi-angle 3D image acquisition device;
above-mentioned, supplementary LED flash light can be one, also can be for the multi-angle be equipped with the supplementary LED flash light of different luminance, also can be the supplementary LED flash light of a plurality of adjustable light filling intensity. According to the shooting parameters, the required light can be obtained, namely the light intensity, the image acquisition speed, the depth of field, the aperture, the focal length and other information which need to be supplemented by the auxiliary LED flash lamp are obtained, so that the multi-angle 3D image acquisition device can assist the LED flash lamp to send a flash instruction according to the shooting parameters.
And step S12, 3D image acquisition based on the shooting parameters is carried out on the shopping cart in the image acquisition area through the multi-angle 3D image acquisition device, and meanwhile, the auxiliary LED flash lamp synchronously flashes according to the flash instruction to obtain the 3D image in the cart after flash light supplement.
Carry out image acquisition through multi-angle 3D image acquisition device to utilize supplementary LED flash light to carry out synchronous flash of light.
In the embodiment, after the shopping cart completely enters the image acquisition area, the light detection device is used for focusing and metering light to obtain the shooting parameters, and image acquisition and synchronous flashing can be further carried out according to the shooting parameters, so that the 3D image in the cart can be obtained under the condition that the light is suitable, further identification and judgment are facilitated, and the shopping settlement and verification efficiency is improved to a certain extent. In addition, light detection device and supplementary LED flash light are connected to locate and be applicable to shopping cart light filling region, and multi-angle 3D image acquisition device can set up in being different from the position of light detection device and supplementary LED flash light carries out image acquisition to the commodity in the shopping cart, and at this moment, light detection device can acquire and be more applicable to supplementary LED flash light current position, more is adapted to the shooting parameter of current environment, and multi-angle 3D image acquisition device can shoot under the shooting parameter condition that is more adapted to current environment, utilizes supplementary LED flash light to carry out the flash of light in step simultaneously, thereby reach better image acquisition effect.
Example 3:
referring to fig. 4, a third embodiment of the present invention provides a shopping cart commodity verification method, based on the second embodiment shown in fig. 3, before the step S10, "acquiring 3D images of shopping carts in an image acquisition area, and obtaining 3D images in carts", the method further includes:
step S100, acquiring a current light brightness value in the shopping cart through a light sensor;
the light sensor is used for acquiring current light in the shopping cart, and can acquire the current light brightness value.
Step S110, comparing the current light brightness value with a preset light threshold range;
the preset light threshold range is preset data representing the acceptable range of the current light. And comparing the light brightness value with the threshold range so as to judge whether the light brightness is in the threshold range. If included, the light may be determined to be insufficient, otherwise, the light may be determined to be sufficient.
And step S120, if the current light brightness value reaches the preset light threshold range, turning on an LED illuminating lamp to provide an illuminating light source.
If the light is insufficient, namely the current light brightness value reaches the threshold range, the LED illuminating lamp is controlled to be turned on so as to facilitate image acquisition. Therefore, the image can be collected under the condition more suitable for the current light, and the image with better quality can be obtained. LED light, on the one hand, can realize the light filling effect when gathering to the image, on the other hand can provide certain illuminating effect to the consumer that checks or other users, improves user experience.
Example 4:
referring to fig. 5, a fourth embodiment of the present invention provides a method for verifying a shopping cart commodity, based on the first embodiment shown in fig. 2, wherein, in step S50, "if yes, it is determined that no abnormal condition has occurred", the method further includes:
step S130, recognizing the 3D image in the shopping cart through a pre-trained commodity confirmation analysis model to determine current commodity information in the shopping cart; the current commodity information comprises commodity varieties and commodity quantity corresponding to the commodity varieties; calculating to obtain the total weight of the current commodity according to the commodity variety, the commodity quantity corresponding to the commodity variety and the commodity weight corresponding to the commodity variety in the current commodity information;
if the abnormal condition is judged not to occur, the commodity in the shopping cart passes the verification on the volume level. In this embodiment, after it is determined that no abnormal condition occurs, further verification of the commodities in the shopping cart through the weight layer can be performed to improve the accuracy of the verification.
Through image recognition, commodity information contained in the in-vehicle 3D image is determined, and the commodity information specifically comprises information such as variety and quantity.
Specifically, the 3D image in the vehicle is recognized according to a pre-trained recognition model, so as to determine the commodity information contained therein. And then calculating the total weight of the current commodity according to the commodity information.
Step S140, calculating the total weight of the list of the commodities according to the shopping list;
the total weight of the list is calculated according to the shopping list.
S150, acquiring the net weight of the commodities in the shopping cart in the image acquisition area through a weight sensor, and taking the net weight as the actual total weight of the commodities in the cart;
the net weight is the net weight obtained by planing off the relevant mass of the shopping cart equal to the mass of the cart independent of the goods in the cart.
Step S160, if the total weight of the current commodities, the actual total weight of the commodities in the shopping cart and the total weight of the list are the same, determining that the commodities in the shopping cart are consistent with the commodities in the shopping list, and ending the verification.
As described above, if the total weight of the current product, the actual total weight of the product in the cart, and the total weight of the list are all the same, it may be determined that the total weight of the product in the list, the actual weighing, and the total weight obtained through the image prediction are the same, and it may be determined that the product in the shopping cart is the same as the product in the shopping list, and the verification may be ended. By utilizing the parameters of the weight layer, the purchased commodities are checked, and the total weight of the commodities predicted by image recognition, the total weight of the list commodities and the total weight data actually weighed are compared to be consistent, so that the check is confirmed to be passed, the accuracy of the check is greatly improved, the condition that the goods are missed or more goods are calculated is avoided, and convenience is provided for managers and consumers.
Example 5:
referring to fig. 6, a fifth embodiment of the present invention provides a shopping cart commodity verification method, based on the first embodiment shown in fig. 2, before the step S20, "acquiring 3D images of shopping carts in an image acquisition area and obtaining 3D images in carts", the method further includes:
s170, carrying out azimuth image acquisition on the shopping cart in the image acquisition area through an azimuth image acquisition device to obtain an azimuth height pre-judgment image;
in this embodiment, since the image capturing area includes the size of the area, there may be a situation that the multi-angle 3D image capturing device cannot obtain a complete in-car 3D image of the in-car merchandise, that is, since the shopping cart enters the image capturing area and is pushed by the consumer, there may be a certain deviation, so that a high-quality image that can be recognized completely cannot be obtained. Therefore, in step S20, before obtaining the in-vehicle 3D image, it is necessary to determine the orientation in which the shopping cart is placed. Or, the positions of the high stacks of the goods in the shopping cart are different, such as the positions are low, and if the multi-angle 3D image capturing device is a fixed position, a clear in-cart 3D image may not be captured.
Above-mentioned, position collection system can also include the infrared sensor that is used for measuring the height including the camera that carries out image acquisition to obtain position height through the camera and judge the image or obtain the signal of characterization height through infrared sensor. The position of the azimuth collecting device can be arranged on a position different from the multi-angle 3D image collecting device, and can also be arranged beside the multi-angle 3D image collecting device and connected with the multi-angle 3D image collecting device.
Step S180, identifying the azimuth height pre-judging image to obtain azimuth information and height information corresponding to articles in the shopping cart;
in the above, the orientation information and the height information are obtained through image recognition, where the orientation information may be positioning information, that is, a coordinate range defined for the image acquisition area, and the orientation information is a coordinate position located in the coordinate range.
And S190, controlling the multi-angle 3D image acquisition device to move to an image acquisition position corresponding to the azimuth information and the height information so as to acquire 3D images of the shopping cart in the image acquisition area at the image acquisition position.
And controlling the multi-angle 3D image acquisition device to move to a corresponding position. Specifically, multi-angle 3D image acquisition device can have slide or arm on equipment to can carry out the removal of relevant position, through removing, adjustable multi-angle 3D image acquisition device's visual scope or the collection scope of image, thereby can further improve image acquisition's quality. In this embodiment, through according to position information and height information, control multi-angle 3D image acquisition device removes to corresponding image acquisition position to further carry out image acquisition on image acquisition position, improved image acquisition's quality, gather the image that is adapted to current position and high field of vision more, further improved discernment effect and efficiency to a certain extent.
Further, referring to fig. 7, the present invention also provides a shopping cart goods verification apparatus, comprising: the system comprises an acquisition module 10, an identification module 20, a construction module 30, a judgment module 40 and a prompt module 50;
the acquisition module 10 is used for acquiring a 3D image of the shopping cart in the image acquisition area to obtain a 3D image in the cart;
the identification module 20 is configured to identify the in-vehicle 3D image to obtain a total volume of a current commodity corresponding to the in-vehicle 3D image;
the building module 30 is configured to obtain a monomer volume of each commodity in the shopping list according to the shopping list corresponding to the commodity in the shopping cart, calculate a total volume of all commodities in the shopping list, determine predicted stacking volumes of all commodities in the shopping cart stacked in different forms according to the total volume and the monomer volume, and build a predicted stacking volume database corresponding to the shopping list;
the judging module 40 is configured to judge whether a predicted stacking volume in a combination form that is the same as the total volume of the current commodity exists in the predicted stacking volume database;
the prompt module 50 is configured to generate a prompt message when it is determined that an abnormal condition occurs, so as to prompt a manager to perform troubleshooting according to the prompt message.
In addition, the invention also provides a user terminal which comprises a memory and a processor, wherein the memory is used for storing the shopping cart commodity verification program, and the processor runs the shopping cart commodity verification program so as to enable the user terminal to execute the shopping cart commodity verification method.
In addition, the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a shopping cart commodity verification program, and the shopping cart commodity verification program realizes the shopping cart commodity verification method when being executed by a processor.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (9)
1. A shopping cart commodity verification method is characterized by comprising the following steps:
3D image acquisition is carried out on the shopping cart in the image acquisition area, and a 3D image in the cart is obtained;
identifying the in-vehicle 3D image to obtain the total volume of the current commodity corresponding to the in-vehicle 3D image;
acquiring the monomer volume of each commodity in the shopping list according to the shopping list corresponding to the commodity in the shopping cart, calculating to obtain the total volume of all the commodities in the shopping list, determining the predicted stacking volume of all the commodities in the shopping cart stacked in different forms according to the total volume and the monomer volume, and constructing a predicted stacking volume database corresponding to the shopping list;
judging whether a predicted stacking volume in a combination form same as the total volume of the current commodity exists in the predicted stacking volume database or not;
if yes, judging that no abnormal condition occurs;
if not, judging that an abnormal condition occurs, and generating a prompt message so as to prompt a manager to check according to the prompt message.
2. The shopping cart merchandise verification method of claim 1, wherein prior to said capturing 3D images of shopping carts within the image capture area to obtain 3D images of carts within said cart, further comprising:
receiving a shopping list corresponding to the commodities in the shopping cart, and generating shopping cart movement prompt information to prompt a user to push the shopping cart into the image acquisition area;
receiving a shopping cart verification instruction generated by triggering an infrared sensor when the shopping cart completely enters the image acquisition area; and according to the shopping cart verification instruction, starting a multi-angle 3D image acquisition device so as to acquire 3D images of the shopping cart in the image acquisition area.
3. The shopping cart merchandise verification method of claim 2, wherein after said "turning on the multi-angle 3D image capturing device according to the shopping cart verification command", further comprising:
focusing and photometry are carried out on the shopping cart through a light ray detection device, and shooting parameters are obtained;
the step of acquiring the 3D image of the shopping cart in the image acquisition area to obtain the 3D image in the cart comprises the following steps:
sending a flash instruction to an auxiliary LED flash lamp connected with the multi-angle 3D image acquisition device through the multi-angle 3D image acquisition device according to the shooting parameters;
3D image acquisition based on the shooting parameters is carried out on the shopping cart in the image acquisition area through the multi-angle 3D image acquisition device, meanwhile, the auxiliary LED flash lamp synchronously flashes according to the flash instruction, and the 3D image in the cart after flash and light supplement is obtained.
4. The shopping cart merchandise verification method of claim 1, wherein prior to said capturing 3D images of shopping carts within the image capture area to obtain 3D images of carts within said cart, further comprising:
acquiring a current light brightness value in the shopping cart through a light sensor;
comparing the current light brightness value with a preset light threshold range;
and if the current light brightness value reaches the preset light threshold range, turning on an LED illuminating lamp to provide an illuminating light source.
5. The shopping cart merchandise verification method of claim 1, wherein after said "if, then determine no abnormal condition has occurred", further comprising:
identifying the 3D image in the shopping cart through a pre-trained commodity confirmation analysis model so as to determine the current commodity information in the shopping cart; the current commodity information comprises commodity varieties and commodity quantity corresponding to the commodity varieties; calculating to obtain the total weight of the current commodity according to the commodity variety, the commodity quantity corresponding to the commodity variety and the commodity weight corresponding to the commodity variety in the current commodity information;
calculating the total weight of the list of the commodities according to the shopping list;
acquiring the net weight of the commodities in the shopping cart in the image acquisition area through a weight sensor, and taking the net weight as the actual total weight of the commodities in the cart;
and if the total weight of the current commodities, the actual total weight of the commodities in the shopping cart and the total weight of the list are the same, judging that the commodities in the shopping cart are consistent with the commodities in the shopping list, and finishing the verification.
6. The shopping cart merchandise verification method of claim 2, wherein prior to said capturing 3D images of shopping carts within the image capture area to obtain 3D images of carts within said cart, further comprising:
acquiring the azimuth image of the shopping cart in the image acquisition area through an azimuth image acquisition device to obtain an azimuth height pre-judgment image;
identifying the azimuth height pre-judgment image to obtain azimuth information and height information corresponding to the articles in the shopping cart;
and controlling the multi-angle 3D image acquisition device to move to an image acquisition position which is adaptive to the azimuth information and the height information so as to acquire 3D images of the shopping cart in the image acquisition area at the image acquisition position.
7. A shopping cart item checkout apparatus, comprising: the device comprises an acquisition module, an identification module, a construction module, a judgment module and a prompt module;
the acquisition module is used for acquiring a 3D image of the shopping cart in the image acquisition area to obtain a 3D image in the cart;
the identification module is used for identifying the in-vehicle 3D image to obtain the total volume of the current commodity corresponding to the in-vehicle 3D image;
the building module is used for obtaining the monomer volume of each commodity in the shopping list according to the shopping list corresponding to the commodity in the shopping cart, calculating to obtain the total volume of all the commodities in the shopping list, determining the predicted stacking volume of all the commodities in the shopping cart stacked in different forms according to the total volume and the monomer volume, and building a predicted stacking volume database corresponding to the shopping list;
the judging module is used for judging whether the predicted stacking volume in the combined form which is the same as the total volume of the current commodity exists in the predicted stacking volume database or not;
and the prompt module is used for generating prompt information when the abnormal condition is judged to occur so as to prompt a manager to check according to the prompt information.
8. A user terminal comprising a memory for storing a shopping cart item verification program and a processor for executing the shopping cart item verification program to cause the user terminal to perform the shopping cart item verification method of any one of claims 1-6.
9. A computer-readable storage medium, wherein the computer-readable storage medium has stored thereon a shopping cart item verification program, which when executed by a processor implements the shopping cart item verification method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810964593.1A CN109166257B (en) | 2018-08-23 | 2018-08-23 | Shopping cart commodity verification method and device thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810964593.1A CN109166257B (en) | 2018-08-23 | 2018-08-23 | Shopping cart commodity verification method and device thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109166257A CN109166257A (en) | 2019-01-08 |
| CN109166257B true CN109166257B (en) | 2021-02-19 |
Family
ID=64896398
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810964593.1A Active CN109166257B (en) | 2018-08-23 | 2018-08-23 | Shopping cart commodity verification method and device thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109166257B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2606754A (en) * | 2021-05-20 | 2022-11-23 | Tj Morris Ltd | Detection of under-scanning at a point-of-sale |
| CN113916335B (en) * | 2021-09-16 | 2023-09-29 | 国网浙江省电力有限公司湖州供电公司 | Material box with counting function and counting method |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US3436968A (en) * | 1965-02-11 | 1969-04-08 | Fairbanks Morse Inc | Processing control system |
| US8055053B2 (en) * | 2006-12-14 | 2011-11-08 | The Hong Kong Polytechnic University | Physimetric property identification of physical object for process control |
| CN107945428A (en) * | 2017-10-20 | 2018-04-20 | 北京火星盒子网络信息技术有限公司 | Commodity checking method and commodity apparatus for checking |
| CN107862814A (en) * | 2017-11-01 | 2018-03-30 | 北京旷视科技有限公司 | Automated Clearing House system and automatic settlement method |
-
2018
- 2018-08-23 CN CN201810964593.1A patent/CN109166257B/en active Active
Non-Patent Citations (1)
| Title |
|---|
| 大型商场智慧购物系统研究;陈立龙;《系统仿真学报》;20161231;2966-2972 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109166257A (en) | 2019-01-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108922026B (en) | Replenishment management method and device for vending machine and user terminal | |
| CN109559454B (en) | Neural network-based commodity identification cash registering method and self-service cash registering desk | |
| US10467454B2 (en) | Synchronization of image data from multiple three-dimensional cameras for image recognition | |
| US10540551B2 (en) | Generation of two-dimensional and three-dimensional images of items for visual recognition in checkout apparatus | |
| KR20200037833A (en) | Item detection system and item detection method based on image monitoring | |
| TW201915860A (en) | Smart meal ordering method and device | |
| JP6261197B2 (en) | Display control apparatus, display control method, and program | |
| CN110866429A (en) | Missed scanning identification method and device, self-service cash register terminal and system | |
| JP2017527057A (en) | Log scanning system | |
| US11557080B2 (en) | Dynamically modeling an object in an environment from different perspectives | |
| CN109559453A (en) | Human-computer interaction device and its application for Automatic-settlement | |
| CN108960132B (en) | Method and device for purchasing commodities in open type vending machine | |
| CN111222870B (en) | Settlement method, device and system | |
| CN112585667A (en) | Intelligent platform counter display system and method | |
| US11281888B2 (en) | Separation of objects in images from three-dimensional cameras | |
| CN109166257B (en) | Shopping cart commodity verification method and device thereof | |
| CN110910628B (en) | Interactive processing method and device for vehicle damage image shooting and electronic equipment | |
| EP3901841A1 (en) | Settlement method, apparatus, and system | |
| CN106203225B (en) | Pictorial element based on depth is deleted | |
| CN110889845B (en) | Measuring method and device, electronic device and storage medium | |
| TWI712903B (en) | Commodity information inquiry method and system | |
| CN107533783A (en) | Product calling mechanism, program and control method | |
| US20140002646A1 (en) | Bottom of the basket surveillance system for shopping carts | |
| CN111507792A (en) | Self-service shopping method, computer readable storage medium and system | |
| CN111428743A (en) | Commodity identification method, commodity processing device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PP01 | Preservation of patent right | ||
| PP01 | Preservation of patent right |
Effective date of registration: 20211123 Granted publication date: 20210219 |
|
| PD01 | Discharge of preservation of patent | ||
| PD01 | Discharge of preservation of patent |
Date of cancellation: 20220415 Granted publication date: 20210219 |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220707 Address after: Room 368, 302, 211 Fute North Road, China (Shanghai) pilot Free Trade Zone, Pudong New Area, Shanghai Patentee after: Shanghai Yuepu Investment Center (L.P.) Address before: 518000 floor 33, Yantian modern industry service center, 3018 Shayan Road, Shatoujiao street, Yantian District, Shenzhen City, Guangdong Province Patentee before: SHENZHEN MALONG TECHNOLOGY Co.,Ltd. |