CN112507820A - Method, device and system for automatically checking goods and electronic equipment - Google Patents
Method, device and system for automatically checking goods and electronic equipment Download PDFInfo
- Publication number
- CN112507820A CN112507820A CN202011351576.4A CN202011351576A CN112507820A CN 112507820 A CN112507820 A CN 112507820A CN 202011351576 A CN202011351576 A CN 202011351576A CN 112507820 A CN112507820 A CN 112507820A
- Authority
- CN
- China
- Prior art keywords
- goods
- target
- image
- determining
- space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 230000015654 memory Effects 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 11
- 238000004891 communication Methods 0.000 claims description 10
- 230000002159 abnormal effect Effects 0.000 claims description 5
- 238000005516 engineering process Methods 0.000 abstract description 5
- 230000016776 visual perception Effects 0.000 abstract description 4
- 238000013136 deep learning model Methods 0.000 description 15
- 230000032258 transport Effects 0.000 description 13
- 238000001514 detection method Methods 0.000 description 10
- 230000011218 segmentation Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000009434 installation Methods 0.000 description 3
- 238000003672 processing method Methods 0.000 description 3
- 230000002547 anomalous effect Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/087—Inventory or stock management, e.g. order filling, procurement or balancing against orders
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
Abstract
The invention provides a method, a device, a system and electronic equipment for automatically checking goods, which comprises the following steps: when the target robot operates to the target goods position, acquiring a first image obtained by shooting goods placed in the target goods position by a camera; and automatically checking the goods of the target goods space based on the first image, and determining a preliminary quantity result and/or state of the goods placed in the target goods space. When the method for automatically checking the goods is used for checking the goods of the target goods position, only the first image of the goods of the target goods position is needed to be analyzed, namely, the visual perception technology is utilized to automatically check the goods of the target goods position, the efficiency is higher, the accuracy is good, the cost is low, and the requirement on the warehouse environment is low.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, a system, and an electronic device for automatically checking goods.
Background
In warehouse testing, inventory and breakage testing is often required for the quantity of items in a warehouse location. The traditional goods quantity checking mainly depends on the identification and management of RFID labels on a tray or a goods box, or manual operation is carried out to take out goods in a storage position and check the quantity of the goods; the breakage detection is mainly performed by a manual detection method.
The mode of checking the quantity of the artificial goods and detecting the artificial damage has low efficiency, high cost and poor accuracy; in the mode of identification and management by means of the RFID tags, the RFID tags are easy to damage, the common RFID tags are not strong in anti-interference capability, certain requirements are placed on the environment of a warehouse, the high-quality RFID tags are high in cost, and extra equipment expenditure is required for reading the RFID tags.
Disclosure of Invention
In view of the above, the present invention provides a method, an apparatus, a system and an electronic device for automatically checking goods, so as to alleviate the technical problems of low efficiency, high cost, poor accuracy and poor interference resistance of the existing goods checking method.
In a first aspect, an embodiment of the present invention provides a method for automatically checking goods, where the method is applied to a server; the server is respectively in communication connection with a target robot and a camera installed on the target robot; the method comprises the following steps: when the target robot operates to a target goods position, acquiring a first image obtained by shooting goods placed in the target goods position by the camera; and automatically checking the goods of the target goods space based on the first image, and determining a preliminary quantity result and/or state of the goods placed in the target goods space.
Further, the method further comprises: acquiring a second image obtained by shooting a target surface of the goods when the goods of the target goods space are put in storage, wherein the target surface is the same as the surface of the goods displayed in the first image; the automatically checking the goods of the target goods space based on the first image, and determining the preliminary quantity result and/or the state of the goods placed in the target goods space comprise: automatically checking the item based on the first image and the second image, determining a preliminary quantity result and/or status of the item.
Further, automatically checking the item based on the first image and the second image, determining a preliminary quantity result and/or status of the item, comprising: determining a status of the item based on the first image; and comparing the first image with the second image, and determining whether the goods in the first image and the goods in the second image have differences or not to obtain the preliminary quantity result of the goods.
Further, if the preliminary quantity results in a difference between the item in the first image and the item in the second image, the method further comprises: controlling the target robot to transport the goods to a first target location; after the goods are conveyed to the first target position, controlling a plurality of cameras of the first target position to shoot different surfaces of the goods to obtain a plurality of images, wherein the different surfaces comprise: front, back, top, left side, and right side; receiving a plurality of images shot by a plurality of cameras at the first target position; determining a quantity of the item based on the plurality of images, thereby determining whether the quantity of the item is the same as a historical quantity.
Further, the method further comprises: if the quantity of the goods is different from the quantity of the historical records, controlling the target robot to convey the goods to an abnormal processing position, and determining difference information based on an image obtained by shooting the goods when the goods in the target goods position are put in storage and the plurality of images; and if the quantity of the goods is the same as the quantity of the historical records, controlling the target robot to transport the goods back to the target goods space.
Further, the method further comprises: and sending the difference information to a specified user terminal.
Further, the method further comprises: controlling the target robot to transport the article to an exception handling location if the state of the article is at least one of broken, deformed, and stained.
In a second aspect, the embodiment of the present invention further provides an apparatus for automatically checking goods, where the apparatus is applied to a server, and the server is respectively in communication connection with a target robot and a camera installed on the target robot; the device comprises: the acquisition unit is used for acquiring a first image obtained by shooting goods placed in a target goods position by the camera after the target robot operates to the target goods position; and the determining unit is used for automatically checking the goods of the target goods space based on the first image and determining the preliminary quantity result and/or the state of the goods placed in the target goods space.
In a third aspect, an embodiment of the present invention further provides a system for automatically checking a good, where the system for automatically checking a good includes: the system comprises a server, a target robot and a camera installed on the target robot; the server is respectively in communication connection with the target robot and the camera; the server is configured to perform the steps of the method for automatically checking goods according to any one of the first aspect.
Further, the target cargo space is a space in a stereoscopic warehouse, the target robot is a stacker, the camera is arranged at a second target position of a cargo carrying platform of the stacker, and the second target position includes any one of the following positions: the upper corner position of the cargo carrying platform and the upper cross beam position of the cargo carrying platform.
Furthermore, the multiple images are obtained by shooting different surfaces of the goods by multiple cameras at the position of the warehousing port.
In a third aspect, an embodiment of the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the method according to any one of the above first aspects when executing the computer program.
In a fourth aspect, an embodiment of the present invention provides a computer storage medium, on which a computer program is stored, and when the computer program runs on a computer, the computer executes the steps of the method according to any one of the first aspect.
In the embodiment of the invention, after the target robot operates to the target goods position, a first image obtained by shooting goods placed in the target goods position by a camera is obtained; and then automatically checking the goods of the target goods space based on the first image, and determining a preliminary quantity result and/or state of the goods placed in the target goods space. According to the description, when the goods of the target goods position are checked, the method for automatically checking the goods only needs to analyze the first image of the goods position, namely, the goods of the goods position can be automatically checked by utilizing the visual perception technology, the efficiency is higher, the accuracy is good, the cost is low, the requirement on the warehouse environment is low, the technical problems that the existing mode for manually detecting the goods of the goods position is low in efficiency, high in labor cost and poor in accuracy are solved, and the technical problems that the existing goods quantity checking by means of RFID tags is poor in anti-interference capability and the equipment cost is high are solved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a schematic diagram of an electronic device according to an embodiment of the present invention;
fig. 2 is a flowchart of a method for automatically checking goods according to an embodiment of the present invention;
FIG. 3 is a flowchart of a method for acquiring a first image according to an embodiment of the present invention;
FIG. 4 is a flow diagram of a method for detecting a preliminary quantity result and/or status of an item according to an embodiment of the present invention;
FIG. 5 is a flow chart of a method for quantity inventory of an item according to an embodiment of the present invention;
fig. 6 is a schematic view of an installation position of a camera on a cargo carrying platform of a stacker according to an embodiment of the present invention;
fig. 7 is an interaction diagram of a method for automatically checking goods according to an embodiment of the present invention;
fig. 8 is a schematic view illustrating an apparatus for automatically checking goods according to an embodiment of the present invention;
fig. 9 is a block diagram of a system for automatically checking goods according to an embodiment of the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the following embodiments, and it should be understood that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1:
first, an electronic device 100 for implementing an embodiment of the present invention, which may be used to run the method of automatically checking goods according to embodiments of the present invention, is described with reference to fig. 1.
As shown in fig. 1, electronic device 100 includes one or more processors 102, one or more memories 104. Optionally, the electronic device 100 may also include an input device 106, an output device 108, a camera 110, which are interconnected via a bus system 112 and/or other form of connection mechanism (not shown). It should be noted that the components and structure of the electronic device 100 shown in fig. 1 are exemplary only, and not limiting, and the electronic device may have other components and structures as desired.
The processor 102 may be implemented in at least one hardware form of a Digital Signal Processor (DSP), a Field-Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), and an asic (application Specific Integrated circuit), and the processor 102 may be a Central Processing Unit (CPU) or other form of Processing Unit having data Processing capability and/or instruction execution capability, and may control other components in the electronic device 100 to perform desired functions.
The memory 104 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. On which one or more computer program instructions may be stored that may be executed by processor 102 to implement client-side functionality (implemented by the processor) and/or other desired functionality in embodiments of the invention described below. Various applications and various data, such as various data used and/or generated by the applications, may also be stored in the computer-readable storage medium.
The input device 106 may be a device used by a user to input instructions and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.
The output device 108 may output various information (e.g., images or sounds) to the outside (e.g., a user), and may include one or more of a display, a speaker, and the like.
The camera 110 is used to capture an image of an item placed in the target cargo space and may also store the captured first image in the memory 104, with the data stored in the memory 104 being available to other components.
For example, the electronic device for implementing the method for automatically checking goods according to the embodiment of the present invention may be implemented as a smart mobile terminal such as a smart phone, a tablet computer, etc., and may also be implemented as any other device with computing capability.
Example 2:
in accordance with an embodiment of the present invention, there is provided an embodiment of a method for automatically inventorying goods, it is noted that the steps illustrated in the flowchart of the figures may be performed in a computer system such as a set of computer executable instructions and that while a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than that presented herein.
Fig. 2 is a flowchart of a method for automatically checking inventory of goods according to an embodiment of the present invention, which is applied to a server, typically a server of a warehouse management system or a warehouse control system, and the server is communicatively connected to a target robot, a camera installed on the target robot, and a camera at another position in a warehouse (e.g., an warehousing camera), as shown in fig. 2, and the method includes the following steps:
step S202, after the target robot operates to the target goods position, a first image obtained by shooting goods placed in the target goods position by the camera is obtained.
In the embodiment of the present invention, the target robot may be different types of handling equipment, such as a stacker, a drum robot, a jack-up robot, a forklift, and the like.
When the target robot is a stacker, the camera may be disposed on a cargo carrying platform of the stacker, and specifically, after the cargo carrying platform of the stacker runs to the target cargo space, the camera shoots the goods placed in the target cargo space to obtain a first image.
It should be noted that the above-mentioned articles may be placed on the target cargo space in a stacked form, which means a stack shape formed by stacking a plurality of articles together. For example, the goods may have a box or box-like outer packaging.
Step S204, automatically checking the goods of the target goods space based on the first image, and determining the initial quantity result and/or the state of the goods placed in the target goods space.
And after the first image is acquired, quickly performing image checking analysis to obtain an initial quantity result and/or state of goods placed in the target goods space. The state of the article is used for indicating whether at least one of breakage, deformation and stain exists in the article. The preliminary quantity result of the goods is used for indicating whether the quantity of the goods in the first image is different from the quantity of the goods in the second image shot when the goods are put in storage, in other words, the preliminary quantity result represents whether the quantity of the goods placed at the target goods position currently is different from the quantity of the goods when the goods are put in storage.
The specific checking mode has various modes, for example, the checking mode is carried out by comparing the collected first image with a pre-stored goods image without damage, deformation and/or stain; or checking the acquired first image and a second image shot when the goods are put in storage in a comparison mode; or, the first image is checked through a pre-trained model to realize the state and/or quantity detection of the goods, wherein the trained model is a model trained based on a deep learning algorithm, for example, a model trained through a CNN neural network.
In the embodiment of the invention, after the target robot operates to the target goods position, a first image obtained by shooting goods placed in the target goods position by a camera is obtained; and then automatically checking the goods of the target goods space based on the first image, and determining a preliminary quantity result and/or state of the goods placed in the target goods space. According to the description, when the goods of the target goods position are checked, the method for automatically checking the goods only needs to analyze the first image of the goods position, namely, the goods of the goods position can be automatically checked by utilizing the visual perception technology, the efficiency is higher, the accuracy is good, the cost is low, the requirement on the warehouse environment is low, the technical problems that the existing mode for manually detecting the goods of the goods position is low in efficiency, high in labor cost and poor in accuracy are solved, and the technical problems that the existing goods quantity checking by means of RFID tags is poor in anti-interference capability and the equipment cost is high are solved.
In addition, the embodiment of the invention not only can automatically count the number of the goods, but also can detect the state of the goods, and can find the goods in time if the goods are damaged, deformed and/or stained so as to be convenient for workers to handle.
The above description briefly describes the method for automatically checking goods according to the present invention, and the detailed description thereof will be given below.
In an alternative embodiment of the present invention, referring to fig. 3, step S202, the step of obtaining a first image of an article placed in the target cargo space by the camera includes:
step S301, a task of detecting goods of the target goods position is sent to the target robot so that the target robot can operate to the target goods position.
Specifically, after the target robot receives a task of detecting goods in the target goods space, the goods carrying platform of the target robot runs to the target goods space.
And step S302, after the target robot runs to the target goods position, controlling the camera to shoot goods on the target goods position.
In step S303, a first image captured by a camera is received.
In an alternative embodiment of the present invention, referring to fig. 4, the method further comprises:
step S401, a second image obtained by shooting the target surface of the goods when the goods of the target goods space are put into the warehouse is obtained.
Wherein the target face is the same face as the face of the article displayed in the first image. When goods are put in a warehouse, checking is carried out at a warehouse inlet, five sides (including the front side, the back side, the upper side, the left side and the right side) of the goods (which can be specifically stacks) are shot during checking, so that warehouse goods images are obtained, and the warehouse goods images can be named and stored according to the serial numbers of storage positions of the goods in the warehouse corresponding to the warehouse goods and the shot surfaces of the goods. When the goods of the target goods position is to be detected, the corresponding second image can be called according to the number corresponding to the target goods position and the shooting surface of the goods.
Step S402, automatically checking the goods based on the first image and the second image, and determining the preliminary quantity result and/or the state of the goods.
The method specifically comprises at least one implementation mode of the following steps:
(1) determining a status of the item based on the first image;
for example, the server according to the embodiment of the present application may be configured with a first deep learning model in advance, where the first deep learning model is obtained by training a neural network through a large number of training samples, and the first deep learning model may be used to perform damage, deformation, and/or stain detection on an image, and perform corresponding labeling. For example, a first image may be input into a first deep learning model and then a determination may be made as to whether the good is broken, deformed, and/or soiled based on the results output by the first deep learning model. If it is determined that the goods have breakage, deformation and/or dirt, the first deep learning model can mark the breakage, deformation and/or dirt area on the first image so as to facilitate subsequent processing of the goods.
(2) Determining a status of the good based on the first image and the second image;
for example, it may be determined by an image processing method whether there is a difference in the goods in the first image and the second image, and thus the state of the goods. If the first image is consistent with the second image, the current state of the goods is consistent with the state of the goods in the warehouse, and no damage, deformation and/or stain exists; if the first image and the second image are not consistent, the current state of the goods is different from the state of the goods when being put in storage, and damage, deformation and/or dirt are presumed to exist. The image processing method adopted here is not limited, and for example, the difference between two images can be found out by adopting the existing traditional image processing method, and then the two images are marked; the difference of the two images can be determined by adopting a trained deep learning model, and a large number of image sample pairs containing partial differences can be adopted for training, so that the deep learning model can find the difference of the two images and mark the difference.
And if the damage, deformation and/or dirt of the goods of the target goods position are determined in any one of the modes, controlling the target robot to convey the goods to the abnormal processing position, and carrying out subsequent processing by manual or automatic equipment. Optionally, the server may also mark the item as anomalous in the system, so that the item will not be available for normal operation in the warehouse.
(3) And comparing the first image with the second image, and determining whether the goods in the first image and the goods in the second image have differences or not to obtain a preliminary quantity result of the goods.
When the first image and the second image are compared, the second deep learning model can be adopted to respectively carry out goods example segmentation on the first image and the second image, and then the results of the two goods example segmentation are compared, so that whether the goods quantity in the first image and the goods quantity in the second image are different or not is determined.
In an optional embodiment of the invention, the item may also be re-inventoried if the preliminary quantity detection results in a difference between the item in the first image and the item in the second image. Referring to fig. 5, the method shown in fig. 2 may further include:
step S501, the target robot is controlled to transport the goods to the first target position.
And a plurality of cameras are arranged at the first target positions, so that different surfaces of the goods can be shot. For example, the first target position may be a position of a warehousing port, and in a specific implementation, the server controls the target robot to transport the goods to the position of the warehousing port.
Step S502, after the goods are transported to the first target position, a plurality of cameras of the first target position are controlled to shoot different surfaces of the goods to obtain a plurality of images.
Wherein, different faces include: front, back, top, left side, and right side; the multiple images obtained by shooting different surfaces of the goods by the multiple cameras can be shot in the process that the target robot conveys the goods to enter from the warehousing port. Specifically, three cameras can be arranged at the position of a warehousing port (similar to a door frame), one camera is arranged at the top end of the warehousing port, the other two cameras are arranged at two sides of the warehousing port, when a target robot transports goods to reach the position of the warehousing port, the top camera shoots the front side of the goods to obtain a front side image of the goods, then the target robot continues to transport the goods to reach the position under the warehousing port, at the moment, the three cameras respectively shoot the upper side, the left side and the right side of the goods to obtain an upper side image of the goods, the left side image of the goods and the right side image of the goods, the target robot continues to transport the goods to pass through the position of the warehousing port, at the moment, the top camera shoots the back side of the goods to obtain a back side image of the goods, and therefore multiple images of the five sides of the.
In step S503, a plurality of images captured by a plurality of cameras at the first target position are received.
Step S504, the quantity of the goods is determined based on the plurality of images, and whether the quantity of the goods is the same as the quantity of the historical records is further determined.
In consideration of the efficiency and the precision of the goods quantity detection, the server of the embodiment of the application is pre-configured with a second deep learning model, and the second deep learning model is obtained by training a neural network through a large number of training samples and can be used for carrying out goods instance segmentation. In an embodiment of the present application, the step of determining the quantity of the goods includes:
1) inputting a plurality of images into a second deep learning model;
2) and determining the quantity of the goods according to the goods example segmentation result output by the second deep learning model.
In the process, an example segmentation algorithm based on deep learning is used for carrying out example segmentation on the goods of a plurality of images, the number of layers of the goods and the distribution of the goods of each layer are determined according to a segmentation result, then the number of all the goods (namely stacking) can be calculated, and then the determined number of the goods is compared with the historical record number to determine whether the goods are the same or not.
If the quantity of the goods is different from the quantity of the historical records, the target robot is controlled to convey the goods to the abnormal processing position, images and a plurality of images obtained by shooting the goods when the goods based on the target goods position are put into the warehouse are determined to obtain difference information, the difference information is sent to the appointed user terminal, and the difference information is processed by manual work and/or automatic equipment. Optionally, the server may also mark the item as anomalous in the system, so that the item will not be available for normal operation in the warehouse.
The difference information is determined according to the example segmentation result of the second deep learning model on the images shot when the goods are put in storage and the example segmentation result of the second deep learning model on the multiple images.
The difference information may be quantity difference information of the goods (for example, the goods in the target goods space are less than X boxes and more than Y boxes), position and quantity difference information of the goods (for example, compared with the images shot when the goods are put in storage, the a position in the multiple images is short of the goods), and the like.
In addition, the sending of the difference information to the specified user terminal may be sending the difference information to a terminal corresponding to a pre-stored mobile phone number, or a terminal corresponding to a pre-stored email, or a terminal corresponding to monitoring software, and the like, where the user terminal may be a smart phone, or a handheld pda (personal Digital assistant) device, and the difference information may be a short message or an email message.
And after receiving the difference information, the user corresponding to the appointed user terminal replenishes or removes the goods of the target goods location according to the difference information so as to ensure that the goods of the target goods location are the same as the quantity of the historical records.
And if the quantity of the goods is the same as the quantity of the historical records, controlling the target robot to transport the goods back to the target goods position.
In an alternative embodiment of the present invention, referring to fig. 6, the target robot is a stacker, and the camera is disposed at a second target position of a cargo bed of the stacker, where the second target position includes any one of: the position of the upper corner of the cargo carrying platform and the position of the upper beam of the cargo carrying platform (the second target position is not particularly limited in the embodiment of the invention, and any other position capable of completely shooting the goods of the target cargo space can be adopted).
In addition, when the cameras are arranged on the goods carrying platforms of the stacking machines, detection distances, detection angles and other parameter standards (lens focal lengths, resolution ratios, storage modes, networking modes, power supply modes, video coding and the like) are determined through experiments, and then the installation number and the installation angles of the cameras are determined, so that the cameras can completely shoot goods of the target goods positions.
In order to more intuitively understand the above process, the method for automatically checking the goods provided by the embodiments is described by taking an interactive diagram of the method for automatically checking the goods shown in fig. 7 as an example, and the method mainly includes the following steps:
s1: the server receives an operation task of detecting goods of the target goods space, and calls an image obtained by shooting when the goods of the target goods space are put in storage;
s2: the server sends an instruction for running to a target goods space to the stacker;
s3: the stacker operates the cargo carrying platform to a target cargo space according to the operation instruction;
s4: after the cargo platform operates to the target cargo space, the server sends a starting instruction to the camera;
s5: the camera executes the starting instruction and shoots goods on the target goods position to obtain a first image;
s6: the camera transmits a shot first image to a server in real time;
s7: the server stores the first image, carries out damage, deformation and/or stain detection on the first image, and determines whether the first image is different from an image shot when a corresponding goods is put in storage;
s8: the server determines that the goods are damaged, deformed and/or stained, and sends an instruction for conveying the goods to an exception handling position to the stacker;
s9: the stacker conveys goods to an exception handling position according to the operation instruction;
s10: the server determines that the first image is different from the corresponding image shot when the goods are put in storage, and sends an instruction for conveying the goods to the position of a storage port to the stacker;
s11: the stacker conveys goods to a warehousing port according to the operation instruction;
s12: after the goods reach the position of the warehousing port, the server sends a starting-up instruction to the warehousing camera;
s13: the warehousing camera executes the starting-up instruction and shoots goods of the target goods space;
s14: the warehousing camera transmits a plurality of shot images to the server in real time;
s15: the server stores a plurality of images, calculates the quantity of goods, and judges whether the quantity of the goods obtained by calculation is the same as the quantity of the historical records;
s16: the server determines that the quantity of the goods obtained by calculation is the same as the quantity of the historical records, and sends an instruction for transporting the goods back to the target goods position to the stacker;
s17: the stacker transports the goods back to the target goods position according to the operation instruction;
s18: the server determines that the number of the goods obtained by calculation is different from the number of the historical records, and sends an instruction for conveying the goods to an exception handling position to the stacker;
s19: the stacker conveys goods to an exception handling position according to the operation instruction;
s20: the server determines that the number of the goods obtained by calculation is different from the number of the historical records, analyzes the image and the multiple images obtained by shooting when the goods are put in storage to obtain difference information, and sends the difference information to the user side;
s21: the goods are processed by manual or automatic equipment, and confirmation information is sent to the server;
s22: the goods processing is completed.
It should be noted that the above is only a specific implementation, and some steps may be changed or adjusted.
Compared with a mode of manually checking goods in a goods position, the method for automatically checking the goods can quickly finish the detection of the goods in the goods position, and is good in accuracy and low in cost; compared with a mode of checking the quantity of goods by means of the RFID tags, the method has the advantages of low requirement on the environment, low equipment cost and good practicability.
Example 3:
the embodiment of the invention also provides a device for automatically checking the goods, which is mainly used for executing the method for automatically checking the goods provided by the embodiment of the invention.
Fig. 8 is a schematic view of an apparatus for automatically inventorying goods according to an embodiment of the present invention, as shown in fig. 8, which is applied to a server communicatively connected to a target robot and a camera installed at the target robot, respectively; the device includes: an acquisition unit 10 and a determination unit 20, wherein:
the acquisition unit is used for acquiring a first image obtained by shooting goods placed in the target goods space by the camera after the target robot operates to the target goods space;
and the determining unit is used for automatically checking the goods of the target goods space based on the first image and determining the preliminary quantity result and/or the state of the goods placed in the target goods space.
In the embodiment of the invention, after the target robot operates to the target goods position, a first image obtained by shooting goods placed in the target goods position by a camera is obtained; and then automatically checking the goods of the target goods space based on the first image, and determining a preliminary quantity result and/or state of the goods placed in the target goods space. According to the description, when the goods of the target goods position are checked, the method for automatically checking the goods only needs to analyze the first image of the goods position, namely, the goods of the goods position can be automatically checked by utilizing the visual perception technology, the efficiency is higher, the accuracy is good, the cost is low, the requirement on the warehouse environment is low, the technical problems that the existing mode for manually detecting the goods of the goods position is low in efficiency, high in labor cost and poor in accuracy are solved, and the technical problems that the existing goods quantity checking by means of RFID tags is poor in anti-interference capability and the equipment cost is high are solved.
Optionally, the obtaining unit is further configured to: acquiring a second image obtained by shooting a target surface of the goods when the goods of the target goods space are put into the warehouse, wherein the target surface is the same as the surface of the goods displayed in the first image; the determining unit is further configured to: an automatic inventory of the item is performed based on the first image and the second image, and a preliminary quantity result and/or status of the item is determined.
Optionally, the determining unit is further configured to: determining a status of the item based on the first image; and comparing the first image with the second image, and determining whether the goods in the first image and the goods in the second image have differences or not to obtain a preliminary quantity result of the goods.
Optionally, if the preliminary quantity results in a difference between the item in the first image and the item in the second image, the apparatus is further configured to: controlling the target robot to transport the goods to the first target location; after the goods are conveyed to the first target position, a plurality of images obtained by shooting different surfaces of the goods by a plurality of cameras controlling the first target position are obtained, wherein the different surfaces comprise: front, back, top, left side, and right side; receiving a plurality of images shot by a plurality of cameras at a first target position; the number of items is determined based on the plurality of images, and it is determined whether the number of items is the same as the history number.
Optionally, the apparatus is further configured to: if the quantity of the goods is different from the quantity of the historical records, controlling the target robot to convey the goods to an abnormal processing position, and determining difference information based on an image obtained by shooting the goods when the goods of the target goods position are put into a warehouse and a plurality of images; and if the quantity of the goods is the same as the quantity of the historical records, controlling the target robot to transport the goods back to the target goods position.
Optionally, the apparatus is further configured to: and sending the difference information to the specified user terminal.
Optionally, the apparatus is further configured to: and controlling the target robot to transport the article to the abnormality treatment position if the state of the article is at least one of the breakage, deformation, and stain.
The device provided by the embodiment of the present invention has the same implementation principle and technical effect as the method embodiments, and for the sake of brief description, reference may be made to the corresponding contents in the method embodiments without reference to the device embodiments.
Example 4:
based on the above method embodiment, an embodiment of the present invention further provides a system for automatically checking a good, referring to a structural block diagram of the system for automatically checking a good shown in fig. 9, where the system includes: a server 90, a target robot 91, and a camera 92 mounted on the target robot; the server 90 is respectively in communication connection with the target robot 91 and the camera 92; the server is used for executing the steps of the method for automatically checking goods in the above method embodiment.
Optionally, the target cargo space is a space in a stereoscopic warehouse, the target robot is a stacker, the camera is disposed at a second target position of a cargo carrying platform of the stacker, and the second target position includes any one of the following positions: the upper corner position of the cargo carrying platform and the upper cross beam position of the cargo carrying platform.
Optionally, the multiple images are obtained by shooting different surfaces of the goods by multiple cameras at the position of the warehousing port.
The system provided by the embodiment of the present invention has the same implementation principle and technical effect as the foregoing method embodiment, and for the sake of brief description, no mention is made in the system embodiment, and reference may be made to the corresponding contents in the foregoing method embodiment.
In another implementation of the present invention, there is further provided a computer storage medium having a computer program stored thereon, the computer program, when executed by a computer, performing the steps of the method of any one of the above method embodiments 2.
In addition, in the description of the embodiments of the present invention, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (13)
1. A method for automatically checking goods is characterized in that the method is applied to a server; the server is respectively in communication connection with a target robot and a camera installed on the target robot; the method comprises the following steps:
when the target robot operates to a target goods position, acquiring a first image obtained by shooting goods placed in the target goods position by the camera;
and automatically checking the goods of the target goods space based on the first image, and determining a preliminary quantity result and/or state of the goods placed in the target goods space.
2. The method of claim 1, further comprising:
acquiring a second image obtained by shooting a target surface of the goods when the goods of the target goods space are put in storage, wherein the target surface is the same as the surface of the goods displayed in the first image;
the automatically checking the goods of the target goods space based on the first image, and determining the preliminary quantity result and/or the state of the goods placed in the target goods space comprise: automatically checking the item based on the first image and the second image, determining a preliminary quantity result and/or status of the item.
3. The method of claim 2, wherein automatically inventorying the item based on the first image and the second image, determining a preliminary quantity result and/or status of the item, comprises:
determining a status of the item based on the first image;
and comparing the first image with the second image, and determining whether the goods in the first image and the goods in the second image have differences or not to obtain the preliminary quantity result of the goods.
4. The method of claim 3, wherein if the preliminary quantity results in a difference between the item in the first image and the item in the second image, the method further comprises:
controlling the target robot to transport the goods to a first target location;
after the goods are conveyed to the first target position, controlling a plurality of cameras of the first target position to shoot different surfaces of the goods to obtain a plurality of images, wherein the different surfaces comprise: front, back, top, left side, and right side;
receiving a plurality of images shot by a plurality of cameras at the first target position;
determining a quantity of the item based on the plurality of images, thereby determining whether the quantity of the item is the same as a historical quantity.
5. The method of claim 4, further comprising:
if the quantity of the goods is different from the quantity of the historical records, controlling the target robot to convey the goods to an abnormal processing position, and determining difference information based on an image obtained by shooting the goods when the goods in the target goods position are put in storage and the plurality of images;
and if the quantity of the goods is the same as the quantity of the historical records, controlling the target robot to transport the goods back to the target goods space.
6. The method of claim 5, further comprising:
and sending the difference information to a specified user terminal.
7. The method of claim 1, further comprising:
controlling the target robot to transport the article to an exception handling location if the state of the article is at least one of broken, deformed, and stained.
8. An automatic goods checking device is characterized in that the device is applied to a server which is respectively in communication connection with a target robot and a camera installed on the target robot; the device comprises:
the acquisition unit is used for acquiring a first image obtained by shooting goods placed in a target goods position by the camera after the target robot operates to the target goods position;
and the determining unit is used for automatically checking the goods of the target goods space based on the first image and determining the preliminary quantity result and/or the state of the goods placed in the target goods space.
9. A system for automatically checking goods, comprising: the system comprises a server, a target robot and a camera installed on the target robot;
the server is respectively in communication connection with the target robot and the camera;
the server is adapted to perform the steps of the method of automatically inventorying goods according to any of claims 1-7.
10. The system of claim 9, wherein the target cargo space is a space in a stereoscopic warehouse, the target robot is a stacker, the camera is disposed at a second target position of a cargo carrying platform of the stacker, and the second target position includes any one of: the upper corner position of the cargo carrying platform and the upper cross beam position of the cargo carrying platform.
11. The system of claim 9, wherein the plurality of images are taken of different sides of the item by a plurality of cameras at the loading bay location.
12. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any of claims 1 to 7 when executing the computer program.
13. A computer storage medium, having a computer program stored thereon, which, when executed by a computer, performs the steps of the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011351576.4A CN112507820A (en) | 2020-11-25 | 2020-11-25 | Method, device and system for automatically checking goods and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011351576.4A CN112507820A (en) | 2020-11-25 | 2020-11-25 | Method, device and system for automatically checking goods and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112507820A true CN112507820A (en) | 2021-03-16 |
Family
ID=74966399
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011351576.4A Pending CN112507820A (en) | 2020-11-25 | 2020-11-25 | Method, device and system for automatically checking goods and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112507820A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113362005A (en) * | 2021-06-21 | 2021-09-07 | 山东产研信息与人工智能融合研究院有限公司 | Intelligent inventory method and system for goods in unmanned warehouse based on environment perception |
| CN113657833A (en) * | 2021-08-20 | 2021-11-16 | 北京京东振世信息技术有限公司 | Material checking method, device, equipment and storage medium |
| CN113998359A (en) * | 2021-11-03 | 2022-02-01 | 龙岩烟草工业有限责任公司 | Data verification method, device and equipment, logistics system and storage medium |
| CN115346318A (en) * | 2022-10-19 | 2022-11-15 | 航信德利信息系统(上海)有限公司 | Escort processing system and use method thereof |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101853387A (en) * | 2010-04-02 | 2010-10-06 | 北京物资学院 | Stereoscopic warehouse goods checking method and system |
| CN207281587U (en) * | 2017-05-05 | 2018-04-27 | 费舍尔物流科技（苏州）有限公司 | A kind of robot based on image recognition and movement control technology |
| CN110136129A (en) * | 2019-05-22 | 2019-08-16 | 广东工业大学 | A kind of commercial quality detection method, device and storage medium |
| US20200293830A1 (en) * | 2018-04-03 | 2020-09-17 | Alibaba Group Holding Limited | Article damage detection |
| CN111832420A (en) * | 2020-06-17 | 2020-10-27 | 深圳达闼科技控股有限公司 | Article processing method, article processing apparatus, medium, and vending machine |
| CN111898935A (en) * | 2020-06-24 | 2020-11-06 | 广州市玄武无线科技股份有限公司 | Storage checking method, system storage and terminal equipment based on image processing |
| US10832476B1 (en) * | 2018-04-30 | 2020-11-10 | State Farm Mutual Automobile Insurance Company | Method and system for remote virtual visualization of physical locations |
-
2020
- 2020-11-25 CN CN202011351576.4A patent/CN112507820A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101853387A (en) * | 2010-04-02 | 2010-10-06 | 北京物资学院 | Stereoscopic warehouse goods checking method and system |
| CN207281587U (en) * | 2017-05-05 | 2018-04-27 | 费舍尔物流科技（苏州）有限公司 | A kind of robot based on image recognition and movement control technology |
| US20200293830A1 (en) * | 2018-04-03 | 2020-09-17 | Alibaba Group Holding Limited | Article damage detection |
| US10832476B1 (en) * | 2018-04-30 | 2020-11-10 | State Farm Mutual Automobile Insurance Company | Method and system for remote virtual visualization of physical locations |
| CN110136129A (en) * | 2019-05-22 | 2019-08-16 | 广东工业大学 | A kind of commercial quality detection method, device and storage medium |
| CN111832420A (en) * | 2020-06-17 | 2020-10-27 | 深圳达闼科技控股有限公司 | Article processing method, article processing apparatus, medium, and vending machine |
| CN111898935A (en) * | 2020-06-24 | 2020-11-06 | 广州市玄武无线科技股份有限公司 | Storage checking method, system storage and terminal equipment based on image processing |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113362005A (en) * | 2021-06-21 | 2021-09-07 | 山东产研信息与人工智能融合研究院有限公司 | Intelligent inventory method and system for goods in unmanned warehouse based on environment perception |
| CN113657833A (en) * | 2021-08-20 | 2021-11-16 | 北京京东振世信息技术有限公司 | Material checking method, device, equipment and storage medium |
| CN113657833B (en) * | 2021-08-20 | 2024-04-16 | 北京京东振世信息技术有限公司 | Material inventory method, device, equipment and storage medium |
| CN113998359A (en) * | 2021-11-03 | 2022-02-01 | 龙岩烟草工业有限责任公司 | Data verification method, device and equipment, logistics system and storage medium |
| CN115346318A (en) * | 2022-10-19 | 2022-11-15 | 航信德利信息系统(上海)有限公司 | Escort processing system and use method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112507820A (en) | Method, device and system for automatically checking goods and electronic equipment | |
| CN109635132B (en) | Automatic warehouse management method, system and terminal equipment | |
| CN106573381B (en) | The visualization of truck unloader | |
| US9594979B1 (en) | Probabilistic registration of interactions, actions or activities from multiple views | |
| TWI816235B (en) | Method and apparatus for storing material, robot, warehousing system and storage medium | |
| EP4156023A1 (en) | Material inventory method and apparatus, and warehousing robot and warehousing system | |
| WO2022105231A1 (en) | Warehousing management method and apparatus, warehouse management system and electronic system | |
| US11288539B1 (en) | Tiered processing for item identification | |
| CN111661548B (en) | Article sorting method, apparatus, device and storage medium | |
| US10805556B1 (en) | Storage units with shifted-lens cameras | |
| CN112824990B (en) | Cargo information detection method and system, robot and processing terminal | |
| US20210374659A1 (en) | Real Time Event Tracking and Digitization for Warehouse Inventory Management | |
| US10713614B1 (en) | Weight and vision based item tracking | |
| JP2020015576A (en) | Automatic warehouse system | |
| CN113393182A (en) | Storage method and device for material box and intelligent storage system | |
| WO2022222801A1 (en) | Warehousing management method and apparatus, warehousing robot, warehousing system, and medium | |
| CN113044450B (en) | Task processing method and device for conveying equipment | |
| CN106429144B (en) | Method, device and system for processing glass cassette | |
| JP5674933B2 (en) | Method and apparatus for locating an object in a warehouse | |
| JP6450576B2 (en) | Book checking device, book checking method, and program for book storage management system | |
| CN116310405A (en) | Goods identification method based on warehouse location, unmanned forklift and storage medium | |
| CN116468366A (en) | Inventory system, inventory method, inventory stand, electronic device and computer medium | |
| JP5745142B1 (en) | Automatic library system | |
| CN113083695A (en) | Goods information rechecking method, device, equipment and storage medium | |
| CN111667628A (en) | Control method, device, equipment and storage medium of unmanned vending device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |