CN113743856A

CN113743856A - Article sorting method and device, and storage medium

Info

Publication number: CN113743856A
Application number: CN202110009590.4A
Authority: CN
Inventors: 李梅; 郭君章; 单纪昌
Original assignee: Beijing Jingdong Qianshi Technology Co Ltd
Current assignee: Beijing Jingdong Qianshi Technology Co Ltd
Priority date: 2021-01-05
Filing date: 2021-01-05
Publication date: 2021-12-03

Abstract

The invention provides an article sorting method, an article sorting device and a storage medium; the article sorting method comprises the steps of acquiring an image of a target area, and processing the image to obtain at least one article image; acquiring target coding information of a target object and matching information corresponding to the target coding information; matching the at least one article image with the matching information respectively to obtain the matching degree of the at least one article image; and determining and sorting the target articles based on the target matching degree. When the articles are sorted, the collected images of the target area do not need to be subjected to complex calculation to determine the sorting sequence, only the target coding information of the target articles and at least one image of the target area need to be obtained, the target matching information is obtained based on the target coding information, the target articles can be sorted according to the matching degree of the target article images of the target area and the target matching information, and the efficiency of article sorting is remarkably improved.

Description

Article sorting method and device, and storage medium

Technical Field

The embodiment of the invention relates to the technical field of logistics and internet, in particular to a method and a device for sorting articles and a storage medium.

Background

In recent years, the intelligent logistics technology is combined with the Internet, and the automation, controllability and intelligence of logistics are realized through fine, dynamic and scientific management, so that the production efficiency is improved, and the operation cost is reduced. The intelligent storage belongs to an important link of intelligent logistics, and not only are articles complicated and different in shape in the existing storage system, but also the operation flow is complicated, and the storage, the movement, the sorting and the combination are realized. Common intelligent technologies such as robot stacking technology, deep learning-based robot picking technology, data mining technology, etc. all require sufficiently comprehensive support of commodity data.

At present, when a robot of a warehousing system sorts, articles are sorted and moved based on results obtained after data analysis. When the articles are sorted and moved, the data of the articles need to be collected firstly, and then the data are analyzed, so that the sorting sequence is determined.

Disclosure of Invention

The article sorting method, the article sorting device and the storage medium provided by the embodiment of the invention can improve the efficiency of article sorting.

The technical scheme of the invention is realized as follows:

the embodiment of the invention provides an article sorting method, which comprises the following steps:

acquiring an image of at least one item within a target area;

classifying the images to obtain at least one article image;

receiving a sorting instruction, wherein the sorting instruction carries target coding information;

responding to the sorting instruction, and acquiring target matching information corresponding to the target coding information from a matching information base;

matching the at least one article image with the target matching information respectively to obtain the matching degree of the at least one article image;

and sorting the target object corresponding to the target coding information based on the matching degree of the at least one object image.

In the above scheme, the sorting the target item corresponding to the target encoding information based on the matching degree of the at least one item includes:

determining a target matching degree corresponding to the target object image corresponding to the target coding information based on the matching degree of the at least one object;

based on the target matching degree, acquiring position information of the target object corresponding to the target matching degree in the image;

and sorting the target object from the position represented by the position information corresponding to the target matching degree.

In the above scheme, the image is a three-dimensional image, and the at least one article image is a three-dimensional image of at least one article.

In the above scheme, the image is a three-dimensional image; the classifying the image to obtain at least one article image includes:

extracting color information in the three-dimensional image and color position information corresponding to the color information;

arranging the color information according to the color position information to form a two-dimensional image of the target area;

and classifying the two-dimensional images to obtain a two-dimensional image of at least one article.

In the foregoing solution, the target matching information includes: the three-dimensional matching image and the size information corresponding to the target coding information;

the matching the at least one article image with the target matching information respectively to obtain the matching degree of the at least one article comprises:

respectively adjusting the three-dimensional matching image and the three-dimensional image of the at least one article to the same size based on the size information to obtain a three-dimensional matching equal-proportion image corresponding to the three-dimensional matching image and at least one three-dimensional equal-proportion image corresponding to the three-dimensional image of the at least one article;

and matching the at least one three-dimensional equal-proportion image with the three-dimensional matching equal-proportion image to obtain the matching degree of the at least one article.

In the foregoing solution, the target matching information includes: the two-dimensional matching image and the size information corresponding to the target coding information;

the base matches the at least one article image with the target matching information respectively to obtain the matching degree of the at least one article, and the method comprises the following steps:

respectively adjusting the two-dimensional matching image and the two-dimensional image of the at least one article to the same size based on the size information to obtain a two-dimensional matching equal-proportion image corresponding to the two-dimensional matching image and at least one two-dimensional equal-proportion image corresponding to the two-dimensional image of the at least one article;

and matching the at least one two-dimensional equal-proportion image with the two-dimensional matching equal-proportion image to obtain the matching degree of the at least one article.

In the above solution, the determining a target matching degree corresponding to the target object image corresponding to the target encoding information based on the matching degree of the at least one object includes:

if the matching degree of the at least one article is larger than the threshold value of the matching degree, determining the matching degree of the target article image corresponding to the target coding information in the at least one article as the target matching degree.

In the above scheme, after the acquiring the image of at least one article in the target area, in response to the sorting instruction, before the obtaining target matching information corresponding to the target encoding information in the matching information base, the method further includes:

scanning the code plate corresponding to the at least one article in the target area to obtain the code information corresponding to the at least one article;

correspondingly, the obtaining, in response to the sorting instruction, target matching information corresponding to the target encoding information from the target matching information base includes:

and if the target coding information belongs to the coding information, responding to the sorting instruction, and acquiring target matching information corresponding to the target coding information from a matching information base.

In the above scheme, after the scanning of the code card corresponding to the at least one article in the target area to obtain the code information corresponding to the at least one article, the method further includes:

and if the target coded information does not belong to the coded information, responding to the sorting instruction and stopping the sorting.

The embodiment of the invention also provides an article sorting method, which comprises the following steps:

acquiring shape information of an article through an image acquisition device;

acquiring coding information corresponding to the article through a code scanning gun based on the shape information, and acquiring description information of the article at a plurality of visual angles through an image acquisition device;

and sending the description information and the coding information to an industrial personal computer.

In the above scheme, the acquiring, by a barcode scanner, the coded information corresponding to the article based on the shape information, and acquiring, by an image acquisition device, the description information of the article at a plurality of viewing angles includes:

collecting, by the image capture device, descriptive information of the item at a plurality of viewing angles in a horizontal direction when the shape information characterizes the item as a first shape, or collecting, by the image capture device, descriptive information of the item above the item when the shape information characterizes the item as a second shape;

and collecting the coded information corresponding to the article through a code scanning gun.

acquiring description information and coding information sent by acquisition equipment; the description information includes: three-dimensional images and dimensional information for multiple perspectives of an article;

forming matching information corresponding to the article based on the three-dimensional images and the size information of the article from multiple visual angles;

and binding the matching information and the coding information and storing the binding information and the coding information in a matching information base.

In the foregoing solution, the forming matching information corresponding to the article based on the three-dimensional image and the size information of the article from multiple viewing angles includes:

fusing the three-dimensional images of the article at multiple visual angles to obtain a three-dimensional matching image corresponding to the article;

forming the matching information based on the three-dimensional matching image and the size information.

extracting color information in three-dimensional images of a plurality of visual angles of the article and color position information corresponding to the color information;

arranging the color information according to the color position information to form two-dimensional matching images of the article at multiple visual angles;

forming the matching information based on the two-dimensional matching image and the size information.

An embodiment of the present invention further provides a collection device, including:

the system comprises a first information acquisition device, an image acquisition device and a first sending device, wherein the first information acquisition device is connected with the image acquisition device, and the image acquisition device is connected with the first sending device;

the first information acquisition device is used for acquiring shape information of an article and sending the shape information to the image acquisition device;

the image acquisition device is used for acquiring the coded information corresponding to the article based on the shape information, acquiring the description information of the article at a plurality of visual angles, and sending the description information and the coded information to the first sending device;

and the first sending device is used for sending the description information and the coding information to an industrial personal computer.

The embodiment of the invention also provides an industrial personal computer, which comprises:

the system comprises a second information receiving device, an information processing device and a second transmitting device, wherein the second information receiving device is connected with the information processing device, and the information processing device is connected with the second transmitting device;

the second information receiving device is used for acquiring the description information and the coding information sent by the acquisition equipment and sending the coding information and the description information to the information processing device; the description information includes: three-dimensional images and dimensional information for multiple perspectives of an article;

the information processing device is used for forming matching information corresponding to the article based on the three-dimensional images and the size information of the multiple visual angles of the article, and sending the matching information and the coding information to a second sending device;

and the second sending device is used for binding the matching information and the coding information and storing the binding information and the coding information in a matching information base.

An embodiment of the present invention provides a sorting apparatus, including:

the sorting device comprises a sorting acquisition device, a sorting processing device, a sorting receiving device and a mechanical arm; the sorting acquisition device and the sorting receiving device are connected with the sorting processing device, and the sorting processing device is connected with the mechanical arm;

the sorting acquisition device is used for acquiring an image of at least one article in a target area and sending the image and the coded information to the sorting processing device;

the sorting processing device is used for classifying the images to obtain at least one article image;

the sorting receiving device is used for receiving a sorting instruction, and the sorting instruction carries target coding information;

the sorting receiving device is further used for responding to the sorting instruction, acquiring target matching information corresponding to the target coding information from a matching information base, and sending the target matching information to the sorting processing device;

the sorting processing device is used for respectively matching the at least one article image with the target matching information to obtain the matching degree of the at least one article image, and calling the mechanical arm to sort the target article corresponding to the target coding information based on the matching degree of the at least one article image.

An embodiment of the present invention provides a sorting device, including a first memory and a first processor, the first memory storing a computer program operable on the first processor, the first processor implementing the steps of the method as described above when executing the program.

An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, wherein the computer program is implemented to implement the steps in the method as described above when being executed by a processor.

An embodiment of the present invention provides an acquisition device, including a second memory and a second processor, where the second memory stores a computer program operable on the second processor, and the second processor implements the steps in the method when executing the program.

The embodiment of the invention provides an industrial personal computer which comprises a third memory and a third processor, wherein the third memory stores a computer program capable of running on the third processor, and the third processor implements the steps of the method when executing the program.

In the embodiment of the invention, at least one article image is obtained by collecting the image of the target area and processing the image; acquiring target coding information of a target object and matching information corresponding to the target coding information; matching the at least one article image with the matching information respectively to obtain the matching degree of the at least one article image; and determining and sorting the target articles based on the target matching degree. When the articles are sorted, the collected images of the target area do not need to be subjected to complex calculation to determine the sorting sequence, only the target coding information of the target articles and at least one image of the target area need to be obtained, the target matching information is obtained based on the target coding information, the target articles can be sorted according to the matching degree of the target article images of the target area and the target matching information, and the efficiency of article sorting is remarkably improved.

Drawings

Fig. 1 is a schematic view of an alternative structure of an article sorting apparatus according to an embodiment of the present invention;

fig. 2 is a schematic flow chart of an alternative method for sorting articles according to an embodiment of the present invention;

fig. 3 is a schematic diagram illustrating an alternative effect of the article sorting method according to the embodiment of the present invention;

fig. 4 is a schematic diagram illustrating an alternative effect of the article sorting method according to the embodiment of the present invention;

fig. 5A is a schematic diagram illustrating an alternative effect of the article sorting method according to the embodiment of the present invention;

fig. 5B is a schematic diagram illustrating an alternative effect of the article sorting method according to the embodiment of the present invention;

fig. 5C is a schematic diagram illustrating an alternative effect of the article sorting method according to the embodiment of the present invention;

fig. 5D is a schematic diagram illustrating an alternative effect of the article sorting method according to the embodiment of the present invention;

fig. 5E is a schematic diagram illustrating an alternative effect of the article sorting method according to the embodiment of the present invention;

fig. 5F is a schematic diagram illustrating an alternative effect of the article sorting method according to the embodiment of the present invention;

fig. 6 is a schematic flow chart of an alternative method for sorting articles according to an embodiment of the present invention;

fig. 7 is a schematic flow chart of an alternative method for sorting articles according to an embodiment of the present invention;

fig. 8 is a schematic flow chart of an alternative method for sorting articles according to an embodiment of the present invention;

fig. 9 is a schematic flow chart of an alternative method for sorting articles according to an embodiment of the present invention;

fig. 10 is a schematic flow chart of an alternative method for sorting articles according to an embodiment of the present invention;

fig. 11 is a schematic flow chart of an alternative method for sorting articles according to an embodiment of the present invention;

fig. 12 is a schematic flow chart of an alternative method for sorting articles according to an embodiment of the present invention;

fig. 13 is a schematic flow chart of an alternative method for sorting articles according to an embodiment of the present invention;

fig. 14 is a schematic flow chart of an alternative method for sorting articles according to an embodiment of the present invention;

fig. 15 is a schematic flow chart of an alternative method for sorting articles according to an embodiment of the present invention;

fig. 16 is an interaction diagram of a method for sorting articles according to an embodiment of the present invention;

fig. 17 is a schematic diagram of a hardware entity of a sorting apparatus according to an embodiment of the present invention;

fig. 18 is a schematic hardware entity diagram of an acquisition device according to an embodiment of the present invention;

fig. 19 is a schematic diagram of a hardware entity of an industrial personal computer according to an embodiment of the present invention;

fig. 20 is an alternative structural schematic diagram of the collecting apparatus according to the embodiment of the present invention;

fig. 21 is an alternative schematic structural diagram of an acquisition device according to an embodiment of the present invention;

fig. 22 is an alternative schematic structural diagram of an industrial personal computer provided in an embodiment of the present invention;

fig. 23 is an alternative structural schematic diagram of the sorting apparatus provided in the embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention are further described in detail with reference to the drawings and the embodiments, the described embodiments should not be construed as limiting the present invention, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

To the extent that similar descriptions of "first/second" appear in this patent document, the description below will be added, where reference is made to the term "first \ second \ third" merely to distinguish between similar objects and not to imply a particular ordering with respect to the objects, it being understood that "first \ second \ third" may be interchanged either in a particular order or in a sequential order as permitted, to enable embodiments of the invention described herein to be practiced in other than the order illustrated or described herein.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to be limiting of the invention.

Fig. 1 is a schematic diagram of an alternative structure of an article sorting apparatus according to an embodiment of the disclosure.

The embodiment of the invention provides an article sorting device, which comprises: collection equipment 1, industrial computer 2 and letter sorting equipment 3.

The system comprises an acquisition device 1, an industrial personal computer 2 and an industrial personal computer 2, wherein the acquisition device 1 is used for acquiring description information and coding information of an article, and the acquisition device 1 is also used for sending the description information and the coding information to the industrial personal computer 2; the industrial personal computer 2 is used for processing based on the description information to obtain matching information; the industrial personal computer 2 is also used for storing the matching information and the coding information in a matching information base; the sorting equipment 3 is used for collecting an image of at least one article when the article is sorted, acquiring target matching information in a matching information base based on the acquired target coding information, matching the image of the at least one article with the target matching information to obtain a matching degree, and sorting the target article based on the matching degree.

In the embodiment of the invention, when the articles enter the warehousing system, a coding plate comprising coding information is attached to the articles. The acquisition device 1 acquires description information and coding information of an article. Wherein, the acquisition device 1 may be at least one of a 2D, 3D camera and a code scanning gun. The acquisition apparatus 1 acquires at least one of size information, shape information, color information, and point cloud information of an article as description information. The acquisition equipment 1 scans the two-dimensional code on the code plate to obtain the code information of the article. The acquisition equipment 1 sends the coding information and the description information to the industrial personal computer 2. In the embodiment of the invention, the acquisition equipment 1 can store the description information and the coding information in a folder named by the coding information, and the acquisition equipment 1 sends the folder to the industrial personal computer 2.

In the embodiment of the invention, the acquisition equipment 1 is in communication connection with the industrial personal computer 2. Wherein, collection equipment 1 and industrial computer 2 can be connected through DB-9 needle serial ports communication line. The collection device 1 may also be connected to the industrial personal computer 2 via a Controller Area Network (CAN). The collecting apparatus 1 can collect color information of six faces of an article and point cloud information of the six faces of the article. The collecting equipment 1 sends the color information of the six surfaces of the article and the point cloud information of the six surfaces of the article to the industrial personal computer 2.

In the embodiment of the present invention, the industrial personal computer 2 may include: a circuit board or an integrated processor or a Micro Control Unit (MCU) with corresponding processing functions. The industrial personal computer 2 can calculate the color information of six surfaces of the article and the point cloud information of the six surfaces of the article through a 3D point cloud segmentation algorithm to obtain the matching information of the article. Wherein the matching information may include: a two-dimensional image of six faces of an article or a three-dimensional image of an article.

In the embodiment of the invention, the industrial personal computer 2 and the sorting equipment 3 establish TCP (Transmission Control protocol) connection. The industrial personal computer 2 can send the matching information and the corresponding coding information to the storage device to form a matching information base. The storage device may be a memory inside the industrial personal computer.

In the embodiment of the invention, the sorting equipment 3 can be connected with the industrial personal computer 2 through a controller local area network. The sorting apparatus 3 may be a robot arm. The sorting equipment 3 is provided with a code scanning device and a camera. The sorting device 3 captures an image of at least one article of the target area by means of a camera arranged on the sorting device 3. Target matching information is obtained in a matching information base based on the obtained target coding information, the image of at least one article is matched with the matching information to obtain matching degree, and the sorting sequence of the target article is determined based on the matching degree, so that the target article is sorted. The images of at least one article in the target area do not need to be subjected to complex calculation to determine the sequence, so that the efficiency of article sorting is remarkably improved.

The acquisition device 1 provided by the embodiment of the present invention may include: bearing device 121, bearing device 121 are used for bearing the weight of the article to change the information acquisition visual angle of article, make things convenient for the collection equipment to gather the description information of article under the condition of different visual angles.

In this embodiment of the present invention, the carrying device 121 may include: a drive device 126 and a bearing surface 125. The driving device 126 may be a servo motor. The support surface 125 may be a flat glass support surface. The driving device drives the bearing surface to rotate.

In one embodiment of the present invention, a coded tag containing coded information is placed on the carrying surface 125 as the item enters the warehousing system. First, the description information and the coding information of one view of the article are collected by the collecting apparatus 1. Then, the driving device 126 drives the bearing surface 125 to rotate clockwise or counterclockwise by 60 °, and the collecting device 1 collects the description information of another view angle of the article again until the article returns to the original state. The acquisition equipment 1 scans the two-dimensional code on the code plate to obtain the code information of the article. The acquisition device 1 sends the encoded information and the acquired description information of the plurality of perspectives to the industrial personal computer 2. The description information of one view may include: color information and point cloud information for an item at one perspective.

In the embodiment of the present invention, the acquisition device 1 may include a first acquisition device and a second acquisition device. The first acquisition device may be a 2D or 3D camera and the second acquisition device may be a code scanning gun 124. The first acquisition equipment can be connected with the industrial personal computer 2 through a controller local area network, and the second acquisition equipment can be connected with the industrial personal computer 2 through a DB-9 pin serial port. The first collecting device may be disposed in a horizontal direction or above the carrying surface 125 of the carrying device 121. A second collection device may be positioned above the bearing surface 125.

In the embodiment of the present invention, if the shape of the article is square, the article sorting device 3 calls the first collecting device in the horizontal direction of the carrying surface 125 to collect the description information of the article from different viewing angles. If the shape of the article is irregular (any one of circular, elliptical, triangular, cylindrical and spherical), a first acquisition device above the bearing surface 125 is invoked to acquire the description information of the article from different perspectives. When the article is in a special shape, the first acquisition equipment above the bearing surface acquires the description information of the article once, then the article is turned over by 180 degrees, and the first acquisition equipment above the bearing surface acquires the description information of the article once again.

In the embodiment of the present invention, when the square articles enter the warehousing system, a code plate including coded information is attached to the square articles and placed on the carrying surface 125 of the carrying device 121. The description information of a visual angle of the article is collected by the first collecting device in the horizontal direction of the bearing surface 125, and the coding information of the article is collected by the second collecting device. Then, the driving device 121 drives the bearing surface 125 to rotate clockwise or counterclockwise by 60 °, and the first collecting device collects the description information of another view angle of the article again until the article returns to the original state. The article is turned over, so that the original side face of the article is vertically upward, the first acquisition equipment acquires the description information of one visual angle of the article again, then the driving device 126 drives the bearing surface 125 to rotate clockwise or anticlockwise for 60 degrees, and the first acquisition equipment acquires the description information of the other visual angle of the article again until the article returns to the turned state. The first acquisition equipment sends the acquired description information of the multiple visual angles of the article and the acquired coding information of the second acquisition equipment to the industrial personal computer 2.

In one embodiment of the present invention, a code plate containing coded information is attached to the spherical articles and placed on the bearing surface 125 as they enter the warehousing system. The above description information of the article is first collected by a first collecting device above the carrying surface 125 and the coding information of the article is collected by a second collecting device. The article is then flipped 180 deg., and the first acquisition device acquires the following description of the article again. The first acquisition equipment sends the acquired description information of the upper and lower surfaces and the encoding information of the articles acquired by the second acquisition equipment to the industrial personal computer 2.

In an embodiment of the present invention, the first collecting device includes: a first camera 122 and a second camera 123; the first camera 122 and the second camera 123 are simultaneously connected with the industrial personal computer 2; the first camera 122 is disposed in the horizontal direction of the bearing surface 125, and the second camera 123 is disposed above the bearing surface 125; the first camera 122 is used for acquiring description information of the article when the article is in the first shape and sending the description information to the industrial personal computer 2; and the second camera 123 is configured to collect description information of the article when the article is in the second shape, and is further configured to send the description information to the industrial personal computer 2.

In the embodiment of the present invention, the side surface of the bearing surface 125 is provided with the light source 127, and the light source 127 is used for polishing an article, so that the collecting device 1 can conveniently collect more accurate description information.

In the embodiment of the present invention, the first shape is a square, and when the square article enters the warehousing system, a code plate including code information is attached to the square article and placed on the carrying surface 125 of the carrying device 121. The description information of a visual angle of the article is collected through a first camera, and the coding information of the article is collected through a second collection device. Then, the driving device 121 drives the carrying surface 125 to rotate clockwise or counterclockwise by n ° (n is a positive integer), and the first camera 122 collects the description information of another viewing angle of the article again until the article returns to the original state. The article is turned over, so that the original side face of the article is vertically upward, the first camera 122 collects the description information of one visual angle of the article again, then the moving device drives the bearing face to rotate clockwise or anticlockwise by n degrees, and the first collecting device collects the description information of the other visual angle of the article again until the article returns to the turned state. The first camera 122 sends the acquired description information of the multiple visual angles of the article and the encoded information acquired by the second acquisition device to the industrial personal computer 2.

In the embodiment of the present invention, when the spherical articles enter the warehousing system, a code plate including coded information is attached to the spherical articles and placed on the carrying surface 125 of the carrying device 121. The above description information of the article is first collected by the second camera 123, and the code information of the article is collected by the second collecting device. The item is then flipped 180 deg., and the second camera 123 again captures the following description of the item. The second camera sends the collected upper and lower description information and the coded information of the article collected by the second collecting device to the industrial personal computer 2.

In the embodiment of the present invention, the description information includes: point cloud information of the article, color information of the article, and size information of the article. The descriptive information may also include a three-dimensional image of the article.

In the embodiment of the invention, if the article is a mobile phone. When the mobile phone enters the warehousing system, a code plate including the code information is attached to the mobile phone and placed on the bearing surface 125 of the bearing device 121, and the code plate is kept upward. Firstly, the color information and point cloud information of the first side face of the mobile phone are collected through the first camera 122, and the coding information of the mobile phone is collected through the second collection device. The driving device 121 drives the bearing surface 125 to rotate 90 degrees clockwise, the first camera 122 collects color information and point cloud information of the second side surface of the mobile phone again, the driving device 121 drives the bearing surface 125 to rotate 90 degrees clockwise again, the first camera 122 collects color information and point cloud information of the third side surface of the mobile phone again, the driving device 121 drives the bearing surface 125 to rotate 90 degrees clockwise again, and the first camera 122 collects color information and point cloud information of the fourth side surface of the mobile phone again. Then, the mobile phone is turned over, so that the first side face of the mobile phone is kept in an upward state, the first camera 122 collects color information and point cloud information of the fifth side face of the mobile phone again, the driving device 121 drives the bearing surface 125 to rotate 90 degrees clockwise, the first camera 122 collects color information and point cloud information of the sixth side face of the mobile phone again, the driving device 121 drives the bearing surface 125 to rotate 90 degrees clockwise again, the first camera 122 collects color information and point cloud information of the seventh side face of the mobile phone again, the driving device 121 drives the bearing surface 125 to rotate 90 degrees clockwise again, and the first camera 122 collects color information and point cloud information of the eighth side face of the mobile phone again. The first camera 122 packages the collected color information and point cloud information of the eight sides of the article and the collected coding information collected by the second collecting device and sends the color information and point cloud information to the industrial personal computer 2.

With reference to fig. 5A to 5F, the industrial personal computer 2 calculates color information and point cloud information of eight sides of the mobile phone through a 3D point cloud segmentation algorithm, and obtains two-dimensional matching images of six corresponding surfaces of the mobile phone. The industrial personal computer 2 sends the two-dimensional matching images of the six surfaces of the mobile phone, the size information of the articles and the coding information to the matching information base.

When the sorting equipment sorts the mobile phone, the image of at least one article in a target area where the mobile phone is located is collected, matching information is obtained in a matching information base based on the obtained target coding information, the image of the at least one article is matched with the matching information to obtain matching degree, the target matching degree is determined based on the matching degree, therefore, the sorting order of the target articles is determined according to the size sequence of the target matching degree, and the target articles are sorted. The images of at least one article in the target area do not need to be subjected to complex calculation to determine the sequence, so that the efficiency of article sorting is remarkably improved.

FIG. 2 is a schematic flow chart of an alternative method for sorting articles according to the disclosed embodiment; the description will be made in conjunction with the steps shown in fig. 2.

S01, an image of at least one item within the target area is acquired.

In the embodiment of the invention, when the target articles need to be sorted, the sorting equipment collects the target areas where the target articles are located. The sorting apparatus acquires an image of at least one item of the target area by a camera.

In the embodiment of the invention, the target area can be a box body where the target object is located; the target area may be a shelf where the target item is located; the target area may also be an item stack where the target item is located.

In the embodiment of the invention, the image of the target area may be a three-dimensional image of the target area acquired by the sorting equipment through the 2D camera and the 3D camera. Wherein. The three-dimensional image includes: and point cloud information of the target area and color information corresponding to the point cloud information. The image of the target area may also be a two-dimensional image of the target area captured by the sorting device via a 2D camera. Wherein the two-dimensional image includes: color information of the target area.

In the embodiment of the invention, a plurality of categories of articles can be placed in the target area, wherein the target articles are included. The sorting equipment can acquire the coding information corresponding to the articles of multiple categories in the target area respectively by scanning the coding board.

Referring to fig. 3, an image of at least one item of a target area is shown. Wherein, the target area has two objects, and the mobile phone is the target object.

And S02, classifying the images to obtain at least one article image.

In the embodiment of the invention, the sorting equipment can label and extract the object images in the images through a target detection model, such as a real-time fast target detection model (YOLO) or a Single-machine multi-card target detection algorithm (SSD), so as to obtain at least one object image.

In the embodiment of the invention, the sorting equipment can extract the article image in the image through the segmentation model to obtain at least one article image.

In the embodiment of the invention, the sorting equipment can extract the three-dimensional image of the article from the three-dimensional image through the target detection model. Wherein the three-dimensional image of the article comprises: and point cloud information of the article and color information corresponding to the point cloud information.

And S03, receiving a sorting instruction, wherein the sorting instruction carries target coding information.

In the embodiment of the invention, the sorting equipment receives the sorting instruction. Wherein the sorting instructions may include target encoded information for a target item to be sorted.

In an embodiment of the present invention, the sorting apparatus may include an input device, and the input device may be a keyboard or a touch screen display. The target code information may be manually input through an input device.

In the embodiment of the invention, the sorting equipment can receive a sorting instruction sent by a remote control center, and the sorting instruction comprises the following steps: target coded information of the target item.

And S04, responding to the sorting instruction, and acquiring the matching information corresponding to the target coding information from the matching information base.

In the embodiment of the invention, the sorting equipment receives the sorting instruction. Wherein the sorting instructions may include target encoded information for the target item. The sorting equipment responds to the sorting instruction according to a preset program and obtains matching information corresponding to the target coding information from the matching information base. Wherein, the matching information base can be stored in the industrial personal computer. The matching information base may also be stored in the sorting device.

In the embodiment of the present invention, the storage device may be an external storage connected to the sorting apparatus, or may be a storage inside the sorting apparatus.

In the embodiment of the present invention, the matching information may include: at least one of size information of the target item, a three-dimensional matching image of the target item, and a two-dimensional matching image of the target item.

With reference to fig. 4, the three-dimensional matching image of the target object mobile phone is shown, and the three-dimensional matching image includes: point cloud information and color information of the target item. Wherein the target object is a mobile phone. And 5A-5F, two-dimensional matching images of the mobile phone are shown. The two-dimensional matching image of the six surfaces of the mobile phone is included. The two-dimensional matching image includes: color information of six faces of the mobile phone.

And S05, matching the at least one article image with the matching information respectively to obtain the matching degree of the at least one article image.

In the embodiment of the invention, the sorting equipment matches the image of at least one article with the three-dimensional matching image or the two-dimensional matching image in the matching information, and the matching degree of the image of at least one article and the three-dimensional matching image or the two-dimensional matching image is determined.

In the embodiment of the invention, the more similar the image of the at least one article is to the three-dimensional matching image or the two-dimensional matching image in the matching information, the greater the matching degree corresponding to the image of the at least one article is.

In an embodiment of the invention, the sorting device may resize the image of the at least one item to a size that matches the size in the information. The sorting apparatus may resize the three-dimensional matching image or the two-dimensional matching image to the size of the size in the matching information. And the sorting device matches the adjusted image of the at least one article with the integrated three-dimensional matching image or two-dimensional matching image, and determines the matching degree of the image of the at least one article with the three-dimensional matching image or the two-dimensional matching image.

And S06, sorting the target object corresponding to the target code information based on the matching degree of at least one object image.

In the embodiment of the invention, if the matching degree of at least one article image is greater than the threshold value of the matching degree, the at least one article image is the target article image. The matching degree is a target matching degree. In the embodiment of the invention, the sorting equipment clamps the target articles according to the size sequence of the target matching degree through the mechanical arm, and sorts the target articles.

In the embodiment of the invention, because the target matching degrees are different in size sequence, the sorting equipment can sort the target matching degrees according to the size sequence.

In the embodiment of the invention, the sorting equipment clamps the target articles according to the size sequence of the target matching degree through the mechanical arm, and sorts the target articles.

In the embodiment of the invention, the sorting equipment can acquire the position information of the target article in the image of at least one article in the target area, the central point or the optimal clamping point of the target article is calculated according to the position information of the target article and the shape of the target article, and the sorting equipment clamps the central point of the target article through the mechanical arm according to the size sequence of the target matching degree to sort the target article.

In the embodiment of the invention, the sorting equipment sorts the target articles to the designated positions or the designated conveying belts through the mechanical arm.

In the embodiment of the invention, at least one article image is obtained by collecting the image of the target area and processing the image; acquiring target coding information of a target object and matching information corresponding to the target coding information; matching the at least one article image with the matching information respectively to obtain the matching degree of the at least one article image; and determining and sorting the target articles based on the target matching degree. When the articles are sorted, the collected images of the target area do not need to be subjected to complex calculation to determine the sorting sequence, only the target coding information of the target articles and at least one image of the target area need to be obtained, the matching information is obtained based on the target coding information, the target articles can be sorted according to the matching degree of the target article images of the target area and the matching information, and the efficiency of article sorting is obviously improved.

In some embodiments, referring to fig. 6, fig. 6 is an optional flowchart of the article sorting method according to the embodiment of the disclosure, and S06 shown in fig. 1 may also be implemented through S07-S09, which will be described in conjunction with the steps.

And S07, determining the target matching degree corresponding to the target article image corresponding to the target code information based on the matching degree of at least one article.

In the embodiment of the present invention, if the matching degree of at least one article is greater than the threshold matching degree, it is determined that the matching degree of the at least one article is the target matching degree, and the at least one article image is the target article image.

In the embodiment of the invention, the sorting equipment matches the at least one article image with the three-dimensional matching image to obtain the three-dimensional matching degree of the at least one article. And if the three-dimensional matching degree of the at least one article is greater than the threshold value of the three-dimensional matching degree, determining that the three-dimensional matching degree of the at least one article is the target matching degree, and determining that the at least one article image is the target article image.

In the embodiment of the invention, the sorting equipment matches the at least one article image with the two-dimensional matching image to obtain the two-dimensional matching degree of the at least one article. And if the two-dimensional matching degree of at least one article is greater than the two-dimensional matching degree threshold value, determining that the two-dimensional matching degree of the at least one article is the target matching degree, and determining that the at least one article image is the target article image.

And S08, acquiring the position information of the target object corresponding to the target matching degree in the image based on the target matching degree.

In an embodiment of the present invention, the image of at least one article in the target area acquired by the sorting device includes: position information and color information. And the sorting equipment acquires the position information and the shape information of the target object in the image of at least one object in the target area according to the target object corresponding to the target matching degree.

And S09, sorting the target articles from the positions represented by the position information corresponding to the target matching degree.

In the embodiment of the invention, the sorting equipment calculates the central point of the target object according to the shape information of the target object, and clamps the target object at the position of the target object through the mechanical arm for sorting. And the clamping can be carried out according to the size sequence of the target matching degree during clamping.

In the embodiment of the invention, the sorting equipment can sort the target articles to the designated positions or the designated conveyor belts.

In some embodiments, referring to fig. 7, fig. 7 is an optional flowchart of the article sorting method according to the embodiment of the disclosure, and S04-S07 shown in fig. 6 can also be implemented through S10-S13, which will be described with reference to the steps.

S10, responding to the sorting instruction, obtaining matching information corresponding to the target coding information from the matching information base, wherein the matching information comprises: and the three-dimensional matching image and the size information corresponding to the target coding information.

In the embodiment of the invention, the sorting equipment receives the sorting instruction. Wherein the sorting instructions may include target encoded information for the target item. And the sorting equipment acquires the matching information corresponding to the target code information from the matching information base according to a preset program. Wherein the matching information includes: and the three-dimensional matching image and the size information of the target object corresponding to the target coding information.

With reference to fig. 4, a three-dimensional matching image of the target object corresponding to the target encoding information is obtained, where the three-dimensional matching image includes: point cloud information and color information of the target item.

And S11, respectively adjusting the three-dimensional matching image and the three-dimensional image of the at least one article to the same size based on the size information to obtain a three-dimensional matching equal-proportion image corresponding to the three-dimensional matching image and at least one three-dimensional equal-proportion image corresponding to the three-dimensional image of the at least one article.

In the embodiment of the invention, the sorting equipment classifies the image of at least one article in the target area to obtain the three-dimensional image of at least one article. And the sorting equipment adjusts the three-dimensional matching image to the size corresponding to the size information to obtain a three-dimensional matching equal-proportion image. And the sorting equipment adjusts the three-dimensional image of at least one article to the size corresponding to the size information to obtain at least one three-dimensional equal-proportion image.

In the embodiment of the invention, the sorting equipment adjusts the three-dimensional matching image to the size of a certain proportion corresponding to the size information to obtain the three-dimensional matching equal-proportion image. The sorting equipment adjusts the three-dimensional image of at least one article to the size of a certain proportion corresponding to the size information to obtain at least one three-dimensional equal-proportion image.

And S12, matching the at least one three-dimensional equal-proportion image with the three-dimensional matching equal-proportion image respectively to obtain the matching degree of the at least one article.

In the embodiment of the invention, the sorting equipment adjusts the three-dimensional matching equal-proportion image and at least one three-dimensional equal-proportion image to the same size, so that the sorting equipment can conveniently match. The sorting equipment matches the three-dimensional equal-proportion images with the three-dimensional matching equal-proportion images to obtain the matching degree of at least one three-dimensional equal-proportion image and the three-dimensional matching equal-proportion image.

In the embodiment of the invention, the sorting equipment can match at least one three-dimensional equal-proportion image with a certain part of the three-dimensional matching equal-proportion image to obtain the matching degree of at least one article.

And S13, if the matching degree of at least one article is greater than the threshold value of the matching degree, determining the matching degree of the target article image corresponding to the target coding information in the at least one article as the target matching degree.

In the embodiment of the invention, when the matching degree of at least one article is greater than the threshold value of the matching degree, the sorting equipment determines that the image of the article corresponding to the matching degree of at least one article is the target article image, and the matching degree of the at least one article is the target matching degree.

In the embodiment of the invention, the matching degree is a numerical value reflecting the similarity between the three-dimensional equal-proportion image and the three-dimensional matching equal-proportion image. The threshold value of the degree of matching is a fixed value. The matching degree may range from 0 to 100 and the threshold matching degree may be 50.

In some embodiments, referring to fig. 8, fig. 8 is an optional flowchart of the article sorting method according to the embodiment of the disclosure, and S02 shown in fig. 7 can also be implemented through S14-S16, which will be described in conjunction with the steps.

S14, color information in the three-dimensional image and color position information corresponding to the color information are extracted.

In the embodiment of the invention, the image of at least one article in the target area acquired by the sorting equipment is a three-dimensional image. And the sorting equipment acquires the color information of each pixel point and the position information corresponding to the color information in the three-dimensional image. Each point cloud information in the three-dimensional image corresponds to one piece of position information and color information.

S15, color information is arranged according to the color position information to form a two-dimensional image of the target region.

In the embodiment of the invention, the sorting equipment arranges the color information according to the position information corresponding to the color information to form the two-dimensional image of the target area.

And S16, classifying the two-dimensional images to obtain a two-dimensional image of at least one article.

In the embodiment of the invention, the sorting equipment can label and extract the article images in the two-dimensional images through a target detection model, such as a real-time fast target detection model (YOLO) or a Single-machine multi-card target detection algorithm (SSD), so as to obtain the two-dimensional image of at least one article.

In some embodiments, referring to fig. 8, fig. 8 is an optional flowchart of the article sorting method according to the embodiment of the disclosure, and S10-S12 shown in fig. 7 can also be implemented through S17-S19, which will be described with reference to the steps.

S17, responding to the sorting instruction, obtaining matching information corresponding to the target coding information from the matching information base, wherein the matching information comprises: and the two-dimensional matching image and the size information corresponding to the target coding information.

In the embodiment of the invention, the sorting equipment receives the sorting instruction. Wherein the sorting instructions may include target encoded information for the target item. And the sorting equipment acquires the matching information corresponding to the target code information from the matching information base according to a preset program. Wherein the matching information includes: and the two-dimensional matching image and the size information of the target object corresponding to the target code information.

With reference to fig. 5A-5F, two-dimensional matching images of the target item are shown. Which includes two-dimensional matching images of six faces of the target article. The two-dimensional matching image includes: color information of six sides of the target article.

S18, based on the size information, the two-dimensional matching image and the two-dimensional image of the at least one article are respectively adjusted to the same size, and a two-dimensional matching equal-proportion image corresponding to the two-dimensional matching image and at least one two-dimensional equal-proportion image corresponding to the two-dimensional image of the at least one article are obtained.

In the embodiment of the invention, the sorting equipment adjusts the two-dimensional matching image to the size corresponding to the size information to obtain the two-dimensional matching equal-proportion image. And the sorting equipment adjusts the two-dimensional image of at least one article to the size corresponding to the size information to obtain at least one two-dimensional equal-proportion image.

In the embodiment of the invention, the sorting equipment adjusts the two-dimensional matching image to the size of a certain proportion corresponding to the size information to obtain the two-dimensional matching equal-proportion image. The sorting equipment adjusts the two-dimensional image of at least one article to the size of a certain proportion corresponding to the size information to obtain at least one two-dimensional equal-proportion image.

And S19, matching the at least one two-dimensional equal-proportion image with the two-dimensional matching equal-proportion image respectively to obtain the matching degree of the at least one article.

In the embodiment of the invention, the sorting equipment adjusts the two-dimensional matching equal-proportion image and the two-dimensional equal-proportion image of at least one article to be the same in size, so that the sorting equipment can conveniently match. The sorting equipment matches the two-dimensional equal-proportion images with the two-dimensional matching equal-proportion images to obtain the matching degree of at least one two-dimensional equal-proportion image and one two-dimensional matching equal-proportion image.

In some embodiments, referring to fig. 9, fig. 9 is an optional flowchart of the article sorting method according to the embodiment of the disclosure, and S04 shown in fig. 1 may also be implemented through S20-S21, which will be described in conjunction with the steps.

S20, scanning the code card corresponding to at least one article in the target area to obtain the code information corresponding to at least one article.

In an embodiment of the present invention, the sorting device may acquire a three-dimensional image of at least one article in the target area through the 2D camera and the 3D camera. The sorting equipment can scan the code cards in the target area through the code scanning gun. The coding board is provided with a bar code corresponding to the target area. The code scanning gun scans the bar code and can acquire the coded information of one corresponding to at least one article corresponding to the target area.

In the embodiment of the invention, a plurality of categories of articles are placed in the target area, wherein the target articles are included. The number of the target objects is not limited, and the target objects can be regularly placed or irregularly placed.

And S21, if the target coding information belongs to the coding information, responding to the sorting instruction, and acquiring target matching information corresponding to the target coding information from the matching information base.

In the embodiment of the invention, the sorting equipment compares the target coded information with the coded information corresponding to at least one article to determine that the target coded information exists in the coded information corresponding to at least one article. Thereby determining that the target object corresponding to the target code information exists in at least one object in the target area. The sort instruction is responded to.

In the embodiment of the present invention, the target encoding information may be: "0001". The coded information corresponding to the at least one article may be: "0001, 0101, 0003". The sorting equipment compares the target coded information with the coded information corresponding to the at least one article to determine that the target coded information exists in the coded information corresponding to the at least one article. Thereby determining that the target object corresponding to the target code information exists in at least one object in the target area.

In some embodiments, referring to fig. 10, fig. 10 is an optional flowchart of the article sorting method according to the embodiment of the disclosure, and S21-S06 shown in fig. 9 can also be implemented through S22, which will be described in conjunction with the steps.

And S22, if the target coded information does not belong to the coded information, responding to the sorting instruction and stopping the sorting.

In the embodiment of the invention, the sorting equipment compares the target coded information with the coded information corresponding to the at least one article, and determines that the target coded information does not exist in the coded information corresponding to the at least one article. Thereby determining that the target object corresponding to the target code information is not present in the at least one object in the target area. No sorting instruction is responded to.

FIG. 11 is a schematic flow chart diagram of an alternative method for sorting articles according to the disclosed embodiment; the description will be made in conjunction with the steps shown in fig. 11.

And S101, acquiring shape information of the article through image acquisition equipment.

In the embodiment of the invention, when the articles enter the warehousing system, the acquisition equipment can acquire the shape information of the articles through the camera. Wherein the shape information characterizes a shape feature of the article.

The shape information may include a first shape and a second shape. The first shape characterizes the article as a square or rectangle. The second shape characterizes the article as being non-square or non-rectangular, e.g. spherical, ellipsoidal.

And S102, acquiring coding information corresponding to the article through a code scanning gun based on the shape information, and acquiring description information of the article at a plurality of visual angles through an image acquisition device.

In an embodiment of the invention, when the articles enter the warehousing system, the article sorting device attaches a coding plate comprising coding information to the articles. Firstly, the description information and the coding information of a plurality of visual angles of the article are collected through an image collecting device of a collecting device. Wherein, the acquisition device can be at least one of a 2D camera, a 3D camera and a code scanning gun. The acquisition device acquires at least one of size information, shape information, color information, and point cloud information of the article as description information.

In the embodiment of the invention, the acquisition equipment can scan the two-dimensional code on the code board through the code scanning gun to obtain the code information of the article. The acquisition device may acquire the description information of the item through the camera.

In the embodiment of the invention, if the shape of the article is the first shape, the camera collects the description information of the side faces of the article at a plurality of visual angles in the horizontal direction of the article, after the description information of the side faces of the article is collected, the article is turned over to keep the state that one side face of the article faces upwards, and the camera collects the description information of the side faces of the article again. If the shape of the article is the second shape (one of a sphere, triangle, ellipse, and cylinder), the camera collects the descriptive information of the article above the article, and after collecting once, turns the article 180 ° to collect the descriptive information of the article once again.

And S103, sending the description information and the target coding information to an industrial personal computer.

In the embodiment of the invention, the acquisition equipment can store the description information and the coding information in a folder named by the coding information, and the acquisition equipment sends the folder to the industrial personal computer.

In the embodiment of the invention, the acquisition equipment is in communication connection with the industrial personal computer in advance. The acquisition equipment and the industrial personal computer can be connected through a controller local area network.

In the embodiment of the invention, the description information of the article is acquired at different visual angles of the article through the acquisition equipment, and the description information of the article and the corresponding coding information are sent to the industrial personal computer, so that more accurate matching information can be obtained when the processing device processes the description information.

In some embodiments, referring to fig. 12, fig. 12 is an optional flowchart of an article sorting method according to an embodiment of the disclosure, and S102 shown in fig. 11 can also be implemented through S104 to S105, which will be described with reference to the steps.

And S104, when the shape information represents that the article is in the first shape, acquiring description information of the article through the image acquisition device at a plurality of visual angles in the horizontal direction, or when the shape information represents that the article is in the second shape, acquiring the description information of the article through the image acquisition device above the article.

In the embodiment of the invention, the acquisition equipment acquires the shape information of the article. The method further includes collecting, by the camera, descriptive information of the article at a plurality of viewing angles in a horizontal direction when the shape information characterizes the article as a first shape, or collecting, by the camera, descriptive information of the article above the article when the shape information characterizes the article as a second shape.

In the embodiment of the invention, if the article is a rectangular mobile phone. When the mobile phone enters the warehousing system, color information and point cloud information of a first side face of the mobile phone are collected through a first camera. The driving device drives the bearing surface to rotate 90 degrees clockwise, the first camera collects color information and point cloud information of the second side face of the mobile phone again, the driving device drives the bearing surface to rotate 90 degrees clockwise again, the first camera collects color information and point cloud information of the third side face of the mobile phone again, the driving device drives the bearing surface to rotate 90 degrees clockwise again, and the first camera collects color information and point cloud information of the fourth side face of the mobile phone again. Then the mobile phone is turned over, the first side face of the mobile phone is kept in an upward state, the second camera collects color information and point cloud information of the fifth side face of the mobile phone again, the driving device drives the bearing face to rotate 90 degrees clockwise, the first camera collects color information and point cloud information of the sixth side face of the mobile phone again, the driving device drives the bearing face to rotate 90 degrees clockwise again, the first camera collects color information and point cloud information of the seventh side face of the mobile phone again, the driving device drives the bearing face to rotate 90 degrees clockwise again, and the first camera collects color information and point cloud information of the eighth side face of the mobile phone again.

In the embodiment of the invention, if the article is a spherical basketball, the second camera collects the description information of the upper surface of the basketball above the basketball, and after the description information is collected once, the basketball is turned by 180 degrees and the description information of the lower surface of the basketball is collected once again.

And S105, collecting the coding information corresponding to the article through a code scanning gun.

In the embodiment of the invention, when the mobile phone enters the warehousing system, the mobile phone is pasted with the coding board comprising the coding information, and the mobile phone is placed on the bearing surface of the bearing device of the acquisition equipment, so that the coding board is kept upward. The collecting equipment collects the coding information of the articles by scanning the coding board through the coding gun.

FIG. 13 is a schematic flow chart diagram of an alternative method for sorting articles according to the disclosed embodiment; the description will be made in conjunction with the steps shown in fig. 13.

S201, acquiring description information and coding information sent by acquisition equipment; the description information includes: three-dimensional images and dimensional information for multiple perspectives of an article.

In the embodiment of the invention, the industrial personal computer acquires the description information and the coding information sent by the acquisition equipment. The description information includes: three-dimensional images and dimensional information for multiple perspectives of an article.

In this embodiment of the present invention, the description information may further include: point cloud information of the article, weight information of the article, and shape information of the article.

In the embodiment of the invention, the industrial personal computer can receive the data packet sent by the acquisition equipment, wherein the data packet comprises the description information and the corresponding coding information.

And S202, forming matching information corresponding to the article based on the three-dimensional images and the size information of the article from multiple visual angles.

In the embodiment of the present invention, the industrial personal computer may include: a circuit board or an integrated processor or a Micro Control Unit (MCU) with corresponding processing functions. The industrial personal computer can calculate the three-dimensional images of the articles at multiple visual angles through a 3D point cloud segmentation algorithm and combine the three-dimensional images with the size information to obtain the matching information of the articles. Wherein the matching information may include: two-dimensional images of six surfaces of the article or three-dimensional images of the article, and size information of the article.

And S203, binding the matching information and the coding information and storing the binding information and the coding information in a matching information base.

In the embodiment of the invention, the industrial personal computer maps the matching information and the corresponding coding information and stores the mapping information in the matching information base.

In the embodiment of the invention, the industrial personal computer can encode information to establish a folder for naming, and store corresponding matching information in the folder and the matching information base.

In the embodiment of the invention, the industrial personal computer can store the matching information base in the database of the industrial personal computer. The industrial personal computer can also send the matching information base to an external storage device, and the storage device is connected with the sorting equipment.

In the embodiment of the invention, the industrial personal computer performs fusion and classification processing on the description information of the multiple visual angles of the article sent by the acquisition device, so that a three-dimensional matching image or a two-dimensional matching image corresponding to the article can be obtained. And the three-dimensional matching image or the two-dimensional matching image is fused with the size information to obtain matching information. When the sorting equipment sorts the target articles, the target articles are sent to the sorting equipment, so that the sorting device does not need to perform complex calculation on the article images, and the sorting efficiency is improved.

In some embodiments, referring to fig. 14, fig. 14 is an optional flowchart of an article sorting method according to an embodiment of the disclosure, and S202 shown in fig. 13 can also be implemented through S204-S205, which will be described with reference to the steps.

And S204, fusing the three-dimensional images of the article from multiple visual angles to obtain a three-dimensional matching image corresponding to the article.

In the embodiment of the invention, the industrial personal computer fuses the three-dimensional images of the articles at multiple visual angles through a 3D point cloud algorithm, so that the three-dimensional matching images corresponding to the articles can be obtained.

In the embodiment of the invention, the industrial personal computer fuses the color information and the point cloud information of the multiple visual angles of the article to obtain the three-dimensional matching image corresponding to the article.

And S205, forming matching information based on the three-dimensional matching image and the size information.

In the embodiment of the invention, the industrial personal computer maps the three-dimensional matching image of the article and the corresponding size information to form matching information.

In the embodiment of the invention, the industrial personal computer can establish a folder by using the coded information corresponding to the article, and stores the three-dimensional matching image and the size information of the article for drinking in the folder.

In some embodiments, referring to fig. 15, fig. 15 is an optional flowchart of an article sorting method according to an embodiment of the disclosure, and S202 shown in fig. 13 may also be implemented through S206-S208, which will be described with reference to the steps.

And S206, extracting color information in the three-dimensional images of the multiple visual angles of the article and color position information corresponding to the color information.

In the embodiment of the invention, the industrial personal computer extracts the color information of the multiple visual angles of the article and the color position information corresponding to the color information from the three-dimensional images of the multiple visual angles of the description information sent by the acquisition device.

Because the three-dimensional image comprises the point cloud information of the article and the corresponding color information, and the point cloud information represents the position information, the industrial personal computer can extract the color information and the color position information corresponding to the color information from the three-dimensional images of a plurality of visual angles.

And S207, arranging the color information according to the color position information to form a two-dimensional matching image of the article from multiple visual angles.

In the embodiment of the invention, the industrial personal computer arranges the color information according to the position represented by the corresponding position information to form two-dimensional matching images of multiple visual angles of the article.

And S208, forming matching information based on the two-dimensional matching image and the size information.

In the embodiment of the invention, the industrial personal computer stores the two-dimensional matching image and the size information in the folder named by the coding information to form the matching information.

Fig. 16 is an interaction diagram of a method for sorting articles according to an embodiment of the present invention; the description will be made in conjunction with the steps of fig. 16.

S301, the acquisition equipment acquires shape information of the article through the image acquisition device.

The specific implementation process is shown as S01 in fig. 11, please refer to S01.

And S302, collecting the coding information corresponding to the article by a code scanning gun based on the shape information by the collecting equipment, and collecting the description information of the article at a plurality of visual angles by the image collecting device.

The specific implementation process is shown as S02 in fig. 11, please refer to S02.

S303, the acquisition equipment sends the description information and the target coding information to an industrial personal computer, and the description information comprises: three-dimensional images and dimensional information for multiple perspectives of an article.

The specific implementation process is shown as S03 in fig. 11, please refer to S03.

And S304, the industrial personal computer forms matching information corresponding to the article based on the three-dimensional images and the size information of the article from multiple visual angles.

The specific implementation process is shown as S02 in fig. 13, please refer to S02.

S305, the industrial personal computer binds the matching information and the coding information and stores the information in a matching information base.

The specific implementation process is shown as S03 in fig. 13, please refer to S03.

S306, the sorting equipment collects images of at least one article in the target area.

The specific implementation process is shown as S01 in fig. 2, please refer to S01.

S307, the sorting equipment classifies the images to obtain at least one article image.

The specific implementation process is shown as S02 in fig. 2, please refer to S02.

S308, the sorting equipment receives a sorting instruction, and the sorting instruction carries target coding information.

The specific implementation process is shown as S03 in fig. 2, please refer to S03.

S309, the sorting equipment responds to the sorting instruction, and matching information corresponding to the target coding information is obtained from the matching information base.

The specific implementation process is shown as S04 in fig. 2, please refer to S04.

S310, the sorting equipment matches the at least one article image with the matching information respectively to obtain the matching degree of the at least one article image.

The specific implementation process is shown as S05 in fig. 2, please refer to S05.

S311, the sorting equipment sorts the target object corresponding to the target coding information based on the matching degree of at least one object image.

The specific implementation process is shown as S06 in fig. 2, please refer to S06.

In the embodiment of the invention, the sorting equipment acquires the image of the target area and processes the image to obtain at least one article image; acquiring target coding information of a target object and matching information corresponding to the target coding information; matching the at least one article image with the matching information respectively to obtain the matching degree of the at least one article image; and determining a sorting order based on the target matching degree, and further sorting the target articles. When the articles are sorted, the collected images of the target area do not need to be subjected to complex calculation to determine the sorting sequence, only the target coding information of the target articles and at least one image of the target area need to be obtained, the target matching information is obtained based on the target coding information, the target articles can be sorted according to the matching degree of the target article images of the target area and the target matching information, and the efficiency of article sorting is remarkably improved.

It should be noted that, in the embodiment of the present invention, if the above-mentioned article sorting method is implemented in the form of a software functional module and sold or used as a standalone product, it may also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for enabling an information consistency detection apparatus (which may be a personal computer or the like) to perform all or part of the methods of the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Thus, embodiments of the invention are not limited to any specific combination of hardware and software.

Correspondingly, the embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the above-mentioned method.

Correspondingly, the embodiment of the present invention provides a sorting device 3, which includes a first memory 302 and a first processor 301, the first memory 302 stores a computer program operable on the first processor 301, and the first processor 301 implements the steps of the method when executing the program.

Here, it should be noted that: the above description of the storage medium and device embodiments is similar to the description of the method embodiments above, with similar advantageous effects as the method embodiments. For technical details not disclosed in the embodiments of the storage medium and the apparatus according to the invention, reference is made to the description of the embodiments of the method according to the invention.

It should be noted that fig. 17 is a schematic diagram of a hardware entity of a sorting apparatus according to an embodiment of the disclosure, and as shown in fig. 17, the hardware entity of the sorting apparatus 3 includes: a first processor 301 and a first memory 302, wherein;

the first processor 301 generally controls the overall operation of the sorting apparatus 3.

The first Memory 302 is configured to store instructions and applications executable by the first processor 301, and may also buffer data (e.g., image data, audio data, voice communication data, and video communication data) to be processed or already processed by the first processor 301 and modules in the sorting apparatus 3, and may be implemented by a FLASH Memory (FLASH) or a Random Access Memory (RAM).

Correspondingly, the embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps in the above method.

Correspondingly, the embodiment of the present invention provides an acquisition apparatus 1, which includes a second memory 102 and a second processor 101, where the second memory 102 stores a computer program that can be executed on the second processor 101, and the processor 101 implements the steps in the above method when executing the program.

It should be noted that fig. 18 is a schematic diagram of a hardware entity of an acquisition device according to an embodiment of the disclosure, as shown in fig. 18, the hardware entity of the acquisition device 1 includes: a second processor 101 and a second memory 102, wherein;

the second processor 101 generally controls the overall operation of the acquisition device 1.

The second Memory 102 is configured to store instructions and applications executable by the processor 101, and may also buffer data (e.g., image data, audio data, voice communication data, and video communication data) to be processed or already processed by each module in the second processor 101 and the acquisition device 1, and may be implemented by a FLASH Memory (FLASH) or a Random Access Memory (RAM).

Correspondingly, the embodiment of the invention provides an industrial personal computer 2, which comprises a third memory 202 and a third processor 201, wherein the third memory 202 stores a computer program capable of running on the third processor 201, and the steps in the method are realized when the processor 201 executes the program.

It should be noted that fig. 19 is a schematic diagram of a hardware entity of an industrial personal computer provided in the embodiments disclosed in the present invention, and as shown in fig. 19, the hardware entity of the industrial personal computer 2 includes: a third processor 201 and a third memory 202, wherein;

the third processor 201 generally controls the overall operation of the industrial personal computer 2.

The third Memory 202 is configured to store instructions and applications executable by the processor 201, and may also buffer data (e.g., image data, audio data, voice communication data, and video communication data) to be processed or already processed by each module in the third processor 201 and the industrial personal computer 2, and may be implemented by a FLASH Memory (FLASH) or a Random Access Memory (RAM).

Fig. 20 is an alternative structural schematic diagram of the acquisition device provided in the embodiment of the present invention.

The embodiment of the invention provides a collecting device 1, which comprises: the system comprises a first information acquisition device 11, an image acquisition device 12 and a first sending device 13, wherein the first information acquisition device 11 is connected with the image acquisition device 12, and the image acquisition device 12 is connected with the first sending device 13; the first information acquisition device 11 is used for acquiring shape information of the article and sending the shape information to the image acquisition device 12; the image acquisition device 12 is used for acquiring the coded information corresponding to the article based on the shape information, acquiring the description information of the article at a plurality of visual angles, and sending the description information and the coded information to the first sending device 13; and the first sending device 13 is used for sending the description information and the coding information to the industrial personal computer 2.

In the embodiment of the present invention, the image capturing device 12 may be: camera, sweep a yard rifle. The camera may be a 2D camera or a 3D camera.

Fig. 21 is an alternative structural schematic diagram of the acquisition device provided in the embodiment of the present invention.

The image acquisition device 12 provided by the embodiment of the invention comprises: a carrying device 121, a first camera 122, a second camera 123 and a code scanning gun 124; the first camera 122, the second camera 123 and the code scanning gun 124 are respectively connected with the first information acquisition device 11 and the first sending device 13; the first camera 122 is arranged in the horizontal direction of the bearing surface 125 of the bearing device 121, and the second camera 123 and the code scanning gun 124 are arranged above the bearing surface 125; the first information acquisition means 11, further configured to send the shape information to the first camera 122 and the second camera 123; a first camera 122, configured to collect description information of the article at multiple viewing angles in the horizontal direction when the shape information represents that the article is in the first shape, and send the description information to the first sending device 13; the first camera 122 is used for collecting description information of the article above the article and sending the description information to the first sending device when the shape information represents that the article is in the second shape; and the code scanning gun 124 is used for collecting the coded information corresponding to the article and sending the coded information to the first sending device 13.

In this embodiment of the present invention, the carrying device 121 may include: a drive device 126 and a bearing surface 125. The driving device 126 may be a servo motor. The support surface 125 may be a flat glass support surface. The driving device 126 drives the bearing surface 125 to rotate.

Fig. 22 is an optional schematic structural diagram of an industrial personal computer provided in an embodiment of the present invention.

The embodiment of the invention provides an industrial personal computer 2, which comprises:

a second information receiving device 21, an information processing device 22 and a second transmitting device 23, wherein the second information receiving device 21 is connected with the information processing device 22, and the information processing device 22 is connected with the second transmitting device 23;

the second information receiving device 21 is used for acquiring the description information and the coding information sent by the acquisition equipment and sending the coding information and the description information to the information processing device 22; the description information includes: three-dimensional images and dimensional information for multiple perspectives of an article; an information processing device 22 for forming matching information corresponding to the article based on the three-dimensional images and the size information of the plurality of visual angles of the article, and transmitting the matching information and the coding information to a second transmitting device 23; and the second sending device 23 is used for binding the matching information and the coding information to form a matching information base.

In the embodiment of the present invention, the information processing apparatus 22 is further configured to fuse three-dimensional images of multiple viewing angles of the article to obtain a three-dimensional matching image corresponding to the article;

in the embodiment of the present invention, the information processing device 22 is further configured to form matching information based on the three-dimensional matching image and the size information, and send the matching information and the encoding information to the second sending device 23.

In the embodiment of the present invention, the information processing device 22 is further configured to extract color information and color position information corresponding to the color information in a three-dimensional image of the article at multiple viewing angles, arrange the color information according to the color position information, form a two-dimensional matching image of the article at multiple viewing angles, form matching information based on the two-dimensional matching image and the size information, and send the matching information and the coding information to the second sending device 23.

An embodiment of the present invention provides a sorting device 3, including:

a sorting acquisition device 31, a sorting processing device 32, a sorting receiving device 33 and a mechanical arm 33; the sorting acquisition device 31 and the sorting receiving device 33 are connected with the sorting processing device 32, and the sorting processing device 32 is connected with the mechanical arm 34;

a sorting acquisition device 31 for acquiring an image of at least one article in the target area and for sending the image and a coded message to a sorting processing device 32;

a sorting processing device 32, configured to perform a sorting process on the images to obtain at least one article image;

the sorting receiving device 33 is used for receiving a sorting instruction, and the sorting instruction carries target coding information;

the sorting receiving device 33 is further configured to respond to the sorting instruction, acquire target matching information corresponding to the target coding information from the matching information base, and send the target matching information to the sorting processing device 32;

and the sorting processing device 32 is configured to match the at least one article image with the target matching information respectively to obtain a matching degree of the at least one article image, and transfer the mechanical arm to sort the target article corresponding to the target coding information based on the matching degree of the at least one article image.

In this embodiment of the present invention, the sorting processing device 32 is further configured to determine, based on the matching degree of at least one article, a target matching degree corresponding to a target article image corresponding to the target encoding information;

the sorting processing device 32 is further configured to obtain, in the image, position information of the target object corresponding to the target matching degree based on the target matching degree;

and the sorting processing device 32 is further configured to invoke the mechanical arm 34 to sort the target item from the position represented by the position information corresponding to the target matching degree based on the target matching degree.

In an embodiment of the invention, the image is a three-dimensional image and the at least one article image is a three-dimensional image of the at least one article.

In the embodiment of the invention, the image is a three-dimensional image; the sorting processing device 32 is further configured to extract color information in the three-dimensional image and color position information corresponding to the color information, arrange the color information according to the color position information, and form a two-dimensional image of the target area;

the sorting processing device 32 is further configured to perform a classification process on the two-dimensional image to obtain a two-dimensional image of at least one article.

In the embodiment of the present invention, the target matching information includes: three-dimensional matching images and size information corresponding to the target coding information;

the sorting processing device 32 is further configured to adjust the three-dimensional matching image and the three-dimensional image of the at least one article to the same size respectively based on the size information, so as to obtain a three-dimensional matching equal-proportion image corresponding to the three-dimensional matching image and at least one three-dimensional equal-proportion image corresponding to the three-dimensional image of the at least one article;

the sorting processing device 32 is further configured to match the at least one three-dimensional equal-proportion image with the three-dimensional matching equal-proportion image, respectively, so as to obtain a matching degree of the at least one article.

In the embodiment of the present invention, the target matching information includes: two-dimensional matching images and size information corresponding to the target coding information;

the sorting processing device 32 is further configured to adjust the two-dimensional matching image and the two-dimensional image of the at least one article to the same size respectively based on the size information, so as to obtain a two-dimensional matching equal-proportion image corresponding to the two-dimensional matching image and at least one two-dimensional equal-proportion image corresponding to the two-dimensional image of the at least one article;

the sorting processing device 32 is further configured to match the at least one two-dimensional equal-proportion image with the two-dimensional matching equal-proportion image, respectively, so as to obtain a matching degree of the at least one article.

In this embodiment of the present invention, the sorting processing device 32 is further configured to determine, when the matching degree of the at least one article is greater than the threshold matching degree, that the matching degree of the target article image corresponding to the target encoding information in the at least one article is the target matching degree.

In the embodiment of the present invention, the sorting and collecting device 31 is further configured to scan a code plate corresponding to at least one article in the target area to obtain code information corresponding to the at least one article, and send the code information corresponding to the at least one article to the sorting and processing device 32;

the sorting receiving device 33 is further configured to, when the target coding information belongs to the coding information corresponding to the at least one article, respond to the sorting instruction to obtain target matching information corresponding to the target coding information from the target matching information base;

the sorting processing device 32 is further configured to stop the sorting in response to the sorting instruction when the target coded information does not belong to the coded information corresponding to the at least one article.

In the embodiment of the invention, the sorting equipment acquires the image of the target area through the sorting acquisition device and processes the image to obtain at least one article image; acquiring target code information of a target object and matching information corresponding to the target code information through a sorting receiving device; matching the at least one article image with the matching information respectively to obtain the matching degree of the at least one article image; and determining a sorting order based on the target matching degree, and then calling the mechanical arm to sort the target articles. When the articles are sorted, the collected images of the target area do not need to be subjected to complex calculation to determine the sorting sequence, only the target coding information of the target articles and at least one image of the target area need to be obtained, the target matching information is obtained based on the target coding information, the target articles can be sorted according to the matching degree of the target article images of the target area and the target matching information, and the efficiency of article sorting is remarkably improved.

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present invention, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention. The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, all the functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.

Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media that can store program codes, such as a removable Memory device, a Read Only Memory (ROM), a magnetic disk, or an optical disk.

Alternatively, the integrated unit of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, a ROM, a magnetic or optical disk, or other various media that can store program code.

The above description is only an embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present invention, and all such changes or substitutions are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.

Claims

1. A method of sorting articles, comprising:

acquiring an image of at least one item within a target area;

classifying the images to obtain at least one article image;

2. The method for sorting the objects according to claim 1, wherein the sorting the object corresponding to the object code information based on the matching degree of the at least one object comprises:

3. The article sorting method according to claim 2,

the image is a three-dimensional image and the at least one article image is a three-dimensional image of at least one article.

4. The article sorting method according to claim 2, wherein the image is a three-dimensional image; the classifying the image to obtain at least one article image includes:

5. The item sorting method according to claim 3, wherein the target matching information includes: the three-dimensional matching image and the size information corresponding to the target coding information;

6. The item sorting method according to claim 4, wherein the target matching information includes: the two-dimensional matching image and the size information corresponding to the target coding information;

7. The method for sorting the objects according to claim 5 or 6, wherein the determining the target matching degree corresponding to the target object image corresponding to the target code information based on the matching degree of the at least one object comprises:

8. The method for sorting objects according to claim 1, wherein after the capturing of the image of at least one object in the target area, before the obtaining of the target matching information corresponding to the target encoding information in the matching information base in response to the sorting instruction, the method further comprises:

9. The method for sorting the articles according to claim 8, wherein after the scanning the coded card corresponding to the at least one article in the target area to obtain the coded information corresponding to the at least one article, the method further comprises:

10. A method of sorting articles, comprising:

acquiring shape information of an article through an image acquisition device;

11. The method for sorting the articles according to claim 10, wherein the collecting, based on the shape information, the coded information corresponding to the articles by a code scanning gun and the collecting the description information of the articles at a plurality of visual angles by an image collecting device comprises:

12. A method of sorting articles, comprising:

13. The method for sorting the objects according to claim 12, wherein the forming the matching information corresponding to the object based on the three-dimensional image and the size information of the object from a plurality of perspectives comprises:

14. The method for sorting the objects according to claim 12, wherein the forming the matching information corresponding to the object based on the three-dimensional image and the size information of the object from a plurality of perspectives comprises:

15. An acquisition device, comprising:

16. An industrial personal computer, comprising:

17. A sorting device, comprising:

18. A sorting device, comprising a first memory storing a computer program operable on a first processor and the first processor implementing the steps of the method according to any one of claims 1 to 9 when executing the program.

19. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 9.

20. An acquisition device comprising a second memory storing a computer program operable on a second processor and a second processor implementing the steps of the method of any one of claims 10 to 11 when the program is executed.

21. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any one of claims 10 to 11.

22. An industrial personal computer, comprising a third memory and a third processor, wherein the third memory stores a computer program operable on the third processor, and the third processor executes the computer program to implement the steps of the method according to any one of claims 12 to 14.

23. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any one of claims 12 to 14.