CN111144322A - Sorting method, device, equipment and storage medium - Google Patents
Sorting method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN111144322A CN111144322A CN201911383380.0A CN201911383380A CN111144322A CN 111144322 A CN111144322 A CN 111144322A CN 201911383380 A CN201911383380 A CN 201911383380A CN 111144322 A CN111144322 A CN 111144322A
- Authority
- CN
- China
- Prior art keywords
- point coordinate
- target object
- information
- sorting
- position information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/02—Measures preceding sorting, e.g. arranging articles in a stream orientating
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/34—Sorting according to other particular properties
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/34—Sorting according to other particular properties
- B07C5/342—Sorting according to other particular properties according to optical properties, e.g. colour
- B07C5/3422—Sorting according to other particular properties according to optical properties, e.g. colour using video scanning devices, e.g. TV-cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Geometry (AREA)
- Multimedia (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a sorting method, a sorting device, sorting equipment and a storage medium, wherein the method comprises the following steps: acquiring a target image, and acquiring type information, position information and contour information of a target object in the target image through an image recognition model; determining the inclination angle of the target object according to the coordinates of the central point and at least one identification point included in the position information; and controlling a sorting device to sort the target object according to the type information, the position information, the outline information and the inclination angle of the target object. According to the technical scheme provided by the embodiment of the invention, the type information, the position information and the contour information of the target object are acquired through the identification function of the image identification model, the inclination angle of the object is acquired according to the position information, and then the sorting operation of the object is completed, so that the object is guaranteed to be sorted in a correct posture while the identification accuracy and the identification efficiency of the object are improved, and the sorting effect is improved.
Description
Technical Field
Embodiments of the present invention relate to image recognition and robotics, and in particular, to a sorting method, apparatus, device, and storage medium.
Background
With the continuous progress of science and technology, the robot technology is rapidly developed, and as an important branch of the robot technology, the sorting robot is widely applied to industrial production.
In the prior art, a template matching method is adopted by a sorting robot through acquired image information, namely, a target object appearing in the image information is matched with a template, the type of the object is determined, and then the object is sorted to a corresponding position through grabbing of the sorting robot.
Disclosure of Invention
The embodiment of the invention provides a sorting method, a sorting device, sorting equipment and a storage medium, which are used for finishing the identification and sorting of articles on the premise of ensuring the correct posture of the articles.
In a first aspect, an embodiment of the present invention provides a sorting method, including:
acquiring a target image, and acquiring type information, position information and contour information of a target object in the target image through an image recognition model; the position information comprises a central point coordinate and at least one identification mark point coordinate;
determining the inclination angle of the target object according to the coordinates of the central point and the coordinates of at least one identification mark point;
and controlling a sorting device to sort the target object according to the type information, the position information, the outline information and the inclination angle of the target object.
In a second aspect, an embodiment of the present invention provides a sorting device, including:
the image recognition execution module is used for acquiring a target image and acquiring type information, position information and contour information of a target object in the target image through an image recognition model; the position information comprises a central point coordinate and at least one identification mark point coordinate;
the inclination angle acquisition module is used for determining the inclination angle of the target object according to the central point coordinate and the at least one identification mark point coordinate;
and the sorting execution module is used for controlling sorting equipment to sort the target object according to the type information, the position information, the outline information and the inclination angle of the target object.
In a third aspect, an embodiment of the present invention further provides an apparatus, where the apparatus includes:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the sorting method of any embodiment of the present invention.
In a fourth aspect, embodiments of the present invention further provide a storage medium containing computer-executable instructions, which when executed by a computer processor implement the sorting method according to any embodiment of the present invention.
According to the technical scheme provided by the embodiment of the invention, the type information, the position information and the contour information of the target object are acquired through the identification function of the image identification model, the inclination angle of the object is acquired according to the position information, and then the sorting operation of the object is completed, so that the object is guaranteed to be sorted in a correct posture while the identification accuracy and the identification efficiency of the object are improved, and the sorting effect is improved.
Drawings
FIG. 1A is a flow chart of a sorting method according to an embodiment of the present invention;
fig. 1B is a schematic diagram of a process of identifying a target image by using a deep convolutional neural network model according to an embodiment of the present invention;
FIG. 1C is a schematic diagram of profile information of a target object according to an embodiment of the present invention;
FIG. 1D is a schematic diagram of a directional line segment of a target object according to an embodiment of the present invention;
fig. 2 is a block diagram of a sorting device according to a second embodiment of the present invention;
fig. 3 is a block diagram of a device according to a third embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1A is a flowchart of a sorting method according to an embodiment of the present invention, where the present embodiment is applicable to a case where an article is sorted by image information, and the method may be executed by a sorting apparatus according to an embodiment of the present invention, where the apparatus may be implemented by software and/or hardware, and may be integrated in a terminal device, and typically, may be integrated in a sorting manipulator, and the method specifically includes the following steps:
s110, obtaining a target image, and obtaining type information, position information and outline information of a target object in the target image through an image recognition model; the position information includes center point coordinates and at least one identification point coordinates.
The target image is derived from a camera installed at a fixed position or on sorting equipment (such as a sorting manipulator), and image recognition operation is carried out on image information shot by the camera through an image recognition model to obtain an article included in the image information; wherein the target image includes one or more items therein.
Optionally, in an embodiment of the present invention, the image recognition model includes a deep convolutional neural network model. Specifically, a training sample is obtained, and a deep convolutional neural network model is trained to obtain the trained deep convolutional neural network model; carrying out image recognition test on the trained deep convolutional neural network model through a test sample, and obtaining the reliability of an output result of the deep convolutional neural network model; judging whether the reliability is greater than or equal to a first preset threshold value; if the reliability is greater than or equal to the first preset threshold value, taking the trained deep convolutional neural network model as an image recognition model; if the reliability is smaller than the first preset threshold, continuing training the deep convolutional neural network model until the reliability of the trained deep convolutional neural network model on the output result of the test sample is larger than or equal to the first preset threshold. The training sample is a marked image, and comprises the steps of marking the outline information, the type information and the position information of an article, and marking whether marking points of the outline information and the position information are blocked or not; particularly, for articles with partial areas being blocked, complete contour information is marked in the training sample, so that the deep convolutional neural network model has an associative memory function; the deep convolutional neural network model trained by the training sample also needs to be subjected to image recognition test of the test sample so as to judge whether the obtained deep convolutional neural network model meets the requirements. Specifically, when training a training sample, the convolutional neural network model completes the processes of information decomposition, feature extraction and information recombination, and finally obtains a model for determining parameter values; when a new image (for example, a test sample) is acquired, the sample is trained based on the learned parameter information, the processes of information decomposition, feature extraction and information recombination are automatically realized in the forward reasoning process, the results of the position information, the contour information and the type information of the article in the training sample are obtained, meanwhile, the reliability of the result is calculated by using a set probability formula, and the value of the reliability is output; the reliability, namely the recognition reliability, is the prediction of the target image recognition accuracy (for example, the reliability is 70%), namely the matching degree of the article information and the real situation calculated by the inference budget of the deep convolutional neural network model; the first preset threshold can be set as required, and when the reliability is greater than or equal to the first preset threshold, the accuracy of the deep convolutional neural network model is high and meets the requirement; and when the reliability is smaller than a first preset threshold value, the accuracy of the deep convolutional neural network model is low, and the requirement is not met.
As shown in fig. 1B, the target image is input to a Backbone network (Backbone) of the deep convolutional neural network model, and the processes of information decomposition, feature extraction and information recombination are implemented to extract image information; the backbone network comprises a convolution module, a pooling module, an activation module, a normalization module and the like, wherein the convolution module is used for performing convolution processing on a target image; the pooling module is used for pooling the target image, namely performing an abstraction process on the target image, the activation module is used for processing the input information through an activation function, and the activation function is used for converting linear data into nonlinear data; the normalization module is used for performing normalization processing on the target image; the backbone network may be derived from a VGG network (Visual Geometry group network), such as VGG16 and VGG19, a Stacked hourglass network (Stacked HourglassNetwork), or a lightweight convolutional neural network such as MobileNet or ShuffleNet. Obtaining a Feature map (Feature Maps) after the target image is operated by a backbone network, wherein the Feature map emphasizes texture information in the target image; continuously carrying out identification processing on the feature map to obtain Region of Interest alignment (ROI alignment); the Region of Interest (ROI) is a Region containing an article, and the regions of Interest are aligned, that is, various feature points of the article, such as a trademark, a name, a barcode, a trademark registration mark, and the like, are located in the Region of Interest. Processing the target image subjected to the alignment processing of the region of interest through a fully-connected convolutional neural network (FC Nets) to obtain Mask (Mask) information; wherein the fully-connected convolutional neural network comprises a plurality of fully-connected layers (FCLayers), the mask information is a binary image (represented by a matrix) which contains a part of the article and has a value of 1; a portion containing no articles, value 0; and then correcting the target image subjected to the region-of-interest alignment processing through the mask information to obtain corrected adjusted region-of-interest alignment (Adaptive ROI Align), and then performing identification processing through one or more full connection layers to finally obtain the article information of the article in the image.
Type information, i.e., the classification of the articles, according to which the articles of the same type are placed in the same area when performing an article sorting operation, for example, the articles may be classified into "snacks", "seafood", "tools", and "living goods"; further sub-divisions of the type of items can be made, for example, in the "snack" type the items are further divided into "beverages", "dried fruits" and "puffed snacks", and in the "seafood" type the items are further divided into "crayfish" and "live crabs"; optionally, in the embodiment of the present invention, the classification of the article type is not particularly limited. The outline information is a line drawn around the edge of the article and indicates the actual outline of the article, and for example, the outline information of the articles "potato chips" and "fried dough twists" is indicated by a dotted line in fig. 1C. The position information represents the position of the article and comprises a central point coordinate and an identification mark point coordinate; the identification mark point coordinates comprise trademark center point coordinates, name center point coordinates, bar code center point coordinates and trademark registration mark center point (namely trademark side coordinates)Of the edge
Identification) of at least one of the coordinates.
Optionally, in an embodiment of the present invention, determining the inclination angle of the target item according to the coordinates of the central point and the coordinates of the at least one identification point includes: and determining the inclination angle of the target object according to the directed line segment from the center point coordinate to at least one identification mark point coordinate. For example, in fig. 1D, point a is the center point of the target item "potato chip", point B is the name center point, and serves as the identification point of the target item "potato chip", and since the inclination angle of the directional line segment pointing to point B at point a is fixed in the item itself (for example, 30 degrees to the left), the inclination degree of the target item in the image (40 degrees to the left) can be determined according to the inclination degree (for example, 70 degrees to the left) of the directional line segment pointing to point B at point a in the image. Specifically, if the position information includes a plurality of identification point coordinates, after acquiring a target image and acquiring type information, position information, and contour information of a target object in the target image through an image recognition model, the method further includes: determining the identification mark point coordinate with the highest priority according to the priority of each identification mark point coordinate; correspondingly, determining the inclination angle of the target object according to the directional line segment from the center point coordinate to at least one identification point coordinate, including: and determining the inclination angle of the target object according to the directed line segment from the center point coordinate to the identification point coordinate with the highest priority. Because the target object may have shielding and part of the identification mark points are shielded and cannot be identified, a plurality of identification mark points can be set for one target object, and when the plurality of identification mark points are obtained, the priority order of the obtained identification mark points is determined according to the set priority order; for example, the priority of the preset trademark center point is higher than that of the name center point, the priority of the name center point is higher than that of the trademark registration mark center point, and the priority of the trademark registration mark center point is higher than that of the bar code center point, wherein the trademark center point and the bar code center point are blocked and cannot be acquired, the name center point coordinate and the trademark registration mark center point coordinate of the article can be acquired, and the name center point coordinate is higher than that of the trademark registration mark center point, so that the name center point coordinate is used as the identification mark point coordinate with the highest priority.
Particularly, besides type information, position information and contour information of a target object in a target image, the reliability of an output result of the image recognition model can be acquired through the image recognition model to represent the prediction of the recognition accuracy of the target object, the accuracy of the current recognition can be judged through comparing the reliability with a set second preset threshold, if the reliability is greater than or equal to the second preset threshold, the current recognition of the target object is represented, and the reliability is high; if the reliability is smaller than the second preset threshold value, the identification of the target object is represented, the reliability is low, and at the moment, the identification of the target object is prevented from making mistakes by sending out an alarm signal or a prompt message, so that the sorting error is generated.
And S120, determining the inclination angle of the target object according to the center point coordinate and the at least one identification mark point coordinate.
S130, controlling a sorting device to sort the target object according to the type information, the position information, the contour information and the inclination angle of the target object.
According to the position information of the target object, the target object is positioned, then the sorting action of the sorting equipment is adjusted according to the contour information, for example, the opening degree of a sorting mechanical arm is adjusted, after the target object is grabbed, the posture of the object is adjusted according to the inclination angle of the object, then the target object is sorted to a designated area according to the type information, and the posture of the object is kept correct. Specifically, if the identification mark point coordinate with the highest priority is a barcode center point coordinate, controlling the sorting device to sort the target object, including: and controlling the sorting equipment to turn the target object by 180 degrees. Since the barcode is usually located on the back of the target object, when the coordinates of the identification point with the highest priority are the coordinates of the point in the area where the barcode is located, the back of the target object faces upward, and the target object needs to be turned over by 180 degrees in addition to adjusting the inclination angle of the target object in the target image, so that the target object is kept in a front display state. Particularly, if the target image comprises a plurality of target articles, one sorting device can be controlled to sequentially sort the target articles, and a plurality of sorting devices can be controlled to simultaneously sort the target articles. The sorting device may include a sorting robot, and may also include other types of sorting robots, and in the embodiment of the present invention, the type of the sorting device is not particularly limited. In the embodiment of the present invention, the lower edge of the target image may be taken as an abscissa axis, i.e., an x axis; the left edge is taken as the ordinate, namely the y axis; a coordinate axis which is perpendicular to the x axis and the y axis and points to the outer side of the target image is used as a z axis; and adjusting the posture of the object, wherein the posture comprises inclination angle adjustment and turning angle adjustment, wherein the inclination angle adjustment is to rotate the target object in a plane formed by an x axis and a y axis, and the turning angle adjustment is to turn the target object in situ along the z axis.
According to the technical scheme provided by the embodiment of the invention, the type information, the position information and the contour information of the target object are acquired through the identification function of the image identification model, the inclination angle of the object is acquired according to the position information, and then the sorting operation of the object is completed, so that the object is guaranteed to be sorted in a correct posture while the identification accuracy and the identification efficiency of the object are improved, and the sorting effect is improved.
Example two
Fig. 2 is a block diagram of a sorting apparatus according to a second embodiment of the present invention, where the apparatus includes: a pattern recognition execution module 210, a tilt angle acquisition module 220, and a sorting execution module 230.
The pattern recognition execution module 210 is configured to obtain a target image, and obtain type information, position information, and contour information of a target object in the target image through an image recognition model; the position information comprises a central point coordinate and at least one identification mark point coordinate;
an inclination angle obtaining module 220, configured to determine an inclination angle of the target object according to the center point coordinate and the at least one identification point coordinate;
a sorting execution module 230, configured to control a sorting apparatus to sort the target object according to the type information, the position information, the profile information, and the tilt angle of the target object.
According to the technical scheme provided by the embodiment of the invention, the type information, the position information and the contour information of the target object are acquired through the identification function of the image identification model, the inclination angle of the object is acquired according to the position information, and then the sorting operation of the object is completed, so that the object is guaranteed to be sorted in a correct posture while the identification accuracy and the identification efficiency of the object are improved, and the sorting effect is improved.
Optionally, on the basis of the above technical solution, the identification mark point coordinate includes at least one of a trademark center point coordinate, a name center point coordinate, a barcode center point coordinate, and a trademark registration mark center point coordinate.
Optionally, on the basis of the above technical solution, the inclination angle obtaining module 220 is specifically configured to:
and determining the inclination angle of the target object according to the directed line segment from the center point coordinate to at least one identification mark point coordinate.
Optionally, on the basis of the above technical solution, if the location information includes a plurality of identification point coordinates, the sorting apparatus further includes:
and the priority determining module is used for determining the identification mark point coordinate with the highest priority according to the priority of each identification mark point coordinate.
Optionally, on the basis of the above technical solution, the inclination angle obtaining module 220 is specifically configured to:
and determining the inclination angle of the target object according to the directed line segment from the center point coordinate to the identification point coordinate with the highest priority.
Optionally, on the basis of the above technical solution, if the identification point coordinate with the highest priority is a barcode center point coordinate, the sorting executing module 230 is further configured to:
and controlling the sorting equipment to turn the target object by 180 degrees.
Optionally, on the basis of the above technical solution, the image recognition model includes a deep convolutional neural network model.
Optionally, on the basis of the above technical solution, the sorting apparatus further includes:
the training execution module is used for acquiring a training sample and training the deep convolutional neural network model to acquire the trained deep convolutional neural network model;
the test execution module is used for carrying out image recognition test on the trained deep convolutional neural network model through a test sample and obtaining the reliability of an output result of the deep convolutional neural network model;
the reliability judging module is used for judging whether the reliability is greater than or equal to a first preset threshold value; if the reliability is greater than or equal to the first preset threshold value, taking the trained deep convolutional neural network model as an image recognition model; if the reliability is smaller than the first preset threshold, continuing training the deep convolutional neural network model until the reliability of the trained deep convolutional neural network model on the output result of the test sample is larger than or equal to the first preset threshold.
The device can execute the sorting method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method. For technical details not described in detail in this embodiment, reference may be made to the method provided in any embodiment of the present invention.
EXAMPLE III
Fig. 3 is a schematic structural diagram of an apparatus according to a third embodiment of the present invention. Fig. 3 illustrates a block diagram of an exemplary device 12 suitable for use in implementing embodiments of the present invention. The device 12 shown in fig. 3 is only an example and should not bring any limitations to the functionality and scope of use of the embodiments of the present invention.
As shown in FIG. 3, device 12 is in the form of a general purpose computing device. The components of device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.
The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory 32. Device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 3, and commonly referred to as a "hard drive"). Although not shown in FIG. 3, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.
The processing unit 16 executes various functional applications and data processing, such as implementing the sorting method provided by embodiments of the present invention, by running a program stored in the system memory 28. Namely: acquiring a target image, and acquiring type information, position information and contour information of a target object in the target image through an image recognition model; the position information comprises a central point coordinate and at least one identification mark point coordinate; determining the inclination angle of the target object according to the coordinates of the central point and the coordinates of at least one identification mark point; and controlling a sorting device to sort the target object according to the type information, the position information, the outline information and the inclination angle of the target object.
Example four
A fourth embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the sorting method according to any embodiment of the present invention; the method comprises the following steps:
acquiring a target image, and acquiring type information, position information and contour information of a target object in the target image through an image recognition model; the position information comprises a central point coordinate and at least one identification mark point coordinate;
determining the inclination angle of the target object according to the coordinates of the central point and the coordinates of at least one identification mark point;
and controlling a sorting device to sort the target object according to the type information, the position information, the outline information and the inclination angle of the target object.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A method of sorting, comprising:
acquiring a target image, and acquiring type information, position information and contour information of a target object in the target image through an image recognition model; the position information comprises a central point coordinate and at least one identification mark point coordinate;
determining the inclination angle of the target object according to the coordinates of the central point and the coordinates of at least one identification mark point;
and controlling a sorting device to sort the target object according to the type information, the position information, the outline information and the inclination angle of the target object.
2. The method of claim 1, wherein the identification mark point coordinates include at least one of a trademark center point coordinate, a name center point coordinate, a barcode center point coordinate, and a trademark registration mark center point coordinate.
3. The method of claim 1 or 2, wherein determining the tilt angle of the target item based on the center point coordinates and at least one of the id point coordinates comprises:
and determining the inclination angle of the target object according to the directed line segment from the center point coordinate to at least one identification mark point coordinate.
4. The method according to claim 3, wherein if the position information includes a plurality of identification point coordinates, after acquiring the target image and acquiring the type information, the position information and the contour information of the target object in the target image through the image recognition model, the method further comprises:
determining the identification mark point coordinate with the highest priority according to the priority of each identification mark point coordinate;
correspondingly, determining the inclination angle of the target object according to the directional line segment from the center point coordinate to at least one identification point coordinate, including:
and determining the inclination angle of the target object according to the directed line segment from the center point coordinate to the identification point coordinate with the highest priority.
5. The method according to claim 4, wherein if the identification point coordinate with the highest priority is a barcode center point coordinate, controlling a sorting device to sort the target item comprises:
and controlling the sorting equipment to turn the target object by 180 degrees.
6. The method of claim 1, wherein the image recognition model comprises a deep convolutional neural network model.
7. The method of claim 6, further comprising, prior to acquiring the target image:
obtaining a training sample, and training a deep convolutional neural network model to obtain the trained deep convolutional neural network model;
carrying out image recognition test on the trained deep convolutional neural network model through a test sample, and obtaining the reliability of an output result of the deep convolutional neural network model;
judging whether the reliability is greater than or equal to a first preset threshold value;
if the reliability is greater than or equal to the first preset threshold value, taking the trained deep convolutional neural network model as an image recognition model;
if the reliability is smaller than the first preset threshold, continuing training the deep convolutional neural network model until the reliability of the trained deep convolutional neural network model on the output result of the test sample is larger than or equal to the first preset threshold.
8. A sorting device, comprising:
the image recognition execution module is used for acquiring a target image and acquiring type information, position information and contour information of a target object in the target image through an image recognition model; the position information comprises a central point coordinate and at least one identification mark point coordinate;
the inclination angle acquisition module is used for determining the inclination angle of the target object according to the central point coordinate and the at least one identification mark point coordinate;
and the sorting execution module is used for controlling sorting equipment to sort the target object according to the type information, the position information, the outline information and the inclination angle of the target object.
9. An apparatus, characterized in that the apparatus comprises:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the sorting method of any of claims 1-7.
10. A storage medium containing computer-executable instructions for performing the sorting method of any one of claims 1-7 when executed by a computer processor.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911383380.0A CN111144322A (en) | 2019-12-28 | 2019-12-28 | Sorting method, device, equipment and storage medium |
| PCT/CN2020/137477 WO2021129528A1 (en) | 2019-12-28 | 2020-12-18 | Sorting method and apparatus, and device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911383380.0A CN111144322A (en) | 2019-12-28 | 2019-12-28 | Sorting method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111144322A true CN111144322A (en) | 2020-05-12 |
Family
ID=70521453
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911383380.0A Pending CN111144322A (en) | 2019-12-28 | 2019-12-28 | Sorting method, device, equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111144322A (en) |
| WO (1) | WO2021129528A1 (en) |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111702761A (en) * | 2020-06-12 | 2020-09-25 | 珠海格力智能装备有限公司 | Control method and device of palletizing robot, processor and sorting system |
| CN112139053A (en) * | 2020-09-14 | 2020-12-29 | 广州市几米物联科技有限公司 | Visual guidance method, system and computer readable storage medium |
| CN112232142A (en) * | 2020-09-27 | 2021-01-15 | 浙江大华技术股份有限公司 | Safety belt identification method and device and computer readable storage medium |
| CN112295949A (en) * | 2020-10-13 | 2021-02-02 | 广州纳诺科技股份有限公司 | Visual intelligent sorting method and system based on deep neural network |
| CN112749735A (en) * | 2020-12-30 | 2021-05-04 | 中冶赛迪重庆信息技术有限公司 | Converter tapping steel flow identification method, system, medium and terminal based on deep learning |
| CN113011823A (en) * | 2021-04-16 | 2021-06-22 | 上海克来机电自动化工程股份有限公司 | Boxing method, device, electronic equipment and storage medium |
| WO2021129528A1 (en) * | 2019-12-28 | 2021-07-01 | 广东拓斯达科技股份有限公司 | Sorting method and apparatus, and device and storage medium |
| CN113222063A (en) * | 2021-05-31 | 2021-08-06 | 平安科技（深圳）有限公司 | Express carton garbage classification method, device, equipment and medium |
| CN113240376A (en) * | 2021-05-31 | 2021-08-10 | 中邮信息科技(北京)有限公司 | Article information determination method, article information determination device, electronic device, and medium |
| CN113245235A (en) * | 2021-04-16 | 2021-08-13 | 嘉联支付有限公司 | Commodity classification method and device based on 3D vision |
| CN113807293A (en) * | 2021-09-24 | 2021-12-17 | 纵目科技(重庆)有限公司 | Deceleration strip detection method, system, equipment and computer readable storage medium |
| CN114043531A (en) * | 2021-12-15 | 2022-02-15 | 东莞市李群自动化技术有限公司 | Table top inclination angle determination method, table top inclination angle use method, table top inclination angle determination device, robot and storage medium |
| CN114273240A (en) * | 2020-09-27 | 2022-04-05 | 深圳顺丰泰森控股(集团)有限公司 | Express delivery single piece separation method, device and system and storage medium |
| CN114871115A (en) * | 2022-04-28 | 2022-08-09 | 五邑大学 | Object sorting method, device, equipment and storage medium |
| CN114951003A (en) * | 2022-05-06 | 2022-08-30 | 苏州嘉诺智能制造有限公司 | Sorting method and device based on distance detection |
| CN116175542A (en) * | 2021-11-28 | 2023-05-30 | 梅卡曼德（北京）机器人科技有限公司 | Grabbing control method, grabbing control device, electronic equipment and storage medium |
| CN116309442A (en) * | 2023-03-13 | 2023-06-23 | 北京百度网讯科技有限公司 | Method for determining picking information and method for picking target object |
| CN116899915A (en) * | 2023-07-19 | 2023-10-20 | 华至云链科技(苏州)有限公司 | Real-time synchronization method and device for equipment data |
| CN117146828A (en) * | 2023-10-30 | 2023-12-01 | 网思科技股份有限公司 | Method and device for guiding picking path, storage medium and computer equipment |
| CN117197248A (en) * | 2023-11-08 | 2023-12-08 | 成都数之联科技股份有限公司 | Electrolytic capacitor direction judging method, device, equipment and storage medium |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113894050B (en) * | 2021-09-14 | 2023-05-23 | 深圳玩智商科技有限公司 | Logistics part sorting method, sorting equipment and storage medium |
| CN113894061B (en) * | 2021-10-23 | 2023-06-27 | 绍兴文理学院元培学院 | Automatic cargo sorting method and system |
| CN113920142B (en) * | 2021-11-11 | 2023-09-26 | 江苏昱博自动化设备有限公司 | Sorting manipulator multi-object sorting method based on deep learning |
| CN114494398B (en) * | 2022-01-18 | 2024-05-07 | 深圳市联洲国际技术有限公司 | Processing method and device of inclined target, storage medium and processor |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102289681A (en) * | 2011-08-05 | 2011-12-21 | 上海邮政科学研究院 | Method for matching envelope images |
| CN106531161A (en) * | 2016-10-17 | 2017-03-22 | 南京理工大学 | Image-recognition-based apparatus and method of automatically sorting and carrying articles by mobile trolley |
| CN106709909A (en) * | 2016-12-13 | 2017-05-24 | 重庆理工大学 | Flexible robot vision recognition and positioning system based on depth learning |
| CN107804514A (en) * | 2017-10-23 | 2018-03-16 | 江苏艾萨克机器人股份有限公司 | A kind of toothbrush method for sorting based on image recognition |
| US20190139256A1 (en) * | 2017-11-06 | 2019-05-09 | Samsung Electronics Co., Ltd. | Electronic device and method for reliability-based object recognition |
| CN109740681A (en) * | 2019-01-08 | 2019-05-10 | 南方科技大学 | A kind of fruit method for sorting, device, system, terminal and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2998544C (en) * | 2015-09-11 | 2023-05-09 | Berkshire Grey, Inc. | Robotic systems and methods for identifying and processing a variety of objects |
| CN110315525A (en) * | 2018-03-29 | 2019-10-11 | 天津工业大学 | A kind of robot workpiece grabbing method of view-based access control model guidance |
| CN109916826B (en) * | 2019-02-20 | 2024-02-06 | 福建南方路面机械有限公司 | Solid waste online identification system and method based on hyperspectral detection |
| CN111144322A (en) * | 2019-12-28 | 2020-05-12 | 广东拓斯达科技股份有限公司 | Sorting method, device, equipment and storage medium |
-
2019
- 2019-12-28 CN CN201911383380.0A patent/CN111144322A/en active Pending
-
2020
- 2020-12-18 WO PCT/CN2020/137477 patent/WO2021129528A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102289681A (en) * | 2011-08-05 | 2011-12-21 | 上海邮政科学研究院 | Method for matching envelope images |
| CN106531161A (en) * | 2016-10-17 | 2017-03-22 | 南京理工大学 | Image-recognition-based apparatus and method of automatically sorting and carrying articles by mobile trolley |
| CN106709909A (en) * | 2016-12-13 | 2017-05-24 | 重庆理工大学 | Flexible robot vision recognition and positioning system based on depth learning |
| CN107804514A (en) * | 2017-10-23 | 2018-03-16 | 江苏艾萨克机器人股份有限公司 | A kind of toothbrush method for sorting based on image recognition |
| US20190139256A1 (en) * | 2017-11-06 | 2019-05-09 | Samsung Electronics Co., Ltd. | Electronic device and method for reliability-based object recognition |
| CN109740681A (en) * | 2019-01-08 | 2019-05-10 | 南方科技大学 | A kind of fruit method for sorting, device, system, terminal and storage medium |
Cited By (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021129528A1 (en) * | 2019-12-28 | 2021-07-01 | 广东拓斯达科技股份有限公司 | Sorting method and apparatus, and device and storage medium |
| CN111702761A (en) * | 2020-06-12 | 2020-09-25 | 珠海格力智能装备有限公司 | Control method and device of palletizing robot, processor and sorting system |
| CN111702761B (en) * | 2020-06-12 | 2022-05-27 | 珠海格力智能装备有限公司 | Control method and device of palletizing robot, processor and sorting system |
| CN112139053A (en) * | 2020-09-14 | 2020-12-29 | 广州市几米物联科技有限公司 | Visual guidance method, system and computer readable storage medium |
| CN114273240A (en) * | 2020-09-27 | 2022-04-05 | 深圳顺丰泰森控股(集团)有限公司 | Express delivery single piece separation method, device and system and storage medium |
| CN112232142A (en) * | 2020-09-27 | 2021-01-15 | 浙江大华技术股份有限公司 | Safety belt identification method and device and computer readable storage medium |
| CN112295949A (en) * | 2020-10-13 | 2021-02-02 | 广州纳诺科技股份有限公司 | Visual intelligent sorting method and system based on deep neural network |
| CN112749735A (en) * | 2020-12-30 | 2021-05-04 | 中冶赛迪重庆信息技术有限公司 | Converter tapping steel flow identification method, system, medium and terminal based on deep learning |
| CN112749735B (en) * | 2020-12-30 | 2023-04-07 | 中冶赛迪信息技术(重庆)有限公司 | Converter tapping steel flow identification method, system, medium and terminal based on deep learning |
| CN113245235A (en) * | 2021-04-16 | 2021-08-13 | 嘉联支付有限公司 | Commodity classification method and device based on 3D vision |
| CN113011823A (en) * | 2021-04-16 | 2021-06-22 | 上海克来机电自动化工程股份有限公司 | Boxing method, device, electronic equipment and storage medium |
| CN113240376A (en) * | 2021-05-31 | 2021-08-10 | 中邮信息科技(北京)有限公司 | Article information determination method, article information determination device, electronic device, and medium |
| CN113222063A (en) * | 2021-05-31 | 2021-08-06 | 平安科技（深圳）有限公司 | Express carton garbage classification method, device, equipment and medium |
| CN113807293A (en) * | 2021-09-24 | 2021-12-17 | 纵目科技(重庆)有限公司 | Deceleration strip detection method, system, equipment and computer readable storage medium |
| CN113807293B (en) * | 2021-09-24 | 2024-02-09 | 纵目科技(重庆)有限公司 | Deceleration strip detection method, deceleration strip detection system, deceleration strip detection equipment and computer readable storage medium |
| CN116175542A (en) * | 2021-11-28 | 2023-05-30 | 梅卡曼德（北京）机器人科技有限公司 | Grabbing control method, grabbing control device, electronic equipment and storage medium |
| CN116175542B (en) * | 2021-11-28 | 2024-01-26 | 梅卡曼德（北京）机器人科技有限公司 | Method, device, electronic equipment and storage medium for determining clamp grabbing sequence |
| CN114043531A (en) * | 2021-12-15 | 2022-02-15 | 东莞市李群自动化技术有限公司 | Table top inclination angle determination method, table top inclination angle use method, table top inclination angle determination device, robot and storage medium |
| CN114043531B (en) * | 2021-12-15 | 2023-10-03 | 东莞市李群自动化技术有限公司 | Table tilt angle determination, use method, apparatus, robot, and storage medium |
| CN114871115A (en) * | 2022-04-28 | 2022-08-09 | 五邑大学 | Object sorting method, device, equipment and storage medium |
| CN114951003B (en) * | 2022-05-06 | 2023-12-26 | 苏州嘉诺智能制造有限公司 | Sorting method and device based on distance detection |
| CN114951003A (en) * | 2022-05-06 | 2022-08-30 | 苏州嘉诺智能制造有限公司 | Sorting method and device based on distance detection |
| CN116309442B (en) * | 2023-03-13 | 2023-10-24 | 北京百度网讯科技有限公司 | Method for determining picking information and method for picking target object |
| CN116309442A (en) * | 2023-03-13 | 2023-06-23 | 北京百度网讯科技有限公司 | Method for determining picking information and method for picking target object |
| CN116899915A (en) * | 2023-07-19 | 2023-10-20 | 华至云链科技(苏州)有限公司 | Real-time synchronization method and device for equipment data |
| CN116899915B (en) * | 2023-07-19 | 2024-01-09 | 华至云链科技(苏州)有限公司 | Real-time synchronization method and device for equipment data |
| CN117146828A (en) * | 2023-10-30 | 2023-12-01 | 网思科技股份有限公司 | Method and device for guiding picking path, storage medium and computer equipment |
| CN117146828B (en) * | 2023-10-30 | 2024-03-19 | 网思科技股份有限公司 | Method and device for guiding picking path, storage medium and computer equipment |
| CN117197248A (en) * | 2023-11-08 | 2023-12-08 | 成都数之联科技股份有限公司 | Electrolytic capacitor direction judging method, device, equipment and storage medium |
| CN117197248B (en) * | 2023-11-08 | 2024-01-26 | 成都数之联科技股份有限公司 | Electrolytic capacitor direction judging method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021129528A1 (en) | 2021-07-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111144322A (en) | Sorting method, device, equipment and storage medium | |
| CN111144426B (en) | Sorting method, sorting device, sorting equipment and storage medium | |
| US11878433B2 (en) | Method for detecting grasping position of robot in grasping object | |
| CN109934121B (en) | Orchard pedestrian detection method based on YOLOv3 algorithm | |
| WO2020177432A1 (en) | Multi-tag object detection method and system based on target detection network, and apparatuses | |
| US9349076B1 (en) | Template-based target object detection in an image | |
| CN110349147B (en) | Model training method, fundus macular region lesion recognition method, device and equipment | |
| CN107563446A (en) | A kind of micro OS object detection method | |
| WO2019207557A1 (en) | Method for distinguishing a real three-dimensional object from a two-dimensional spoof of the real object | |
| CN111507908B (en) | Image correction processing method, device, storage medium and computer equipment | |
| CN108428248B (en) | Vehicle window positioning method, system, equipment and storage medium | |
| Yin et al. | Graph neural network for 6D object pose estimation | |
| Zhang et al. | Prioritizing robotic grasping of stacked fruit clusters based on stalk location in RGB-D images | |
| CN112936257A (en) | Workpiece grabbing method and device, computer equipment and storage medium | |
| Yin et al. | Overview of robotic grasp detection from 2D to 3D | |
| CN117372928A (en) | Video target detection method and device and related equipment | |
| CN112947458A (en) | Robot accurate grabbing method based on multi-mode information and computer readable medium | |
| CN112857746A (en) | Tracking method and device of lamplight detector, electronic equipment and storage medium | |
| Lin et al. | Robot grasping based on object shape approximation and LightGBM | |
| US20160253570A1 (en) | Feature vector extraction device | |
| CN115995017A (en) | Fruit identification and positioning method, device and medium | |
| CN115861956A (en) | Yolov3 road garbage detection method based on decoupling head | |
| CN115319739A (en) | Workpiece grabbing method based on visual mechanical arm | |
| CN115205806A (en) | Method and device for generating target detection model and automatic driving vehicle | |
| Bhuyan et al. | Structure‐aware multiple salient region detection and localization for autonomous robotic manipulation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |