WO2021129527A1 - 分拣方法、装置、设备和存储介质 - Google Patents

分拣方法、装置、设备和存储介质 Download PDF

Info

Publication number
WO2021129527A1
WO2021129527A1 PCT/CN2020/137476 CN2020137476W WO2021129527A1 WO 2021129527 A1 WO2021129527 A1 WO 2021129527A1 CN 2020137476 W CN2020137476 W CN 2020137476W WO 2021129527 A1 WO2021129527 A1 WO 2021129527A1
Authority
WO
WIPO (PCT)
Prior art keywords
inclination angle
information
target item
position information
target
Prior art date
Application number
PCT/CN2020/137476
Other languages
English (en)
French (fr)
Inventor
徐必业
罗小军
赵磊
王亚平
袁仁辉
吴丰礼
Original Assignee
广东拓斯达科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广东拓斯达科技股份有限公司 filed Critical 广东拓斯达科技股份有限公司
Publication of WO2021129527A1 publication Critical patent/WO2021129527A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/34—Sorting according to other particular properties
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C2501/00—Sorting according to a characteristic or feature of the articles or material to be sorted
    • B07C2501/0081—Sorting of food items

Definitions

  • the embodiments of the present application relate to image recognition and robotics technology, for example, to a sorting method, device, equipment, and storage medium.
  • the sorting robot adopts a template matching method through the acquired image information, that is, matches the target item appearing in the graphic information with the template, determines the type of the item, and then sorts the item through the picking of the sorting robot To the corresponding position, but this image recognition method has low accuracy and efficiency in judging the item type.
  • this image recognition method has low accuracy and efficiency in judging the item type.
  • after grabbing the item it cannot guarantee that the item will be sorted in the correct posture. For example, a beverage bottle is a bottle. The way the mouth is upside down or upside down is grabbed and placed, which seriously affects the sorting effect.
  • the embodiments of the present application provide a sorting method, device, equipment, and storage medium, so as to complete the identification and sorting of the items on the premise that the posture of the items is correct.
  • the embodiment of the present application provides a sorting method, including:
  • the position information includes the first tilt angle
  • the sorting device is controlled to sort the target item.
  • the embodiment of the present application provides a sorting device, including:
  • a graphic recognition execution module configured to obtain a target image, and obtain type information, position information, and contour information of a target item in the target image through an image recognition model; the position information includes a first tilt angle;
  • the second inclination angle determination module is configured to determine the smallest rectangular frame of the target item according to the position information and the outline information, and determine the second inclination angle of the smallest rectangular frame;
  • a true tilt angle calculation module configured to determine the true tilt angle of the target item according to the first tilt angle and the second tilt angle
  • the sorting execution module is configured to control the sorting device to sort the target items according to the type information, the position information, the contour information, and the real inclination angle of the target items.
  • An embodiment of the present application also provides a device, and the device includes:
  • One or more processors are One or more processors;
  • Storage device set to store one or more programs
  • the one or more processors When the one or more programs are executed by the one or more processors, the one or more processors implement the sorting method described in any embodiment of the present application.
  • the embodiments of the present application also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, implement the sorting method described in any embodiment of the present application.
  • FIG. 1A is a flowchart of a sorting method provided in Embodiment 1 of the present application.
  • FIG. 1B is a schematic diagram of outline information of a target item provided in Embodiment 1 of the present application.
  • FIG. 1C is a schematic diagram of the smallest rectangular frame of the target article provided in Embodiment 1 of the present application.
  • FIG. 2 is a flowchart of a sorting method provided in Embodiment 2 of the present application.
  • FIG. 3 is a structural block diagram of a device provided in Embodiment 3 of the present application.
  • FIG. 1A is a flowchart of a sorting method provided in Embodiment 1 of this application. This embodiment can be applied to the situation that items are sorted by image information.
  • the method can be executed by the sorting device in the embodiment of this application.
  • the device can be implemented by software and/or hardware, and can be integrated in a terminal device, for example, can be integrated in a sorting robot.
  • the method includes the following steps:
  • the target image comes from a camera installed at a fixed location or on a sorting device (for example, a sorting manipulator).
  • the image information captured by the camera is imaged through the image recognition model to obtain the items included in the image information.
  • One or more items can be included in the image.
  • the image recognition model is obtained through training and has the function of image recognition.
  • the image recognition model includes a deep convolutional neural network model.
  • the training sample is an annotated image, including the outline information, type information, and location information of the item. It also includes the annotation of whether the labeled points of the outline information and location information are occluded; for items that are partially occluded, the training sample
  • the complete contour information is also marked in the, so that the deep convolutional neural network model has the associative memory function; the deep convolutional neural network model trained by the training sample needs to pass the image recognition test of the test sample to determine the acquisition Whether the obtained deep convolutional neural network model meets the requirements.
  • the convolutional neural network model has completed the process of information decomposition, feature extraction and information reorganization, and obtained model parameter values.
  • the model parameter training sample of the When a new image (for example, test sample) is obtained, it is based on the learned In the process of forward reasoning, the model parameter training sample of the, automatically realizes the process of information decomposition, feature extraction and information reorganization, and obtains the recognition result, that is, the position information, contour information and type information of the items in the training sample, and at the same time uses the established
  • the probability formula calculates the reliability of the result and outputs the value of the reliability.
  • Reliability that is, recognition reliability
  • the first preset threshold can be set as required.
  • the reliability is greater than or equal to the first preset threshold, it means that the accuracy of the deep convolutional neural network model is high and meets the requirements; when the reliability is less than the first preset threshold When, it means that the accuracy of the deep convolutional neural network model is low and does not meet the requirements.
  • the item information includes type information, location information, and outline information.
  • Type information that is, the classification of the items.
  • the items of the same type are placed in the same area during the item sorting operation.
  • items can be divided into “snacks” and “aquatic products”. , “Tools” and “daily necessities”; you can also subdivide the types of items.
  • the “snacks” category the items are divided into “drinks", “dried fruits” and “puffed snacks", and in “aquatic products”
  • the items are divided into “crayfish” and "hairy crabs” in the type.
  • the classification of the item types is not limited.
  • the outline information is a line drawn around the edge of the article, which represents the actual outline of the article.
  • the outline information of the article "potato chips” and "twist” is represented by a dotted line.
  • the position information indicates the position of the item.
  • the first angle of inclination in the position information indicates the degree of inclination of the item in the image information. Leaning to the left is a negative angle, and to the right is a positive angle.
  • the threshold of the first inclination angle The range is negative 180 degrees to positive 180 degrees. Take Figure 1B as an example.
  • the target image includes two items, namely "Potato Chips" and "Twist".
  • the position information further includes combination information or vertex coordinate information; wherein, the combination information includes center point coordinates and side length information. If the item is rectangular, the side length information output by the image recognition model includes the length of the short side and the length of the long side.
  • the vertex coordinate information includes the coordinates of the four vertices; if the item is square or diamond, the side length information includes the length of one side and the coordinates of the vertex.
  • the information includes the coordinates of the four vertices; if the item is a circle, the side length information includes the radius length, and the vertex coordinate information includes the coordinates of the center of the circle and the coordinates of any point on the circle; if the item is an ellipse, the side length information includes the short radius and the long radius
  • the vertex coordinate information includes the coordinates of four vertices; if the item is a triangle, the side length information includes the length of three sides, and the vertex coordinate information includes the coordinates of three vertices.
  • it can be determined according to the outline information of the article.
  • the image recognition model can obtain the type information, location information and contour information of the target object in the target image, as well as the reliability of the output result of the image recognition model, which can be used to predict the accuracy of the target object recognition. Compare with the set second preset threshold to judge the accuracy of this identification. If the reliability is greater than or equal to the second preset threshold, it means that the identification of the target item this time has high reliability; if it is reliable If the performance is less than the second preset threshold, it means that the reliability of the identification of the target item this time is low. At this time, an alarm signal or a prompt message can be issued to avoid errors in the identification of the target item and sorting errors.
  • the smallest rectangular frame is the smallest rectangle that completely covers the article, and the smallest rectangular frame of the target article can be determined by a moment estimation algorithm, and the second inclination angle of the smallest rectangular frame can be determined.
  • the second inclination angle obtained by the moment estimation algorithm is only the inclination angle of the smallest rectangular frame that completely covers the object, without directivity. Therefore, the second inclination angle determined by the smallest rectangular frame includes two values that are 180 degrees apart from each other. Taking FIG. 1C as an example, the second inclination angle of the smallest rectangular frame of the target item "potato chips" includes two values of minus 35 degrees and plus 145 degrees.
  • the second inclination angle obtained through the smallest rectangular frame has no directionality and includes two values that are 180 degrees apart from each other, the second inclination angle needs to be corrected by the first inclination angle.
  • determining the true inclination angle of the target object according to the first inclination angle and the second inclination angle includes: determining the first inclination angle according to the first inclination angle Two effective values of the tilt angle, and the effective value is determined as the true tilt angle of the target item.
  • the first inclination angle of the item "potato chips" in the image in FIG. 1B is minus 30 degrees
  • the second inclination angle of the item in the image in FIG. 1C includes two values of minus 35 degrees and plus 145 degrees.
  • the second inclination angle of minus 35 degrees and the first inclination angle of minus 30 degrees are closer in value. Therefore, the effective value of the second inclination angle is determined by the first inclination angle, and the effective value is used as the article The true tilt angle of the "potato chips”.
  • determining the true inclination angle of the target object according to the first inclination angle and the second inclination angle includes: determining the first inclination angle according to the first inclination angle Two effective values in the tilt angle; determine the true value of the target item according to the first tilt angle, the effective value, the preset weight coefficient of the first tilt angle, and the preset weight coefficient of the effective value slope.
  • the effective values in the first inclination angle and the second inclination angle can be respectively set with different weight values, and then the weight values of the first inclination angle and the first inclination angle are multiplied, and the effective values in the second inclination angle are compared with the second inclination angle.
  • the weight value of the effective value in the tilt angle is multiplied, and the result of multiplying the weight value of the first tilt angle and the first tilt angle and the weight of the effective value in the second tilt angle and the effective value in the second tilt angle.
  • the results of multiplying the values are summed, and the final calculated value obtained is used as the true tilt angle.
  • S140 Control a sorting device to sort the target item according to the type information, the position information, the contour information, and the true tilt angle of the target item.
  • controlling the sorting device to sort the target item also includes: controlling the sorting device to turn the target item 180 degrees. Since the barcode is usually located on the back of the target item, when the barcode information of the target item is recognized, the back of the target item is facing up.
  • the target item In addition to adjusting the inclination angle of the target item in the target image, the target item needs to be turned over. 180 degrees to keep the target item displayed in front. If the target image includes multiple target items, one sorting device can be controlled to sort multiple target items in turn, or multiple sorting devices can be controlled to sort multiple target items at the same time.
  • the sorting equipment may include a sorting manipulator or other types of sorting robots. In the embodiment of the present application, the type of the sorting equipment is not limited.
  • the lower edge of the target image can be used as the abscissa axis, that is, the x axis; the left edge can be used as the ordinate, that is, the y axis; the coordinate axis perpendicular to the x and y axes and pointing to the outside of the target image can be used as z axis; adjustment of the posture of the item, including tilt angle adjustment and flip angle adjustment, where the tilt angle adjustment is to rotate the target item in the plane composed of the x axis and the y axis, and the flip angle adjustment is along z In-situ reversal of the axis direction.
  • the technical solution provided by the embodiment of the application obtains the type information, position information and contour information of the target item through the recognition function of the image recognition model, and obtains the true tilt angle of the item in combination with the smallest rectangular frame, and then completes the sorting of the item Operation, while improving the recognition accuracy and efficiency of the items, it ensures that the items are sorted in the correct posture, and the sorting effect is improved.
  • FIG. 2 is a structural block diagram of a sorting device provided in the second embodiment of the present application.
  • the device includes: a graphic recognition execution module 210, a second tilt angle determination module 220, a true tilt angle calculation module 230, and a sorting execution module 240 .
  • the graphic recognition execution module 210 is configured to obtain a target image, and obtain the type information, position information, and contour information of the target item in the target image through the image recognition model; the position information includes the first tilt angle.
  • the second inclination angle determination module 220 is configured to determine the smallest rectangular frame of the target item according to the position information and the outline information, and determine the second inclination angle of the smallest rectangular frame.
  • the true tilt angle calculation module 230 is configured to determine the true tilt angle of the target object according to the first tilt angle and the second tilt angle.
  • the sorting execution module 240 is configured to control the sorting device to sort the target item according to the type information, the position information, the contour information, and the true tilt angle of the target item.
  • the technical solution provided by the embodiment of the application obtains the type information, position information and contour information of the target item through the recognition function of the image recognition model, and obtains the true tilt angle of the item in combination with the smallest rectangular frame, and then completes the sorting of the item Operation, while improving the recognition accuracy and efficiency of the items, it ensures that the items are sorted in the correct posture, and the sorting effect is improved.
  • the position information further includes combination information or vertex coordinate information; wherein, the combination information includes center point coordinates and side length information.
  • the image recognition model includes a deep convolutional neural network model.
  • the sorting device further includes: a training execution module configured to obtain training samples and train the deep convolutional neural network model to obtain the trained deep convolutional neural network Model; test execution module, set to pass the test sample, perform image recognition test on the trained deep convolutional neural network model, and obtain the reliability of the output result of the trained deep convolutional neural network model; reliability judgment module, set To determine whether the reliability is greater than or equal to the first preset threshold; if the reliability is greater than or equal to the first preset threshold, use the trained deep convolutional neural network model as the image recognition model; If the reliability is less than the first preset threshold, the deep convolutional neural network model will continue to be trained until the trained deep convolutional neural network model has a reliability greater than or equal to the test sample output result. Up to the first preset threshold.
  • a training execution module configured to obtain training samples and train the deep convolutional neural network model to obtain the trained deep convolutional neural network Model
  • test execution module set to pass the test sample, perform image recognition test on the trained deep convolutional neural network
  • the second inclination angle determining module 220 is configured to determine the smallest rectangular frame of the target item through a moment estimation algorithm according to the position information and the contour information, and Determine the second inclination angle of the smallest rectangular frame.
  • the true tilt angle calculation module 230 includes: a first true tilt angle calculation unit configured to determine the effective value of the second tilt angle according to the first tilt angle , And determine the effective value as the true tilt angle of the target item.
  • the true tilt angle calculation module 230 includes: a valid value determining unit configured to determine the valid value in the second tilt angle according to the first tilt angle; second The true tilt angle calculation unit is configured to determine the true value of the target item according to the first tilt angle, the effective value, the preset weight coefficient of the first tilt angle, and the preset weight coefficient of the effective value. slope.
  • the above-mentioned device can execute the sorting method provided by any embodiment of the present application, and is equipped with functional modules corresponding to the execution method.
  • functional modules corresponding to the execution method.
  • FIG. 3 is a schematic structural diagram of a device provided in Embodiment 3 of this application.
  • Figure 3 shows a block diagram of an exemplary device 12 suitable for implementing embodiments of the present application.
  • the device 12 shown in FIG. 3 is only an example, and should not bring any limitation to the function and scope of use of the embodiments of the present application.
  • the device 12 is represented in the form of a general-purpose computing device.
  • the components of the device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).
  • the bus 18 represents one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any bus structure among multiple bus structures.
  • these architectures include but are not limited to Industry Standard Architecture (ISA) bus, MicroChannel Architecture (MCA) bus, enhanced ISA bus, Video Electronics Standards Association (Video Electronics Standards Association) , VESA) local bus and Peripheral Component Interconnect (PCI) bus.
  • the device 12 includes a variety of computer system readable media. These media can be any available media that can be accessed by the device 12, including volatile and non-volatile media, removable and non-removable media.
  • the system memory 28 may include a computer system readable medium in the form of a volatile memory, such as a random access memory (RAM) 30 and/or a cache memory 32.
  • the device 12 may also include other removable/non-removable, volatile/nonvolatile computer system storage media.
  • the storage system 34 may be configured to read and write a non-removable, non-volatile magnetic medium (not shown in FIG. 3, usually referred to as a "hard drive").
  • a disk drive configured to read and write to a removable non-volatile disk (such as a "floppy disk") and a removable non-volatile optical disk (such as a compact disk read-only memory) can be provided.
  • each drive can be connected to the bus 18 through one or more data media interfaces.
  • the system memory 28 may include at least one program product, the program product having a set (for example, at least one) program modules, and these program modules are configured to perform the functions of multiple embodiments of the present application.
  • a program/utility tool 40 having a set of (at least one) program module 42 may be stored in, for example, the system memory 28.
  • Such program module 42 includes but is not limited to an operating system, one or more application programs, other program modules, and programs Data, each of these examples or some combination may include the realization of the network environment.
  • the program module 42 usually executes the functions and/or methods in the embodiments described in this application.
  • the device 12 may also communicate with one or more external devices 14 (such as keyboards, pointing devices, displays 24, etc.), and may also communicate with one or more devices that enable a user to interact with the device 12, and/or communicate with
  • the device 12 can communicate with any device (such as a network card, modem, etc.) that can communicate with one or more other computing devices. This communication can be performed through an input/output (Input/Output, I/O) interface 22.
  • the device 12 may also communicate with one or more networks (for example, a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) through the network adapter 20. As shown in FIG. 3, the network adapter 20 communicates with other modules of the device 12 through the bus 18.
  • LAN local area network
  • WAN wide area network
  • public network such as the Internet
  • the processing unit 16 executes a variety of functional applications and data processing by running programs stored in the system memory 28, for example, to implement the sorting method provided in the embodiments of the present application. That is: acquiring a target image, and acquiring the type information, position information, and contour information of the target item in the target image through the image recognition model; the position information includes the first tilt angle; according to the position information and the contour information , Determine the minimum rectangular frame of the target item, and determine the second inclination angle of the minimum rectangular frame; determine the true inclination angle of the target item according to the first inclination angle and the second inclination angle; The type information, the position information, the contour information, and the true inclination angle of the target item are controlled to sort the target item by the sorting device.
  • the fourth embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored.
  • the sorting method as described in any of the embodiments of the present application is implemented; the method includes: obtaining Target image, and obtain the type information, position information, and contour information of the target item in the target image through the image recognition model; the position information includes the first tilt angle; the position information and the contour information are used to determine the The minimum rectangular frame of the target item, and the second inclination angle of the minimum rectangular frame is determined; the true inclination angle of the target item is determined according to the first inclination angle and the second inclination angle; according to the target item
  • the type information, the position information, the contour information and the true tilt angle of the control sorting equipment are controlled to sort the target items.
  • the computer storage medium of the embodiment of the present application may adopt any combination of one or more computer-readable media.
  • the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above.
  • Examples of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, RAM, read-only memory (ROM), erasable memory Erasable Programmable Read-Only Memory (EPROM) or flash memory, optical fiber, portable compact CD-ROM, optical storage device, magnetic storage device, or any suitable combination of the above.
  • the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device.
  • the computer-readable signal medium may include a data signal propagated in baseband or as a part of a carrier wave, and computer-readable program code is carried therein. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium.
  • the computer-readable medium may send, propagate, or transmit the program for use by or in combination with the instruction execution system, apparatus, or device .
  • the program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the above.
  • any suitable medium including but not limited to wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the above.
  • the computer program code used to perform the operations of this application can be written in one or more programming languages or a combination thereof.
  • the programming languages include object-oriented programming languages—such as Java, Smalltalk, C++, and also conventional Procedural programming language-such as "C" language or similar programming language.
  • the program code can be executed entirely on the user's computer, partly on the user's computer, executed as an independent software package, partly on the user's computer and partly executed on a remote computer, or entirely executed on the remote computer or server.
  • the remote computer may be connected to the user's computer through any kind of network including LAN or WAN, or may be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Sorting Of Articles (AREA)

Abstract

一种分拣方法、装置、设备及存储介质，该方法包括：获取目标图像，并通过图像识别模型获取所述目标图像中目标物品的类型信息、位置信息和轮廓信息（S110）；确定所述目标物品的最小矩形框，并确定所述最小矩形框的第二倾斜角度（S120）；根据第二倾斜角度和所述位置信息中的第一倾斜角度，确定所述目标物品的真实倾斜角度（S130）；根据所述目标物品的类型信息、位置信息、轮廓信息和所述真实倾斜角度，控制分拣设备对所述目标物品进行分拣（S140）。

Description

分拣方法、装置、设备和存储介质
本申请要求在2019年12月28日提交中国专利局、申请号为201911383379.8的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及图像识别和机器人技术，例如涉及一种分拣方法、装置、设备和存储介质。
背景技术
随着科技的不断进步，机器人技术也得到迅速发展，分拣机器人作为机器人技术的一个重要分支，已被广泛应用于工业生产中。
相关技术中，分拣机器人通过获取的图像信息，采用模板匹配的方法，即将图形信息中出现的目标物品与模板进行匹配，确定物品的种类，进而通过分拣机器人的抓取将该物品分拣到相应位置，但这样的图像识别方法，对物品类型判断的准确性和效率较低，同时，抓取到该物品后，并不能保证物品以正确的姿态被分拣，例如，饮料瓶以瓶口倒置或者倒放的方式被抓取并摆放，严重影响分拣效果。
发明内容
本申请实施例提供了一种分拣方法、装置、设备和存储介质，以保证物品姿态正确的前提下，完成物品的识别和分拣。
本申请实施例提供了一种分拣方法，包括：
获取目标图像，并通过图像识别模型获取所述目标图像中目标物品的类型信息、位置信息和轮廓信息；所述位置信息包括第一倾斜角度；
根据所述位置信息和所述轮廓信息，确定所述目标物品的最小矩形框，并确定所述最小矩形框的第二倾斜角度；
根据所述第一倾斜角度以及所述第二倾斜角度，确定所述目标物品的真实倾斜角度；
根据所述目标物品的所述类型信息、所述位置信息、所述轮廓信息和所述真实倾斜角度，控制分拣设备对所述目标物品进行分拣。
本申请实施例提供了一种分拣装置，包括：
图形识别执行模块，设置为获取目标图像，并通过图像识别模型获取所述目标图像中目标物品的类型信息、位置信息和轮廓信息；所述位置信息包括第一倾斜角度；
第二倾斜角度确定模块，设置为根据所述位置信息和所述轮廓信息，确定所述目标物品的最小矩形框，并确定所述最小矩形框的第二倾斜角度；
真实倾斜角度计算模块，设置为根据所述第一倾斜角度以及所述第二倾斜角度，确定所述目标物品的真实倾斜角度；
分拣执行模块，设置为根据所述目标物品的所述类型信息、所述位置信息、所述轮廓信息和所述真实倾斜角度，控制分拣设备对所述目标物品进行分拣。
本申请实施例还提供了一种设备，所述设备包括：
一个或多个处理器；
存储装置，设置为存储一个或多个程序；
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现本申请任意实施例所述的分拣方法。
本申请实施例还提供了一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时实现本申请任意实施例所述的分拣方法。
附图说明
图1A是本申请实施例一提供的一种分拣方法的流程图；
图1B是本申请实施例一提供的目标物品的轮廓信息示意图；
图1C是本申请实施例一提供的目标物品的最小矩形框示意图；
图2是本申请实施例二提供的一种分拣方法的流程图；
图3是本申请实施例三提供的一种设备的结构框图。
具体实施方式
下面结合附图和实施例对本申请进行说明。可以理解的是，此处所描述的具体实施例仅用于解释本申请，而非对本申请的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与本申请相关的部分而非全部结构。
实施例一
图1A为本申请实施例一提供的一种分拣方法的流程图，本实施例可适用于 通过图像信息对物品进行分拣的情况，该方法可以由本申请实施例中的分拣装置来执行，该装置可以通过软件和/或硬件实现，并可以集成在终端设备中，例如，可以集成在分拣机械手中，该方法包括如下步骤：
S110、获取目标图像，并通过图像识别模型获取所述目标图像中目标物品的类型信息、位置信息和轮廓信息；所述位置信息包括第一倾斜角度。
目标图像来源于安装在固定位置或分拣设备(例如，分拣机械手)上的摄像头，通过图像识别模型对摄像头拍摄的图像信息进行图像识别操作，获取图像信息中包括的物品。图像中可以包括一个或多个物品。
图像识别模型是经过训练获取的，具有图像识别功能。可选的，在本申请实施例中，所述图像识别模型包括深度卷积神经网络模型。获取训练样本，并对深度卷积神经网络模型进行训练，以获取训练后的深度卷积神经网络模型；通过测试样本，对训练后的深度卷积神经网络模型进行图像识别测试，得到识别结果，并获取训练后的深度卷积神经网络模型输出结果的可靠性；判断所述可靠性是否大于或等于第一预设阈值；若所述可靠性大于或等于所述第一预设阈值，则将训练后的深度卷积神经网络模型作为图像识别模型；若所述可靠性小于所述第一预设阈值，则对所述深度卷积神经网络模型继续进行训练，直至训练后的深度卷积神经网络模型对所述测试样本输出结果的可靠性大于或等于所述第一预设阈值为止。训练样本是经过标注的图像，包括对物品的轮廓信息、类型信息以及位置信息的标注，还包括对轮廓信息和位置信息的标注点是否被遮挡的标注；对于部分区域被遮挡的物品，训练样本中同样标注了完整的轮廓信息的标注，以使深度卷积神经网络模型具备联想记忆功能；经过训练样本训练后的深度卷积神经网络模型，还需要经过测试样本的图像识别测试，以判断获取到的深度卷积神经网络模型是否符合要求。卷积神经网络模型在进行训练样本训练的情况下，已经完成了信息分解、特征提取和信息重组的过程，获得模型参数值，在获取到新的图像(例如，测试样本)时，基于学习到的模型参数训练样本，在前向推理的过程中，自动实现信息分解、特征提取和信息重组的过程，获得识别结果，即训练样本中物品的位置信息、轮廓信息和类型信息，同时利用既定的概率公式，计算出该结果的可靠性，并将该可靠性的数值输出。可靠性，即识别可靠性，就是对目标图像识别准确性的预测(例如，可靠性为70％)，也即通过深度卷积神经网络模型的推理预算计算出来的物品信息与真实情况的匹配程度。第一预设阈值可以根据需要设定，当可靠性大于或等于第一预设阈值时，表示该深度卷积神经网络模型的准确性较高，符合要求；当可靠性小于第一预设阈值时，表示该深度卷积神经网络模型的准确性较低，不符合要求。
物品信息包括类型信息、位置信息和轮廓信息。类型信息，也即物品的分类，根据物品的所述类型，在进行物品分拣操作时，将同类型的物品放置于相同区域内，例如，可以将物品分为“零食”、“水产品”、“工具”和“生活用品”；还可以对物品的类型进行细划分，例如，在“零食”类型中将物品分为“饮料”、“干果”和“膨化零食”，在“水产品”类型中将物品分为“小龙虾”和“大闸蟹”。可选的，在本申请实施例中，对物品类型的划分不作限定。轮廓信息，是围绕物品的边缘描点的连线，表示了物品的实际轮廓，例如，图1B中用虚线表示了物品“薯片”和“麻花”的轮廓信息。位置信息，表示了物品所处的位置，位置信息中的第一倾斜角度表示了物品在图像信息中的倾斜程度，向左倾斜为负角度，向右倾斜为正角度，第一倾斜角度的阈值范围为负180度至正180度，以图1B为例，目标图像中包括了两种物品，分别为“薯片”和“麻花”，其中，“薯片”在图像中倾斜了负30度，“麻花”在图像中倾斜了正120度。可选的，在本申请实施例中，所述位置信息还包括组合信息或顶点坐标信息；其中，所述组合信息包括中心点坐标和边长信息。如果物品为矩形，那么图像识别模型输出的边长信息包括短边长度和长边长度，顶点坐标信息包括四个顶点的坐标；如果物品为正方形或菱形，边长信息包括单边长度，顶点坐标信息包括四个顶点的坐标；如果物品为圆形，边长信息包括半径长度，顶点坐标信息包括圆心坐标和圆上任一点的坐标；如果物品为椭圆形，边长信息包括短半径长度和长半径长度，顶点坐标信息包括四个顶点的坐标；如果物品为三角形，边长信息包括三个边的长度，顶点坐标信息包括三个顶点的坐标。对于物品的形状判断，可以根据物品的轮廓信息确定。
通过图像识别模型可以获取到目标图像中目标物品的类型信息、位置信息和轮廓信息外，还可以获取到图像识别模型输出结果的可靠性，用来表示对目标物品识别准确性的预测，可以通过与设定的第二预设阈值进行比较，来判断本次识别的准确性，如果可靠性大于或等于第二预设阈值，则表示本次对目标物品的识别，可靠性较高；如果可靠性小于第二预设阈值，则表示本次对目标物品的识别，可靠性较低，此时可以通过发出报警信号或提示消息，避免对目标物品的识别出错，进而出现分拣错误。
S120、根据所述位置信息和所述轮廓信息，确定所述目标物品的最小矩形框，并确定所述最小矩形框的第二倾斜角度。
最小矩形框是完整覆盖物品的最小矩形，可以通过矩估计(Moment Estimation)算法确定目标物品的最小矩形框，并确定所述最小矩形框的第二倾斜角度。但通过矩估计算法获取的第二倾斜角度，只是完全覆盖物体的最小矩形框的倾斜角度，不带有方向性，因此通过最小矩形框确定的第二倾斜角度包括彼此相差180度的两个数值，以图1C为例，目标物品“薯片”的最小矩形框 的第二倾斜角度包括负35度和正145度两个数值。
S130、根据所述第一倾斜角度以及所述第二倾斜角度，确定所述目标物品的真实倾斜角度。
由于通过最小矩形框获取的第二倾斜角度，不带有方向性，包括了彼此相差180度的两个数值，因此，需要通过第一倾斜角度对第二倾斜角度进行修正。
可选的，在本申请实施例中，根据所述第一倾斜角度以及所述第二倾斜角度，确定所述目标物品的真实倾斜角度，包括：根据所述第一倾斜角度，确定所述第二倾斜角度中的有效数值，并将所述有效数值确定为所述目标物品的真实倾斜角度。以上述技术方案为例，图1B中物品“薯片”在图像中的第一倾斜角度为负30度，图1C中该物品在图像中第二倾斜角度包括负35度和正145度两个数值，显然第二倾斜角度中负35度与第一倾斜角度负30度在数值上更接近，因此，通过第一倾斜角度确定了第二倾斜角度中的有效数值，并将该有效数值作为了物品“薯片”的真实倾斜角度。
可选的，在本申请实施例中，根据所述第一倾斜角度以及所述第二倾斜角度，确定所述目标物品的真实倾斜角度，包括：根据所述第一倾斜角度，确定所述第二倾斜角度中的有效数值；根据所述第一倾斜角度、所述有效数值、所述第一倾斜角度的预设权重系数以及所述有效数值的预设权重系数，确定所述目标物品的真实倾斜角度。第一倾斜角度和第二倾斜角度中的有效数值可以分别设定不同的权重值，再将第一倾斜角度与第一倾斜角度的权重值相乘和第二倾斜角度中的有效数值与第二倾斜角度中的有效数值的权重值相乘，再对第一倾斜角度与第一倾斜角度的权重值相乘的结果和第二倾斜角度中的有效数值与第二倾斜角度中的有效数值的权重值相乘的结果求和，获取的最终计算数值作为真实倾斜角度。
S140、根据所述目标物品的所述类型信息、所述位置信息、所述轮廓信息和所述真实倾斜角度，控制分拣设备对所述目标物品进行分拣。
根据目标物品的位置信息，对目标物品进行定位，再根据轮廓信息调整分拣设备的分拣动作，例如，调整分拣机械手的张开程度，在抓取到目标物品后，根据物品的真实倾斜角度，调整物品姿态，进而根据类型信息将目标物品分拣至指定区域，并保持物品的姿态正确。如果在目标物品的轮廓信息内识别到了条形码信息，则控制分拣设备对所述目标物品进行分拣，还包括：控制分拣设备将所述目标物品翻转180度。由于条形码通常位于目标物品的背面，因此，当识别到目标物品的条形码信息时，此时目标物品的背面朝上，除了需要调整目标物品在目标图像中的倾斜角度外，还需要将目标物品翻转180度，以使目标物品保持正面展示状态。如果目标图像中包括多个目标物品，可以控制一台 分拣设备依次分拣多个目标物品，也可以控制住多台分拣设备同时分拣多个目标物品。分拣设备可以包括分拣机械手，也可以包括其他类型的分拣机器人，在本申请实施例中，对分拣设备的类型不作限定。在本申请实施例中，可以将目标图像的下边沿作为横坐标轴，即x轴；左边沿作为纵坐标，即y轴；垂直于x轴和y轴，并指向目标图像外侧的坐标轴作为z轴；对物品的姿态调整，包括倾斜角度调整和翻转角度调整，其中，对倾斜角度的调整，即在x轴和y轴组成的平面中旋转目标物品，对翻转角度的调整，即沿z轴方向进行的原地翻转。
本申请实施例提供的技术方案，通过图像识别模型的识别作用，获取到目标物品的类型信息、位置信息和轮廓信息，并结合最小矩形框获取到物品的真实倾斜角度，进而完成物品的分拣操作，在提高了物品的识别准确性和识别效率的同时，保证了物品以正确姿态被分拣，提升了分拣效果。
实施例二
图2是本申请实施例二所提供的一种分拣装置的结构框图，该装置包括：图形识别执行模块210、第二倾斜角度确定模块220、真实倾斜角度计算模块230和分拣执行模块240。
图形识别执行模块210，设置为获取目标图像，并通过图像识别模型获取所述目标图像中目标物品的类型信息、位置信息和轮廓信息；所述位置信息包括第一倾斜角度。
第二倾斜角度确定模块220，设置为根据所述位置信息和所述轮廓信息，确定所述目标物品的最小矩形框，并确定所述最小矩形框的第二倾斜角度。
真实倾斜角度计算模块230，设置为根据所述第一倾斜角度以及所述第二倾斜角度，确定所述目标物品的真实倾斜角度。
分拣执行模块240，设置为根据所述目标物品的所述类型信息、所述位置信息、所述轮廓信息和所述真实倾斜角度，控制分拣设备对所述目标物品进行分拣。
本申请实施例提供的技术方案，通过图像识别模型的识别作用，获取到目标物品的类型信息、位置信息和轮廓信息，并结合最小矩形框获取到物品的真实倾斜角度，进而完成物品的分拣操作，在提高了物品的识别准确性和识别效率的同时，保证了物品以正确姿态被分拣，提升了分拣效果。
可选的，在上述技术方案的基础上，所述位置信息还包括组合信息或顶点坐标信息；其中，所述组合信息包括中心点坐标和边长信息。
可选的，在上述技术方案的基础上，所述图像识别模型包括深度卷积神经 网络模型。
可选的，在上述技术方案的基础上，分拣装置，还包括：训练执行模块，设置为获取训练样本，并对深度卷积神经网络模型进行训练，以获取训练后的深度卷积神经网络模型；测试执行模块，设置为通过测试样本，对训练后的深度卷积神经网络模型进行图像识别测试，并获取训练后的深度卷积神经网络模型输出结果的可靠性；可靠性判断模块，设置为判断所述可靠性是否大于或等于第一预设阈值；若所述可靠性大于或等于所述第一预设阈值，则将训练后的深度卷积神经网络模型作为图像识别模型；若所述可靠性小于所述第一预设阈值，则对所述深度卷积神经网络模型继续进行训练，直至训练后的深度卷积神经网络模型对所述测试样本输出结果的可靠性大于或等于所述第一预设阈值为止。
可选的，在上述技术方案的基础上，第二倾斜角度确定模块220，是设置为根据所述位置信息和所述轮廓信息，通过矩估计算法，确定所述目标物品的最小矩形框，并确定所述最小矩形框的第二倾斜角度。
可选的，在上述技术方案的基础上，真实倾斜角度计算模块230，包括：第一真实倾斜角度计算单元，设置为根据所述第一倾斜角度，确定所述第二倾斜角度中的有效数值，并将所述有效数值确定为所述目标物品的真实倾斜角度。
可选的，在上述技术方案的基础上，真实倾斜角度计算模块230，包括：有效数值确定单元，设置为根据所述第一倾斜角度，确定所述第二倾斜角度中的有效数值；第二真实倾斜角度计算单元，设置为根据所述第一倾斜角度、所述有效数值、所述第一倾斜角度的预设权重系数以及所述有效数值的预设权重系数，确定所述目标物品的真实倾斜角度。
上述装置可执行本申请任意实施例所提供的分拣方法，具备执行方法相应的功能模块。未在本实施例中描述的技术细节，可参见本申请任意实施例提供的方法。
实施例三
图3为本申请实施例三提供的一种设备的结构示意图。图3示出了适于用来实现本申请实施方式的示例性设备12的框图。图3显示的设备12仅仅是一个示例，不应对本申请实施例的功能和使用范围带来任何限制。
如图3所示，设备12以通用计算设备的形式表现。设备12的组件可以包括但不限于：一个或者多个处理器或者处理单元16，系统存储器28，连接不同系统组件(包括系统存储器28和处理单元16)的总线18。
总线18表示几类总线结构中的一种或多种，包括存储器总线或者存储器控 制器，外围总线，图形加速端口，处理器或者使用多种总线结构中的任意总线结构的局域总线。举例来说，这些体系结构包括但不限于工业标准体系结构(Industry Standard Architecture，ISA)总线，微通道体系结构(MicroChannel Architecture，MCA)总线，增强型ISA总线、视频电子标准协会(Video Electronics Standards Association，VESA)局域总线以及外围组件互连(Peripheral Component Interconnect，PCI)总线。
设备12包括多种计算机系统可读介质。这些介质可以是任何能够被设备12访问的可用介质，包括易失性和非易失性介质，可移动的和不可移动的介质。
系统存储器28可以包括易失性存储器形式的计算机系统可读介质，例如随机存取存储器(Random Access Memory，RAM)30和/或高速缓存存储器32。设备12还可以包括其它可移动/不可移动的、易失性/非易失性计算机系统存储介质。仅作为举例，存储系统34可以设置为读写不可移动的、非易失性磁介质(图3未显示，通常称为“硬盘驱动器”)。尽管图3中未示出，可以提供设置为对可移动非易失性磁盘(例如“软盘”)进行读写的磁盘驱动器，以及对可移动非易失性光盘(例如磁盘只读存储器(Compact Disc Read-Only Memory，CD-ROM),数字视盘只读存储器(Digital Video Disc Read-Only Memory，DVD-ROM)或者其它光介质)进行读写的光盘驱动器。在这些情况下，每个驱动器可以通过一个或者多个数据介质接口与总线18相连。系统存储器28可以包括至少一个程序产品，该程序产品具有一组(例如至少一个)程序模块，这些程序模块被配置以执行本申请多个实施例的功能。
具有一组(至少一个)程序模块42的程序/实用工具40，可以存储在例如系统存储器28中，这样的程序模块42包括但不限于操作系统、一个或者多个应用程序、其它程序模块以及程序数据，这些示例中的每一个或某种组合中可能包括网络环境的实现。程序模块42通常执行本申请所描述的实施例中的功能和/或方法。
设备12也可以与一个或多个外部设备14(例如键盘、指向设备、显示器24等)通信，还可与一个或者多个使得用户能与该设备12交互的设备通信，和/或与使得该设备12能与一个或多个其它计算设备进行通信的任何设备(例如网卡，调制解调器等等)通信。这种通信可以通过输入/输出(Input/Output，I/O)接口22进行。并且，设备12还可以通过网络适配器20与一个或者多个网络(例如局域网(Local Area Network，LAN)，广域网(Wide Area Network，WAN)和/或公共网络，例如因特网)通信。如图3所示，网络适配器20通过总线18与设备12的其它模块通信。应当明白，尽管图3中未示出，可以结合设备12使用其它硬件和/或软件模块，包括但不限于：微代码、设备驱动器、冗余处理 单元、外部磁盘驱动阵列、独立磁盘冗余阵列(Redundant Arrays of Independent Disks，RAID)系统、磁带驱动器以及数据备份存储系统等。
处理单元16通过运行存储在系统存储器28中的程序，从而执行多种功能应用以及数据处理，例如实现本申请实施例所提供的分拣方法。也即：获取目标图像，并通过图像识别模型获取所述目标图像中目标物品的类型信息、位置信息和轮廓信息；所述位置信息包括第一倾斜角度；根据所述位置信息和所述轮廓信息，确定所述目标物品的最小矩形框，并确定所述最小矩形框的第二倾斜角度；根据所述第一倾斜角度以及所述第二倾斜角度，确定所述目标物品的真实倾斜角度；根据所述目标物品的所述类型信息、所述位置信息、所述轮廓信息和所述真实倾斜角度，控制分拣设备对所述目标物品进行分拣。
实施例四
本申请实施例四还提供了一种计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现如本申请任意实施例所述的分拣方法；该方法包括：获取目标图像，并通过图像识别模型获取所述目标图像中目标物品的类型信息、位置信息和轮廓信息；所述位置信息包括第一倾斜角度；根据所述位置信息和所述轮廓信息，确定所述目标物品的最小矩形框，并确定所述最小矩形框的第二倾斜角度；根据所述第一倾斜角度以及所述第二倾斜角度，确定所述目标物品的真实倾斜角度；根据所述目标物品的所述类型信息、所述位置信息、所述轮廓信息和所述真实倾斜角度，控制分拣设备对所述目标物品进行分拣。
本申请实施例的计算机存储介质，可以采用一个或多个计算机可读的介质的任意组合。计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、RAM、只读存储器(Read-Only Memory，ROM)、可擦式可编程只读存储器(Erasable Programmable Read-Only Memory，EPROM)或闪存、光纤、便携式紧凑CD-ROM、光存储器件、磁存储器件、或者上述的任意合适的组合。在本文件中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。
计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算 机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。
计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括——但不限于无线、电线、光缆、射频(Radio Frequency，RF)等，或者上述的任意合适的组合。
可以以一种或多种程序设计语言或其组合来编写用于执行本申请操作的计算机程序代码，所述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括LAN或WAN—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。

Claims (10)

  1. 一种分拣方法，包括：
    获取目标图像，并通过图像识别模型获取所述目标图像中目标物品的类型信息、位置信息和轮廓信息；所述位置信息包括第一倾斜角度；
    根据所述位置信息和所述轮廓信息，确定所述目标物品的最小矩形框，并确定所述最小矩形框的第二倾斜角度；
    根据所述第一倾斜角度以及所述第二倾斜角度，确定所述目标物品的真实倾斜角度；
    根据所述目标物品的所述类型信息、所述位置信息、所述轮廓信息和所述真实倾斜角度，控制分拣设备对所述目标物品进行分拣。
  2. 根据权利要求1所述的方法，其中，所述位置信息还包括组合信息或顶点坐标信息；所述组合信息包括中心点坐标和边长信息。
  3. 根据权利要求1所述的方法，其中，所述图像识别模型包括深度卷积神经网络模型。
  4. 根据权利要求3所述的方法，在获取目标图像前，还包括：
    获取训练样本，并对深度卷积神经网络模型进行训练，以获取训练后的深度卷积神经网络模型；
    通过测试样本，对所述训练后的深度卷积神经网络模型进行图像识别测试，并获取所述训练后的深度卷积神经网络模型输出结果的可靠性；
    判断所述可靠性是否大于或等于第一预设阈值；
    响应于所述可靠性大于或等于所述第一预设阈值的判断结果，将所述训练后的深度卷积神经网络模型作为图像识别模型；
    响应于所述可靠性小于所述第一预设阈值的判断结果，对所述深度卷积神经网络模型继续进行训练，直至所述训练后的深度卷积神经网络模型对所述测试样本输出结果的可靠性大于或等于所述第一预设阈值为止。
  5. 根据权利要求1所述的方法，其中，根据所述位置信息和所述轮廓信息，确定所述目标物品的最小矩形框，并确定所述最小矩形框的第二倾斜角度，包括：
    根据所述位置信息和所述轮廓信息，通过矩估计算法，确定所述目标物品的最小矩形框，并确定所述最小矩形框的第二倾斜角度。
  6. 根据权利要求1所述的方法，其中，所述第二倾斜角度包括相差180°的两个数值；
    根据所述第一倾斜角度以及所述第二倾斜角度，确定所述目标物品的真实倾斜角度，包括：
    根据所述第一倾斜角度，确定所述第二倾斜角度中的有效数值，并将所述有效数值确定为所述目标物品的真实倾斜角度。
  7. 根据权利要求1所述的方法，其中，所述第二倾斜角度包括相差180°的两个数值；
    根据所述第一倾斜角度以及所述第二倾斜角度，确定所述目标物品的真实倾斜角度，包括：
    根据所述第一倾斜角度，确定所述第二倾斜角度中的有效数值；
    根据所述第一倾斜角度、所述有效数值、所述第一倾斜角度的预设权重系数以及所述有效数值的预设权重系数，确定所述目标物品的真实倾斜角度。
  8. 一种分拣装置，包括：
    图形识别执行模块，设置为获取目标图像，并通过图像识别模型获取所述目标图像中目标物品的类型信息、位置信息和轮廓信息；所述位置信息包括第一倾斜角度；
    第二倾斜角度确定模块，设置为根据所述位置信息和所述轮廓信息，确定所述目标物品的最小矩形框，并确定所述最小矩形框的第二倾斜角度；
    真实倾斜角度计算模块，设置为根据所述第一倾斜角度以及所述第二倾斜角度，确定所述目标物品的真实倾斜角度；
    分拣执行模块，设置为根据所述目标物品的所述类型信息、所述位置信息、所述轮廓信息和所述真实倾斜角度，控制分拣设备对所述目标物品进行分拣。
  9. 一种设备，包括：
    至少一个处理器；
    存储装置，设置为存储至少一个程序，
    当所述至少一个程序被所述至少一个处理器执行，使得所述至少一个处理器实现如权利要求1-7中任一所述的分拣方法。
  10. 一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行如权利要求1-7中任一所述的分拣方法。
PCT/CN2020/137476 2019-12-28 2020-12-18 分拣方法、装置、设备和存储介质 WO2021129527A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911383379.8 2019-12-28
CN201911383379.8A CN111144426B (zh) 2019-12-28 2019-12-28 一种分拣方法、装置、设备和存储介质

Publications (1)

Publication Number Publication Date
WO2021129527A1 true WO2021129527A1 (zh) 2021-07-01

Family

ID=70521446

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/137476 WO2021129527A1 (zh) 2019-12-28 2020-12-18 分拣方法、装置、设备和存储介质

Country Status (2)

Country Link
CN (1) CN111144426B (zh)
WO (1) WO2021129527A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115488876A (zh) * 2022-06-22 2022-12-20 湖北商贸学院 一种基于机器视觉的机器人分拣方法和装置

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111144426B (zh) * 2019-12-28 2023-05-30 广东拓斯达科技股份有限公司 一种分拣方法、装置、设备和存储介质
CN112700454B (zh) * 2020-12-28 2024-05-14 北京达佳互联信息技术有限公司 图像裁剪方法、装置、电子设备及存储介质
CN112784875B (zh) * 2020-12-30 2022-05-17 深兰智能科技(上海)有限公司 机械手的控制方法及相关装置
CN113379681B (zh) * 2021-05-20 2022-11-04 深圳技术大学 Led芯片倾斜角度的获取方法、系统、电子装置及存储介质
CN113547520A (zh) * 2021-07-21 2021-10-26 广东拓斯达科技股份有限公司 一种机械手移动对位方法、装置、系统及存储介质
CN114005183B (zh) * 2021-12-31 2023-07-18 北京瑞莱智慧科技有限公司 动作识别方法、装置、设备及存储介质
CN114570674A (zh) * 2022-01-19 2022-06-03 福建南方路面机械股份有限公司 一种基于高度传感器的自动分拣方法、装置及可读介质
CN115258508A (zh) * 2022-07-21 2022-11-01 京东科技控股股份有限公司 物品拣选方法及装置、计算机可读存储介质
CN115690132A (zh) * 2022-10-20 2023-02-03 北京霍里思特科技有限公司 一种图像处理方法和系统
CN116309442B (zh) * 2023-03-13 2023-10-24 北京百度网讯科技有限公司 挑拣信息的确定方法及目标对象的挑拣方法
CN116899915B (zh) * 2023-07-19 2024-01-09 华至云链科技(苏州)有限公司 设备数据实时同步方法及装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015089589A (ja) * 2013-11-05 2015-05-11 ファナック株式会社 バラ積みされた物品をロボットで取出す装置及び方法
CN104781848A (zh) * 2012-10-09 2015-07-15 Sk电信有限公司 用于估计单体的倾斜度的图像监测装置及其方法
CN107804514A (zh) * 2017-10-23 2018-03-16 江苏艾萨克机器人股份有限公司 一种基于图像识别的牙刷分拣方法
CN109688885A (zh) * 2016-08-14 2019-04-26 生活机器人学股份有限公司 餐具清洁系统及机器人装置
CN109916826A (zh) * 2019-02-20 2019-06-21 福建南方路面机械有限公司 一种基于高光谱检测的固废在线识别系统及识别方法
CN111144426A (zh) * 2019-12-28 2020-05-12 广东拓斯达科技股份有限公司 一种分拣方法、装置、设备和存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6037125B2 (ja) * 2013-02-18 2016-11-30 株式会社サタケ 光学式粒状物選別機
KR102242948B1 (ko) * 2016-09-23 2021-04-21 가부시키가이샤 아-스 테크니카 선별장치 및 선별방법
CN107895173B (zh) * 2017-11-06 2021-08-17 国网重庆市电力公司电力科学研究院 标注图像目标的方法、装置、设备及可读存储介质
CN109225941B (zh) * 2018-10-10 2021-03-19 常熟理工学院 内螺纹攻丝情况自动检测与分拣系统及方法
CN110222703B (zh) * 2019-06-12 2023-07-11 江苏通佑视觉科技有限公司 图像轮廓识别方法、装置、设备和介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104781848A (zh) * 2012-10-09 2015-07-15 Sk电信有限公司 用于估计单体的倾斜度的图像监测装置及其方法
JP2015089589A (ja) * 2013-11-05 2015-05-11 ファナック株式会社 バラ積みされた物品をロボットで取出す装置及び方法
CN109688885A (zh) * 2016-08-14 2019-04-26 生活机器人学股份有限公司 餐具清洁系统及机器人装置
CN107804514A (zh) * 2017-10-23 2018-03-16 江苏艾萨克机器人股份有限公司 一种基于图像识别的牙刷分拣方法
CN109916826A (zh) * 2019-02-20 2019-06-21 福建南方路面机械有限公司 一种基于高光谱检测的固废在线识别系统及识别方法
CN111144426A (zh) * 2019-12-28 2020-05-12 广东拓斯达科技股份有限公司 一种分拣方法、装置、设备和存储介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115488876A (zh) * 2022-06-22 2022-12-20 湖北商贸学院 一种基于机器视觉的机器人分拣方法和装置

Also Published As

Publication number Publication date
CN111144426B (zh) 2023-05-30
CN111144426A (zh) 2020-05-12

Similar Documents

Publication Publication Date Title
WO2021129527A1 (zh) 分拣方法、装置、设备和存储介质
WO2021129528A1 (zh) 分拣方法、装置、设备和存储介质
US11878433B2 (en) Method for detecting grasping position of robot in grasping object
EP3712805A1 (en) Gesture recognition method, device, electronic device, and storage medium
WO2020107833A1 (zh) 基于骨架的行为检测方法、终端设备及计算机存储介质
WO2020078064A1 (zh) 基于地面纹理图像的导航方法、装置、设备及存储介质
WO2018121737A1 (zh) 关键点预测、网络训练及图像处理方法和装置、电子设备
WO2021103868A1 (zh) 一种行人结构化方法、装置、设备和存储介质
US11132575B2 (en) Combinatorial shape regression for face alignment in images
CN112016638B (zh) 一种钢筋簇的识别方法、装置、设备及存储介质
US11361587B2 (en) Age recognition method, storage medium and electronic device
CN110222703B (zh) 图像轮廓识别方法、装置、设备和介质
US20220414910A1 (en) Scene contour recognition method and apparatus, computer-readable medium, and electronic device
CN111368878B (zh) 一种基于ssd目标检测的优化方法、计算机设备和介质
WO2021179485A1 (zh) 图像矫正处理方法、装置、存储介质及计算机设备
WO2022199360A1 (zh) 运动物体的定位方法、装置、电子设备及存储介质
CN114140418A (zh) 一种基于rgb图像和深度图像的七自由度抓取姿势检测方法
CN110598647B (zh) 一种基于图像识别的头部姿态识别方法
CN114332927A (zh) 课堂举手行为检测方法、系统、计算机设备和存储介质
CN111985432B (zh) 一种基于贝叶斯定理和自适应权值调整的多模态数据融合方法
WO2024012367A1 (zh) 视觉目标跟踪方法、装置、设备以及存储介质
CN117372928A (zh) 一种视频目标检测方法、装置及相关设备
US10013602B2 (en) Feature vector extraction device based on cell positioning
Liu et al. Mos: a low latency and lightweight framework for face detection, landmark localization, and head pose estimation
CN111240195A (zh) 基于机器视觉的自动控制模型训练、目标物回收方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20907589

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20907589

Country of ref document: EP

Kind code of ref document: A1