Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
FIG. 1 illustrates an exemplary system architecture suitable for use in implementing embodiments of the present application.
As shown in fig. 1, the system architecture may include a vision system 101, a transfer robot 102. The vision system 101 and the transfer robot 102 may communicate data using TCP/IP communication protocols.
The cameras of the vision system 101 can shoot and collect images towards the container stacking area, and the vision system 101 can determine containers located at corners in the container stacking area as candidate containers based on the collected images, and further can determine target containers needing to be moved away from the container stacking area. The vision system 101 may transmit the determined position information of the target container, and a number indicating the type of the target container to the transfer robot 102 in an IP message.
The control system of the transfer robot 102 may control the transfer robot to position the target container using the determined position information of the target container to be moved away from the container stacking area, and control the gripping member of the transfer robot to grip the target container. The control system of the transfer robot 102 may determine the type of the target container according to the number indicating the type of the target container, and may further determine the transfer direction of the target container according to the type of the target container, and the control system of the transfer robot 102 may control the clamping member to move in the determined transfer direction, so as to transfer the target container out of the stacking area.
During each transportation process, a target container located at a corner can be determined by the vision system 101, and the transportation robot 102 transports the target container located at the corner determined by the vision system 101 to a container stacking area.
Referring to fig. 2, a flow of one embodiment of a container handling method according to the present application is shown. The method comprises the following steps:
step 201, obtaining position information of containers in the container stacking area, and determining candidate containers in the container stacking area based on the position information of the containers in the container stacking area.
In this embodiment, the camera can be used to face the container stacking area, and the angle of the camera is preset, so that the candidate containers at least partially located at the corners of the container stacking area are contained in the image collected by the camera.
In this embodiment, the containers located in the container stacking area may be detected from the image collected by the camera, and the position information of the containers in the detected container stacking area may be acquired.
For example, the image captured by the camera may be detected by a detection method such as edge detection using a Canny operator or a susan (small uniform segmentation sampling nucleus) operator, and a plurality of containers in the stacking area may be detected. Meanwhile, the coordinates of the detected corner points of the containers can be acquired, and then, the coordinates of the center point of each container can be further calculated. The position information of the container may include: coordinates of corner points, coordinates of center points.
In this embodiment, after the plurality of containers are detected, it is ensured in advance that the captured image includes at least one container candidate located at a corner of the container stacking area, and therefore the plurality of containers detected include at least one container candidate. The candidate containers in all the detected containers can be determined based on the position information of each detected container, namely, the candidate containers in the container stacking area are determined. The number of candidate containers in the container stacking area can be multiple.
In this embodiment, the candidate container satisfies candidate conditions, and the candidate conditions include: for each preset direction set in one preset direction set, no other adjacent container exists in each direction in the preset direction set. A container may be a candidate container when it has no other containers adjacent to it in each of a set of predetermined orientations. The candidate containers of other containers in the stacking area, all of which are adjacent in the preset direction set, can be determined according to the position information of each detected container.
For example, only one direction is included in the preset direction set. All containers are stacked on the ground, and no other container is stacked above each container. One set of preset directions contains the positive direction of the X-axis in a coordinate system established on a plane parallel to the ground, and the other set of preset directions contains the negative direction of the X-axis in a coordinate system established on a plane parallel to the ground. The X-axis in a coordinate system established on a plane parallel to the ground is parallel to the horizontal direction. When one container has no other container in the positive direction of the X axis or the negative direction of the X axis, the container is a candidate container. The position information of each container can be used to detect that there is no container adjacent to the other container in the positive direction or negative direction of the X axis.
In some alternative implementations of this embodiment, all containers within the container stacking area may be stacked within the stacking area in a stacked manner. A plurality of stacked containers form a stack of containers. The coordinate system may be established in a plane parallel to the horizontal plane. For example, a coordinate system in which the X-axis is parallel to the horizontal direction is established in a plane parallel to the horizontal plane. A set of predetermined directions includes one of: the positive direction of the X-axis and the positive direction of the Y-axis in the established coordinate system, the positive direction of the X-axis and the negative direction of the Y-axis in the established coordinate system, the negative direction of the X-axis and the negative direction of the Y-axis in the established coordinate system, and the negative direction of the X-axis and the positive direction of the Y-axis in the established coordinate system.
When a container satisfies that there are no other containers adjacent to each direction in a preset direction set, the container can be used as a candidate container. When one container has no adjacent container in the positive directions of the X axis and the Y axis, the container can be used as a candidate container. When one container has no adjacent container in the negative direction of the X axis and the positive direction of the Y axis, the container can be used as a candidate container. When one container has no adjacent container in the negative direction of the X axis and the negative direction of the Y axis, the container can be used as a candidate container. When one container has no adjacent container in the negative direction of the X axis and the positive direction of the Y axis, the container can be used as a candidate container.
In step 202, a target container in the candidate containers is determined, and a carrying direction of the target container is determined.
In this embodiment, after all the candidate containers are determined, the target container in all the candidate containers may be determined. One container can be selected from all the candidate containers as a target container.
For example, the container candidate closest to the gripping member of the transfer robot may be selected as the target container based on the coordinates of the center point of the container candidate and the coordinates of the gripping member of the transfer robot in the position information of each container candidate.
In some optional implementations of the present embodiment, when determining the target container among all the candidate containers, the candidate container with the smallest error in the position information among the candidate containers may be taken as the target container. For example, the position information of the candidate container includes coordinates of a center point of the candidate container determined from coordinates of corner points of the candidate container. The position information of the container is determined by edge detection by using a Canny operator, a susan (small uniform Segment optimizing) operator and the like, and the position information of each candidate container has an error. The candidate container whose error of the coordinates of the center point in the position information is the smallest may be taken as the target container.
In this embodiment, after the target container is determined, the carrying direction of the target container may be further determined.
For example, each preset direction set includes only one direction. All containers are stacked on the ground, and no other container is stacked above each container. One set of predetermined directions includes a positive direction of an X-axis of a coordinate system established on a plane parallel to the ground, and the other set of predetermined directions includes a negative direction of the X-axis in the coordinate system established on the plane parallel to the ground. The X-axis is parallel to the horizontal direction. When one container has no other container in the positive direction of the X axis or the negative direction of the X axis, the container can be used as a candidate container. When one container has no adjacent other container in the positive direction of the X axis, the container may be conveyed in the positive direction of the X axis. When one container has no adjacent other container in the negative X-axis direction, the container may be transported in the negative X-axis direction.
For another example, all the preset direction sets include: the method comprises the following steps of establishing a coordinate system, establishing a preset direction set, wherein the preset direction set comprises a positive direction of an X axis and a positive direction of a Y axis in the established coordinate system, the preset direction set comprises a positive direction of the X axis and a negative direction of the Y axis in the established coordinate system, the preset direction set comprises a negative direction of the X axis and a negative direction of the Y axis in the established coordinate system, and the preset direction set comprises a negative direction of the X axis and a positive direction of the Y axis in the established coordinate system. When one container has no other adjacent container in each direction of a preset direction set, the preset direction set may also be referred to as a target preset direction set. The included angle formed by the determined carrying direction of the target container and each direction in the target preset direction set can be smaller than 90 degrees. Therefore, the determined carrying direction of the target container inclines for a certain angle relative to each direction in the target preset direction set.
In some optional implementations of this embodiment, when there is no other container adjacent to one container in each direction of a preset direction set, the preset direction set may also be referred to as a target preset direction set. For each direction in the target preset direction set, the determined included angle between the carrying direction of the target container and the direction is a preset angle corresponding to the direction.
For example, all the preset direction sets include: the preset direction set comprises a preset direction set consisting of a positive direction of an X axis and a positive direction of a Y axis, a preset direction set consisting of a positive direction of an X axis and a negative direction of a Y axis, a preset direction set consisting of a negative direction of an X axis and a negative direction of a Y axis, and a preset direction set consisting of a negative direction of an X axis and a positive direction of a Y axis.
When the target preset direction set is a preset direction set consisting of the positive direction of the X axis and the positive direction of the Y axis, the carrying direction of the target container
Can be the angle of presetting that the positive direction of X axle corresponds with the contained angle in the positive direction of X axle, the transport direction of target packing box and the contained angle in the positive direction of Y axle are the angle of presetting that the positive direction of Y axle corresponds, and the angle of presetting that the positive direction of X axle corresponds, the angle of presetting that the positive direction of Y axle corresponds can all be less than 90 degrees. When the target preset direction set is a preset direction set consisting of a positive direction of an X axis and a negative direction of a Y axis, an included angle between the conveying direction of the target container and the positive direction of the X axis is a preset angle corresponding to the positive direction of the X axis, an included angle between the conveying direction of the target container and the negative direction of the Y axis is a preset angle corresponding to the negative direction of the Y axis, and the preset angle corresponding to the negative direction of the Y axis can be smaller than 90 degrees. When the target preset direction set is a preset direction set consisting of a negative direction of an X axis and a negative direction of a Y axis, an included angle between the carrying direction of the target container and the negative direction of the X axis is a preset angle corresponding to the negative direction of the X axis, an included angle between the carrying direction of the target container and the negative direction of the Y axis is a preset angle corresponding to the negative direction of the Y axis, and the preset angle corresponding to the negative direction of the X axis can be smaller than 90 degrees. When the target preset direction set is a preset direction set consisting of the negative direction of the X axis and the positive direction of the Y axis, the included angle between the conveying direction of the target container and the negative direction of the X axis is a preset angle corresponding to the negative direction of the X axis, and the included angle between the conveying direction of the target container and the positive direction of the Y axis is a preset angle corresponding to the positive direction of the Y axis.
In this embodiment, each preset direction set may correspond to a type, and when there is no other adjacent container in each direction of a target container in a preset direction set, the type of the target container is the type corresponding to the preset direction set.
For example, when there is no adjacent container in both the positive direction of the X axis and the positive direction of the Y axis, that is, each direction of the target container in the preset direction set composed of the positive direction of the X axis and the positive direction of the Y axis is an adjacent other container, the type of the target container is class 1. When a target container has no adjacent containers in both the negative X-axis direction and the positive Y-axis direction, then the type of the target container is class 2. When a target container has no adjacent containers in both the negative X-axis direction and the negative Y-axis direction, then the type of the target container is category 3. When a target container has no adjacent containers in both the negative X-axis direction and the negative Y-axis direction, then the type of the target container is category 4.
In this embodiment, the type of the target container may be determined by the vision system, and the vision system encapsulates the number indicating the type of the target container in an IP message and sends the IP message to the transfer robot. And the carrying robot determines a preset direction set corresponding to the type according to the type of the target container, and further determines a carrying direction.
And 203, conveying the target container out of the stacking area based on the position information of the target container and the conveying direction of the target container.
In this embodiment, the target container may be positioned by the transfer robot according to the position information of the target container, the control system of the transfer robot may control the clamping device, such as the suction cup, to clamp the target container, and then the transfer robot may transfer the target container out of the stacking area along the determined transfer direction.
For example, the gripping device, such as a suction cup, of the transfer robot may grip the candidate container, and then the gripping device of the transfer robot may move in the determined transfer direction to transfer the target container out of the container stacking area.
Referring to fig. 3, a schematic diagram of one effect of determining a target container is shown.
All containers in the stacking area are stacked in a stacked manner in the container stacking area. The vision system may be used to detect 4 types of candidate containers located at the corners, and then, the vision system may be used to further determine a candidate container with the smallest error of the position information, and the candidate container with the smallest error of the position information may be used as the target container. The vision system may send a number representing the type of target container in an IP message to the transfer robot. The carrying robot can determine a preset direction set corresponding to the type according to the type of the target container, and further determine the carrying direction of the target container. When the target container is of type 1, the carrying direction and the positive directions of the X axis and the Y axis respectively have preset included angles. When the target container is of type 2, the carrying direction and the positive direction of the X axis and the negative direction of the Y axis both have preset included angles. When the target container is of type 3, the carrying direction and the X-axis negative direction and the Y-axis negative direction both have preset included angles. When the target container is of type 4, the carrying direction and the negative direction of the X axis and the positive direction of the Y axis both have preset included angles.
Because when the target container is conveyed to the goods box stacking area, the conveying direction and the direction of the corresponding coordinate axis all have a certain included angle, the situation that the container around is scraped in the conveying process can be avoided.
Referring to fig. 4, as an implementation of the method shown in the above figures, the present application provides an embodiment of a container handling apparatus, which corresponds to the embodiment of the method shown in fig. 2. Specific implementations of operations that the respective units in the apparatus are configured to perform may refer to the corresponding specific implementations of operations described in the method embodiments.
As shown in fig. 4, the container handling apparatus of the present embodiment includes: processing means 401, determining means 402, and conveying means 403. Wherein the processing unit 401 is configured to obtain position information of a container in the container stacking area, and determine a candidate container satisfying candidate conditions in the container stacking area based on the position information of the container in the container stacking area, wherein the candidate conditions include: each direction in the preset direction set is not provided with other adjacent containers; the determination unit 402 is configured to determine a target container among the candidate containers, and determine a carrying direction of the target container; the conveying unit 403 is configured to convey the target container out of the stacking area based on the position information of the target container and the conveying direction of the target container.
In some optional implementations of this embodiment, one set of preset directions includes one of: the X-axis positive direction and the Y-axis positive direction, the X-axis positive direction and the Y-axis negative direction, the X-axis negative direction and the Y-axis positive direction, wherein the X-axis and the Y-axis are coordinate axes in a plane parallel to a horizontal plane.
In some optional implementation manners of this embodiment, for each direction in the set of target preset directions, an included angle between the carrying direction of the target container and the direction is a preset angle corresponding to the direction, and there is no other adjacent container in each direction in the set of target preset directions for the target container.
In some optional implementations of this embodiment, the determining unit is further configured to: and taking the candidate container with the smallest error of the position information in the candidate container as the target container.
FIG. 5 illustrates a schematic structural diagram of a computer system suitable for use in implementing the electronic device of an embodiment of the present application.
As shown in fig. 5, the computer system includes a Central Processing Unit (CPU)501 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM503, various programs and data necessary for the operation of the computer system are also stored. The CPU 501, ROM 502, and RAM503 are connected to each other via a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
To the I/O interface 505, AN input section 506, AN output section 507, a storage section 508 including a hard disk or the like, and a communication section 509 including a network interface card such as L AN card, a modem or the like, the communication section 509 performs communication processing via a network such as the internet, the drive 510 is also connected to the I/O interface 505 as necessary, a removable medium 511 such as a magnetic disk, AN optical disk, a magneto-optical disk, a semiconductor memory or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary.
In particular, the processes described in the embodiments of the present application may be implemented as computer programs. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising instructions for carrying out the method illustrated in the flow chart. The computer program can be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 511. The computer program performs the above-described functions defined in the method of the present application when executed by the Central Processing Unit (CPU) 501.
The present application further provides an electronic device that may be configured with one or more processors; a memory for storing one or more programs, the one or more programs may include instructions for performing the operations described in the above embodiments. The one or more programs, when executed by the one or more processors, cause the one or more processors to perform the instructions of the operations described in the above embodiments.
The present application also provides a computer readable medium, which may be included in an electronic device; or the device can be independently arranged and not assembled into the electronic equipment. The computer-readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to perform the operations described in the embodiments above.
It should be noted that the computer readable medium described herein can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with a message execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with a message execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable messages for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer messages.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by a person skilled in the art that the scope of the invention as referred to in the present application is not limited to the embodiments with a specific combination of the above-mentioned features, but also covers other embodiments with any combination of the above-mentioned features or their equivalents without departing from the inventive concept. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.