WO2023213070A1

WO2023213070A1 - Method and apparatus for obtaining goods pose based on 2d camera, device, and storage medium

Info

Publication number: WO2023213070A1
Application number: PCT/CN2022/133338
Authority: WO
Inventors: 沈维国
Original assignee: 劢微机器人科技(深圳)有限公司
Priority date: 2022-05-06
Filing date: 2022-11-21
Publication date: 2023-11-09
Also published as: CN115062737A

Abstract

The present application relates to the technical field of image processing, and discloses a method and apparatus for obtaining a goods pose based on a 2D camera, a device, and a storage medium. The method comprises: obtaining a goods QR code image; determining QR code corner information according to the goods QR code image; determining, according to the QR code corner information and a preset calibration parameter, three-dimensional space coordinates corresponding to the goods QR code image; and determining a goods pose according to the QR code three-dimensional space coordinates.

Description

Methods, devices, equipment and storage media for obtaining cargo pose based on 2D camera

This application claims priority to the Chinese patent application with application number 202210486489.2 filed on May 6, 2022, the entire content of which is incorporated into this application by reference.

Technical field

This application relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for obtaining the pose of goods based on a 2D camera.

Background technique

With the development of smart industry and smart logistics, warehouse management is becoming more and more unmanned, and cargo size measurement plays a very important role in warehouse management. How an unmanned forklift picks up goods that have been placed randomly is an important issue in the smart warehousing industry. The most important of these is how to accurately determine the position of the goods. The current general method is to use a 3D camera to obtain the point cloud of the goods. Then the position and attitude of the goods are determined based on the point cloud. However, this method is costly, highly dependent on the lighting environment, and requires the goods to be of the same specifications. How to achieve the acquisition of cargo position more economically has become an urgent technical problem that needs to be solved.

The above content is only used to assist in understanding the technical solutions of the present application, and does not represent an admission that the above content is prior art.

technical problem

The main purpose of this application is to provide a method, device, equipment and storage medium for obtaining the posture of goods based on a 2D camera, aiming to solve the technical problem of high cost in determining the posture of goods in the existing technology.

Technical solutions

In order to achieve the above purpose, this application provides a method for obtaining the pose of goods based on a 2D camera. The method includes the following steps:

Obtain the QR code image of the goods;

Determine the QR code corner information based on the cargo QR code image;

Determine the three-dimensional space coordinates corresponding to the cargo QR code image based on the QR code corner information and preset calibration parameters;

The position and orientation of the goods are determined according to the three-dimensional spatial coordinates of the two-dimensional code.

In one embodiment, determining the three-dimensional spatial coordinates of the QR code based on the QR code corner information and preset calibration parameters includes:

Get standard corner point information;

The three-dimensional space coordinates of the two-dimensional code are determined according to the two-dimensional code corner information, the standard corner information and the preset calibration parameters.

In one embodiment, an information area and a positioning area are provided in the QR code image, and the three-dimensional spatial coordinates of the QR code are determined based on the QR code corner information, the standard corner information and preset calibration parameters. ,include:

Determine the corner point of the information area and positioning information based on the QR code corner point information;

Determine the homography matrix of the two-dimensional code based on the corner points of the information area and the standard corner point information;

Determine corner point information of the positioning area according to the homography matrix and standard corner point information;

The three-dimensional space coordinates of the two-dimensional code are determined according to the corner point information of the positioning area, the standard corner point information and the preset calibration parameters.

In one embodiment, determining the three-dimensional spatial coordinates of the QR code based on the corner point information of the positioning area, the standard corner point information and preset calibration parameters includes:

Determine multiple positioning area corner points and labels corresponding to the positioning area corner points according to the positioning area corner point information;

Determine multiple standard positioning area corner points and labels corresponding to the standard positioning area corner points according to the standard corner point information;

Determine the corresponding relationship between the corner points of the positioning area and the corner points of the standard positioning area based on the labels corresponding to the corner points of the positioning area and the labels corresponding to the corner points of the standard positioning area;

The three-dimensional spatial coordinates of the two-dimensional code are determined according to the corresponding relationship and the preset calibration parameters.

In one embodiment, determining the three-dimensional spatial coordinates of the two-dimensional code based on the corresponding relationship and preset calibration parameters includes:

Determine the relative positional relationship between the corner points of each positioning area and the corner points of the standard positioning area according to the corresponding relationship;

Determine the actual spatial position of the corner points of each positioning area according to the relative position relationship and the preset calibration parameters;

The three-dimensional spatial coordinates of the two-dimensional code are determined according to the actual spatial positions of the corner points of each positioning area.

In one embodiment, the unmanned forklift is equipped with a fill light. Before obtaining the QR code image of the goods, the method further includes:

When it is detected that the distance of the goods is less than the preset distance, control the fill light to supplement the light;

When the fill light supplements the light, the QR code image of the goods is recognized.

In one embodiment, determining the QR code corner information based on the cargo QR code image includes:

Perform adaptive threshold segmentation on the QR code image to obtain the image outline;

The two-dimensional code corner information is determined according to the image outline.

In addition, in order to achieve the above purpose, this application also proposes a device for obtaining the posture of goods based on a 2D camera. The device for obtaining the posture of goods based on a 2D camera includes:

Acquisition module, used to obtain QR code images of goods;

A processing module configured to determine the QR code corner information based on the QR code image of the goods;

The processing module is also used to determine the three-dimensional spatial coordinates of the QR code based on the QR code corner information and preset calibration parameters;

The processing module is also used to determine the position and orientation of the goods according to the three-dimensional spatial coordinates of the two-dimensional code.

In addition, in order to achieve the above purpose, this application also proposes a device for obtaining the posture of goods based on a 2D camera. The equipment for obtaining the posture of goods based on a 2D camera includes: a memory, a processor, and a device that is stored on the memory and can be used on the A program for acquiring the cargo pose based on the 2D camera running on the processor is configured to implement the steps of the method for acquiring the cargo pose based on the 2D camera as described above.

In addition, in order to achieve the above purpose, this application also proposes a storage medium, which stores a program for obtaining the posture of goods based on a 2D camera. When the program for obtaining the posture of goods based on a 2D camera is executed by the processor, the above implementation is implemented. The steps of the method of obtaining cargo pose based on 2D camera.

beneficial effects

This application obtains the QR code image of the goods; determines the QR code corner information based on the QR code image of the goods; determines the 3D corresponding to the QR code image of the goods based on the QR code corner information and preset calibration parameters. Spatial coordinates; determine the posture of the cargo according to the three-dimensional spatial coordinates of the QR code. Through the above method, the calibration information is used to convert the points in the plane of the QR code into points in the three-dimensional space. Since the QR code is set at a fixed position of the goods, the relative position of the points in the QR code and the goods is and posture are fixed, so only the QR code can be used to locate the three-dimensional posture of the goods. The posture of the goods can still be determined without the need for a 3D camera, which saves the unmanned truck the trouble of determining the posture of the goods. cost.

Description of the drawings

Figure 1 is a schematic structural diagram of a device for acquiring cargo pose based on a 2D camera in the hardware operating environment involved in the embodiment of the present application;

Figure 2 is a schematic flowchart of the first embodiment of the method for obtaining the pose of goods based on a 2D camera according to this application;

Figure 3 is a schematic diagram of a QR code according to one embodiment of the method for obtaining the pose of goods based on a 2D camera;

Figure 4 is a schematic flowchart of the second embodiment of the method for obtaining the pose of goods based on a 2D camera in this application;

Figure 5 is a structural block diagram of the first embodiment of the device for obtaining the position and orientation of goods based on a 2D camera in this application.

The realization of the purpose, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings.

Embodiments of the invention

It should be understood that the specific embodiments described here are only used to explain the present application and are not used to limit the present application.

Referring to Figure 1, Figure 1 is a schematic structural diagram of a device for acquiring cargo pose based on a 2D camera in the hardware operating environment involved in the embodiment of the present application.

As shown in Figure 1, the device for acquiring cargo pose based on a 2D camera may include: a processor 1001, such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize connection communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard). In one embodiment, the user interface 1003 may also include a standard wired interface and a wireless interface. In an embodiment, the network interface 1004 may include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. In an embodiment, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

Those skilled in the art can understand that the structure shown in Figure 1 does not constitute a limitation on the device for acquiring cargo posture based on a 2D camera. It may include more or less components than shown in the figure, or some components may be combined or different. component layout.

As shown in Figure 1, memory 1005, which is a storage medium, may include an operating system, a network communication module, a user interface module, and a program for obtaining the posture of goods based on a 2D camera.

In the device for acquiring cargo pose based on a 2D camera shown in Figure 1, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; this application acquires the cargo pose based on a 2D camera The processor 1001 and the memory 1005 in the device can be configured in the device for acquiring the cargo pose based on the 2D camera. The device for acquiring the cargo pose based on the 2D camera calls the program for acquiring the cargo pose based on the 2D camera stored in the memory 1005 through the processor 1001. , and execute the method for obtaining the cargo pose based on the 2D camera provided by the embodiment of this application.

The embodiment of the present application provides a method for obtaining the posture of goods based on a 2D camera. Refer to Figure 2. Figure 2 is a schematic flow chart of a method of obtaining the posture of goods based on a 2D camera according to the first embodiment of the present application.

In this embodiment, the method for obtaining the pose of goods based on a 2D camera includes the following steps:

Step S10: Obtain the QR code image of the goods.

It should be noted that the execution subject of this embodiment is a cargo posture detection system. The cargo posture detection system is installed on an unmanned handling equipment. The unmanned handling equipment can be an unmanned forklift or an AGV car. , it can also be an unmanned transport robot, or even other equipment that has the same or similar functions as an unmanned forklift. This embodiment is not limited to this, and is only explained using an unmanned forklift as an example.

It can be understood that this embodiment is applied to the process of detecting cargo poses by unmanned trucks, unmanned forklifts, and handling robots. The accuracy of the cargo pose detection process directly affects the process of cargo loading and unloading. In man-made factories, 3D scanning equipment such as lidar is used to 3D scan goods to obtain a 3D lattice model of the goods. However, this method requires the assembly of a large number of high-precision sensors, which is costly and requires a lot of data to be processed during pose detection. The amount of data is huge, the memory usage is high, and the handling process is easily affected by long response times and lags. Therefore, this embodiment proposes a solution that only needs to be equipped with a 2D camera to accurately detect the pose, so as to obtain a more economical pose. testing method.

It should be noted that the QR code image of the goods can be obtained through the image collection equipment on the unmanned forklift. For example, by taking a photo, the image information is obtained, and then the image is positioned on the QR code to obtain the QR code image.

Specifically, the goods QR code is a QR code identification fixed on the goods, which stores the identification information of the goods, and the position of the QR code is very fixed, that is, each position in the QR code is relative to each position of the goods. The relative distances are fixed, which also lays the foundation for determining the acquisition pose based on the QR code pose.

In this embodiment, a fill light is provided on the unmanned forklift. Before acquiring the QR code image of the goods, the method further includes: when it is detected that the distance of the goods is less than a preset distance, controlling the fill light to supplement the light; When the fill light supplements the light, the QR code image of the goods is recognized.

It should be noted that the unmanned forklift in this solution can be equipped with an ordinary camera and a fill light. During use, the fill light is placed near the camera and fixed on the unmanned forklift together with the camera. When the forklift drives to the cargo attachment and stops, the fill light will illuminate and the camera will start taking pictures, thereby obtaining a two-dimensional image of the cargo. code image. This can operate in more scenes with poor lighting. Since this embodiment uses 2D pose detection technology, the impact of light is greater, so a fill light can be used to assist to improve the accuracy of pose detection.

Step S20: Determine the QR code corner information based on the cargo QR code image.

It can be understood that the corner point information of the QR code can be obtained directly by performing corner point recognition on the QR code. By performing corner point identification on the QR code, all corner position information in the QR code can be obtained, for example: in Figure 3 The corner positions of the upper left, upper right, lower left, lower right corners of the information area. Store the images and coordinates of these corner points as QR code corner point information.

It should be noted that although corner detection can identify the position of each corner point, it cannot identify the identity of each corner point. Therefore, it is necessary to further determine the identity of each QR code corner point. Only by identifying the identity can each corner point be determined. Where should the normal placement be?

In this embodiment, adaptive threshold segmentation is performed on the QR code image to obtain an image outline; the QR code corner information is determined based on the image outline.

It should be noted that since the error of direct corner point recognition may be very large, adaptive threshold segmentation can be performed on the QR code image, that is, the color image is converted into a black and white image, so that the black and white boundary, that is, the image outline, can be determined. According to the image outline The success rate of corner point identification will be greatly improved, and the corner information of the QR code will be finally determined.

Step S30: Determine the three-dimensional space coordinates corresponding to the cargo QR code image based on the QR code corner information and preset calibration parameters.

It is understandable that based on the QR code corner information, it can be judged whether the unmanned forklift is facing the goods at this time, and then the position of the unmanned forklift can be adjusted so that the goods are in an ideal relative position, for example: through the 4 information areas in Figure 3 The corner points determine whether the connected graphics are squares, determine whether the direction is correct, and then determine whether the distance is correct based on the size of the square. When the direction and distance are both ideal, the corner points of each information area are converted according to the preset calibration parameters to determine each The three-dimensional coordinates of the corner points are enough.

Among them, the preset calibration parameters can be internal calibration parameters and external calibration parameters. The internal calibration parameters are used to determine the accurate position of each point in the image in the camera coordinate system, and the external calibration parameters are used to convert the coordinates in the camera coordinate system into The unmanned forklift is the spatial coordinate of the reference system.

Step S40: Determine the posture of the goods according to the three-dimensional spatial coordinates of the QR code.

It can be understood that since the relative position of the QR code and the entire cargo is fixed, after obtaining the three-dimensional space coordinates of the QR code, this can be calculated based on the positional relationship between the three-dimensional space coordinates of the original QR code and the three-dimensional space coordinates of the cargo. The three-dimensional space coordinates of the cargo are obtained to determine the posture.

This embodiment obtains the QR code image of the goods; determines the QR code corner information based on the QR code image of the goods; determines the corresponding QR code image of the goods based on the QR code corner information and preset calibration parameters. Three-dimensional space coordinates; determine the posture of the cargo according to the three-dimensional space coordinates of the QR code. Through the above method, the calibration information is used to convert the points in the plane of the QR code into points in the three-dimensional space. Since the QR code is set at a fixed position of the goods, the relative position of the points in the QR code and the goods is and posture are fixed, so only the QR code can be used to locate the three-dimensional posture of the goods. The posture of the goods can still be determined without the need for a 3D camera, which saves the unmanned truck the trouble of determining the posture of the goods. cost.

Referring to Figure 4, Figure 4 is a schematic flowchart of a second embodiment of a method for obtaining the pose of goods based on a 2D camera according to the present application.

Based on the above-mentioned first embodiment, in this embodiment, the method of obtaining the pose of the goods based on the 2D camera also includes:

Step S31: Obtain standard corner point information.

It should be noted that the standard corner point information can be the location of each corner point in the QR code during calibration, or other corner point information used to determine the deviation between the currently detected corner point and the standard corner point position. For example: For example, the corner coordinates of the four corners of the information area in Figure 3 are (2, 2) (-2, 1) (-2, -1) (2, -2) and the standard corner coordinates are (2, 2 ) (-2, 2) (-2, -2) (2, -2), that is, it is determined that the goods are not facing the forklift but deflected to the right. This is because the length distance between the two corner points The farther the image acquisition device is, the shorter the length displayed in the image. Therefore, it can be obtained by comparing the standard corner points and the QR code corner points based on the 4 corner points. The pose of the current QR code is compared with the standard 2D code. How much deviation has occurred in the pose of the code, thereby inferring the pose of the current QR code.

Step S32: Determine the three-dimensional spatial coordinates of the two-dimensional code based on the two-dimensional code corner information, the standard corner information and preset calibration parameters.

It can be understood that by comparing the QR code corner point information with the standard corner point information, the degree of deviation of the posture from the standard posture can be known, and then the conversion of pixel coordinates and actual space coordinates can be performed in combination with the preset calibration parameters. Determine the three-dimensional coordinates of the QR code.

In this embodiment, the information area corner point and positioning information are determined based on the QR code corner point information; the homography matrix of the QR code is determined based on the information area corner point and standard corner point information; The homography matrix and standard corner point information determine the corner point information of the positioning area; the three-dimensional space coordinates of the two-dimensional code are determined according to the corner point information of the positioning area, the standard corner point information and the preset calibration parameters.

It should be noted that although corner point detection can identify the position of each corner point, it cannot identify the identity of each corner point. Therefore, in this process, this embodiment actually first identifies and determines the QR code in the information area. The position of the QR code in the image is determined at the same time as the goods ID, and the QR code corners of the four information areas are determined based on the position of the information area.

It can be understood that the homography matrix is used to reflect the correlation between two images on the same plane in space at any different shooting angles. As shown in Figure 3, the corner points of the information area on the camera photo can be first arrayed with The homography matrix is determined by using the standard information area corners in the standard corner point information to obtain the spatial relationship between the QR code image under the standard image and the QR code image under the current image; and then based on the homography matrix and the standard image Use the corner points of the positioning area under the current image to initially determine the approximate position of the corner points of the positioning area under the current image, and then use the approximate positions of these positioning area corner points to filter out the corner points of the positioning area, so that the accurate corner points of the positioning area can be obtained information.

In this embodiment, multiple positioning area corner points and corresponding labels of the positioning area corner points are determined based on the positioning area corner point information; multiple standard positioning area corner points and standard positioning area corner points are determined based on the standard corner point information. The corresponding label of the point; determine the corresponding relationship between the corner point of the positioning area and the corner point of the standard positioning area according to the label corresponding to the corner point of the positioning area and the label corresponding to the corner point of the standard positioning area; according to the corresponding relationship and the preset calibration The parameters determine the three-dimensional coordinates of the QR code.

It can be understood that, as shown in Figure 3, the QR code is divided into a positioning area and an information area. Generally speaking, the position and posture of the goods can be calculated by comparing the corner points of the positioning area with the standard corner points. However, in fact, it is only based on The calculation accuracy of 4 points is very limited, so this embodiment proposes an optimal solution with higher accuracy, as follows: Obtain the homography matrix based on the corner points of the four information areas and the standard corner point information, and then calculate the homography matrix based on the homography The matrix obtains the corner points of the black and white part of the image (corner points of the positioning area). It can be seen that there are 44 corner points of the information area in Figure 3. The corner points have high accuracy and a large number of corner points. Then the three-dimensional coordinates of the corner point are obtained based on the internal and external parameters of the camera. The position and attitude of the QR code are obtained through three-dimensional coordinates. The logic of obtaining the corner points of the information area based on the homography matrix is to use the corner points of the information area as the basis to calculate which corners are the corner points of the positioning area, so as to avoid useless corner points from being mixed into the calculation process, and each information area corner point is not processed label, and then compare it with the corner point corresponding to the label in the standard corner point information. The actual logic is consistent with the calculation process of corner points in the information area, except that there are more corner points in the positioning area, and the corner points in the positioning area cover a wider range. Therefore, the accuracy is higher, which will not be described in detail in this embodiment.

In this embodiment, the relative positional relationship between the corner points of each positioning area and the corner points of the standard positioning area is determined according to the corresponding relationship; the actual spatial position of the corner points of each positioning area is determined according to the relative positional relationship and the preset calibration parameters; The three-dimensional spatial coordinates of the two-dimensional code are determined according to the actual spatial positions of the corner points of each positioning area.

It should be noted that the relative positional relationship between the corner points of each positioning area and the corner points of the standard positioning area is determined based on the corresponding relationship. For example: assuming that each corner point of the positioning area in Figure 3 has a corresponding label, respectively, N1~N44, Then the corresponding standard information area corner points in the standard corner point information also have corresponding numbers N1~N44. The corner points with the same label are matched to determine the relative position relationship of each pair of corner points, thereby deducing the standard QR code and the current The pose relationship of the QR code can be used to calculate the three-dimensional coordinates of each corner point of the current QR code based on the three-dimensional coordinates of each corner point in the standard QR code.

In specific implementation, this embodiment proposes a preferred implementation method, for example: fixing the QR code to the goods, fixing the position of the QR code to the position of the goods, and knowing the position and posture of the QR code means knowing the goods. position posture. So what we are looking for here is the position and posture of the QR code. Process the image acquired by the camera, perform adaptive threshold segmentation to obtain a binary image, extract the contour, obtain the quadrilateral outline, decode the black and white on the image, and obtain the ID of the QR code and the four corner points of the QR code, as shown in the figure 3 shown. The homography matrix is obtained based on these four corner points and the original image corner points, and then the corner points of the black and white part of the image are obtained based on the homography matrix. The corner points have high accuracy and a large number of corner points. Then the three-dimensional coordinates of the corner point are obtained based on the internal and external parameters of the camera, and the position and attitude of the QR code are obtained through the three-dimensional coordinates.

This embodiment obtains standard corner point information; determines the three-dimensional space coordinates of the two-dimensional code based on the two-dimensional code corner information, the standard corner point information and preset calibration parameters. Through the above method, standard corner points are introduced so that the unmanned forklift can detect the posture of the cargo without adjusting its own posture, which improves the flexibility of cargo posture detection.

In addition, embodiments of the present application also propose a storage medium on which is stored a program for acquiring cargo poses based on a 2D camera. When the program for acquiring cargo poses based on a 2D camera is executed by a processor, the above-mentioned steps are implemented. Steps of obtaining cargo pose based on 2D camera.

Referring to Figure 5, Figure 5 is a structural block diagram of a first embodiment of a device for obtaining the position and orientation of goods based on a 2D camera according to the present application.

As shown in Figure 5, the device for obtaining the pose of goods based on a 2D camera proposed in the embodiment of this application includes:

The acquisition module 10 is used to acquire the QR code image of the goods;

The processing module 20 is used to determine the QR code corner information based on the QR code image of the goods;

The processing module 20 is also used to determine the three-dimensional spatial coordinates of the QR code based on the QR code corner information and preset calibration parameters;

The processing module 20 is also used to determine the position and orientation of the goods according to the three-dimensional spatial coordinates of the two-dimensional code.

In one embodiment, the processing module 20 is also used to obtain standard corner point information;

In one embodiment, the processing module 20 is also used to determine the corner point of the information area and the positioning information based on the corner point information of the QR code;

In one embodiment, the processing module 20 is further configured to determine multiple positioning area corner points and labels corresponding to the positioning area corner points according to the positioning area corner point information;

In one embodiment, the processing module 20 is further configured to determine the relative positional relationship between the corner points of each positioning area and the corner points of the standard positioning area according to the corresponding relationship;

In one embodiment, the processing module 20 is also used to control the fill light to supplement the light when it is detected that the distance of the goods is less than the preset distance;

In one embodiment, the processing module 20 is also used to perform adaptive threshold segmentation on the QR code image to obtain the image outline;

It should be understood that the above are only examples and do not constitute any limitation on the technical solution of the present application. In specific applications, those skilled in the art can make settings as needed, and the present application does not impose any limitations on this.

In this embodiment, the acquisition module 10 obtains the QR code image of the goods; the processing module 20 determines the QR code corner information based on the QR code image of the goods; the processing module 20 determines the QR code corner information based on the QR code corner information and preset calibration parameters. The three-dimensional space coordinates corresponding to the two-dimensional code image of the goods; the processing module 20 determines the posture of the goods according to the three-dimensional space coordinates of the two-dimensional code. Through the above method, the calibration information is used to convert the points in the plane of the QR code into points in the three-dimensional space. Since the QR code is set at a fixed position of the goods, the relative position of the points in the QR code and the goods is and posture are fixed, so only the QR code can be used to locate the three-dimensional posture of the goods. The posture of the goods can still be determined without the need for a 3D camera, which saves the unmanned truck the trouble of determining the posture of the goods. cost.

It should be noted that the workflow described above is only illustrative and does not limit the scope of protection of the present application. In practical applications, those skilled in the art can select part or all of it for implementation according to actual needs. The purpose of this embodiment is not limited here.

In addition, for technical details that are not described in detail in this embodiment, please refer to the method for obtaining cargo pose based on a 2D camera provided in any embodiment of this application, and will not be described again here.

Furthermore, it should be noted that, as used herein, the terms "include", "comprises" or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or system that includes a list of elements includes not only those elements, but also other elements not expressly listed or elements inherent to the process, method, article or system. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

The above serial numbers of the embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is better. implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or that contributes to the existing technology. The computer software product is stored in a storage medium (such as a read-only memory). , ROM)/RAM, magnetic disk, optical disk), including several instructions to cause a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of this application.

The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of the present application may be directly or indirectly used in other related technical fields. , are all equally included in the patent protection scope of this application.

Claims

A method of obtaining the posture of goods based on a 2D camera, in which the QR code of the goods is set at a fixed position of the goods. The method of obtaining the posture of the goods based on a 2D camera includes:

Obtain the QR code image of the goods;

Determine the QR code corner information based on the cargo QR code image;

Determine the three-dimensional space coordinates corresponding to the cargo QR code image based on the QR code corner information and preset calibration parameters;

The position and orientation of the goods are determined according to the three-dimensional spatial coordinates of the two-dimensional code.
The method of claim 1, wherein determining the three-dimensional spatial coordinates of the two-dimensional code based on the two-dimensional code corner information and preset calibration parameters includes:

Get standard corner point information;

The three-dimensional space coordinates of the two-dimensional code are determined according to the two-dimensional code corner information, the standard corner information and the preset calibration parameters.
The method of claim 2, wherein the two-dimensional code image is provided with an information area and a positioning area, and the two-dimensional code is determined based on the two-dimensional code corner information, the standard corner information and preset calibration parameters. The three-dimensional space coordinates of the code include:

Determine the corner point of the information area and positioning information based on the QR code corner point information;

Determine the homography matrix of the two-dimensional code based on the corner points of the information area and the standard corner point information;

Determine corner point information of the positioning area according to the homography matrix and standard corner point information;

The three-dimensional space coordinates of the two-dimensional code are determined according to the corner point information of the positioning area, the standard corner point information and the preset calibration parameters.
The method of claim 3, wherein determining the three-dimensional spatial coordinates of the two-dimensional code based on the corner point information of the positioning area, the standard corner point information and preset calibration parameters includes:

Determine multiple positioning area corner points and labels corresponding to the positioning area corner points according to the positioning area corner point information;

Determine multiple standard positioning area corner points and labels corresponding to the standard positioning area corner points according to the standard corner point information;

Determine the corresponding relationship between the corner points of the positioning area and the corner points of the standard positioning area based on the labels corresponding to the corner points of the positioning area and the labels corresponding to the corner points of the standard positioning area;

The three-dimensional spatial coordinates of the two-dimensional code are determined according to the corresponding relationship and the preset calibration parameters.
The method of claim 4, wherein determining the three-dimensional spatial coordinates of the two-dimensional code according to the corresponding relationship and preset calibration parameters includes:

Determine the relative positional relationship between the corner points of each positioning area and the corner points of the standard positioning area according to the corresponding relationship;

Determine the actual spatial position of the corner points of each positioning area according to the relative position relationship and the preset calibration parameters;

The three-dimensional spatial coordinates of the two-dimensional code are determined according to the actual spatial positions of the corner points of each positioning area.
The method according to any one of claims 1 to 5, wherein the unmanned forklift is provided with a fill light, and before obtaining the QR code image of the goods, it also includes:

When it is detected that the distance of the goods is less than the preset distance, control the fill light to supplement the light;

When the fill light supplements the light, the QR code image of the goods is recognized.
The method according to any one of claims 1 to 5, wherein determining the QR code corner information based on the cargo QR code image includes:

Perform adaptive threshold segmentation on the QR code image to obtain the image outline;

The two-dimensional code corner information is determined according to the image outline.
A device for acquiring cargo pose based on a 2D camera, wherein the device for acquiring cargo pose based on a 2D camera includes:

Acquisition module, used to obtain QR code images of goods;

A processing module configured to determine the QR code corner information based on the QR code image of the goods;

The processing module is also used to determine the three-dimensional spatial coordinates of the QR code based on the QR code corner information and preset calibration parameters;

The processing module is also used to determine the position and orientation of the goods according to the three-dimensional spatial coordinates of the two-dimensional code.
A device for acquiring cargo poses based on a 2D camera, wherein the device includes: a memory, a processor, and a program for acquiring cargo poses based on a 2D camera that is stored on the memory and can run on the processor, the The program for acquiring the cargo pose based on the 2D camera is configured to implement the steps of the method for acquiring the cargo pose based on the 2D camera as described in any one of claims 1 to 7.
A storage medium, wherein a program for acquiring cargo poses based on a 2D camera is stored on the storage medium, and when the program for acquiring cargo poses based on a 2D camera is executed by a processor, the implementation is as described in any one of claims 1 to 7 The steps of obtaining the cargo pose based on 2D camera.