WO2023169186A1

WO2023169186A1 - Positioning method and related device

Info

Publication number: WO2023169186A1
Application number: PCT/CN2023/077019
Authority: WO
Inventors: 王伟杰; 郑月; 张小龙
Original assignee: 华为技术有限公司
Priority date: 2022-03-10
Filing date: 2023-02-18
Publication date: 2023-09-14
Also published as: CN116772804A

Abstract

A positioning method and a related device. The positioning method comprises the following steps: when a target object is in a first position, a processing apparatus acquiring first positioning coordinates of a mark point on the target object in a first coordinate system of a first camera, and second positioning coordinates of the mark point on the target object in a second coordinate system of a second camera; when the target object is in a second position, the processing apparatus acquiring third positioning coordinates of the mark point on the target object in the first coordinate system, and fourth positioning coordinates of the mark point on the target object in the second coordinate system, the first position and the second position being located in a photographing overlap area of the first camera and the second camera; according to the first positioning coordinates, the second positioning coordinates, the third positioning coordinates and the fourth positioning coordinates, the processing apparatus determining a first coordinate system transformation parameter between the first coordinate system and the second coordinate system; and then, according to the first coordinate system transformation parameter, the processing apparatus positioning an object to be detected. Further provided are a processing apparatus and a positioning system which use the method.

Description

A positioning method and related equipment

This application claims priority to the Chinese patent application submitted to the State Intellectual Property Office of China on March 10, 2022, with application number 202210240303.5 and the application title "A positioning method and related equipment", the entire content of which is incorporated herein by reference. Applying.

Technical field

The present application relates to the field of positioning technology, and in particular, to a positioning method and related equipment.

Background technique

Positioning technology has been increasingly widely used in many industries in the past few years and plays an increasingly important role in our daily lives. From map navigation to social networks, location services play a key role. Among them, indoor positioning technology is becoming a hot area of academic research and industry application. Common indoor positioning systems usually require the application of computer vision technology and wireless communication technology. Specifically, the image information captured by the camera during the positioning process is compared with the pre-collected visual photo database of the positioning venue, and then the location coordinates are determined through a computer vision algorithm.

In practical applications, considering that large buildings such as shopping malls, stadiums, and medical centers have huge indoor spaces, multiple cameras need to be deployed to expand the positioning area. However, since each camera is placed in a different position, the positioning information determined by the image information collected by different cameras is expressed in different coordinate systems, making it impossible to use unified coordinates to represent each position in the indoor space. positioning information.

Contents of the invention

Embodiments of the present application provide a positioning method and related equipment to facilitate determining the positioning coordinates of any position within the camera coverage in a multi-camera scenario.

In the first aspect, this application provides a positioning method. Specifically, when the target object is at the first position, the processing device obtains the first positioning coordinates of the mark point on the target object in the first coordinate system of the first camera, and the second coordinates of the mark point on the target object in the second camera. The second positioning coordinate in the system. When the target object is at the second position, the processing device obtains the third positioning coordinates of the mark point on the target object in the first coordinate system, and the fourth positioning coordinates of the mark point on the target object in the second coordinate system. Wherein, the first position and the second position are both located in the overlapping area of the first camera and the second camera. Afterwards, the processing device determines the first coordinate system transformation parameter between the first coordinate system and the second coordinate system based on the first positioning coordinates, the second positioning coordinates, the third positioning coordinates and the fourth positioning coordinates. Furthermore, the processing device locates the object to be measured according to the first coordinate system transformation parameter.

In this embodiment, this application provides a specific implementation method for external parameter calibration of multiple cameras. The coordinate systems of multiple cameras can be unified through the obtained coordinate system transformation parameters, and the subsequent acquisition is based on each camera. The positioning coordinates obtained can be represented by a unified coordinate system, which facilitates the determination of the positioning coordinates of any position within the camera coverage in a multi-camera scenario.

In some possible implementations, after the processing device acquires the first positioning coordinates and the second positioning coordinates, and before the processing device acquires the third positioning coordinates and the fourth positioning coordinates, the method further includes: the processing device controls the target object to move from the first position. to the second position. Through the above method, this application adopts a fully automated system, which does not require manual intervention and is more practical.

In some possible implementations, the processing device obtaining the first positioning coordinates of the mark point on the target object in the first coordinate system and the second positioning coordinates of the mark point on the target object in the second coordinate system includes: when the target object is in At the first position, the processing device obtains the first imaging information of the marked point on the target object by the first camera and the second imaging information of the marked point on the target object by the second camera, determines the first positioning coordinates according to the first imaging information, and determines the first positioning coordinates according to the first imaging information. The second imaging information determines second positioning coordinates. The processing device obtains the third positioning coordinates of the mark point on the target object in the first coordinate system and the fourth positioning coordinates of the mark point on the target object in the second coordinate system, including: when the target object is at the second position, the processing device obtains The third imaging information of the marked point on the target object by the first camera and the fourth imaging information of the marked point on the target object by the second camera are used to determine the third positioning coordinates based on the third imaging information and the fourth positioning information is determined based on the fourth imaging information. coordinate. In this embodiment, a specific implementation method for obtaining the positioning coordinates of a target object based on a camera is provided, which enhances the realizability of this solution.

In some possible implementations, the first coordinate system transformation parameters include a rotation matrix and a translation vector, so as to accurately obtain the relative positional relationship between the two camera coordinate systems.

[Amended in accordance with Rule 26 24.03.2023]
In some possible implementations, the first positioning coordinates, the second positioning coordinates and the first coordinate system transformation parameters satisfy the formula, and the third positioning coordinates, the fourth positioning coordinates and the first coordinate system transformation parameters satisfy the formula. Formulas include: Among them, θ represents the rotation angle between the first coordinate system and the second coordinate system, represents the rotation matrix, represents the translation vector, Indicates the first positioning coordinate or the third positioning coordinate, Indicates the second positioning coordinate or the fourth positioning coordinate. Through the above method, a specific method for calculating the first coordinate system transformation parameters is provided, which further improves the realizability of this solution.

In some possible implementations, the processing device locating the object to be measured according to the first coordinate system transformation parameters includes: the processing device unifies the first coordinate system and the second coordinate system according to the first coordinate system transformation parameters to obtain the target coordinate system, Obtain target imaging information of the object to be measured by the first camera or the second camera, and determine the target positioning coordinates of the object to be measured in the target coordinate system based on the target imaging information.

In some possible implementations, the processing device unifying the first coordinate system and the second coordinate system according to the first coordinate system transformation parameters to obtain the target coordinate system includes: the processing device converts the second coordinate system according to the first coordinate system transformation parameters. is the first coordinate system, and the target coordinate system is the first coordinate system. Alternatively, the processing device converts the first coordinate system into the second coordinate system according to the first coordinate system transformation parameter, and the target coordinate system is the second coordinate system. In this embodiment, the coordinate system after unification can be either the first coordinate system or the second coordinate system, which improves the flexibility of this solution.

In some possible implementations, the method further includes: when the target object is at the first position, the processing device obtains the fifth positioning coordinates of the marker point on the target object in the third coordinate system of the third camera. The first position is located in a shooting overlap area of the first camera, the second camera and the third camera. When the target object is at the second position, the processing device obtains the sixth positioning coordinates of the mark point on the target object in the third coordinate system. Wherein, the second position is located in the shooting overlap area of the first camera, the second camera and the third camera; the processing device is based on the first positioning coordinates, the second positioning coordinates, the third positioning coordinates, the fourth positioning coordinates, the fifth positioning coordinates and The sixth positioning coordinate determines the second coordinate system transformation parameter between the first coordinate system and the third coordinate system and the third coordinate system transformation parameter between the second coordinate system and the third coordinate system. The processing device unifies the first coordinate system and the third coordinate system according to the transformation parameters of the second coordinate system, and unifies the second coordinate system and the third coordinate system according to the transformation parameters of the third coordinate system. In this embodiment, multiple cameras that have completed external parameter calibration can be combined to perform external parameter calibration on another camera, thereby improving the calibration accuracy.

In some possible implementations, the processing device unifying the first coordinate system and the third coordinate system according to the second coordinate system transformation parameter includes: the processing device converts the third coordinate system into the first coordinate according to the second coordinate system transformation parameter. system, or the processing device converts the first coordinate system into the third coordinate system according to the second coordinate system transformation parameter. The processing device unifying the second coordinate system and the third coordinate system according to the third coordinate system transformation parameter includes: the processing device converts the third coordinate system into the second coordinate system according to the third coordinate system transformation parameter, or the processing device according to the third coordinate system transformation parameter. The three-coordinate system transformation parameter converts the second coordinate system into the third coordinate system. In this implementation, different coordinate systems can be used as a unified coordinate system, which improves the scalability of this solution.

In a second aspect, this application provides a processing device, including: a detection module and a calculation module. The detection module is used to: when the target object is at the first position, obtain the first positioning coordinates of the mark point on the target object in the first coordinate system of the first camera, and the second coordinates of the mark point on the target object in the second camera. The second positioning coordinate in the system. The first position is located in the overlapping area of the first camera and the second camera. When the target object is at the second position, the third positioning coordinates of the mark point on the target object in the first coordinate system and the fourth positioning coordinates of the mark point on the target object in the second coordinate system are obtained. Wherein, the second position is located in the overlapping area of the first camera and the second camera; the calculation module is used to: determine the first coordinate system and the third positioning coordinate according to the first positioning coordinates, the second positioning coordinates, the third positioning coordinates and the fourth positioning coordinates. The first coordinate system transformation parameter between the two coordinate systems is used to position the object to be measured according to the first coordinate system transformation parameter.

In some possible implementations, the processing device further includes a control module. After the detection module obtains the first positioning coordinates and the second positioning coordinates, and before the detection module obtains the third positioning coordinates and the fourth positioning coordinates, the control module is used to: control the target object to move from the first position to the second position.

In some possible implementations, the detection module is specifically configured to: when the target object is at the first position, obtain the first imaging information of the marked point on the target object by the first camera and the third imaging information of the marked point on the target object by the second camera. Two imaging information, determine the first positioning coordinates according to the first imaging information, and determine the second positioning coordinates according to the second imaging information. When the target object is at the second position, the third imaging information of the marked point on the target object by the first camera and the fourth imaging information of the marked point on the target object by the second camera are obtained, and the third positioning coordinates are determined based on the third imaging information. , determine the fourth positioning coordinates according to the fourth imaging information.

In some possible implementations, the first coordinate system transformation parameter includes a rotation matrix and a translation vector.

[Amended in accordance with Rule 26 24.03.2023]
In some possible implementations, the first positioning coordinates, the second positioning coordinates and the first coordinate system transformation parameters satisfy the formula, and the third positioning coordinates, the fourth positioning coordinates and the first coordinate system transformation parameters satisfy the formula. Formulas include: Among them, θ represents the rotation angle between the first coordinate system and the second coordinate system, represents the rotation matrix, represents the translation vector, Indicates the first positioning coordinate or the third positioning coordinate, Indicates the second positioning coordinate or the fourth positioning coordinate.

In some possible implementations, the calculation module is specifically configured to: unify the first coordinate system and the second coordinate system according to the first coordinate system transformation parameters to obtain the target coordinate system, and obtain the image of the object to be measured by the first camera or the second camera. Target imaging information, determine the target positioning coordinates of the object to be measured in the target coordinate system based on the target imaging information.

In some possible implementations, the calculation module is specifically configured to: convert the second coordinate system into the first coordinate system according to the first coordinate system transformation parameter, and the target coordinate system is the first coordinate system. Alternatively, the first coordinate system is converted into the second coordinate system according to the first coordinate system transformation parameter, and the target coordinate system is the second coordinate system.

In some possible implementations, the detection module is further configured to: when the target object is at the first position, obtain the fifth positioning coordinates of the mark point on the target object in the third coordinate system of the third camera. The first position is located in a shooting overlap area of the first camera, the second camera and the third camera. When the target object is at the second position, the sixth positioning coordinates of the mark point on the target object in the third coordinate system are obtained. The second position is located in a shooting overlap area of the first camera, the second camera and the third camera. The calculation module is also used to: determine the distance between the first coordinate system and the third coordinate system based on the first positioning coordinate, the second positioning coordinate, the third positioning coordinate, the fourth positioning coordinate, the fifth positioning coordinate and the sixth positioning coordinate. The second coordinate system transformation parameter and the third coordinate system transformation parameter between the second coordinate system and the third coordinate system are used to unify the first coordinate system and the third coordinate system according to the second coordinate system transformation parameter, and according to the third coordinate system The system transformation parameters unify the second coordinate system and the third coordinate system.

In some possible implementations, the calculation module is specifically configured to: convert the third coordinate system into the first coordinate system according to the second coordinate system transformation parameter, or convert the first coordinate system into the first coordinate system according to the second coordinate system transformation parameter. Three coordinate system. The third coordinate system is converted into the second coordinate system according to the third coordinate system transformation parameter, or the second coordinate system is converted into the third coordinate system according to the third coordinate system transformation parameter.

In a third aspect, this application provides a positioning system, including: a processing device, a first camera, a second camera, and a target object, wherein the processing device is used to perform the positioning method described in any embodiment of the first aspect.

In a fourth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by hardware, it can implement any of the methods executed by the processing device in the first aspect. some or all of the steps.

In the embodiment of the present application, the first camera and the second camera have a shooting overlap area, and the target object can be controlled to move within the shooting overlap area. Specifically, when the target object is at the first position, the positioning coordinates of the mark point on the target object in the coordinate system of the first camera and the coordinate system of the second camera are respectively obtained. When the target object moves to the second position, the positioning coordinates of the mark point on the target object in the coordinate system of the first camera and the coordinate system of the second camera are respectively obtained. Furthermore, the coordinate system transformation parameters between the coordinate system of the first camera and the coordinate system of the second camera are determined based on the obtained multiple positioning coordinates, and the object to be measured is positioned based on the coordinate system transformation parameters. Through the above method, this application provides a specific implementation method for external parameter calibration of multiple cameras. The coordinate systems of multiple cameras can be unified through the obtained coordinate system transformation parameters. Subsequently, based on the obtained coordinate system of each camera, Positioning coordinates can be represented by a unified coordinate system, which makes it easy to determine the positioning coordinates of any position within the camera coverage in a multi-camera scenario.

Description of the drawings

Figure 1 is a schematic diagram of a scene for multi-camera positioning in the embodiment of the present application;

Figure 2 is a schematic diagram of the coordinate systems of different cameras in the embodiment of the present application;

Figure 3 is a schematic diagram of a positioning system provided by an embodiment of the present application;

Figure 4 is a schematic diagram of an embodiment of the positioning method in the embodiment of the present application;

Figure 5 is a schematic diagram of another positioning system provided by an embodiment of the present application;

Figure 6 is a schematic diagram of another embodiment of the positioning method in the embodiment of the present application;

Figure 7 is a schematic structural diagram of a processing device in an embodiment of the present application;

Figure 8 is another structural schematic diagram of a processing device in an embodiment of the present application.

Detailed ways

The embodiments of the present application provide a positioning method and related equipment, which can unify the coordinate systems of multiple cameras according to the coordinate system transformation parameters. Subsequent positioning coordinates obtained based on each camera can be represented by the unified coordinate system, which facilitates In a multi-camera scenario, determine the positioning coordinates of any location within the camera coverage.

Figure 1 is a schematic diagram of a scene for multi-camera positioning in an embodiment of the present application. As shown in Figure 1, camera 1, camera 2 and camera 3 each have their own shooting area. Since the shooting area of a single camera is limited, multiple cameras are usually considered to expand the positioning area. One or more marking points can be installed on the object to be measured to locate the object to be measured through the coordinates of each marking point. The five circular marks on the object to be measured shown in Figure 1 can be regarded as mark points. The relative position relationship between these mark points can be represented by a certain coordinate system. For example, taking one of the mark points as the origin can The coordinates of other marker points are determined, and these coordinates are called the calibration coordinates of the marker points. The camera will capture and image the object to be measured, and the pixel coordinates of each marker point can be extracted based on the captured images. Combining the calibration coordinates of the marker points and the pixel coordinates of the marker points to perform N-point perspective (perspective n point, PnP) calculations can determine the positioning coordinates of each marker point. The positioning coordinates of the marker points specifically refer to the camera as the origin. The coordinates of each marker point in the coordinate system. It should be understood that the positioning information determined by the image information collected by different cameras is expressed in different coordinate systems, which will result in the inability to use unified coordinates to represent the positioning information of each position in the indoor space.

Figure 2 is a schematic diagram of the coordinate systems of different cameras in the embodiment of the present application. As shown in Figure 2, the coordinate system of the camera is the coordinate system with the optical center of the camera as the origin. Taking camera 1 in Figure 2 as an example, the optical center of camera 1 is O ₁ , f ₁ is the focal length of camera 1 , and C ₁ is the optical center on image plane 1 of camera 1 . The two mutually perpendicular directions on the image plane 1 of the camera 1 can be defined as the X ₁ axis and the Y ₁ axis. The direction passing through the optical center O ₁ and perpendicular to the image plane 1 can be positioned as the Z ₁ axis, then O ₁ -X ₁ Y ₁ Z ₁ is the coordinate system of camera 1. In the same way, O ₂ -X ₂ Y ₂ Z ₂ is the coordinate system of camera 2. It should be understood that converting the coordinate system of one camera into the coordinate system of another camera requires determining the coordinate system transformation parameters between the two coordinate systems. This process can be called external parameter calibration of the camera. After completing the external parameter calibration of the camera, the positioning coordinates of multiple cameras can be represented by the same coordinate system. This coordinate system can be called a unified coordinate system.

Figure 3 is a schematic diagram of a positioning system provided by an embodiment of the present application. As shown in Figure 3, the positioning system includes: a processing device, camera 1, camera 2, communication equipment and target objects. Specifically, the processing device is used to send instructions to the target object through the communication device to control the movement of the target object. When the target object is located in the overlapping shooting area of camera 1 and camera 2, camera 1 and camera 2 respectively send the imaging information obtained by shooting the target object to the processing device through the communication device. The processing device can perform analysis and calculation based on the imaging information fed back by camera 1 and camera 2 to determine the coordinate system transformation parameters between the coordinate system of camera 1 and the coordinate system of camera 2, thereby obtaining a unified coordinate system. After that, in the process of positioning the object to be measured, the positioning coordinates of the object to be measured can be expressed according to the unified coordinate system.

It should be understood that the information transmission between the processing device, the camera 1, the camera 2 and the target object can be communicated through a wireless network or a wired network. For example, the communication device can be a router. In practical applications, the communication device and the processing device may be independent of each other or integrated together, and the details are not limited here.

The positioning method provided by this application will be introduced below with reference to specific implementation modes.

Figure 4 is a schematic diagram of a positioning method according to the embodiment of the present application. This method is specifically implemented based on the positioning system introduced in Figure 3 above. In this example, the positioning method includes the following steps.

401. When the target object is at the first position, the processing device obtains the first positioning coordinates of the mark point on the target object in the first coordinate system of camera 1, and the first positioning coordinates of the mark point on the target object in the second coordinate system of camera 2. Second positioning coordinates.

In this embodiment, one or more marking points are provided on the target object, and the first position where the target object stays is in the overlapping area of photography by camera 1 and camera 2 . The camera 1 images the marked points on the target object to obtain first imaging information, and sends the first imaging information to the processing device. The camera 2 images the marked points on the target object to obtain second imaging information, and sends the second imaging information to the processing device. The processing device may determine the first positioning coordinates of the marker point on the target object in the first coordinate system of the camera 1 based on the first imaging information, and determine the location of the marker point on the target object in the second coordinate system of the camera 2 based on the second imaging information. the second positioning coordinates.

402. When the target object is at the second position, the processing device obtains the third positioning coordinates of the mark point on the target object in the first coordinate system of camera 1, and the third positioning coordinates of the mark point on the target object in the second coordinate system of camera 2. The fourth positioning coordinate.

The processing device can control the target object to move from the first position to the second position, and the second position where the target object stays is also in the overlapping area of the camera 1 and the camera 2 . The camera 1 images the marked points on the target object to obtain third imaging information, and sends the third imaging information to the processing device. The camera 2 images the marked point on the target object to obtain fourth imaging information, and sends the fourth imaging information to the processing device. The processing device may determine the third positioning coordinates of the marker point on the target object in the first coordinate system of the camera 1 based on the third imaging information, and determine the location of the marker point on the target object in the second coordinate system of the camera 2 based on the fourth imaging information. The fourth positioning coordinates.

It should be understood that in the above-mentioned steps 401 and 402, the processing device determines the specific implementation method of determining the positioning coordinates of the target object in the camera coordinate system based on the imaging information of the camera. Reference can be made to the relevant introduction in Figure 1 above, which will not be described again here. It should also be understood that when the target object stays at the first position or the second position, the camera 1 and the camera 2 can photograph the target object at the same time, or they can photograph the target object at different times, which is not limited here.

403. The processing device determines the first coordinate system transformation parameter between the first coordinate system and the second coordinate system based on the first positioning coordinates, the second positioning coordinates, the third positioning coordinates and the fourth positioning coordinates.

In this embodiment, the first coordinate system transformation parameters include a rotation matrix (rotation matrix) and a translation vector (translation vector). Among them, the rotation matrix is a matrix that when multiplied by a vector has the effect of changing the direction of the vector but not changing the size and maintaining chirality. For example, if the origins of coordinate system 1 and coordinate system 2 are coincident but not parallel, the transformation of a spatial point from coordinate system 1 to coordinate system 2 is represented by a rotation matrix. If coordinate system 1 and coordinate system 2 are parallel but not coincident, the transformation of a spatial point from coordinate system 1 to coordinate system 2 is represented by a translation vector. In other words, the coordinates of the space point in coordinate system 1 plus the translation vector are the coordinates of the space point in coordinate system 2. Specifically, the processing device may calculate the first coordinate system transformation parameter according to the following formula.

[Amended in accordance with Rule 26 24.03.2023]
The formula is:

[Amended in accordance with Rule 26 24.03.2023]
Among them, θ represents the rotation angle between the first coordinate system and the second coordinate system, represents the rotation matrix, represents the translation vector, Represents the positioning coordinates obtained based on camera 1 (such as the first positioning coordinates or the third positioning coordinates), Indicates the positioning coordinates obtained based on camera 2 (such as the second positioning coordinates or the fourth positioning coordinates). It should be understood that the first positioning coordinates, the second positioning coordinates and the first coordinate system transformation parameters satisfy this formula, and the third positioning coordinates, the fourth positioning coordinates and the first coordinate system transformation parameters also satisfy this formula. That is to say, the first positioning coordinate and the second positioning coordinate are substituted into the formula as one equation, and the third positioning coordinate and the fourth positioning coordinate are substituted into the formula as another equation. The calculation can be obtained by combining the two equations. A coordinate system transformation parameter.

It should be understood that the above-mentioned first positioning coordinates and the second positioning coordinates can be regarded as the first set of positioning information, and the above-mentioned third positioning coordinates and the fourth positioning coordinates can be regarded as the second set of positioning information. In order to calculate the above-mentioned first coordinate system transformation parameters, at least two sets of positioning information are required. In some possible implementations, the processing device can also control the target to move to more locations and collect more sets of positioning information, so as to calculate the first coordinate system transformation parameters based on more sets of positioning information, which can reduce calculation errors. .

404. The processing device locates the object to be measured according to the first coordinate system transformation parameters.

After the processing device calculates the first coordinate system transformation parameters, it first unifies the first coordinate system and the second coordinate system according to the first coordinate system transformation parameters to obtain the target coordinate system. Afterwards, whether the object to be measured is positioned based on camera 1 or camera 2, the positioning coordinates of the object to be measured will be represented by the target coordinate system. Specifically, the processing device can obtain the target imaging information of the object to be measured by the camera 1 or the camera 2, and determine the target positioning coordinates of the object to be measured in the target coordinate system based on the target imaging information.

It should be understood that this application does not specifically limit the target coordinate system. For example, the processing device may convert the second coordinate system into the first coordinate system according to the first coordinate system transformation parameter, and the first coordinate system is the target coordinate system. For another example, the processing device may convert the first coordinate system into the second coordinate system according to the first coordinate system transformation parameter, and the second coordinate system is the target coordinate system. For another example, the target coordinate system can also be a custom coordinate system other than the first coordinate system and the second coordinate system. Of course, the premise is that the relationship between the custom coordinate system and the first coordinate system or the second coordinate system needs to be determined. coordinate system transformation parameters.

It should be noted that the above embodiment introduces the process of external parameter calibration between two cameras. In some possible implementations, multiple cameras that have completed external parameter calibration can also be combined to perform external parameter calibration on another camera, thereby improving the calibration accuracy. Further introduction will be made below on the basis of the above embodiments.

Figure 5 is a schematic diagram of another positioning system provided by an embodiment of the present application. As shown in Figure 5, different from the positioning system shown in Figure 3 above, the positioning system provided in this implementation also includes a camera 3. When the target object is located in the shooting overlap area of camera 1, camera 2 and camera 3, camera 1, camera 2 and camera 3 respectively send the imaging information obtained by shooting the target object to the processing device through the communication device. Among them, the external parameter calibration between camera 1 and camera 2 has been completed according to the method introduced in the embodiment shown in Figure 4 above. The processing device can perform analysis and calculation based on the imaging information fed back by camera 1, camera 2 and camera 3 to determine the coordinate system transformation parameters between the coordinate system of camera 1 and the coordinate system of camera 3 and the coordinate system of camera 2 and the coordinates of camera 3. The coordinate system transformation parameters between the systems are used to obtain a unified coordinate system. After that, in the process of positioning the object to be measured, the positioning coordinates of the object to be measured can be expressed according to the unified coordinate system.

Specifically, the first position and the second position in the embodiment shown in FIG. 4 are also in the shooting area of the camera 3 . When the target object is at the first position, the camera 3 images the marker point on the target object to obtain fifth imaging information, and sends the fifth imaging information to the processing device. The processing device may determine the fifth positioning coordinates of the mark point on the target object in the third coordinate system of the camera 3 based on the fifth imaging information. When the target object is at the second position, the camera 3 images the marker point on the target object to obtain sixth imaging information, and sends the sixth imaging information to the processing device. The processing device may determine the sixth positioning coordinates of the mark point on the target object in the third coordinate system of the camera 3 based on the sixth imaging information. After that, the processing device determines the second position between the first coordinate system and the third coordinate system based on the first positioning coordinate, the second positioning coordinate, the third positioning coordinate, the fourth positioning coordinate, the fifth positioning coordinate and the sixth positioning coordinate. coordinate system transformation parameters and third coordinate system transformation parameters between the second coordinate system and the third coordinate system. Furthermore, the processing device unifies the first coordinate system and the third coordinate system according to the second coordinate system transformation parameter, and unifies the second coordinate system and the third coordinate system according to the third coordinate system transformation parameter.

[Amended in accordance with Rule 26 24.03.2023]
It should be noted that the second coordinate system transformation parameters and the third coordinate system transformation parameters also include rotation matrices and translation vectors. As an example, the second coordinate system transformation parameter and the third coordinate system transformation parameter can be calculated according to the set of equations provided below. The system of equations includes:

[Amended in accordance with Rule 26 24.03.2023]
Among them, the rotation matrix from camera i to camera j is expressed as The rotation angle from camera i to camera j is expressed as θ _i,j , and the translation variable from camera i to camera j is expressed as t _i,j , Represents the positioning coordinates obtained based on camera 1 (such as the first positioning coordinates or the third positioning coordinates), Represents the positioning coordinates obtained based on camera 2 (such as the second positioning coordinates or the fourth positioning coordinates), Indicates the positioning coordinates obtained based on camera 3 (such as the fifth positioning coordinates or the sixth positioning coordinates). By substituting the above positioning coordinates obtained based on camera 1, camera 3 and camera 3 into the set of equations, the second coordinate system transformation parameters and the third coordinate system transformation parameters can be calculated.

It should be noted that the processing device can convert the third coordinate system into the first coordinate system according to the second coordinate system transformation parameter, or convert the first coordinate system into the third coordinate system according to the second coordinate system transformation parameter, thereby completing the process. External parameter calibration between camera 1 and camera 3. In the same way, the processing device can convert the third coordinate system into the second coordinate system according to the third coordinate system transformation parameter, or convert the second coordinate system into the third coordinate system according to the third coordinate system transformation parameter, thereby completing the camera 2 External parameter calibration with camera 3. Since the external parameter calibration between camera 1 and camera 2 has been completed, the external parameter calibration between camera 1, camera 2 and camera 3 can be completed through the above method. In practical applications, the coordinate system of any of the above cameras can be used as a unified coordinate system. After that, no matter which camera is used to locate the object to be measured, the positioning coordinates of the object to be measured will be expressed in the unified coordinate system.

A more specific embodiment will be introduced below, taking the positioning system shown in Figure 3 above as an example.

Figure 6 is a schematic diagram of another embodiment of the positioning method in the embodiment of the present application. In this example, the positioning method includes the following steps.

601. Determine the initial coordinate system.

When the processing device controls the movement of the target object, it will first enter the shooting area of a certain camera. For example, if the camera is camera 1, then the coordinate system of camera 1 can be determined as the initial coordinate system.

602. Determine whether the target object enters the overlapping shooting area of multiple cameras and the external parameters between the multiple cameras are not calibrated. If not, perform step 603; if yes, perform step 604.

603. Control the target to continue moving.

604. Control the target object to stop for a period of time every time it walks a certain distance to allow each camera to image the target object.

It should be understood that the duration of each stop of the target object should be greater than the duration required for camera shooting. For example, if the camera shooting frame rate is 30 frames per second, the duration of each stop of the target object should be greater than 33 milliseconds. Each camera takes a picture of the stopped target and sends the acquired imaging information to the processing device.

605. Determine whether the target object has walked out of the overlapping shooting area of multiple cameras or the number of sets of positioning information has reached the upper limit. If not, continue to step 604; if so, execute step 606.

It should be understood that when the target stays at the same position, the positioning coordinates obtained based on each camera can be used as a set of positioning information. If the number of sets of positioning information reaches the upper limit, it can be considered that the required positioning information is enough, and subsequent operations will be performed. Of course, if the target has walked out of the overlapping shooting area of multiple cameras, it will no longer be able to collect a new set of positioning information, and the following operations will be performed.

606. Determine whether the number of groups of positioning information is greater than or equal to 2. If not, perform step 607; if yes, perform step 608.

It should be understood that according to the above-mentioned introduction to the embodiment shown in Figure 4, at least two sets of positioning information are required to complete the external parameter calibration between multiple cameras. Therefore, the processing device needs to determine whether at least two sets of positioning information have been collected before performing external parameter calibration.

607. Stop this external parameter calibration.

608. Perform external parameter calibration between multiple cameras based on the obtained multiple sets of positioning information, and establish a unified coordinate system to facilitate subsequent positioning operations.

It should be understood that for the specific implementation of step 608, reference can be made to the relevant introduction of steps 403 and 404 in the embodiment shown in FIG. 4, and will not be described again here.

Based on the introduction of the above embodiments, it can be seen that this application provides a specific implementation method for external parameter calibration of multiple cameras. The obtained coordinate system transformation parameters can be used to unify the coordinate systems of multiple cameras. Subsequently, based on each The positioning coordinates obtained by each camera can be represented by a unified coordinate system, which facilitates the determination of the positioning coordinates of any position within the camera coverage in a multi-camera scenario. In addition, the positioning system provided by this application is a fully automated system that does not require manual intervention and is more practical.

The processing device of the above positioning system is introduced below.

Figure 7 is a schematic structural diagram of a processing device in an embodiment of the present application. As shown in Figure 7, the processing device includes a detection module 701, a calculation module 702 and a control module 703. Specifically, the detection module 701 can be used to determine the positioning coordinates of the target object based on the imaging information fed back by each camera. The calculation module 702 is used to calculate coordinate system transformation parameters between cameras and establish a unified coordinate system. The control module 703 is used to control the movement of the target object. In a possible implementation, the detection module 701 is used to perform step 401 and step 402 in the embodiment shown in Figure 4, and the calculation module 702 is used to perform step 403 and step 404 in the embodiment shown in Figure 4. . In a possible implementation, the detection module 701 is used to perform step 602, step 605 and step 606 in the above-mentioned embodiment shown in FIG. 6 . The calculation module 702 is used to execute step 601, step 607 and step 608 in the embodiment shown in FIG. 6 . The control module 703 is used to execute steps 603 and 604 in the embodiment shown in FIG. 6 . It should be understood that the above-mentioned detection module 701, calculation module 702 and control module 703 are only modules divided in terms of functional implementation. In practical applications, the detection module 701, calculation module 702 and control module 703 can be independent modules, or they can integrated together to achieve.

Figure 8 is another structural schematic diagram of a processing device in an embodiment of the present application. As shown in Figure 8, the processing device includes a processor 801, a memory 802 and a transceiver 803. The processor 801, the memory 802 and the transceiver 803 are connected to each other through lines, where the transceiver 803 is used to exchange data or instructions with the camera and the target object. Memory 802 is used to store program instructions and data. The processor 801 is used to perform the operation steps in the embodiment shown in FIG. 4 and FIG. 6 . It should be noted that the processor shown in Figure 8 above can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit ASIC, or at least one integrated circuit for executing related programs. , to implement the technical solutions provided by the embodiments of this application. The memory shown in Figure 8 above can store the operating system and other application programs. When the technical solutions provided by the embodiments of this application are implemented through software or firmware, the program code used to implement the technical solutions provided by the embodiments of this application is stored in the memory and executed by the processor. In one embodiment, the processor may include memory internally. In another embodiment, the processor and memory are two separate structures.

Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be described again here.

Those of ordinary skill in the art can understand that all or part of the steps to implement the above embodiments can be completed by hardware, or can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. The above-mentioned The storage media mentioned may be read-only memory, random access memory, etc. Specifically, for example, the above-mentioned processing unit or processor may be a central processing unit, a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices. , transistor logic devices, hardware components, or any combination thereof. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

When implemented using software, the method steps described in the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, e.g., the computer instructions may be transferred from a website, computer, server, or data center Transmission to another website, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more available media integrated. The available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)), etc.

Claims

A positioning method, characterized by including:

When the target object is at the first position, the processing device obtains the first positioning coordinates of the mark point on the target object in the first coordinate system of the first camera, and the position of the mark point on the target object in the first coordinate system of the second camera. The second positioning coordinates in the second coordinate system, wherein the first position is located in the shooting overlap area of the first camera and the second camera;

When the target object is at the second position, the processing device obtains the third positioning coordinates of the marker point on the target object in the first coordinate system, and the marker point on the target object is at The fourth positioning coordinate in the second coordinate system, wherein the second position is located in the overlapping area of the first camera and the second camera;

The processing device determines a third position between the first coordinate system and the second coordinate system based on the first positioning coordinate, the second positioning coordinate, the third positioning coordinate and the fourth positioning coordinate. a coordinate system transformation parameter;

The processing device locates the object to be measured according to the first coordinate system transformation parameter.
The method according to claim 1, characterized in that, after the processing device obtains the first positioning coordinates and the second positioning coordinates, the processing device obtains the third positioning coordinates and the fourth positioning coordinates. Before coordinates, the method also includes:

The processing device controls the movement of the target object from the first position to the second position.
The method according to claim 1 or 2, characterized in that the processing device obtains the first positioning coordinates of the mark point on the target object in the first coordinate system and the first positioning coordinate of the mark point on the target object. The second positioning coordinates of the marker point in the second coordinate system include:

When the target object is at the first position, the processing device obtains the first imaging information of the marked point on the target object by the first camera and the first imaging information of the mark point on the target object by the second camera. Second imaging information of the mark point, determining the first positioning coordinates according to the first imaging information, and determining the second positioning coordinates according to the second imaging information;

The processing device obtains the third positioning coordinates of the mark point on the target object in the first coordinate system and the fourth positioning coordinates of the mark point on the target object in the second coordinate system. include:

When the target object is at the second position, the processing device acquires the third imaging information of the marked point on the target object by the first camera and the third imaging information of the mark point on the target object by the second camera. The fourth imaging information of the mark point, the third positioning coordinates are determined according to the third imaging information, and the fourth positioning coordinates are determined according to the fourth imaging information.
The method according to any one of claims 1 to 3, characterized in that the first coordinate system transformation parameters include a rotation matrix and a translation vector.
[Amended in accordance with Rule 26 24.03.2023]
The method according to claim 4, characterized in that the first positioning coordinates, the second positioning coordinates and the first coordinate system transformation parameters satisfy the formula, the third positioning coordinates, the fourth positioning coordinates The coordinates and the first coordinate system transformation parameters satisfy the formula;

The formula includes:

Wherein, the θ represents the rotation angle between the first coordinate system and the second coordinate system, and the represents the rotation matrix, the represents the translation vector, the represents the first positioning coordinate or the third positioning coordinate, and the Indicates the second positioning coordinate or the fourth positioning coordinate.
The method according to any one of claims 1 to 5, characterized in that the processing device positioning the object to be measured according to the first coordinate system transformation parameter includes:

The processing device unifies the first coordinate system and the second coordinate system according to the first coordinate system transformation parameters to obtain a target coordinate system, and obtains the first camera or the second camera's response to the target coordinate system. The target imaging information of the object to be measured is determined according to the target imaging information, and the target positioning coordinates of the object to be measured in the target coordinate system are determined.
The method according to claim 6, characterized in that, the processing device unifying the first coordinate system and the second coordinate system according to the first coordinate system transformation parameters to obtain the target coordinate system includes:

The processing device converts the second coordinate system into the first coordinate system according to the first coordinate system transformation parameter, and the target coordinate system is the first coordinate system;

or,

The processing device converts the first coordinate system into the second coordinate system according to the first coordinate system transformation parameter, and the target coordinate system is the second coordinate system.
The method according to any one of claims 1 to 7, characterized in that the method further includes:

When the target object is at the first position, the processing device obtains the fifth positioning coordinates of the mark point on the target object in the third coordinate system of the third camera, wherein the first position Located in the shooting overlap area of the first camera, the second camera and the third camera;

When the target object is at the second position, the processing device obtains the sixth positioning coordinates of the mark point on the target object in the third coordinate system, wherein the second position is located at the second position. The shooting overlap area of the first camera, the second camera and the third camera;

The processing device determines the first positioning coordinate, the second positioning coordinate, the third positioning coordinate, the fourth positioning coordinate, the fifth positioning coordinate and the sixth positioning coordinate. a second coordinate system transformation parameter between the first coordinate system and the third coordinate system and a third coordinate system transformation parameter between the second coordinate system and the third coordinate system;

The processing device unifies the first coordinate system and the third coordinate system according to the second coordinate system transformation parameter, and unifies the second coordinate system and the third coordinate system according to the third coordinate system transformation parameter. The three coordinate systems are unified.
The method according to claim 8, wherein the processing device unifying the first coordinate system and the third coordinate system according to the second coordinate system transformation parameter includes:

The processing device converts the third coordinate system into the first coordinate system according to the second coordinate system transformation parameter, or the processing device converts the first coordinate system into the first coordinate system according to the second coordinate system transformation parameter. The system is converted into the third coordinate system;

The processing device unifying the second coordinate system and the third coordinate system according to the third coordinate system transformation parameter includes:

The processing device converts the third coordinate system into the second coordinate system according to the third coordinate system transformation parameter, or the processing device converts the second coordinate system into the second coordinate system according to the third coordinate system transformation parameter. The coordinate system is converted into the third coordinate system.
A processing device, characterized by comprising: a detection module and a calculation module;

The detection module is configured to: when the target object is at the first position, obtain the first positioning coordinates of the marker point on the target object in the first coordinate system of the first camera, and the marker point on the target object. The second positioning coordinates in the second coordinate system of the second camera, wherein the first position is located in the shooting overlap area of the first camera and the second camera;

When the target object is at the second position, the third positioning coordinates of the marker point on the target object in the first coordinate system are obtained, and the marker point on the target object is at the second position. The fourth positioning coordinate in the coordinate system, wherein the second position is located in the shooting overlap area of the first camera and the second camera;

The calculation module is configured to: determine the relationship between the first coordinate system and the second coordinate system based on the first positioning coordinates, the second positioning coordinates, the third positioning coordinates and the fourth positioning coordinates. The first coordinate system transformation parameter between;

The object to be measured is positioned according to the first coordinate system transformation parameter.
The processing device according to claim 10, characterized in that the processing device further includes a control module; after the detection module obtains the first positioning coordinates and the second positioning coordinates, the detection module obtains the Before the third positioning coordinate and the fourth positioning coordinate, the control module is used to control the target object to move from the first position to the second position.
The processing device according to claim 10 or 11, characterized in that the detection module is specifically used for:

When the target object is at the first position, the first imaging information of the marked point on the target object by the first camera and the third imaging information of the marked point on the target object by the second camera are obtained. two imaging information, determining the first positioning coordinates according to the first imaging information, and determining the second positioning coordinates according to the second imaging information;

When the target object is at the second position, the third imaging information of the marked point on the target object by the first camera and the third imaging information of the marked point on the target object by the second camera are obtained. Four imaging information, the third positioning coordinates are determined according to the third imaging information, and the fourth positioning coordinates are determined according to the fourth imaging information.
The processing device according to any one of claims 10 to 12, wherein the first coordinate system transformation parameter includes a rotation matrix and a translation vector.
[Amended in accordance with Rule 26 24.03.2023]
The processing device according to claim 13, characterized in that the first positioning coordinates, the second positioning coordinates and the first coordinate system transformation parameters satisfy the formula, the third positioning coordinates, the fourth positioning coordinates The positioning coordinates and the first coordinate system transformation parameters satisfy the formula;

The formula includes:

Wherein, the θ represents the rotation angle between the first coordinate system and the second coordinate system, and the represents the rotation matrix, the represents the translation vector, the represents the first positioning coordinate or the third positioning coordinate, and the Indicates the second positioning coordinate or the fourth positioning coordinate.
The processing device according to any one of claims 10 to 14, characterized in that the calculation module is specifically used for:

Unify the first coordinate system and the second coordinate system according to the first coordinate system transformation parameters to obtain a target coordinate system, and obtain the target of the object to be measured by the first camera or the second camera. Imaging information: determine the target positioning coordinates of the object to be measured in the target coordinate system according to the target imaging information.
The processing device according to claim 15, characterized in that the calculation module is specifically used for:

Convert the second coordinate system to the first coordinate system according to the first coordinate system transformation parameter, and the target coordinate system is the first coordinate system;

or,

The first coordinate system is converted into the second coordinate system according to the first coordinate system transformation parameter, and the target coordinate system is the second coordinate system.
The processing device according to any one of claims 10 to 16, characterized in that the detection module is also used for:

When the target object is at the first position, the fifth positioning coordinates of the mark point on the target object in the third coordinate system of the third camera are obtained, wherein the first position is located at the third coordinate system. The shooting overlap area of one camera, the second camera and the third camera;

When the target object is at the second position, obtain the sixth positioning coordinates of the mark point on the target object in the third coordinate system, wherein the second position is located at the first camera , the shooting overlap area of the second camera and the third camera;

The calculation module is also used to: according to the first positioning coordinate, the second positioning coordinate, the third positioning coordinate, the fourth positioning coordinate, the fifth positioning coordinate and the sixth positioning coordinate , determine the second coordinate system transformation parameter between the first coordinate system and the third coordinate system and the third coordinate system transformation parameter between the second coordinate system and the third coordinate system;

The first coordinate system and the third coordinate system are unified according to the second coordinate system transformation parameter, and the second coordinate system and the third coordinate system are unified according to the third coordinate system transformation parameter. Unite.
The processing device according to claim 17, characterized in that the calculation module is specifically used for:

Convert the third coordinate system to the first coordinate system according to the second coordinate system transformation parameter, or convert the first coordinate system to the third coordinate system according to the second coordinate system transformation parameter Tie;

Convert the third coordinate system to the second coordinate system according to the third coordinate system transformation parameter, or convert the second coordinate system to the third coordinate system according to the third coordinate system transformation parameter Tie.
A positioning system, characterized in that the positioning system includes: a processing device, a first camera, a second camera and a target object, wherein the processing device is used to perform as described in any one of claims 1 to 9 Methods.
A computer-readable storage medium, characterized by comprising computer instructions, which when the computer instructions are run on a computer device, cause the computer device to perform the method according to any one of claims 1 to 9.