WO2021139176A1

WO2021139176A1 - Pedestrian trajectory tracking method and apparatus based on binocular camera calibration, computer device, and storage medium

Info

Publication number: WO2021139176A1
Application number: PCT/CN2020/111780
Authority: WO
Inventors: 程晓; 雷晨雨; 张国辉; 宋晨
Original assignee: 平安科技（深圳）有限公司
Priority date: 2020-07-30
Filing date: 2020-08-27
Publication date: 2021-07-15
Also published as: CN111899282A

Abstract

Disclosed in the present application are a pedestrian trajectory tracking method and apparatus based on binocular camera calibration, a computer device, and a storage medium. The present application relates to artificial intelligence and blockchain technology. The method comprises: by means of a calibration object image set, acquiring monocular calibration parameters of a binocular camera so as to carry out binocular correction on a test picture to obtain a left correction picture and a right correction picture, and obtaining a re-projection matrix; by means of a StereoBM algorithm, performing calculation on the left correction picture and the right correction picture to obtain a view difference; acquiring a target image set uploaded by the binocular camera and corresponding to a target to be tracked, calling a trajectory tracking algorithm to acquire target two-dimensional image coordinates of each frame of a target image; and according to a sparse perspective change algorithm and the view difference, correspondingly converting the target two-dimensional image coordinates into corresponding target 3D coordinates to constitute a target 3D coordinate set. A two-dimensional image coordinate system photographed by a binocular camera is converted into a real-world 3D coordinate system, and accurate 3D coordinates of a target pedestrian under the camera can be acquired.

Description

Pedestrian track tracking method, device, computer equipment and storage medium based on binocular camera calibration

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on July 30, 2020, the application number is 202010752907.9, and the invention title is "Pedestrian trajectory tracking method and device based on binocular camera calibration", the entire content of which is incorporated by reference Incorporated in this application.

Technical field

This application relates to the field of artificial intelligence image detection technology, and in particular to a pedestrian trajectory tracking method, device, computer equipment and storage medium based on binocular camera calibration.

Background technique

Visual tracking and target detection are the earliest research directions in the field of computer vision. After decades of accumulation, these two directions have achieved remarkable development, and are widely used in robot navigation, intelligent surveillance video, target behavior analysis, and transportation. Management and security prevention and control and other fields.

The main task of visual tracking and target detection is to locate multiple targets of interest in a given video at the same time, maintain their IDs, and record their trajectories. The goal can be arbitrary, and the most researched one is "pedestrian tracking". Multi-target tracking technology adopts Detection-Based Tracking strategy to perform specific type of target detection or motion detection in a given frame of the video, and then perform sequential or batch tracking, and connect the detection hypothesis to the trajectory, so as to achieve the camera's visual range Multiplayer trajectory tracking.

However, the inventor realized that most of the current tracking and target detection technologies only provide the coordinates in the two-dimensional image taken by the camera, and cannot fully reflect the position of the pedestrian in the real three-dimensional world. For example, when a pedestrian moves longitudinally along the camera shooting direction, the accurate position movement of the pedestrian cannot be judged by the coordinates of the two-dimensional image. In addition, the use of the two-dimensional image coordinates of the camera has very strict requirements on the installation position and angle of the camera, which brings great difficulty to the deployment of the system and reduces the versatility of the system.

Summary of the invention

The embodiments of this application provide a pedestrian trajectory tracking method, device, computer equipment, and storage medium based on binocular camera calibration, aiming to solve the tracking and target detection technology in the prior art, and only provide the two-dimensional image captured by the camera. The coordinates do not fully reflect the position of pedestrians in the real three-dimensional world.

In the first aspect, an embodiment of the present application provides a pedestrian trajectory tracking method based on binocular camera calibration, which includes:

Obtain the single target fixed parameters of the binocular camera through the calibration object image set; wherein, the single target fixed parameters include left camera internal parameters, left camera external parameters, left camera distortion parameters, right camera internal parameters, right camera external parameters, and right camera Distortion parameter

Acquiring a test picture, performing binocular correction on the test picture by using the single target setting parameters to obtain a left-corrected picture and a right-corrected picture, and obtain a reprojection matrix;

Call the pre-stored StereoBM algorithm, and calculate the view difference between the left correction picture and the right correction picture through the StereoBM algorithm;

Obtaining the target image set uploaded by the binocular camera and corresponding to the target to be tracked, and calling a pre-stored trajectory tracking algorithm to obtain the target two-dimensional image coordinates of each frame of the target image in the target image set; and

The target two-dimensional image coordinates of each frame of the target image in the target image set are converted into corresponding target 3D coordinates to form the target 3D according to the called sparse perspective change algorithm and the view difference. Coordinate collection.

In the second aspect, an embodiment of the present application provides a pedestrian trajectory tracking device based on binocular camera calibration, which includes:

The single target setting unit is used to obtain the single target setting parameters of the binocular camera through the calibration object image set; wherein, the single target setting parameters include the left camera internal parameter, the left camera external parameter, the left camera distortion parameter, the right camera internal parameter, and the right camera internal parameter. External camera parameters and distortion parameters of the right camera;

The binocular correction unit is used to obtain a test picture, perform binocular correction on the test picture by using the single target setting parameters to obtain a left-corrected picture and a right-corrected picture, and obtain a reprojection matrix;

A view difference calculation unit, configured to call a pre-stored StereoBM algorithm, and calculate the view difference from the left correction picture and the right correction picture through the StereoBM algorithm;

The target two-dimensional coordinate acquiring unit is used to acquire the target image set uploaded by the binocular camera and corresponding to the target to be tracked, and call a pre-stored trajectory tracking algorithm to acquire the target two-dimensional image coordinates of each frame of target image in the target image set;

The target 3D coordinate collection acquisition unit is used to convert the target two-dimensional image coordinates of each frame of the target image in the target image set into the corresponding two-dimensional image coordinates according to the called sparse perspective change algorithm and the view difference The 3D coordinates of the target to form a set of target 3D coordinates.

In a third aspect, an embodiment of the present application provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and running on the processor, and the processor executes the computer The following steps are implemented during the program:

In a fourth aspect, the embodiments of the present application also provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, which when executed by a processor causes the processor to perform the following operations :

The embodiments of the application provide a pedestrian trajectory tracking method, device, computer equipment, and storage medium based on binocular camera calibration, including obtaining single target parameters of binocular cameras through a calibration object image set; obtaining test pictures through single target Perform binocular correction on the test picture by setting parameters to obtain the left and right correction pictures and get the reprojection matrix; call the StereoBM algorithm, and calculate the left and right correction pictures through the StereoBM algorithm to obtain the view difference; get the binocular camera upload The target image set corresponding to the target to be tracked, the trajectory tracking algorithm is called to obtain the target two-dimensional image coordinates of each frame of the target image in the target image set; the target two-dimensional image coordinates of each frame of the target image in the target image set are changed according to the called sparse perspective The algorithm and the view are different, and the two-dimensional image coordinates of each target are converted into the corresponding target 3D coordinates to form a target 3D coordinate set. The two-dimensional image coordinate system captured by the binocular camera is converted into a real-world 3D coordinate system, and the accurate 3D coordinates of the target pedestrian under the camera can be obtained.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. Ordinary technicians can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic diagram of an application scenario of a pedestrian trajectory tracking method based on binocular camera calibration provided by an embodiment of the application;

2 is a schematic flowchart of a pedestrian trajectory tracking method based on binocular camera calibration according to an embodiment of the application;

3 is a schematic block diagram of a pedestrian trajectory tracking device based on binocular camera calibration provided by an embodiment of the application;

Fig. 4 is a schematic block diagram of a computer device provided by an embodiment of the application.

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

It should be understood that when used in this specification and appended claims, the terms "including" and "including" indicate the existence of the described features, wholes, steps, operations, elements and/or components, but do not exclude one or The existence or addition of multiple other features, wholes, steps, operations, elements, components, and/or collections thereof.

It should also be understood that the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in the specification of this application and the appended claims, unless the context clearly indicates other circumstances, the singular forms "a", "an" and "the" are intended to include plural forms.

It should be further understood that the term "and/or" used in the specification and appended claims of this application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations .

Please refer to Figures 1 and 2. Figure 1 is a schematic diagram of an application scenario of a pedestrian trajectory tracking method based on binocular camera calibration provided by an embodiment of the application; Figure 2 is a pedestrian trajectory tracking based on binocular camera calibration provided by an embodiment of the application A schematic flow chart of the method. The pedestrian trajectory tracking method based on binocular camera calibration is applied to a server, and the method is executed by application software installed in the server.

As shown in Figure 2, the method includes steps S110 to S150.

S110. Obtain the single target parameter of the binocular camera through the calibration object image set; wherein, the single target parameter includes the left camera internal parameter, the left camera external parameter, the left camera distortion parameter, the right camera internal parameter, the right camera external parameter, and Distortion parameter of the right camera.

In this embodiment, through the calibration object image set (the calibration object image collection includes multiple checkerboard pictures in specific implementation, and the corresponding viewing angles of each checkerboard picture are different) through the checkerboard corner detection to calculate the binocular camera The parameters corresponding to the left camera and the parameters corresponding to the right camera. It is specifically calculated that the parameters corresponding to the left camera include the left camera internal parameters, the left camera external parameters, and the left camera distortion parameters; the parameters corresponding to the right camera include the right camera internal parameters, the right camera external parameters, and the right camera distortion parameters.

The process of obtaining the parameters corresponding to the right camera with a single target is the same as obtaining the parameters corresponding to the left camera with a single target. At this time, only the parameters corresponding to the left camera obtained by the single target setting are taken as an example to illustrate the various parameters obtained after the single target setting.

Specifically, the left camera internal parameters include 1/dx, 1/dy, u0, v0, and f; dx represents the length occupied by a pixel in the x direction, dy represents the length occupied by a pixel in the y direction, and u0 represents the center pixel coordinates of the image and the image The number of horizontal pixels that differ between the pixel coordinates of the origin, v0 represents the number of vertical pixels that differ between the center pixel coordinates of the image and the pixel coordinates of the image origin, and f represents the focal length of the left camera.

The left camera external parameters include the rotation matrix R and the translation matrix T from the world coordinate system to the camera coordinate system of the camera.

The left camera distortion parameters include {k1, k2, p1, p2, k3}, where k1, k2, and k3 represent radial distortion coefficients, and p1 and p2 represent tangential distortion coefficients.

In an embodiment, step S110 includes:

Receive the left checkerboard picture set sent by the left camera in the binocular camera, and receive the right checkerboard picture set sent by the right camera; wherein the left checkerboard picture set and the right checkerboard picture set form a calibration object image set, and Each left checkerboard picture in the set of left checkerboard pictures corresponds to a right checkerboard picture in the set of right checkerboard pictures;

Acquiring one of the left checkerboard pictures in the left checkerboard picture set as the target left checkerboard picture, and obtaining the target right checkerboard picture corresponding to the target left checkerboard picture in the right checkerboard picture set;

Calling a pre-stored Harris corner detection function to obtain the Harris corner feature of the left image in the left checkerboard of the target, and obtain the Harris corner feature of the right image in the right checkerboard of the target;

Least square estimation is performed by using the Harris corner feature of the left image and the Harris corner feature of the right image to obtain the single target parameter of the binocular camera.

In this embodiment, when calibrating the left and right cameras in the binocular camera, it is necessary to print 10-20 checkerboard pictures taken from different angles (wherein the checkerboard picture is the surface of the checkerboard and the imaging plane of the camera). The included angle must be less than 45 degrees) for the left camera and right camera to calibrate. In the single target timing of the left camera, a left checkerboard picture is first extracted from the left checkerboard picture set as the target left checkerboard picture, and then the Harris corner detection function is called to detect the target left checkerboard Based on the multiple left image Harris corner features in the grid picture, the least square estimation is performed according to the multiple left image Harris corner features, and the single target parameter of the left camera is obtained. Refer to the single target parameter setting process of the left camera, and similarly, the single target parameter setting of the right camera can also be obtained.

S120. Obtain a test picture, perform binocular correction on the test picture by using the single target setting parameter to obtain a left-corrected picture and a right-corrected picture, and obtain a re-projection matrix.

In this embodiment, the process of performing binocular correction is generally based on the left camera, and then the left camera and the right camera shoot the same object at the same time to obtain the left camera test picture and the right camera test picture. After that, the left camera test picture and the right camera test picture are processed, so that the two pictures finally achieve the following goal: that is, the same object has the same size in the two images and is horizontally in a straight line.

Since the left camera external parameters of the left camera obtained previously include the left rotation matrix R1 (that is, the above-mentioned rotation matrix R) and the left translation matrix T1 (that is, the above-mentioned translation matrix T), the right camera external parameters include The right rotation matrix R2 and the right translation matrix T2. At this time, if the left camera is used as the reference, the right rotation matrix R2 and the right translation matrix T2 can be decomposed into the rotation matrix R21 and R22, and the translation matrix T21 and T22, which are rotated by half of the left and right cameras, by using the cvStereoRectify algorithm of OpenCV. Then calculate the correction lookup mapping table of the left correction picture and the right correction picture to obtain the reprojection matrix Q.

In an embodiment, step S120 includes:

Linearly transform the image coordinates of each pixel in the test picture according to the left camera internal parameters and the right camera internal parameters to obtain the left actual imaging plane coordinates of each pixel and the right actual imaging plane coordinates of each pixel;

The left actual imaging plane coordinates of each pixel are converted according to the left camera distortion parameters to obtain the left ideal plane imaging coordinates of each pixel, and the right actual imaging plane coordinates of each pixel are converted according to the right camera distortion parameters. Obtain the right ideal plane imaging coordinates of each pixel;

Perform perspective projection transformation on the left ideal plane imaging coordinates of each pixel according to the left camera internal parameters to obtain the left camera 3D coordinates of each pixel, and perform perspective projection transformation on the right ideal plane imaging coordinates of each pixel according to the right camera internal parameters to obtain 3D coordinates of the right camera of each pixel;

The left camera 3D coordinates of each pixel are rigid body converted according to the left camera external parameters to obtain the left actual 3D coordinates of each pixel, and the right camera 3D coordinates of each pixel are rigid body converted according to the right camera external parameters to obtain each pixel The actual right 3D coordinates of the point;

Obtain a left correction picture according to the left actual 3D coordinates of each pixel, and obtain a right correction picture according to the right actual 3D coordinates of each pixel;

According to the mapping relationship between the left actual 3D coordinates of each pixel and the right actual 3D coordinates of each pixel, the reprojection matrix is obtained.

In this embodiment, the essence of the binocular correction of the left and right cameras is to convert the picture from the image pixel coordinate system to the actual imaging plane coordinate, and then from the actual imaging plane coordinate to the ideal plane imaging coordinate system, and then from the ideal plane imaging coordinate system. The system is converted to the camera 3D coordinate system, and finally from the camera 3D coordinate system to the actual 3D coordinate system, the left correction image is obtained according to the left actual 3D coordinates of each pixel, and the right correction image is obtained according to the right actual 3D coordinates of each pixel According to the mapping relationship between the left actual 3D coordinates of each pixel and the right actual 3D coordinates of each pixel, the reprojection matrix is finally obtained.

S130. Invoke a pre-stored StereoBM algorithm, and calculate the view difference between the left correction picture and the right correction picture through the StereoBM algorithm.

In this embodiment, if the feature of a point on a picture is matched to a corresponding point on another two-dimensional image space, this process will be very time-consuming. In order to reduce the computational complexity of matching search, epipolar constraints are used to reduce the matching of corresponding points from a two-dimensional search space to a one-dimensional search space. At this time, the StereoBM algorithm of OpenCV can be used to calculate the left-corrected picture and the right-corrected picture to obtain a disparity map.

In an embodiment, step S130 includes:

Performing single-channel grayscale conversion on the left-corrected picture to obtain a left single-channel grayscale image;

Performing single-channel grayscale conversion on the right-corrected picture to obtain a right single-channel grayscale image;

Call the preset disparity search range and sliding window size in the StereoBM algorithm, and use the left single-channel grayscale image, right single-channel grayscale image, disparity search range, and sliding window size as input parameters of the StereoBM algorithm Calculate to get the view difference.

In this embodiment, the left correction picture needs to be read first and converted into a left single-channel grayscale image. For example, the name of the left correction picture is zjztp1.jpg. Specifically, the left correction picture is first read through the cv2.imread() instruction of OpenCV. The correction picture zjztp1.jpg, that is, imgL=cv2.imread('zjztp1.jpg'); then the left correction picture is converted into a left single-channel grayscale image through OpenCV's cv2.cvtColor() command, that is, imgLG=cv2. cvtColor(imgL, cv2.COLOR_BGR2GRAY), where imgLG represents the left single-channel grayscale image.

When converting the right-corrected picture into a right single-channel grayscale image, it is necessary to read the right-corrected picture and convert it into a right single-channel grayscale image. For example, the name of the right-corrected picture is yjztp1.jpg. Read the right correction picture yjztp1.jpg through OpenCV's cv2.imread() instruction, that is, imgR=cv2.imread('yjztp1.jpg'); then use OpenCV's cv2.cvtColor() instruction to convert the right correction picture into The right single-channel grayscale image, that is, imgRG=cv2.cvtColor(imgR, cv2.COLOR_BGR2GRAY), where imgRG represents the right single-channel grayscale image. Among them, the cv2.imread() instruction of OpenCV is an image reading instruction, and the cv2.cvtColor() instruction of OpenCV is an image graying instruction.

After obtaining the left single-channel grayscale image imgLG and the right single-channel grayscale image imgRG, the StereoBM algorithm of OpenCV stereo=cv2.StereoBM_create(numDisparities=16*9, blocksize=45), disp=stereo.compute(imgLG, imgRG) is calculated, and the view difference is obtained; the cv2.StereoBM_create instruction in the StereoBM algorithm of OpenCV is the disparity search range and sliding window size call instruction, and the disp=stereo.compute(imgLG, imgRG) in the StereoBM algorithm of OpenCV is the view difference Calculation instructions.

S140: Obtain a target image set uploaded by the binocular camera and corresponding to the target to be tracked, and call a pre-stored trajectory tracking algorithm to obtain the target two-dimensional image coordinates of each frame of the target image in the target image set.

In this embodiment, in order to track the pedestrian route, the pre-stored trajectory tracking algorithm can be called at this time to obtain the target two-dimensional image coordinates of each frame of the target image in the target image set corresponding to the target to be tracked uploaded by the binocular camera.

Wherein, the specific implementation of the trajectory tracking algorithm adopts a multi-target tracking algorithm. In order to have a detailed understanding of the multi-target tracking algorithm, the multi-target tracking algorithm will be introduced below.

The problem of Multiple Object Tracking (MOT) is raised: There is a video, and the video is composed of N consecutive frames. From the first frame to the last frame, there are multiple targets inside, constantly moving in and out. The purpose of multi-target tracking is to distinguish each target from other targets and track its trajectory in different frames. The most classic application of multi-target tracking is to monitor pedestrians at intersections.

In fact, the multi-target tracking problem can be understood as a multi-variable estimation problem, and we give its formal definition. Given a sequence of images,

Represents the state of the i-th target in the t-th frame,

Represents the state sequence _{of all targets M t} in the t-th frame,

Represents the state sequence of the i-th target, where i _s and i _c represent the first and last image of the target i, respectively, S _1:t = {S ₁ ,S ₂ ,...,S _t } means all The state sequence of the target from frame 1 to frame t. It should be noted that the ID of the target in each frame may be different. Correspondingly, under the most commonly used tracking-by-detection structure,

Represents the i-th observation target in the t-th frame,

Indicates the observation target _{of all targets M t} in the t-th frame _{, O 1:t} ={O ₁ ,O ₂ ,...,O _t } represents the observation target sequence of all targets from the first frame to the t-th frame. The purpose of multi-target tracking is to find the best state sequence of all targets. The conditional distribution on the state sequence of all observed targets can be obtained by generalization modeling using the MAP (maximal a posteriori) estimation method:

Through the Kalman filter method based on probability prediction, the solution of the model corresponding to equation (1) can be calculated to obtain the target two-dimensional image coordinates of the target image of each frame.

S150. The target two-dimensional image coordinates of each frame of the target image in the target image set are converted into corresponding target 3D coordinates according to the called sparse perspective change algorithm and the view difference, so as to form The target 3D coordinate collection.

In this embodiment, the target two-dimensional image coordinates output by the trajectory tracking algorithm are converted. Specifically, through the above-mentioned disparity map, the two-dimensional points are reprojected to the three-dimensional reprojection matrix Q, and the cvPerspectiveTransform algorithm (ie sparse perspective change algorithm) of OpenCV is used to convert the two-dimensional image coordinates of each target into the corresponding target 3D Coordinates to form a set of target 3D coordinates. This application can be applied to smart city management/smart transportation scenarios to promote the construction of smart cities. Moreover, after obtaining the 3D coordinates of each target pedestrian, it can be used to draw a pedestrian trajectory map, accurately calculate the distance moved by the target pedestrian, and accurately calculate the distance between the target pedestrian and the target object.

In an embodiment, after step S150, the method further includes:

Upload the target 3D coordinate set to the blockchain network.

In this embodiment, the server can be used as a blockchain node device to upload the target 3D coordinate set to the blockchain network, making full use of the non-tamperable characteristics of the blockchain data to achieve solidified storage of pedestrian trajectory data.

Wherein, the corresponding summary information is obtained based on the target 3D coordinate set. Specifically, the summary information is obtained by hashing the target 3D coordinate set, for example by using the sha256 algorithm. Uploading summary information to the blockchain can ensure its security and fairness and transparency to users. The server can download the summary information from the blockchain to verify whether the target 3D coordinate set has been tampered with. The blockchain referred to in this example is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information for verification. The validity of the information (anti-counterfeiting) and generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

This method realizes the conversion of the two-dimensional image coordinate system taken by the binocular camera into the real-world 3D coordinate system, and can obtain the accurate 3D coordinates of the target pedestrian under the camera.

The embodiment of the present application also provides a pedestrian trajectory tracking device based on binocular camera calibration. The pedestrian trajectory tracking device based on binocular camera calibration is used to implement any embodiment of the aforementioned pedestrian trajectory tracking method based on binocular camera calibration. Specifically, please refer to FIG. 3, which is a schematic block diagram of a pedestrian trajectory tracking device based on binocular camera calibration provided by an embodiment of the present application. The pedestrian trajectory tracking device 100 based on binocular camera calibration can be configured in a server.

As shown in FIG. 3, the pedestrian trajectory tracking device 100 based on binocular camera calibration includes: a single target positioning unit 110, a binocular correction unit 120, a view difference calculation unit 130, a target two-dimensional coordinate acquisition unit 140, and a target 3D coordinate collection acquisition Unit 150.

The single target setting unit 110 is used to obtain the single target setting parameters of the binocular camera through the calibration object image set; wherein, the single target setting parameters include left camera internal parameters, left camera external parameters, left camera distortion parameters, right camera internal parameters, External parameters of the right camera and distortion parameters of the right camera.

In an embodiment, the single target setting unit 110 includes:

The calibration object image set acquisition unit is used to receive the left checkerboard picture set sent by the left camera in the binocular camera, and the right checkerboard picture set sent by the right camera; wherein, the left checkerboard picture set and the right checkerboard picture set The set forms a calibration object image set, and each left checkerboard picture in the left checkerboard picture set corresponds to a right checkerboard picture in the right checkerboard picture set;

The target checkerboard picture obtaining unit is used to obtain one of the left checkerboard pictures in the left checkerboard picture set as the target left checkerboard picture, and to obtain the right checkerboard picture set corresponding to the target left checkerboard picture The right checkerboard picture of the target;

Harris corner point feature detection unit, used to call a pre-stored Harris corner point detection function to obtain the Harris corner point feature of the left image in the left checkerboard of the target, and obtain the right image in the right checkerboard of the target Harris corner features;

The least squares estimation unit is used to perform least squares estimation by using the Harris corner feature of the left image and the Harris corner feature of the right image to obtain the single target parameter of the binocular camera.

In this embodiment, when calibrating the left and right cameras in the binocular camera, it is necessary to print 10-20 checkerboard pictures taken from different angles (wherein the checkerboard picture is the surface of the checkerboard and the imaging plane of the camera). The included angle must be less than 45 degrees) for the left camera and right camera to calibrate. In the single target timing of the left camera, a left checkerboard picture is first extracted from the left checkerboard picture set as the target left checkerboard picture, and then the Harris corner detection function is called to detect the target left checkerboard The multiple left image Harris corner features in the grid picture, and finally the least square estimation is performed according to the multiple left image Harris corner features, and the single target parameter of the left camera is obtained. Refer to the single target parameter setting process of the left camera, and similarly, the single target parameter setting of the right camera can be obtained.

The binocular correction unit 120 is configured to obtain a test picture, perform binocular correction on the test picture by using the single target parameter to obtain a left-corrected picture and a right-corrected picture, and obtain a reprojection matrix.

Since the left camera external parameters of the left camera previously obtained include the left rotation matrix R1 (that is, the above-mentioned rotation matrix R) and the left translation matrix T1 (that is, the above-mentioned translation matrix T), the right camera external parameters include The right rotation matrix R2 and the right translation matrix T2. At this time, if the left camera is used as the reference, the right rotation matrix R2 and the right translation matrix T2 can be decomposed into the rotation matrix R21 and R22, and the translation matrix T21 and T22, which are rotated by half of the left and right cameras, by using the cvStereoRectify algorithm of OpenCV. Then calculate the correction lookup mapping table of the left correction picture and the right correction picture to obtain the reprojection matrix Q.

In an embodiment, the binocular correction unit 120 includes:

The first conversion unit is used for linearly converting the image coordinates of each pixel in the test picture according to the left camera internal parameters and the right camera internal parameters, to obtain the left actual imaging plane coordinates of each pixel, and to obtain the right of each pixel. Actual imaging plane coordinates;

The second conversion unit is used to convert the left actual imaging plane coordinates of each pixel according to the left camera distortion parameters to obtain the left ideal plane imaging coordinates of each pixel, and to calculate the right actual imaging plane coordinates of each pixel according to the right The camera distortion parameter performs coordinate conversion to obtain the right ideal plane imaging coordinates of each pixel;

The third conversion unit is used to perform perspective projection transformation of the left ideal plane imaging coordinates of each pixel point according to the left camera internal parameters to obtain the left camera 3D coordinates of each pixel point, and the right ideal plane imaging coordinates of each pixel point according to the right camera The internal parameter performs perspective projection transformation to obtain the right camera 3D coordinates of each pixel;

The fourth conversion unit is used to perform rigid body conversion of the left camera 3D coordinates of each pixel according to the external parameters of the left camera to obtain the actual left 3D coordinates of each pixel, and the 3D coordinates of the right camera of each pixel according to the external parameters of the right camera Perform rigid body transformation to obtain the actual right 3D coordinates of each pixel;

The correction picture acquisition unit is configured to obtain a left correction picture according to the left actual 3D coordinates of each pixel, and obtain a right correction picture according to the right actual 3D coordinates of each pixel;

The re-projection matrix obtaining unit is configured to obtain the re-projection matrix according to the mapping relationship between the left actual 3D coordinates of each pixel and the right actual 3D coordinates of each pixel.

The view difference calculation unit 130 is configured to call a pre-stored StereoBM algorithm, and calculate the view difference from the left correction picture and the right correction picture through the StereoBM algorithm.

In an embodiment, the view difference calculation unit 130 includes:

The first gray scale conversion unit is configured to perform single-channel gray scale conversion on the left corrected picture to obtain a left single-channel gray scale image;

The second gray scale conversion unit is configured to perform single-channel gray scale conversion on the right corrected picture to obtain a right single-channel gray scale image;

The view difference acquisition unit is used to call the preset disparity search range and sliding window size in the StereoBM algorithm, and use the left single-channel grayscale image, right single-channel grayscale image, disparity search range, and sliding window size as all The input parameters of the StereoBM algorithm are calculated to obtain the view difference.

When converting the right-corrected picture into a right single-channel grayscale image, it is necessary to read the right-corrected picture and convert it into a right single-channel grayscale image. For example, the name of the right-corrected picture is yjztp1.jpg. Read the right correction picture yjztp1.jpg through OpenCV's cv2.imread() instruction, that is, imgR=cv2.imread('yjztp1.jpg'); then use OpenCV's cv2.cvtColor() instruction to convert the right correction picture into The right single-channel grayscale image, that is, imgRG=cv2.cvtColor(imgR, cv2.COLOR_BGR2GRAY), where imgRG represents the right single-channel grayscale image.

After obtaining the left single-channel grayscale image imgLG and the right single-channel grayscale image imgRG, the StereoBM algorithm of OpenCV stereo=cv2.StereoBM_create(numDisparities=16*9, blocksize=45), disp=stereo.compute (imgLG, imgRG) After calculation, the view difference is obtained.

The target two-dimensional coordinate acquiring unit 140 is configured to acquire the target image set uploaded by the binocular camera and corresponding to the target to be tracked, and call a pre-stored trajectory tracking algorithm to acquire the target two-dimensional image coordinates of each frame of the target image in the target image set.

Represents the state of the i-th target in the t-th frame,

Represents the state sequence _{of all targets M t} in the t-th frame,

Represents the i-th observation target in the t-th frame,

Indicates the observation target _{of all targets M t} in the t-th frame _{, O 1:t} ={O ₁ ,O ₂ ,...,O _t } represents the observation target sequence of all targets from the first frame to the t-th frame. The purpose of multi-target tracking is to find the best state sequence of all targets. On the conditional distribution of the state sequence of all observed targets, the above formula (1) can be obtained by generalization modeling using MAP (maximal a posteriori) estimation method By using the Kalman filter method based on probability prediction, the solution of the model corresponding to formula (1) can be calculated to obtain the target two-dimensional image coordinates of the target image of each frame.

The target 3D coordinate set acquisition unit 150 is configured to convert the target two-dimensional image coordinates of each frame of the target image in the target image set according to the called sparse perspective change algorithm and the view difference to convert each target two-dimensional image coordinate into Corresponding target 3D coordinates to form a target 3D coordinate set.

In this embodiment, the target two-dimensional image coordinates output by the trajectory tracking algorithm are converted. Specifically, through the above-mentioned disparity map, the two-dimensional points are reprojected to the three-dimensional reprojection matrix Q, and the cvPerspectiveTransform algorithm (ie sparse perspective change algorithm) of OpenCV is used to convert the two-dimensional image coordinates of each target into the corresponding target 3D Coordinates to form a set of target 3D coordinates.

In an embodiment, the pedestrian trajectory tracking device 100 based on binocular camera calibration further includes:

The data link unit is used to upload the target 3D coordinate set to the blockchain network.

The device realizes the conversion of the two-dimensional image coordinate system captured by the binocular camera into the real-world 3D coordinate system, and can obtain the accurate 3D coordinates of the target pedestrian under the camera.

The aforementioned pedestrian trajectory tracking device based on binocular camera calibration can be implemented in the form of a computer program, and the computer program can be run on a computer device as shown in FIG. 4.

Please refer to FIG. 4, which is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device 500 is a server, and the server may be an independent server or a server cluster composed of multiple servers.

Referring to FIG. 4, the computer device 500 includes a processor 502, a memory, and a network interface 505 connected through a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.

The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 can execute a pedestrian trajectory tracking method based on binocular camera calibration.

The processor 502 is used to provide calculation and control capabilities, and support the operation of the entire computer device 500.

The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute the pedestrian trajectory tracking method based on binocular camera calibration.

The network interface 505 is used for network communication, such as providing data information transmission. Those skilled in the art can understand that the structure shown in FIG. 4 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

The processor 502 is configured to run a computer program 5032 stored in a memory to implement the pedestrian trajectory tracking method based on binocular camera calibration disclosed in the embodiment of the present application.

Those skilled in the art can understand that the embodiment of the computer device shown in FIG. 4 does not constitute a limitation on the specific configuration of the computer device. In other embodiments, the computer device may include more or less components than those shown in the figure. Or some parts are combined, or different parts are arranged. For example, in some embodiments, the computer device may only include a memory and a processor. In such embodiments, the structures and functions of the memory and the processor are the same as those of the embodiment shown in FIG. 4, and will not be repeated here.

It should be understood that in this embodiment of the application, the processor 502 may be a central processing unit (Central Processing Unit, CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor.

In another embodiment of the present application, a computer-readable storage medium is provided. The computer-readable storage medium may be non-volatile or volatile. The computer-readable storage medium stores a computer program, where the computer program is executed by a processor to implement the pedestrian trajectory tracking method based on binocular camera calibration disclosed in the embodiments of the present application.

Those skilled in the art can clearly understand that, for the convenience and conciseness of description, the specific working process of the above-described equipment, device, and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here. A person of ordinary skill in the art may be aware that the units and algorithm steps of the examples described in the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both, in order to clearly illustrate the hardware and software Interchangeability, in the above description, the composition and steps of each example have been generally described in accordance with the function. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.

In the several embodiments provided in this application, it should be understood that the disclosed equipment, device, and method may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, or the units with the same function may be combined into one. Units, for example, multiple units or components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may also be electrical, mechanical or other forms of connection.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments of the present application.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this application is essentially or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium. It includes several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), magnetic disk or optical disk and other media that can store program codes.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Anyone familiar with the technical field can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

A pedestrian trajectory tracking method based on binocular camera calibration, which includes:

Obtain the single target fixed parameters of the binocular camera through the calibration object image set; wherein, the single target fixed parameters include left camera internal parameters, left camera external parameters, left camera distortion parameters, right camera internal parameters, right camera external parameters, and right camera Distortion parameter

Acquiring a test picture, performing binocular correction on the test picture by using the single target setting parameters to obtain a left-corrected picture and a right-corrected picture, and obtain a reprojection matrix;

Call the pre-stored StereoBM algorithm, and calculate the view difference between the left correction picture and the right correction picture through the StereoBM algorithm;

Obtaining the target image set uploaded by the binocular camera and corresponding to the target to be tracked, and calling a pre-stored trajectory tracking algorithm to obtain the target two-dimensional image coordinates of each frame of the target image in the target image set; and

The target two-dimensional image coordinates of each frame of the target image in the target image set are converted into corresponding target 3D coordinates to form the target 3D according to the called sparse perspective change algorithm and the view difference. Coordinate collection.
The pedestrian trajectory tracking method based on binocular camera calibration according to claim 1, wherein said obtaining the single target fixed parameters of the binocular camera through the calibration object image set comprises;

Receive the left checkerboard picture set sent by the left camera in the binocular camera, and receive the right checkerboard picture set sent by the right camera; wherein the left checkerboard picture set and the right checkerboard picture set form a calibration object image set, and Each left checkerboard picture in the set of left checkerboard pictures corresponds to a right checkerboard picture in the set of right checkerboard pictures;

Acquiring one of the left checkerboard pictures in the left checkerboard picture set as the target left checkerboard picture, and obtaining the target right checkerboard picture corresponding to the target left checkerboard picture in the right checkerboard picture set;

Calling a pre-stored Harris corner detection function to obtain the Harris corner feature of the left image in the left checkerboard of the target, and obtain the Harris corner feature of the right image in the right checkerboard of the target;

Least square estimation is performed by using the Harris corner feature of the left image and the Harris corner feature of the right image to obtain the single target parameter of the binocular camera.
The pedestrian trajectory tracking method based on binocular camera calibration according to claim 1, wherein said acquiring a test picture performs binocular correction on said test picture by setting parameters of said single target to obtain a left correction picture and a right correction Picture, and get the reprojection matrix, including:

Linearly transform the image coordinates of each pixel in the test picture according to the left camera internal parameters and the right camera internal parameters to obtain the left actual imaging plane coordinates of each pixel and the right actual imaging plane coordinates of each pixel;

The left actual imaging plane coordinates of each pixel are converted according to the left camera distortion parameters to obtain the left ideal plane imaging coordinates of each pixel, and the right actual imaging plane coordinates of each pixel are converted according to the right camera distortion parameters. Obtain the right ideal plane imaging coordinates of each pixel;

Perform perspective projection transformation on the left ideal plane imaging coordinates of each pixel according to the left camera internal parameters to obtain the left camera 3D coordinates of each pixel, and perform perspective projection transformation on the right ideal plane imaging coordinates of each pixel according to the right camera internal parameters to obtain 3D coordinates of the right camera of each pixel;

The left camera 3D coordinates of each pixel are rigid body converted according to the left camera external parameters to obtain the left actual 3D coordinates of each pixel, and the right camera 3D coordinates of each pixel are rigid body converted according to the right camera external parameters to obtain each pixel The actual 3D coordinates of the right of the point;

Obtain a left correction picture according to the left actual 3D coordinates of each pixel, and obtain a right correction picture according to the right actual 3D coordinates of each pixel;

According to the mapping relationship between the left actual 3D coordinates of each pixel and the right actual 3D coordinates of each pixel, the reprojection matrix is obtained.
The pedestrian trajectory tracking method based on binocular camera calibration according to claim 1, wherein said calculating said left correction picture and said right correction picture through said StereoBM algorithm to obtain a view difference, comprises:

Performing single-channel grayscale conversion on the left-corrected picture to obtain a left single-channel grayscale image;

Performing single-channel grayscale conversion on the right-corrected picture to obtain a right single-channel grayscale image;

Call the preset disparity search range and sliding window size in the StereoBM algorithm, and use the left single-channel grayscale image, right single-channel grayscale image, disparity search range, and sliding window size as input parameters of the StereoBM algorithm Calculate to get the view difference.
The pedestrian trajectory tracking method based on binocular camera calibration according to claim 1, wherein the invoking a pre-stored trajectory tracking algorithm to obtain the target two-dimensional image coordinates of each frame of the target image in the target image set comprises:

The multi-target tracking algorithm corresponding to the trajectory tracking algorithm is called to obtain the target two-dimensional image coordinates of each frame of the target image in the target image set.
The pedestrian trajectory tracking method based on binocular camera calibration according to claim 1, further comprising:

Upload the target 3D coordinate set to the blockchain network.
The pedestrian trajectory tracking method based on binocular camera calibration according to claim 4, wherein the single-channel grayscale conversion of the left correction picture to obtain a left single-channel grayscale image comprises:

The left correction picture is read through the OpenCV picture reading instruction, and the left correction picture is converted into a left single-channel grayscale image through the OpenCV picture grayscale instruction instruction.
The pedestrian trajectory tracking method based on binocular camera calibration according to claim 4, wherein said calling the preset disparity search range and sliding window size in the StereoBM algorithm, the left single-channel grayscale image, right The single-channel grayscale image, the disparity search range and the sliding window size are calculated as the input parameters of the StereoBM algorithm to obtain the view difference, including:

The disparity search range and the sliding window size are called by the disparity search range and sliding window size call instructions in the StereoBM algorithm of OpenCV, and the view difference is calculated by the view difference calculation instructions in the StereoBM algorithm of OpenCV; the disparity search range is 16*9, sliding The window size is 45.
A pedestrian trajectory tracking device based on binocular camera calibration, which includes:

The single target setting unit is used to obtain the single target setting parameters of the binocular camera through the calibration object image set; wherein, the single target setting parameters include the left camera internal parameter, the left camera external parameter, the left camera distortion parameter, the right camera internal parameter, and the right camera internal parameter. External camera parameters and distortion parameters of the right camera;

The binocular correction unit is used to obtain a test picture, perform binocular correction on the test picture by using the single target setting parameters to obtain a left-corrected picture and a right-corrected picture, and obtain a reprojection matrix;

A view difference calculation unit, configured to call a pre-stored StereoBM algorithm, and calculate the view difference from the left correction picture and the right correction picture through the StereoBM algorithm;

The target two-dimensional coordinate acquiring unit is used to acquire the target image set uploaded by the binocular camera and corresponding to the target to be tracked, and call a pre-stored trajectory tracking algorithm to acquire the target two-dimensional image coordinates of each frame of target image in the target image set;

The target 3D coordinate collection acquisition unit is used to convert the target two-dimensional image coordinates of each frame of the target image in the target image set into the corresponding two-dimensional image coordinates according to the called sparse perspective change algorithm and the view difference The 3D coordinates of the target to form a set of target 3D coordinates.
A computer device includes a memory, a processor, and a computer program that is stored on the memory and can run on the processor, wherein the processor implements the following steps when the processor executes the computer program:

Obtain the single target fixed parameters of the binocular camera through the calibration object image set; wherein, the single target fixed parameters include left camera internal parameters, left camera external parameters, left camera distortion parameters, right camera internal parameters, right camera external parameters, and right camera Distortion parameter

Acquiring a test picture, performing binocular correction on the test picture by using the single target setting parameters to obtain a left-corrected picture and a right-corrected picture, and obtain a reprojection matrix;

Call the pre-stored StereoBM algorithm, and calculate the view difference between the left correction picture and the right correction picture through the StereoBM algorithm;

Obtaining the target image set uploaded by the binocular camera and corresponding to the target to be tracked, and calling a pre-stored trajectory tracking algorithm to obtain the target two-dimensional image coordinates of each frame of the target image in the target image set; and

The target two-dimensional image coordinates of each frame of the target image in the target image set are converted into corresponding target 3D coordinates to form the target 3D according to the called sparse perspective change algorithm and the view difference. Coordinate collection.
The computer device according to claim 10, wherein said obtaining the single target fixed parameter of the binocular camera through the calibration object image set comprises;

Receive the left checkerboard picture set sent by the left camera in the binocular camera, and receive the right checkerboard picture set sent by the right camera; wherein the left checkerboard picture set and the right checkerboard picture set form a calibration object image set, and Each left checkerboard picture in the set of left checkerboard pictures corresponds to a right checkerboard picture in the set of right checkerboard pictures;

Acquiring one of the left checkerboard pictures in the left checkerboard picture set as the target left checkerboard picture, and obtaining the target right checkerboard picture corresponding to the target left checkerboard picture in the right checkerboard picture set;

Calling a pre-stored Harris corner detection function to obtain the Harris corner feature of the left image in the left checkerboard of the target, and obtain the Harris corner feature of the right image in the right checkerboard of the target;

Least square estimation is performed by using the Harris corner feature of the left image and the Harris corner feature of the right image to obtain the single target parameter of the binocular camera.
10. The computer device according to claim 10, wherein said acquiring a test picture, performing binocular correction on the test picture by using the single target parameter to obtain a left-corrected picture and a right-corrected picture, and obtaining a reprojection matrix, include:

Linearly transform the image coordinates of each pixel in the test picture according to the left camera internal parameters and the right camera internal parameters to obtain the left actual imaging plane coordinates of each pixel and the right actual imaging plane coordinates of each pixel;

The left actual imaging plane coordinates of each pixel are converted according to the left camera distortion parameters to obtain the left ideal plane imaging coordinates of each pixel, and the right actual imaging plane coordinates of each pixel are converted according to the right camera distortion parameters. Obtain the right ideal plane imaging coordinates of each pixel;

Perform perspective projection transformation on the left ideal plane imaging coordinates of each pixel according to the left camera internal parameters to obtain the left camera 3D coordinates of each pixel, and perform perspective projection transformation on the right ideal plane imaging coordinates of each pixel according to the right camera internal parameters to obtain 3D coordinates of the right camera of each pixel;

The left camera 3D coordinates of each pixel are rigid body converted according to the left camera external parameters to obtain the left actual 3D coordinates of each pixel, and the right camera 3D coordinates of each pixel are rigid body converted according to the right camera external parameters to obtain each pixel The actual right 3D coordinates of the point;

Obtain a left correction picture according to the left actual 3D coordinates of each pixel, and obtain a right correction picture according to the right actual 3D coordinates of each pixel;

According to the mapping relationship between the left actual 3D coordinates of each pixel and the right actual 3D coordinates of each pixel, the reprojection matrix is obtained.
11. The computer device according to claim 10, wherein said calculating said left correction picture and said right correction picture by said StereoBM algorithm to obtain a view difference, comprises:

Performing single-channel grayscale conversion on the left-corrected picture to obtain a left single-channel grayscale image;

Performing single-channel grayscale conversion on the right-corrected picture to obtain a right single-channel grayscale image;

Call the preset disparity search range and sliding window size in the StereoBM algorithm, and use the left single-channel grayscale image, right single-channel grayscale image, disparity search range, and sliding window size as input parameters of the StereoBM algorithm Calculate to get the view difference.
The computer device according to claim 10, wherein said calling a pre-stored trajectory tracking algorithm to obtain the target two-dimensional image coordinates of each frame of target image in the target image set comprises:

The multi-target tracking algorithm corresponding to the trajectory tracking algorithm is called to obtain the target two-dimensional image coordinates of each frame of the target image in the target image set.
The computer device according to claim 10, further comprising:

Upload the target 3D coordinate set to the blockchain network.
11. The computer device according to claim 13, wherein said performing single-channel grayscale conversion on the left-corrected picture to obtain a left single-channel grayscale image comprises:

The left correction picture is read through the OpenCV picture reading instruction, and the left correction picture is converted into a left single-channel grayscale image through the OpenCV picture grayscale instruction.
The computer device according to claim 13, wherein the disparity search range and the sliding window size preset in the StereoBM algorithm are invoked, and the left single-channel grayscale image, the right single-channel grayscale image, and the disparity search The range and sliding window size are calculated as the input parameters of the StereoBM algorithm to obtain the view difference, including:

The disparity search range and sliding window size are called through the disparity search range and sliding window size call instructions in the StereoBM algorithm of OpenCV, and the view difference is calculated by the view difference calculation instructions in the StereoBM algorithm of OpenCV; where the disparity search range is 16*9, sliding The window size is 45.
A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the following operations:

Obtain the single target fixed parameters of the binocular camera through the calibration object image set; wherein, the single target fixed parameters include left camera internal parameters, left camera external parameters, left camera distortion parameters, right camera internal parameters, right camera external parameters, and right camera Distortion parameter

Acquiring a test picture, performing binocular correction on the test picture by using the single target setting parameters to obtain a left-corrected picture and a right-corrected picture, and obtain a reprojection matrix;

Call the pre-stored StereoBM algorithm, and calculate the view difference between the left correction picture and the right correction picture through the StereoBM algorithm;

Obtaining the target image set uploaded by the binocular camera and corresponding to the target to be tracked, and calling a pre-stored trajectory tracking algorithm to obtain the target two-dimensional image coordinates of each frame of the target image in the target image set; and

The target two-dimensional image coordinates of each frame of the target image in the target image set are converted into corresponding target 3D coordinates to form the target 3D according to the called sparse perspective change algorithm and the view difference. Coordinate collection.
18. The computer-readable storage medium according to claim 18, wherein said obtaining the single target fixed parameter of the binocular camera through the calibration object image set comprises;

Receive the left checkerboard picture set sent by the left camera in the binocular camera, and receive the right checkerboard picture set sent by the right camera; wherein the left checkerboard picture set and the right checkerboard picture set form a calibration object image set, and Each left checkerboard picture in the set of left checkerboard pictures corresponds to a right checkerboard picture in the set of right checkerboard pictures;

Acquiring one of the left checkerboard pictures in the left checkerboard picture set as the target left checkerboard picture, and obtaining the target right checkerboard picture corresponding to the target left checkerboard picture in the right checkerboard picture set;

Calling a pre-stored Harris corner detection function to obtain the Harris corner feature of the left image in the left checkerboard of the target, and obtain the Harris corner feature of the right image in the right checkerboard of the target;

Least square estimation is performed by using the Harris corner feature of the left image and the Harris corner feature of the right image to obtain the single target parameter of the binocular camera.
The computer-readable storage medium according to claim 18, wherein said acquiring a test picture performs binocular correction on said test picture by using said single target parameter to obtain a left-corrected picture and a right-corrected picture, and obtain re The projection matrix includes:

Linearly transform the image coordinates of each pixel in the test picture according to the left camera internal parameters and the right camera internal parameters to obtain the left actual imaging plane coordinates of each pixel and the right actual imaging plane coordinates of each pixel;

The left actual imaging plane coordinates of each pixel are converted according to the left camera distortion parameters to obtain the left ideal plane imaging coordinates of each pixel, and the right actual imaging plane coordinates of each pixel are converted according to the right camera distortion parameters. Obtain the right ideal plane imaging coordinates of each pixel;

Perform perspective projection transformation on the left ideal plane imaging coordinates of each pixel according to the left camera internal parameters to obtain the left camera 3D coordinates of each pixel, and perform perspective projection transformation on the right ideal plane imaging coordinates of each pixel according to the right camera internal parameters to obtain 3D coordinates of the right camera of each pixel;

The left camera 3D coordinates of each pixel are rigid body converted according to the left camera external parameters to obtain the left actual 3D coordinates of each pixel, and the right camera 3D coordinates of each pixel are rigid body converted according to the right camera external parameters to obtain each pixel The actual right 3D coordinates of the point;

Obtain a left correction picture according to the left actual 3D coordinates of each pixel, and obtain a right correction picture according to the right actual 3D coordinates of each pixel;

According to the mapping relationship between the left actual 3D coordinates of each pixel and the right actual 3D coordinates of each pixel, the reprojection matrix is obtained.