CN112884832B - Intelligent trolley track prediction method based on multi-view vision - Google Patents

Abstract

The invention discloses an intelligent trolley track prediction method based on multi-view vision, which solves the defects of instable real-time positioning, easy signal disconnection, use limitation, low working efficiency and high cost of the existing indoor intelligent trolley.

Description

A method for predicting the trajectory of an intelligent car based on multi-eye vision

technical field

The invention relates to indoor positioning technology, in particular to a method for predicting the trajectory of an intelligent car based on multi-eye vision.

Background technique

In the prior art, the outdoor smart car is based on global positioning and navigation technology (GPS) to realize real-time positioning and trajectory prediction, but in some areas where the GPS signal is weak or cannot be covered by GPS at all, the smart car cannot be accurately detected. Positioning, for example, indoor smart car positioning, the current smart car positioning technology relies on sensors to process external signals to obtain real-time pose information. However, since the sensor will be greatly affected by the outside world, when the signal is interfered, the positioning cannot be accurately achieved. For example, in some places that cannot be covered by GPS, WiFi positioning, Bluetooth positioning, and radio frequency identification positioning can be used. These methods have the disadvantage of instability. When the indoor smart car works far away from the receiving source, the signal disconnection may occur. Therefore, the existing technology has high cost, certain limitations, and low work efficiency.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a method for predicting the trajectory of an intelligent car based on multi-eye vision, which can locate in real time based on machine vision, and has low cost and high stability.

The above-mentioned technical purpose of the present invention is achieved through the following technical solutions:

A method for predicting the trajectory of an intelligent car based on multi-eye vision, comprising the following steps:

S1. Install several cameras fixedly, and take pictures of different poses of the checkerboard with the set number, calibrate the cameras by Zhang Zhengyou's calibration method, and obtain the internal parameters and distortion parameters of the cameras;

S2. Paste the visual label on the smart car, and the camera shoots in real time to obtain the real-time two-dimensional coordinates of the smart car by locating the pose of the visual label;

S3. Establish the origin of the world coordinates according to the PnP algorithm, define the correspondence between the world space coordinates of multiple points and the two-dimensional coordinates, and obtain the external parameters of the camera at this time;

S4, transform the coordinate system of the two-dimensional coordinate pose output by the visual label, and convert the two-dimensional coordinates into three-dimensional space coordinates;

S5. Construct a pose measurement model through a multi-eye stereo vision model, introduce the least squares method to obtain the three-dimensional space pose, and optimize the intelligent car robotic arm space pose by combining with the triangular center of gravity method;

S6, obtaining the pose information of the smart car through the three-dimensional space coordinates, drawing the trajectory of the car's motion trajectory and performing error analysis.

Preferably, the information of the visual label includes four corner pixels, center pixels, a homography matrix, and an ID corresponding to each label.

Preferably, the visual label adopts the AprilTag visual system.

Preferably, the transformation of three-dimensional space coordinates is specifically:

Through the different placement of the three cameras, the visual labels on the smart car are obtained at the same time and in the same scene;

Through the PnP algorithm, given the coordinates of N 3D points in the world and the 2D coordinates on the image, the camera extrinsic parameters are solved;

Convert a series of two-dimensional coordinates into three-dimensional coordinates according to the camera's internal and external parameters.

Preferably, the three cameras are installed at different positions and at different angles in the space.

To sum up, the present invention has the following beneficial effects:

Using multi-eye machine vision and visual labeling technology to locate the pose of the smart car, realize the indoor real-time positioning of the sensorless smart car, and improve the existing smart car positioning technology; The monocular camera depth calculation problem has higher accuracy for the binocular camera, which can achieve more accurate indoor smart car positioning, and realize the multi-angle and large-scale real-time positioning requirements of the smart car.

Description of drawings

Fig. 1 is a schematic flow diagram of the method;

Figure 2 is a schematic diagram of a multi-view smart car pose measurement;

Figure 3 is a diagram of the multi-eye vision pose measurement model.

Detailed ways

The present invention will be further described in detail below with reference to the accompanying drawings.

According to one or more embodiments, a method for predicting the trajectory of an intelligent car based on multi-eye vision is disclosed, as shown in FIG. 1 and FIG. 2 , including the following steps:

S1. Install several cameras fixedly, and take pictures of different poses of a set number of checkerboards. The cameras are calibrated by Zhang Zhengyou's calibration method, and the internal parameters and distortion parameters of the cameras are obtained.

The installation of cameras is as follows: the number of cameras is preferably three, three cameras are installed at different positions in the indoor space, and the angles of each camera are different, and three cameras are used to achieve multi-eye vision, and the multi-eye vision measurement system can cover a larger area. Compared with a single binocular vision, the measurement system with three binocular visions has better robustness and has a wider range of uses in practical and complex application scenarios.

The calibration of the internal parameters of the camera is as follows: making a checkerboard calibration board, and three fixed cameras synchronously shoot 20 pictures of the calibration board at different positions and rotation angles, and collect data synchronously with multiple objects. The camera is calibrated to obtain the internal parameters and distortion coefficients of each camera.

According to the camera calibration, the translation matrix and rotation matrix in the camera coordinate system can be calculated, and then the three-dimensional space coordinates of the center of the object can be obtained according to the translation matrix, so as to realize the six-degree-of-freedom attitude estimation.

S2. Paste the visual label on the smart car, and the camera shoots in real time to obtain the real-time two-dimensional coordinates of the smart car by locating the pose of the visual label.

The information of the visual label includes four corner pixels, center pixel, homography matrix, and ID corresponding to each label.

The visual label adopts the AprilTag vision system, which is widely used in the field of robot, AR, and camera calibration. It is similar to the two-dimensional code (QR) technology, but reduces the complexity, can quickly detect markers, and calculate the marker Relative position, through this tag, the two-dimensional coordinate pose of the smart car can be accurately estimated, and the remote real-time positioning technology can be realized.

Multiple visual labels are attached to the indoor smart car body, and the position of the visual label on the smart car is captured in real time. When the smart car is working, the three cameras shoot video information in real time and transmit it to the computer. The computer uses the visual labels on the smart car body to identify. The 2D coordinates of the center point of the visual label AprilTag.

S3. Establish the origin of the world coordinates according to the PnP algorithm, define the correspondence between the world space coordinates of multiple points and the two-dimensional coordinates, and obtain the external parameters of the camera at this time. The PnP algorithm is used to obtain the external parameters of the camera according to the two-dimensional coordinates and spatial coordinates of the images of multiple fixed points, through the known four corner points and the calibrated camera internal parameters and distortion, and at the same time, the origin of the world coordinate system can be determined.

S4 , transform the coordinate system of the two-dimensional coordinate pose output by the visual label, and convert the two-dimensional coordinates into three-dimensional space coordinates, as shown in FIG. 3 .

According to the information of the camera input in the computer in advance and the spatial position relationship of each device, the two-dimensional coordinates can be converted into camera coordinates. When the camera coordinates are converted into the spatial position coordinates in the world coordinate system, the conversion of the three-dimensional spatial coordinates is as follows:

Through the different placement of the three cameras, the visual labels on the smart car are obtained at the same time and in the same scene; through the PnP algorithm, given the coordinates of N 3D points in the world and the two-dimensional coordinates on the image, Solve the camera external parameters; convert a series of two-dimensional coordinates into three-dimensional coordinates according to the camera internal parameters and external parameters, see formula 1, A matrix is the camera internal parameter matrix, B matrix is the camera external parameter matrix, u, v are two-dimensional coordinates, X _W , Y _W , Z _W are the three-dimensional coordinates, and Z _C is the scale factor.

S5. Construct a pose measurement model through a multi-eye stereo vision model, as shown in Figure 3, introduce the least squares method to obtain the three-dimensional spatial pose, and combine the triangular center of gravity method to optimize the space pose of the smart car.

In practical applications, since the data is always noisy, the trinocular vision fusion by the least squares method can obtain that the three-dimensional coordinates of the measured object are three disjoint points, and the optimal center of gravity method can be obtained. three-dimensional coordinates.

S6, obtaining the pose information of the smart car through the three-dimensional space coordinates, drawing the trajectory of the car's motion trajectory and performing error analysis.

It can realize real-time trajectory prediction for the smart car, meet the work requirements for accurate positioning of the indoor smart car, and improve the flexibility and controllability.

Through multi-eye machine vision, visual labeling system and other technologies for pose positioning, sensorless indoor real-time positioning of smart cars is realized, which improves the existing smart car positioning technology. AprilTag is a visual reference system for vision tasks such as robot positioning and camera calibration, which can calculate the precise position and orientation of the camera in a two-dimensional coordinate system. Therefore, the cost of purchasing an expensive sensor smart car can be saved, and it has good robustness and economy. Users can get rid of the troubles caused by the inconvenience and low work efficiency caused by the inconvenience and low work efficiency caused by the areas that cannot be located by the global positioning system, such as indoor intelligent car positioning. The visual labeling system used is also economical and reliable. The accuracy of multi-eye vision is more accurate than that of binocular vision, which greatly improves the efficiency of indoor intelligent car estimation and prediction. The calculation of 3D pose through multi-eye vision can not only solve the problem of monocular vision The problem of camera depth calculation has higher accuracy for binocular cameras, which can achieve more accurate indoor smart car positioning, and realize the multi-angle and large-scale real-time positioning requirements of smart cars.

This specific embodiment is only an explanation of the present invention, and it does not limit the present invention. Those skilled in the art can make modifications without creative contribution to the present embodiment as required after reading this specification, but as long as the rights of the present invention are used All claims are protected by patent law.

Claims (2)

1. a kind of intelligent car trajectory prediction method based on multi-eye vision, is characterized in that, comprises the following steps: S1. Install several cameras fixedly, and take pictures of different poses of the checkerboard with the set number, calibrate the cameras by Zhang Zhengyou's calibration method, and obtain the internal parameters and distortion parameters of the cameras; S2. Paste the visual label on the smart car, and the camera shoots in real time. By locating the pose of the visual label, the real-time two-dimensional coordinates of the smart car are obtained; the information of the visual label includes four corner pixels, center pixels, homography Matrix, each label corresponds to an ID; the visual label adopts the AprilTag vision system; S3. Establish the origin of the world coordinates according to the PnP algorithm, define the correspondence between the world space coordinates of multiple points and the two-dimensional coordinates, and obtain the external parameters of the camera at this time; S4. Transform the coordinate system of the two-dimensional coordinate pose output by the visual label, and convert the two-dimensional coordinates into three-dimensional space coordinates; through the different placement of the three cameras, at the same time and in the same scene, the vision on the smart car is The label is used to obtain pictures; the PnP algorithm is used to solve the camera external parameters given the coordinates of N 3D points in the world and the two-dimensional coordinates on the image; a series of two-dimensional coordinates are converted into three-dimensional coordinates according to the camera internal parameters and external parameters; S5. Construct a pose measurement model through a multi-eye stereo vision model, introduce the least squares method to obtain the three-dimensional space pose, and optimize the intelligent car space pose with the triangular center of gravity method; S6, obtaining the pose information of the smart car through the three-dimensional space coordinates, drawing the trajectory of the car's motion trajectory and performing error analysis. 2 . The method for predicting the trajectory of an intelligent car based on multi-eye vision according to claim 1 , wherein the three cameras are installed at different positions and at different angles in the space. 3 .

Images