CN111784749A

CN111784749A - Space positioning and motion analysis system based on binocular vision

Info

Publication number: CN111784749A
Application number: CN201911282790.6A
Authority: CN
Inventors: 刘明; 周政; 杨鹏; 张笑宇; 惠梅; 董立泉; 赵跃进
Original assignee: Beijing Institute of Technology BIT
Current assignee: Beijing Institute of Technology BIT
Priority date: 2019-12-13
Filing date: 2019-12-13
Publication date: 2020-10-16

Abstract

The invention discloses a binocular vision-based space positioning and motion analysis system. The two array camera systems are used for replacing binocular cameras to collect scene images from two specified directions, so that the field angle of the collected images can be remarkably improved; the system combines the gesture analysis and the motion trail analysis to record the motion process of the athlete, and is used for analyzing the action and the position of the athlete frame by frame and observing the motion state of the athlete at different moments.

Description

Space positioning and motion analysis system based on binocular vision

Technical Field

The invention belongs to the field of image processing and machine vision, and particularly relates to a binocular vision-based space positioning and motion analysis system.

Background

Common positioning methods include ultrasonic positioning, laser positioning, infrared positioning, optical positioning and the like, and are mainly applied to the fields of military affairs, industrial measurement, building construction and the like. The currently used vision measuring technology mainly comprises the following 3 methods, namely a binocular stereo vision method, a structured light method and a geometric optics method.

The binocular stereo vision positioning has the advantages of non-contact, automatic measurement, no harm to human eyes and the like. The most common method is a parallel optical axis model, two cameras are horizontally arranged at a distance of a base line, the same characteristic point only has horizontal parallax in two images through distortion correction and epipolar correction, the parallax of the corresponding point is obtained by using an image registration method, and the depth information of an object point in a scene is finally obtained by using the relation between the parallax and the depth. At present, a large number of binocular stereo vision positioning methods are proposed and applied to various engineering fields. However, the conventional binocular positioning technology often has the disadvantage of insufficient range, and in order to increase the range and ensure the positioning accuracy, the baseline distance needs to be enlarged, that is, the optical center distance between the two cameras needs to be increased, so that the field angle of the binocular camera is reduced, the positioning range is also reduced, and the positioning requirement of large places such as ice rinks is difficult to meet. And the motion analysis of the moving object in the traditional binocular stereo vision positioning is difficult due to the characteristic of small field angle. Therefore, there is a need in engineering to develop a positioning system that can satisfy both the range and the wide viewing angle.

Disclosure of Invention

In view of the above, the invention uses the image stitching technology to enlarge the field angle of the binocular camera, can solve the problem of insufficient positioning range of the binocular stereoscopic vision system at low cost, and simultaneously combines the posture and motion trajectory analysis technology to assist the motion training.

The invention provides a binocular vision-based space positioning and motion analysis system scheme. The technical scheme is as follows:

1) two array camera systems for capturing scene images from two designated orientations instead of binocular cameras

2) And performing feature extraction on an image acquired by each lens of the array camera, and tracking and gesture recognition on a specified target in the image.

3) After the images obtained by the two array camera systems are corrected, the images are spliced into a large-view-field image

4) Calculating to obtain the space position through two large view field images and analyzing to obtain the motion track

In the step 1), the array cameras are erected on a high platform, and two array cameras are respectively aligned to the playground by adopting a placing mode of a binocular vision model with intersected optical axes (the intersected optical axes are based on a sub-lens optical axis serving as a splicing reference during panoramic splicing), so that the whole playground is ensured to be within a common view field of the two cameras. And calibrating the array camera by adopting a Zhangyingyou calibration method to acquire the internal and external parameters of the camera.

In the step 2), the image needs to be preprocessed, the distortion is corrected, the noise is eliminated, and the quality of the image is improved.

In the step 3), the characteristic points are selected at the ground during splicing, so that accurate information of the ground during splicing is ensured.

In the step 4), a plurality of mark points are selected on the motion field, the coordinate positions of the mark points are measured, and the mark point images are respectively intercepted from the videos shot by the two array cameras. And obtaining the position coordinates in the two-dimensional image of the unprocessed target object through the processing, and analyzing and calculating the target space position by combining the internal and external parameters of the camera and the distortion coefficient.

Compared with the prior art, the invention has the following advantages:

1) most of the sport event analysis systems in the market need athletes to carry portable sensing equipment, and the invention only needs to adopt a video image processing means to collect data of the athletes, thereby ensuring that the data can not influence and burden the behavior and actions of the athletes while collecting the data, and simultaneously leading the training environment to be closer to the real competition environment.

2) The invention combines the posture analysis and the motion trail analysis for the first time, records the complete motion process of the athlete, and is used for analyzing the action and the position of the athlete frame by frame and observing the virtual three-dimensional models of the athlete at different moments.

Drawings

FIG. 1 is a flow chart of an embodiment of the present invention;

FIG. 2 is a schematic diagram of a network flow for automatically detecting an athlete and performing gesture recognition

FIG. 3 is a binocular positioning vision model according to the present invention;

fig. 4 is a relationship diagram of four types of coordinate systems in the three-dimensional reconstruction process, the four types of coordinate systems: a pixel coordinate system, an image coordinate system, a camera coordinate system, and a world coordinate system.

Detailed Description

The invention is described in detail below with reference to the figures and specific embodiments. It should be noted that the described examples are only intended to facilitate the understanding of the invention, and do not have any limiting effect thereon.

As shown in fig. 1, the embodiment provides a binocular vision-based spatial localization and motion analysis system, and the specific implementation process of the system includes the following steps:

1) and calibrating the array camera. And (3) adopting a Zhangyingyou calibration method, ensuring that each sub-lens of each array camera can capture photos of different positions and angles of a plurality of calibration plates, and calculating internal and external parameters of each sub-lens of the camera.

2) Image acquisition and preprocessing. Because the system adopts a binocular vision model with intersected optical axes (the intersected optical axes are based on the optical axes of the sub-lenses serving as splicing references during panoramic splicing), strict requirements on the placement of the cameras are not required, only a larger baseline distance needs to be reserved between the cameras to ensure the precision, and a larger common view field is required between the cameras.

3) And (3) feature extraction, as shown in fig. 2, extracting features in the acquired picture by using an image processing method, so that subsequent splicing and gesture recognition are facilitated.

4) And (3) gesture recognition, as shown in fig. 2, inputting the extracted features into a candidate network of the region to perform semantic segmentation on the picture, and collecting the detected position information of the human body and the feature points of the human body, so that human motion description information of the moving personnel in the image can be obtained.

5) Target detection and tracking, as shown in fig. 2, identifies players in the field based on the characteristics and tracks designated players in the captured video using a tracking algorithm.

6) And (4) panoramic stitching, namely stitching the images collected by the array camera into a panoramic image of the sports stadium. And when splicing is carried out, the characteristic points on the ground of the sports field are used for calculating a conversion matrix so as to ensure that the position information of the athlete is accurate.

7) Stereo matching, as shown in fig. 3, the binocular vision model with intersecting optical axes does not have strict requirements on the placement positions of the two cameras. The cameras are placed relatively randomly, and the inclination angles of the cameras and the placement distance between the two cameras can be flexibly adjusted according to the requirements of a venue and the actual characteristics of an acquired object. And calculating the parallax of the corresponding image by using a stereo matching algorithm, and realizing mathematical transformation from the point coordinates on the two-dimensional images shot at two angles to the three-dimensional actual space coordinates.

8) And (4) track reduction, namely reducing the space track of the sportsman in the field according to the target tracking and stereo matching results.

9) Three-dimensional reconstruction, known from a binocular vision measurement model, can determine the position of any point P after the position in a three-dimensional space is observed by a binocular camera, and because of the calculation requirement, we usually provide four types of coordinate systems: a pixel coordinate system, an image coordinate system, a camera coordinate system, and a world coordinate system. As shown in FIG. 4, a rectangular coordinate system uO is defined on the image₀v, denoted as pixel coordinate system, coordinate system xO₁y, as image coordinate system, coordinate system OX_cY_cZ_cRecorded as camera coordinate system, world coordinate system is represented by X_wAxis, Y_wAxis and Z_wThe shafts are jointly composed. Let the point of the three-dimensional world coordinate of the point P be [ X, Y, Z,1 ]]The two-dimensional pixel coordinate of its projection point p is [ u, v,1 ]]The relationship between world coordinates and pixel coordinates is as follows:

M₁i.e. the internal reference matrix of the camera, M₂Is an external parameter matrix of the camera, M₃₄Referred to as the projection matrix. And reconstructing the three-dimensional models of the athlete and the field through the results of posture detection and three-dimensional matching.

In summary, the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A space positioning and motion analysis system based on binocular vision is characterized by comprising the following steps:

2) And performing feature extraction on an image acquired by each lens of the array camera, and performing target tracking and gesture recognition on a specified moving target.

4) And calculating to obtain the position of the target in the three-dimensional space through the two large-field images, and obtaining the motion track of the target by analyzing the continuous video images.

2. The binocular vision based spatial localization and motion analysis system of claim 1, wherein a binocular vision model of intersecting optical axes (the intersecting optical axes are based on the optical axes of the sub-lenses serving as the reference of the stitching in the panoramic stitching) is adopted, and the requirement of camera placement limitation is less.

3. The binocular vision based spatial orientation and motion analysis system of claim 1, wherein the input image is pre-processed to correct distortion, remove noise, and improve image quality.

4. The binocular vision based spatial localization and motion analysis system of claim 1, wherein the feature points are selected at the ground during stitching to ensure that ground information is accurate during stitching.

5. The binocular vision based spatial orientation and motion analysis system of claim 1, wherein a plurality of marker points are selected on the motion field, coordinate positions of the marker points are measured, and images of the marker points are respectively captured from videos captured by two array cameras. After the above processing, the position coordinates in the two-dimensional image of the unprocessed target object are obtained, and the position of the target in the actual three-dimensional space is calculated based on the position coordinates.

6. The binocular vision based spatial localization and motion analysis system of claim 1, wherein the target and scene depth information is obtained using camera internal and external parameters, distortion coefficients and image feature point correspondences.