TWI499999B - The 3D ring car image system based on probability calculation and its obtaining method - Google Patents

Description

3D ring car image system based on probability calculation and its obtaining method

The invention relates to a 3D loop car image, in particular to a calculation using a modified probability neural network architecture to obtain a 3D loop image.

Advanced driver assistance systems have become one of the most important and urgent research topics in the world. According to the survey, the global car shipments in 2014 will be 85 million, and the installation rate of video security products will be 63 million. In the field of vehicle safety, machine vision is used to obtain environmental image information, and the road condition and the surrounding environment of the vehicle are analyzed during the path or between the vehicles to achieve various optimal dynamic response effects. This is also a visual intelligent transportation system (Vision). -based intelligent transportation system) development goals.

Early parking assistance systems often use ultrasonic or photographic lenses, using audible alarms or displaying camera images behind the vehicle to help the driver determine the distance between the vehicle and the obstacle at the blind corner. With the ultrasonic alarm method, the distance indication is not intuitive and the actual obstacle distance cannot be accurately determined. With the rear camera, the traditional vehicle image monitoring system is limited by the camera's image capturing range. The image displayed by the car display often cannot effectively cover the surrounding range of the vehicle, so it can only be developed for the rear of the vehicle or the sides of the vehicle. The blind spot monitoring display.

In view of the above problems of the prior art, the object of the present invention is to provide a 3D loop image system based on probability calculation and a method for obtaining the same, so as to solve the problem that the vehicle display in the vehicle cannot effectively monitor the surroundings of the vehicle. problem.

The invention is a 3D ring car image system based on the calculation of a probability method, which is suitable for use on a vehicle, and comprises a capturing unit and a processing unit. The capturing unit is configured to capture a plurality of bird's-eye images in a plurality of directions around the vehicle and transmit the bird's-eye view image. The processing unit has a correction module, a splicing module and a computing module, and the processing unit is configured to receive the bird's-eye image. The calibration module corrects the coordinates of the bird's-eye view image to the same coordinate system, and uses the splicing module to convert the bird's-eye view image into a virtual plane image to be spliced into a bird's-eye view image, and then the spliced bird's-eye view image is used by the operation module. Use algorithms to get 3D loop images.

Preferably, the capture unit of the 3D loop image system based on the probabilistic calculation of the present invention may be, for example, a wide-angle camera.

Preferably, the capture unit of the 3D loop image system based on the probability calculation of the present invention can be disposed, for example, on the front side body, the rear body and the second mirror on the vehicle.

Preferably, the algorithm of the 3D loop image system based on the probabilistic calculation of the present invention may be, for example, a Modified Probabilistic Neural Network (MPNN).

Preferably, the improved probability neural network architecture of the 3D loop image system based on the probabilistic calculation of the present invention may have, for example, an input layer, a class layer, a sum layer and an output layer, respectively, where c = { c ₁ , _{c 2, ..., c m}} , m is the number of class c, the input x, its closest _I class c has a corresponding output vector y _i, which may be expressed as y = {y _1, y ₂ ,..., y _m }, where m is the number of categories y , and the probability density function used by the improved probability neural network architecture is Where σ is the smoothing coefficient of the Gaussian function, absolute output for , where z _i is the number of samples in the class vector c _i , the vector of the network output , versus Represents the X-axis and Y-axis coordinates of the point, respectively.

The present invention is a method for obtaining a 3D loop image based on a probabilistic calculation, comprising the steps of: capturing a plurality of bird's-eye view images of a plurality of directions around the vehicle by the capture unit, and transmitting the bird's-eye view image; The coordinates of the bird's-eye view image are corrected to the same coordinate system; the bird's-eye view image is converted into a virtual plane image to be stitched into a bird's-eye view image; and the stitched bird's-eye view image is calculated by an algorithm to obtain a 3D car image.

Preferably, the capturing unit of the method for obtaining a 3D loop image based on the probability calculation of the present invention may be, for example, a wide-angle camera.

Preferably, the capturing unit of the method for obtaining a 3D car image based on the probability method of the present invention can be disposed, for example, on the front side body, the rear body and the second mirror on the vehicle.

Preferably, the algorithm for obtaining the 3D loop image based on the probabilistic method of the present invention may be, for example, an improved probability neural network architecture.

Preferably, the improved probability neural network architecture of the method for obtaining a 3D loop image based on the probabilistic method of the present invention may have, for example, an input layer, a class layer, a sum layer, and an output layer, respectively, where c = { c ₁ , c ₂ ,..., c _m }, m is the number of categories c , for the input x , the closest to the category c _i has a corresponding output vector y _i , which can be expressed as y = { y ₁ , y ₂ ,..., y _m }, where m is the number of categories y , and the probability density function used by the modified probability neural network architecture is Where σ is the smoothing coefficient of the Gaussian function, absolute output for , where z _i is the number of samples in the class vector c _i , the vector of the network output , versus Represents the X-axis and Y-axis coordinates of the point, respectively.

In summary, the 3D loop image system based on the probabilistic calculation of the present invention and the obtaining method thereof have the following advantages:

(1) The correction module of the processing unit of the present invention can correct the bird's-eye view image captured by the camera to the same coordinate system, so that the captured bird's-eye view image can be corrected to achieve the complete restoration effect.

(2) The splicing module of the processing unit of the present invention converts the bird's-eye view image into a virtual image plane and then splicing the bird's-eye view image to ensure that the corrected bird's-eye view image can be accurately stitched and stitched.

(3) The 3D loop image system based on the probability calculation of the present invention and the obtaining method thereof mainly provide the use of the stereoscopic 3D loop to present the relative position of the obstacle and the vehicle by improving the shortcomings of the current bird's-eye view image. The actual three-dimensional dynamics of the vehicle will not cause the angle of the vehicle body to be out of alignment due to the orientation and height of the obstacle.

9‧‧‧ Vehicles

10‧‧‧Capture unit

20‧‧‧Processing unit

21‧‧‧ Calibration Module

22‧‧‧Splicing module

23‧‧‧ Computing Module

101‧‧‧A bird's eye view of the front body

102‧‧‧A bird's eye view of the rear body

103‧‧‧ Bird's eye view of the mirror

Aerial view of the 104‧‧ ‧ reversing mirror

S10~S40‧‧‧Steps

Figure 1 is a block diagram of the system of the 3D loop image system based on the probability calculation of the present invention.

2 is a first schematic diagram of a 3D loop image system based on the probability calculation of the present invention and a method for obtaining the same.

Figure 3 is a 3D loop image based on the probability calculation of the present invention. A second schematic diagram of the system and its method of acquisition.

Figure 4 is a third schematic diagram of the 3D loop image system based on the probability calculation of the present invention and a method for obtaining the same.

Figure 5 is a schematic diagram of the coordinate correction of the 3D loop image system based on the probabilistic calculation of the present invention and the method of obtaining the same.

Figure 6 is a flow chart showing the steps of the 3D loop image system based on the probability calculation of the present invention and the obtaining method thereof.

Please refer to Figure 1 and further refer to Figures 2 through 5. The 3D loop image system based on the probability calculation of the present invention is applicable to a vehicle and includes a plurality of capture units 10 and a processing unit 20. The capturing unit 10 is configured to capture a plurality of bird's-eye images (as shown in FIG. 2) in a plurality of directions around the vehicle 9 and transmit a bird's-eye view image. The capturing unit 10 can be, for example, a wide-angle camera. The limitation is as long as the image that can be captured to the periphery of the vehicle 9 is suitable for the present invention, and the capturing unit can be disposed, for example, on the front side of the vehicle 9 , the rear body and the second mirror for capturing the vehicle 9 The bird's-eye view image 101 of the square body, the bird's-eye view image 102 of the rear body, and the bird's-eye view images 103, 104 of the two mirrors.

In the above, the processing unit 20 of the present invention has a calibration module 21, a splicing module 22, and a computing module 23. The processing unit 20 is configured to receive a bird's-eye image, and the calibration module 21 corrects the coordinates of the bird's-eye image to the same coordinate. The system uses the splicing module 22 to convert the bird's-eye view image into a virtual plane image (as shown in FIG. 5) to form a bird's-eye view image (as shown in FIG. 3) by stitching (Image Stitching), and then stitching the image. The aerial view image is obtained by the computing module using an algorithm to obtain a 3D loop image (as shown in Fig. 4). The above coordinate correction method converts the image plane coordinates into a world coordinate plane, and then converts the world coordinate plane into a virtual plane seat. Standard.

The above algorithm may be, for example, an improved probabilistic neural network architecture, wherein the improved probabilistic neural network architecture has an input layer, a class layer, a sum layer, and an output layer, respectively, where c = { c ₁ , c ₂ , .. , c _m }, m is the number of categories c , for the input x , the closest to the category c _i has a corresponding output vector y _i , which can be expressed as y = { y ₁ , y ₂ , .. y _m }, where m is the number of categories y , and the probability density function used by the improved probability neural network architecture is Where σ is the smoothing coefficient of the Gaussian function, absolute output for , where z _i is the number of samples in the class vector c _i , the vector of the network output , versus Represents the X-axis and Y-axis coordinates of the point, respectively.

Please refer to FIG. 6 , which is a flow chart of the steps of the 3D loop image system based on the probabilistic calculation of the present invention and the obtaining method thereof. The present invention includes the following steps: First, (S10) capturing a plurality of bird's-eye view images of a plurality of directions around the vehicle, and transmitting a bird's-eye view image, and the capturing unit may be, for example, a wide-angle camera, and the capturing unit may be disposed, for example, in the vehicle. On the front side of the car body, the rear car body and the second mirror. Then (S20) correcting the coordinates of the received bird's-eye image to the same coordinate system, the coordinate correction method is to convert the image plane coordinate into a world coordinate plane, and then convert the world coordinate plane into a virtual plane coordinate (such as Figure 5).

Then, (S30) converting the bird's-eye view image into a virtual plane image to be spliced into a bird's-eye view image. For example, it is possible to find a similar feature point by using a plurality of separately captured images, when the number of feature points is matched and When the coordinate position meets the threshold value, it will correspond The image stitching and stitching are performed at an angle and a distance, and the image stitching and stitching can be performed, for example, by the Open CV image processing library. Finally, (S40), the stitched bird's-eye view image is calculated by an algorithm to obtain a 3D loop image. The algorithm in S40 can be, for example, an improved probability neural network architecture.

In summary, the 3D ring car image system based on the probability calculation method of the present invention and the obtaining method thereof are designed and ingenious, and the bird's-eye view images are sequentially ordered by the correction module, the splicing module and the computing module. Perform image correction, stitching and stitching, and use the improved probabilistic neural network architecture to calculate 3D loop images. This 3D loop image allows the user on the vehicle to more accurately understand the relative distance between the vehicle body and the obstacle. .

The above is intended to be illustrative only and not limiting. Any equivalent modifications or alterations to the spirit and scope of the invention are intended to be included in the scope of the appended claims.

10‧‧‧Capture unit

20‧‧‧Processing unit

21‧‧‧ Calibration Module

22‧‧‧Splicing module

23‧‧‧ Computing Module

Claims (5)

A 3D loop image system based on a probabilistic calculation is applied to a vehicle. The 3D loop image system includes: a plurality of capture units for capturing a plurality of bird's-eye views of the plurality of directions around the vehicle, and Transmitting the bird's-eye view images, wherein the capture unit is a wide-angle camera; and a processing unit having a correction module, a splicing module and a computing module, the processing unit is configured to receive the bird's eye view The image is corrected by the correction module to coordinate the coordinates of the bird's-eye view image into the same coordinate system, and the splicing module converts the bird's-eye view images into virtual plane images to be spliced into a bird's-eye view image, and then spliced The aerial view image uses the algorithm to obtain a 3D loop image by using the algorithm, wherein the algorithm is a modified Probabilistic Neural Network (MPNN), wherein the improved probability neural network The network architecture has an input layer, a pattern layer, a summing layer, and an output layer, where c = { c ₁ , c ₂ , ..., c _{m}, m} is the number of class c, the input x, its closest _I class c has a corresponding output vector y _i, which may be expressed as y = {y _1, y ₂ ,..., y _m }, where m is the number of categories y , and the probability density function used by the modified probabilistic neural network architecture is Where σ is the smoothing coefficient of the Gaussian function, absolute output for , , where z _i is the number of samples in the class vector c _i , the vector of the network output , versus Represents the X-axis and Y-axis coordinates of the point, respectively. The 3D loop image system based on the probabilistic calculation described in claim 1 is wherein the picking units are disposed on the front side body, the rear body and the second mirror on the vehicle. A method for obtaining a 3D loop image based on a probabilistic calculation includes the following steps: capturing a plurality of bird's-eye view images of a plurality of directions around a vehicle by using a capture unit, and transmitting the bird's-eye images; The coordinates of the bird's-eye view images are corrected to the same coordinate system; the bird's-eye view images are converted into virtual plane images to be stitched into a bird's-eye view image; and the stitched aerial bird's-eye view image is calculated by an algorithm A 3D loop image, wherein the algorithm is an improved probability neural network architecture, wherein the improved probability neural network architecture has an input layer, a class layer, a sum layer, and an output layer, respectively, where c = { c ₁ , c ₂ ,..., c _m }, m is the number of categories c , for the input x , the closest to the category c _i has a corresponding output vector y _i , which can be expressed as y = { y ₁ , y ₂ ,..., y _m }, where m is the number of categories y , and the probability density function used by the improved probability neural network architecture is Where σ is the smoothing coefficient of the Gaussian function, absolute output for , where z _i is the number of samples in the class vector c _i , the vector of the network output , versus Represents the X-axis and Y-axis coordinates of the point, respectively. The method for obtaining a 3D loop image based on the probability calculation described in claim 3, wherein the capture units are wide-angle cameras. The method for obtaining a 3D loop image based on the probability calculation described in claim 3, wherein the picking units are disposed on the front side body body, the rear side body body and the second rear view mirror of the vehicle.