CN111754574A - Distance testing method, device, system and storage medium based on binocular camera - Google Patents

Abstract

The invention discloses a distance testing method, device, system and storage medium based on a binocular camera. The method first obtains the blind spot position of the binocular camera within a preset image range, and when it is determined that the target to be tested is in the blind spot position, the method is adopted. The monocular measures the distance of the target to be measured, and outputs the distance measurement result obtained by the monocular distance measurement; when it is determined that the target to be measured is not in the blind spot, the monocular and binocular are used to measure the target to be measured separately. Ranging, and after the weight calculation of the obtained monocular ranging results and binocular ranging results, the comprehensive ranging results are obtained and output. This method combines the respective advantages of the binocular parallax ranging principle and the monocular ranging principle, and uses the monocular and binocular combined ranging to improve the ranging accuracy, thereby solving the limitation of the traditional binocular camera in obstacle distance detection. The technical problem of low ranging accuracy caused by blind spots.

Description

Distance testing method, device, system and storage medium based on binocular camera

technical field

The invention relates to the technical field of binocular camera imaging, in particular to a distance testing method, device, system and storage medium based on a binocular camera.

Background technique

With the development of sensor technology and machine vision technology, obstacle testing technology based on binocular camera is increasingly widely used in robotics, smart cars and other fields. However, in the prior art, the target distance detection is mostly performed by analyzing the binocular disparity, which is easily subject to many limitations such as the environment and the blind spot of binocular matching. .

SUMMARY OF THE INVENTION

To this end, embodiments of the present invention provide a method, device, system, and storage medium for distance measurement based on a binocular camera, so as to at least partially solve the ranging accuracy caused by the limitation of blind spots in the distance detection of obstacles by traditional binocular cameras Low technical issues.

In order to achieve the above purpose, the embodiments of the present invention provide the following technical solutions:

A distance testing method based on a binocular camera, characterized in that the method comprises:

Obtain the blind spot position of the binocular camera within the preset image range;

If it is determined that the target to be measured is located in the blind area, the target to be measured is measured using a monocular, and the ranging result obtained by the monocular ranging is output;

It is determined that the target to be measured is not located in the blind area, then the target to be measured is measured separately by monocular and binocular, and after the obtained monocular ranging result and binocular ranging result are weighted, the result is obtained. Output comprehensive ranging results.

Further, the method also includes:

Collecting a binocular image of the target to be measured, extracting the position information of the target to be measured in the binocular image, and marking the position information through a feature extraction frame;

obtaining the effective parallax ratio of the target to be tested within the range of the feature extraction frame;

If it is determined that the effective parallax ratio is lower than the set threshold, and the target to be measured is in the blind area, the target to be measured is measured using a monocular, and the ranging result obtained by the monocular distance measurement is output;

It is determined that the effective parallax ratio is higher than the set threshold, and the target to be measured is not located in the blind spot, then the target to be measured is measured separately by monocular and binocular, and the obtained monocular ranging result is compared with that. After the binocular ranging results are weighted, the comprehensive ranging results are obtained and output.

Further, collecting the binocular image of the target to be measured, extracting the position information of the target to be measured in the binocular image, and marking the position information through a feature extraction frame, specifically including:

Collect the left eye image and the right eye image under the same frame image including the target to be tested, and obtain a disparity map;

Extract the position information of the object to be tested in the disparity map through machine learning or feature detection;

Use a rectangular box to frame the location information.

Further, the comprehensive ranging result is obtained by the formula Dz=w*dist1+(1-w)*dist2;

Wherein, w is a weight, w is a parameter positively related to the effective disparity ratio, dist1 is a binocular ranging result, and dist2 is a monocular ranging result.

Further, the functional relationship between the weight and the effective disparity ratio includes:

Wherein, n is the effective parallax ratio, and _nmax is the maximum value of the set threshold.

Further, the obtaining the blind spot position of the binocular camera within the preset image range specifically includes:

Obtain the internal and external parameters and field of view of the binocular camera;

The blind spot position is acquired according to the internal and external parameters and the field of view angle.

The present invention also provides a distance testing device based on a binocular camera for implementing the above method, the device comprising:

The blind spot acquisition unit is used to obtain the blind spot position of the binocular camera within the preset image range;

The first ranging result output unit is used to determine that when the target to be measured is located in the blind area, use a monocular to measure the distance of the target to be measured, and output the ranging result obtained by the monocular ranging;

The second ranging result output unit is used to determine that the target to be measured is not located in the blind area, then use monocular and binocular to measure the distance of the target to be measured separately, and compare the obtained monocular ranging result with the binocular range. After the weight of the visual ranging result is calculated, the comprehensive ranging result is obtained and output.

Further, the device also includes:

a position acquisition unit, configured to collect a binocular image of the target to be measured, extract the position information of the target to be measured in the binocular image, and mark the position information through a feature extraction frame;

A parallax ratio acquiring unit, configured to acquire the effective parallax ratio of the object to be tested within the range of the feature extraction frame;

The first ranging result output unit is further configured to determine that the effective parallax ratio is lower than a set threshold and the target to be measured is located in a blind area, then use a monocular to measure the distance to the target to be measured, and Output the ranging result obtained by monocular ranging;

The second ranging result output unit is further configured to determine that when the effective parallax ratio is higher than a set threshold and the target to be measured is not located in the blind spot, then monocular and binocular are used to measure the target to be measured. Ranging is measured separately, and after the obtained monocular ranging results and binocular ranging results are weighted, the comprehensive ranging results are obtained and output.

The present invention also provides a distance testing system based on a binocular camera, the system comprising: a processor and a memory;

the memory for storing one or more program instructions;

The processor is configured to execute one or more program instructions for executing the method as described above.

The present invention also provides a computer storage medium, wherein the computer storage medium contains one or more program instructions, and the one or more program instructions are used to be executed by a binocular camera-based distance testing system as described above. method.

The method, device, system and storage medium for distance testing based on binocular cameras provided by the present invention firstly obtain the blind spot position of the binocular camera within the preset image range, and when it is determined that the target to be tested is in the blind spot position, the single Measure the distance of the target to be measured by eye, and output the distance measurement result obtained by the monocular distance measurement; when it is determined that the target to be measured is not in the blind spot, then use the monocular and binocular to measure the target to be measured separately. After calculating the weight of the obtained monocular ranging results and the binocular ranging results, the comprehensive ranging results are obtained and output. This method solves the problem that the target distance detection purely through binocular parallax is easily limited by the environment and the blind spot of binocular matching. It combines the respective advantages of the binocular parallax ranging principle and the monocular ranging principle. Combined with the distance measurement, the distance measurement accuracy is improved, thus solving the technical problem of low range measurement accuracy caused by the limitation of the blind spot when the traditional binocular camera detects the distance of obstacles.

Description of drawings

In order to illustrate the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that are required to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only exemplary, and for those of ordinary skill in the art, other implementation drawings can also be obtained according to the extension of the drawings provided without creative efforts.

The structures, proportions, sizes, etc. shown in this specification are only used to cooperate with the contents disclosed in the specification, so as to be understood and read by those who are familiar with the technology, and are not used to limit the conditions for the implementation of the present invention, so there is no technical The substantive meaning above, any modification of the structure, the change of the proportional relationship or the adjustment of the size should still fall within the technical content disclosed in the present invention without affecting the effect and the purpose that the present invention can produce. within the range that can be covered.

1 is a flowchart of a specific embodiment of a distance testing method provided by the present invention;

Figure 2 is a schematic diagram of the binocular ranging principle of the binocular stereo camera;

FIG. 3 is a schematic diagram of a ranging blind area obtained based on the binocular ranging of FIG. 2;

FIG. 4 is a schematic diagram of the monocular ranging principle of the binocular stereo camera;

Fig. 5 is the implementation flow chart under a kind of real-time scene of the distance test method shown in Fig. 1;

6 is a structural block diagram of a specific embodiment of the distance testing device provided by the present invention;

FIG. 7 is a structural block diagram of a specific implementation manner of the distance testing system provided by the present invention.

Description of reference numbers:

100-blind zone acquisition unit 200-first ranging result output unit

300-Second ranging result output unit 400-Position acquisition unit 500-Parallax ratio acquisition unit

Detailed ways

The embodiments of the present invention are described below by specific specific embodiments. Those who are familiar with the technology can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the present invention. , not all examples. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The distance testing method based on the binocular camera provided by the present invention uses the combined testing method of the monocular camera and the binocular camera to avoid the problem of blind spots existing in the binocular ranging, so as to solve the obstacle distance detection of the traditional binocular camera. The problem of low ranging accuracy caused by the time limit of blind spots. In a specific embodiment, as shown in Figure 1, the method includes:

S1: Acquire the blind spot position of the binocular camera within the preset image range; the obtaining of the blind spot position specifically includes acquiring the internal and external parameters and field of view of the binocular camera, and obtaining the blind spot position according to the internal and external parameters and the field of view.

In the actual operation process, according to the principle of binocular ranging shown in Figure 2, the following formula can be obtained from the triangular geometric relationship in Figure 2:

Among them, f represents the focal length of the camera, and B represents the distance between the centers of the two cameras. If D=X _left -X _right , the distance in the z direction can be obtained as:

It can be seen from this that binocular ranging needs to find the imaging position of a certain point in space on the left and right cameras to calculate the distance. If the point can only appear on the imaging surface of one camera, the corresponding parallax cannot be obtained. There is no distance information, thus forming a ranging blind area as shown in Figure 3.

S2: It is determined that the target to be measured is located in the blind spot, and the target to be measured is measured using a monocular, and the ranging result obtained by the monocular ranging is output. That is to say, when it is determined that the target to be measured is just in the blind spot of the camera When the area is large, binocular measurement cannot be detected. In this case, the left-eye camera or the right-eye camera needs to be used for measurement, and a relatively accurate ranging result is output through the monocular.

As shown in Figure 4, according to the principle of monocular ranging, the parameter information of the target to be measured in the blind spot can be obtained by the method of monocular ranging. In Figure 4, θe represents the angle between the main optical axis of the camera and the horizontal ground, OE represents the main optical axis of the camera, OB represents the intersection of the target to be ranged and the ground and the line where the optical center of the camera is located, f represents the focal length of the camera, Hcam represents the installation height of the camera from the ground, and θb represents the angle between the OB line and the OE line. Usually, the V point projected by OZ onto the imaging surface is called the vanishing point, which can be obtained by calibrating the camera pose. In the image coordinate system, ye represents the y-direction coordinate of the vanishing point, y0 represents the y-direction coordinate of the optical center point, which can be obtained by calibrating the camera's internal parameters, and yb represents the y-direction coordinate of the intersection point B of the target to be ranged and the ground.

The Z-direction distance of the target to be measured is obtained through the geometric relationship as:

S3: It is determined that the target to be measured is not located in the blind area, then the target to be measured is measured separately by monocular and binocular, and after the obtained monocular ranging result and binocular ranging result are weighted, Obtain and output the comprehensive ranging result, that is to say, when it is determined that the target to be measured is not in the blind spot, binocular and monocular data are collected respectively, and the weight of the obtained data is calculated to output a more accurate final result.

Specifically, in a specific embodiment, as shown in FIG. 5 , the implementation process of the method includes:

Collect the left eye image and the right eye image under the same frame image including the target to be tested, and obtain a disparity map;

Extract the position information of the object to be tested in the disparity map through machine learning or feature detection;

Use a rectangular frame to select the location information;

obtaining the effective parallax ratio of the target to be tested within the range of the feature extraction frame;

If it is determined that the effective parallax ratio is lower than the set threshold, and the target to be measured is in the blind area, the target to be measured is measured using a monocular, and the ranging result obtained by the monocular distance measurement is output;

It is determined that the effective parallax ratio is higher than the set threshold, and the target to be measured is not located in the blind spot, then the target to be measured is measured separately by monocular and binocular, and the obtained monocular ranging result is compared with that. After the binocular ranging results are weighted, the comprehensive ranging results are obtained and output.

The comprehensive ranging result is obtained by the formula Dz=w*dist1+(1-w)*dist2;

Wherein, w is a weight, w is a parameter positively related to the effective disparity ratio, dist1 is a binocular ranging result, and dist2 is a monocular ranging result.

Further, the functional relationship between the weight and the effective disparity ratio includes:

Wherein, n is the effective parallax ratio, and _nmax is the maximum value of the set threshold.

In the above-mentioned specific embodiments, the distance testing method based on the binocular camera provided by the present invention first obtains the blind spot position of the binocular camera within the preset image range, and when it is determined that the target to be tested is in the blind spot position, the monocular camera is used to measure the blind spot. When it is determined that the target to be measured is not in the blind spot position, the target to be measured is measured by monocular and binocular respectively, and the After the obtained monocular ranging results and binocular ranging results are weighted, the comprehensive ranging results are obtained and output. This method solves the problem that the target distance detection purely through binocular parallax is easily limited by the environment and the blind spot of binocular matching. It combines the respective advantages of the binocular parallax ranging principle and the monocular ranging principle. Combined with the distance measurement, the distance measurement accuracy is improved, thus solving the technical problem of low range measurement accuracy caused by the limitation of the blind spot when the traditional binocular camera detects the distance of obstacles.

In addition to the above distance measuring method, the present invention also provides a distance testing device based on a binocular camera, which is used to implement the above method. In a specific implementation manner, as shown in FIG. 6 , the device includes:

The blind spot obtaining unit 100 is used to obtain the blind spot position of the binocular camera within the preset image range; obtaining the blind spot position specifically includes obtaining the internal and external parameters and the field of view of the binocular camera, and obtaining all the internal and external parameters and the field of view according to the internal and external parameters. the blind spot location.

The first ranging result output unit 200 is used to determine that when the target to be measured is located in the blind area, the target to be measured is measured using a monocular, and the ranging result obtained by the monocular ranging is output, that is, when When it is determined that the target to be measured is just in the blind spot area of the camera, the binocular measurement cannot be detected. In this case, the left-eye camera or the right-eye camera must be used for measurement, and a more accurate ranging result is output through the monocular.

The second ranging result output unit 300 is used to determine that the target to be measured is not located in the blind spot, then use monocular and binocular to measure the distance of the target to be measured separately, and compare the obtained monocular ranging result with the After the binocular ranging results are weighted, the comprehensive ranging results are obtained and output. That is to say, when it is determined that the target to be measured is not in the blind spot, binocular and monocular data are collected respectively, and the obtained data is weighted calculation to output a more accurate final result.

In order to obtain the accurate blind spot position, the device also includes:

The position acquisition unit 400 is configured to collect a binocular image of the target to be measured, extract the position information of the target to be measured in the binocular image, and mark the position information through a feature extraction frame;

A parallax ratio acquiring unit 500, configured to acquire the effective parallax ratio of the object to be measured within the range of the feature extraction frame;

At this time, the first ranging result output unit is further configured to determine that the effective parallax ratio is lower than a set threshold and the target to be measured is located in a blind area, and the target to be measured is measured using a monocular. , and output the ranging result obtained by monocular ranging;

The second ranging result output unit is further configured to determine that the effective parallax ratio is higher than a set threshold and the target to be measured is not located in the blind spot, then use monocular and binocular to respectively measure the target to be measured. Ranging, and after the weight calculation of the obtained monocular ranging results and binocular ranging results, the comprehensive ranging results are obtained and output.

In the above specific embodiment, the distance testing device based on the binocular camera provided by the present invention first obtains the blind spot position of the binocular camera within the preset image range, and when it is determined that the target to be tested is in the blind spot position, the monocular camera is used to measure the blind spot. When it is determined that the target to be measured is not in the blind spot position, the target to be measured is measured by monocular and binocular respectively, and the After the obtained monocular ranging results and binocular ranging results are weighted, the comprehensive ranging results are obtained and output. This method solves the problem that the target distance detection purely through binocular parallax is easily limited by the environment and the blind spot of binocular matching. It combines the respective advantages of the binocular parallax ranging principle and the monocular ranging principle. Combined with the distance measurement, the distance measurement accuracy is improved, thus solving the technical problem of low range measurement accuracy caused by the limitation of the blind spot when the traditional binocular camera detects the distance of obstacles.

According to a third aspect of the embodiments of the present invention, the present invention further provides a distance testing system based on a binocular camera. As shown in FIG. 7 , the system includes: a processor 201 and a memory 202;

the memory for storing one or more program instructions;

The processor is configured to execute one or more program instructions for executing the method as described above.

Corresponding to the foregoing embodiments, an embodiment of the present invention further provides a computer storage medium, where the computer storage medium includes one or more program instructions. Wherein, the one or more program instructions are used to perform the method as described above by a distance testing system.

In this embodiment of the present invention, the processor may be an integrated circuit chip, which has signal processing capability. The processor may be a general-purpose processor, a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field programmable gate array (FPGA for short), or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

Various methods, steps, and logical block diagrams disclosed in the embodiments of the present invention can be implemented or executed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in conjunction with the embodiments of the present invention may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art. The processor reads the information in the storage medium, and completes the steps of the above method in combination with its hardware.

The storage medium may be memory, eg, may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory.

Among them, the non-volatile memory may be a read-only memory (Read-Only Memory, referred to as ROM), a programmable read-only memory (Programmable ROM, referred to as PROM), an erasable programmable read-only memory (Erasable PROM, referred to as EPROM) , Electrically Erasable Programmable Read-Only Memory (Electrically EPROM, EEPROM for short) or flash memory.

The volatile memory may be a random access memory (Random Access Memory, RAM for short), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic RAM (DRAM), Synchronous DRAM, referred to as SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, referred to as DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, referred to as ESDRAM), synchronous connection dynamic random access memory (Synchlink DRAM) , referred to as SLDRAM) and direct memory bus random access memory (DirectRambus RAM, referred to as DRRAM).

The storage medium described in the embodiments of the present invention is intended to include, but not limited to, these and any other suitable types of memory.

Those skilled in the art should appreciate that, in one or more of the above examples, the functions described in the present invention may be implemented by a combination of hardware and software. When the software is applied, the corresponding functions may be stored in or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage medium can be any available medium that can be accessed by a general purpose or special purpose computer.

The above specific embodiments further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention, and are not intended to limit the protection scope of the present invention. On the basis of the technical solutions of the present invention, any modifications, equivalent replacements, improvements, etc. made shall be included within the protection scope of the present invention.

Claims (10)

1. a distance testing method based on binocular camera, is characterized in that, described method comprises: Obtain the blind spot position of the binocular camera within the preset image range; If it is determined that the target to be measured is located in the blind area, the target to be measured is measured using a monocular, and the ranging result obtained by the monocular ranging is output; It is determined that the target to be measured is not located in the blind area, then the target to be measured is measured separately by monocular and binocular, and after the obtained monocular ranging result and binocular ranging result are weighted, the result is obtained. Output comprehensive ranging results. 2. distance testing method according to claim 1, is characterized in that, described method also comprises: Collecting a binocular image of the target to be measured, extracting the position information of the target to be measured in the binocular image, and marking the position information through a feature extraction frame; obtaining the effective parallax ratio of the target to be tested within the range of the feature extraction frame; If it is determined that the effective parallax ratio is lower than the set threshold, and the target to be measured is in the blind area, the target to be measured is measured using a monocular, and the ranging result obtained by the monocular distance measurement is output; It is determined that the effective parallax ratio is higher than the set threshold, and the target to be measured is not located in the blind spot, then the target to be measured is measured separately by monocular and binocular, and the obtained monocular ranging result is compared with that. After the binocular ranging results are weighted, the comprehensive ranging results are obtained and output. 3. The distance testing method according to claim 2, wherein the binocular image of the target to be measured is collected, the position information of the target to be measured in the binocular image is extracted, and the feature extraction frame is used to mark the described Location information, including: Collect the left eye image and the right eye image under the same frame image including the target to be tested, and obtain a disparity map; Extract the position information of the object to be tested in the disparity map through machine learning or feature detection; Use a rectangular box to frame the location information. 4. The distance testing method according to claim 3, wherein the comprehensive ranging result is obtained by the formula Dz=w*dist1+(1-w)*dist2; Wherein, w is a weight, w is a parameter positively related to the effective disparity ratio, dist1 is a binocular ranging result, and dist2 is a monocular ranging result. 5. The distance testing method according to claim 4, wherein the functional relationship between the weight and the effective parallax ratio comprises:

Wherein, n is the effective parallax ratio, and _nmax is the maximum value of the set threshold. 6. The distance testing method according to any one of claims 1-5, wherein the acquiring the blind spot position of the binocular camera within a preset image range specifically includes: Obtain the internal and external parameters and field of view of the binocular camera; The blind spot position is acquired according to the internal and external parameters and the field of view angle. 7. A distance testing device based on a binocular camera for implementing the method according to claims 1-6, wherein the device comprises: The blind spot acquisition unit is used to obtain the blind spot position of the binocular camera within the preset image range; The first ranging result output unit is used to determine that when the target to be measured is located in the blind area, use a monocular to measure the distance of the target to be measured, and output the ranging result obtained by the monocular ranging; The second ranging result output unit is used to determine that the target to be measured is not located in the blind area, then use monocular and binocular to measure the distance of the target to be measured separately, and compare the obtained monocular ranging result with the binocular range. After the weight of the visual ranging result is calculated, the comprehensive ranging result is obtained and output. 8. The distance testing device according to claim 7, wherein the device further comprises: a position acquisition unit, configured to collect a binocular image of the target to be measured, extract the position information of the target to be measured in the binocular image, and mark the position information through a feature extraction frame; A parallax ratio acquiring unit, configured to acquire the effective parallax ratio of the object to be tested within the range of the feature extraction frame; The first ranging result output unit is further configured to determine that the effective parallax ratio is lower than a set threshold and the target to be measured is located in a blind area, then use a monocular to measure the distance to the target to be measured, and Output the ranging result obtained by monocular ranging; The second ranging result output unit is further configured to determine that when the effective parallax ratio is higher than a set threshold and the target to be measured is not located in the blind spot, then monocular and binocular are used to measure the target to be measured. Ranging is measured separately, and after the obtained monocular ranging results and binocular ranging results are weighted, the comprehensive ranging results are obtained and output. 9. A distance testing system based on a binocular camera, wherein the system comprises: a processor and a memory; the memory for storing one or more program instructions; The processor is configured to execute one or more program instructions for performing the method of any one of claims 1-6. 10. A computer storage medium, wherein the computer storage medium contains one or more program instructions, and the one or more program instructions are used to be executed by a binocular camera-based distance testing system as claimed in the claim The method of any one of claims 1-6.

Images