CN116228889A

CN116228889A - Mobile calibration device, camera array system calibration device and method

Info

Publication number: CN116228889A
Application number: CN202310469271.0A
Authority: CN
Inventors: 蒲先坤; 王昕�; 高欣健; 马一鸣; 石磊; 高隽; 杨天; 王玲妹
Original assignee: Hefei University of Technology
Current assignee: Hefei University of Technology
Priority date: 2023-04-27
Filing date: 2023-04-27
Publication date: 2023-06-06
Anticipated expiration: 2043-04-27
Also published as: CN116228889B

Abstract

The invention provides a mobile calibration device, a camera array system calibration device and a method, which belong to the technical field of computational imaging, wherein the method comprises the following steps: adopting a camera of a camera array system to acquire images of a calibration plate of the mobile calibration device to obtain X multiplied by Y images in a plurality of two-dimensional arrays with depths corresponding to the respective depths, wherein the two-dimensional arrays are X rows and Y columns, and X and Y are positive integers; based on X multiplied by Y images in the two-dimensional array corresponding to the plurality of depths, the camera plane parallax calibration of the camera array system, the camera plane depth calibration of the camera array system and the calibration of each viewpoint base line of the camera plane of the camera array system are executed, and the automatic calibration of the depths, the parallaxes and the base lines can be realized, so that the complexity of calibration is reduced, and the calibration efficiency and the calibration precision are improved.

Description

Mobile calibration device, camera array system calibration device and method

Technical Field

The invention relates to the technical field of computational imaging, in particular to a mobile calibration device, a camera array system calibration device and a camera array system calibration method.

Background

In the related art, aiming at the fields of camera array computational imaging and target detection, a manual calibration mode is adopted to calibrate the camera array, the calibration complexity is high, and the calibration efficiency and the calibration precision are low.

Therefore, how to reduce the complexity and improve the calibration efficiency and precision when calibrating the camera array becomes a problem to be solved.

Disclosure of Invention

The invention provides a mobile calibration device, a camera array system calibration device and a camera array system calibration method, which are used for solving the problems of high complexity and low calibration efficiency and precision of the calibration of a camera array in the prior art.

The invention provides a mobile calibration device for calibrating a camera array system, which comprises: the device comprises a calibration plate 17, a liftable upright post 20 of the calibration plate, a sliding platform 21, a guide rail sliding block 22, a scale 24, a guide rail sliding block base 23, a translation sliding table 27, a positioning pointer 25, an angle adjusting optical axis 19, a second servo motor 28, a second servo driver 29 and a position depth calibration device 30;

wherein the liftable upright post 20 of the calibration plate is arranged on the slidable platform 21;

the guide rail sliding block 22 and the scale 24 are arranged at preset positions on the guide rail sliding block base 23, and the translation sliding table 27 is arranged on the guide rail sliding block base 23;

the slidable platform 21 is connected with the translation sliding table 27 so that the slidable platform 21 is slidable on the guide rail slider 22;

The positioning pointer 25 is mounted on the slidable platform 21, and the positioning pointer 25 is parallel to the scale 24;

the angle adjusting optical axis 19 is vertically arranged on a liftable upright post 20 of the calibration plate, and the calibration plate is horizontally arranged on the angle adjusting optical axis 19;

the second servo motor 28 is mounted on the translation sliding table 27, and the second servo motor 28 is connected with a first servo driver in the camera array system, and the second servo motor 28 is connected with a second servo driver 29.

The invention provides a camera array system calibration device, comprising: a camera array system and the mobile calibration device;

the imaging plane of the camera array system is parallel to the calibration plate of the mobile calibration device.

The invention provides a camera array system calibration method, which is applied to the camera array system calibration device, and comprises the following steps:

adopting a camera of a camera array system to acquire images of a calibration plate of the mobile calibration device to obtain X multiplied by Y images in a plurality of two-dimensional arrays with depths corresponding to the respective depths, wherein the two-dimensional arrays are X rows and Y columns, and X and Y are positive integers;

And performing parallax calibration of a camera plane of the camera array system, depth calibration of the camera plane of the camera array system and base line calibration of each view point of the camera plane of the camera array system based on the X multiplied by Y images in the two-dimensional array corresponding to the plurality of depths respectively.

According to the camera array system calibration method provided by the invention, the image acquisition is carried out on the calibration plate of the mobile calibration device, and the method comprises the following steps:

and acquiring images of the calibration plates of the mobile calibration device by a two-dimensional plane array motion image acquisition mode or a two-dimensional plane array fixed image acquisition mode.

According to the camera array system calibration method provided by the invention, before the camera of the camera array system is adopted, the calibration plate of the mobile calibration device is subjected to image acquisition in a two-dimensional plane array motion image acquisition mode to obtain X multiplied by Y images in a plurality of two-dimensional arrays with respective depths, the method further comprises the following steps:

measuring distances between the calibration plate and the camera plane at a plurality of positions by a laser range finder;

calculating the average depth of the calibration plate and the camera plane under the condition that the calibration plate is parallel to the camera plane, and taking the average depth as the initial depth of the two-dimensional plane array motion;

And determining the positions of the viewpoints on the two-dimensional plane and the baseline distance between the viewpoints.

According to the camera array system calibration method provided by the invention, the camera plane parallax calibration of the camera array system is executed, and the method comprises the following steps:

taking an image corresponding to a central viewpoint as a reference for X multiplied by Y images in a two-dimensional array corresponding to each depth, and obtaining parallax values between each viewpoint and the central viewpoint based on image corner position coordinates of each viewpoint in the X multiplied by Y images in the two-dimensional array corresponding to the depth;

and obtaining a depth-parallax relation model based on parallax values between each viewpoint and the central viewpoint in the two-dimensional array corresponding to the plurality of depths.

According to the camera array system calibration method provided by the invention, the method further comprises the following steps:

calculating and obtaining parallax values of all viewpoints corresponding to the depth to be detected based on the depth to be detected and the depth-parallax relation model;

taking the parallax value of each viewpoint corresponding to the depth to be detected as an array element to obtain a homography matrix;

transforming the images of the viewpoints corresponding to the depth to be detected to the center view angle based on the homography matrix to obtain the transformed images of the 4 directions of the viewpoints;

Respectively fusing the transformed images of the 4 directions of each viewpoint to obtain a calculated imaging result under the depth to be detected;

and obtaining the polarization state, the polarization degree and the polarization angle information based on the calculated imaging result.

According to the camera array system calibration method provided by the invention, the camera plane depth calibration of the camera array system is executed, and the method comprises the following steps:

taking a reference distance measured by a laser range finder in a camera array system as an initial depth position, and taking a position fed back by an encoder in a moving state as a depth position;

and calibrating the depth direction according to the initial depth position and the depth position fed back by the encoder.

According to the calibration method of the camera array system provided by the invention, the calibration of the base line of each viewpoint of the camera plane of the camera array system is executed, and the method comprises the following steps:

taking the zero position fed back by an absolute position encoder in a camera array system as a reference, taking one step length position of the camera moving on the translation sliding table as a base line distance, taking the position fed back by the encoder on the translation sliding table as the position corresponding to different view points in the row direction, and collecting images corresponding to the positions fed back by the encoder on the translation sliding table under a plurality of depths;

calibrating the baseline distance of each viewpoint in the row direction according to the zero point position and the position fed back by the encoder on the translation sliding table;

According to the plurality of depths, parallax between the view point corresponding to the position fed back by the encoder on the translation sliding table and the central view point under the plurality of depths and the baseline distance of each view point in the row direction, a camera array model is obtained, and the camera array model is used for representing the relation between at least two of the depths, the parallax and the baseline.

The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor realizes any one of the camera array system calibration methods when executing the program.

The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements any of the camera array system calibration methods described above.

The invention also provides a computer program product comprising a computer program which when executed by a processor implements any of the camera array system calibration methods described above.

According to the mobile calibration device and the camera array system calibration method using the same, the X multiplied by Y images in the two-dimensional arrays corresponding to the depths can be obtained through image acquisition of the calibration plate of the mobile calibration device, the camera plane parallax calibration of the camera array system, the camera plane depth calibration of the camera array system and the camera plane view point base line calibration of the camera array system can be respectively executed based on the X multiplied by Y images in the two-dimensional arrays corresponding to the depths, the depth-parallax-base line model is obtained, and the automatic calibration of the depth, the parallax and the base line can be realized through joint motion with the camera array system and the mobile calibration device, so that the complexity of calibration is reduced, and the calibration efficiency and the calibration precision are improved.

Drawings

In order to more clearly illustrate the invention or the technical solutions of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the invention, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of a mobile calibration device according to the present invention;

FIG. 2 is a schematic diagram of the structure of the square calibration plate provided by the invention;

FIG. 3 is a front view of a mobile calibration device provided by the present invention;

FIG. 4 is a side view of a mobile calibration device provided by the present invention;

FIG. 5 is a top view of the mobile calibration device provided by the present invention;

FIG. 6 is a schematic diagram of a calibration device of a camera array system according to the present invention;

FIG. 7 is a schematic diagram of a camera array system according to the present invention;

FIG. 8 is a schematic flow chart of a calibration method of a camera array system provided by the invention;

FIG. 9 is a schematic diagram of a coordinate system of a camera array system provided by the present invention;

FIG. 10 is a schematic view of the camera array system provided by the present invention for each viewpoint of a camera plane;

Fig. 11 is a horizontal depth-disparity plot for all viewpoints in a range of depths provided by the present invention;

fig. 12 is a vertical depth-disparity plot for all viewpoints in a range of depths provided by the present invention;

fig. 13 is one of the depth-parallax graphs of the horizontal direction of the viewpoint 1 in a certain depth range provided by the present invention;

FIG. 14 is a second view of a horizontal depth-disparity plot for view 1 in a depth range provided by the present invention;

fig. 15 is a third view of a depth-parallax graph of the horizontal direction of the viewpoint 1 in a certain depth range provided by the present invention;

fig. 16 is one of the depth-parallax graphs of the vertical direction of the viewpoint 1 in a certain depth range provided by the present invention;

FIG. 17 is a second view of a depth-disparity plot for the vertical direction of view 1 in a range of depths provided by the present invention;

fig. 18 is a third of the depth-parallax graphs of the vertical direction of the viewpoint 1 in a certain depth range provided by the present invention;

fig. 19 is one of the depth-parallax graphs of the horizontal direction of the viewpoint 2 in a certain depth range provided by the present invention;

fig. 20 is one of the depth-parallax graphs of the vertical direction of the viewpoint 2 in a certain depth range provided by the present invention;

FIG. 21 is a second view of a horizontal depth-disparity plot for view 2 in a depth range provided by the present invention;

FIG. 22 is a second view of a depth-disparity plot for the vertical direction of viewpoint 2 in a range of depths provided by the present invention;

FIG. 23 is a schematic view of a camera array model according to the present invention;

FIG. 24 is a schematic view of one of the depth-parallax-baseline curves provided by the present invention;

FIG. 25 is a schematic view of a scatter plot of camera array parameters in 3D space provided by the present invention;

FIG. 26 is a second view of a camera array model according to the present invention;

FIG. 27 is a second schematic view of a depth-parallax-baseline curve provided by the present invention;

FIG. 28 is a second view of a scatter plot of camera array parameters in 3D space according to the present invention;

FIG. 29 is a third view of a camera array model according to the present invention;

FIG. 30 is a third schematic view of a depth-disparity-baseline curve provided by the present invention;

FIG. 31 is a third view of the scatter plot of camera array parameters in 3D space provided by the present invention;

fig. 32 is a schematic structural diagram of an electronic device provided by the present invention.

Reference numerals:

1: an industrial personal computer; 2: a display device; 3: a motion control card; 4: a first servo driver; 5: an IO control card; 6: a first servo motor; 7: a translation sliding table; 8: lifting the electric push rod; 9: an electric push rod fixing plate; 10: a camera fixing plate; 11: a camera; 12: a laser range finder; 13: a planar fixing plate; 14: a bracket surrounding structure; 15: a fixed base bracket; 16A: a first photoelectric switch; 16B: a second photoelectric switch; 17: a calibration plate; 18: a calibration plate fixing bracket; 19: angle-adjusting the optical axis; 20: liftable stand columns of the calibration plate; 21: a slidable platform; 22: a guide rail slide block; 23: a guide rail slide block base; 24: a ruler; 25: positioning a pointer; 26: a sliding platform fixing plate; 27: a translation sliding table; 28: and a second servo motor.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The following will be described first:

camera array computed imaging (synthetic aperture imaging) is a very powerful computational imaging method that can sharpen objects at arbitrary focal planes, enhancing the signal-to-noise ratio of camouflage targets in the image. When the polarization array calculates imaging, when the measured targets are at different depths, the parallaxes among the cameras are different, the base line distances of the cameras are different, and the parallaxes are also different. In the related art, in the calculation imaging, a target image with a certain known depth is shot, the relative depth of a target and the target is measured to calculate the relative position relation between cameras, and then the known depth of the target and the relative depth of the target are used to calculate the parallax relation between cameras at the depth, so that the calculation imaging of the target is realized. According to the method, a target image is shot when each target detection task is needed, the relative pose of each camera viewpoint is solved, and the relative distance of the target relative to the target is measured, so that target detection is inconvenient to realize well.

The integration and automation of the target detection equipment are one of key factors for measuring the performance of the detection equipment. Aiming at the field of mobile array target detection, the related art does not have a calibration device and an automatic calibration method for a polarization array, the accuracy of the calibration of an array system is related to the accuracy of target detection of a computing imaging system, equipment is required to be calibrated in detection tasks of different scenes, the automatic calibration device is applied to the array target detection system, the accuracy of target detection can be improved, the complex calibration process is reduced, and the calibration efficiency and accuracy are improved.

In order to solve the problems, the invention provides a mobile calibration device, a camera array system calibration device and a camera array system calibration method, which are used for researching the parallax relation between homography matrixes of different depths of a target in an environment when a polarization array is calculated, and realizing homography matrix acquisition and parallax calibration when the target is at the different depths.

FIG. 1 is a schematic structural diagram of a mobile calibration device according to the present invention, for calibrating a camera array system, as shown in FIG. 1, the mobile calibration device includes: the device comprises a calibration plate 17, a liftable upright post 20 of the calibration plate, a sliding platform 21, a guide rail sliding block 22, a scale 24, a guide rail sliding block base 23, a translation sliding table 27, a positioning pointer 25, an angle adjusting optical axis 19, a second servo motor 28, a second servo driver 29 and a position depth calibration device 30;

Alternatively, the camera in the present invention may be a polarization camera;

alternatively, the image in the present invention may be a polarized image;

alternatively, the camera array may be a polarized camera array.

Optionally, the mobile calibration device may also include a calibration plate mounting bracket 18.

Optionally, the mobile calibration device may also include a sliding platform securing plate 26.

Alternatively, the calibration plate fixing bracket 18 may be mounted on the angle-adjusting optical axis 19.

Alternatively, the calibration plate 17 may be fixed to the calibration plate fixing bracket 18.

Alternatively, the slidable platform 21 and the translation stage 27 may be connected by a slide platform fixing plate 26.

Alternatively, the second servomotor 28 may contain a position encoder e.

Alternatively, the rail slider 22 may be a fixed rail slider.

Alternatively, the calibration plate 17 may be a black and white checkered calibration plate.

Fig. 2 is a schematic structural diagram of the square grid calibration plate provided by the invention, as shown in fig. 2, square grids exist on the square grid calibration plate, and the number of the square grids in the row direction and the column direction is not odd or even.

Optionally, the square grid number side length and the number of the grid calibration plates can be properly adjusted according to different application requirements.

FIG. 3 is a front view of the mobile calibration device provided by the present invention, as shown in FIG. 3, including a front view of the mobile calibration device.

FIG. 4 is a side view of the mobile calibration device provided by the present invention, as shown in FIG. 4, including a side view of the mobile calibration device.

FIG. 5 is a top view of the mobile calibration device provided by the present invention, as shown in FIG. 5, including a top view of the mobile calibration device.

Alternatively, to reduce calibration errors, the calibration personnel may calibrate the mobile calibration device before using the mobile calibration device.

Optionally, the calibration personnel can adjust the foot height of the guide rail slide block base 23 to fix the calibration plate fixing bracket 18 and horizontally calibrate the calibration plate 17 on the movable calibration device, adjust the angle adjustment optical axis 19, and vertically calibrate the same.

Optionally, after the calibration of the mobile calibration device is completed, no recalibration is required until the next time the mobile calibration device is installed.

The mobile calibration device provided by the invention is used for calibrating a camera array system, and can realize automatic calibration of depth, parallax and base line by matching with a camera array system calibration method, so that the complexity of calibration is reduced, and the calibration efficiency and precision are improved.

Fig. 6 is a schematic structural diagram of a calibration device for a camera array system according to the present invention, and as shown in fig. 6, the calibration device for a camera array system includes: a camera array system and the mobile calibration device;

Fig. 7 is a schematic structural diagram of a camera array system provided by the present invention, as shown in fig. 7, the camera array system includes: the device comprises an industrial personal computer 1, a display device 2, a motion control card 3, a first servo driver 4, an IO control card 5, a first servo motor 6, a translation sliding table 7, a lifting electric push rod 8, an electric push rod fixing plate 9, a camera fixing plate 10, a camera 11, a laser range finder 12, a plane fixing plate 13, a bracket surrounding structure 14, a fixing base bracket 15, a first photoelectric switch 16A and a second photoelectric switch 16B.

Optionally, the first servomotor 6 in the camera array system contains an absolute position encoder a.

Optionally, the elevating electric push rod 8 in the camera array system comprises a support frame 8a, a stepper motor 8b and a stepper driver 8c, wherein the stepper motor 8b comprises a relative position encoder d.

Alternatively, in the camera array system, the bracket surrounding structure 14 and the fixed base bracket 15 may be integrally installed and fixed, the left and right elevating electric pushers 8 may be respectively installed on the platform of the fixed base bracket 15 by using the electric pushers fixing plate 9, and the plane fixing plate 13 may be installed on the supporting frame 8a of the electric pushers 8.

Alternatively, in the camera array system, the first servo motor 6, the photoelectric switch 16A, and the photoelectric switch 16B may be respectively mounted on the translation slide table 7, and the translation slide table 7 may be fixedly mounted on the plane fixing plate 13.

Alternatively, in the camera array system, the camera 11 and the laser rangefinder 12 may be mounted on the translation slide table 7 through the camera fixing plate 10, and the camera 11 may be perpendicular to the movement direction of the translation slide table 7.

Alternatively, in the camera array system, the electronic components such as the industrial personal computer 1, the display device 2, the motion control card 3, the servo driver 4, the io control card 5, the first servo motor 6, the stepping motor 8b, the stepping driver 8c, and the like may be connected.

Alternatively, to reduce calibration errors, a calibration person may calibrate the camera array system prior to calibrating using the camera array system.

Alternatively, the calibration personnel may perform overall horizontal calibration on the camera array system, which includes performing horizontal calibration on the plane fixing plate 13 and the camera fixing plate 10, respectively, so that the installation positions of the camera 11 and the laser rangefinder 12 on the plane on which the camera fixing plate 10 is located are horizontal.

Optionally, after the camera array system is calibrated, no recalibration is required until the next time the camera array system is installed.

Optionally, in order to reduce calibration errors, the calibration personnel may calibrate the camera array system calibration device before using the camera array system for calibration, and after completing calibration of the camera array system and the mobile calibration device.

Alternatively, the calibration personnel may make the imaging plane of the camera 11 of the camera array system parallel to the calibration plate 17 of the mobile calibration device, and finally make a joint adjustment of the camera array system and the calibration plate 17 as a whole.

According to the camera array system calibration device, the camera array system is calibrated by moving the calibration device, and the automatic calibration of depth, parallax and base lines can be realized by matching with the camera array system calibration method, so that the complexity of calibration is reduced, and the calibration efficiency and precision are improved.

Fig. 8 is a schematic flow chart of a camera array system calibration method provided by the invention, as shown in fig. 8, the camera array system calibration method is applied to a camera array system calibration device, and the method comprises the following steps:

step 800, adopting a camera of a camera array system to acquire images of a calibration plate of the mobile calibration device, and obtaining X multiplied by Y images in a plurality of two-dimensional arrays with depths corresponding to each other, wherein the two-dimensional arrays are X rows and Y columns, and X and Y are positive integers;

alternatively, the depth may be the distance of the plane of the calibration plate of the mobile calibration device from the plane of the camera array.

Alternatively, X rows in the two-dimensional array may be preset values, and the calibration personnel may set the value of X as needed.

Alternatively, the Y column in the two-dimensional array may be a preset value, and the calibration personnel may set the value of Y as needed.

For example, the calibration personnel may set the value of X to 5 and the value of Y to 10, i.e., the two-dimensional array is X rows and Y columns.

Step 810, performing parallax calibration of a camera plane of the camera array system, depth calibration of the camera plane of the camera array system, and baseline calibration of each viewpoint of the camera plane of the camera array system based on the x×y images in the two-dimensional arrays corresponding to the plurality of depths, respectively.

Alternatively, in order to obtain the horizontal parallax and vertical parallax parameter table of each view in the camera array, it is necessary to perform parallax calibration of the camera plane of the camera array system based on x×y images in the two-dimensional array to which a plurality of depths respectively correspond.

Alternatively, in order to obtain a two-dimensional planar array multi-viewpoint image for each depth, it is necessary to perform camera plane depth calibration of the camera array system based on x×y images in a two-dimensional array to which a plurality of depths respectively correspond.

Alternatively, in order to obtain the baseline distance of each viewpoint in the row direction, it is necessary to perform the camera plane each viewpoint baseline calibration of the camera array system based on x×y images in the two-dimensional array to which a plurality of depths respectively correspond.

According to the camera array system calibration method provided by the invention, through image acquisition of the calibration plate of the mobile calibration device, X multiplied by Y images in the two-dimensional array corresponding to the plurality of depths respectively can be obtained, based on the X multiplied by Y images in the two-dimensional array corresponding to the plurality of depths respectively, the camera plane parallax calibration of the camera array system, the camera plane depth calibration of the camera array system and the base line calibration of each viewpoint of the camera plane of the camera array system can be respectively executed, a depth-parallax-base line model is obtained, and the automatic calibration of the depths, the parallaxes and the base lines is realized by combined motion with the camera array system and the mobile calibration device, so that the complexity of calibration is reduced, and the calibration efficiency and the calibration precision are improved.

Optionally, the image acquisition on the calibration board of the mobile calibration device includes:

Optionally, the two-dimensional plane array motion imaging mode may include the following steps:

before the camera of the camera array system performs image acquisition on a calibration plate of the mobile calibration device, a slidable platform on a linear translation sliding table of the mobile calibration device can be placed at an initial position, and the camera in the camera array system can be located at an origin position of the translation sliding table.

Optionally, the slidable platform on the linear translation sliding table of the mobile calibration device may be placed at an initial position, where the initial position may be manually set as required, for example, setting the upper right, lower right, or upper left of the camera array system as the initial position, which is not limited in the present invention.

Optionally, the origin position of the translation sliding table may be set in an absolute position encoder, and the calibration personnel may set the origin position of the translation sliding table according to actual needs, for example, set the leftmost end, the rightmost end or the uppermost end of the translation sliding table as the origin position, which is not limited in the present invention.

And secondly, the camera of the camera array system can start to acquire images from the original position of the translation sliding table, when the absolute position encoder a in the first servo motor moves by one position, a signal is fed back and the camera is triggered to acquire an image when the absolute position encoder a reaches a set base line position, after all viewpoints are acquired in the row direction, the sliding platform of the movable calibrating device drives the camera of the camera array system to quickly return to the original position of the translation sliding table, and meanwhile, the electric lifting electric push rod of the camera array system moves by one base line distance in the vertical direction.

And step three, repeating the step two, and collecting the view points on each row until all the view points on the two-dimensional plane are collected.

And step four, in the depth direction, a calibrator can set a depth step length and a depth range to be calibrated according to actual requirements.

For example, the calibration personnel can set the depth step to be calibrated to be 0.5 meter and the depth range to be 20 meters.

For example, a calibration person may set the depth step to be calibrated to 1 meter and the depth range to 10 meters.

And fifthly, according to the set depth step length and depth range, a second servo driver of the movable calibration device can drive a second servo motor to rotate, so as to drive a calibration plate on the slidable platform to move a position, a position encoder e of the second servo motor records the position value and transmits the position value to a computer, and then all view points on a two-dimensional plane of the position are acquired until all view points on the two-dimensional plane of each position in the depth direction are acquired and stored.

In one embodiment of the invention, a calibration personnel adopts a two-dimensional plane array motion image acquisition mode of a polarization camera to acquire images of black and white square grid calibration plates in a mobile calibration device by the following steps:

in the first step, the polarization camera array system runs by itself, the slidable platform 21 on the linear translation sliding table 27 is placed at the upper left position of the polarization camera array system, and the polarization camera 11 in the polarization camera array system is located at the leftmost end of the translation sliding table 7.

Step two, starting to run, the sliding platform 21 drives the polarization camera 11 to run to the leftmost end of the translation sliding table 7, the laser range finder 12 measures the distance between the black and white square grid calibration plate 17 and the camera array plane in the mobile calibration device at a plurality of positions, and after verification of parallelism, the average depth is calculated and is set as the initial depth.

And step three, the slidable platform 21 drives the polarization camera 11 to return to the leftmost end of the translation sliding table 7, and the number of view points in the row direction and the column direction and the baseline distance between view points are input.

And step four, the start of acquisition is executed, the polarization camera 11 starts to acquire images from the leftmost end of the translation sliding table 7, when the absolute position encoder a moves by one position, a signal is fed back and the camera is triggered to acquire an image, after all viewpoints are acquired in the row direction, the sliding platform 21 drives the polarization camera 11 to quickly return to the leftmost end of the translation sliding table 7, and meanwhile, the electric lifting electric push rod 8 moves by one baseline distance in the vertical direction.

And fifthly, repeating the process of the third step, and collecting the viewpoints on each row until all the viewpoints on the two-dimensional plane are collected.

Step six, setting the depth step length to be calibrated to be 1 meter and setting the depth range to be calibrated to be 10 meters in the depth direction.

And step seven, the second servo driver 28 drives the second servo motor to rotate so as to drive the black and white square calibration plate on the slidable platform 21 to move for 1 meter, the encoder e of the second servo motor 28 records the position value and transmits the position value to a computer, and then the step four and the step five are executed until the black and white square calibration plate at each position within the range of 10 meters in the depth direction is acquired and stored in a two-dimensional image acquisition mode by the polarization camera array system.

Alternatively, a two-dimensional planar array fixed imaging mode may be used for calibration of the fixed camera array.

According to the camera array system calibration method provided by the invention, the calibration plate of the mobile calibration device can be subjected to image acquisition in a two-dimensional plane array motion image acquisition mode or a two-dimensional plane array fixed image acquisition mode, so that X multiplied by Y images in a plurality of two-dimensional arrays with depths corresponding to each other are obtained.

Optionally, before the camera of the camera array system is used to acquire images of the calibration board of the mobile calibration device in a two-dimensional plane array motion image acquisition mode to obtain x×y images in a two-dimensional array with a plurality of depths corresponding to each other, the method further includes:

Optionally, taking the calibration plate as a black-white square calibration plate as an example, the slidable platform of the mobile calibration device can drive the camera of the camera array system to move to the original position of the translation sliding table, the laser range finder can measure the distance between the black-white square calibration plate and the plane of the camera array in the mobile calibration device at a plurality of positions, the average depth is calculated after the calibration plate is verified to be parallel to the plane of the camera, and the depth is set to be the initial depth of the two-dimensional plane array motion.

Alternatively, the initial depth of the two-dimensional planar array motion may be used as the starting depth position for the array system camera plane depth calibration process.

Alternatively, the camera array system may determine the locations of the viewpoints on the two-dimensional plane, as well as the baseline distance between the viewpoints.

For example, after the initial depth of the two-dimensional plane array motion is determined, the slidable platform of the mobile calibration device can drive the camera to return to the initial position, and the number of view points in the row direction and the column direction and the baseline distance between view points are input.

According to the camera array system calibration method provided by the invention, the initial depth of the two-dimensional plane array motion can be determined by measuring the distances between the calibration plate and the camera plane at a plurality of positions through the laser range finder, and X multiplied by Y images in the two-dimensional array, which correspond to the plurality of depths respectively, can be obtained through the two-dimensional plane array motion image acquisition mode based on the initial depth of the two-dimensional plane array motion.

Optionally, performing a camera plane parallax calibration of the camera array system includes:

Optionally, there is some variation in the depth-disparity of each camera view in the camera array. The following relationship exists for any two views in the camera array:

（1）/>

wherein the method comprises the steps of

For focal length->

Representing the baseline distance between cameras, " >

Representing the pixel size of the camera sensor, +.>

Representing the difference between the images, i.e. the disparity.

Representing the depth of the target from the camera.

Alternatively, assuming that all cameras are on the same plane, since the camera array is fixed and the camera parameters are known, then

，

，

Is known. According to equation (1), it is assumed that there are the following three models that can be used to represent +.>

And->

Is the relation of:

（2）

（3）

（4）

these three models can be referred to as a power function model, a rational number model, and

an exponential function model. In formula (2), the->

Representing depth argument ++>

，

Is a proportional coefficient->

To compensate for the amount. In formula (3), the->

Representing depth argument ++>

，

Is a proportional coefficient->

For index number of times of adjustment, +.>

To compensate for the amount. In formula (4), the drug is->

Representing depth argument ++>

，

And->

Is an exponential coefficient>

And->

Is a proportionality coefficient.

Alternatively, the coefficients may be calculated by nonlinear iterative optimization, provided that the three functional models better represent the relationship of the model parameters.

Alternatively, the camera array system may order the two-dimensional array images of different depths acquired in the camera two-dimensional planar array motion imaging mode in an automatically named manner.

Alternatively, the angular point positions of the square grid in the view point calibration plate image in the row direction and the column direction can be recorded corresponding to the angular point positions of the square grid in the center view point calibration plate image when the mobile calibration device is at each depth position

For calibrating the pixel coordinates of the corner points of the plate image, +.>

Coordinates representing the row direction>

Representing the coordinates of the column direction.

Optionally, for the x×y array images in the two-dimensional array at a certain depth, the camera array system may use the central viewpoint image as a reference, and acquire the angular point position coordinates of each calibration block image of the x×y viewpoints by using an angular point detection algorithm

The camera array system can calculate the angular point position coordinates in the respective viewpoint images>

Coordinates of the position of the corner in relation to the centre viewpoint image>

Obtaining parallax matrixes of X Y group of viewpoint images and central viewpoint images, removing outliers of each group of parallaxes in the X Y group of parallax matrixes by using 3 sigma, and then obtaining average value as parallax value of the viewpoint and the central viewpoint to obtain parallax of each viewpoint in X Y viewpoints relative to the central viewpoint>

。

Alternatively, the camera array system may calculate Z X Y viewpoints at corresponding depths with reference to the center viewpoint image for X X Y array images in a depth range, such as a two-dimensional array of Z steps

Lower parallax.

Fig. 9 is a schematic diagram of a coordinate system of a camera array system provided by the present invention, as shown in fig. 9, a depth direction Z, an array row direction X, and an array column direction Y of the present invention are all based on the present coordinate system, where an array arrangement manner can be adjusted by inputting row and column view points in an upper computer, that is, a row direction baseline and an array direction baseline can be adjusted according to practical application occasions.

Alternatively, the camera array system may decompose the parallaxes of the respective viewpoints in the row direction, the respective viewpoints in the column direction and the central viewpoint at each depth, respectively, to obtain the parallaxes of the respective viewpoints in the row direction relative to the central viewpoint

And parallax +/of each view point with respect to the center view point in the column direction>

。

Alternatively, the camera array system may iterate the calculation using a nonlinear least squares method and LM algorithm to obtain the parameters of the depth-disparity model equations (2), (3), and (4). For different views, the disparity can be represented by the following expression:

（5）

（6）

（7）

（8）

（9）

（10）

wherein in equation (5)

Representing horizontal parallax, & lt & gt>

Is a proportional coefficient->

Indicating depth->

For the baseline distance between cameras, +.>

Is the compensation quantity;

And->

Is an exponential coefficient; in equation (6)>

Representing horizontal parallax, & lt & gt>

Is a proportional coefficient->

Indicating depth->

For index number of times of adjustment, +.>

Is the compensation quantity; in equation (7)>

Representing horizontal parallax, & lt & gt>

And->

Is a proportional coefficient->

And->

Is an exponential coefficient; in equation (8)>

Representing vertical parallax, ++>

Is a proportional coefficient->

Indicating depth->

Is the compensation quantity; in equation (9)>

Representing horizontal parallax, & lt & gt >

Is a proportional coefficient->

Indicating depth->

For index number of times of adjustment, +.>

Is the compensation quantity; in equation (10)>

Representing horizontal parallax, & lt & gt>

And->

Is a proportional coefficient->

Indicating depth->

And

the index coefficient is recorded as follows:

（11）

optionally, for other views in the camera array, the camera array system can also use a nonlinear least square method and an LM algorithm to perform iterative computation, so as to finally obtain depth-parallax relation model parameters of all views in the horizontal direction and the vertical direction.

According to the camera array system calibration method provided by the invention, the depth-parallax relation model can be obtained based on the parallax values between each viewpoint and the central viewpoint in the two-dimensional array corresponding to the plurality of depths by obtaining the parallax values between each viewpoint and the central viewpoint in the two-dimensional array corresponding to the plurality of depths.

Optionally, the camera array system calibration method further includes:

Alternatively, the depth to be detected may be artificially determined, for example, the depth to be detected may be 100 meters if the detection requirement is 100 meters, or 30 meters if the detection requirement is 30 meters, or 30 meters.

Alternatively, the 4 directions may be 0 °,45 °,90 °, and 135 ° directions of the viewpoint.

Optionally, based on the depth to be detected and the depth-parallax relation model, a horizontal parallax parameter table and a vertical parallax parameter table of each view in the camera array can be obtained, and the camera array system can call the parameter table and the model formula to directly calculate the parallax value of each viewpoint corresponding to the depth to be detected.

Optionally, for x×y images in the two-dimensional array at a certain depth, the camera array system may convert each view image into a central view angle by using homography transformation with the image corresponding to the central view point as a reference, so as to obtain x×y 3-order square matrices, where parallax is a matrix element at a certain position in the homography matrix of the homography transformation.

Optionally, for an x×y image in a depth range, such as a two-dimensional array in Z steps, the camera array system may convert each view image into a central view angle by using homography transformation with the image corresponding to the central view point as a reference, to obtain z×x×y 3-order square matrices, and store the z×x×y 3-order square matrices in the 5D array.

Optionally, the camera array system may calculate parallax between each viewpoint and the central viewpoint on the camera array plane in different polarization array arrangement modes, that is, different viewpoint distribution in the row direction and the column direction, and the camera array system may iteratively calculate

Different arrangements, i.e. different number of points and baseline distance +.>

Zhang Shendu-a parallax relationship model parameter table.

Alternatively, the camera array system may iteratively calculate a homography matrix for each view angle relative to a central view angle in the camera array at a depth using a multi-view geometry method and an LM algorithm.

Optionally, after the depth of the reconstruction layer is input, the camera array system may call the depth-parallax relation model parameter table and the model formula, automatically write the parallax value of each viewpoint into the array element of the homography matrix corresponding to the viewpoint, and then transform the view to the central viewing angle by using the new homography matrix, so as to obtain the transformed 4-direction images of each viewpoint.

Optionally, the camera array system may respectively fuse the polarized images of 4 directions of each view angle to obtain the computed imaging results of the polarized images of 4 directions at a certain depth.

Optionally, the camera array system can calculate the polarization state, the polarization degree and the polarization angle information again by using a polarization stokes formula based on the calculated imaging result, and the polarization information is fused and displayed to realize target polarization calculation imaging and target polarization state detection at a certain depth.

According to the camera array system calibration method provided by the invention, based on the depth to be detected and the depth-parallax relation model, the parallax value of each viewpoint corresponding to the depth to be detected can be calculated and obtained, the homography matrix can be obtained by taking the parallax value of each viewpoint corresponding to the depth to be detected as an array element, the transformed images of 4 directions of each viewpoint can be obtained based on the homography matrix, the calculated imaging result under the depth to be detected is obtained after fusion, the polarization state, the polarization degree and the polarization angle information are calculated based on the calculated imaging result, and the polarization information is fused and displayed, so that the target polarization calculation imaging and target polarization state detection under a certain depth are realized.

Optionally, performing camera plane depth calibration of the camera array system includes:

Optionally, when the camera plane depth calibration of the camera array system is performed, the reference distance measured by the laser range finder of the camera array system in the camera array system calibration device may be taken as the initial depth position, the position fed back by the position encoder e of the second servo motor in the mobile calibration device is taken as the depth position, and the depth direction is calibrated according to the initial depth position and the depth position fed back by the position encoder e.

Optionally, each time the calibration plate of the mobile calibration device moves by one position, the position encoder e can feed back the position to the upper computer, the camera array system can collect an array image of the position, and when the mobile calibration device runs out, namely all depth values are recorded, a depth position table of the depth-two-dimensional plane array multi-viewpoint image is obtained.

According to the camera array system calibration method, the depth position table of the depth-two-dimensional plane array multi-view image can be obtained by executing the camera plane depth calibration of the camera array system.

Optionally, performing a camera plane each viewpoint baseline calibration of the camera array system includes:

taking the zero position fed back by an absolute position encoder in a camera array system as a reference, taking one step length position of the camera moving on the translation sliding table as a base line distance, taking the position fed back by the encoder on the translation sliding table as positions corresponding to different view points in the row direction, and collecting images corresponding to the positions fed back by the encoder on the translation sliding table under a plurality of depths;

Optionally, each time the camera on the translation sliding table moves by one position in the camera array system, the encoder can feed back the position to the upper computer and trigger the camera to collect an image of the position, and after the camera moves by M-1 baseline distances, M images in the row direction can be collected to obtain M viewpoints in the row direction. The translation sliding table runs through a stroke, all base line distances of the line are recorded, and a line base line distance-line multi-viewpoint image is obtained.

Optionally, the lifting electric push rod of the camera array system drives the translation sliding table to ascend at a position, the encoder can feed back the position to the upper computer, and according to the fed back position, the camera on the translation sliding table can return to the original position, so that the acquisition of M viewpoint images on the second row is completed.

Optionally, the lifting electric push rod of the camera array system can ascend for N positions, and the camera on the translation sliding table collects N viewpoints altogether at N positions in the column direction. The translation sliding table runs through a stroke, all base line distances of the array are recorded, and a multi-view image of the array base line distance-array is obtained.

Optionally, the camera on the translation sliding table of the camera array system may collect m×n viewpoint images, that is, the base line distances in all the row directions and the column directions are recorded, so as to obtain a row-column base line distance-two-dimensional plane array multi-viewpoint image.

Optionally, different camera array arrangements

Under this, the change of the base line distance B and depth D of each viewpoint causes parallax +.>

After the baseline distance B is introduced, the depth parallax relation model can be expanded to a higher dimension.

Alternatively, based on the equation

A camera array model may be proposed:

（12）

wherein the expression (12) comprises three model expressions,

，

Representing the depth D of the object from the camera plane and the parallax +.of the viewpoint from the central viewpoint, respectively>

。

，

，

，

，

Is the various parameters of the model.

Optionally, the camera array system may acquire multiple sets of camera array calibration plate images at different baseline distances (different camera array distributions) and different depths, and obtain parallax between each viewpoint and the central viewpoint in each set of camera array by using corner detection and 3 sigma rule. And (3) calibrating the depth of the camera plane of the camera array system, wherein the depth D and the baseline distance B of each viewpoint in the row direction can be respectively obtained in the baseline calibration of each viewpoint of the camera plane of the camera array system.

Alternatively, iterative optimization using nonlinear least squares and belief-domain algorithms can be fitted to a depth-disparity-baseline 3D model, i.e

The model returns corresponding three groups of model parameters at the same time;

The model provides a basis for parameter deduction and quantification for camera array imaging.

Alternatively, the camera array model may be used to characterize any of the following:

1. a relationship between depth and parallax;

2. relationship between depth and baseline;

3. a relationship between parallax and baseline;

4. depth, parallax, and baseline.

Alternatively, the camera array model may be used for unknown parameter budgets, or array distribution calculations.

Alternatively, get

After the model is formed, calibration personnel can verify the camera plane parallax, depth and baseline calibration result of the camera array system.

In one embodiment, the calibration personnel can verify the camera plane parallax, depth, baseline calibration results of the camera array system by:

extracting partial depth parallax data in a certain depth range, expanding the partial depth parallax data by using a small amount of camera array distribution to obtain more array distribution, and calculating horizontal parallaxes and vertical parallaxes corresponding to all views of the obtained arrays;

step two, utilizing a depth value of a depth-two-dimensional plane array multi-viewpoint image obtained by a camera plane depth calibration method of a camera array system;

step three, reading the camera array model parameters obtained by fitting, the parallax D in the step one and the depth D in the step two, and utilizing

The model formula (12) calculates the baseline distance, and selects a plurality of groups of values which do not participate in the iterative calculation of the model to participate in the calculation.

Reading row direction and column direction base line distance of row-column array multi-view image obtained by camera array system camera plane each view base line calibration method

、

Is marked as- >

。

Step five, respectively calculating the mean square error of the line direction viewpoint baseline distance and the column direction viewpoint baseline distance

、

。

Further, analyzing differences between the baseline calibration actual measurement values and the model calculation values of all the viewpoints of the camera plane of the camera array system to obtain

And (5) analyzing the error of the model. Meanwhile, the calibration results of parallax, depth and base line are also verified.

According to the camera array system calibration method provided by the invention, the line-row base line distance-two-dimensional plane array multi-viewpoint images can be obtained by executing the calibration of the base line of each viewpoint of the camera plane of the camera array system, and the parallax between each viewpoint and the central viewpoint in each group of camera arrays is based

The depth D and the base line distance B of each viewpoint in the row direction can be fitted to obtain a depth-parallax-base line 3D model, and the depth, the parallax and the base line can be automatically calibrated by combined motion with a camera array system and a mobile calibration device, so that the complexity of calibration is reduced, and the calibration efficiency and precision are improved.

Fig. 10 is a schematic view of each view point of a camera plane of the camera array system provided by the invention, and as shown in fig. 10, the array distribution includes an array row direction view point and an array column direction view point.

Optionally, the array system may set different viewpoints, such as different array arrangements of 5×5, 5×7, 5×9, etc., according to actual needs. Different arrangement modes lead the number of the visual points and the distance between the base lines to be different, and the parallaxes are also different, so that the three calibration modes obtain results, namely the parallaxes

Verification between baseline B, depth D is necessary.

Optionally, taking a nonlinear iteration fitted curve as an error reference standard, and executing the camera array system calibration method of the invention, wherein the error in the horizontal direction in the calculated imaging result is within 0.5 pixel, and the error is within 0.5 pixel when parallax is calculated, so that the accuracy of the camera array system calibration method of the invention is fully proved.

In one embodiment of the invention, a nonlinear iteration fitted curve is used as an error reference standard, the camera array system after the calibration method of the camera array system is executed calculates that the error in the horizontal direction in the imaging result is 0.05 pixel, and the error is 0.07 pixel when parallax is calculated.

In another embodiment of the present invention, a curve fitted by nonlinear iteration is used as an error reference standard, and the error in the horizontal direction in the imaging result is calculated to be 0.1 pixel, and when parallax is calculated, the error is 0.09 pixel in the camera array system after the calibration method of the camera array system is executed.

TABLE 1 depth-parallax relationship model calculation and measured MSE

Table 1 calculates and measures the mean square error (Mean Square Error, MSE) for the depth-disparity relationship model. As shown in table 1, the optimal parameters of three depth-parallax relation models (powers, ratios, exponentials) of each view are calculated by using a nonlinear least square method and an LM algorithm in an iterative manner, and the corresponding optimal expressions are obtained. And then, taking depth values which do not participate in iterative calculation of the model into three model expressions corresponding to the visual angles, namely a depth range [20.8, 23.6] with a step length of 0.4m, respectively calculating the parallax value of each visual angle, and obtaining the actually measured parallax value. Finally, the mean square error between the model calculation value and the actual measurement value is quantized, as shown in table 1, the MSE of the parallax in the horizontal and vertical directions calculated by the power function model is minimum, and the average value is only 0.03 and 0.49 pixels. Therefore, the power function module is closer to the true value and is the optimal model. The experimental result shows that the parallax calibration result of the camera array is accurate.

Fig. 11 is a graph of depth-parallax in the horizontal direction of all viewpoints in a depth range provided by the present invention, as shown in fig. 11, in one embodiment of the present invention, in a 5×17 array arrangement mode of a depth range, a parallax curve of all viewpoints and a central viewpoint in the horizontal direction is drawn, wherein the parallax curve is obtained by performing array sampling with a step length of 40mm under a depth of 6-24 m. The parallax of the array in the horizontal direction of the column view point of the array in the 5 x 17 array arrangement mode is coincident, and the total is 17 curves. Indicating a higher accuracy of the camera array system.

Fig. 12 is a graph of depth-parallax in the vertical direction of all viewpoints in a depth range provided by the present invention, as shown in fig. 12, in one embodiment of the present invention, in a 5×17 array arrangement mode of a depth range, a parallax curve of all viewpoints and a central viewpoint in the vertical direction, in which a 16mm lens is drawn at a depth of 6-24m, and the step size is 40mm, for array sampling. The parallax of the row view points of the array in the vertical direction is coincident in a 5 multiplied by 17 array arrangement mode, and the total is 5 curves. Indicating a higher accuracy of the camera array system.

Fig. 13 is one of the horizontal depth-parallax graphs of the viewpoint 1 in a certain depth range provided by the present invention, as shown in fig. 13, in one embodiment of the present invention, the designed camera array system has a row direction length of 1800mm, an array viewpoint arrangement manner of 5×11, and a row direction baseline distance calculated according to the feedback value of the absolute position encoder a of 112.8484mm.

And drawing a parallax change trend line graph of parallax scattering point distribution of the viewpoint 1 and the central viewpoint of the 35mm lens, which are subjected to array sampling at intervals of 40mm in depth of 22-23m, in the horizontal direction and extending to 24m, wherein the parallax relation between the fitted viewing angle 1 and the central viewing angle under 5×13 array distribution is shown in fig. 13. In the array arrangement mode, parallax scattering points of the view point 1 and the central view angle in the horizontal direction are near trend lines.

Fig. 14 is a second graph of depth-parallax in the horizontal direction of the viewpoint 1 in a certain depth range provided by the present invention, as shown in fig. 14, in one embodiment of the present invention, the designed camera array system has a row direction length of 1800mm, the array viewpoint arrangement mode is 5×13, and the row direction baseline distance calculated according to the feedback value of the absolute position encoder a is 150.4645mm.

And drawing a parallax change trend line graph of parallax scattering point distribution of the viewpoint 1 and the central viewpoint of the 35mm lens, which are subjected to array sampling at intervals of 40mm in depth of 22-23m, in the horizontal direction and extending to 24m, wherein the parallax relation between the fitted viewing angle 1 and the central viewing angle under 5×13 array distribution is shown in fig. 14. In the array arrangement mode, parallax scattering points of the view point 1 and the central view angle in the horizontal direction are near trend lines.

Fig. 15 is a third view of a depth-parallax curve diagram in the horizontal direction of the viewpoint 1 in a certain depth range, as shown in fig. 15, in one embodiment of the present invention, the designed camera array system has a row direction length of 1800mm, an array viewpoint arrangement manner of 5×17, and a row direction baseline distance calculated according to the feedback value of the absolute position encoder a of 180.5574mm.

And drawing a parallax change trend line graph of parallax scattering point distribution of the viewpoint 1 and the central viewpoint of the 35mm lens, which are subjected to array sampling at intervals of 40mm in depth of 22-23m, in the horizontal direction and extending to 24m, wherein the parallax relation between the fitted viewing angle 1 and the central viewing angle under 5×17 array distribution is shown in fig. 15. In the array arrangement mode, parallax scattering points of the view point 1 and the central view angle in the horizontal direction are near trend lines.

Fig. 16 is one of the vertical depth-parallax graphs of the view point 1 in a certain depth range provided by the present invention, as shown in fig. 16, in one embodiment of the present invention, the designed camera array system has a row direction length of 1800mm, an array view point arrangement manner of 5×11, and a row direction baseline distance calculated according to the feedback value of the absolute position encoder a of 112.8484mm.

And drawing a parallax change trend line graph of parallax scattering point distribution of the viewpoint 1 and the central viewpoint of the 35mm lens, which are subjected to array sampling at intervals of 40mm in depth of 22-23m, in the vertical direction and extending to 24m, wherein the parallax relation between the fitted viewing angle 1 and the central viewing angle in 5×11 array distribution is shown in fig. 16. In the array arrangement mode, parallax scattering points of the view point 1 and the central view angle in the vertical direction are near trend lines.

Fig. 17 is one of the vertical depth-parallax graphs of the view point 1 in a certain depth range provided by the present invention, as shown in fig. 17, in one embodiment of the present invention, the designed camera array system has a row direction length of 1800mm, the array view point arrangement mode is 5×13, and the row direction baseline distance calculated according to the feedback value of the absolute position encoder a is 150.4645mm.

And drawing a parallax change trend line graph of parallax scattering point distribution of the viewpoint 1 and the central viewpoint of the 35mm lens, which are subjected to array sampling at intervals of 40mm in depth of 22-23m, in the vertical direction and extending to 24m, wherein the parallax relation between the fitted viewing angle 1 and the central viewing angle in 5 multiplied by 13 array distribution is shown in figure 17. In the array arrangement mode, parallax scattering points of the view point 1 and the central view angle in the vertical direction are near trend lines.

Fig. 18 is one of the vertical depth-parallax graphs of the view point 1 in a certain depth range provided by the present invention, as shown in fig. 18, in one embodiment of the present invention, the designed camera array system has a row direction length of 1800mm, the array view point arrangement mode is 5×17, and the row direction baseline distance calculated according to the feedback value of the absolute position encoder a is 180.5574mm.

And drawing a parallax change trend line graph of parallax scattering point distribution of the viewpoint 1 and the central viewpoint of the 35mm lens, which are subjected to array sampling at intervals of 40mm in depth of 22-23m, in the vertical direction and extending to 24m, wherein the parallax relation between the fitted viewing angle 1 and the central viewing angle in 5×17 array distribution is shown in fig. 18. In the array arrangement mode, parallax scattering points of the view point 1 and the central view angle in the vertical direction are near trend lines.

According to the fitting result, parallax of the viewpoint 1 in the horizontal direction and the vertical direction is respectively extended, parallax under any depth can be obtained, the control precision of the camera array system is higher, and the parallax errors in the horizontal direction and the vertical direction are lower than or equal to 0.3 pixel, so that the precision can be ensured and automatic calibration of the camera array system can be realized by the mobile calibration device, and parallax data augmentation of any depth can be performed.

Fig. 19 is one of the horizontal depth-parallax graphs of the viewpoint 2 in a certain depth range provided by the present invention, as shown in fig. 19, in one embodiment of the present invention, the parallax relationship between the viewpoint 2 and the central viewing angle in the 5×11,5×13, and 5×17 array distribution is drawn, where the parallax is equal to the horizontal parallax of the central viewpoint in the viewpoint 2 in the array sampling at a depth of 40mm between 22 and 23 m. As can be seen from the contour diagram, in three different array arrangements, the parallax in the horizontal direction of the viewpoint 2 and the central viewing angle is different from the parallax in the horizontal direction of the viewpoint 1 and the central viewing angle in fig. 13, the parallax of the viewpoint 2 and the central viewing angle is not coincident, and the differences between them are 10.5 and 12.8 pixels, which is also because the base line distance varies, resulting in equidistant differences from the central viewing angle at different depths.

Fig. 20 is one of the vertical depth-parallax graphs of the viewpoint 2 in a certain depth range provided by the present invention, as shown in fig. 20, in one embodiment of the present invention, the parallax relationship between the viewpoint 2 and the central viewing angle in 5×11,5×13, and 5×17 array distribution is drawn, where the parallax is equal to the vertical parallax of the central viewpoint in the viewpoint 2 in which the array sampling is performed by using a 35mm lens at a depth of 40mm under a depth of 22-23 m. FIG. 21 is a second view of a horizontal depth-disparity plot for view 2 in a depth range provided by the present invention; as shown in fig. 21, horizontal parallax is fitted according to the parallax contour, and these points in the horizontal contour map are fitted into a curve, and as can be seen from the fitted map, the horizontal parallax values of the viewpoint 2 and the central viewpoint in three array arrangement modes are very stable and tend to be a straight line, which proves that the parallax result calibrated by the mobile calibration device has higher stability based on the camera array system.

FIG. 22 is a second view of a depth-disparity plot for the vertical direction of viewpoint 2 in a range of depths provided by the present invention; as shown in fig. 22, the parallax in the vertical direction is fitted according to the parallax contour, the points in the horizontal direction contour are fitted into a curve, and as can be seen from the fitted curve, the vertical parallax values of the viewpoint 2 and the central viewpoint in three array arrangement modes are very stable and tend to be a straight line, so that the stability of the parallax result calibrated by the mobile calibration device is higher based on the camera array system.

FIG. 23 is a schematic view of a camera array model according to the present invention, as shown in FIG. 23, in one embodiment of the present invention {5×9, 5×7, 5×6, 5×5, 5×3} distribution is obtained by using {5×17, 5×13, 5×11} camera array distribution expansion. The corresponding base line distances for these arrays were calculated to be {112.85, 150.46, 180.56, 225.70, 300.93, 361.11, 451.39, 902.79 } mm, respectively.

Fig. 24 is a schematic diagram of a depth-parallax-baseline curve provided by the present invention, as shown in fig. 24, according to the scattered points of the camera array parameters and the depth-parallax-baseline curve in the 3D space, combining the camera array model equation (12), adopting a nonlinear least square method, selecting a trust domain algorithm for iterative computation, fitting the model (1) in the equation (12) to obtain a curved surface corresponding to the model, and returning to the parameters corresponding to the model.

Fig. 25 is one of the schematic diagrams of the scatter points of the camera array parameters in the 3D space provided by the present invention, as shown in fig. 25, according to the scatter points of the camera array parameters and the depth-parallax-baseline curve in the 3D space, combining the camera array model equation (12), adopting a nonlinear least square method, selecting the trust domain algorithm for iterative computation, fitting the model (1) in the equation (12) to obtain a curved surface corresponding to the model, and returning the parameters corresponding to the model.

FIG. 26 is a schematic diagram of a camera array model according to the second embodiment of the present invention, as shown in FIG. 26, in one embodiment of the present invention {5×9, 5×7, 5×6, 5×5, 5×3} distribution is obtained by using a 5×17 camera array distribution extension. The corresponding base line distances for these arrays were calculated to be {112.85, 150.46, 180.56, 225.70, 300.93, 361.11, 451.39, 902.79 } mm, respectively.

Fig. 27 is a schematic diagram of a depth-parallax-baseline curve provided by the present invention, as shown in fig. 27, according to the scattered points of the camera array parameters and the depth-parallax-baseline curve in the 3D space, combining the camera array model equation (12), adopting a nonlinear least square method, selecting a trust domain algorithm for iterative computation, fitting the model (2) in the equation (12) to obtain a curved surface corresponding to the model, and returning the parameters corresponding to the model.

Fig. 28 is one of the schematic diagrams of the scattered points of the camera array parameters in the 3D space provided by the invention, as shown in fig. 28, according to the scattered points of the camera array parameters and the depth-parallax-baseline curve in the 3D space, combining the camera array model equation (12), adopting a nonlinear least square method, selecting the trust domain algorithm for iterative computation, fitting the model (2) in the equation (12) to obtain a curved surface corresponding to the model, and returning the parameters corresponding to the model.

Fig. 29 is a schematic view of a camera array model according to the present invention, as shown in fig. 29, in one embodiment of the present invention, {5×9, 5×7, 5×6, 5×5, 5×3} distribution is obtained by using a 5×17 camera array distribution extension. The corresponding base line distances for these arrays were calculated to be {112.85, 150.46, 180.56, 225.70, 300.93, 361.11, 451.39, 902.79 } mm, respectively.

Fig. 30 is a schematic diagram of a depth-parallax-baseline curve provided by the present invention, as shown in fig. 30, according to the scattered points of the camera array parameters and the depth-parallax-baseline curve in the 3D space, combining the camera array model equation (12), adopting a nonlinear least square method, selecting a trust domain algorithm for iterative computation, fitting the model (3) in the equation (12) to obtain a curved surface corresponding to the model, and returning the parameters corresponding to the model.

Fig. 31 is one of the schematic diagrams of the scatter points of the camera array parameters in the 3D space provided by the present invention, as shown in fig. 31, according to the scatter points of the camera array parameters in the 3D space and the depth-parallax-baseline curve, combining the camera array model equation (12), adopting a nonlinear least square method, selecting the trust domain algorithm for iterative computation, fitting the model (3) in the equation (12) to obtain a curved surface corresponding to the model, and returning the parameters corresponding to the model.

TABLE 2 RMSE for theoretical and actual measured values for camera array model

Table 2 is the root mean square error (Root Mean Squared Error, RMSE) of the camera array model baseline theoretical and actual measured values. As shown in table 2, three sets of depth data { [7.52, 14.52], step,1m }, { [15.52, 22.52], step,1m }, { [16, 23], step,1m }, which did not participate in the iterative computation of the model, were selected. Taking the calculation of baseline values as an example: the depth-disparity is taken into the camera array model to calculate the baseline value, which is then calculated as RMSE with the true measurement. { V5, V6, …, V17} represents the viewpoint distribution, { Model1, model2, model3} represents the three models in equation (12), and the last column of the table is the average RSME value.

Fig. 32 illustrates a physical structure diagram of an electronic device, which may include: a processor 3210, a communication interface (Communications Interface) 3220, a memory 3230 and a communication bus 3240, wherein the processor 3210, the communication interface 3220 and the memory 3230 perform communication with each other through the communication bus 3240. The processor 3210 may invoke logic instructions in the memory 3230 to perform a tire-way contact three-way force sensor decoupling model determination method comprising: adopting a camera of a camera array system to acquire images of a calibration plate of the mobile calibration device to obtain X multiplied by Y images in a plurality of two-dimensional arrays with depths corresponding to the respective depths, wherein the two-dimensional arrays are X rows and Y columns, and X and Y are positive integers; and performing parallax calibration of a camera plane of the camera array system, depth calibration of the camera plane of the camera array system and base line calibration of each view point of the camera plane of the camera array system based on the X multiplied by Y images in the two-dimensional array corresponding to the plurality of depths respectively.

Further, the logic instructions in memory 3230 described above can be implemented in the form of software functional units and stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

In another aspect, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of executing the tire-road contact three-way force sensor decoupling model determining method provided by the above methods, the method comprising: adopting a camera of a camera array system to acquire images of a calibration plate of the mobile calibration device to obtain X multiplied by Y images in a plurality of two-dimensional arrays with depths corresponding to the respective depths, wherein the two-dimensional arrays are X rows and Y columns, and X and Y are positive integers; and performing parallax calibration of a camera plane of the camera array system, depth calibration of the camera plane of the camera array system and base line calibration of each view point of the camera plane of the camera array system based on the X multiplied by Y images in the two-dimensional array corresponding to the plurality of depths respectively.

In yet another aspect, the present invention further provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the method for determining a decoupling model of a tire-road contact three-way force sensor provided by the above methods, the method comprising: adopting a camera of a camera array system to acquire images of a calibration plate of the mobile calibration device to obtain X multiplied by Y images in a plurality of two-dimensional arrays with depths corresponding to the respective depths, wherein the two-dimensional arrays are X rows and Y columns, and X and Y are positive integers; and performing parallax calibration of a camera plane of the camera array system, depth calibration of the camera plane of the camera array system and base line calibration of each view point of the camera plane of the camera array system based on the X multiplied by Y images in the two-dimensional array corresponding to the plurality of depths respectively.

The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A mobile calibration device for calibrating a camera array system, the mobile calibration device comprising: the device comprises a calibration plate (17), a liftable upright post (20) of the calibration plate, a slidable platform (21), a guide rail sliding block (22), a scale (24), a guide rail sliding block base (23), a translation sliding table (27), a positioning pointer (25), an angle adjusting optical axis (19), a second servo motor (28), a second servo driver (29) and a position depth calibration device (30);

wherein, the liftable stand column (20) of the calibration plate is arranged on the slidable platform (21);

the guide rail sliding block (22) and the scale (24) are arranged at preset positions on the guide rail sliding block base (23), and the translation sliding table (27) is arranged on the guide rail sliding block base (23);

the slidable platform (21) is connected with the translation sliding table (27) so that the slidable platform (21) can slide on the guide rail sliding block (22);

the positioning pointer (25) is arranged on the sliding platform (21), and the positioning pointer (25) is parallel to the scale (24);

The angle adjusting optical axis (19) is vertically arranged on a lifting upright post (20) of the calibration plate, and the calibration plate is horizontally arranged on the angle adjusting optical axis (19);

the second servo motor (28) is mounted on the translation sliding table (27), the second servo motor (28) is connected with a first servo driver in the camera array system, and the second servo motor (28) is connected with a second servo driver (29).

2. A camera array system calibration apparatus, comprising: a camera array system, a mobile calibration device as claimed in claim 1;

3. A camera array system calibration method, applied to the camera array system calibration apparatus of claim 2, the method comprising:

4. A camera array system calibration method according to claim 3, wherein the image capturing of the calibration plate of the mobile calibration device comprises:

5. The method for calibrating a camera array system according to claim 4, wherein before the camera of the camera array system is used to acquire images of the calibration plate of the mobile calibration device by a two-dimensional planar array motion image acquisition mode to obtain X Y images in a plurality of two-dimensional arrays with respective depths, the method further comprises:

6. The camera array system calibration method of claim 5, wherein performing camera plane parallax calibration of the camera array system comprises:

7. The camera array system calibration method of claim 6, further comprising:

8. The camera array system calibration method of claim 5, wherein performing camera plane depth calibration of the camera array system comprises:

9. The camera array system calibration method according to claim 5, wherein performing the camera plane each viewpoint base line calibration of the camera array system includes:

10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the camera array system calibration method of any one of claims 3 to 9 when the program is executed by the processor.