CN114157852A - Virtual camera array three-dimensional imaging method and system based on rotating double prisms - Google Patents
Virtual camera array three-dimensional imaging method and system based on rotating double prisms Download PDFInfo
- Publication number
- CN114157852A CN114157852A CN202111437830.7A CN202111437830A CN114157852A CN 114157852 A CN114157852 A CN 114157852A CN 202111437830 A CN202111437830 A CN 202111437830A CN 114157852 A CN114157852 A CN 114157852A
- Authority
- CN
- China
- Prior art keywords
- virtual camera
- camera
- camera array
- dimensional
- visual axis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/346—Image reproducers using prisms or semi-transparent mirrors
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B35/00—Stereoscopic photography
- G03B35/02—Stereoscopic photography by sequential recording
Abstract
The invention discloses a virtual camera array three-dimensional imaging method and system based on a rotating biprism, and belongs to the technical field of three-dimensional computational imaging. Based on the prism refraction effect, the visual axis of the fixed single camera deflects under the action of the double prisms at a certain rotation angle; selecting an intersection point of a visual axis of the initial virtual camera and a preset virtual plane as the position of a sub-camera in the virtual camera array, and determining the rotation angle of the double prisms according to the relation between the position of the sub-camera in the virtual camera array and the azimuth angle and the pitch angle of the visual axis deflection; the method comprises the steps that the visual axis of a fixed single camera is changed by rotating a double prism for multiple times, so that dynamic scanning of a target scene is achieved, and an uncorrected multi-image sequence with sufficient angle and spatial resolution is obtained; the uncorrected multi-image sequence is converted into an image sequence acquired by a corresponding plane virtual camera array, and the three-dimensional information reconstruction of a target scene is realized by combining a light field imaging theory, so that the compactness and the flexibility of three-dimensional imaging are improved.
Description
Technical Field
The invention relates to a virtual camera array three-dimensional imaging method and system based on a rotating biprism, and belongs to the technical field of three-dimensional computational imaging.
Background
Three-dimensional imaging technology is one of the most important technologies in applications such as robotics, unmanned driving, and smart sensing. Passive three-dimensional measurement schemes have been widely used in recent years in a number of target scenarios, since they can be laser independent. The passive three-dimensional measurement system can be divided into a monocular camera, a binocular camera and a multi-view camera three-dimensional reconstruction method. However, the multi-view camera is one of the popular research directions in the three-dimensional imaging technology because it can acquire image information of a target scene from multiple angles.
The traditional multi-camera three-dimensional reconstruction mainly comprises the following steps: the system has the advantages that a real camera array is used, dozens of real cameras are generally required to be arranged at fixed positions in a scene, and meanwhile, light information is acquired for a target scene, and the system has the defects that the size is large, and the distance between the cameras is limited by the size of a single camera; the micro-lens array is used for carrying out secondary imaging on the image of the main lens, and the system uses the same sensor as a receiving surface of the secondary imaging, so that the resolution of the acquired image is seriously limited by the size of the angular resolution; the system has the advantages that the single camera is used for shooting the same target scene in multiple angles by using the moving single camera matched with the guide rail and through displacement on the two-dimensional guide rail, the actual size is relatively large due to the guide rail moving factor, the moving range is limited by the length of the guide rail, and the camera is easily influenced by the precision and the straightness of the guide rail.
Disclosure of Invention
The invention mainly aims to provide a virtual camera array three-dimensional imaging method and a system based on a rotating double prism, based on the refraction effect of the prism, a visual axis of a fixed single camera deflects under the action of the double prism at a certain rotating angle, at the moment, an image acquired by the fixed single camera can be regarded as a non-corrected image acquired by a required initial virtual camera with different pose parameters, the intersection point of the visual axis of the initial virtual camera and a preset virtual plane is selected as the position of a sub-camera in a virtual camera array, and the rotating angle of the double prism is determined according to the relation between the position of the sub-camera in the required virtual camera array and the azimuth angle and the pitch angle of the deflection of the visual axis; the method comprises the steps that the visual axis of a fixed single camera is changed by rotating a double prism for multiple times, so that dynamic scanning of a target scene is achieved, and an uncorrected multi-image sequence with sufficient angle and spatial resolution is obtained; the obtained multi-angle uncorrected image sequence is converted, the uncorrected multi-image sequence is converted into an image sequence acquired by a corresponding plane virtual camera array, and three-dimensional information reconstruction of a target scene is realized by combining a light field imaging theory according to a two-dimensional image acquired by the plane virtual camera array, namely the three-dimensional imaging compactness and flexibility are improved by the virtual camera array based on the rotating biprism. The compactness refers to that multi-view images collected by a multi-camera array can be acquired by deflecting and fixing the visual axis of a single camera and changing the pose of a virtual camera; the flexibility means that the number and the positions of the cameras in the virtual camera array can be changed by changing the combination of the rotation angle values of the double prisms, and the flexibility can adapt to different three-dimensional imaging scenes without changing the position of each real single camera relative to a real camera array.
The invention discloses a virtual camera array three-dimensional imaging method based on a rotating biprism, which comprises the following steps:
the method comprises the following steps: based on the refraction effect of the prism, the visual axis of the fixed single camera will be deflected under the action of the biprism at a certain rotation angle, and the visual axis will be deflected to different degrees when passing through each surface of the biprism in the system. At the moment, an image acquired by the fixed single camera can be regarded as a non-corrected image acquired by a required initial virtual camera with different pose parameters, an intersection point of a visual axis of the initial virtual camera and a preset virtual plane is selected as the position of a sub-camera in the virtual camera array, and the rotation angle of the double prisms is determined according to the relationship between the position of the sub-camera in the virtual camera array and the azimuth angle and the pitch angle of the visual axis deflection.
Based on the refraction effect of the prisms, the visual axis of the fixed single camera will be deflected by the biprisms at a certain rotation angle, the visual axis of which passes through each face of the biprisms in the systemAt all times, different degrees of deflection will occur, the deflected light vector Ri(i ═ 1,2,3,4) is represented by:
wherein n isiAnd ni-1Corresponding to the refractive indices of the front and rear incident refraction surfaces, NiCorresponding to the normal vector of each facet.
According to the analysis of the refraction direction of the light after passing through each prism surface and the combination of the parameters and the spacing of the prisms, the intersection point S of the visual axis and each surface can be obtainedi(i ═ 1,2,3,4,5), expressed as:
wherein, tiRepresents the corresponding deflection parameter, which is determined by the direction of the deflected light and the pitch of each facet.
At the moment, an image acquired by the fixed single camera can be regarded as a non-corrected image acquired by a required initial virtual camera with different pose parameters, an intersection point of a visual axis of the initial virtual camera and a preset virtual plane is selected as the position of a sub-camera in the virtual camera array, and the rotation angle of the double prisms is determined according to the relationship between the position of the sub-camera in the required virtual camera array and the azimuth angle and the pitch angle of the visual axis deflection.
Initial virtual camera C corresponding to the uncorrected imagevIs represented as:
wherein m isvRepresented as a two-dimensional projection of a three-dimensional point M, Kv、RvAnd TvRespectively, an initial virtual camera CvInternal parameters of (a), a rotation matrix, and an estimated translation vector.
Step two: and according to the relation between the position of the sub-camera in the required virtual camera array and the rotation angle of the double prisms, changing the visual axis of the fixed single camera by rotating the double prisms for multiple times, and realizing dynamic scanning of the target scene, so that an uncorrected multi-image sequence with sufficient angle and spatial resolution is obtained.
Step three: in order to enable the obtained Epipolar Plane Image (EPI) information to be detected more easily, the pose of the Image obtained by the virtual camera is transformed, equivalent virtual cameras with different internal references are integrated and transformed into the same virtual Plane in the same direction, an equivalent virtual camera array composed of a plurality of parallel, equidistant and coplanar multi-camera structures is formed, and an uncorrected multi-Image sequence is transformed into a two-dimensional Image collected by the corresponding planar virtual camera array.
Sub-camera C in virtual camera arrayvrIs represented as:
wherein m isvrRepresented as a two-dimensional projection of a three-dimensional point M, Kvr、RvrAnd TvrRespectively, sub-camera C in the corrected virtual camera arrayvrInternal parameters, rotation matrix, and translation vector.
Transformation matrix H between initial virtual camera and corrected virtual camerakExpressed as:
wherein the content of the first and second substances,
representing a pseudo-inverse matrix calculation. And obtaining images acquired by the sub-cameras in the virtual camera array through conversion between the virtual cameras and the corrected virtual cameras.
Step four: according to the two-dimensional image acquired by the planar virtual camera array, the image set after the transformation correction is used for realizing the three-dimensional information reconstruction of the target scene by combining the light field theory, namely the three-dimensional imaging compactness and flexibility are improved by the virtual camera array based on the rotating double prisms.
The invention also discloses a virtual camera array three-dimensional imaging system based on the rotating double prisms, which is used for realizing the virtual camera array three-dimensional imaging method based on the rotating double prisms.
Under the action of the biprism at a certain rotation angle, the real fixed single camera is used for acquiring images which can be regarded as being acquired by an initial virtual camera with different pose parameters from the real camera.
The miniature stepping motor is used for driving the bevel gear embedded with the prism to rotate for multiple times, so that the visual axis of the fixed single camera is changed for multiple times, and the deflection of the system visual axis and the scanning of a target scene are realized.
Based on the refraction effect of the prism, the double prisms are used for deflecting the visual axis of the real fixed single camera, and the rotation angle of the double prisms is determined by the relation between the position of the sub camera in the required virtual camera array and the azimuth angle and the pitch angle of the visual axis deflection. The independent prism combination is used for constructing the relation between the fixed single camera and the virtual camera array. The prism combination is coaxially arranged right in front of the fixed single camera and is separated by a fixed distance value.
The magnetic encoder is annular, is coaxial with the prism and the real fixed single camera, and is used for detecting the rotation position of the prism in real time, so that the virtual camera array three-dimensional imaging system has a closed-loop feedback control function.
The sub-cameras in the virtual camera array are used for uniformly acquiring information of light propagation in a target scene, so that three-dimensional calculation imaging is realized.
The invention also discloses a working method of the virtual camera array three-dimensional imaging system based on the rotating biprism, which comprises the following steps:
and signals for controlling the double prisms and outputting camera images are transmitted and received by opening corresponding system serial ports. The prism can be rotated to a preset angle in a high-precision manner by driving the gear to rotate by using the pulse signal driving stepping motor, and angle adjustment control is performed on the rotation angle of the gear by receiving angle value data as feedback information in real time. And when the preset angle value is reached, acquiring an uncorrected image sequence acquired by the initial virtual camera. The method comprises the steps of obtaining uncorrected images of multiple angles under the rotation angles of multiple groups of rotating double prisms, and obtaining a two-dimensional image sequence collected by a virtual camera array by transforming the uncorrected image sequence. By acquiring the acquired two-dimensional image sequence, the analysis of the light propagation information can be realized, namely, the three-dimensional information reconstruction of a target scene is realized by combining the image set after the transformation correction with the light field theory according to the two-dimensional image acquired by the planar virtual camera array, so that the compactness and the flexibility of the three-dimensional imaging are improved.
Has the advantages that:
1. the invention discloses a virtual camera array three-dimensional imaging method and a system based on a rotating double prism, wherein based on the refraction effect of the prism, the visual axis of a fixed single camera deflects under the action of the double prism at a certain rotating angle, at the moment, an image acquired by the fixed single camera can be regarded as a non-corrected image acquired by a required initial virtual camera with different pose parameters, the intersection point of the visual axis of the initial virtual camera and a preset virtual plane is selected as the position of a sub-camera in a virtual camera array, and the rotating angle of the double prism is determined according to the relation between the position of the sub-camera in the required virtual camera array and the azimuth angle and the pitch angle of the visual axis deflection; the method comprises the steps that the visual axis of a fixed single camera is changed by rotating a double prism for multiple times, so that dynamic scanning of a target scene is achieved, and an uncorrected multi-image sequence with sufficient angle and spatial resolution is obtained; the obtained multi-angle uncorrected image sequence is converted, the uncorrected multi-image sequence is converted into an image sequence acquired by a corresponding plane virtual camera array, and three-dimensional information reconstruction of a target scene is realized by combining a light field imaging theory according to a two-dimensional image acquired by the plane virtual camera array, namely the three-dimensional imaging compactness and flexibility are improved by the virtual camera array based on the rotating biprism.
2. The invention discloses a virtual camera array three-dimensional imaging method and system based on a rotating double prism, which can obtain images acquired by a multi-camera array by deflecting the visual axis of a real camera and changing the pose of an original virtual camera, thereby improving the compactness of a three-dimensional imaging system.
3. The invention discloses a virtual camera array three-dimensional imaging method and system based on a rotating biprism, which can change the number and the position of cameras in a virtual camera array by changing the combination of the scanning angle values of the biprism, do not need to change the position of each real camera relative to a real camera array, can adapt to different three-dimensional imaging scenes, and further improve the flexibility of a three-dimensional imaging system.
4. Compared with the existing three-dimensional information acquisition system, the invention discloses a virtual camera array three-dimensional imaging method and system based on a rotating double prism, which realize dynamic scanning of a target scene by changing the visual axis of a fixed single camera through rotating the double prism for multiple times, thereby acquiring an uncorrected multi-image sequence with sufficient angles and spatial resolution and acquiring multi-angle information of the scene without sacrificing the spatial resolution.
5. The invention discloses a virtual camera array three-dimensional imaging method and a virtual camera array three-dimensional imaging system based on a rotating double prism.
Drawings
FIG. 1 is a three-dimensional schematic diagram of a rotating biprism based virtual camera array system according to the present invention.
FIG. 2 is a flow chart of a method for three-dimensional imaging of a virtual camera array based on a rotating biprism according to the present invention.
FIG. 3 is a schematic diagram of the present invention for acquiring images captured by a virtual camera array.
Fig. 4 is a diagram of an example of the virtual camera array based on the rotating biprism according to the present invention.
Fig. 5 is a flow chart of the detailed operation of the system of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
As shown in fig. 1, the virtual camera array three-dimensional imaging system based on the rotating double prisms disclosed in this example includes a prism 110, a prism 111, a real fixed single camera 120, a sub-camera 130 in the virtual camera array, a magnetic encoder (not shown), a micro stepper motor (not shown), and a gear set and a supporting structure (not shown).
Under the action of the prism 110 and the prism 111 at a certain rotation angle, the real fixed single camera 120 is used to acquire an image that can be regarded as being captured by an initial virtual camera having different pose parameters from those of the real camera.
The miniature stepping motor is used for driving the corresponding 45-degree helical gear embedded with the prism to rotate for multiple times, so that the visual axis of the fixed single camera 120 is changed for multiple times, and the deflection of the system visual axis and the scanning of a target scene are realized.
Each micro-stepper motor is equipped with a motor driver (not shown) connected thereto as a control signal input for driving the micro-stepper motor.
Based on the refraction effect of the prism, the double prism is used to deflect the visual axis of the real fixed single camera 120, and the rotation angle is determined by the relationship between the position of the sub-camera 130 in the desired virtual camera array and the azimuth angle and the pitch angle of the visual axis deflection. The prism 110 and the prism 111 are used for constructing the relationship between the fixed single camera 120 and the sub-camera 130 in the virtual camera array. The prism combination is coaxially placed right in front of the fixed single camera and separated by a distance value of 33 mm.
The magnetic encoder is annular, is coaxial with the prism 110, the prism 111 and the real fixed single camera 120, and is used for detecting the rotation position of the prism in real time, so that the virtual camera array three-dimensional imaging system has a closed-loop feedback control function.
The sub-cameras 130 in the virtual camera array are used for uniformly acquiring information of light propagation in a target scene, so that three-dimensional computational imaging is realized.
In the embodiment, a 5 × 5 virtual camera array is formed by rotating a biprism and the spacing between the sub-cameras making up the virtual camera array is 0.6 mm.
As shown in fig. 2, the method for three-dimensional imaging of a virtual camera array based on a rotating biprism disclosed in this embodiment includes the following steps:
the method comprises the following steps: based on the refraction effect of the prism, the visual axis of the fixed single camera 120 will be deflected by the double prism at a certain rotation angle, as shown in fig. 3, when the visual axis passes through each surface of the double prism in the system, the visual axis will be deflected to different degrees, and the deflected light ray vector Ri(i ═ 1,2,3,4) is represented by:
wherein n isiAnd ni-1Corresponding to the refractive indices of the front and rear incident refraction surfaces, NiCorresponding to the normal vector of each facet.
According to the analysis of the refraction direction of the light after passing through each prism surface and the combination of the parameters and the spacing of the prisms, the intersection point S of the visual axis and each surface can be obtainedi(i ═ 1,2,3,4,5), which can be expressed as:
wherein, tiRepresents the corresponding deflection parameter, which is determined by the direction of the deflected light and the pitch of each facet.
At this time, the image acquired by the fixed single camera 120 can be regarded as a non-corrected image acquired by a required initial virtual camera with different pose parameters, an intersection point of a visual axis of the initial virtual camera and a preset virtual plane is selected as a position of a sub-camera in the virtual camera array, and the rotation angle of the double prisms is determined according to the relationship between the position of the sub-camera 130 in the required virtual camera array and an azimuth angle and a pitch angle of the visual axis deflection.
Initial virtual camera C corresponding to the uncorrected imagevIs represented as:
wherein m isvRepresented as a two-dimensional projection of a three-dimensional point M, Kv、RvAnd TvRespectively, an initial virtual camera CvInternal parameters of (a), a rotation matrix, and an estimated translation vector.
Step two: according to the relationship between the position of the sub-camera 130 in the required virtual camera array and the rotation angle of the biprism, the dynamic scanning of the target scene is realized by rotating the biprism multiple times to change the visual axis of the fixed single camera 120, thereby obtaining an uncorrected multi-image sequence with sufficient angle and spatial resolution.
Step three: in order to enable the obtained Epipolar Plane Image (EPI) information to be detected more easily, the position and posture of the Image obtained by the virtual camera are transformed, equivalent virtual cameras with different internal references are integrated and transformed into the same direction and the same virtual Plane, an equivalent virtual camera array consisting of a plurality of parallel, equidistant and same-Plane multi-camera structures is formed, namely an uncorrected multi-Image sequence is transformed into an Image sequence collected by a corresponding Plane virtual camera array, and a sub-camera C in the virtual camera array is usedvrThe projection matrix of (a) may be expressed as:
wherein m isvrRepresented as a two-dimensional projection of a three-dimensional point M, Kvr、RvrAnd TvrRespectively, sub-camera C in the corrected virtual camera arrayvrInternal parameters, rotation matrix, and translation vector.
Transformation matrix H between initial virtual camera and corrected virtual camerakCan be expressed as:
wherein the content of the first and second substances,
representing a pseudo-inverse matrix calculation. Through the above process, the image acquired by the sub-camera 130 in the virtual camera array can be obtained.
Step four: according to the two-dimensional image acquired by the planar virtual camera array, the image set after the transformation correction is used for realizing the three-dimensional information reconstruction of the target scene by combining the light field theory, namely the three-dimensional imaging compactness and flexibility are improved by the virtual camera array based on the rotating double prisms.
As shown in fig. 4, according to the example image collected by the virtual camera array based on the rotating double prisms disclosed in this embodiment, the images are arranged in an array and numbered in sequence, wherein each image is obtained at a rotation angle corresponding to a prism, then the collected images are converted into an EPI format, a depth value at a corresponding point is calculated according to different linear slopes in the EPI, a disparity map is generated, and a depth map of a target scene is obtained according to the disparity map and by combining the sub-camera distance and the focal distance.
As shown in fig. 5, the embodiment further discloses a working method of the virtual camera array three-dimensional imaging system based on the rotating biprism, which includes:
and signals for controlling the double prisms and outputting camera images are transmitted and received by opening corresponding system serial ports. By driving the stepping motor with a pulse signal to rotate the gear, the prism can be rotated to a high precisionThe angle is preset, and angle adjustment control is performed on the rotation angle of the gear by receiving angle value data serving as feedback information in real time. And when the preset angle value is reached, acquiring an uncorrected image acquired by the initial virtual camera. By obtaining a multi-angle uncorrected image sequence under the rotation angles of a plurality of groups of rotating double prisms and combining a matrix HkAnd realizing the transformation of the uncorrected image sequence so as to acquire the two-dimensional image sequence acquired by the virtual camera array. By acquiring the acquired two-dimensional image sequence, the light propagation information can be analyzed, so that three-dimensional imaging is realized.
The invention provides a virtual camera array three-dimensional imaging system and method based on a rotating biprism, which aim to realize array three-dimensional imaging more flexibly and have the characteristics of compact structure and stable control.
The above detailed description is intended to illustrate the objects, aspects and advantages of the present invention, and it should be understood that the above detailed description is only exemplary of the present invention and is not intended to limit the scope of the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (6)
1. A virtual camera array three-dimensional imaging method based on a rotating biprism is characterized in that: comprises the following steps of (a) carrying out,
the method comprises the following steps: based on the refraction effect of the prism, the visual axis of the fixed single camera deflects under the action of the double prisms at a certain rotation angle, and the visual axis of the fixed single camera deflects to different degrees when passing through each surface of the double prisms in the system; at the moment, an image acquired by the fixed single camera can be regarded as a non-corrected image acquired by a required initial virtual camera with different pose parameters, an intersection point of a visual axis of the initial virtual camera and a preset virtual plane is selected as the position of a sub-camera in the virtual camera array, and the rotation angle of the double prisms is determined according to the relationship between the position of the sub-camera in the virtual camera array and the azimuth angle and the pitch angle of the visual axis deflection;
step two: according to the relation between the position of the sub-camera in the required virtual camera array and the rotation angle of the double prisms, the double prisms are rotated for multiple times to change the visual axis of the fixed single camera, so that the dynamic scanning of the target scene is realized, and an uncorrected multi-image sequence with sufficient angle and spatial resolution is obtained;
step three: in order to enable the obtained Epipolar Plane Image (EPI) information to be detected more easily, the position and posture of the Image obtained by the virtual camera are transformed, equivalent virtual cameras with different internal references are integrated and transformed into the same direction and the same virtual Plane, an equivalent virtual camera array consisting of a plurality of parallel, equidistant and coplanar multi-camera structures is formed, namely, an uncorrected multi-Image sequence is transformed into a two-dimensional Image collected by a corresponding planar virtual camera array;
step four: according to the two-dimensional image acquired by the planar virtual camera array, the image set after the transformation correction is used for realizing the three-dimensional information reconstruction of the target scene by combining the light field theory, namely the three-dimensional imaging compactness and flexibility are improved by the virtual camera array based on the rotating double prisms.
2. The method for three-dimensional imaging of the virtual camera array based on the rotating biprism of claim 1, wherein: the first implementation method comprises the following steps of,
based on the refraction effect of the prism, the visual axis of the fixed single camera is deflected under the action of the double prisms at a certain rotation angle, the visual axis of the fixed single camera is deflected to different degrees when passing through each surface of the double prisms in the system, and the deflected light ray vector Ri(i ═ 1,2,3,4) is represented by:
wherein n isiAnd ni-1Corresponding to the refractive indices of the front and rear incident refraction surfaces, NiA normal vector corresponding to each face;
according to the analysis of the refraction direction of the light after passing through each prism surface and the combination of the parameters and the spacing of the prisms, the intersection point S of the visual axis and each surface can be obtainedi(i ═ 1,2,3,4,5), expressed as:
wherein, tiRepresents the corresponding deflection parameters, which are determined by the direction of the deflected light and the pitch of each facet;
at the moment, an image acquired by the fixed single camera can be regarded as a non-corrected image acquired by a required initial virtual camera with different pose parameters, an intersection point of a visual axis of the initial virtual camera and a preset virtual plane is selected as the position of a sub-camera in the virtual camera array, and the rotation angle of the double prisms is determined according to the relationship between the position of the sub-camera in the required virtual camera array and the azimuth angle and the pitch angle of the visual axis deflection;
initial virtual camera C corresponding to the uncorrected imagevIs represented as:
wherein m isvRepresented as a two-dimensional projection of a three-dimensional point M, Kv、RvAnd TvRespectively, an initial virtual camera CvInternal parameters of (a), a rotation matrix, and an estimated translation vector.
3. The method as claimed in claim 2, wherein the method comprises the following steps: in the third step, the first step is that,
sub-camera C in virtual camera arrayvrIs represented as:
wherein m isvrRepresented as a two-dimensional projection of a three-dimensional point M, Kvr、RvrAnd TvrRespectively, sub-camera C in the corrected virtual camera arrayvrInternal parameters, rotation matrix and translation vector of (a);
transformation matrix H between initial virtual camera and corrected virtual camerakExpressed as:
wherein the content of the first and second substances,
representing a pseudo-inverse matrix calculation; and obtaining images acquired by the sub-cameras in the virtual camera array through conversion between the virtual cameras and the corrected virtual cameras.
4. The method as claimed in claim 3, wherein the method comprises the following steps: the compactness refers to that multi-view images collected by a multi-camera array can be acquired by deflecting and fixing the visual axis of a single camera and changing the pose of a virtual camera; the flexibility means that the number and the positions of the cameras in the virtual camera array can be changed by changing the combination of the rotation angle values of the double prisms, and the flexibility can adapt to different three-dimensional imaging scenes without changing the position of each real single camera relative to a real camera array.
5. A virtual camera array three-dimensional imaging system based on a rotating biprism, which is used for implementing the virtual camera array three-dimensional imaging method based on the rotating biprism as claimed in claim 1,2,3 or 4, and is characterized in that: the system comprises a real fixed single camera, sub-cameras in a virtual camera array, a double prism, a magnetic encoder, a micro stepping motor, a gear set and a supporting structure;
under the action of a double prism at a certain rotation angle, the real fixed single camera is used for acquiring an image which can be regarded as being collected by an initial virtual camera with a pose parameter different from that of the real camera;
the miniature stepping motor is used for driving the helical gear embedded with the prism to rotate for multiple times, so that the visual axis of the fixed single camera is changed for multiple times, and the deflection of the system visual axis and the scanning of a target scene are realized;
based on the refraction effect of the prism, the double prisms are used for deflecting the visual axis of the real fixed single camera, and the rotation angle of the double prisms is determined by the relation between the position of the sub camera in the required virtual camera array and the azimuth angle and the pitch angle of the deflection of the visual axis; the independent prism combination is used for constructing the relation between the fixed single camera and the virtual camera array; the prism combination is coaxially arranged right in front of the fixed single camera and is separated by a fixed distance value;
the magnetic encoder is annular, is coaxial with the prism and the real fixed single camera, and is used for detecting the rotation position of the prism in real time, so that the virtual camera array three-dimensional imaging system has a control function of closed-loop feedback;
the sub-cameras in the virtual camera array are used for uniformly acquiring information of light propagation in a target scene, so that three-dimensional calculation imaging is realized.
6. The rotating biprism based virtual camera array three dimensional imaging system of claim 5, wherein: the working method is that,
signals for controlling the double prisms and outputting camera images are sent and received by opening corresponding system serial ports; the prism can be rotated to a preset angle in a high-precision mode by driving the gear to rotate by using the pulse signal driving stepping motor, and the angle adjustment control is carried out on the rotation angle of the gear by receiving angle value data serving as feedback information in real time; when the preset angle value is judged to be reached, acquiring an uncorrected image sequence acquired by the initial virtual camera; acquiring an uncorrected image of multiple angles under the rotation angles of a plurality of groups of rotating double prisms, and acquiring a two-dimensional image sequence acquired by a virtual camera array by transforming the uncorrected image sequence; by acquiring the acquired two-dimensional image sequence, the analysis of the light propagation information can be realized, namely, the three-dimensional information reconstruction of a target scene is realized by combining the image set after the transformation correction with the light field theory according to the two-dimensional image acquired by the planar virtual camera array, so that the compactness and the flexibility of the three-dimensional imaging are improved.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111437830.7A CN114157852B (en) | 2021-11-30 | 2021-11-30 | Virtual camera array three-dimensional imaging method and system based on rotating double prisms |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111437830.7A CN114157852B (en) | 2021-11-30 | 2021-11-30 | Virtual camera array three-dimensional imaging method and system based on rotating double prisms |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114157852A true CN114157852A (en) | 2022-03-08 |
| CN114157852B CN114157852B (en) | 2022-12-13 |
Family
ID=80454821
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111437830.7A Active CN114157852B (en) | 2021-11-30 | 2021-11-30 | Virtual camera array three-dimensional imaging method and system based on rotating double prisms |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114157852B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117115004A (en) * | 2023-04-28 | 2023-11-24 | 荣耀终端有限公司 | Image processing method and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108253939A (en) * | 2017-12-19 | 2018-07-06 | 同济大学 | Variable optical axis single eye stereo vision measuring method |
| CN109597275A (en) * | 2018-11-29 | 2019-04-09 | 同济大学 | A kind of axial Distributed Three-dimensional imaging method based on double-wedge prism |
| CN110243283A (en) * | 2019-05-30 | 2019-09-17 | 同济大学 | A kind of variable optical axis vision measurement system and method |
| US20210014473A1 (en) * | 2018-03-22 | 2021-01-14 | Arizona Board Of Regents On Behalf Of The University Of Arizona | Methods of rendering light field images for integral-imaging-based light field display |
| CN112330794A (en) * | 2020-10-09 | 2021-02-05 | 同济大学 | Single-camera image acquisition system based on rotary bipartite prism and three-dimensional reconstruction method |
-
2021
- 2021-11-30 CN CN202111437830.7A patent/CN114157852B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108253939A (en) * | 2017-12-19 | 2018-07-06 | 同济大学 | Variable optical axis single eye stereo vision measuring method |
| US20210014473A1 (en) * | 2018-03-22 | 2021-01-14 | Arizona Board Of Regents On Behalf Of The University Of Arizona | Methods of rendering light field images for integral-imaging-based light field display |
| CN109597275A (en) * | 2018-11-29 | 2019-04-09 | 同济大学 | A kind of axial Distributed Three-dimensional imaging method based on double-wedge prism |
| CN110243283A (en) * | 2019-05-30 | 2019-09-17 | 同济大学 | A kind of variable optical axis vision measurement system and method |
| CN112330794A (en) * | 2020-10-09 | 2021-02-05 | 同济大学 | Single-camera image acquisition system based on rotary bipartite prism and three-dimensional reconstruction method |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117115004A (en) * | 2023-04-28 | 2023-11-24 | 荣耀终端有限公司 | Image processing method and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114157852B (en) | 2022-12-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10863164B2 (en) | Stereo camera and method of controlling stereo camera | |
| CN110243283B (en) | Visual measurement system and method with variable visual axis | |
| CN109767476B (en) | Automatic focusing binocular camera calibration and depth calculation method | |
| US6496218B2 (en) | Stereoscopic image display apparatus for detecting viewpoint and forming stereoscopic image while following up viewpoint position | |
| CN102494609B (en) | Three-dimensional photographing process based on laser probe array and device utilizing same | |
| CN109859272B (en) | Automatic focusing binocular camera calibration method and device | |
| EP3144880B1 (en) | A method and an apparatus for generating data representative of a light field | |
| US6236748B1 (en) | Compound eye image pickup device utilizing plural image sensors and plural lenses | |
| CN109242914B (en) | Three-dimensional calibration method of movable vision system | |
| EP0811876A1 (en) | Method and apparatus for three-dimensional measurement and imaging having focus-related convergence compensation | |
| US6839081B1 (en) | Virtual image sensing and generating method and apparatus | |
| CN107449374A (en) | The vision auxiliary laser galvanometer scanning system and its field calibration method of flexible layout | |
| CN113175899B (en) | Camera and galvanometer combined three-dimensional imaging model of variable sight line system and calibration method thereof | |
| CN106340045B (en) | Calibration optimization method in three-dimensional facial reconstruction based on binocular stereo vision | |
| CN111080705B (en) | Calibration method and device for automatic focusing binocular camera | |
| CN111854636B (en) | Multi-camera array three-dimensional detection system and method | |
| US20120113231A1 (en) | 3d camera | |
| CN112904766A (en) | Rotating double-prism pointing control system and method based on pixel error feedback | |
| KR20180053668A (en) | Method and apparatus for generating data representing a pixel beam | |
| CN110360929A (en) | High speed line scan sensor and its scaling method | |
| CN108596960B (en) | Sub-aperture image alignment method of light field camera | |
| CN114157852B (en) | Virtual camera array three-dimensional imaging method and system based on rotating double prisms | |
| CN114359406A (en) | Calibration of auto-focusing binocular camera, 3D vision and depth point cloud calculation method | |
| CN111800588A (en) | Optical unmanned aerial vehicle monitoring system based on three-dimensional light field technology | |
| US5390024A (en) | Optical transform generating apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |