CN114157852B - Virtual camera array three-dimensional imaging method and system based on rotating double prisms - Google Patents
Virtual camera array three-dimensional imaging method and system based on rotating double prisms Download PDFInfo
- Publication number
- CN114157852B CN114157852B CN202111437830.7A CN202111437830A CN114157852B CN 114157852 B CN114157852 B CN 114157852B CN 202111437830 A CN202111437830 A CN 202111437830A CN 114157852 B CN114157852 B CN 114157852B
- Authority
- CN
- China
- Prior art keywords
- virtual camera
- camera
- camera array
- visual axis
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 54
- 230000000007 visual effect Effects 0.000 claims abstract description 56
- 238000000034 method Methods 0.000 claims abstract description 19
- 230000000694 effects Effects 0.000 claims abstract description 11
- 230000009471 action Effects 0.000 claims abstract description 10
- 239000011159 matrix material Substances 0.000 claims description 14
- 230000009466 transformation Effects 0.000 claims description 8
- 238000013519 translation Methods 0.000 claims description 6
- 238000005290 field theory Methods 0.000 claims description 5
- 238000004458 analytical method Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000012937 correction Methods 0.000 claims description 3
- 230000008859 change Effects 0.000 claims description 2
- 238000006243 chemical reaction Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000001131 transforming effect Effects 0.000 description 3
- 238000005259 measurement Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/346—Image reproducers using prisms or semi-transparent mirrors
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B35/00—Stereoscopic photography
- G03B35/02—Stereoscopic photography by sequential recording
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a virtual camera array three-dimensional imaging method and system based on a rotating biprism, and belongs to the technical field of three-dimensional computational imaging. Based on the prism refraction effect, the visual axis of the fixed single camera deflects under the action of the double prisms at a certain rotation angle; selecting an intersection point of a visual axis of the initial virtual camera and a preset virtual plane as the position of a sub-camera in the virtual camera array, and determining a rotating angle of the double prisms according to the relation between the position of the sub-camera in the virtual camera array and the azimuth angle and the pitch angle of the visual axis deflection; the method comprises the steps that the visual axis of a fixed single camera is changed by rotating a double prism for multiple times, so that dynamic scanning of a target scene is achieved, and an uncorrected multi-image sequence with sufficient angle and spatial resolution is obtained; the uncorrected multi-image sequence is converted into an image sequence acquired by a corresponding plane virtual camera array, and the three-dimensional information reconstruction of a target scene is realized by combining a light field imaging theory, so that the compactness and the flexibility of three-dimensional imaging are improved.
Description
Technical Field
The invention relates to a virtual camera array three-dimensional imaging method and system based on a rotating biprism, and belongs to the technical field of three-dimensional computational imaging.
Background
Three-dimensional imaging technology is one of the most important technologies in applications such as robotics, unmanned driving, and smart sensing. Passive three-dimensional measurement schemes have been widely used in recent years in a number of target scenarios, since they can be laser independent. The passive three-dimensional measurement system can be divided into a monocular camera, a binocular camera and a multi-view camera three-dimensional reconstruction method. However, the multi-view camera is one of the popular research directions in the three-dimensional imaging technology because it can acquire image information of a target scene from multiple angles.
The traditional multi-camera three-dimensional reconstruction mainly comprises the following steps: the system has the advantages that a real camera array is used, dozens of real cameras are generally required to be arranged at fixed positions in a scene, and meanwhile, light information is acquired for a target scene, and the system has the defects that the size is large, and the distance between the cameras is limited by the size of a single camera; the micro-lens array is used for carrying out secondary imaging on the image of the main lens, and the system uses the same sensor as a receiving surface of the secondary imaging, so that the resolution of the acquired image is seriously limited by the size of the angular resolution; the single camera is used for shooting the same target scene at multiple angles by using the moving single camera matched with the guide rail and through displacement on the two-dimensional guide rail, the actual size of the system is relatively large due to the motion factor of the guide rail, the moving range of the system is limited by the length of the guide rail, and the camera is easily influenced by the precision and the straightness of the guide rail.
Disclosure of Invention
The invention mainly aims to provide a virtual camera array three-dimensional imaging method and a system based on a rotating double prism, based on the refraction effect of the prism, the visual axis of a fixed single camera deflects under the action of the double prism at a certain rotating angle, at the moment, an image acquired by the fixed single camera can be regarded as a non-corrected image acquired by a required initial virtual camera with different pose parameters, the intersection point of the visual axis of the initial virtual camera and a preset virtual plane is selected as the position of a sub-camera in a virtual camera array, and the rotating angle of the double prism is determined according to the relation between the position of the sub-camera in the required virtual camera array and the azimuth angle and the pitch angle of the deflection of the visual axis; the visual axis of a fixed single camera is changed by rotating a double prism for multiple times, so that dynamic scanning of a target scene is realized, and an uncorrected multi-image sequence with sufficient angle and spatial resolution is obtained; the obtained multi-angle uncorrected image sequence is converted, the uncorrected multi-image sequence is converted into an image sequence acquired by a corresponding plane virtual camera array, and three-dimensional information reconstruction of a target scene is realized by combining a light field imaging theory according to a two-dimensional image acquired by the plane virtual camera array, namely the three-dimensional imaging compactness and flexibility are improved by the virtual camera array based on the rotating biprism. The compactness refers to that multi-view images collected by a multi-camera array can be acquired by deflecting and fixing the visual axis of a single camera and changing the pose of a virtual camera; the flexibility means that the number and the positions of the cameras in the virtual camera array can be changed by changing the combination of the rotation angle values of the double prisms, and the flexibility can adapt to different three-dimensional imaging scenes without changing the position of each real single camera relative to a real camera array.
The invention discloses a virtual camera array three-dimensional imaging method based on a rotating biprism, which comprises the following steps:
the method comprises the following steps: based on the refraction effect of the prism, the visual axis of the fixed single camera will be deflected under the action of the biprism at a certain rotation angle, and the visual axis will be deflected to different degrees when passing through each surface of the biprism in the system. At the moment, an image acquired by the fixed single camera can be regarded as a non-corrected image acquired by a required initial virtual camera with different pose parameters, an intersection point of a visual axis of the initial virtual camera and a preset virtual plane is selected as the position of a sub-camera in the virtual camera array, and the rotation angle of the double prisms is determined according to the relationship between the position of the sub-camera in the virtual camera array and the azimuth angle and the pitch angle of the visual axis deflection.
Based on the refraction effect of the prism, the visual axis of the fixed single camera deflects under the action of the double prisms at a certain rotation angle, when the visual axis passes through each surface of the double prisms in the system, the visual axis deflects to different degrees, and the deflected light ray vector R i (i =1,2,3,4) is expressed as:
wherein n is i And n i-1 Corresponding to the refractive indexes before and after the incident refraction surface, N i Corresponding to the normal vector of each facet.
According to the analysis of the refraction direction of the light after passing through each prism surface and the combination of the parameters and the spacing of the prisms, the intersection point S of the visual axis and each surface can be obtained i (i =1,2,3,4,5), expressed as:
wherein, t i Representing corresponding deflection parametersWhich is determined by the direction of the deflected light and the spacing of each facet.
At the moment, an image acquired by the fixed single camera can be regarded as a non-corrected image acquired by a required initial virtual camera with different pose parameters, an intersection point of a visual axis of the initial virtual camera and a preset virtual plane is selected as the position of a sub-camera in the virtual camera array, and the rotation angle of the double prisms is determined according to the relation between the position of the sub-camera in the required virtual camera array and the azimuth angle and the pitch angle of visual axis deflection.
Initial virtual camera C corresponding to the uncorrected image v Is represented as:
wherein m is v Represented as a two-dimensional projection of a three-dimensional point M, K v 、R v And T v Are respectively the initial virtual camera C v Internal parameters of (a), a rotation matrix, and an estimated translation vector.
Step two: and according to the relation between the position of the sub-camera in the required virtual camera array and the rotation angle of the double prisms, changing the visual axis of the fixed single camera by rotating the double prisms for multiple times, and realizing dynamic scanning of the target scene, so that an uncorrected multi-image sequence with sufficient angle and spatial resolution is obtained.
Step three: in order to enable the obtained Epipolar Plane Image (EPI) information to be detected more easily, the pose of the Image obtained by the virtual camera is transformed, equivalent virtual cameras with different internal references are integrated and transformed into the same virtual Plane in the same direction, an equivalent virtual camera array composed of a plurality of parallel, equidistant and coplanar multi-camera structures is formed, and an uncorrected multi-Image sequence is transformed into a two-dimensional Image collected by the corresponding planar virtual camera array.
Sub-camera C in virtual camera array vr Is represented as:
wherein m is vr Represented as a two-dimensional projection of a three-dimensional point M, K vr 、R vr And T vr Respectively, sub-camera C in the corrected virtual camera array vr Internal parameters, rotation matrix, and translation vector.
Transformation matrix H between initial virtual camera and corrected virtual camera k Expressed as:
wherein,
representing a pseudo-inverse matrix calculation. And obtaining images acquired by the sub-cameras in the virtual camera array through conversion between the virtual cameras and the corrected virtual cameras.
Step four: according to the two-dimensional image acquired by the planar virtual camera array, the image set after the transformation correction is used for realizing the three-dimensional information reconstruction of the target scene by combining the light field theory, namely the three-dimensional imaging compactness and flexibility are improved by the virtual camera array based on the rotating biprism.
The invention also discloses a virtual camera array three-dimensional imaging system based on the rotating double prisms, which is used for realizing the virtual camera array three-dimensional imaging method based on the rotating double prisms.
Under the action of the biprism at a certain rotation angle, the real fixed single camera is used for acquiring images which can be regarded as being acquired by an initial virtual camera with different pose parameters from the real camera.
The miniature stepping motor is used for driving the bevel gear embedded with the prism to rotate for multiple times, so that the visual axis of the fixed single camera is changed for multiple times, and the deflection of the system visual axis and the scanning of a target scene are realized.
Based on the refraction effect of the prism, the double prisms are used for deflecting the visual axis of the real fixed single camera, and the rotation angle of the double prisms is determined by the relation between the position of the sub camera in the required virtual camera array and the azimuth angle and the pitch angle of the visual axis deflection. The independent prism combination is used for constructing the relation between the fixed single camera and the virtual camera array. The prism combination is coaxially arranged right in front of the fixed single camera and is separated by a fixed distance value.
The magnetic encoder is annular, is coaxial with the prism and the real fixed single camera, and is used for detecting the rotation position of the prism in real time, so that the virtual camera array three-dimensional imaging system has a closed-loop feedback control function.
The sub-cameras in the virtual camera array are used for uniformly acquiring information of light propagation in a target scene, so that three-dimensional calculation imaging is realized.
The invention also discloses a working method of the virtual camera array three-dimensional imaging system based on the rotating biprism, which comprises the following steps:
and signals for controlling the double prisms and outputting camera images are transmitted and received by opening corresponding system serial ports. By driving the stepping motor to rotate the gear using the pulse signal, the prism can be rotated to a preset angle in a high-precision manner, and the angle adjustment control of the rotation angle of the gear is performed by receiving angle value data as feedback information in real time. And when the preset angle value is reached, acquiring an uncorrected image sequence acquired by the initial virtual camera. The method comprises the steps of obtaining uncorrected images of multiple angles under the rotation angles of multiple groups of rotating double prisms, and obtaining a two-dimensional image sequence collected by a virtual camera array by transforming the uncorrected image sequence. By acquiring the acquired two-dimensional image sequence, the analysis of light propagation information can be realized, namely, the three-dimensional information reconstruction of a target scene is realized by combining a transformed and corrected image set with a light field theory according to the two-dimensional image acquired by the planar virtual camera array, and the compactness and the flexibility of three-dimensional imaging are improved.
Has the beneficial effects that:
1. the invention discloses a virtual camera array three-dimensional imaging method and a system based on a rotating double prism, wherein based on the refraction effect of the prism, the visual axis of a fixed single camera deflects under the action of the double prism at a certain rotating angle, at the moment, an image acquired by the fixed single camera can be regarded as a non-corrected image acquired by a required initial virtual camera with different pose parameters, the intersection point of the visual axis of the initial virtual camera and a preset virtual plane is selected as the position of a sub-camera in a virtual camera array, and the rotating angle of the double prism is determined according to the relation between the position of the sub-camera in the required virtual camera array and the azimuth angle and the pitch angle of the visual axis deflection; the method comprises the steps that the visual axis of a fixed single camera is changed by rotating a double prism for multiple times, so that dynamic scanning of a target scene is achieved, and an uncorrected multi-image sequence with sufficient angle and spatial resolution is obtained; the method comprises the steps of transforming an obtained multi-angle uncorrected image sequence, transforming the uncorrected multi-image sequence into an image sequence acquired by a corresponding plane virtual camera array, and realizing three-dimensional information reconstruction of a target scene by combining a light field imaging theory according to a two-dimensional image acquired by the plane virtual camera array, namely improving the compactness and the flexibility of three-dimensional imaging by the rotating biprism-based virtual camera array.
2. The invention discloses a virtual camera array three-dimensional imaging method and system based on a rotating double prism, which can obtain images acquired by a multi-camera array by deflecting the visual axis of a real camera and changing the pose of an original virtual camera, thereby improving the compactness of a three-dimensional imaging system.
3. The invention discloses a method and a system for three-dimensional imaging of a virtual camera array based on a rotating biprism.
4. Compared with the existing three-dimensional information acquisition system, the invention discloses a virtual camera array three-dimensional imaging method and system based on a rotating double prism, which realize dynamic scanning of a target scene by changing the visual axis of a fixed single camera through rotating the double prism for multiple times, thereby acquiring an uncorrected multi-image sequence with sufficient angles and spatial resolution and acquiring multi-angle information of the scene without sacrificing the spatial resolution.
5. The invention discloses a virtual camera array three-dimensional imaging method and a virtual camera array three-dimensional imaging system based on a rotating double prism.
Drawings
FIG. 1 is a three-dimensional schematic diagram of a rotating biprism based virtual camera array system of the present invention.
FIG. 2 is a flow chart of a method for three-dimensional imaging of a virtual camera array based on a rotating biprism according to the present invention.
FIG. 3 is a schematic diagram of acquiring an image captured by a virtual camera array according to the present invention.
Fig. 4 is a diagram of an example of the virtual camera array based on the rotating biprism according to the present invention.
Fig. 5 is a flow chart of the detailed operation of the system of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
As shown in fig. 1, a three-dimensional imaging system based on a rotating double prism virtual camera array disclosed in this example includes a prism 110, a prism 111, a real fixed single camera 120, a sub-camera 130 in the virtual camera array, a magnetic encoder (not shown), a micro stepper motor (not shown), and a gear set and a supporting structure (not shown).
Under the action of the prism 110 and the prism 111 at a certain rotation angle, the real fixed single camera 120 is used to acquire an image that can be regarded as being captured by an initial virtual camera with different pose parameters from those of the real camera.
The miniature stepping motor is used for driving the corresponding 45-degree helical gear embedded with the prism to rotate for multiple times, so that the visual axis of the fixed single camera 120 is changed for multiple times, and the deflection of the system visual axis and the scanning of a target scene are realized.
Each micro-stepper motor is equipped with a motor driver (not shown) connected thereto as a control signal input for driving the micro-stepper motor.
Based on the refraction effect of the prism, the double prism is used to deflect the visual axis of the real fixed single camera 120, and the rotation angle is determined by the relationship between the position of the sub-camera 130 in the desired virtual camera array and the azimuth angle and the pitch angle of the visual axis deflection. The prism 110 and the prism 111 are used for constructing the relationship between the fixed single camera 120 and the sub-camera 130 in the virtual camera array. The prism combination is coaxially placed right in front of the fixed single camera and separated by a distance value of 33mm.
The magnetic encoder is annular, is coaxial with the prism 110, the prism 111 and the real fixed single camera 120, and is used for detecting the rotation position of the prism in real time, so that the virtual camera array three-dimensional imaging system has a closed-loop feedback control function.
The sub-cameras 130 in the virtual camera array are used for uniformly acquiring information of light propagation in a target scene, so that three-dimensional computational imaging is realized.
In the embodiment, a 5 × 5 virtual camera array is formed by rotating a biprism, and the interval between sub-cameras constituting the virtual camera array is 0.6mm.
As shown in fig. 2, the method for three-dimensional imaging of a virtual camera array based on a rotating biprism disclosed in this embodiment includes the following steps:
the method comprises the following steps: based on the refraction effect of the prism, the visual axis of the fixed single camera 120 will be deflected by the double prism at a certain rotation angle, as shown in FIG. 3, when the visual axis passes through each face of the double prism in the systemWill all deflect to different extents, the deflected ray vector R i (i =1,2,3,4) is expressed as:
wherein n is i And n i-1 Corresponding to the refractive indices of the front and rear incident refraction surfaces, N i Corresponding to the normal vector of each facet.
The direction of the light refraction after passing through each prism surface is analyzed, and the parameters and the intervals of the prisms are combined, so that the intersection point S of the visual axis and each surface can be obtained i (i =1,2,3,4,5), which can be expressed as:
wherein, t i Represents the corresponding deflection parameter, which is determined by the direction of the deflected light and the pitch of each facet.
At this time, the image obtained by the fixed single camera 120 can be regarded as a non-corrected image acquired by a required initial virtual camera with different pose parameters, an intersection point of a visual axis of the initial virtual camera and a preset virtual plane is selected as a position of a sub-camera in the virtual camera array, and a rotation angle of the double prisms is determined according to a relation between the position of the sub-camera 130 in the required virtual camera array and an azimuth angle and a pitch angle of visual axis deflection.
Initial virtual camera C corresponding to the uncorrected image v Is represented as:
wherein m is v Represented as a two-dimensional projection of a three-dimensional point M, K v 、R v And T v Respectively, an initial virtual camera C v The rotation matrix and the estimated translation vector.
Step two: according to the relationship between the position of the sub-camera 130 in the required virtual camera array and the rotation angle of the biprism, the dynamic scanning of the target scene is realized by rotating the biprism multiple times to change the visual axis of the fixed single camera 120, thereby obtaining an uncorrected multi-image sequence with sufficient angle and spatial resolution.
Step three: in order to enable the obtained Epipolar Plane Image (EPI) information to be detected more easily, equivalent virtual cameras with different internal parameters are gathered and transformed to the same direction and the same virtual Plane by carrying out pose transformation on images obtained by the virtual cameras to form an equivalent virtual camera array consisting of a plurality of parallel, equidistant and coplanar multi-camera structures, namely, an uncorrected multi-Image sequence is transformed into an Image sequence collected by a corresponding planar virtual camera array, and a sub-camera C in the virtual camera array is used for detecting the Epipolar Plane Image (EPI) information more easily vr The projection matrix of (a) may be expressed as:
wherein m is vr Represented as a two-dimensional projection of a three-dimensional point M, K vr 、R vr And T vr Respectively, sub-camera C in the corrected virtual camera array vr Internal parameters, rotation matrix, and translation vector.
Transformation matrix H between initial virtual camera and corrected virtual camera k Can be expressed as:
wherein,
representing a pseudo-inverse matrix calculation. Through the above process, the image acquired by the sub-camera 130 in the virtual camera array can be obtained.
Step four: according to the two-dimensional image acquired by the planar virtual camera array, the image set after the transformation correction is used for realizing the three-dimensional information reconstruction of the target scene by combining the light field theory, namely the three-dimensional imaging compactness and flexibility are improved by the virtual camera array based on the rotating biprism.
As shown in fig. 4, according to the example image collected by the virtual camera array based on the rotating biprism disclosed in this embodiment, the images are arranged in an array and numbered in sequence, wherein each image is obtained at a rotation angle corresponding to the prism, then the collected images are converted into an EPI format, a depth value at a corresponding point is calculated according to different straight line slopes in the EPI, a disparity map is generated, and a depth map of a target scene is obtained according to the disparity map and the sub-camera distance and the focal length.
As shown in fig. 5, the embodiment further discloses a working method of the virtual camera array three-dimensional imaging system based on the rotating biprism, which includes:
and signals for controlling the double prisms and outputting camera images are transmitted and received by opening corresponding system serial ports. The prism can be rotated to a preset angle in a high-precision manner by driving the gear to rotate by using the pulse signal driving stepping motor, and angle adjustment control is performed on the rotation angle of the gear by receiving angle value data as feedback information in real time. And when the preset angle value is reached, acquiring an uncorrected image acquired by the initial virtual camera. By obtaining a multi-angle uncorrected image sequence under the rotation angles of a plurality of groups of rotating double prisms and combining a matrix H k And realizing the transformation of the uncorrected image sequence so as to acquire the two-dimensional image sequence acquired by the virtual camera array. By acquiring the acquired two-dimensional image sequence, the light propagation information can be analyzed, so that three-dimensional imaging is realized.
The invention provides a virtual camera array three-dimensional imaging system and method based on a rotating biprism, which aim to realize array three-dimensional imaging more flexibly and have the characteristics of compact structure and stable control.
The above detailed description is intended to illustrate the objects, aspects and advantages of the present invention, and it should be understood that the above detailed description is only exemplary of the present invention and is not intended to limit the scope of the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (5)
1. A virtual camera array three-dimensional imaging method based on a rotating biprism is characterized in that: comprises the following steps of (a) carrying out,
the method comprises the following steps: based on the refraction effect of the prism, the visual axis of the fixed single camera deflects under the action of the double prisms at a certain rotation angle, and the visual axis of the fixed single camera deflects to different degrees when passing through each surface of the double prisms in the system; at the moment, an image acquired by the fixed single camera can be regarded as a non-corrected image acquired by a required initial virtual camera with different pose parameters, an intersection point of a visual axis of the initial virtual camera and a preset virtual plane is selected as the position of a sub-camera in the virtual camera array, and the rotation angle of the double prisms is determined according to the relationship between the position of the sub-camera in the virtual camera array and the azimuth angle and the pitch angle of the visual axis deflection;
step two: according to the relation between the position of the sub-camera in the required virtual camera array and the rotation angle of the double prisms, the double prisms are rotated for multiple times to change the visual axis of the fixed single camera, so that the dynamic scanning of the target scene is realized, and an uncorrected multi-image sequence with sufficient angle and spatial resolution is obtained;
step three: in order to enable the obtained Epipolar Plane Image (EPI) information to be detected more easily, the position and posture of the Image obtained by the virtual camera are transformed, equivalent virtual cameras with different internal references are integrated and transformed into the same direction and the same virtual Plane, an equivalent virtual camera array consisting of a plurality of parallel, equidistant and coplanar multi-camera structures is formed, namely, an uncorrected multi-Image sequence is transformed into a two-dimensional Image collected by the corresponding planar virtual camera array;
step four: according to two-dimensional images acquired by a planar virtual camera array, a light field theory is combined to use a converted and corrected image set for realizing three-dimensional information reconstruction of a target scene, namely the three-dimensional imaging compactness and flexibility are improved by the virtual camera array based on the rotating biprism;
the first implementation method comprises the following steps of,
based on the refraction effect of the prism, the visual axis of the fixed single camera deflects under the action of the double prisms at a certain rotation angle, when the visual axis passes through each surface of the double prisms in the system, the visual axis deflects to different degrees, and the deflected light ray vector R i (i =1,2,3,4) is expressed as:
wherein n is i And n i-1 Corresponding to the refractive indices of the front and rear incident refraction surfaces, N i A normal vector corresponding to each face;
according to the analysis of the refraction direction of the light after passing through each prism surface and the combination of the parameters and the spacing of the prisms, the intersection point S of the visual axis and each surface can be obtained i (i =1,2,3,4,5), expressed as:
wherein, t i Represents the corresponding deflection parameters, which are determined by the direction of the deflected light and the pitch of each facet;
at the moment, an image acquired by the fixed single camera can be regarded as a non-corrected image acquired by a required initial virtual camera with different pose parameters, an intersection point of a visual axis of the initial virtual camera and a preset virtual plane is selected as the position of a sub-camera in the virtual camera array, and the rotation angle of the double prisms is determined according to the relationship between the position of the sub-camera in the required virtual camera array and the azimuth angle and the pitch angle of the visual axis deflection;
initial virtual camera C corresponding to the uncorrected image v Is represented as:
wherein m is v Represented as a two-dimensional projection of a three-dimensional point M, K v 、R v And T v Are respectively the initial virtual camera C v Internal parameters of (a), a rotation matrix, and an estimated translation vector.
2. The method for three-dimensional imaging of the virtual camera array based on the rotating biprism of claim 1, wherein: in the third step, the first step is that,
sub-camera C in virtual camera array vr Is represented as:
wherein m is vr Represented as a two-dimensional projection of a three-dimensional point M, K vr 、R vr And T vr Respectively, sub-camera C in the corrected virtual camera array vr Internal parameters, rotation matrix and translation vector of (a);
transformation matrix H between initial virtual camera and corrected virtual camera k Expressed as:
wherein,
representing a pseudo-inverse matrix calculation; and obtaining images acquired by the sub-cameras in the virtual camera array through conversion between the virtual cameras and the corrected virtual cameras.
3. The method for three-dimensional imaging of the virtual camera array based on the rotating biprism of claim 2 wherein: the compactness refers to that multi-view images collected by a multi-camera array can be acquired by deflecting and fixing the visual axis of a single camera and changing the pose of a virtual camera; the flexibility means that the number and the positions of the cameras in the virtual camera array can be changed by changing the combination of the rotation angle values of the double prisms, and the flexibility can adapt to different three-dimensional imaging scenes without changing the position of each real single camera relative to a real camera array.
4. A virtual camera array three-dimensional imaging system based on a rotating biprism, which is used for implementing the virtual camera array three-dimensional imaging method based on the rotating biprism as claimed in claim 1,2,3 or 4, and is characterized in that: the system comprises a real fixed single camera, sub-cameras in a virtual camera array, a double prism, a magnetic encoder, a micro stepping motor, a gear set and a supporting structure;
under the action of a double prism at a certain rotation angle, the real fixed single camera is used for acquiring an image which can be regarded as being collected by an initial virtual camera with a pose parameter different from that of the real camera;
the miniature stepping motor is used for driving the helical gear embedded with the prism to rotate for multiple times, so that the visual axis of the fixed single camera is changed for multiple times, and the deflection of the system visual axis and the scanning of a target scene are realized;
based on the refraction effect of the prism, the double prisms are used for deflecting the visual axis of the real fixed single camera, and the rotation angle of the double prisms is determined by the relation between the position of the sub camera in the required virtual camera array and the azimuth angle and the pitch angle of the deflection of the visual axis; the independent prism combination is used for constructing the relation between the fixed single camera and the virtual camera array; the prism combination is coaxially arranged right in front of the fixed single camera and is separated by a fixed distance value;
the magnetic encoder is annular, is coaxial with the prism and the real fixed single camera, and is used for detecting the rotation position of the prism in real time, so that the virtual camera array three-dimensional imaging system has a control function of closed-loop feedback;
the sub-cameras in the virtual camera array are used for uniformly acquiring information of light propagation in a target scene, so that three-dimensional computational imaging is realized.
5. The rotating biprism based virtual camera array three dimensional imaging system of claim 4, wherein: the working method is that,
signals for controlling the double prisms and outputting camera images are sent and received by opening corresponding system serial ports; the prism can be rotated to a preset angle in a high-precision mode by driving the gear to rotate by using the pulse signal driving stepping motor, and the angle adjustment control is carried out on the rotation angle of the gear by receiving angle value data serving as feedback information in real time; when the preset angle value is judged to be reached, acquiring an uncorrected image sequence acquired by the initial virtual camera; acquiring an uncorrected image of multiple angles under the rotation angles of a plurality of groups of rotating double prisms, and converting an uncorrected image sequence to acquire a two-dimensional image sequence acquired by a virtual camera array; by acquiring the acquired two-dimensional image sequence, the analysis of the light propagation information can be realized, namely, the three-dimensional information reconstruction of a target scene is realized by combining the image set after the transformation correction with the light field theory according to the two-dimensional image acquired by the planar virtual camera array, so that the compactness and the flexibility of the three-dimensional imaging are improved.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111437830.7A CN114157852B (en) | 2021-11-30 | 2021-11-30 | Virtual camera array three-dimensional imaging method and system based on rotating double prisms |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111437830.7A CN114157852B (en) | 2021-11-30 | 2021-11-30 | Virtual camera array three-dimensional imaging method and system based on rotating double prisms |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114157852A CN114157852A (en) | 2022-03-08 |
| CN114157852B true CN114157852B (en) | 2022-12-13 |
Family
ID=80454821
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111437830.7A Active CN114157852B (en) | 2021-11-30 | 2021-11-30 | Virtual camera array three-dimensional imaging method and system based on rotating double prisms |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114157852B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114842179B (en) * | 2022-05-20 | 2024-09-17 | 青岛海信医疗设备股份有限公司 | Matching method of organ three-dimensional model and intraoperative organ image and electronic equipment |
| CN114937099B (en) * | 2022-05-31 | 2024-09-03 | 北京理工大学 | Three-dimensional computing ghost imaging method and system based on rotatable prism combination |
| CN117115004B (en) * | 2023-04-28 | 2024-07-12 | 荣耀终端有限公司 | Image processing method and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108253939A (en) * | 2017-12-19 | 2018-07-06 | 同济大学 | Variable optical axis single eye stereo vision measuring method |
| CN109597275A (en) * | 2018-11-29 | 2019-04-09 | 同济大学 | A kind of axial Distributed Three-dimensional imaging method based on double-wedge prism |
| CN110243283A (en) * | 2019-05-30 | 2019-09-17 | 同济大学 | A kind of variable optical axis vision measurement system and method |
| CN112330794A (en) * | 2020-10-09 | 2021-02-05 | 同济大学 | Single-camera image acquisition system based on rotary bipartite prism and three-dimensional reconstruction method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP7185331B2 (en) * | 2018-03-22 | 2022-12-07 | アリゾナ ボード オブ リージェンツ オン ビハーフ オブ ザ ユニバーシティ オブ アリゾナ | How to render light field images for integral imaging light field displays |
-
2021
- 2021-11-30 CN CN202111437830.7A patent/CN114157852B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108253939A (en) * | 2017-12-19 | 2018-07-06 | 同济大学 | Variable optical axis single eye stereo vision measuring method |
| CN109597275A (en) * | 2018-11-29 | 2019-04-09 | 同济大学 | A kind of axial Distributed Three-dimensional imaging method based on double-wedge prism |
| CN110243283A (en) * | 2019-05-30 | 2019-09-17 | 同济大学 | A kind of variable optical axis vision measurement system and method |
| CN112330794A (en) * | 2020-10-09 | 2021-02-05 | 同济大学 | Single-camera image acquisition system based on rotary bipartite prism and three-dimensional reconstruction method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114157852A (en) | 2022-03-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114157852B (en) | Virtual camera array three-dimensional imaging method and system based on rotating double prisms | |
| US10863164B2 (en) | Stereo camera and method of controlling stereo camera | |
| CN110243283B (en) | Visual measurement system and method with variable visual axis | |
| TWI521255B (en) | Automatic focusing method, and automatic focusing device, image capturing device using the same | |
| EP3144880B1 (en) | A method and an apparatus for generating data representative of a light field | |
| CN109767476A (en) | A kind of calibration of auto-focusing binocular camera and depth computing method | |
| CN102494609A (en) | Three-dimensional photographing process based on laser probe array and device utilizing same | |
| US6839081B1 (en) | Virtual image sensing and generating method and apparatus | |
| CN105654484A (en) | Light field camera external parameter calibration device and method | |
| CN113175899B (en) | Camera and galvanometer combined three-dimensional imaging model of variable sight line system and calibration method thereof | |
| CN102253057B (en) | Endoscope system and measurement method using endoscope system | |
| CN102679959A (en) | Omnibearing 3D (Three-Dimensional) modeling system based on initiative omnidirectional vision sensor | |
| CN111854636B (en) | Multi-camera array three-dimensional detection system and method | |
| CN112904766A (en) | Rotating double-prism pointing control system and method based on pixel error feedback | |
| KR20180053668A (en) | Method and apparatus for generating data representing a pixel beam | |
| CN112634379B (en) | Three-dimensional positioning measurement method based on mixed vision field light field | |
| CN114359406A (en) | Calibration of auto-focusing binocular camera, 3D vision and depth point cloud calculation method | |
| US5390024A (en) | Optical transform generating apparatus | |
| CN108805921A (en) | Image-taking system and method | |
| CN112950727B (en) | Large-view-field multi-target simultaneous ranging method based on bionic curved compound eye | |
| CN108053468A (en) | A kind of monocular vision focuses on storehouse acquisition and scene reconstruction method | |
| CN116051651A (en) | Equivalent multi-vision model of variable-vision imaging system and calibration method thereof | |
| WO2020244273A1 (en) | Dual camera three-dimensional stereoscopic imaging system and processing method | |
| CN109565583B (en) | Method and apparatus for generating data representing a pixel beam | |
| CN115174789B (en) | Micro-lens array focusing type light field camera based on micro-motion platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |