WO2018000892A1

WO2018000892A1 - Imaging method, apparatus and system for panoramic stereo image

Info

Publication number: WO2018000892A1
Application number: PCT/CN2017/080195
Authority: WO
Inventors: 刘亚辉; 沈靖程; 王士博; 王超
Original assignee: 深圳市圆周率软件科技有限责任公司
Priority date: 2016-06-27
Filing date: 2017-04-12
Publication date: 2018-01-04
Also published as: CN106990668A; CN106990668B

Abstract

An imaging method, an apparatus and a system for a panoramic stereo image comprises: step A, arranging a plurality of image capturing units randomly in a space (S21); step B, calibrating internal and external parameters of the respective image capturing unit (S22); step C, controlling the respective image capturing unit to work coordinately and synchronously acquiring and transmitting a coordinate of image point (S23); step D, randomly specifying a spherical coordinate system of a left eye and a spherical coordinate system of a right eye in the space (S24); step E, respectively calculating a mapping function of the image capturing unit from the coordinate of image point to the spherical coordinate system of the left eye and the spherical coordinate system of the right eye to generate a spherical image of the left eye and a spherical image of the right eye (S25); step F, projecting the spherical images of the left eye and of the right eye into the left eye and the right eye respectively to complete stereo display of the panoramic image (S26). A panoramic stereo image acquisition at a large degree of freedom is achieved, thereby effectively reducing constraints on consistency in parameters of the respective image capturing units and uniformity in position arrangement in the building process of the panorama shooting equipment, and improving shooting freedom and flexibility of the image capturing unit.

Description

Imaging method, device and system for panoramic stereoscopic image

Technical field

The invention belongs to the field of panoramic imaging technology, and in particular relates to an imaging method, device and system for panoramic stereoscopic images.

Background technique

At present, the methods of panoramic imaging are mainly divided into two categories: first, using a single camera unit (or multiple camera unit) to obtain all the images in the scene space by rotating around the axis to form a panoramic image; second, using multiple camera units The images are combined in a fixed manner to obtain all the images in the scene space at the same time and then stitched together to form a panoramic image. The advantage of the above first method is that a panoramic image of a static scene can be captured by a smaller number of camera units; the disadvantage is that the imaging effect on the dynamic scene is not good, and the panoramic video cannot be obtained; the advantage of the second method described above is that Instant imaging can effectively image both static and dynamic scenes; the disadvantage is that the consistency of internal parameters of each camera unit and the uniformity of position arrangement between camera units are high. This consistency and uniformity constraints Undoubtedly reduce the flexibility of the shooting method and the possibility of free combination of multiple devices.

Summary of the invention

In summary, in order to solve the above technical problem, the present invention provides a method, a device and a system for imaging panoramic stereoscopic images, so as to realize flexible acquisition of panoramic stereoscopic images, and reduce parameters of each camera unit during the construction process of the panoramic imaging device. Consistency of uniformity and uniformity of positional placement.

In a first aspect, the present invention provides an imaging method for panoramic stereoscopic images, comprising the following steps:

Step A, arbitrarily arranging a plurality of camera units in the space according to different application scenarios;

Step B: calibrating the internal and external parameters of each camera unit in the step A that has been arranged;

Step C, controlling the camera units in each of the step B to perform cooperative work, and performing synchronous acquisition and transmission of image point coordinates;

Step D, arbitrarily designating a left-eye spherical coordinate system O _L and a right-eye spherical coordinate system O _R in the space;

Step E, according to the left-eye spherical coordinate system O _L and the right-eye spherical coordinate system O _{R in} the step D, respectively, the image point coordinates of the imaging unit in the step C are solved to the left-eye spherical coordinate system O _L and the right a mapping function of the spherical coordinate system O _R , and respectively generating a left-eye spherical image corresponding to the left-eye spherical coordinate system O _L and a right-eye spherical image corresponding to the right-eye spherical coordinate system O _R according to the solved mapping function;

In step F, the left-eye spherical image and the right-eye spherical image in the step E are respectively thrown into the left eye and the right eye to complete the stereoscopic display of the panoramic image.

Further, the solution method of the mapping function includes the following steps:

Step E1, determining the relationship between the object point coordinates and the image point coordinates in the camera unit coordinate system;

Step E2, calibrating the spherical coordinate system and the camera unit coordinate system, and determining a transformation relationship between the camera unit coordinate system and the spherical coordinate system;

Step E3, according to the relationship between the object point coordinates and the image point coordinates in the step E1 and the transformation relationship between the camera unit coordinate system and the spherical coordinate system in the step E2, the coordinates of the object point in the spherical coordinate system are solved;

In step E4, according to the coordinates of the object point in the step E3 in the spherical coordinate system, the spherical coordinates of the object point projected onto the spherical coordinate system with the radius r are solved.

Further, the distance between the center of the spherical image of the left eye and the center of the spherical image of the right eye is equal to the distance between the eyes of the human eye.

In a second aspect, the present invention provides an imaging apparatus for panoramic stereoscopic images, including:

The camera unit is arranged to arrange a plurality of camera units in the space according to the unused application scenarios;

a system calibration module, configured to calibrate internal and external parameters of each camera unit that has been arranged in the camera unit arrangement module, and is connected to the camera unit arrangement module;

a hardware communication module, configured to control each camera unit of the camera unit arrangement module to perform cooperative work and perform synchronous acquisition of image point coordinates, receive image point coordinates collected by each camera unit in the camera unit arrangement module, and transmit Connected to the camera unit arrangement module;

The left-eye and right-eye spherical coordinate system specifies a module for arbitrarily designating a left-eye spherical coordinate system O _L and a right-eye spherical coordinate system O _R in the space;

a mapping function solving module, configured to receive image point coordinates transmitted by the hardware communication module, and specify a left-eye spherical coordinate system O _L and a right-eye spherical coordinate system O _R according to the left-eye and right-eye spherical coordinate systems respectively Solving a mapping function of the image point coordinates of the camera unit in the camera unit arrangement module to the left-eye spherical coordinate system O _L and the right-eye spherical coordinate system O _R , and respectively generating a left eyeball according to the solved mapping function a left-eye spherical image corresponding to the surface coordinate system O _L and a right-eye spherical image corresponding to the right-eye spherical coordinate system O _R , which are connected to the hardware communication module and the left-eye and right-eye spherical coordinate system specifying modules;

An image display module, configured to cast a left-eye spherical image and a right-eye spherical image in the mapping function solving module into the left eye and the right eye respectively to complete stereoscopic display of the panoramic image, which is compared with the mapping function solving module connection.

Further, the mapping function solving module includes:

a coordinate relationship determining unit, configured to determine a relationship between the object point coordinates and the image point coordinates in the camera unit coordinate system;

a coordinate system transformation relationship determining unit is configured to calibrate the spherical coordinate system and the camera unit coordinate system, and determine a transformation relationship between the camera unit coordinate system and the spherical coordinate system;

a spherical coordinate system coordinate solving unit of the object point, configured to calculate a coordinate of the object point in the spherical coordinate system according to the relationship between the object point coordinate and the image point coordinate and the transformation relationship between the camera unit coordinate system and the spherical coordinate system And connecting to the coordinate relationship determining unit and the coordinate system transformation relationship determining unit;

a spherical coordinate solving unit of the object point, configured to calculate a spherical coordinate of the object point projected onto the spherical coordinate system with the radius r according to the coordinate of the object point in the spherical coordinate system, and the spherical coordinate system coordinate of the object point The solution units are connected.

In a third aspect, the present invention provides an imaging system for panoramic stereoscopic images, the imaging system of the panoramic stereoscopic image comprising N imaging units randomly distributed or equally spaced on a circumference of the same circle, wherein N is greater than or equal to A positive integer of 2.

The present invention provides an imaging system for panoramic stereoscopic images, the imaging system of the panoramic stereoscopic image comprising eight imaging units arbitrarily distributed or symmetrically arranged on four sides of the same quadrilateral, the field of view of the imaging unit More than 180 degrees.

The present invention provides an imaging system for panoramic stereoscopic images, the imaging system of the panoramic stereoscopic image comprising four camera units A randomly arranged or equally spaced on the circumference of the same circle, and two backs aligned in an axisymmetric arrangement and The optical axis is connected to the imaging unit B of the center of the plane formed by the four imaging units A and perpendicular to the plane, and the field of view of the imaging unit A is greater than 180 degrees, and the imaging unit B is The field of view is greater than 180 degrees.

The invention provides an imaging system for panoramic stereoscopic images, the imaging system of the panoramic stereoscopic image comprises eight camera units C arranged at an equal or equal interval on the circumference of the same circle, and the two backs are aligned in an axisymmetric setting. And the optical axis is connected to the imaging unit D which is perpendicular to the plane of the plane formed by the eight imaging units C, and the field of view of the imaging unit C is greater than 90 degrees, the imaging unit D The field of view is greater than 180 degrees.

Compared with the prior art, the present invention realizes panoramic stereoscopic image acquisition with a large degree of freedom by means of a combination of multiple camera units, and based on this, solves the true multi-view stereoscopic display of the panoramic image by using the method of multiple panoramic ball pairs. The invention effectively reduces the constraint of the consistency of the parameters of the camera unit and the uniformity of the position arrangement during the construction process of the panoramic photographing device, and greatly improves the degree of freedom and flexibility of the photographing unit.

DRAWINGS

1 is a schematic flowchart of a method for imaging a panoramic stereoscopic image according to a first embodiment of the present invention;

2 is a schematic flow chart of a solution method of the mapping function in FIG. 1;

3 is a schematic structural diagram of an imaging apparatus for panoramic stereoscopic images according to a second embodiment of the present invention;

4 is a schematic structural diagram of a mapping function solving module in FIG. 3;

5 is a schematic diagram showing a mapping relationship between images formed by a single camera unit in an arbitrary spherical coordinate system;

6 is a schematic diagram showing a mapping relationship between overlapping imaging regions of a plurality of imaging units in an arbitrary spherical coordinate system;

FIG. 7 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a third embodiment of the present invention; FIG.

FIG. 8 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a fourth embodiment of the present invention; FIG.

FIG. 9 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a fifth embodiment of the present invention; FIG.

FIG. 10 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a sixth embodiment of the present invention; FIG.

11 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a seventh embodiment of the present invention;

12 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to an eighth embodiment of the present invention;

FIG. 13 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a ninth embodiment of the present invention; FIG.

14 is a schematic diagram showing the positional relationship of a spherical coordinate system pair when the panoramic stereoscopic image forming apparatus of the present invention is viewed from the front;

15 is a schematic diagram showing the positional relationship of a spherical coordinate system pair when the panoramic stereoscopic image forming apparatus of the present invention is turned to the right front;

16 is a schematic diagram showing the positional relationship of a spherical coordinate system pair when the panoramic stereoscopic image forming apparatus of the present invention is turned to the right side;

FIG. 17 is a schematic diagram showing the positional relationship of a spherical coordinate system pair when the panoramic stereoscopic image forming apparatus of the present invention is viewed from an arbitrary viewing angle.

Detailed ways

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The implementation of the present invention will be described in detail below with reference to specific embodiments.

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of a method for imaging a panoramic stereoscopic image according to a first embodiment of the present invention. As shown in FIG. 1, the method for imaging a panoramic stereoscopic image includes the following steps:

Step S20, arbitrarily arranging a plurality of camera units in space according to different application scenarios;

Step S21, calibrating the internal and external parameters of the respective image capturing units that have been arranged;

Step S22, controlling each camera unit to perform cooperative work, and performing synchronous acquisition and transmission of image point coordinates;

Step S23, arbitrarily designating a left-eye spherical coordinate system O _L and a right-eye spherical coordinate system O _R in the space;

Step S24, the spherical coordinate left eye and the right eye O _L O _R spherical coordinate system solver imaging unit to image eye point coordinates are spherical coordinates mapping function O _L and O _R right spherical coordinate system, and according to The solved mapping function respectively generates a left-eye spherical image corresponding to the left-eye spherical coordinate system O _L and a right-eye spherical image corresponding to the right-eye spherical coordinate system O _R ;

In step S25, the left-eye spherical image and the right-eye spherical image are respectively thrown into the left eye and the right eye to complete the stereoscopic display of the panoramic image.

In the first embodiment of the present invention, the distance between the center of the spherical surface of the left-eye spherical image and the center of the spherical image of the right eye is equal to the distance between the human eye and the eye.

Please refer to FIG. 2. FIG. 2 is a schematic flowchart of a solution method of the mapping function in FIG. 1. As shown in FIG. 2, the mapping function solving method includes the following steps:

Step S70, determining the relationship between the object point coordinates and the image point coordinates in the camera unit coordinate system;

Step S71, calibrating the spherical coordinate system and the camera unit coordinate system, and determining a transformation relationship between the camera unit coordinate system and the spherical coordinate system;

Step S72, according to the relationship between the object point coordinates and the image point coordinates, and the camera unit coordinate system and the spherical coordinate The transformation relationship between the systems, and the coordinates of the object point in the spherical coordinate system;

In step S73, the spherical coordinates of the object point projected onto the spherical coordinate system with radius r are solved according to the coordinates of the object point in the spherical coordinate system.

Referring to FIG. 3, FIG. 3 is a schematic structural diagram of an imaging apparatus for a panoramic stereoscopic image according to a second embodiment of the present invention. As shown in FIG. 3, the imaging apparatus of the panoramic stereoscopic image includes:

The camera unit arrangement module 31 is configured to arbitrarily arrange a plurality of camera units in the space according to the unused application scenarios;

The system calibration module 32 is configured to calibrate internal and external parameters of the image capturing unit 31 that have been arranged in the camera unit arrangement module 31, and is connected to the camera unit arrangement module 31;

The hardware communication module 33 is configured to control the camera units of the camera unit arrangement module 31 to perform coordinated operation and perform synchronous acquisition of image point coordinates, and receive image point coordinates collected by each camera unit in the camera unit arrangement module 31 and transmit the image coordinates. It is connected to the camera unit arrangement module 31;

The left-eye and right-eye spherical coordinate system specifying module 34 is configured to arbitrarily designate a left-eye spherical coordinate system O _L and a right-eye spherical coordinate system O _R in the space;

The mapping function solving module 35 is configured to receive the pixel coordinates transmitted by the hardware communication module 33, and specify the left-eye spherical coordinate system O _L and the right-eye spherical coordinate system O _R in the module 34 according to the left-eye and right-eye spherical coordinate systems. Solving the mapping function of the image point coordinates of the camera unit in the camera unit arrangement module 31 to the left-eye spherical coordinate system O _L and the right-eye spherical coordinate system O _R respectively, and respectively generating a left eyeball according to the solved mapping function. a left-eye spherical image corresponding to the surface coordinate system O _L and a right-eye spherical image corresponding to the right-eye spherical coordinate system O _R , which are connected to the hardware communication module 33 and the left-eye and right-eye spherical coordinate system specifying modules 34;

The image display module 36 is configured to cast the left-eye spherical image and the right-eye spherical image in the mapping function solving module 35 into the left eye and the right eye respectively to complete the stereoscopic display of the panoramic image, which is connected to the mapping function solving module 35. .

In the second embodiment of the present invention, the distance between the center of the spherical surface of the left-eye spherical image and the center of the spherical image of the right eye is equal to the distance between the human eye and the eye.

Please refer to FIG. 4. FIG. 4 is a schematic structural diagram of the mapping function solving module in FIG. 3. As shown in FIG. 4, the mapping function solving module includes:

a coordinate relationship determining unit 81, configured to determine a relationship between the object point coordinates and the image point coordinates in the camera unit coordinate system;

The coordinate system transformation relationship determining unit 82 is configured to calibrate the spherical coordinate system and the camera unit coordinate system, and determine a transformation relationship between the camera unit coordinate system and the spherical coordinate system;

The spherical coordinate system coordinate solving unit 83 of the object point is configured to calculate the coordinates of the object point in the spherical coordinate system according to the relationship between the object point coordinates and the image point coordinates and the transformation relationship between the camera unit coordinate system and the spherical coordinate system, and The coordinate relationship determining unit 81 and the coordinate system transformation relationship determining unit 82 are connected;

The spherical coordinate solving unit 84 of the object point is configured to solve the spherical coordinate of the object point projected onto the spherical coordinate system with the radius r according to the coordinates of the object point in the spherical coordinate system, and the spherical coordinate system coordinate solving unit 83 of the object point Connected.

Referring to FIG. 5, FIG. 5 is a schematic diagram of a mapping relationship between an image formed by a single imaging unit and an arbitrary spherical coordinate system. As shown in FIG. 5, the space between the arbitrary spherical coordinate system 501 and the imaging unit 502 of an arbitrary spatial posture is shown. The relationship, as well as the internal parameters of the camera unit, can establish a mapping function of the point coordinates I(x ₁ ', y ₁ ', f), the object distance d to the spherical coordinates, wherein the spherical coordinate system is represented as OXYZ, and the camera unit coordinate system represents For O'X'Y'Z', the internal parameters of the camera unit are recorded as K in the form of vectors, including optical imaging coordinates, focal length, integrated distortion parameters, and lens angle of view.

Figure 5 is a simplified example of a spatial mapping relationship in a two-dimensional plane (OXZ). According to the relative positional relationship between the camera unit coordinate system (O'X'Y'Z') and the spherical coordinate system, the optical center in the plane (OXZ) can be obtained. The coordinate O'(x ₀ , z ₀ ) on the top, and the image coordinate I(x ₁ ', y ₁ ', f) can be converted into the coordinate I(x ₁ , z ₁ ) on the plane (OXZ); The point coordinates P(x ₃ ', y ₃ ', z ₃ ') can be obtained by the imaging unit imaging formula and the object distance d; according to the camera unit coordinate system (O'X'Y'Z') and the spherical coordinate system OXYZ For the spatial relationship, find the coordinate value (R, θ) of the object point P(x ₃ ′, y ₃ ', z ₃ ') in the spherical coordinate system, R is the radius of the sphere, and θ is the vector OP on the OXZ plane. The angle of the X axis. Similarly, the mapping relationship of the above two-dimensional plane can be extended to the mapping relationship of the three-dimensional space.

From the geometric relationship shown in FIG. 5, any image point I(x _i ', y _i ', f) in the camera unit coordinate system can be mapped into the spherical coordinate system, and the mapping function can be expressed as

Where O represents the spherical coordinate OXYZ, O' represents the camera unit coordinate system O'X'Y'Z', and K = (x ₀ ', y ₀ ', f, Δx', Δy') represents the internal parameter vector of the camera unit, wherein (x ₀ ', y ₀ ') represents the coordinates of the optical center on the image plane, f represents the focal length, (Δx', Δy') represents the imaging integrated distortion, d represents the object distance, and (x _i ', y _i ') represents the image Point coordinates, where the object distance d can be manually specified or measured using a distance sensor, and the remaining parameters are obtained by system calibration.

Specifically, the mapping function can be implemented by the following calculation process:

1. The relationship between the object point P(x ₃ ', y ₃ ', z ₃ ') and the image point coordinate I(x _i ', y _i ', f) under the camera unit coordinate system O'X'Y'Z' As shown in Equation 1:

2. Under the premise that O and O' have been calibrated, the transformation relationship between the spherical coordinate system and the camera unit coordinate system is as shown in Equation 2:

O=R×O'+T (Formula 2)

A rotation matrix R of 3 rows and 3 columns and a translation matrix T of 3 rows and 1 column can be obtained by Equation 2.

3. From the results of Equations 1 and 2, the coordinates P(x ₃ , y ₃ , z ₃ ) of the object point P in the spherical coordinate system can be obtained from Equation 3:

4. In summary, the coordinates of the spherical surface of the spherical coordinate system OXYZ projected by the object point P to the radius r can be obtained by Equation 4:

Please refer to FIG. 6. FIG. 6 is a schematic diagram showing the mapping relationship between the overlapping imaging regions of the plurality of imaging units in an arbitrary spherical coordinate system, as shown in FIG. 6, in the overlapping imaging regions of the multiple imaging units, under different camera unit coordinate systems. The object distance d can be obtained by solving the multi-eye stereoscopic relationship. Therefore, in the arbitrary spherical coordinate system 601, according to the spatial relationship between the

imaging units

602 and 603 which are mutually superimposed and imaged, the object point P can be solved relative to the imaging unit 602. According to the analysis of FIG. 3, in the case where the object distance and the image point coordinate of P are known, the point P can be obtained in the spherical coordinate system according to the spatial relationship between the spherical coordinate system OXYZ and the imaging unit coordinate system OXY. The spatial relationship between the coordinate system OXYZ and the camera unit coordinate system O ₁ X ₁ Y, and the coordinates of the point P in the spherical coordinate system are obtained. Therefore, for the two imaging units of the overlapping imaging, the mapping of the object point P to the spherical coordinate system can be performed. Relational representation

K ₁ and K ₂ represent internal parameter vectors of the imaging unit 602 and the imaging unit 603, respectively, (x _i1 , y _i1 ) and (x _i2 , y _i2 ) are images of the object point P in the imaging unit 602 and the imaging unit 603, respectively. Point coordinates, the method does not need to predict the object distance d, and can automatically calculate the spatial three-dimensional coordinates of the object point.

The invention provides an imaging system for panoramic stereoscopic images, comprising N imaging units arranged arbitrarily or equally spaced on the circumference of the same circle, wherein N is a positive integer greater than or equal to 2. Please refer to FIG. 7. FIG. 7 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a third embodiment of the present invention. As shown in FIG. 7, the imaging system of the panoramic stereoscopic image includes two cameras with back alignments arranged in an axisymmetric manner. The unit 41 has an angle of view of the camera unit 41 greater than 180 degrees.

Please refer to FIG. 8. FIG. 8 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a fourth embodiment of the present invention. As shown in FIG. 8, the imaging system of the panoramic stereoscopic image includes three arbitrary distributions on the circumference of the same circle. Or an imaging unit 42 that is equally spaced, the field of view of the imaging unit 42 is greater than 120 degrees.

Please refer to FIG. 9. FIG. 9 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a fifth embodiment of the present invention. As shown in FIG. 9, the imaging system of the panoramic stereoscopic image includes four arbitrary distributions on the circumference of the same circle. Or an imaging unit 43 disposed at equal intervals, the angle of view of the imaging unit 43 being greater than 180 degrees.

Referring to FIG. 10, FIG. 10 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a sixth embodiment of the present invention. As shown in FIG. 10, the imaging system of the panoramic stereoscopic image includes eight randomly distributed on the circumference of the same circle. Or an imaging unit 44 disposed at equal intervals, the angle of view of the imaging unit 44 being greater than 90 degrees.

Please refer to FIG. 11. FIG. 11 is a schematic structural diagram of an imaging system for a panoramic stereoscopic image according to a seventh embodiment of the present invention. As shown in FIG. 11, the imaging system of the panoramic stereoscopic image includes four arbitrary distributions on the circumference of the same circle. Or an equally spaced camera unit A45 and two camera units 46 that are aligned in an axisymmetric manner and whose optical axis is connected through the center of the plane formed by the four camera units A45 and perpendicular to the plane. The angle of view of the imaging unit A45 is greater than 180 degrees, and the angle of view of the imaging unit B46 is greater than 180 degrees.

Referring to FIG. 12, FIG. 12 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to an eighth embodiment of the present invention. As shown in FIG. 12, the imaging system of the panoramic stereoscopic image includes eight randomly distributed on the circumference of the same circle. Or an equally spaced camera unit C47, and two camera units D48 that are aligned in an axisymmetric arrangement and whose optical axis is connected through the center of the plane formed by the eight camera units C47 and perpendicular to the plane, The field of view of the camera unit C47 is greater than 90 degrees, and the field of view of the camera unit D48 is greater than 180 degrees.

The present invention provides an imaging system for another panoramic stereoscopic image, comprising M imaging units randomly distributed on the sides of the same regular polygon or symmetrically arranged to each other, wherein M is a positive integer greater than or equal to 2. Referring to FIG. 13, FIG. 13 is a schematic structural diagram of an imaging system for a panoramic stereoscopic image according to a ninth embodiment of the present invention. As shown in FIG. 13, the imaging system of the panoramic stereoscopic image includes eight arbitrary sides on the same side of the same quadrilateral. The imaging unit 49 is symmetrically disposed or symmetrically arranged, and the field of view of the imaging unit 49 is greater than 180 degrees.

The panoramic stereoscopic image forming method of the present invention can map the object points captured by the panoramic stereoscopic image forming device to a spherical coordinate system of an arbitrary center to form a panoramic sphere of the image, so that the object point can be mapped to a plurality of sets of spherical distances. The method of observing the spherical coordinate system of the pupil distance, simulating the panoramic sphere of the left and right eyes under different viewing angles, and projecting the corresponding left and right eye panoramic sphere images of the spherical coordinate system to the left and right eyes through the split screen display device respectively. Realize stereoscopic display of panoramic images. Please refer to FIG. 14. FIG. 14 is a schematic diagram showing the positional relationship of the spherical coordinate system pair when the panoramic stereoscopic image forming apparatus of the present invention is viewed from the front. As shown in FIG. 14, O _L represents a left-eye panoramic spherical coordinate system, and O _R represents a right-eye panoramic view. In the spherical coordinate system, in order to give the observer a more natural experience, the distance D between the left-eye panoramic spherical coordinate system O _L and the right-eye panoramic spherical coordinate system O _R can be set as the observer's pupil distance according to the demand, and the left-eye panoramic sphere And the radius of the right-eye panoramic sphere is set to the same value R; please refer to FIG. 15, FIG. 15 is a schematic diagram showing the positional relationship of the spherical coordinate system pair when the panoramic stereoscopic image forming apparatus of the present invention is turned to the right front; FIG. 16 is FIG. FIG. 17 is a schematic diagram showing the positional relationship of a spherical coordinate system pair when the panoramic stereoscopic image forming apparatus of the present invention is viewed from an arbitrary viewing angle; FIG. 17 is a schematic diagram showing a positional relationship of a spherical coordinate system pair when the panoramic stereoscopic image forming apparatus of the present invention is turned to the right side; .

The invention realizes the panoramic stereoscopic image acquisition with large degree of freedom by combining multiple camera units, and solves the problem of the real multi-view stereoscopic display of the panoramic image by using the method of multiple panoramic ball pairs, and the invention effectively reduces the problem. The panorama shooting device constrains the consistency of the parameters of each camera unit and the uniformity of position arrangement during the construction process, which greatly improves the freedom and flexibility of the camera unit.

The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. Within the scope.

Claims

A method for imaging a panoramic stereoscopic image, comprising the steps of:

Step A, arbitrarily arranging a plurality of camera units in the space according to different application scenarios;

Step B: calibrating the internal and external parameters of each camera unit in the step A that has been arranged;

Step C, controlling the camera units in each of the step B to perform cooperative work, and performing synchronous acquisition and transmission of image point coordinates;

Step D, arbitrarily designating a left-eye spherical coordinate system O L and a right-eye spherical coordinate system O R in the space;

Step E, according to the left-eye spherical coordinate system O L and the right-eye spherical coordinate system O R in the step D, respectively, the image point coordinates of the imaging unit in the step C are solved to the left-eye spherical coordinate system O L and the right a mapping function of the spherical coordinate system O R , and respectively generating a left-eye spherical image corresponding to the left-eye spherical coordinate system O L and a right-eye spherical image corresponding to the right-eye spherical coordinate system O R according to the solved mapping function;

In step F, the left-eye spherical image and the right-eye spherical image in the step E are respectively thrown into the left eye and the right eye to complete the stereoscopic display of the panoramic image.
The method for imaging a panoramic stereoscopic image according to claim 1, wherein the method for solving the mapping function comprises the following steps:

Step E1, determining the relationship between the object point coordinates and the image point coordinates in the camera unit coordinate system;

Step E2, calibrating the spherical coordinate system and the camera unit coordinate system, and determining a transformation relationship between the camera unit coordinate system and the spherical coordinate system;

Step E3, according to the relationship between the object point coordinates and the image point coordinates in the step E1 and the transformation relationship between the camera unit coordinate system and the spherical coordinate system in the step E2, the coordinates of the object point in the spherical coordinate system are solved;

In step E4, according to the coordinates of the object point in the step E3 in the spherical coordinate system, the spherical coordinates of the object point projected onto the spherical coordinate system with the radius r are solved.
The method for imaging a panoramic stereoscopic image according to claim 1, wherein said left eye The distance between the center of the sphere where the spherical image is located and the center of the spherical image of the right eye is equal to the distance between the eyes of the human eye.
An imaging device for panoramic stereoscopic images, comprising:

The camera unit is arranged to arrange a plurality of camera units in the space according to the unused application scenarios;

a system calibration module, configured to calibrate internal and external parameters of each camera unit that has been arranged in the camera unit arrangement module, and is connected to the camera unit arrangement module;

a hardware communication module, configured to control each camera unit of the camera unit arrangement module to perform cooperative work and perform synchronous acquisition of image point coordinates, receive image point coordinates collected by each camera unit in the camera unit arrangement module, and transmit Connected to the camera unit arrangement module;

The left-eye and right-eye spherical coordinate system specifies a module for arbitrarily designating a left-eye spherical coordinate system O L and a right-eye spherical coordinate system O R in the space;

a mapping function solving module, configured to receive image point coordinates transmitted by the hardware communication module, and specify a left-eye spherical coordinate system O L and a right-eye spherical coordinate system O R according to the left-eye and right-eye spherical coordinate systems respectively Solving a mapping function of the image point coordinates of the camera unit in the camera unit arrangement module to the left-eye spherical coordinate system O L and the right-eye spherical coordinate system O R , and respectively generating a left eyeball according to the solved mapping function a left-eye spherical image corresponding to the surface coordinate system O L and a right-eye spherical image corresponding to the right-eye spherical coordinate system O R , which are connected to the hardware communication module and the left-eye and right-eye spherical coordinate system specifying modules;

An image display module, configured to cast a left-eye spherical image and a right-eye spherical image in the mapping function solving module into the left eye and the right eye respectively to complete stereoscopic display of the panoramic image, which is compared with the mapping function solving module connection.
The apparatus for imaging a panoramic stereoscopic image according to claim 4, wherein the mapping function solving module comprises:

a coordinate relationship determining unit, configured to determine a relationship between the object point coordinates and the image point coordinates in the camera unit coordinate system;

Coordinate system transformation relationship determining unit for calibrating the spherical coordinate system and the camera unit coordinate system, determining a transformation relationship between the camera unit coordinate system and the spherical coordinate system;

a spherical coordinate system coordinate solving unit of the object point, configured to calculate a coordinate of the object point in the spherical coordinate system according to the relationship between the object point coordinate and the image point coordinate and the transformation relationship between the camera unit coordinate system and the spherical coordinate system And connecting to the coordinate relationship determining unit and the coordinate system transformation relationship determining unit;

a spherical coordinate solving unit of the object point, configured to calculate a spherical coordinate of the object point projected onto the spherical coordinate system with the radius r according to the coordinate of the object point in the spherical coordinate system, and the spherical coordinate system coordinate of the object point The solution units are connected.
The panoramic stereoscopic image forming apparatus according to claim 4, wherein a distance between a center of the spherical surface of the left-eye spherical image and a spherical center of the right-eye spherical image is equal to a human eyelid distance.
An imaging system for panoramic stereoscopic images, characterized in that the imaging system of the panoramic stereoscopic image comprises N imaging units randomly distributed or equally spaced on the circumference of the same circle, wherein N is a positive integer greater than or equal to 2 .
An imaging system for panoramic stereoscopic images, characterized in that the imaging system of the panoramic stereoscopic image comprises eight imaging units arbitrarily distributed or symmetrically arranged on four sides of the same quadrilateral, the angle of view of the imaging unit More than 180 degrees.
An imaging system for panoramic stereoscopic images, characterized in that the imaging system of the panoramic stereoscopic image comprises four camera units A randomly arranged or equally spaced on the circumference of the same circle, and two backs aligned in an axisymmetric arrangement and The optical axis is connected to the imaging unit B of the center of the plane formed by the four imaging units A and perpendicular to the plane, and the field of view of the imaging unit A is greater than 180 degrees, and the imaging unit B is The field of view is greater than 180 degrees.
An imaging system for panoramic stereoscopic images, characterized in that the imaging system of the panoramic stereoscopic image comprises eight camera units C arbitrarily distributed or equally spaced on the circumference of the same circle, and the two backs are aligned in an axisymmetric setting And the optical axis is connected to the imaging unit D which is perpendicular to the plane of the plane formed by the eight imaging units C, and the field of view of the imaging unit C is greater than 90 degrees, the imaging unit D The field of view is greater than 180 degrees.