WO2018137495A1

WO2018137495A1 - Light-field panoramic camera

Info

Publication number: WO2018137495A1
Application number: PCT/CN2018/072035
Authority: WO
Inventors: 虞晶怡; 曹煊
Original assignee: 叠境数字科技（上海）有限公司
Priority date: 2017-01-24
Filing date: 2018-01-10
Publication date: 2018-08-02
Also published as: CN107450265A

Abstract

A light-field panoramic camera, comprising N light-field cameras (1), wherein N≥2. The light-field cameras (1) are uniformly distributed and fixed on a circular support (2). The photographing angle of each light-field camera (1) is FOV_Cam, and the overlap angle between light-field images acquired by two adjacent light-field cameras (1) is FOV_Overlap. A panoramic light-field image captured by the light-field panoramic camera allows refocusing at any viewing angle within 360°, so that the eyes can selectively focus on any object when viewing the panoramic image, and the eyes can obtain a more realistic panoramic viewing effect.

Description

Light field panoramic camera

Technical field

The invention relates to the field of panoramic camera imaging technology, in particular to a light field panoramic camera.

Background technique

The existing panoramic camera can only capture 360° 2D images, that is, only the color and brightness information of the light can be collected, and the direction information of the light cannot be collected. Therefore, existing panoramic cameras can only provide viewing of "fixed" focused images, but not "refocus".

Referring to Figures 1a and 1b, the 360° panorama seen by both eyes when using an existing panoramic camera can only be focused on a cylindrical surface of a fixed radius. While the human eye is watching the three-dimensional world, the ciliary muscles can relax and contract, thereby focusing on different objects at different distances. Referring to Figures 2a and 2b, the human eye can choose to focus on a cylindrical surface of different radii. When focusing on a distant (ie, a cylindrical surface with a larger radius), the ciliary muscle relaxes and focuses in the vicinity (ie, the radius is smaller). On the cylindrical surface, the ciliary muscles are tight. The existing panoramic camera loses the direction information of the light, and the human eye can only see the fixed focus image, and cannot refocus. Since this does not conform to the visual characteristics of the human eye to view the three-dimensional world, the visual experience of viewing the panorama by the human eye is reduced. Therefore, there is an urgent need to invent a panoramic camera that allows a human eye to see a refocusable image.

Summary of the invention

It is an object of the present invention to provide a light field panoramic camera capable of capturing a light field in a range of 360°, which can simultaneously collect information on the color, brightness and light direction of the light, thereby realizing refocusing at any viewpoint of the panorama.

The technical solution adopted by the present invention includes N light field cameras, and N≥2, the light field cameras are evenly distributed and fixed on a circular support, and each light field camera takes an angle of FOV _Cam , two adjacent The light field image acquired by the light field camera overlaps at an angle of FOV _Overlap ; the need to satisfy: combine the color, brightness and light direction of the light collected in the FOV _Cam of the adjacent two light field camera angles into a larger light. Field image; splicing the light field image acquired by N light field cameras can obtain the color, brightness and light direction information of the light in the 360° panoramic range, and refocusing can be realized at any viewpoint within 360°. Wherein, the light field camera comprises a combination of a conventional camera and a microlens array, as shown in FIG. 3, the microlens array is embedded between the main lens overlap factor and the imaging sensor.

Compared with the prior art, the present invention has the following advantages:

1) The overlapping angles of two adjacent light field cameras change dynamically with the number of cameras. When the number of light field cameras is small, considering the shooting angle of a single light field camera is large, the imaging distortion is serious, so two adjacent The light field camera captures a large overlap angle to ensure that the light field images captured by two adjacent light field cameras can be correctly stitched together.

2) Not only the brightness and color information of the light collected by the adjacent two light field cameras are fused, but also the direction information of the light is fused, so that the spliced image also contains the color, brightness and light direction information.

3) By splicing N light field cameras, the simultaneous acquisition of the color, brightness and light direction information of the light in the 360° range is achieved, so that the focus can be refocused at any angle within the 360° range.

DRAWINGS

Figure 1a is a schematic view (top view) of only one fixed focus plane when the human eye views a conventional panorama;

Figure 1b is a schematic view (side view) of only one fixed focus plane when the human eye views a conventional panorama;

2a is a schematic view (top view) of refocusing at different distances in the distance when the human eye views the panoramic light field map collected by the present invention;

2b is a schematic view (side view) of refocusing at different distances in the distance when the human eye views the panoramic light field map collected by the present invention;

Figure 3 is a schematic structural diagram of a light field camera;

4 is a schematic structural view of a first embodiment of the present invention (when N=2);

Figure 5 is a schematic structural view of a second embodiment of the present invention (when N=3);

6 is a schematic structural view of a third embodiment of the present invention (N=4);

FIG. 7 is a schematic structural view of a fourth embodiment of the present invention (when N=8).

detailed description

The traditional panoramic camera can only capture 360° panoramic 2D images, only contains the color and brightness information of the light within 360°, but lacks the direction information of the light; therefore, the traditional panoramic camera cannot achieve refocusing.

The present invention will be further described in detail below with reference to the specific embodiments of the invention.

Referring to FIG. 3, the light field camera 001 of the embodiment is modified from a conventional camera by embedding a microlens array 102 between the main lens 101 of the conventional camera and the imaging sensor 103. The following embodiments of the present invention all adopt the light field camera 001 modified in this manner. Embodiments when N=2, N=3, N=4, and N=8 are respectively described below, and the overlap factor λ=0.3 between adjacent two cameras in all embodiments.

It should be specially pointed out that the angle of each light field camera is FOV _Cam , and the angle of the light field image collected by two adjacent light field cameras overlaps with FOV _Overlap ;

Where λ is the overlap factor and λ ≥ 0; (the camera shooting range angle can not be any angle, but is determined when the camera and the lens are purchased, so the lower limit of the FOV _Cam can be specified). The overlap factor λ is an artificially set parameter whose magnitude reflects the overlap angle of two adjacent cameras. If λ is equal to zero, 360° is equally divided into N equal parts, camera shooting range angle

Yes, it is not required to overlap the shooting range of adjacent cameras. In this case, the panoramic camera of the light field can also be formed, but the stitched panorama has a large dead zone (ie, the area where the camera cannot capture images). As the overlap factor λ increases, the overlapping area of the image captured by the adjacent camera increases, and the dead zone of the panorama decreases, but the camera is required to have a larger shooting angle, which causes the camera distortion to increase. Therefore, the value of λ needs to be reasonably determined according to the angle of the shooting range of the light field camera. Generally, the value of λ is between [0.2, 0.4]. There is no upper limit for the value of the overlap factor, and the lower limit is zero, the empirical value is between [0.2, 0.4], and the embodiment takes 0.3, which is an example sufficient to support the content of the claims.

Embodiment 1

Referring to FIG. 4, when N=2, only two light field cameras 1 are fixed back to one bracket 2, and the normal lines of the two light field camera lenses coincide. According to the formula

The shooting angle of the single light field camera FOV _Cam ≥ 234°, the FOV of the light field camera selected in this embodiment is 240°, which satisfies the above requirements. Each light field camera simultaneously collects information on the color, brightness and light direction of the light at an angle of 240°, and stores the three aspects of the collected information as a light field image. The light field images captured by the two light field cameras partially overlap, and the light field images of the overlapping portions are fused, that is, the color, brightness and light direction of the overlapping portions of the light are merged, thereby obtaining a 360° panoramic light field image.

Embodiment 2

Referring to FIG. 5, when N=3, the three light field cameras 1 are evenly distributed and fixed on a circular bracket 2, and the normal lines of the adjacent two light field camera 1 lenses are 120°, according to the formula.

The shooting angle of the single light field camera 1 is FOV _Cam ≥ 156°, and the FOV of the light field camera 1 selected in this embodiment is 160°, which satisfies the above requirements. Each light field camera simultaneously collects information on the color, brightness and light direction of the light at an angle of 160°, and stores the three aspects of the collected information as a light field image. The light field images captured by any two adjacent light field cameras partially overlap, and the light field images captured by all the adjacent two light field cameras are spliced, thereby obtaining a 360° panoramic light field image.

Embodiment 3

Referring to FIG. 6, when N=4, the four light field cameras 1 are uniformly distributed and fixed on a circular bracket 2, and the normal lines of the adjacent two light field camera lenses are 90°, according to the formula.

The shooting angle of the single light field camera 1 is FOV _Cam ≥ 117°, and the FOV of the light field camera 1 selected in this embodiment is 120°, which satisfies the above requirements. Each light field camera 1 simultaneously collects information on the color, brightness and light direction of the light at an angle of 120°, and stores the three aspects of the collected information as a light field image. The light field images captured by any two adjacent light field cameras partially overlap, and the light field images captured by all the adjacent two light field cameras are spliced, thereby obtaining a 360° panoramic light field image.

Embodiment 4

Referring to FIG. 7, when N=8, the eight light field cameras 1 are uniformly distributed and fixed on a circular support 2, and the normal lines of the adjacent two light field camera 1 lenses are 45°. According to the formula

The shooting angle of the single light field camera is FOV _Cam ≥ 58.5°, and the FOV of the light field camera selected in this embodiment is 60°, which satisfies the above requirements. Each light field camera simultaneously collects information on the color, brightness and light direction of the light at an angle of 60°, and stores the three aspects of the collected information as a light field image. The light field images captured by any two adjacent light field cameras partially overlap, and the light field images captured by all the adjacent two light field cameras are spliced, thereby obtaining a 360° panoramic light field image.

In the first embodiment, since the number of light field cameras is small (N = 2), it is required that each light field camera has a large angle of shooting range. If the distortion of the 240° camera cannot be accepted, or if the price of the 240° lens is expensive, the overlap factor can be reduced, so that λ=0.2, the shooting angle requirement of the single light field camera is relaxed to FOV _Cam ≥216°) . Similarly, in other embodiments, when a light field camera with a large shooting angle is unwilling to be used due to price or distortion, etc., the overlap factor λ can be appropriately reduced to reduce the requirement for the light field camera FOV _Cam .

For the fourth embodiment, the number of light field cameras is relatively large, and only the light field camera FOV _Cam ≥ 58.5° is needed. In the fourth embodiment, the FOV _Cam = 60° light field camera is used, the distortion is small, and the lens price is moderate, so λ = 0.3 can be maintained, and the value of λ can be further increased.

The above is an example in which N=2, N=3, N=4, and N=8, respectively. It can be seen that the case of the present invention is special when N=2 and N=3, so the first embodiment is listed separately. Box embodiment 2; and when N ≥ 4, the embodiment of the present invention has certain universality, so the embodiment of the fourth embodiment when N=8 is added while exemplifying the third embodiment. It will be readily apparent to those skilled in the art that when N>4 is required to adopt other values, it can be reasoned with reference to the above-described third and fourth embodiments without creative labor.

In summary, compared with the conventional panoramic camera, the beneficial effects of the present invention are mainly: collecting the color, brightness and light direction information of the light in the 360° range, and stitching the images of all the adjacent light field cameras by the feature point registration. The linear transition fusion method is used to fuse the images of the overlapping regions, and finally a panoramic light field map is obtained, which can be refocused at any viewpoint within 360°. When the eye is viewing the panorama, it can be selectively focused on any object, and the human eye can obtain a more realistic panoramic viewing effect.

The embodiments of the present invention are described above with reference to the accompanying drawings and embodiments, which are not to be construed as limiting the scope of the invention. Various modifications or modifications are made within the scope of protection.

Claims

A light field panoramic camera, comprising: N light field cameras, and N≥2, the light field cameras are evenly distributed and fixed on a circular support, and each light field camera takes an angle of FOV Cam The angle of the image of the light field collected by two adjacent light field cameras is FOV Overlap ;

FOV Overlap ≥[(360°/N)×λ], λ is the overlap factor and λ≥0; the color, brightness and light direction of the light collected in the FOV Cam of the adjacent two light field camera angles are combined into one Larger light field image; splicing the light field image acquired by N light field cameras to obtain the color, brightness and light direction information of the light in the 360° panoramic range, and refocusing can be achieved at any viewpoint within 360°.
A light field panoramic camera according to claim 1, wherein said light field camera comprises a combination of microlens arrays embedded between the main lens and the imaging sensor.