WO2019014845A1

WO2019014845A1 - Prism-based light field synthesis method

Info

Publication number: WO2019014845A1
Application number: PCT/CN2017/093348
Authority: WO
Inventors: 李乔
Original assignee: 辛特科技有限公司; 李乔
Priority date: 2017-07-18
Filing date: 2017-07-18
Publication date: 2019-01-24
Also published as: CN111194430B; CN111194430A

Abstract

A prism-based light field synthesis method, comprising: (a) arranging a plurality of projection heads in an array to form projection groups (401), and arranging the plurality of projection groups (401) in an array to form a projection wall (400), a group of isosceles right triangle prisms (402) being disposed in front of the projection wall (400), and the camera wall (400) acquiring complete spatial image information collected by a camera wall (100); (b) the plurality of projection groups (401) in the projection wall (400) playing back images having different viewing angles respectively, each projection head in the same projection group (401) simultaneously playing back images having different spatial depths at the same viewing angle, and images played back by a plurality of projection heads in the same projection group (401) forming a spatial depth image comprising depth information at the same viewing angle; and (c) the spatial depth images at different viewing angles that are played back by the plurality of projection groups (401) being synthesized into an entire light field. With the present invention, an entire light field may be quickly and completely restored, and the light field may be restored at different viewing angles and different depths of a spatial area.

Description

Method for synthesizing light field based on prism

Technical field

The invention relates to the technical field of light field synthesis, in particular to a method for synthesizing a light field based on a prism.

Background technique

Almost any 3D imaging technology has been developed based on this principle of polarization. In 1839, the British scientist Winster discovered a wonderful phenomenon. The distance between the eyes of a person is about 5cm (European average). When looking at any object, the angles of the two eyes do not coincide, that is, there are two angles of view. This subtle difference in viewing angles is transmitted to the brain through the retina, which distinguishes the front and back of the object and produces a strong three-dimensional effect. This is the principle of polarization, and almost any 3D imaging technology has been developed based on this principle so far.

However, 3D equipment based on the principle of "polarization" cannot solve the dizziness problem caused by people during use. In the natural environment, the left and right parallax and the eye focus system can confirm each other, so that the brain knows that these two functions are tacit cooperation. When the user is watching the 3D image based on the "polarization principle", due to the lack of the participation of the eye focus system, the two sets of the brain's distance sensing system and the observation in the natural environment are different, and this difference makes the brain very uncomfortable. The sense of vertigo is created.

In order to solve the problem of vertigo in 3D video, the industry has introduced a light field theory solution. One of the more representative companies in the field of 3D photography is Lytro. Lytro's light field camera uses a microlens array method to solve the position and direction of recording a single light, but the lens array method currently has several major drawbacks: large pixel loss, resolution of 40 million pixel light field photos. The normal photo output resolution is only 2450×1634 (about 4 million pixels); the speed is slow, because the amount of data recorded by a single photo is as high as 50M, when the camera is too fast, the storage operation of the camera can not keep up, each picture It takes a few seconds to load.

A representative company in the field of 3D broadcasting is a solution based on light field theory made by Magic Leap, but this solution uses optical fiber scanning technology to realize light field display, fiber Due to the control of the rotation, angle and illumination of the optical fiber, it has certain difficulties in control. In addition, the multi-focus display method proposed by Magic Leap uses an eye detection system to detect the eye observation point and then re-render the picture, adjust the picture projected to the eye, and each time an image of the depth information is cast, it is difficult to completely restore the entire light field. It is difficult to perform light field restoration from different space schedules.

Therefore, in order to solve the above problems, there is a need for a prism-based synthetic light field capable of rapidly and completely reducing the entire light field from different spatial angles.

Summary of the invention

It is an object of the present invention to provide a method of synthesizing a light field based on a prism, the method comprising:

a) forming a projection group by a plurality of projection head arrays, the plurality of projection groups being arrayed into a projection wall, and arranging a set of isosceles right angle prisms in front of each of the projection groups, the projection wall acquiring the complete collection by the camera wall Spatial image information;

b) a plurality of the projection groups in the projection wall respectively play images of different viewing angles, and each of the projection heads in the same projection simultaneously plays images of different spatial depths under the same viewing angle, and the same projection group The images played by the plurality of the projection heads are combined to form a spatial depth image of the same view space region;

c) spatially deep images of different viewing angles played by a plurality of said projection groups synthesize the entire light field.

Preferably, the isosceles right angle prisms are arranged in the same direction.

Preferably, each of the projection heads in the same projection group plays an image of different spatial depths at the same viewing angle and is projected onto a right angle side of the triangular prism.

Preferably, the complete spatial image information is collected by:

A1) forming a camera group by a plurality of camera arrays, wherein the plurality of camera groups are arrayed into a camera wall, and the plurality of cameras in the same camera group have different focal lengths;

A2) a plurality of the camera groups of the camera wall collect image information of different viewing angles, and a plurality of the cameras in the same camera group collect image information of different spatial depths in the same viewing angle;

A3) the camera sends the collected image information to the image processing computer, and the image processing computer performs denoising processing and image information verification on the collected image information, Complete spatial image information.

Preferably, the spatial image information collected by each of the camera groups is in one-to-one correspondence with the spatial image played by each of the projection groups.

Preferably, the camera wall is arrayed by a plurality of the camera groups on a planar or spherical base.

Preferably, the unfocused portion of the acquired image information is removed by a denoising process. The invention provides a method for synthesizing a light field based on a prism, and the image information collected by the camera wall of the array and the image information of different spatial depths is played by the plurality of projection heads through different spatial depths. It can quickly and completely restore the entire light field, and can realize the light field reduction of different depths of different viewing angles in the space region.

It is to be understood that the foregoing general descriptions

DRAWINGS

Further objects, features, and advantages of the present invention will be made apparent by the following description of the embodiments of the invention.

1a to 1b are schematic views showing a light field collecting device of the present invention;

2 is a flow chart showing the light field collection method of the present invention;

3 is a schematic view showing the image of the camera wall of the present invention collecting different viewing angles;

4 is a schematic diagram showing an image of a corresponding spatial region of a camera group in an embodiment of the present invention;

FIG. 5 is a schematic diagram showing the same camera group collecting different spatial depth images according to the present invention; FIG.

Figure 6 is a flow chart showing the synthesis of a light field based on a prism according to the present invention;

7 is a schematic view showing the projection wall of the present invention playing different perspective images;

Figure 8 is a schematic diagram showing the projection head of the present invention playing an image having different spatial depths onto a synthetic light field on a prism.

Detailed ways

The objects and functions of the present invention and the use thereof are achieved by reference to exemplary embodiments. And functional methods will be clarified. However, the invention is not limited to the exemplary embodiments disclosed below; it can be implemented in various forms. The essence of the description is merely to assist those skilled in the relevant art to understand the specific details of the invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the figures, the same reference numerals are used to refer to the same or similar parts, or the same or similar steps, unless otherwise stated.

The content of the present invention will be described in detail below through specific embodiments with reference to the accompanying drawings. In order to clearly explain the light field reduction method of the present invention, the process of the light field collection of the present invention is first explained. 1a-1b, a schematic diagram of a light field collecting device according to an embodiment of the present invention includes a convex spherical base, a camera wall 100 mounted on the outer side of the spherical base, and In the image processing computer, in this embodiment, a spherical base is used, and the camera wall 100 is mounted on the outer side 100b of the spherical base to enable the camera wall 100 to collect image information of the spatial area in all directions. In some embodiments, the base for mounting the camera wall 100 can be planar or spherical. This embodiment preferably employs a spherical base.

The camera wall 100 includes a plurality of camera groups 110 of an array, the camera group 110 includes a plurality of cameras 111 of an array, and a plurality of cameras 111 in the same camera group 110 are closely arranged, and each camera can acquire the same viewing angle. The complete image information of the spatial region, the plurality of cameras 111 within the same camera group 110 have different focal lengths. The camera 111 realizes data transmission with the image processing computer on the inner side 100a of the spherical base. Specifically, the data transmission may be a wired connection for data transmission or a wireless data transmission.

The plurality of camera groups 110 of the camera wall 100 are used to collect image information of different viewing angles. The plurality of camera heads 111 in the same camera group 110 are used to collect image information of different spatial depths in the same viewing angle, and the image processing computer collects image information. Perform denoising processing and image information verification.

As shown in FIG. 2, the method for collecting the complete spatial image information disclosed in the present invention includes:

S101: acquiring image information, wherein the plurality of light field camera arrays are formed into a camera group, the plurality of camera groups are arrayed into a camera wall, and the plurality of cameras in the same camera group have different focal lengths, and the camera wall is arrayed by the plurality of camera groups. Outside the spherical base, it is used to capture image information of the space area.

Among them, multiple camera groups of the camera wall collect image information of different viewing angles, the same photo A plurality of cameras in the head group collect image information of different spatial depths in the same viewing angle. As shown in FIG. 3, the camera wall of the present invention collects images of different viewing angles. The camera wall 100 is mounted on the outer side of the spherical base. Different camera groups 110 collect image information of different viewing angles. In the embodiment, the adjacent three camera groups respectively collect image information of the space area A, the space area B, and the space area C. The spatial area acquired between adjacent camera sets should have overlapping portions to ensure the integrity of the acquired spatial image information.

A plurality of cameras in the same camera group are closely arranged, and each camera can acquire complete image information of the same viewing space area. FIG. 4 is a schematic diagram showing an image of a corresponding spatial region of a camera group according to an embodiment of the present invention. FIG. 5 is a schematic diagram of capturing a different spatial depth image of the same camera group according to the present invention. The plurality of cameras in the same camera group simultaneously collect image information of different spatial depths. In this embodiment, the plurality of cameras in the same camera group use different image distances to acquire image information of different spatial depths at the same focal length. In some embodiments, multiple cameras in the same camera group can acquire image information of different spatial depths with different focal lengths and the same image distance.

In this embodiment, the spatial region A corresponding to the mth camera group is taken as an example, and the image information of the spatial region A corresponding to the collected spatial region A of the camera group includes a first image (puppy) 201, a second image 202 (tree), and The third image (sun) 203, the first image (puppy) 201 is closest to the camera wall, the second image (tree) is 202, and the third image (sun) 203 is the farthest from the camera wall. The plurality of cameras of the mth camera group array respectively collect image information of different spatial depths. According to the light field collection method disclosed by the present invention, multiple cameras of the same camera group use different focal lengths, and image information with different spatial depths is always Focus imaging in a camera.

The following is an exemplary description. The first camera in the mth camera group focuses the first image (puppy) 201, and the first image (puppy) 201 is clearly imaged in the image information captured by the first camera. The second image 202 (tree) and the third image (sun) 203 are blurred. Similarly, in the image information captured by the second camera, the second image (tree) 202 is clearly imaged, and the first image 201 (puppy) and the third image (sun) 203 are blurred; the nth camera captures In the image information, the third image (sun) 203 is clearly imaged, and the first image 201 (puppy) and the second image (tree) 202 are blurredly imaged. It should be understood that each image in the embodiment also has a different spatial depth, and the depth of different spaces of the same image is The cameras respectively acquire image information of different spatial depths. For example, for the first image (puppy) 201, the eye of the puppy of the camera wall is closer to the camera wall than the camera wall, and the tail of the puppy is farther from the camera wall, and the first image is respectively acquired by the cameras with different focal lengths ( Puppy) 201 spatial depth image information. Through the acquisition of multiple cameras, the image information of the spatial depth of the complete spatial space is collected.

S102: Image information denoising processing, and each camera of the camera wall sends the collected image information to the image processing computer. Since each camera only focuses on image information having a certain depth of space, the image information collected by each camera has only a single focus point, and the other unfocused portions are denoised, and the collected information is removed by denoising processing. The portion of the image information that is not in focus. The method for denoising is denoised according to the prior art of those skilled in the art, and the denoising process is preferably performed by a method of map.

S103: Image information verification, and performing image information verification on the image information collected by each camera after the denoising process, thereby ensuring that the image information collected by each camera has only one focus point.

S104, spatial image collection, image information collected by multiple cameras in the same camera group is combined with A region image information with full spatial depth, and multiple camera groups in the camera wall synthesize image information of different viewing angles with complete space Deep spatial imagery for complete light field acquisition.

According to the method for synthesizing a light field based on a prism provided by the present invention, the following describes the process of synthesizing the light field in the embodiment, and the flow chart of the prism-based synthetic light field of the present invention is shown in FIG. Methods of synthesizing the light field include:

S301, the projection wall acquires spatial image information with complete depth information, and the plurality of projection head arrays are formed into a projection group, and the plurality of projection groups are arrayed into a projection wall, and a set of isosceles right angle prisms are arranged in front of each projection group, and the projection wall is acquired. Complete spatial imagery information captured by the camera wall.

S302, respectively playing images of different viewing angles, as shown in FIG. 7 , the projection wall of the present invention plays a different perspective image, the projection wall 400 arrays the plurality of projection groups 401, and the plurality of projection groups 401 in the projection wall 400 respectively play different viewing angles. The image, for example, image information of the A area, the B area, and the C area collected by three adjacent camera groups, plays the images A', B', and C' through three adjacent projection groups.

S303. Simultaneously playing images of different spatial depths under the same viewing angle, according to the present invention For example, each projection group 401 is arranged in front of a set of right angle triangular prisms 402, and each isosceles right angle prism is arranged in the same direction. The image information of the A region, the B region, and the C region is restored to the light fields A", B", and C" by the triangular prism group 402.

S304. The plurality of projection heads in the same projection group form a spatial depth image of the same view space region through the triangular prism group, and the present invention is described below by taking a reduction process of the spatial image information of the A region as an example. As shown in FIG. 8 , the projection head of the present invention plays a picture with different spatial depths to the synthesized light field on the prism, and the image information of different spatial depths collected by the space area A corresponding to the mth camera group includes the first image. (puppy), second image (tree) and third image (sun).

An isosceles right angle prism set 402 is arranged in front of the projection head group 401 for playing the A area image, and each of the projection heads of the projection group playing the A area image simultaneously plays the first image (puppies) 201a of different spatial depths in the same viewing angle. The second image (tree) 202a and the third image (sun) 203a are projected onto a right-angled side of the isosceles right-angled prism, and are reflected by the triangular prism group 402 to restore the spatial image of different depths of the A region to A".

S305: Synthesize the entire light field, and spatially deep images of different viewing angles played by the plurality of projection groups are combined to synthesize the entire light field.

The invention provides a method for synthesizing a light field based on a prism, and the image information collected by the camera wall of the array and the image information of different spatial depths is played by the plurality of projection heads through different spatial depths. The entire light field can be restored completely and quickly.

Other embodiments of the invention will be apparent to those skilled in the <RTIgt; The description and the examples are to be considered as illustrative only, and the true scope and spirit of the invention are defined by the claims.

Claims

A method for synthesizing a light field based on a prism, characterized in that the method comprises:

a) forming a projection group by a plurality of projection head arrays, the plurality of projection groups being arrayed into a projection wall, and arranging a set of isosceles right angle prisms in front of each of the projection groups, the projection wall acquiring the complete collection by the camera wall Spatial image information;

b) a plurality of the projection groups in the projection wall respectively play images of different viewing angles, and each of the projection heads in the same projection group simultaneously play images of different spatial depths in the same viewing angle, in the same projection group The images played by the plurality of projection heads constitute a spatial depth image of the same viewing space area;

c) spatially deep images of different viewing angles played by a plurality of said projection groups synthesize the entire light field.
The method of claim 1 wherein said isosceles right angle prisms are arranged in the same direction.
The method according to claim 1, wherein each of the projection heads in the same projection group plays an image of different spatial depths at the same viewing angle and projects onto a right angle side of the triangular prism.
The method of claim 1 wherein said complete spatial image information is acquired by:

A1) forming a camera group by a plurality of camera arrays, wherein the plurality of camera groups are arrayed into a camera wall, and the plurality of cameras in the same camera group have different focal lengths;

A2) the plurality of camera groups of the camera wall collect image information of different viewing angles, and the plurality of the cameras in the same camera group collect image information of different spatial depths in the same viewing angle;

A3) The camera sends the collected image information to the image processing computer, and the image processing computer performs denoising processing and image information verification on the collected image information to obtain complete spatial image information.
The method according to claim 4, wherein the spatial image information acquired by each of the camera groups is in one-to-one correspondence with the spatial image played by each of the projection groups.
The method of claim 4 wherein said camera wall is comprised of a plurality The camera set is arrayed on a planar or spherical base.
The method according to claim 4, wherein the unfocused portion of the acquired image information is removed by a denoising process.