CN102404598B - Image generation system and method for stereoscopic 3D display - Google Patents

Abstract

The invention discloses a system and method for stereoscopic three-dimensional display image generation. Cooperate with the display principle of light field reconstruction and field of view splicing to obtain the image sources of most existing stereoscopic 3D display systems. The generating system includes a two-dimensional display unit array, a lens array, an aperture array, a directional scattering screen, a precision guide rail, an image acquisition system and a computer. The steps of the generating method include: circularly scanning display points; capturing and identifying by an image acquisition system; obtaining mapping coordinate relationships; transforming perspective image sources; moving the image acquisition system to repeat operations; superimposing each image to obtain an image source required for final display. The present invention adopts a flexible and operable image processing method depending on the specific three-dimensional display device, and has the advantage that it can be used to obtain image sources in a stereoscopic three-dimensional display system based on a flat panel display or multi-projection. The system and method comprehensively consider the problem of system imaging aberration and system precision, and can obtain corrected images with relatively low computational complexity.

Description

Image generation system and method for stereoscopic three-dimensional display

technical field

The invention relates to image generation and display methods, in particular to an image generation system and method for stereoscopic three-dimensional display.

Background technique

Image mosaic technology has always been a research hotspot in the field of computer vision and image processing. With the rapid development of 3D panoramic display technology, the image generation system designed in 3D has become the main means of image source acquisition for many 3D display devices. The image generation system designed in a three-dimensional manner reproduces three-dimensional image information on a two-dimensional display system based on a perspective projection drawing method. Most of the existing stereoscopic 3D display devices provide enough viewing angles by splicing the field of view horizontally or vertically, allowing the observer's two eyes to span different viewing angles to obtain a delicate 3D perception. According to this principle, the image information reproduced on the 2D display system is related to the specific viewing angle design parameters of the 3D display device, and often needs to go through an image processing process corresponding to the viewing angle information to be presented. At present, the commonly used processing ideas in the field of computer graphics are based on light field reconstruction and pixel scanning to obtain the image information required for the corresponding viewing angle. This method is quite universal in the computer field, and the images obtained from various perspectives have high resolution and authenticity, but the images generated by this scheme do not consider the actual aberration and system accuracy of the 3D display system, so The generated image is often the image required for imaging under theoretical conditions, and the effect displayed by the projection of the three-dimensional display system is not satisfactory.

Usually, a successful 3D display device needs to comprehensively consider many aspects such as image resolution, 3D display effect, and computing cost. Currently, the stereoscopic 3D display devices that have been developed are limited by factors such as system structure, which often lead to images generated in theoretical conditions. It cannot be perfectly displayed, and it is more practical to explore an image generation system and method that easily takes into account the system imaging aberration and device accuracy. The main purpose of the present invention is to construct a system and method for generating a variety of stereoscopic three-dimensional display images with universality, which includes a projection type three-dimensional display device and an image acquisition and recognition system with high scalability. In consideration of system imaging aberration and device accuracy, the image source required for display under real conditions can be obtained from the perspective of actual projection display of image points, and can be widely used in multi-projection display or time-sharing display splicing principles. Stereoscopic three-dimensional display device.

Contents of the invention

The object of the present invention is to overcome the deficiencies of the prior art and image generation methods, and provide an image generation system and method for stereoscopic three-dimensional display.

The image generation system for stereoscopic three-dimensional display includes a two-dimensional display unit array, a lens array, an aperture array, a directional scattering screen and a precision guide rail arranged in sequence, wherein the directional scattering screen and the precision guide rail are arranged concentrically, and the precise An image acquisition system is installed on the guide rail. The images displayed by all the display units in the two-dimensional display unit array are projected to the other side of the directional scattering screen through the corresponding lens array and aperture array for imaging. The computer is connected with the image acquisition system and the two-dimensional display unit respectively. Arrays are connected. the

The two-dimensional display unit array, lens array and aperture array used in the stereoscopic three-dimensional display image generating system are N*1 arrays of lateral parallax, or N*M arrays of lateral and vertical parallax.

The two-dimensional display unit array used in the stereoscopic three-dimensional display image generation system is a single two-dimensional display or an array of multiple two-dimensional displays, wherein the two-dimensional display is LCD, LCOS, PDP, LED, CRT, OLED or a projector.

The image acquisition system used in the stereoscopic three-dimensional display image generation system is a CCD or CMOS shooting device.

The steps of the method for generating stereoscopic three-dimensional display images are as follows:

1) The image acquisition system is first set on the symmetrical center line of the system, and shoots at the imaging center of the system;

2) The display unit in the two-dimensional display unit array displays a coordinate point (X ₀ , Y ₀ );

3) If the image acquisition system captures the corresponding coordinate point information, it is determined that the point displayed on the display unit belongs to the image to be displayed at the viewing angle, and proceed to step 4); if the image acquisition system does not capture the corresponding coordinate point information, then determine Because the point displayed on the display unit does not belong to the image to be displayed at this viewing angle, return to step 2) cyclically scan and display the next coordinate point;

4) Record the coordinate point information (X ₀ , Y ₀ ) of the captured display point in the two-dimensional image display unit array and the coordinate point information (X ₁ , Y ₁ ) in the image acquisition system, and send them to the computer to generate Corresponding mapping relationship; cyclically scan and record the coordinate point mapping information until the scanning of all display points of the display units in the two-dimensional display unit array ends;

5) The computer obtains the original image of the corresponding viewing angle of the three-dimensional object to be presented, and changes from the coordinate point (X ₁ , Y ₁ ) to the coordinate point (X ₀ , Y ₀ ) according to the mapping relationship, and obtains the corresponding viewing angle of the three-dimensional object to be presented in two Dimension display image in cell array;

6) Move the image acquisition system to the left or right to the next adjacent viewing angle position, wherein the angular interval of the image acquisition system moving along the precision guide rail matches the viewing angle interval of the 3D display system, and repeat steps 1) to 5) , until the computer obtains images of all viewing angles of the three-dimensional object to be presented in the two-dimensional display unit array;

7) The computer superimposes the images in the two-dimensional display unit array of the corresponding viewing angles of all the three-dimensional objects to be presented to generate the final image displayed by the two-dimensional display unit array and sends it to the corresponding display unit.

The original perspective image of the 3D object used in the stereoscopic 3D display image generation method is the image of each perspective of the virtual 3D model captured by computer simulation, or the image of the 3D object actually captured by the camera at the corresponding perspective.

The main advantage of the present invention is that it proposes a system and method for generating a variety of stereoscopic three-dimensional display images with universality, which includes a projection-type three-dimensional display device and an image acquisition and recognition system with high scalability. The original intention is to obtain the image source required for display under real conditions from the perspective of actual projection display of image points, taking into account the problems of system imaging aberration and device accuracy, and can be widely used based on the principle of multi-projection display or time-sharing display stitching Stereoscopic 3D display device.

Description of drawings

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

1 is a schematic diagram of the basic structure of a device for an image display method for stereoscopic three-dimensional display;

Fig. 2 is a flow chart and a schematic diagram of an image generation method for stereoscopic three-dimensional display;

3 is a schematic diagram of the basic structure of a display unit of a three-dimensional display device with 15 horizontal viewing angles;

Fig. 4 and Fig. 5 are schematic diagrams of the image calibration method corresponding to the three-dimensional display device with 15 horizontal viewing angles;

In the figure, a two-dimensional image display unit array 1 , a lens array 2 , an aperture array 3 , a directional scattering screen 4 , a precision guide rail 5 , an image acquisition system 6 , and a computer 7 .

Detailed ways

As shown in Figure 1, the image generation system for stereoscopic three-dimensional display includes a two-dimensional display unit array 1, a lens array 2, an aperture array 3, a directional scattering screen 4 and a precision guide rail 5 arranged in sequence, wherein the directional scattering screen 4 Arranged concentrically with the precision guide rail 5, an image acquisition system 6 is arranged on the precision guide rail 5, and the images displayed by all the display units in the two-dimensional display unit array 1 are projected onto the directional scattering screen 4 through the corresponding lens array 2 and aperture array 3 For imaging on the other side, the computer 7 is connected to the image acquisition system 6 and the two-dimensional display unit array 1 respectively. the

The two-dimensional display unit array 1 , the lens array 2 and the aperture array 3 are an N*1 array of lateral parallax, or an N*M array of lateral and vertical parallax. The two-dimensional display unit array 1 is a single two-dimensional display or an array composed of multiple two-dimensional displays, wherein the two-dimensional display is LCD, LCOS, PDP, LED, CRT, OLED or projector. The image acquisition system 6 is a CCD or CMOS shooting device.

As shown in Figure 2, the steps of the method for generating stereoscopic 3D display images are as follows:

1) The image acquisition system 6 is first set on the symmetrical center line of the system, and is aligned with the imaging center of the system to shoot;

2) The display unit in the two-dimensional display unit array 1 displays a coordinate point (X ₀ , Y ₀ );

3) If the image acquisition system 6 captures the corresponding coordinate point information, it is determined that the point displayed on the display unit belongs to the image to be displayed at the viewing angle, and proceed to step 4); if the image acquisition system 6 does not capture the corresponding coordinate point information, Then it is judged that the point displayed on the display unit does not belong to the image to be displayed at this viewing angle, and returns to step 2) cyclically scans and displays the next coordinate point;

4) Record the coordinate point information (X ₀ , Y ₀ ) of the captured display point in the two-dimensional image display unit array 1 and the coordinate point information (X ₁ , Y ₁ ) in the image acquisition system 6 respectively, and send them to The computer 7 generates the corresponding mapping relationship; circularly scans and records the coordinate point mapping information until the scanning of all display points of the display units in the two-dimensional display unit array 1 ends;

5) The computer 7 obtains the original image of the corresponding viewing angle of the three-dimensional object to be presented, and changes from the coordinate point (X ₁ , Y ₁ ) to the coordinate point (X ₀ , Y ₀ ) according to the mapping relationship, and obtains the corresponding viewing angle of the three-dimensional object to be presented at Two-dimensionally display the images in the cell array 1;

6) Move the image acquisition system 6 to the left or right to the next adjacent viewing angle position, wherein the angular interval of the image acquisition system 6 moving along the precision guide rail 5 matches the viewing angle interval of the three-dimensional display system, and repeat steps 1) to Step 5), until the computer 7 obtains the images of all viewing angles of the three-dimensional object to be presented in the two-dimensional display unit array 1;

7) The computer 7 superimposes the images in the two-dimensional display unit array 1 of the corresponding viewing angles of all the three-dimensional objects to be presented to generate the final image displayed by the two-dimensional display unit array 1 and sends it to the corresponding display unit.

The original perspective image of the three-dimensional object used in the stereoscopic three-dimensional display image generation method is the image of each perspective of the virtual three-dimensional model captured by computer simulation, or the image of the three-dimensional object actually captured by the camera at the corresponding perspective.

Taking an example of an image generated by a field-of-view stitching 3D display device including 15 horizontally stitched images to generate an image of one viewing angle, other viewing angles can be derived by analogy. The system structure is shown in Figure 1. The image display array 1 includes 15 projectors arranged in dislocations. As shown in Figure 3, the so-called projector is composed of a part of the image display array 1, a lens and an aperture. Attached to the front of the lens; the directional scattering screen 2 is arc-shaped, with the system center O as the center, scattering light in the longitudinal direction, and not scattering light in the horizontal direction. The 15 projectors are aligned with the set system center O in the horizontal direction, and aligned with the same height on the diffusion screen 2 in the vertical direction. Then the images of the 15 projectors are all spread out in the longitudinal direction through the longitudinal scattering screen 2, so that a long strip image can be seen on the other side, and the 15 images are just spliced into a whole complete image.

Before generating an image, an image calibration is first required. Firstly, the image acquisition system 6 is placed at the position of the middle viewing angle, such as the position of the acquisition system indicated by the solid line in FIG. 1 . Then, a white point is displayed on the image display array 1 with coordinates (X ₀ , Y ₀ ), and the white point is scanned line by line from the upper left corner of the display array, as shown in Figure 4; Move one pixel to capture a perspective image, and analyze whether there is a white point in the obtained image; if there is, it means that the white point at this position belongs to this perspective, and its coordinates in the acquired perspective image (X ₁ , Y ₁ ) is recorded, as shown in Figure 5, and a mapping relationship is established with the coordinates (X ₀ , Y ₀ ) of the white point in the image display array. In this way, when all the pixels on the image display array 1 have been scanned, the (X ₁ , Y ₁ ) coordinates of all the white points that have appeared in the area captured by the image acquisition system 6 are mapped to the corresponding pixel points on the image display array 1 (X ₀ , Y ₀ ), and present a distribution area, as shown in the dotted box in Figure 5, this area is the image range that the system can display. Such a viewing angle is calibrated.

After completing the image calibration of a viewing angle, set all (X ₀ , Y ₀ ) belonging to the viewing angle to white, and set the rest to black. At this time, the image acquisition system 6 should collect a piece of 15 white vertical bars. White area; the image acquisition system 6 moves to the right along the precision guide rail 5 until a certain vertical bar image completely turns black, then it is considered that the image acquisition system 6 has reached the next viewing angle, and the angle turned is the interval between adjacent viewing angles . Repeat the previous image calibration part for the next viewing angle until all viewing angles are calibrated, and finally the mapping relationship between (X ₁ , Y ₁ ) and corresponding (X ₀ , Y ₀ ) on all viewing angles can be obtained.

After the calibration is completed, the image can be generated according to the obtained mapping relationship without changing the system parameters without re-calibration. The final 3D model or scene to be displayed is obtained according to the viewing angle interval of the calibration part to obtain 2D viewing angle images. Scale the perspective image to the size of the image range that the system can display, and correspond the pixels of the perspective image with (X ₁ , Y ₁ ) on the corresponding perspective, and assign them to (X ₀ , Y ₀ ) according to the mapping relationship. Then all the pixels (X ₀ , Y ₀ ) of the viewing angle are obtained by mapping the viewing angle image of the viewing angle. The rest of the viewing angle operations can be deduced by analogy, and finally the image to be displayed by the entire image display array 1 can be obtained. As long as the finally generated image is displayed on the image display array 1, a three-dimensional model or scene can be seen in the observation area.

Although the present invention is further described here by way of illustration and example, it should be recognized that the present invention is not limited to the above-mentioned embodiments and examples, and the foregoing descriptions are only considered as illustrative rather than restrictive , those skilled in the art can make various changes or modifications, as long as they do not depart from the scope and spirit established in the appended claims, they are all deemed to be within the protection scope of the present invention.

Claims (2)

1. A method for generating stereoscopic three-dimensional display images using a system, characterized in that the system includes a two-dimensional display unit array (1), a lens array (2), an aperture array (3), The directional scattering screen (4) and the precision guide rail (5), wherein the directional scattering screen (4) and the precision guide rail (5) are arranged concentrically, an image acquisition system (6) is arranged on the precision guide rail (5), and the two-dimensional display The images displayed by all the display units in the unit array (1) are projected to the other side of the directional scattering screen (4) for imaging through the corresponding lens array (2) and aperture array (3), and the computer (7) communicates with the image acquisition system ( 6) connected with the two-dimensional display unit array (1); Its steps are as follows: 1) The image acquisition system (6) is first set on the symmetrical center line of the system, and shoots at the imaging center of the system; 2) The display unit in the two-dimensional display unit array (1) displays a coordinate point (X ₀ , Y ₀ ); 3) If the image acquisition system (6) captures the corresponding coordinate point information, it is determined that the point displayed on the display unit belongs to the image to be displayed at this viewing angle, and proceed to step 4); if the image acquisition system (6) does not capture the corresponding coordinate point information coordinate point information, it is determined that the point displayed on the display unit does not belong to the image to be displayed at this viewing angle, and returns to step 2) cyclically scans and displays the next coordinate point; 4) Record the coordinate point information (X ₀ , Y ₀ ) of the captured display point in the two-dimensional image display unit array (1) and the coordinate point information (X ₁ , Y ₁ ) in the image acquisition system (6) ), send it to the computer (7) to generate the corresponding mapping relationship; circularly scan and record the coordinate point mapping information until the scanning of all display points of the display unit in the two-dimensional display unit array (1) ends; 5) The computer (7) obtains the original image of the corresponding angle of view of the three-dimensional object to be presented, and changes from the coordinate point (X ₁ , Y ₁ ) to the coordinate point (X ₀ , Y ₀ ) according to the mapping relationship, and obtains the corresponding image of the three-dimensional object to be presented. The image of the viewing angle in the two-dimensional display cell array (1); 6) Move the image acquisition system (6) to the left or right to the next adjacent viewing angle position, wherein the angular interval of the image acquisition system (6) moving along the precision guide rail (5) matches the viewing angle interval of the three-dimensional display system , repeating step 1) to step 5), until the computer (7) obtains images of all viewing angles of the three-dimensional object to be presented in the two-dimensional display unit array (1); 7) The computer (7) superimposes the images in the two-dimensional display unit array (1) of the corresponding viewing angles of all the three-dimensional objects to be presented to generate the final image displayed by the two-dimensional display unit array (1) and sends it to the corresponding display unit. 2. A method for generating images for stereoscopic three-dimensional display according to claim 1, characterized in that the original perspective images of the three-dimensional objects are images of various perspectives of the virtual three-dimensional model taken by computer simulation, or the corresponding images are captured by the camera The perspective actually captures the image of the three-dimensional object.

Images