WO2016108339A1 - Method for implementing display system capable of switching between three-dimensional and two-dimensional images in integral imaging using mask panel - Google Patents

Abstract

The present invention relates to a method for implementing a display system capable of switching between three-dimensional and two-dimensional images in integral imaging using a mask panel and, more particularly, to a method for implementing a display system capable of switching between three-dimensional and two-dimensional images in integral imaging using a mask panel, in which elemental images stored in an image acquisition device are successively projected through a lens array and a mask panel onto a space and displayed as a three-dimensional image by a display device, wherein, when the display device displays white color and a two-dimensional image is displayed on the mask panel, an observer can observe only the two-dimensional image.

Description

A method for implementing a display system capable of converting an integrated image using a mask panel into a 3D and 2D image

The present invention relates to a method for implementing a display system that can be switched to three-dimensional and two-dimensional images in an integrated image using a mask panel, and more particularly, to selectively express two-dimensional and three-dimensional images using a mask panel. Alternatively, the present invention relates to a method for implementing a display system capable of converting from an integrated image using a mask panel capable of simultaneously displaying a 2D image and a 3D image to 3D and 2D images.

Glasses-free three-dimensional display device is considered as one of the next generation display technology that can lead the new market of the display industry.

However, there are no glasses-free three-dimensional display devices that have been successfully commercialized yet, but various technologies are being developed.

Among these technologies, integrated imaging has recently attracted attention.

This technique was first proposed by Professor Lippman in 1908, but did not evolve significantly due to the limitation of the resolution of the display device.

This integrated imaging technology is a method of making a 3D image by using a lens array in addition to a conventional display device.

Therefore, the structure of the system is simple and there is an advantage that can easily create a three-dimensional color image.

For practical application of integrated imaging technology, integrated imaging technology having a 3D / 2D conversion function is required.

Recently, various studies for this have been attempted.

Various devices are additionally used for this 3D / 2D conversion function.

For example, there is an active pinhole array, an LED array, or the like.

However, this method adds complex devices to the existing integrated image display system to add two-dimensional functionality.

1 is a schematic diagram showing the basic principle of the integrated image system.

Basically, the principle of reproducing the 3D object 110 as a 3D image 210 is that element images are captured by the image acquisition device 130 after the light from the 3D object 110 passes through the lens array 120. The element images collected by the image acquisition step 100 and the image acquisition step 100 are displayed on the display apparatus 230 and then reproduced as a 3D image 210 in space through the lens array 220. It consists of a video playback step 200.

That is, the integrated image technology is largely divided into the image acquisition step 100 and the image reproduction step 200 as shown in FIG.

In the image acquisition step 100, various images formed by the lens array 120 are recorded in an image acquisition device 130 such as a CCD camera.

In this case, the images recorded in the image acquisition device 130 are called elemental images.

On the contrary, in the image reproducing step 200, the recorded element images are displayed on the display apparatus 230 so that the 3D image can be restored in the space through the lens array 220 as in the case of acquiring the element images. do.

Subsequently, the element images of the image acquisition step 100 and the element images 230 of the image reproduction step 200 are substantially the same.

In the conventional literature of such a three-dimensional integrated image display method is a method of compressing an element image by applying a region segmentation technique to the element image compression device in Patent No. 0891160, (a) from each other through a lens array from a three-dimensional object Obtaining an element image having another parallax; (b) dividing the obtained element image into similar regions having a plurality of similar images according to similar correlations; (c) rearranging the images included in each of the similar regions into a one-dimensional element image array; And (d) compressing the rearranged and generated one-dimensional element image array.

According to another exemplary embodiment of the present invention, in a method of restoring an integrated image using an element image picked up through a lens array in Korean Patent No. 0942271, the element image is enlarged to a predetermined size, and the enlarged respective elements Generating a reconstructed image by adding pixels located at the same coordinates of the image; Measuring a blur metric value of each reconstructed image; Selecting a reconstructed image corresponding to an inflection point of the blur metric value according to a focal length as a focus image; Generating an erosion image through an erosion operation of subtracting each pixel value of a corresponding erosion mask from each pixel value of the focus image; And it is described an integrated image restoration method comprising the step of mapping the erosion image to the reconstruction image.

Integrated imaging technology can be classified into two types according to the distance g between the lens array and the display device.

The first method can be divided into g ≠ f (focal length of the base lens), and the second method can be divided into g = f.

The former g? F is called resolution-priority integral imaging (RPII) or defocused mode integral imaging.

One point of the element image is imaged through the lens to form an integrated beam.

In this way, the resolution of the 3D image can be increased, but the depth region is drastically reduced.

In the latter case, g = f, a point of the element image is converted into a parallel beam through the lens to produce an integrated beam.

This case is called depth-priority integral imaging (DPII) or focused mode integral imaging.

In this method, the depth area for displaying a 3D image can be maximized, but the resolution of the 3D image is low.

As described above, in the depth-first integrated imaging technology having a basic structure, a two-dimensional image cannot be displayed because the lens array is placed in front of the display device.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and provides an integrated image display system capable of selectively expressing two-dimensional and three-dimensional images using a mask panel, or simultaneously expressing two-dimensional and three-dimensional images. The user can enjoy various modes of video according to user's taste.

According to the present invention, a method for implementing a display system capable of converting an integrated image using a mask panel into a 3D and 2D image is performed by sequentially displaying a lens array and a mask panel by a display device. When the display device is displayed in white and the 2D image is displayed on the mask panel, the viewer can observe only the 2D image.

Therefore, the present invention provides a method for implementing a display system which can be switched to 3D and 2D images in an integrated image using a mask panel, or selectively expressing 2D images and 3D images, or displaying 2D images and 3D images. By expressing on one screen at the same time, there is a remarkable effect such as viewing various images.

1 is a schematic diagram showing the basic principle of the integrated image system.

2 is a schematic diagram showing an integrated image display system using a mask panel.

3 is a schematic view of an operation state in the three-dimensional mode of the integrated image display system using a mask panel.

4 is a schematic view of an operation state in the two-dimensional mode of the integrated image display system using a mask panel.

FIG. 5 is a schematic view of an operating state in a two-dimensional and three-dimensional mixed mode of an integrated image display system using a mask panel; FIG.

6 is an experimental view of an integrated image display system using a mask panel.

7 is a photograph of an experimental apparatus that actually configures an integrated image display system using a mask panel.

8 is a resultant image photograph in three-dimensional mode of an integrated image display system using a mask panel.

FIG. 9 is an enlarged photograph of a portion of the image shown in FIG. 8. FIG.

10 is a resultant image photograph in a two-dimensional mode of an integrated image display system using a mask panel.

11 is an image photograph in a mixed mode of an integrated image display system using a mask panel.

<Explanation of symbols for the main parts of the drawings>

100. Image Acquisition Step

110. 3-D object 120. Lens array 130. Image acquisition device

200. Video playback stage

210. 3D image 220. Lens array 230. Display device

240. Mask panel 241. Transmission area 242. Blocking area

The present invention provides a method for implementing a display system capable of converting an integrated image using a mask panel into a 3D and a 2D image, wherein element images stored in the image acquisition device 130 are connected to the lens array 220 by the display device 230. When the mask panel 240 is sequentially viewed and displayed as a 3D image 210 in a space, the display apparatus 230 is displayed in white and the mask panel 240 is displayed as a 2D image. Only two-dimensional images can be observed.

When only a portion of the display apparatus 230 is displayed in white and a 2D image is displayed in the region of the mask panel 240 corresponding to the white region of the display apparatus 230, the observer of the display apparatus 230 may be configured. The 2D image corresponding to the white area can be observed.

In addition, in the remaining areas of the display apparatus 230 that are not displayed in white, the element images pass through the lens array 220 and the mask panel 240 to display a three-dimensional image in space so that the viewer does not display white. The 3D image corresponding to the remaining area of the display apparatus 230 may be observed.

Hereinafter, a method for implementing a display system that can be switched to 3D and 2D images in an integrated image using the mask panel according to the present invention will be described in detail with reference to the accompanying drawings.

To simplify the terminology before the description, the elementary lens refers to one small lens constituting the lens array, and elemental images of an imaging region on an image plane corresponding to the size of the elementary lens. image).

This set of element images is also called an elemental image array.

2 is a schematic diagram illustrating an integrated image display system using a mask panel.

As shown in FIG. 2, the mask panel displaying the mask pattern is closely attached to the lens array.

Here, the mask panel 240 may be a SLM (Spatial Light Modulator) capable of transmitting light (ON) or blocking (OFF), and LC panels are used a lot.

The distance between the display device 230 and the lens array 220 to display the element images is equal to f, which is a focal length of the base lens.

In the integrated image display method using the mask panel, a mask pattern displayed on the mask panel 240 placed in front of the lens array 220 is changed, and the corresponding element images are temporally multiplexed.

That is, the mask panel 240 is composed of a transmission area 241 through which the element images are transmitted and a blocking area 242 through which the element images are blocked. The mask panel 240 includes a transmission area 241 and a blocking area 242. As the pattern changes, it means that the element images are temporally multiplexed.

As a result, the resolution of the displayed 3D image 210 is improved.

It is assumed that the resolution of the 3D image 210 is increased by n × n times by applying time multiplexing.

Then, n × n different mask patterns (transmission area and blocking area) and corresponding n × n element arrays are required.

When the mask pattern and the element images formed on the mask panel 240 are synchronized with each other, a proper three-dimensional voxel (Volume pixel, voxel) is produced.

In the integrated image display system using the mask panel, the 3D mode is implemented as shown in FIG. 3.

3 is a schematic view of an operation state in a three-dimensional mode of an integrated image display system using a mask panel.

In order to display a 3D image, element images are displayed on the display apparatus 230, and a mask pattern corresponding to the element images is displayed on the mask panel 240.

For convenience, explanation is given by time multiplexing (t ₁ , t ₂ ) dividing the elementary lens into two in the longitudinal direction.

Thus, FIG. 3A shows the operating principle of the display system when t = t ₁ .

As shown in FIG. 3A, a mask pattern t = t ₁ is displayed on a mask panel and element images t = t ₁ are simultaneously displayed on a display device.

Then, the light rays starting from the element images are reduced in the size of the light beam while passing through the mask pattern, and cross each other in space to generate voxels.

Therefore, in the case of FIG. 3A, the generated voxel has a size of p / 2 equal to the size of the transmissive region 241 which is the ON region of the mask panel 240.

This means that the resolution of the reconstructed 3D image is doubled since the size of the reconstructed 3D image is 1/2 smaller than that of the conventional method without using the mask panel 240.

And in Figure 3 (b) shows a case of t = t _2.

The mask pattern t = t ₂ is expressed on the mask panel 240, and the element images t = t ₂ are displayed on the display device 230.

As shown in (a) of FIG. 3, the voxel size generated in space is p / 2, and the three-dimensional resolution is two times higher than that of the conventional method.

However, the difference between FIG. 3 (a) and FIG. 3 (b) is that the generation positions of the voxels are different, contributing to the improvement of the resolution by occupying the complementary positions.

A voxel is a compound word combining a volume and a pixel, and refers to a basic unit of a 3D image corresponding to a pixel, which is a basic unit of a 2D image.

In addition, the transmissive region 241 and the blocking region 242 of the mask panel 240 alternately alternate positions with time so that a clearer 3D image 210 is displayed.

When the alternation of the transmissive region 241 and the blocking region 242 of the mask panel 240 and the corresponding element images on the display apparatus 230 are performed at a very high speed, the change is caused by the persistence of human vision. I can't recognize it.

The method for implementing the 2D mode in the integrated image display system using the mask panel is shown in FIG. 4.

4 is a schematic view of an operation state in the two-dimensional mode of the integrated image display system using a mask panel.

In the two-dimensional mode, the mask panel serves as a display panel displaying a two-dimensional image.

In the display device 230, a set of element images is represented in the 3D mode, but in the 2D mode, all pixels of the display device 230 are displayed in white to function as a backlight.

Therefore, when the light from the display device (backlight) illuminates the mask panel (two-dimensional display) displaying the two-dimensional image, the viewer can see the two-dimensional image.

In some cases, when the backlight function, an image of a special image may be generated by displaying a color such as blue or gray instead of white.

For example, if the backlight is used as blue, theoretically, red and green are not expressed, so only the blue two-dimensional image is visible, but in fact, the blue light of the backlight may partially pass through the red and green filters of the two-dimensional panel. Because of this, unexpected color mixing occurs to generate an image of a special image.

Gray means lower value than white (level 255), and the 2D image is darkened overall.

That is, it can be seen that it has the same effect as the software method of adjusting the contrast to gray compared to the hardware method of adjusting the contrast through the OSD to adjust the brightness of the display device.

In this case, the resolution of the observed 2D image is the same as that of the mask panel 240.

Therefore, using the mask panel 240 having a high resolution, the observer can experience a high-resolution clear two-dimensional image.

In addition, the combination of the display device (backlight) 230 and the lens array 220 may be regarded as an active backlight, and may serve two functions.

One is to adjust the brightness of the backlight according to the user's purpose in a software way, and the second is to generate a directional light source having a directional light by generating only some pixels of the display device 230.

In the integrated image display system used in the present invention, it is possible to display a mixture of a 3D image and a 2D image on one screen.

To this end, the three-dimensional image region may be displayed independently in the three-dimensional mode and the two-dimensional image region in the two-dimensional mode.

FIG. 5 is a schematic diagram of an operation state of a mixed mode of 2D and 3D of an integrated image display system using a mask panel.

5 shows the operation principle of the 3D / 2D mixed mode.

In the 2D image area, a white image (light source) having the same size and shape as the 2D image is displayed on the display apparatus 230, and the 2D image is independently displayed on the mask panel 240.

Meanwhile, in the 3D image area, element images are displayed on the display apparatus 230, and a mask pattern is displayed on the mask panel 240.

This 3D / 2D mixed mode function may be a major function in a practical 3D display.

In the above, the three modes (three-dimensional mode, two-dimensional mode, three-dimensional / two-dimensional mixed mode) has been described theoretically.

FIG. 6 is an experimental view of an integrated image display system using a mask panel, and shows a display experiment structure for experimentally confirming. FIG.

It is assumed that the lens array 220 and the mask panel 240 are in close contact with each other at a position of z = 0.

That is, the thickness of the mask panel 240 was ignored.

The LCD is positioned at z = -f, which is the focal length of the primary lens.

However, color separation occurs inevitably due to the fact that the distance between the lens array and the LCD is f and the physical structure of the pixels of the LCD.

Therefore, a diffuser is required to compensate for this.

Although not confirmed in FIG. 7, a diffuser film was attached to the LCD surface.

Specifications of the optical and electronic devices used in FIG. 7 are as follows.

The lens array has a size of 24 ″ and the number of basic lenses is 330 × 180.

The focal length f of the base lens is 8 mm, the size of the base lens is 1.6 x 1.6 mm, and the viewing angle is about 11 degrees.

The LCD is a BenQ XL-2411t with a 24 "screen size.

The resolution is 1920 × 1080 and the maximum scanning rate in the vertical direction is 120 Hz.

As the mask panel 240, an LC panel inside the same model product was used.

First, the element image generation required for the display experiment was performed through CG pickup.

As shown in FIG. 6, two planar objects (flower background, 'DSL') are assumed to be at −15f and 10f, respectively.

Since f = 8 mm, the positions of the two objects correspond to -120 mm and 80 mm, respectively.

Elemental images of the two objects were acquired through a virtual pinhole array having the same period as the lens array.

At this time, unlike the existing method, a new element image generation principle suitable for an integrated image method using a mask panel was applied.

The lens division number (or time multiplexing number) is 2 × 2.

One mask pattern and the element images generated corresponding thereto are paired.

A total of four pairs of mask patterns and element images were sequentially and repeatedly displayed on the LC panel and the LCD of FIG. 7, respectively.

120 pairs of mask patterns and elementary images per second were displayed by the computer in synchronization.

Thus, the observer has the same visual effect as seeing 30 (= 120/4) conventional images per second.

8 is a resultant image photograph in a 3D mode of an integrated image display system using a mask panel.

That is, Figure 8 is a picture of the picture taken by the camera to play the video.

8 (a) and 8 (b) are taken from the left and right, respectively.

Accordingly, in FIG. 8, it is possible to confirm the parallax of the reproduced image and give a three-dimensional effect.

FIG. 9 is an enlarged photograph of a portion of the image illustrated in FIG. 8, and shows a result of an experiment for confirming the resolution of a reproduced 3D image.

In this case, the experiment was performed assuming that the front object ('DSU') is at z = 0.

FIG. 9A is an enlarged view of a part of an image reproduced by a conventional method without using a mask, and FIG. 9B is an enlarged view of a part of an image reproduced by a method using a mask panel.

Accordingly, it can be experimentally confirmed from FIG. 9 that the resolution of the 3D image displayed in the integrated image method using the mask panel is improved by 2 × 2 times than the conventional method.

FIG. 10 is a resultant image photograph in the two-dimensional mode of the integrated image display system using the mask panel, and shows the result of the experiment in the two-dimensional mode.

In FIG. 7 described above, when only a monochrome image having a white color is displayed on the entire screen of the LCD, the LCD serves as a backlight.

In addition, two-dimensional mode was realized by displaying a high-resolution two-dimensional image instead of a mask pattern on the LC panel.

10 (a) is an original image of 1920 x 1080 resolution displayed on the LC panel, Figure 10 (b) is an observed image, it can be seen that the same high-resolution image representation as the original image can be confirmed.

Meanwhile, element images and mask patterns for the 3D / 2D mixed mode are generated through a two-step process.

In the case of element images, in step 1, element images are generated by placing a background object and a front object in a desired position as shown in FIG.

In the second step, all of the pixels constituting the element images, the colors of the pixels overlapping with the pixels constituting the front object are changed to white.

That is, the second step is the same as synthesizing the white object image and the element images having the shape and size of the two-dimensional image.

This area becomes the backlight for the two-dimensional image located in front.

In the case of a mask pattern, a mask pattern is first created as in the 3D mode.

As a next step, a 2D image is added to a mask pattern and synthesized.

When the synthesized pairs of element images and mask patterns are synchronized and displayed on the LCD and the mask panel, respectively, the mixed image of the 3D image and the 2D image can be observed.

At this time, the 2D image is positioned on the surface of the lens array, that is, z = 0.

In the actual experiment, it is assumed that the background object is z = -10f and two plane characters are z = 10f.

In this experiment, the 3D mode was implemented by the conventional method without using the mask panel, and only the 2D image was represented on the LC panel.

That is, only the mask pattern area of the mask panel was changed to the simple transmission area.

For this purpose, a composite image in which only two-dimensional images exist on a white background is displayed on the LC panel.

Experimental results can be seen in FIG.

FIG. 11 is an image photograph in a mixed mode of an integrated image display system using a mask panel, in which FIG. 11 (a) is an image observed from the left viewpoint and FIG. 11 (b) is an image observed from the right viewpoint.

Accordingly, in FIG. 11, parallax and three-dimensional effects between the background image, the 2D image, and the front image can be confirmed.

And it can be seen that the two-dimensional image is also expressed in high resolution.

As a result, the present invention proposed a display system that can be switched to an integrated image based 3D and 2D using a mask panel, and performed an optical display experiment.

A method of generating element images and mask patterns suitable for three modes has been described, and a computer generated method is used. An experiment of optically displaying the generated element images and mask patterns by synchronizing them is performed.

As a result, it can be confirmed through the photographs of the experiment results of FIGS. 8 to 10 that the 3D mode using the mask panel can express a great depth, the resolution is improved, and the high resolution 2D image can be expressed in the 2D mode. Can be.

In addition, as illustrated in FIG. 11, the 3D image and the 2D image may be simultaneously displayed.

Therefore, the present invention provides a method for implementing a display system which can be switched to 3D and 2D images in an integrated image using a mask panel, or selectively expressing 2D images and 3D images, or displaying 2D images and 3D images. By expressing on one screen at the same time, there is a remarkable effect such as viewing various images.

Claims (3)

1. The element images stored in the image acquisition device 130 are sequentially displayed through the lens array 220 and the mask panel 240 by the display device 230 and displayed as a 3D image 210 in space. When the device 230 is displayed in white and a two-dimensional image is displayed on the mask panel 240, the viewer can observe only the two-dimensional image. A method of implementing a switchable display system.
2. The method of claim 1,

   When only a portion of the display apparatus 230 is displayed in white and a 2D image is displayed in the region of the mask panel 240 corresponding to the white region of the display apparatus 230, the observer of the display apparatus 230 may be configured. A method of implementing a display system that can be switched to three-dimensional and two-dimensional images in an integrated image using a mask panel characterized in that the two-dimensional image corresponding to the white area can be observed.
3. The method of claim 2,

   In the remaining areas of the display device 230 that are not displayed in white, the element images pass through the lens array 220 and the mask panel 240 to display a three-dimensional image in space so that the viewer does not display white. A method of implementing a display system that can be switched to three-dimensional and two-dimensional images in an integrated image using a mask panel characterized in that the three-dimensional image corresponding to the remaining area of the device 230 can be observed.

Images