KR101614121B1 - Apparatus and method for displaying integrated image - Google Patents

Abstract

Provided are an apparatus for displaying an integrated image and a method for displaying an integrated image. The apparatus for displaying an integrated image of the present invention includes: a light field microscope obtaining a light field image in which pieces of different viewpoint information of an object are recorded by using a first lens array and an object lens; a second lens array; a processor cropping the light field image into a predetermined size based on an F-number of the object lens and an F-number of the second lens array and generating an elemental image from the cropped light field image; and a display panel spatially displaying a three-dimensional integrated image showing the object by causing the elemental image to pass through the second lens array.

Description

[0001] Apparatus and method for displaying integrated images [

The present disclosure relates to a method and apparatus for generating an elemental image using an optical field image and displaying the integrated image based on the generated elemental image.

Recently, there has been a great demand for a three-dimensional image display device capable of realizing images more effectively and effectively in various fields such as movies, games, advertisements, medical images, education, and military. Accordingly, various techniques for displaying three-dimensional images have been proposed, and various types of three-dimensional image display devices have already been commercialized.

For example, the three-dimensional image display device includes a spectacle method and a non-spectacle method. In the stereoscopic method, there are a lenticular method using a plurality of cylindrical lens arrays and a parallax barrier method having a plurality of openings and openings. have.

An integrated image method has been proposed as a three-dimensional image display method in which the depth perception recognized by the brain matches the focal point of the eye and can provide full parallax.

The integral imaging technique is a method of non-spectacular three-dimensional display, which provides both vertical and horizontal parallax using a lens-array to provide three-dimensional images to viewers without special glasses There is a way. Each element lens constituting the lens array can display a three-dimensional virtual image before or after the integrated image display by adjusting the optical information distribution of the specially produced element image.

According to the present embodiments, there is provided a method and apparatus for displaying a three-dimensional integrated image on a space by performing cropping on an optical field image in which different viewpoint information of an object is recorded.

The integrated image display device according to the first aspect includes an optical element for acquiring a light field image in which different viewpoint information of an object is recorded using a first lens array and an objective lens, microscope; A second lens array; Based on the F-number of the objective lens and the F-number of the second lens array, the optical field image is cropped to a predetermined size, and the elemental image is extracted from the cropped optical field image. image; And a display panel for passing the elemental image through the second lens array and displaying a three-dimensional integrated image representing the object on a space.

In addition, the processor can perform a predetermined pixel mapping on the cropped optical field image to generate an element image.

Also, the predetermined pixel mapping may be to rotate the cropped light field image 180 degrees.

The processor can also adjust the size of the cropped light field image according to the resolution of the elemental image.

Further, the resolution of the element image can be determined by the pitch size of the pixels in the display panel and the pitch size of one lens in the second lens array.

Further, the optical field microscope includes: a light source for irradiating an object with a predetermined light field; An objective lens for focusing an optical field passing through the object; A first lens array for receiving a focused optical field; And a sensor for detecting an optical field passing through the first lens array and obtaining an optical field image.

In addition, the processor generates a plurality of element images for the object at predetermined intervals, and the display panel sequentially passes the plurality of element images to the second lens array, and displays the integrated image reflecting the change of the object as a moving image Can be displayed.

According to another aspect, a direct image display method includes: obtaining a light field image in which different viewpoint information of an object is recorded using a first lens array and an objective lens; Based on the F-number of the objective lens and the F-number of the second lens array, the optical field image is cropped to a predetermined size, and the elemental image is extracted from the cropped optical field image. image; And displaying the three-dimensional integrated image representing the object on the space by passing the element image through the second lens array.

According to embodiments of the present invention, the integrated video display device according to the present disclosure can display an integrated image of an object in real time using a light field microscope.

In addition, according to embodiments of the present invention, the integrated image display device can display an integrated image of an object on a space without distortion.

In addition, according to embodiments of the present invention, the integrated image display device performs simple image processing such as cropping or pixel mapping on the obtained optical field image, thereby increasing the speed of generating an element image for an integrated image of the object .

The present invention may be readily understood by reference to the following detailed description and the accompanying drawings, in which reference numerals refer to structural elements.
1 is a view for explaining an integrated image display device according to the present disclosure.
2 is a diagram for explaining a more specific embodiment in which an optical field microscope acquires an optical field image of an object, according to an embodiment.
3 is a diagram for explaining an embodiment of a cropping and pixel mapping performed by a processor.
4 is a diagram for explaining an embodiment of a cropping performed by a processor according to an embodiment.
5 is a diagram illustrating an embodiment in which a processor generates an elemental image, according to one embodiment.
6 is a diagram for explaining an embodiment in which the display panel displays an integrated image.
7 is a view for explaining an embodiment in which an integrated video display device displays an integrated image of an object.
8 is a view for explaining an integrated image display method performed by the integrated image display device according to an embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The following examples are intended to illustrate the invention and are not intended to limit or limit the scope of the invention. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. Also, in certain cases, there may be a term selected arbitrarily by the applicant, in which case the meaning thereof will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term, not on the name of a simple term, but on the entire contents of the present invention.

When an element is referred to as "including" an element throughout the specification, it is to be understood that the element may include other elements as well, without departing from the spirit or scope of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. In order to clearly explain the present invention in the drawings, parts not related to the description will be omitted.

1 is a view for explaining an integrated image display device 100 according to the present disclosure.

The integrated image display device 100 according to an embodiment includes a light field microscope 110 using a first lens array 112 and an objective lens 114, a processor 120, An array 140 and a display panel 130. The integrated video display device 100 shown in Fig. 1 shows only the components related to the present embodiment. Therefore, it will be understood by those skilled in the art that other general-purpose components other than the components shown in FIG. 1 may be further included.

The optical field microscope 110 may be configured to provide an optical field image (e.g., an optical field image) in which different viewpoint information of an object is recorded, using the first lens array 112 and the objective lens 114, light field image. According to one embodiment, the optical field microscope 110 may capture an optical field through an object to obtain an optical field image. The light field is a concept of expressing the state of light distribution in space through the distribution of light rays. According to this concept, the light reflected or generated in an object is defined as going straight into space and entering the human eye, and the three-dimensional space can be composed of a myriad of optical fields. An example of a mathematical representation of an individual optical field may be a 5-dimensional Pleoptic function, where three-dimensional spatial coordinates (x, y, z) of a point through which a ray passes through a plane on a particular plane in space, Can be represented by the luminance with respect to the spatial direction angles ([theta], [phi]). Thus, the optical field microscope 110 can be obtained by informing the pleofoptic function value of the optical field passing through the object. For example, the optical field microscope 110 can obtain the luminance values of (?,?) Of the optical field passing through the object with respect to each of the spatial coordinates (x, y, z).

More specifically, the optical field microscope 110 will be described below with reference to FIG.

2 is a diagram for explaining a more specific embodiment in which an optical field microscope acquires an optical field image of an object, according to an embodiment.

The optical field microscope 110 may include a light source 202, an objective lens 114, a first lens array 112, and a sensor 206, according to one embodiment. The optical field microscope 110 shown in Fig. 2 shows only the components related to the present embodiment. Accordingly, it will be understood by those skilled in the art that other general-purpose components other than the components shown in FIG. 2 may be further included.

Light source 202 may illuminate a given light field toward object 204, according to one embodiment. The object can be a microscopic organism with a size of less than 0.1 mm beyond the visible limit of the naked eye, and can be, for example, a pretty nematode (Caenorhabditis elegans). Light source 202 may be an incoherent light source, according to one embodiment.

Objective lens 114 may focus an optical field through object 204, according to one embodiment. The optical field microscope 110 can be designed so that the objective lens 114 has a high resolving power. In addition, the optical field microscope 110 can be designed to have a high resolution by adjusting the numerical aperture (NA) of the objective lens 114.

The first lens array 112 can receive the optical field focused by the objective lens 114. The first lens array 112 may comprise a plurality of lenses in a micro-sized unit in a two-dimensional array. According to one embodiment, the first lens array 112 may be located at a point focused by the objective lens 114.

The sensor 206 can detect the light field that has passed through the first lens array 112. [ The sensor 206 may also capture an optical field through the first lens array 112 to obtain an optical field image for the object 204. The sensor 206 may be an image sensor, such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), according to one embodiment.

In FIG. 2, for convenience of explanation, the light field that has passed through a predetermined point 208 of the object 204 is illustrated as an example, and the viewpoint information of the predetermined point 208 of the object 204 Although the optical field 209 is shown, it is equally applicable to other points in the object 204. Thus, the optical field microscope 110 can capture an optical field containing different viewpoint information of an object, thereby obtaining an optical field image for the object.

The image 210 of FIG. 2 represents a portion of the optical field image acquired by the optical field microscope.

The image 210 represents the optical field captured in the sensor 206 relative to the 2 x 2 lens array region in the first lens array 112. That is, the image 210 represents a part of the optical field image. Since the image 210 does not match the F-number of the objective lens 114 and the F-number of the first lens array 112, the optical field of the object is captured in the circle area 212 . That is, an aperture stop in the objective lens 114 can form a circle area 212. In addition, the square area in the circle area 212 indicates an area to be subjected to cropping, which will be described later with reference to FIG.

The processor 120 of FIG. 1 includes an image processor 120 for cropping the optical field image acquired by the optical field microscope 110 to a predetermined size, and for displaying an integrated image of the object from the cropped image an elemental image can be generated. Generally, the F-number of the objective lens 114 and the F-number of the second lens array 112 do not match each other, so that an integrated image generated directly from the optical field image acquired by the optical field microscope 110 It is possible to distort the three-dimensional image of the object.

Therefore, in order to prevent distortion of the integrated image, the processor 120 obtains the F-number by the optical field microscope 110 based on the F-number of the objective lens 114 and the F-number of the second lens array 112 It is possible to crop the optical field image to a predetermined size.

Cropping refers to cropping a portion of a predetermined image to adjust the size of a predetermined image to fit the desired size. Further, the F-number can be defined by the following equation (1).

In Equation (1), N denotes f-number, f denotes a focal length, p denotes a pitch indicating the diameter of the lens, and NA denotes a numerical aperture. A specific embodiment in which the processor 120 crops the optical field image will be described with reference to FIGS. 3 and 4. FIG.

The processor 120 may adjust the size of the light field image or the cropped light field image according to the resolution of the elemental image, according to one embodiment. According to one embodiment, the resolution of the elemental image may be determined according to the pitch size of the pixels in the display panel 130 and the pitch size of one lens included in the second lens array 140. As a specific example, if the pitch size of the display panel 130 is 125 μm and the pitch size of one lens included in the second lens array 140 is 1 mm, the resolution of the element image is determined to be 8 × 8. Accordingly, when the resolution of the optical field image is 31 × 31 and the resolution of the elementary image is 8 × 8, the processor 120 resizes the size of the optical field image or the cropped optical field image to 8 × 8, (resizing). An example of resizing is undersampling.

 Processor 120 may perform predetermined pixel mapping on an optical field image or a cropped optical field image, according to one embodiment. Pixel mapping refers to changing the position of a pixel or changing a predetermined value of a pixel within a two-dimensional matrix of pixels constituting an image. Processor 120 may perform predetermined pixel mapping on an optical field image or a cropped optical field image to solve the pseudoscopic problem of the integrated image generated from the optical field image, . The pseudoscopic problem means that when an integrated image of an object is displayed on a space, an observer observes an integrated image of the object appears in a perspective relationship that is opposite to that of an observer directly observing the object.

According to one embodiment, the processor 120 may perform pixel mapping to solve the pseudoscopic problem using Equation (2) below.

(I, j, t) corresponding to the (s, t) position with respect to the lens region corresponding to the (i, j) position of the first lens array 112, (I ', j', s ', t') is a pixel coordinate value in the second lens array 140 (s ', j' , t ') position of the display panel 130. [0060] As shown in FIG. Also, k is a numerical value indicating the distance that an observer views an integrated image of an object. That is, when k = 0, the integrated image is displayed at the position closest to the observer, and as the value of k increases, the integrated image is displayed at a position farther from the observer. Also, n represents the number of pixels in the sensor 206 with reference to any one of the lens area areas in the first lens array or the second lens array. According to one embodiment, the processor 120 may set k = 0 in Equation (2).

According to one embodiment, the processor 120 may rotate the optical field image or the cropped optical field image 180 degrees to perform pixel mapping.

Accordingly, the processor 120 may crop an optical field image obtained by the optical field microscope 110 to a predetermined size, thereby generating an element image for displaying an integrated image of the object 204. [ In addition, the processor 120 may generate the element image by adjusting the size of the optical field image or the cropped optical field image according to the resolution of the elementary image. In addition, the processor 120 may perform predetermined pixel mapping on the light field image or the cropped light field image to generate an element image.

The display panel 130 of Figure 1 may pass an elemental image generated by the processor 120 to the second lens array 140 to display an integrated image representing the object in space, according to one embodiment . The second lens array 140 may include a plurality of lenses in a unit of micro size and may be configured in a two-dimensional array, and may receive an element image to display an integrated image. The second lens array 140 may function as an optical element for an autostereoscopic three-dimensional image, in order to display a three-dimensional integrated image, according to one embodiment. The display panel 130 may be a flat display panel according to an embodiment. The details of display of the integrated image of the object by the display panel 130 will be described with reference to FIG.

According to one embodiment, the integrated image display device 100 performs simple image processing such as cropping or pixel mapping on the optical field image obtained from the optical field microscope 110, And as a result, a plurality of elementary images can be generated every short period or in real time. That is, when the position or state of the object changes, the integrated image display device 100 sequentially passes a plurality of element images generated in real time to the second lens array, As shown in FIG.

FIG. 3 is a diagram for explaining an embodiment of cropping and pixel mapping performed by the processor 120. FIG.

The light field image 310 represents a captured light field that includes different viewpoint information with respect to an area corresponding to each of the plurality of lenses constituting the first lens array 112. [

The processor 120 may crop a region 315 of a predetermined size for each region of the optical field image 310 and generate a cropped optical field image 320. [

In addition, the processor 120 may perform pixel mapping on the cropped optical field image 320 according to one embodiment. For example, the processor 120 may perform pixel mapping to rotate the cropped light field image 320 180 degrees.

Thus, the processor 120 may perform pixel mapping on the cropped light field image 320 to generate the element image 330.

4 is a diagram for explaining an embodiment of a cropping performed by the processor 120, according to an embodiment.

The table 410 shown in FIG. 4 shows the specifications of the first lens array 112, the objective lens 114, and the second lens array 140.

The processor 120 may crop the region 425 of the captured optical field image 420 relative to the lens region of either of the first lens arrays 112, according to one embodiment. The processor 120 may crop the region 425 of the optical field image 420 based on the ratio of the F-number of the objective lens 114 to the F-number of the second lens array 112 . More specifically, according to Equation 1 described above, a value of '1' is calculated for the F-number of the objective lens 114 and a value of '2' is calculated for the F-number of the second lens array 112 . Accordingly, the processor 120 calculates the optical field image (100 x 100 pixels) based on the ratio of the F-number of the objective lens 114 and the F-number of the second lens array 112, The area 425 composed of 50 x 50 pixels can be cropped. Thus, the processor 120 can perform cropping by the size of the region 425 for each of the plurality of lens regions included in the first lens array 112, and generate an elemental image based thereon .

5 is a diagram illustrating an embodiment in which processor 120 generates an elemental image, in accordance with one embodiment.

The optical field microscope 110 includes an optical field image 510 for an object consisting of a letter 'N' located on the focal plane, a letter 'S' located 25 μm below the focal plane and a letter 'U' Can be obtained. That is, the optical field microscope 110 can acquire an optical field image 510 in which different viewpoint information about an object made up of characters 'S', 'N', and 'U' having a size of 150 μm is recorded.

The processor 120 crops the optical field image 510 to a predetermined size based on the F-number of the objective lens 114 and the F-number of the second lens array 112, An image 520 may be generated. According to one embodiment, the processor 120 may perform undersampling to resize the size of the cropped optical field image 520. [

The processor 120 may perform predetermined pixel mapping on the cropped light field image 520. [ According to one embodiment, the processor 120 may rotate the cropped light field image 180 degrees to perform pixel mapping to generate an elemental image 530 of the object.

6 is a view for explaining an embodiment in which the display panel 130 displays an integrated image.

The display panel 130 may irradiate the second lens array 140 with the element image generated by the processor 120. [ The display panel 130 can display the three-dimensional integrated image 610 representing the object 204 in space by passing the elemental image through the second lens array 140. Accordingly, the observer 620 can observe the integrated image 610 that is larger than the actual size of the object 204 in the three-dimensional space.

7 is a view for explaining an embodiment in which the integrated image display device 100 displays an integrated image of an object.

The optical field microscope 110 may obtain an optical field image 710 in which different viewpoint information of an object is recorded, according to one embodiment.

The processor 120 may crop the optical field image 710 to a predetermined size and generate an elemental image for displaying an integrated image of the object from the cropped optical field image.

Accordingly, the display panel 130 can pass the elemental image to the second lens array 140, and display the integrated image of the object on the space.

The image 720 represents an integrated image displayed on the space. The image 720 represents the views viewed from the five observer positions (Top, Center, Right, Left, and Bottom) to indicate that the three-dimensional image is displayed . Accordingly, as can be seen from the image 720, it can be confirmed that a three-dimensional image is displayed in that a parallax exists for each of the images.

8 is a view for explaining an integrated video display method performed by the integrated video display device 100 (hereinafter referred to as the device 100) according to an embodiment.

The method shown in FIG. 8 can be performed by each component of the apparatus 100 of FIG. 1, and redundant descriptions are omitted.

In step S810, the apparatus 100 may acquire a light field image in which different viewpoint information of the object Object is recorded, using the first lens array and the objective lens. According to one embodiment, the apparatus 100 may capture an optical field through an object to obtain an optical field image. More specifically, the apparatus 100 can illuminate a given light field towards the object and focus the light field through the object to the point where the first lens array is located. The device 100 may then detect the light field that has passed through the first lens array to obtain an optical field image.

In step S820, the apparatus 100 crops the optical field image to a predetermined size based on the F-number of the objective lens and the F-number of the second lens array, An elemental image can be generated from the extracted optical field image.

The apparatus 100 may adjust the size of the optical field image or the cropped optical field image according to the resolution of the elemental image, according to one embodiment. Accordingly, when the resolution of the optical field image is 31 × 31 and the resolution of the elementary image is 8 × 8, the processor 120 resizes the size of the optical field image or the cropped optical field image to 8 × 8, (resizing). An example of resizing is undersampling.

The apparatus 100 may perform predetermined pixel mapping on an optical field image or a cropped optical field image, according to one embodiment. Device 100 may perform a predetermined pixel mapping on an optical field image or a cropped optical field image to solve the pseudoscopic problem of the integrated image generated from the optical field image, have. According to one embodiment, the apparatus 100 may rotate the optical field image or the cropped optical field image 180 degrees to perform pixel mapping.

Accordingly, the apparatus 100 can crop an optical field image to a predetermined size, and generate an element image for displaying a three-dimensional integrated image of the object. In addition, the apparatus 100 may generate an elementary image by adjusting the size of the optical field image or the cropped optical field image according to the resolution of the elementary image. In addition, the apparatus 100 may perform predetermined pixel mapping on an optical field image or a cropped optical field image to generate an elemental image.

In step S830, the apparatus 100 passes the elemental image through the second lens array, and can display a three-dimensional integrated image representing the object in space.

The integrated image display apparatus and method according to the present invention are not limited to the configuration and the method of the embodiments described above, but the embodiments may be modified so that all or some of the embodiments may be modified Or may be selectively combined.

 The specific implementations described in this embodiment are illustrative and do not in any way limit the scope of the invention. For brevity of description, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of such systems may be omitted. Also, the connections or connecting members of the lines between the components shown in the figures are illustrative of functional connections and / or physical or circuit connections, which may be replaced or additionally provided by a variety of functional connections, physical Connection, or circuit connections.

In this specification (particularly in the claims), the use of the terms "above" and similar indication words may refer to both singular and plural. In addition, when a range is described, it includes the individual values belonging to the above range (unless there is a description to the contrary), and the individual values constituting the above range are described in the detailed description. Finally, if there is no explicit description or contradiction to the steps constituting the method, the steps may be performed in an appropriate order. It is not necessarily limited to the description order of the above steps. The use of all examples or exemplary terms (e. G., The like) is merely intended to be illustrative of technical ideas and is not to be limited in scope by the examples or the illustrative terminology, except as by the appended claims. It will also be appreciated by those skilled in the art that various modifications, combinations, and alterations may be made depending on design criteria and factors within the scope of the appended claims or equivalents thereof.

Claims (13)

A light field microscope for acquiring a light field image in which different viewpoint information of an object is recorded using a first lens array and an objective lens;
A second lens array;
The optical field image is cropped to a predetermined size based on the F-number of the objective lens and the F-number of the second lens array, A processor for generating an elemental image; And
And a display panel for passing the elemental image through the second lens array and displaying a three-dimensional integrated image representing the object on a space,
The light field image
Wherein the first lens array includes regions corresponding to each of the plurality of lenses constituting the first lens array,
The processor comprising:
Determining a size of an area to be cropped in the areas based on a ratio of an F-number of the objective lens and an F-number of the second lens array, and cropping each of the areas according to the determined size And generates the elemental image from the cropped regions. The method according to claim 1,
The processor comprising:
And performing predetermined pixel mapping on the cropped regions to generate the elementary image. 3. The method of claim 2,
Wherein the predetermined pixel mapping comprises:
And rotates the cropped regions by 180 degrees. The method according to claim 1,
The processor comprising:
And adjusts the size of the cropped regions according to the resolution of the elementary image. 5. The method of claim 4,
Wherein the resolution of the elementary image is determined by a pitch size of pixels in the display panel and a pitch size of one lens in the second lens array. The method according to claim 1,
In the optical field microscope,
A light source for irradiating the object with a predetermined light field;
The objective lens focusing the light field passing through the object;
The first lens array receiving the focused optical field; And
And a sensor for detecting the light field passing through the first lens array and obtaining the light field image. The method according to claim 1,
The processor comprising:
Generating a plurality of element images for the object every predetermined period,
The display panel includes:
And sequentially passes the plurality of elemental images through the second lens array to display an integrated image reflecting a change of the object as a moving image. Using a first lens array and an objective lens, obtaining a light field image in which different viewpoint information of an object is recorded;
The method comprising: cropping the optical field image to a predetermined size based on an F-number of the objective lens and an F-number of the second lens array; Generating an elemental image; And
Passing the elemental image through the second lens array and displaying a three-dimensional integrated image representing the object on a space,
The light field image
Wherein the first lens array includes regions corresponding to each of the plurality of lenses constituting the first lens array,
Wherein the step of generating the elementary image comprises:
Determining a size of an area to be cropped in the areas based on a ratio of an F-number of the objective lens and an F-number of the second lens array, and cropping each of the areas according to the determined size And generating the elemental image from the cropped regions. 9. The method of claim 8,
Wherein the step of generating the elementary image comprises:
Further comprising performing predetermined pixel mapping on the cropped regions. ≪ Desc / Clms Page number 13 > 10. The method of claim 9,
Wherein the predetermined pixel mapping comprises:
And rotating the cropped regions by 180 degrees. 9. The method of claim 8,
Wherein the step of generating the elementary image comprises:
Further comprising adjusting the size of the cropped regions according to the resolution of the elemental image. 9. The method of claim 8,
Wherein obtaining the light field image comprises:
Irradiating the object with a predetermined light field;
And detecting an optical field passing through the object to obtain the optical field image. 9. The method of claim 8,
Generating a plurality of element images for the object at every predetermined period; And
Further comprising the step of sequentially passing the plurality of elemental images through the second lens array and displaying an integrated image reflecting a change of the object as a moving image.

Images