KR102660735B1 - Apparatus, Method and System For Generating Three-Dimensional Image Using a Coded Phase Mask - Google Patents

Abstract

The present disclosure relates to a three-dimensional imaging device, method, and system using a coded mask. The three-dimensional imaging device according to an embodiment of the present disclosure includes a transceiver for transmitting and receiving signals, and a processor for controlling the transceiver. , the processor synthesizes complex data about the object based on the image obtained by photographing the object, and creates a three-dimensional image of the object based on the complex data about the object and the PSF (Point Spread Function) image for the point light source. can be created.

Description

Apparatus, Method and System For Generating Three-Dimensional Image Using a Coded Phase Mask}

The present disclosure relates to an apparatus, method, and system for generating a three-dimensional image using a coded phase mask.

The camera focuses light reflected from an object and uses the collected light to create an image that expresses the shape and color of the subject through a recording medium. Cameras generally use a lens to converge light reflected from an object, but at this time, various aberrations occur due to the curvature of the lens sphere, etc. These optical aberrations interfere with the creation of clear images and can be a major factor in degrading camera performance. Accordingly, additional technology is required to remove optical aberrations of the lens, and the optical system using the lens is structured somewhat complicatedly.

In order to solve various optical problems of these lens-based cameras, various image generation technologies based on pinhole camera structures have been proposed. Figure 1 is a diagram of the process of generating an image by borrowing a pinhole camera structure. A pinhole camera is an ideal camera that is free from the various optical problems of lens-based cameras. It is a device that lets light enter through a very small hole (pinhole) and creates an image on a recording medium at a certain distance. However, pinhole cameras have a problem in that the amount of light is relatively small compared to lens cameras. In order to solve this problem of pinhole cameras, a camera that uses a coded aperture while adopting the pinhole camera structure as shown in Figure 1 has been developed. It has been suggested.

The coded aperture is an aperture with a plurality of pinholes located at random positions, and light reflected from an object is focused on each pinhole through the coded aperture and recorded on a recording medium. When a decoding process is performed based on a plurality of recorded images (patterns), one image of the object is generated.

The purpose of the present disclosure is to efficiently generate a three-dimensional image using a coded phase mask.

The purpose of the present disclosure is to provide a cost-reduced three-dimensional image generation device, method, and system using a coded phase mask.

The purpose of the present disclosure is to provide an apparatus, method, and system for generating a very compact three-dimensional image using a coded phase mask.

Other objects and advantages of the present disclosure can be understood by the following description and will be more clearly understood by the examples of the present disclosure. In addition, it will be readily apparent that the objects and advantages of the present disclosure can be realized by means and combinations thereof as indicated in the claims.

In order to achieve the above object, a three-dimensional image generating device according to an embodiment of the present disclosure includes a transceiver for transmitting and receiving signals, and a processor for controlling the transceiver, wherein the processor produces an image obtained by photographing an object. Based on this, complex data about the object can be synthesized, and a three-dimensional image of the object can be generated based on the complex data about the object and the PSF (Point Spread Function) image of the point light source.

Meanwhile, an image obtained by photographing the object may be acquired by a polarization image sensor through a coded phase mask (CPM).

Meanwhile, the coded phase mask may be a coded half-wave phase mask.

Meanwhile, the coded half-wave plate phase mask may be based on a geometric phase element manufactured using a scanning method.

Meanwhile, there are a plurality of images obtained by photographing the object, acquired through one photographing using the polarization image sensor, and may include phase-modulated brightness information about the object.

Meanwhile, the polarization image sensor is a color polarization image sensor, and the phase-modulated brightness information for the object may include brightness information for each color.

Meanwhile, the PSF image for the point light source may be a PSF image corresponding to the position of the image obtained by photographing the object, extracted from the PSF library for the point light source.

Meanwhile, the PSF library may include PSF images obtained by changing the depth of a plurality of point light sources.

Meanwhile, the PSF image obtained by changing the depth of the point light source may be generated by synthesizing complex data for the point light source based on the image obtained by photographing the point light source at each depth.

Meanwhile, the 3D image of the object may be generated through a convolution operation between the PSF image and complex data about the object.

In order to achieve the above object, a method for generating a 3D image according to an embodiment of the present disclosure includes the steps of synthesizing complex data about an object based on an image obtained by photographing the object, complex data about the object and a point light source It may include generating a three-dimensional image of the object based on a Point Spread Function (PSF) image.

Meanwhile, the image obtained by photographing the object may be acquired by a polarization image sensor through a coded phase mask (CPM), and the coded phase mask is a coded half-wave plate phase mask (coded half-wave mask). It may be a wave phase mask).

Meanwhile, the coded half-wave plate phase mask may be based on a geometric phase element manufactured using a scanning method.

Meanwhile, there are a plurality of images obtained by photographing the object, acquired through one photographing using the polarization image sensor, and may include phase-modulated brightness information about the object.

Meanwhile, the polarization image sensor is a color polarization image sensor, and the phase-modulated brightness information for the object may include brightness information for each color.

Meanwhile, the PSF image for the point light source may be a PSF image corresponding to the position of the image obtained by photographing the object, extracted from the PSF library for the point light source.

Meanwhile, the PSF library may include PSF images obtained by changing the depth of a plurality of point light sources.

Meanwhile, the PSF image obtained by changing the depth of the point light source may be generated by synthesizing complex data for the point light source based on the image obtained by photographing the point light source at each depth.

Meanwhile, the 3D image of the object may be generated through a convolution operation between the PSF image and complex data about the object.

In order to achieve the above object, a three-dimensional image generation system according to an embodiment of the present disclosure includes a coded phase mask (CPM) that modulates the phase of reflected light of an object, and a phase modulation from the coded phase mask. A polarization image sensor that records the brightness information of the object as an image based on the reflected light of the object, and synthesizes complex data about the object based on the image of the object, and complex data about the object and the point light source It may include a 3D image generating device that generates a 3D image of the object based on a Point Spread Function (PSF) image.

According to the present disclosure, an apparatus, method, and system for generating a three-dimensional image using a coded phase mask can be expected.

According to the present disclosure, a three-dimensional image can be generated at reduced cost using a coded phase mask.

According to the present disclosure, an efficient and clear three-dimensional image can be generated using a coded phase mask.

The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

Figure 1 is a diagram showing the process of generating an image based on a coded aperture by borrowing a conventional pinhole camera structure.
FIG. 2 is a diagram of a Coded Aperture Correlation Holography (COACH) system capable of generating a three-dimensional image applicable to the present disclosure.
FIG. 3 is a diagram of a coded phase mask (CPM) used in FIG. 2 applicable to the present disclosure, pinhole brightness information, and object brightness information.
4 is a diagram of a monochromatic polarization image sensor applicable to the present disclosure.
Figure 5 is a diagram of a color polarization image sensor applicable to the present disclosure.
FIG. 6 is a diagram illustrating a process for obtaining complex data using a color polarization image sensor applicable to the present disclosure.
Figure 7 is a diagram of a system for manufacturing a scanning-type geometric phase element applicable to the present disclosure.
8 is a diagram of a geometric phase element applicable to the present disclosure.
FIG. 9 is a diagram of a 3D image generation system using a coded phase mask according to an embodiment of the present disclosure.
FIG. 10 is a diagram illustrating a data processing process of a 3D image generation system using a coded phase mask according to an embodiment of the present disclosure.
FIG. 11 is a diagram of a method for generating a 3D image using a coded phase mask according to an embodiment of the present disclosure.
FIG. 12 is a diagram of a 3D image generating device using a coded phase mask according to an embodiment of the present disclosure.
FIG. 13 is a diagram of a 3D image generating device using a coded phase mask according to another embodiment of the present disclosure.

Hereinafter, with reference to the attached drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily practice them. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein.

In describing embodiments of the present disclosure, if it is determined that detailed descriptions of known configurations or functions may obscure the gist of the present disclosure, detailed descriptions thereof will be omitted. In addition, in the drawings, parts that are not related to the description of the present disclosure are omitted, and similar parts are given similar reference numerals.

In the present disclosure, distinct components are intended to clearly explain each feature, and do not necessarily mean that the components are separated. That is, a plurality of components may be integrated to form one hardware or software unit, or one component may be distributed to form a plurality of hardware or software units. Accordingly, even if not specifically mentioned, such integrated or distributed embodiments are also included in the scope of the present disclosure.

In the present disclosure, components described in various embodiments do not necessarily mean essential components, and some may be optional components. Accordingly, embodiments consisting of a subset of the elements described in one embodiment are also included in the scope of the present disclosure. Additionally, embodiments that include other components in addition to the components described in the various embodiments are also included in the scope of the present disclosure.

In the present disclosure, components described in various embodiments do not necessarily mean essential components, and some may be optional components. Accordingly, embodiments consisting of a subset of the elements described in one embodiment are also included in the scope of the present disclosure. Additionally, embodiments that include other components in addition to the components described in the various embodiments are also included in the scope of the present disclosure.

In the present disclosure, terms such as first and second are used only for the purpose of distinguishing one component from other components, and do not limit the order or importance of the components unless specifically mentioned. Therefore, within the scope of the present disclosure, the first component in one embodiment may be referred to as a second component in another embodiment, and similarly, the second component in one embodiment may be referred to as the first component in another embodiment. It may also be called.

When a component of the present disclosure is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but other components may exist in between. It must be understood that it may be possible. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Meanwhile, in the present disclosure, the 3D image may include a complex hologram.

Meanwhile, in the present disclosure, image generation, imaging, and image restoration may be used interchangeably.

Meanwhile, in the present disclosure, imaging and image generation may be used interchangeably.

Meanwhile, in the present disclosure, point light source and pinhole may be used interchangeably.

Meanwhile, in the present disclosure, complex hologram, complex hologram data, and complex data may be used interchangeably.

Hereinafter, various embodiments of the present disclosure will be described with reference to the drawings.

FIG. 2 is a diagram of a Coded Aperture Correlation Holography (COACH) system capable of generating a three-dimensional image applicable to the present disclosure, and FIG. 3 is a diagram of a coded aperture correlation holography (COACH) system used in the spatial light modulator of the system of FIG. 2 applicable to the present disclosure. This is a diagram of a coded phase mask (CPM), brightness information of a point light source, and brightness information of an object.

More specifically, Figure 2 proposes a system that generates a digital three-dimensional image by introducing the principle of Figure 1 into computational photography and recognizing the defocus pattern by coded aperture. I'm doing it. In addition, (a) in Figure 3 shows an example of a coded phase mask, (b) shows brightness information of a point light source according to the coded phase mask as in (a), and (c) shows coding as in (a). This is a diagram about the brightness information of an object according to the phase mask.

Figure 2 shows a binary type coded mask, mainly used in the field of computational photography, as a random phase mask on a phase-only spatial light modulator (SLM). This is a drawing of a system named COACH that generates a 3D image (hologram) using .

Unlike the coded aperture camera of conventional computational photography, which requires a high-load iterative algorithm for 3D image restoration, the system of Figure 2 uses a point spread function (PSF) using an initial point light source. It is characterized by using a library and may include an objective lens (L2, 201), a polarizer (P1, 202), a spatial light modulator (SLM, 203), and an image sensor (204). Here, the spatial light modulator may be a phase-only spatial light modulator.

When external light is collected through the objective lens 201, the light is input to the spatial light modulator 203 through the polarizer 202, which filters only light components of a specific polarity.

Thereafter, an image including brightness information about a point light source can be obtained from the image sensor 204 by floating a coded phase mask (CPM) representing various phase angles from 0 to 360 degrees on the spatial light modulator 203. Here, the coded phase mask may be as shown in (a) of FIG. 3, and the image acquired from the image sensor may be as shown in (b) of FIG. 3. Additionally, the point light source can be moved in the depth direction on the optical axis, and an image containing brightness information of the point light source for each location can be generated. That is, since images containing brightness information of a point light source can be generated according to positional intervals, there may be a plurality of images. These images can be grouped to build a PSF library for that point light source.

Once the PSF library is built, an object is placed in the space where the point light source moved, and the previously used coded phase mask is floated on the spatial light modulator to create an image in which the brightness information of the object is recorded for that point, as shown in (c) of Figure 3. can be created. Here, in order to remove the noise component that appears due to the non-uniform brightness distribution of the point light source, three different coded phase masks can be used, based on three brightness information generated from the image sensor. Additionally, the phase angle distribution of the coded phase mask can be basically determined in the form of a random function. Three images with recorded brightness information about an object can be combined into one piece of complex data. One sheet of complex data is generated by the following formula.

here is the position vector of each pixel, obtained with the kth exposure. and depth value This is the brightness information corresponding to .

One piece of complex data about an object is convolutioned with a PSF image (e.g., (b) in Figure 3) in which the brightness information of a point light source corresponding to a specific position in the previously constructed PSF library is recorded, and the corresponding point is calculated. It is possible to create a 3D image of an object.

Meanwhile, the image sensor may be the monochromatic polarization image sensor of FIG. 4 or the color polarization image sensor of FIG. 5, which will be described in more detail with reference to FIGS. 4 and 5 below.

FIG. 4 is a diagram illustrating the pixel structure of a monochromatic polarization image sensor applicable to the present disclosure. More specifically, a structure in which photodiodes 403, to which a micro lens array 401 and a polarizer array 402 are attached, are arranged in two dimensions to obtain polarization information of the target object is used. This is a diagram showing the pixel structure of an image sensor.

As an example, a microlens array 401 may be attached on the polarizer array 402. For example, the polarizer array may be configured to have a 2 x 2 configuration for the pixels, and each polarizer may be rotated by 0 degrees, 45 degrees, -45 degrees, and 90 degrees, as mentioned above. As shown, this is to control the geometric phase of the light wave. Meanwhile, the degree to which each polarizing plate is rotated corresponds to one embodiment and is not limited to the above example.

Meanwhile, the image sensor in FIG. 4 is a monochromatic polarization image sensor, and the color polarization image sensor may mean that, for example, a color filter is additionally attached to the image sensor in FIG. 4. The color polarization image sensor will be described in more detail with reference to FIG. 5 below.

FIG. 5 is a diagram illustrating the pixel arrangement of a polarization image sensor applicable to the present disclosure. More specifically, this is a diagram to explain the pixel arrangement of a color polarization image sensor with a color filter attached along with the polarizer arrangement.

As an example, in the case of a color polarization image sensor, four polarization components can be expressed in three color channels (e.g., R, G, and B) with one shot of the target object (e.g., R, G, B). G, G, B). At this time, each color channel may consist of, for example, four pixels, and each pixel may be based on a different wire-grid direction. As an example, according to this, a total of four polarization components can be expressed as RGGB.

FIG. 6 is a diagram illustrating a process for obtaining complex data using a color polarization image sensor applicable to the present disclosure.

As an example, the color polarization image sensor of FIG. 6 may be the color polarization image sensor of FIG. 5, and may be a color polarization image sensor with a color filter attached in addition to the monochromatic polarization image sensor including the polarizer array of FIG. 4. and may be a sensor used in a 3D image generating device, method, and system using a coded phase mask of the present disclosure.

An image of brightness information about a target object recorded by a color polarization image sensor may be a raw image. As an example, a color polarization image sensor can express four polarization components as three colors (eg, R, G, and B). At this time, the polarizers may each have different phase values. For example, they may have phase values of 0 degrees, 45 degrees, 90 degrees, and 135 degrees, respectively.

A color polarization image sensor can distinguish polarization components according to phase and perform complex hologram data recombination. Based on this, each component can be collected for each color channel (demosaicing).

Figure 7 is a diagram of a system for manufacturing a scanning-type geometric phase element applicable to the present disclosure. Geometric phase refers to the phase affected by geometric movement in parameter space without change in optical retardation that occurs along the optical path or the axis on which polarized light is projected. Mainly in optics, it refers to a phase that occurs due to a change in polarization state, and is also called a PB (panchartnam-berry) phase. According to the recently introduced geometric phase device using liquid crystal, the phase can vary depending on the orientation angle of the liquid crystal, so a desired phase device can be freely created as long as there is information on the two-dimensional orientation angle of the liquid crystal. There are various manufacturing systems, but one of the systems applicable to this disclosure is the scanning method as shown in FIG. 7. A substrate coated with a photo-alignment material is placed at the position of the hologram 701 in FIG. 7, the sections are moved using a 2D positioning system 702, and the sections assigned to the sections are used using a 1D polarization control stage. The polarization state of incident light can be defined by the orientation angle. According to this, linear orientation information can be generated in the photo-alignment film on the substrate, and when liquid crystal is subsequently coated, phase devices with spatially different orientation angles can be manufactured through self-alignment.

8 is a diagram of a geometric phase element applicable to the present disclosure.

More specifically, a random phase retarder that can be generated by inputting coded phase mask information as shown in (a) of FIG. 3 as an input value to the 1D polarization control stage of the scanning type phase device manufacturing system as shown in FIG. 7. This is a diagram showing (random phase retarder). The random phase retarder of FIG. 8 has a random phase value arrangement, and by adjusting the layers of the liquid crystal, the liquid crystal in each section can have the same half-wave plate characteristics. Therefore, phase modulation from 0 to 360 degrees can be possible depending on the characteristics of the geometric phase. Figure 8 shows the orientation angle of the liquid crystal for each space based on the coded phase mask or the phase delay value accordingly using color.

FIG. 9 is a diagram of a 3D image generation system using a coded phase mask according to an embodiment of the present disclosure, and FIG. 10 is a diagram of a 3D image generation system using a coded phase mask according to an embodiment of the present disclosure. This is a diagram of the data processing process.

As an embodiment, a 3D image generation system using a coded phase mask includes an object to be captured 901, an objective lens 902, a phase mask 903, a polarization image sensor 904, and a 3D image generating device ( 905) may be included.

As an example, when explaining the data processing process of FIG. 10, the explanation is based on the 3D image generation system of FIG. 9 for clarity of explanation. However, since the 3D image generation system is only an embodiment of the present disclosure, It does not necessarily have to be configured as shown in Figure 9.

Additionally, the data processing process of FIG. 10 may be performed by a 3D image generating device as well as a 3D image generating system, and the 3D image generating device may be as shown in FIG. 12 . However, it is not limited to this.

As an embodiment, the 3D image generation system may perform the data processing process of FIG. 10 and/or the 3D image generation method of FIG. 11, and may perform the 3D image generation method of the 3D image generation device of FIG. 12. It may be included as device 905.

Prior to generating a 3D image, the 3D image generation system may perform a process (1001) of acquiring complex data for a point light source for the first time, as shown in FIG. 10. First, when the 3D image generation system is configured (1000), a point light source (pinhole) is installed (1010) and photographed (1011), and the polarized image is separated based on the captured image of the point light source to synthesize complex data ( 1012) You can. Polarization image separation is performed through a phase mask (FIG. 9, 903), and an image containing four modulated brightness information based on an image of a point light source obtained in one shooting is obtained using an image sensor (FIG. 9, 904). It may include a creation process. Meanwhile, this process can be repeated while moving the point light source (pinhole) at a certain interval in the depth direction on the optical axis (1013). For example, a PSF image can be created by moving a point light source from the depth position z0 to zN at an interval of Δz, and the set depth range and interval are the depth acquisition range and depth resolution of the system. It can be. Accordingly, it is possible to generate a PSF image for a point light source based on complex data synthesized at each location. Therefore, there may be multiple PSF images for a point light source. A PSF library 1002 can be created based on PSF images for a plurality of point light sources. The PSF library corresponding to the system's response characteristics for each point light source can then be used to restore a three-dimensional image from complex data about the object.

After the preceding PSF library construction process, the object to be photographed is installed (1003), photographed (1004), and when the reflected light of the object is incident on the objective lens 902, the light information is concentrated and can pass through the phase mask 903. there is. At this time, the phase mask 903 that modulates the phase of the reflected light of the object may be a coded phase mask (CPM) and may include a coded half-wave phase mask. there is. Here, the coded half-waveplate phase mask includes that shown in FIG. 8 and may include the geometric phase elements generated by FIG. 7 . The phase mask 903 can randomly modulate phase information of input light.

Thereafter, the polarization image sensor 904 may record the brightness information of the object as an image based on the reflected light of the object whose phase has been modulated from the coded phase mask. Based on the image generated as the object is photographed (1004), each polarized image can be separated, and based on this, the 3D image generating device 905 can synthesize complex data about the object (1005). More specifically, the polarization image sensor 904 converts the phase information of the input light at the position corresponding to the pixel into unique brightness information and records it in the pixel, based on the assigned angle of the polarizing plate corresponding to each pixel. You can. Here, the polarization image sensor 904 may include the monochromatic polarization image sensor of FIG. 4 and/or the color polarization image sensor of FIG. 5 . Complex data synthesized by the 3D image generating device 905 may be included in the object data 1006. When the polarization image sensor 904 is a color polarization image sensor, the synthesized complex data is converted to the complex data of FIG. 6. It may have been created by performing a process. The 3D image generating device 905 synthesizes complex data for the object based on the image of the object, and synthesizes complex data for the object and the PSF (Point Spread Function) image for the point light source. A 3D image can be created. When the polarization image sensor 904 is color, the brightness information for each color is recorded in the pixel, so the 3D image generating device 905 can restore a full-color 3D image of the object.

At this time, when the object is photographed at a specific depth zk, the 3D image generating device 905 generates a PSF image for a point light source photographed at a specific depth (e.g., car) and the object included in the PSF library. The three-dimensional image of the object can be restored (1007) based on the synthesized complex data. At this time, the combined complex data of the PSF image for the point light source and the object may be subjected to a convolution operation.

Meanwhile, in the above, the lens 902, the phase mask 903, the polarization image sensor 904, and the 3D image generating device 905 are separated for clarity of explanation, so the phase mask 903 and the polarization image sensor 904 may be comprised of a single device, or the phase mask 903 and/or the polarization image sensor 904 may be included in the 3D image generating device 905. However, since this corresponds to one embodiment, the present disclosure is not limited thereto.

FIG. 11 is a diagram of a method for generating a 3D image using a coded phase mask according to an embodiment of the present disclosure.

As an example, the 3D image generating method of FIG. 11 may be performed by the 3D image generating system of FIG. 9 or the 3D image generating device of FIG. 12, but is not limited thereto.

As another example, the method of generating a 3D image of FIG. 11 may be performed after step 1001 of the data processing process of FIG. 10 is performed, but is not limited thereto.

As an example, complex data about an object may be synthesized (S1101) based on an image obtained by photographing the object. Here, the image acquired by photographing an object may include an image including phase modulation information obtained by photographing an object and modulating it through a phase mask, as described above with reference to other drawings. Here, the phase mask includes a coded phase mask, and the coded phase mask may include a coded half-wave plate phase mask. Here, the coded half-wave plate phase mask may be based on a geometric phase element manufactured using a scanning method. As an example, there may be a plurality of images obtained by photographing an object, may be acquired through a single photographing using a polarization image sensor, and may include phase-modulated brightness information about the object. Additionally, the polarization image sensor used may be the monochromatic polarization image sensor of FIG. 4 or the color polarization image sensor of FIG. 5. If the polarization image sensor used is a color polarization image sensor, the phase-modulated brightness information for the object may include brightness information for each color. Additionally, the synthesis of complex data for an object may be based on the complex data synthesis process of FIG. 6 described above, but is not limited thereto. In addition, the process of synthesizing complex data about an object based on an image acquired by photographing the object (S1101) involves photographing the object (FIG. 10, 1004), separating the polarized image, and synthesizing complex data (FIG. 10, 1005). It may include the process of:

A three-dimensional image of the object can be generated (S1102) based on the synthesized complex data for the object and the PSF image for the point light source. The PSF image for a point light source may be a PSF image extracted from a PSF library for a point light source and corresponding to the position of the image obtained by photographing the object. The process of building the PSF library may be the same as the process described with reference to FIGS. 9 and 10, and may correspond to processes 1001 and 1002 of FIG. 10. More specifically, the PSF library may include a plurality of PSF images obtained by changing the depth of a point light source, as mentioned above. A PSF image obtained by changing the depth of a point light source may be generated by synthesizing complex data about the point light source based on images obtained by photographing the point light source at each depth. Here, complex data may be included in object data 1006 of FIG. 10. Additionally, a 3D image of an object can be generated through a convolution operation between a PSF image for a point light source and complex data for the object. This may correspond to 1007 in FIG. 10.

Meanwhile, since FIG. 11 is only an example of the present disclosure, the order of FIG. 11 may be changed, steps other than those of FIG. 11 may be included, or some steps may be excluded.

FIG. 12 is a diagram of a 3D image generating device using a coded phase mask according to an embodiment of the present disclosure.

As an embodiment, the apparatus 1200 for generating a 3D image using a coded phase mask may include a transceiver 1202 that transmits and receives signals and a processor 1201 that controls the transceiver 1202.

As an example, the 3D image generating device of FIG. 12 may execute the 3D image generating method using the coded phase mask of FIG. 11 and may be included in the system of FIG. 9, but is not limited thereto.

As an embodiment, the processor 1201 synthesizes complex data about an object based on an image obtained by photographing the object, and combines complex data about the object and a point spread function (PSF) image about a point light source. A three-dimensional image of the object can be created. An image obtained by photographing an object may be acquired by a polarization image sensor through a coded phase mask (CPM). The coded phase mask may be a coded half-wave phase mask, and the coded half-wave phase mask may be based on a geometric phase element manufactured by scanning, which is shown in FIGS. 5 and 5 The content described with reference to 6 may be included. There are a plurality of images obtained by photographing an object, acquired through a single photographing using a polarization image sensor, and may include phase-modulated brightness information about the object. Here, the mentioned polarization image sensor may include the polarization image sensor of FIGS. 4 and 5. In the case of a color polarization image sensor, the phase-modulated brightness information for the object may include brightness information for each color. . The PSF image for a point light source may be a PSF image corresponding to the position of an image obtained by photographing an object, extracted from a PSF library for a point light source. The PSF library may include PSF images obtained by changing the depth of a plurality of point light sources. A PSF image obtained by changing the depth of a point light source may be generated by synthesizing complex data about the point light source based on an image obtained by photographing the point light source at each depth. A 3D image of an object can be created through a convolution operation between the PSF image and complex data about the object.

Meanwhile, although not shown in FIG. 12, the 3D image generating device using a coded phase mask includes memory including RAM (Random Access Memory) and ROM (Read Only Memory), a user interface input device, a user interface output device, and storage. , may further include a network interface and a bus.

Additionally, there may be one or more processors 1201 in FIG. 12 and may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in memory and/or storage. Memory and storage may include various types of volatile or non-volatile storage media.

The existing system used a phase-only spatial light modulator to express the encoded phase mask, but since it is an active device, it requires a driving circuit and additional power, and there is a limit to increasing the diameter of the aperture, and the price is currently in the tens of millions of won. there is. However, according to the present disclosure, the system can be constructed simply and inexpensively by using an encoded half-wave plate phase mask using a geometric phase element.

In addition, according to the present disclosure, instead of acquiring an image using a time-sequential coded phase mask (CPM) using a polarization image sensor, four pieces of modulated brightness information can be obtained with a single exposure. Therefore, more efficient video acquisition may be possible.

In addition, according to the present disclosure, a very compact 3D image generation system without aberration caused by a lens or chromatic dispersion effect due to the wavelength dependence of diffraction can be manufactured as a highly mass-producible device without additional power consumption.

The steps of the method or algorithm described in connection with the embodiments disclosed herein may be implemented directly in hardware, software modules, or a combination of the two executed by a processor. Software modules may reside in a storage medium (i.e., memory and/or storage) such as RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM. An exemplary storage medium is coupled to a processor, the processor capable of reading information from and writing information to the storage medium. Alternatively, the storage medium may be integral with the processor. The processor and storage medium may reside within an application specific integrated circuit (ASIC). The ASIC may reside within the user terminal. Alternatively, the processor and storage medium may reside as separate components within the user terminal.

Exemplary methods of the present disclosure are expressed as a series of operations for clarity of explanation, but this is not intended to limit the order in which the steps are performed, and each step may be performed simultaneously or in a different order, if necessary. In order to implement the method according to the present disclosure, other steps may be included in addition to the exemplified steps, some steps may be excluded and the remaining steps may be included, or some steps may be excluded and additional other steps may be included.

The various embodiments of the present disclosure do not list all possible combinations but are intended to explain representative aspects of the present disclosure, and matters described in the various embodiments may be applied independently or in combination of two or more.

Additionally, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. For hardware implementation, one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general purpose It can be implemented by a processor (general processor), controller, microcontroller, microprocessor, etc. For example, it is obvious that it can be implemented in the form of a program stored in a non-transitory computer-readable medium that can be used at the end or edge, or in the form of a program stored in a non-transitory computer-readable medium that can be used at the edge or in the cloud. do. Additionally, it can be implemented by combining various hardware and software.

The scope of the present disclosure is software or machine-executable instructions (e.g., operating system, application, firmware, program, etc.) that allow operations according to the methods of various embodiments to be executed on a device or computer, and such software or It includes non-transitory computer-readable medium in which instructions, etc. are stored and can be executed on a device or computer.

For example, a 3D image generation program using a coded mask according to an embodiment of the present disclosure synthesizes complex data about an object based on an image obtained by photographing the object on a computer, and combines the synthesized complex data and It may be a program stored on a non-transitory computer-readable medium that generates a three-dimensional image of an object based on a PSF image for a point light source.

The present disclosure described above may be subject to various substitutions, modifications, and changes without departing from the technical spirit of the present disclosure to those skilled in the art to which the present disclosure pertains. Therefore, the scope of the present disclosure is limited to the above. It is not limited to one embodiment and the attached drawings.

901: Object to be photographed
902: Objective lens
903: Phase mask
904: Polarization image sensor
905: 3D image generating device

Claims (20)

A transceiver unit that transmits and receives signals;
Including a processor that controls the transceiver unit,
The processor is
Complex data for the object is synthesized based on the image obtained by photographing the object, and a three-dimensional image of the object is generated based on the complex data for the object and the PSF (Point Spread Function) image for the point light source,
The image obtained by photographing the object is an image acquired by a polarization image sensor through a coded phase mask (CPM),
The PSF (Point Spread Function) image for the point light source is a PSF image corresponding to the position of the image obtained by photographing the object,
The polarization image sensor,
A color polarization image sensor comprising a polarizer array composed of a plurality of polarizers at different angles for a plurality of pixels,
The image obtained by photographing the object includes phase-modulated brightness information including brightness information for each color of the object,
3D image generating device.
delete According to claim 1,
The coded phase mask is
A three-dimensional image generating device, a coded half-wave phase mask.
According to clause 3,
The coded half-wave plate phase mask is
A three-dimensional image generating device based on a geometric phase element manufactured by scanning method.
According to claim 1,
There are multiple images obtained by photographing the object,
A three-dimensional image generating device obtained through one shot using the polarization image sensor.
delete According to claim 1,
The PSF image for the point light source is,
A three-dimensional image generating device, which is a PSF image extracted from the PSF library for the point light source.
According to clause 7,
The PSF library is,
A three-dimensional image generating device comprising a PSF image obtained by changing the depth of a plurality of point light sources.
According to clause 7,
The PSF image obtained by changing the depth of the point light source is,
A three-dimensional image generating device that is generated by synthesizing complex data about the point light source based on images obtained by photographing the point light source at each depth.
According to claim 1,
The three-dimensional image of the object is,
A three-dimensional image generating device generated through a convolution operation of the PSF image and complex data about the object.
synthesizing complex data about an object based on an image obtained by photographing the object;
Generating a three-dimensional image of the object based on complex data for the object and a Point Spread Function (PSF) image for a point light source,
The image obtained by photographing the object is an image acquired by a polarization image sensor through a coded phase mask (CPM),
The PSF image for the point light source is a PSF image corresponding to the position of the image obtained by photographing the object,
The polarization image sensor,
A color polarization image sensor including a polarizer array composed of a plurality of polarizers at different angles for a plurality of pixels,
The image obtained by photographing the object includes phase-modulated brightness information including brightness information for each color of the object,
How to create a 3D image.
According to claim 11,
The coded phase mask is
A method of creating a three-dimensional image, a coded half-wave phase mask.
According to claim 12,
The coded half-wave plate phase mask is
A three-dimensional image generation method based on geometric phase elements produced by scanning.
According to claim 11,
There are multiple images obtained by photographing the object,
A method of generating a three-dimensional image, obtained through one shot using the polarization image sensor.
delete According to claim 11,
The PSF image for the point light source is,
A method of generating a three-dimensional image, which is a PSF image corresponding to the position of the image obtained by photographing the object, extracted from the PSF library for the point light source.
According to claim 16,
The PSF library is,
A three-dimensional image generation method comprising a PSF image obtained by changing the depth of a plurality of point light sources.
According to claim 16,
The PSF image obtained by changing the depth of the point light source is,
A three-dimensional image generation method that is generated by synthesizing complex data about the point light source based on images obtained by photographing the point light source at each depth.
According to claim 11,
The three-dimensional image of the object is,
A three-dimensional image generation method generated through a convolution operation of the PSF image and complex data for the object.
A coded phase mask (CPM) that modulates the phase of reflected light from an object;
a polarization image sensor that records brightness information of the object as an image based on reflected light of the object whose phase has been modulated from the coded phase mask; and
A three-dimensional image that synthesizes complex data about the object based on the image of the object and generates a three-dimensional image of the object based on complex data about the object and a PSF (Point Spread Function) image for a point light source. Including a generating device;
The polarization image sensor,
A color polarization image sensor comprising a polarizer array composed of a plurality of polarizers at different angles for a plurality of pixels,
The image obtained by photographing the object includes phase-modulated brightness information including brightness information for each color of the object,
3D image generation system.

Images