KR20210151474A - Ultrathin camera device using microlens array, and Multi-functional imaging method using the same - Google Patents

Abstract

An ultrathin camera device includes: an optical module including a microlens array in which microlenses are arranged; an image sensor which outputs electrical image signals by sensing light coming through the microlens array; spacers which form a focal length by separating the optical module from the image sensor; and a processor which outputs a final image by reconstructing array images generated from the image signals with a designated imaging process depending on a distance at which the object is located, wherein each microlens convexly protrudes toward the image sensor. Therefore, it is possible to simultaneously image objects located in multiple planes using the ultrathin camera device.

Description

Ultra-thin camera device using microlens array, and multi-functional imaging method thereof

This disclosure relates to imaging technology.

Recently, in the field of vision technology, studies are being conducted to develop various applications using acquired images. However, there is a limit to the applications that can be provided with a single lens or a dual lens.

The conventional lens has a longer focal length than the imaging plane of a near object, so there is a problem of out of focus. Therefore, the existing camera has a limitation in performing both long-distance imaging and near-field imaging at the same time. To compensate for this, short-distance, medium-distance, and long-distance imaging can be performed using micro lenses having different focal lengths, but there is a problem in that the resolution is lowered when reconstructing images due to different viewing angles of images acquired from each lens.

In conventional cameras, image distortion may occur due to the rolling shutter effect. On the other hand, global shutter image sensors require pixel-level memory than rolling shutter image sensors, have a low fill factor of photodiodes, increase noise during imaging in dark areas, and are expensive to manufacture due to their complex structure.

The task to be solved is to provide various applications by simultaneously imaging objects located in multiple planes at different distances using a microlens array having a very short focal length, and variously processing the acquired array images. To provide an ultra-thin camera device device.

The problem to be solved is a microscopic image, a 3D depth image, and a high dynamic range (HDR) image using an ultra-thin camera device that provides near-field imaging, intermediate-range imaging, and far-field imaging. ), etc. to provide a multifunctional imaging method.

The task to be solved is an ultra-thin camera that uses a rolling shutter-type image sensor and a microlens array to acquire images of different instants and combines them to generate high-frame rate images/high-speed images to provide the device.

As a camera device according to an embodiment, an optical module including a microlens array in which microlenses are arranged, an image sensor sensing light entering through the microlens array and outputting electrical image signals, the optical module and the image A processor for outputting a final image by reconstructing the spacers to form a focal length by separating the sensor, and an imaging method designated according to the distance at which the object is located, the array images generated by the image signals. Each microlens is convexly protruded toward the image sensor.

The optical module includes a transparent substrate, a pinhole array layer that transmits light entering through the transparent substrate to the microlens array through pinholes filled with a transparent material, and the microlens array arranged to correspond to the pinholes of the pinhole array layer may include.

Each spacer is a micro-pole to which an adhesive is applied, and one end may be fixed to the image sensor and the other end may be fixed to the optical module.

The processor may generate a microscope image by stitching the array images captured in the near plane.

The processor may generate a 3D depth image by estimating a depth based on a parallax of the array images captured in a mid-range plane.

The processor may generate a high-resolution image by superimposing the array images captured in a distant plane.

The processor may combine the array images captured at different moments to produce high frame rate images.

The processor may determine a distance plane in which the object is located by using a visual difference seen by the microlenses.

When the distance between the two microlenses, the focal length of the microlens, and the distance calculated using the visual difference are within the imaging range of one microlens, the processor may determine that the object is for the near plane.

A multifunctional imaging method of a processor according to an embodiment, comprising: acquiring array images captured through microlenses arranged in a plane; and reconstructing the array images to obtain a microscope image, a 3D depth image, a high-resolution image, and a high-frame rate image generating at least one of them.

In the generating of the at least one, when partial regions of an object are captured through at least some microlenses, the array images are stitched and connected to generate the microscopic image, or the viewing direction of the object viewed through each microlens is different. In this case, the 3D depth image is generated by estimating the depth based on the parallax of the array images, or when the viewing directions of objects seen through each microlens are the same, the high-resolution image can be generated by overlapping the array images .

The generating of the at least one may generate the high frame rate images by combining the array images captured at different moments through each microlens.

In the generating of the at least one, a distance plane on which an object is located is determined using a visual difference seen by the microlenses, and the array images are reconstructed by an imaging method specified in the determined distance plane to generate a final image.

The array images are captured through the microlenses having a fixed focal length, and a method of reconstructing the array images may vary according to a visual difference seen from the microlenses.

The microlenses may be formed to match the pinholes in a pinhole array layer in which pinholes filled with a transparent material are formed, and may refract light entering through the pinholes to form a focus on the image sensor.

According to an embodiment, the ultra-thin camera device can be used to simultaneously image objects located in multiple planes, and to image a very fast moving object at different instants, so that microscopic imaging at a short distance, three-dimensional imaging at a medium distance, and three-dimensional imaging at a distance It can provide multifunctional applications such as high-resolution imaging and high-speed imaging.

According to an embodiment, the ultra-thin camera device has a very short focal length, so that not only long-distance imaging but also very close objects can be imaged without defocusing. According to an embodiment, the ultra-thin camera device may perform close-up photography for various uses, such as fingerprint recognition and skin imaging, and may generate a microscope image.

According to an embodiment, since the ultra-thin camera device has the same viewing angle and the same pitch, the distance of the object may be predicted through the degree of overlap depending on the location of the object. According to an embodiment, the ultra-thin camera device may generate a 3D image by reconstructing the images based on the distance information of the object.

According to an embodiment, the ultra-thin camera device may generate a high-resolution image by superimposing images obtained from lenses.

According to an embodiment, the ultra-thin camera device may generate high frame rate images by momentarily capturing a very fast moving object while reducing image distortion caused by the rolling shutter effect.

According to an embodiment, since the ultra-thin camera device is manufactured to be ultra-thin, it may be mounted on a micro device such as a smartphone or a drone.

1 is a configuration diagram of an ultra-thin camera device according to an embodiment.
2 is a view for explaining a multi-function application of the ultra-thin camera device according to an embodiment.
3 is a view for explaining a method of determining a distance range of a multi-plane using a microlens array according to an embodiment.
4 is a view for explaining short-distance imaging of an ultra-thin camera device according to an embodiment.
5 is a view for explaining a short-distance imaging result of the ultra-thin camera device according to an exemplary embodiment.
6 and 7 are diagrams for explaining a mid-range imaging result of the ultra-thin camera device according to an exemplary embodiment.
8 is a view for explaining a long-distance imaging result of the ultra-thin camera device according to an embodiment.
9 is a view for explaining the rolling shutter phenomenon.
10 and 11 are diagrams for explaining a high frame rate imaging result of the ultra-thin camera device according to an exemplary embodiment.
12 is a view for explaining a packaging method of an ultra-thin camera device according to an embodiment.
13 is a flowchart of a multi-function imaging method of an ultra-thin camera device according to an embodiment.

Hereinafter, with reference to the accompanying drawings, the embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part "includes" a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated. In addition, terms such as “…unit”, “…group”, and “module” described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software. have.

1 is a configuration diagram of an ultra-thin camera device according to an embodiment.

Referring to FIG. 1 , the ultra-thin camera device 100 simultaneously images objects located in multiple planes at different distances using a microlens array, and variously processes the array images acquired through multi-plane imaging. Thus, various applications can be provided. The ultra-thin camera device 100 includes, for example, a microscopic image generated through near-field imaging, a 3D depth image generated through medium-range imaging, and a high-resolution image generated through long-distance imaging (high dynamic). range (HDR) image) and the like, and high frame rate images (high-speed images) composed of images captured at different moments.

This ultra-thin camera device 100 is an optical module including a transparent substrate (glass wafer) 110 and a microlens array 120, sensing the light entering through the microlens array 120 to output an electrical image signal. It includes an image sensor 130 , and a spacer 140 spaced apart from the optical module and the image sensor 130 to form a focal length. The ultra-thin camera device 100 may further include a processor 200 for processing image signals sensed by the image sensor 130 . For convenience, the optical module including the microlens array 120 , the image sensor 130 , and the spacer 140 are referred to as a packaged camera module. The ultra-thin camera device 100 may further include an image signal and a memory for storing the image.

The optical module of the ultra-thin camera device 100 may further include a pinhole array layer 150 that transmits light entering through the transparent substrate 110 to the microlens array 120 . The pinhole array layer 150 is an absorption layer made of a material that absorbs light, and pinholes filled with a transparent material are formed to correspond to the pattern of the microlens array 120 . Light passes only through the pinhole of a transparent material, and the light transmitted to the microlens array 120 by absorbing light other than the pinhole is blocked. The pinhole array layer 150 may have a multilayer structure in which an absorption layer absorbing light and a transparent layer passing light are alternately stacked. In this case, in the absorption layer, an opening may be formed by a mask pattern manufactured to correspond to the pattern of the microlens array 120 . The pinhole may serve as the aperture of the camera.

The transparent substrate 110 may be a substrate made of a transparent glass material that allows light to pass therethrough.

The microlens array 120 has a structure in which a plurality of micro-scale microlenses 121 are arranged, and may include, for example, microlenses arranged in a planar grid pattern. Each microlens 121 is aligned with the pinhole of the pinhole array layer 150 and has a convexly protruding shape. The microlens 121 refracts light entering through the transparent substrate 110 to form a focus on the image sensor 130 , and only light entering through the pinhole of the pinhole array layer 150 may be received. In this case, since the microlens 121 has a very short focal length, an ultra-thin camera can be manufactured.

The microlens array 120 is packaged such that the convex surface of the microlens 121 faces the image sensor 130 . Through this, the microlens array 120 may focus the light entering through the pinhole array layer 150 to the image sensor 130 .

The image sensor 130 is spaced apart from the pinhole array layer 150 by the spacer 140 , and receives light passing through the microlens array 120 . The image sensor 130 may be, for example, a CMOS image sensor, but the types of image sensors may be various. The image sensor 130 is a device that converts light entering through the microlens array 120 into an electrical signal, and detects the light corresponding to the red channel, the green channel, and the blue channel through the RGB color filter to increase the intensity of the corresponding color. can output a signal representing

The spacer 140 couples the optical module including the microlens array 120 and the image sensor 130 to be spaced apart by a focal length. The spacer 140 is made of micro-pillars, and both ends of the micro-pillars may be attached to the image sensor 130 and the optical module through an epoxy adhesive. Through this, ultra-thin camera device packaging may be possible. Here, the optical module to which the spacer 140 is attached may be the transparent substrate 110 on which the microlens array 120 is formed or the pinhole array layer 150 . The spacer 140 may be designed in various ways. The height of the spacer 140 is the same as the focal length of the microlens 121 , and the tolerance of the height of the fine pillars may be set according to the depth of focus of the microlens 121 .

On the other hand, the conventional camera using a microlens array has a limitation in performing short-range imaging because it is difficult to package a lens having a short focal length. That is, in order to perform close-range imaging without defocus, a lens with a focal length of about 170 μm must be precisely packaged with the image sensor. In order to solve this problem, the spacer 140 is fixed to the image sensor 130 by applying epoxy using a dispenser, and then aligning the microlens array with the image sensor 130 using a flip-chip bonding technique. The optical module including 120 and the image sensor 130 may be precisely packaged.

The processor 200 may generate a plurality of images (array images) imaged by a plurality of microlenses by using the image signals output from the image sensor 130 . In addition, the processor 200 may process the array images with various image processing algorithms to generate a microscope image, a 3D depth image, a high-resolution image, and a high frame rate image. In order to distinguish the image acquired from the microlens and the image generated by reconstructing the images acquired from the microlenses, the image acquired from the microlens is called an array image. In addition, in order to classify imaging through each microlens, a channel identifier may be assigned to each microlens to distinguish.

Meanwhile, the processor 200 may be divided into a processor that generates array images by processing electrical image signals obtained from the image sensor 130 , and a processor that processes the array images using various image processing algorithms. For example, the main processor of the device equipped with the ultra-thin camera device 100 executes a multi-function imaging application, and processes the array images with various image processing algorithms, such as a microscope image, a 3D depth image, a high-resolution image, and a high frame rate. Images can be generated, but the description assumes that the processor 200 performs multi-function imaging processing.

2 is a view for explaining a multi-function application of the ultra-thin camera device according to an embodiment.

Referring to FIG. 2 , the insect captures many scenes of an object at once through visual information obtained from facet lenses. The ultra-thin camera device 100 is a device simulating such insect stereosis, and combines the microlens array 120 in which a plurality of microlenses 121 corresponding to facet lenses are arranged with the image sensor 130 . to image In this case, the pigment cells blocking light between the lens and the lens may be simulated as the pinhole array layer 150 in which pinholes are formed in the light absorbing layer.

The ultra-thin camera device 100 may capture focused array images through the microlens array 120 , and the field-of-view seen by each microlens is different according to the distance from the object. Accordingly, the ultra-thin camera apparatus 100 may reconstruct the captured array images with different observation ranges according to distances in different ways to image objects located in multiple planes of short, medium, and long distances.

For example, in the case of the object 10 located in a short distance, a partial region of the object 10 is captured through each microlens that closes the object. Accordingly, the ultra-thin camera device 100 may generate one microscope image 300 by combining the array images obtained by obtaining a partial area of an object. For example, the ultra-thin camera device 100 uses an image stitching technique to find similar patterns in images acquired from each microlens, and to connect similar parts to generate one large panoramic image. can

In the case of the objects 20 and 22 located at an intermediate distance, the viewing directions of the objects viewed through each microlens are different. Accordingly, the ultra-thin camera device 100 extracts disparity of the array images and estimates the depth (distance) of each object based on the disparity. The ultra-thin camera device 100 may generate a 3D depth image 400 including depth information. The ultra-thin camera device 100 may extract objects from images obtained from each microlens, and may extract parallax of each object by overlapping array images.

In the case of the distant object 30, the object 30 is captured in the same viewing direction through each microlens. That is, each array image may include the object 30 identically. Accordingly, the ultra-thin camera device 100 may generate a high-resolution image 500 by overlapping the array images. After aligning the array images with respect to the center of the microlens, the ultra-thin camera device 100 may upsample the array images through interpolation to increase image resolution.

In addition, in the case of the distant object 32, the object 32 may be sequentially captured through each microlens in the rolling shutter direction by the rolling shutter effect of the image sensor. The ultra-thin camera device 100 may generate high frame rate images 600 by combining the captured array images in a relatively short time.

3 is a view for explaining a method of determining a distance range of a multi-plane using a microlens array according to an embodiment.

Referring to FIG. 3 , a plane on which an object is positioned may be divided into a short-distance, a medium-distance, and a long-distance using a visual difference seen from each lens. The ultra-thin camera device 100 may predict the distance of the object through the degree of overlap according to the location of the object.

Referring to (a), the visual difference (x-x') between the two lenses may be defined as in Equation 1. In Equation 1, B is the baseline distance between the two lenses, f is the focal length, and z is the vertical distance from the lens to the object P.

Multi-planes can be divided into distance ranges as shown in Table 1. In Table 1, d is the imaging range of one lens.

multi-plane distance range close range

mid-range

far away

Referring to (b), the areas viewed by the lenses may overlap. As the distance from the lens increases, the overlapping range increases.

As such, the ultra-thin camera device 100 may perform short-range imaging as well as intermediate and long-distance imaging by using the microlens array 120 having a very short focal length. It is not easy to package the microlens array 120 , but the microlens array 120 and the image sensor 130 can be packaged using the spacer 140 mounted by flip-chip bonding.

FIG. 4 is a diagram illustrating short-range imaging of the ultra-thin camera device according to an embodiment, and FIG. 5 is a diagram illustrating a short-distance imaging result of the ultra-thin camera device according to an embodiment.

Referring to FIG. 4 , the ultra-thin camera device 100 has a small diameter aperture through the pinhole array layer 150 and the microlens array 120 .

When looking at a plane focused according to the size of the aperture and object distance, commercial cameras capture blurry images due to out of focus at near objects, whereas the ultra-thin camera device 100 has all Clear images can be obtained at close range without defocusing. According to the Gaussian lens formula (1/a+1/b=1/f), commercial lenses with long focal lengths become farther from the focal length as the object gets closer because the distance from the image plane changes as the distance of the object gets closer. On the other hand, a microlens with a short focal length has a small change in the distance from the image plane even when the distance to the object is close, so that it does not move away from the focal length even when the object approaches.

As such, the short focal length of the microlens enables the ultra-thin camera device 100 to perform all-in-focus imaging from a short distance to a far distance, and may reduce a minimum object distance (MOD). Here, the minimum object distance means the object distance closest to the camera from which a clear image can be obtained.

Referring to FIG. 5A , the ultra-thin camera device 100 may image an object located in a short distance through each microlens of the microlens array 120 . The short-range imaging result is array images obtained through microlenses, and each array image may include a partial area of an object.

Referring to (b), the ultra-thin camera device 100 may generate a single microscope image by combining array images obtained through short-range imaging. The ultra-thin camera device 100 may generate one large panoramic image by connecting images acquired from each microlens using an image stitching technique.

6 and 7 are diagrams for explaining a mid-range imaging result of the ultra-thin camera device according to an exemplary embodiment.

Referring to FIG. 6 , (a) is a result of imaging objects of various distances located at an intermediate distance through the ultra-thin camera device 100 . Array images have different viewing directions from each microlens. For example, it can be seen that the range in which the block of number 2 overlaps the block of number 1 looks different depending on the lens position. Here, a channel may be provided to distinguish an array image acquired through a microlens.

(b) is a graph of the relationship between disparity according to the object distance, and (c) is a diagram for explaining the relationship between disparity and object distance occurring between microlenses. As a result of checking the disparity of each microlens using objects located at various distances in the intermediate range, it can be seen that the disparity increases as the distance between the lenses increases, and the disparity increases as the distance between the objects increases.

Referring to FIG. 7 , (a) is a result of Red-cyan anaglyph using medium-range imaging results, and (b) is a result of depth estimation obtained through medium-range imaging.

The ultra-thin camera device 100 may extract a red-cyan anaglyph disparity by using the array images obtained by medium-range imaging, and estimate the depth (distance) of each object based on the disparity. . The ultra-thin camera device 100 may generate a 3D depth image including depth information.

8 is a view for explaining a long-distance imaging result of the ultra-thin camera device according to an embodiment.

Referring to FIG. 8 , (a) is a result of imaging a distant object through the ultra-thin camera device 100, (b) is an image obtained through a specific lens among the array images, (c) is an array As an image obtained by combining images, it can be seen that contrast and sharpness are increased.

The ultra-thin camera device 100 may generate a high-resolution image by overlapping the array images.

9 is a view for explaining a rolling shutter phenomenon, and each of FIGS. 10 and 11 is a view for explaining a high frame rate imaging result of the ultra-thin camera device according to an embodiment.

Referring to FIG. 9 , (a) is an imaging method of a single-lens camera, and (b) is a diagram conceptually explaining an imaging method of the ultra-thin camera device 100 using a microlens array.

A single-lens camera uses one image sensor for one lens, and a rolling shutter type image sensor that is mainly used stores images in chronological order. Therefore, single-lens cameras cause image distortion in the case of fast-moving objects.

On the other hand, since the ultra-thin camera device 100 images light from several microlenses with one image sensor, it is possible to acquire images from each microlens in a relatively short time.

Accordingly, the ultra-thin camera device 100 may obtain array images clearly captured by each microlens in the direction of the rolling shutter.

Therefore, the ultra-thin camera device 100 can image a very fast moving object without distortion, and at the same time capture different instantaneous objects using the microlens array 120 , and combine the array images at a high frame rate. You can provide an image.

Referring to FIG. 10 , (a) is a rotating fan, (b) is an image taken with a shutter speed of 1 ms in a general single-lens camera, (c) is a shutter speed of 1 ms in the ultra-thin camera device 100 Array images taken with , (d) are images obtained by merging array images of each row by HDR merging technology.

In (b), the shape of the fan is distorted due to the effect of the rolling shutter effect, and one star appears as three. However, it can be seen that in each array image of (c), there is little distortion in the form of a fan, and one star is visible.

Referring to FIG. 11 , (a) is a color disc divided into 6 equal parts, (b) is an image obtained by a single lens camera, and (ce) is an image obtained by an ultra-thin camera device 100 having different lens diameters . The lens diameter is (c) 300 μm. (d) 150 μm, (e) 100 μm. Comparing (c-e), it can be seen that the color area uniformity of the original plate increases as the lens diameter decreases. The image (c) photographed with a 300 μm lens has a large green area, whereas the image (e) photographed with a 100 μm lens has an almost uniform area of the color disk.

12 is a view for explaining a packaging method of an ultra-thin camera device according to an embodiment.

Referring to FIG. 12 , in order to manufacture the ultra-thin camera device 100 having a thin camera, it is important to package a microlens with a short focal length and an image sensor. In FIG. 12 , it is described that the microlens array 120 is attached to the transparent substrate 110 , but an optical module including the microlens array 120 may be designed in various ways as shown in FIG. 1 .

Epoxy, which is an adhesive, is applied to one end of the spacer 140 made of fine pillars by a dispensing technique, and the spacer 140 is fixed to the image sensor 130 . Epoxy is similarly applied to the other end of the spacer 140 , and the optical module including the microlens array 120 is fixed on the spacer 140 . In this case, flip-chip bonding equipment may be used, and the microlens array 120 and the image sensor 130 are aligned and packaged.

The spacer 140 is positioned at the corner to fix the image sensor and the lens, but the position may be changed depending on the shape of the lens. The height of the spacer 140 is the focal length of the lens, and a tolerance of the height of the fine pillar may be set according to the depth of focus of the lens. The total height of the adhesive and micro-pillars should be made equal to the focal length.

The manufactured ultra-thin camera device may have a total track length of 1 mm or less.

13 is a flowchart of a multi-function imaging method of an ultra-thin camera device according to an embodiment.

Referring to FIG. 13 , the processor 200 acquires array images captured through the microlens array 120 ( S110 ).

The processor 200 determines a distance plane (eg, near, intermediate, and far) where an object is located by using the visual difference seen from the microlenses, and reconstructs the array images by an imaging method specified in the distance plane to obtain a final image. output (S120).

When the distance plane on which the object is located is near, an image processing algorithm for generating a panoramic image/microscopic image by stitching and connecting the array images may be designated. When the distance plane on which the object is located is an intermediate distance, an image processing algorithm for generating a 3D depth image by estimating the depth of the object based on the parallax of the array images may be designated. When the distance plane where the object is located is far away, an image processing algorithm that creates a high-resolution image (HDR image) by superimposing the array images may be specified. In addition, when the distance plane on which the object is located is far, an image processing algorithm that generates array images as high frame rate images (high speed images) may be specified.

As such, the ultra-thin camera device 100 can simultaneously image objects in various planes without changing the focal length by using the microlens array 120 , and reconstruct the array images with an image processing algorithm according to the distance plane for various applications. can provide

According to an embodiment, an ultra-thin camera device can be used to simultaneously image objects located in multiple planes, and to image a very fast moving object at different instants, so that microscopic imaging at a short distance, three-dimensional imaging at a medium distance, and a long distance It can provide multifunctional applications such as high-resolution imaging and high-speed imaging. According to an embodiment, the ultra-thin camera device has a very short focal length, so that not only long-distance imaging but also very close objects can be imaged without defocusing. According to an embodiment, the ultra-thin camera device may be used for various microscopic imaging, such as fingerprint recognition and skin imaging. According to an embodiment, since the ultra-thin camera device has the same viewing angle and the same pitch, the distance of the object may be predicted through the degree of overlap according to the location of the object. According to an embodiment, the ultra-thin camera device may generate a 3D image by reconstructing the images based on the distance information of the object. According to an embodiment, the ultra-thin camera device may generate a high-resolution image by superimposing images obtained from lenses. According to an embodiment, the ultra-thin camera device may generate high frame rate imaging by momentarily capturing a very fast moving object while reducing image distortion caused by the rolling shutter effect. According to an embodiment, since the ultra-thin camera device is manufactured to be ultra-thin, it may be mounted on a micro device such as a smartphone or a drone.

The embodiment of the present invention described above is not implemented only through the apparatus and method, and may be implemented through a program for realizing a function corresponding to the configuration of the embodiment of the present invention or a recording medium in which the program is recorded.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto. is within the scope of the right.

Claims (15)

An optical module comprising a microlens array in which microlenses are arranged;
an image sensor that senses light entering through the microlens array and outputs electrical image signals;
Spacers spaced apart from the optical module and the image sensor to form a focal length, and
a processor for outputting a final image by reconstructing the array images generated by the image signals by a designated imaging method according to the distance at which the object is located;
Each microlens has a shape protruding convexly toward the image sensor. In claim 1,
The optical module is
transparent substrate,
A pinhole array layer that transmits light entering through the transparent substrate to the microlens array through pinholes filled with a transparent material, and
the microlens array arranged to correspond to the pinholes of the pinhole array layer
comprising, a camera device. In claim 1,
Each spacer is a micro-pole to which an adhesive is applied, and one end is fixed to the image sensor and the other end is fixed to the optical module. In claim 1,
the processor is
A camera device for generating a microscope image by stitching and connecting the array images captured in a near plane. In claim 1,
the processor is
A camera device for generating a 3D depth image by estimating a depth based on a parallax of the array images captured in a mid-range plane. In claim 1,
the processor is
A camera device for generating a high-resolution image by superimposing the array images captured in a distant plane. In claim 1,
the processor is
A camera device for generating high framerate images by combining the array images captured at different moments. In claim 1,
the processor is
A camera device for determining a distance plane on which an object is located by using a visual difference seen by the microlenses. In claim 8,
the processor is
If the distance between the two microlenses, the focal length of the microlens, and the distance calculated using the visual difference are within the imaging range of one microlens, it is determined that the object is for the near plane. A method for multi-function imaging of a processor, comprising:
acquiring array images captured through microlenses arranged in a plane; and
reconstructing the array images to generate at least one of a microscope image, a 3D depth image, a high resolution image, and a high frame rate image;
A multifunctional imaging method comprising a. In claim 10,
The step of creating the at least one
When partial regions of an object are captured through at least some microlenses, the array images are stitched to create the microscopic image, or
When the viewing direction of the object seen through each microlens is different, the 3D depth image is generated by estimating the depth based on the parallax of the array images,
When the field of view of an object seen through each microlens is the same, the high-resolution image is generated by superimposing the array images. In claim 10,
The step of creating the at least one
Combining the array images captured at different moments through each microlens to produce the high frame rate images. In claim 10,
The step of creating the at least one
A multifunctional imaging method of determining a distance plane where an object is located by using the visual difference seen from the microlenses, and reconstructing the array images by an imaging method designated on the determined distance plane to generate a final image. In claim 10,
The array images are captured through the microlens having a fixed focal length,
A method for reconstructing the array images is different according to a visual difference seen in the microlenses. 15. In claim 14,
The microlenses are
A multifunctional imaging method for forming a focus on an image sensor by forming pinholes filled with a transparent material to match the pinholes in the formed pinhole array layer, and refracting light entering through the pinholes.

Images