KR20150107571A - Image pickup apparatus and image pickup method of generating image having depth information - Google Patents

Abstract

Disclosed are an image pickup apparatus of generating an image having depth information and an image pickup method to improve the resolution of an image having depth information generated by a light field technique. An image pickup apparatus according to the disclosed embodiment obtains images having different depths of a focus by moving a main lens, and can extract depth information from each of the images obtained by that. According to an image pickup apparatus and an image pickup method according to the disclosed embodiment, several imaging processes are performed by changing the position of the main lens. Thereby, different images can be obtained. So, it has no need to increase the size of a micro lens to increase the number of images at different points.

Description

TECHNICAL FIELD The present invention relates to an image capturing apparatus and an image capturing apparatus for generating an image having depth information,

The disclosed embodiments relate to an image acquiring apparatus and an image acquiring method for generating an image having depth information, and more particularly, to an image acquiring apparatus and an image acquiring method capable of improving resolution of an image having depth information generated using a light field technique, And an image acquiring method.

A typical two-dimensional (2D) camera acquires intensity information of light received through an objective lens with an image sensor to generate an image. For example, the light intensity information for one point of the subject can be obtained by collecting a plurality of rays coming from a point of the object on a point of the image sensor by using an objective lens and accumulating for a predetermined time, It is possible to make one image by using the intensity information of the light obtained from the plurality of pixels of the image. However, with this 2D camera image acquisition method, information on the intensity and direction of individual rays coming from a point of the object can not be obtained.

A light field technique is a technique that allows to generate images corresponding to an arbitrary point of view or an arbitrary focus by acquiring information about the individual intensities and directions of a plurality of rays from a point of the object to be. The light field technique has a refocusing effect that can arbitrarily focus on all subjects in a three-dimensional (3D) camera or a field of view, which can obtain information on a plurality of viewpoints of a subject and depth information of the subject The camera can be implemented.

A camera to which the light field technique is applied can be typically implemented using a main lens and a microlens array. For example, a microlens array having a plurality of microlenses arranged two-dimensionally can be disposed between the main lens and the image sensor. Here, one microlens in the microlens array may correspond to a plurality of pixels of the image sensor. Accordingly, images at different points in time can be obtained from a plurality of pixels corresponding to one micro lens. For example, when one microlens covers 7 占 7 pixels, 7 占 7 images having different viewpoints can be obtained at the same time.

However, in a light field camera, the resolution of an image is determined not by the pixel pitch of the image sensor but by the pitch of the microlens. For example, as the size of the microlens increases, the resolution of the image decreases. When the size of the microlens decreases, the resolution of the image increases. Therefore, if the size of the microlens is increased to increase the number of images having different viewpoints, the resolution of the image is lowered. If the size of the microlens is decreased to increase the resolution of the image, As shown in FIG. As a result, a trade-off occurs between the resolution of the image and the parallax detection capability.

There is provided an image acquiring apparatus and an image acquiring method capable of improving resolution of an image having depth information generated using a light field technique.

According to an embodiment of the present invention, there is provided an image acquisition apparatus including: a main lens for condensing incident light; An image sensor having a plurality of two-dimensionally arranged pixels for detecting an incident light to form an image; A microlens array disposed between the main lens and the image sensor, the microlens array having a plurality of microlenses arranged two-dimensionally; And a control unit for receiving an image signal from the image sensor and generating an image, wherein the control unit acquires a plurality of images having different subject depths by changing a distance between the main lens and the image sensor, And generate at least one depth map each from at least one of the images.

Each of the microlenses may be arranged to correspond to at least two pixels of the image sensor.

The control unit is configured to generate a depth map by using outputs of at least two pixels arranged corresponding to the same microlens, and to generate an image by summing outputs of at least two pixels arranged corresponding to the same microlens .

In addition, the controller may be configured to vary the distance between the main lens and the image sensor, with the depth of focus (DOF) as a basic step unit.

Wherein the control unit acquires an image through the image sensor and generates a depth map every time the distance between the main lens and the image sensor is changed in increments of depth of focus, May be configured to acquire the correct image.

The control unit may be configured to store a subject area having a minimum depth value in each depth map generated each time the distance between the main lens and the image sensor is changed in units of depth of focus.

The controller may select a depth map having a minimum depth value with respect to the selected subject area and output an image corresponding to the selected depth map when a specific subject area is selected in the plurality of images.

And a distance between the main lens and the image sensor when focusing on the shortest focal length is Dh and a distance between the main lens and the image sensor when the focal point is focused on the shortest focal distance is Dc, And the distance between the main lens and the image sensor is changed in units of depth of focus between Dh and Dc.

Wherein the control unit initially selects the distance Dh between the main lens and the image sensor and sequentially sets the distance between the main lens and the image sensor to the depth of focus until the distance between the main lens and the image sensor becomes Dc Or the like.

Also, the control unit initially obtains an image and a depth map by selecting a distance Dh between the main lens and the image sensor, analyzes the depth map, and obtains an image only for a subject depth in which the subject exists. And adjust the distance between the image sensors in units of depth of focus.

For example, the depth of focus is determined by 2 × (aperture ratio of the main lens) × allowable confusion circle (CoC), and the size of the allowable confusion circle may be equal to one pitch or two pitches of the microlenses.

The control unit may identify the background of the initial image and the candidate object of interest through the depth map of the initial image obtained by focusing on the focal length, select an object of interest according to a predefined condition among the identified objects of interest, And the depth value of each of the depths of the selected object.

In addition, the controller may be configured to adjust the number of objects of interest selected according to the battery and the remaining amount of memory.

Meanwhile, an image acquisition method according to an embodiment of the image acquisition apparatus including a main lens, an image sensor, and a microlens disposed between the main lens and the image sensor and having a plurality of microlenses arrayed in two dimensions, Changing a distance between the lens and the image sensor; Acquiring a plurality of images having different object depths by acquiring an image every time the distance between the main lens and the image sensor changes; And generating at least one depth map from at least one of the plurality of acquired images, respectively.

Each of the microlenses may be arranged to correspond to at least two pixels of the image sensor.

The image capturing method includes: generating a depth map using outputs of at least two pixels arranged corresponding to the same microlens; And generating an image by summing outputs of at least two pixels arranged corresponding to the same microlens.

The distance between the main lens and the image sensor may be changed by a basic step unit of depth of focus (DOF).

According to the image capturing method, every time the distance between the main lens and the image sensor is changed in units of depth of focus, an image is acquired and a depth map is generated. Thus, all objects located from an infinite distance to a shortest focal distance are focused The image can be acquired.

The image capturing method may further include storing a subject area having a minimum depth value in each of the generated depth maps each time the distance between the main lens and the image sensor is changed in units of depth of focus.

The image capturing method includes: selecting a specific subject area within the plurality of images; Selecting a depth map having a minimum depth value for the selected subject area; And outputting an image corresponding to the selected depth map.

According to the image acquisition method, the distance between the main lens and the image sensor when focusing on the over focal distance is Dh, and the distance between the main lens and the image sensor when focusing on the shortest focal distance is Dc , The distance between the main lens and the image sensor can be changed in units of depth of focus between Dh and Dc.

The step of changing the distance between the main lens and the image sensor may include an initial step of selecting a distance between the main lens and the image sensor as Dh; And sequentially changing the distance between the main lens and the image sensor in units of depth of focus until the distance between the main lens and the image sensor becomes Dc.

Also, the step of changing the distance between the main lens and the image sensor may include an initial step of selecting a distance Dh between the main lens and the image sensor; And a step of analyzing the depth map obtained in the initial step to change the distance between the main lens and the image sensor in units of depth of focus so that an image is obtained only for a subject depth in which the subject exists.

The image acquisition method includes: acquiring an initial image by focusing on a focal length; Identifying a background in the initial image and a candidate object of interest through a depth map of the acquired initial image; Selecting an object of interest according to a predetermined condition among the identified objects of interest; And performing a photographing operation on a subject depth range in which an object of interest is present using the depth values of the selected objects of interest.

Further, the image acquiring method may include: checking a battery and a remaining amount of memory; And adjusting the number of objects of interest selected according to the battery and the remaining amount of memory.

According to an embodiment of the present invention, there is provided a method of re-focusing, comprising: acquiring a plurality of images each having a different depth of field; Generating a depth map from each of the acquired images; Storing a subject area having a minimum depth value in each of the depth maps; Selecting a specific subject area within the plurality of images; Selecting a depth map having a minimum depth value for the selected subject area; And outputting an image corresponding to the selected depth map.

An image acquisition device according to the disclosed embodiment includes a microlens array disposed between a main lens and an image sensor. Here, one microlens of the microlens array may correspond to two or more pixels or four or more pixels of the image sensor. In addition, the image acquiring apparatus according to the disclosed embodiment can acquire a plurality of images having different depth of fields by moving the main lens, and extract depth information from each of the acquired images. According to the image acquiring apparatus and the image acquiring method according to the disclosed embodiments, since a plurality of images having different viewpoints can be obtained by performing a plurality of photographing operations by changing the position of the main lens, There is no need to increase the size of the microlenses. Therefore, it is possible to minimize the resolution of the image by minimizing the size of the microlens without reducing the number of images having different viewpoints.

Figure 1 schematically illustrates an exemplary configuration of an image acquisition device according to one embodiment.
2A shows an exemplary positional relationship between pixels of an image sensor and individual microlenses.
Figure 2B shows another exemplary positional relationship between the pixels of the image sensor and the individual microlenses.
Fig. 3 exemplarily shows the positions of a plurality of objects photographed by the image acquiring apparatus shown in Fig.
4A to 4C illustrate exemplary focusing states according to distances between a subject and an image capturing apparatus.
5 shows the positional difference between the two pixel columns when an image formed by the main lens is located behind the micro lens array.
Figure 6 shows the positional difference of the image between two rows of pixels when the image formed by the primary lens is correctly located on the microlens array.
7 shows the positional difference between the two pixel columns in the case where the image formed by the main lens is located in front of the microlens array.
8A to 8D illustrate a depth map of an image obtained by photographing the subjects shown in FIG.
FIG. 9 illustrates a process of sequentially performing focus bracketing by moving a position of a main lens according to an image acquisition method according to an embodiment.
Figures 10 and 11 show the relationship between the circle of confusion (CoC) and the depth of focus (DOF).
12 is a graph showing a light spot size when a subject is formed on the image sensor according to the distance of the subject when the size of the allowable confusion circle is set to one microlens pitch.
13 is a graph showing the size of a light spot when a subject is formed on the image sensor according to the distance of the subject when the size of the allowable confusion circle is set to two pitches of microlenses.
FIG. 14 illustrates a process of performing focus bracketing only on a subject depth at which a subject exists according to an image acquisition method according to another embodiment.
FIGS. 15A to 15C illustrate a depth map of an image obtained by photographing the subjects shown in FIG. 14 by way of example.
FIG. 16 exemplarily shows a depth map of an image having two central subjects.
FIG. 17 is an exemplary flowchart illustrating a process of automatically performing a focus bracketing operation only on an object depth in which an object of interest is selected by automatically selecting an object of interest.
FIG. 18 exemplarily shows a depth map of an image in which a plurality of objects of interest are uniformly distributed.
19 is an exemplary flowchart illustrating a process of automatically performing a focus bracketing operation only on a subject depth in which a selected core subject exists by automatically selecting only a core subject in order to minimize a focus bracketing step.

Hereinafter, an image acquiring apparatus and an image acquiring method for generating an image having depth information will be described in detail with reference to the accompanying drawings. In the following drawings, like reference numerals refer to like elements, and the size of each element in the drawings may be exaggerated for clarity and convenience of explanation. Furthermore, the embodiments described below are merely illustrative, and various modifications are possible from these embodiments. Also, in the layer structures described below, the expressions "top" or "on top"

1 schematically illustrates an exemplary configuration of an image acquisition device 100 according to one embodiment. 1, an image capturing apparatus 100 according to the present embodiment includes a main lens 110 for condensing incident light, an image sensor 130 having a plurality of pixels arranged two-dimensionally to form an image by detecting incident light, And a microlens array 120 disposed between the main lens 110 and the image sensor 130. [ The image capturing apparatus 100 includes an actuator 115 for moving the main lens 110 for focusing on a subject, a driving unit 141 for providing an operation signal to the actuator 115, And a controller 140 receiving the image signal from the image processor 140 and generating an image having depth information. The control unit 140 may also control the operation of the driving unit 141 and the actuator 115 to change the distance between the main lens 100 and the image sensor 130.

Although FIG. 1 illustrates the primary lens 110 as being composed of one single lens element for convenience, the primary lens 110 may include a plurality of lens elements for correction of aberrations and the like. In addition, when the main lens 110 includes a plurality of lens elements, the actuator 115 may move the entire main lens 110, but it is also possible to move only some lens elements among the plurality of lens elements Do. Therefore, in the following description, the movement of the main lens 110 means that not only all the lens elements of the main lens 110 move together but also only some lens elements of the main lens 110 move I must understand.

The actuator 115 may be disposed in the microlens array 120 and the image sensor 130 instead of the main lens 110 to move the microlens array 120 and the image sensor 130. In this case, the control unit 140 controls the movement of the microlens array 120 and the image sensor 130 instead of the main lens 110 to change the distance between the main lens 100 and the image sensor 130 can do. The description below refers to the movement of the main lens 100 for the sake of convenience, but it is a relative movement of the main lens 100 with respect to the image sensor 130 and a change in the distance between the main lens 100 and the image sensor 130 .

The microlens array 120 may include a plurality of microlenses 121 arranged in two dimensions. Like the normal light field technique, the microlens array 120 may be disposed between the main lens 110 and the image sensor 130. [ The plurality of microlenses 121 in the microlens array 120 may all have the same focal length and diameter.

The image sensor 130 converts the intensity of the incident light into an electrical signal and outputs the electrical signal. For example, the image sensor 130 may be a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) image sensor. The image sensor 130 may include a plurality of pixels arranged two-dimensionally. Each pixel independently senses incident light and can output an electrical signal according to the intensity of the incident light.

The control unit 140 may process electrical signals output from the plurality of pixels of the image sensor 130 to generate an image. In addition, the controller 140 extracts depth information of a subject in the image to generate a depth map, and generates an image based on the depth map, The control unit 115 may control the focus bracketing operation. In addition, the control unit 140 may perform a re-focusing function to focus on a specific subject according to a user's command using a plurality of images having different depths of the subject. These operations will be described later in more detail.

At least two pixels of the image sensor 130 may correspond to each microlens 121 in the microlens array 120 in order to obtain depth information on a plurality of objects in the image. Then, at least two pixels of the image sensor 130 corresponding to one micro lens 121 can detect light beams having different parallaxes for the same object, respectively.

2A shows an exemplary positional relationship between the pixels 131a and 131b of the image sensor 130 and the individual microlenses 121. As shown in FIG. As shown in FIG. 2A, only two pixels 131a and 131b can be arranged in the horizontal direction with respect to one microlens 121. The two pixels 131a and 131b corresponding to one microlens 121 can sense light at different points of view from the same point of the object. For example, the left pixel 131a senses a light ray passing through the right area of the entrance pupil of the main lens 110, and the right pixel 131b senses a light ray passing through the left area of the incident lens of the main lens. Lt; / RTI > In this case, two images having different viewpoints in the horizontal direction can be obtained. However, the two images thus obtained have the same viewpoint in the vertical direction with no time difference. Therefore, depth information between objects arranged in the horizontal direction can be obtained, but depth information between objects arranged in the vertical direction can not be obtained.

FIG. 2B shows another exemplary positional relationship between the pixels 131a, 131b, 131c, and 131d of the image sensor 130 and the individual microlenses 121. FIG. Referring to FIG. 2B, four pixels 131a, 131b, 131c, and 131d arranged in a 2 × 2 matrix may be disposed on one microlens 121. In this case, not only the parallax in the horizontal direction but also the parallax in the vertical direction can be obtained. Therefore, it is possible to obtain both the depth information between the objects arranged in the horizontal direction and the depth information between the objects arranged in the vertical direction.

On the other hand, when a general color image is generated, each microlens 121 corresponds to a unit pixel of a color image. For example, one color image signal can be obtained by summing outputs of two pixels 131a and 131b or four pixels 131a, 131b, 131c, and 131d disposed on one microlens 121 . That is, the controller 140 generates depth information using the outputs of the plurality of pixels 131a, 131b, 131c, and 131d disposed on one microlens 121, and outputs the depth information to one microlens 121 The color image can be generated by summing the outputs of the plurality of pixels 131a, 131b, 131c, and 131d arranged. For this purpose, the same color filter may be disposed in the plurality of pixels 131a, 131b, 131c, and 131d disposed to one microlens 121. [ For example, a red color filter is disposed on the plurality of pixels 131a, 131b, 131c, and 131d disposed with respect to the first microlens 121, and pixels 132a 132b, 132c and 132d are arranged with a green color filter and the pixels 133a, 133b, 133c and 133d arranged with respect to the third micro lens 123 are arranged with a blue color filter. That is, the plurality of pixels 131a, 131b, 131c, and 131d disposed for one microlens 121 may be configured to sense light of the same color.

The resolution of the color image is determined by the size of each microlens 121 in the microlens array 120 regardless of the actual resolution of the image sensor 130. [ 2A, the resolution of the color image is 1/2 of the actual resolution of the image sensor 130, and in FIG. 2B, the resolution of the color image is 1/4 of the actual resolution of the image sensor 130 . 2A and 2B illustratively show two pixels 131a and 131b or four pixels 131a, 131b, 131c and 131d arranged on one microlens 121, but at the same time, The number of pixels of the image sensor 130 disposed for one microlens 121 may be increased to obtain a plurality of images having a viewpoint. However, the resolution of the color image can be reduced as much.

The image capturing apparatus 100 according to the present embodiment acquires a plurality of images having different depths of focus through focus bracketing to obtain a plurality of images having different viewpoints while suppressing the resolution degradation of the color image. Hereinafter, a specific operation method of the image acquisition apparatus 100 according to the present embodiment will be described in detail.

3 illustrates an example of the position of a plurality of objects photographed by the image capturing apparatus 100 according to the present embodiment. For example, a first subject 210 is disposed at a distance D1 from the image capturing apparatus 100, a second subject 220 is disposed at a distance D2, and a third subject 230 is placed at a distance D3 . In Fig. 3, D0 represents the hyperfocal distance. And focal length are the shortest distances from the camera where the image is clearly defined when the focus of the camera is set to infinity. Therefore, when the focal point of the image capturing apparatus 100 is set to infinity for a subject farther than D0, a clear image is always formed irrespective of the distance to the image capturing apparatus 100. [

Since the images obtained by photographing the subjects 210, 220 and 230 using a conventional camera are generally displayed on the two-dimensional plane of the subjects 210, 220 and 230, the images of the subjects 210, 220 and 230 The exact distance information will disappear. However, since only a correctly-focused subject is clearly displayed, and an unfocused subject appears blurred due to blur, only an unfocused subject is located in front of or behind the focused subject Able to know.

On the other hand, in the case of the image capturing apparatus 100 according to the present embodiment, since depth information can be obtained as described with reference to FIGS. 4A to 4C and FIGS. 5 to 7, The distance between them. 4A to 4C illustrate focusing states according to the distances between the objects 210, 220, and 230 and the image capturing apparatus 100. FIG. 4A to 4C, it is assumed that only the second object 220 is focused for convenience. 5 shows the positional difference between the two pixel columns when an image formed by the main lens 110 is located behind the microlens array 120. Fig. 7 shows the positional difference between the two pixel columns when the image formed by the main lens 110 is accurately positioned on the microlens array 120 and FIG. The positional difference between the two pixel columns is shown.

In general, the focal length of a lens assumes that the incident light is parallel light (i.e., the object is located at infinity). Accordingly, the actual object is formed at a distance longer than the focal length of the lens, and the closer the distance between the object and the lens is, the longer the distance at which the image of the object is formed becomes longer. For example, referring to FIG. 4A, a first object 210 located closest to the image acquisition device 100 is formed through the microlens array 120 to form an image 210 '. In this case, since the microlens array 120 can not correctly capture the image 210 'of the first object 210 on the image sensor 130, the image of the first subject 210' 210) appear blurry.

The conventional camera only obtains information that the first subject 210 is out of focus. 5, in the case of the image acquisition apparatus 100 according to the present embodiment, the left pixel column 130a and the right pixel column 130b arranged in the microlenses of one column of the microlens array 120, A pixel disparity occurs between the video signals generated by the left pixel column 130a and the right pixel column 130b due to the parallax between the columns 130b. Such a pixel mismatch may be inversely multiplied by the depth value for the first object 210. Therefore, the image acquisition apparatus 100 according to the present embodiment can obtain depth information on the first object 210. [

4B, an image 220 'of a second subject 220 farther from the first subject 210 is formed on the microlens array 120 precisely. In this case, the microlens array 120 can accurately connect the image 220 'of the second subject 220 to the image sensor 130. Therefore, the second subject 220 is clearly displayed in the image output by the image sensor 130. [ In this case, as shown in FIG. 6, between the image signals generated by the left pixel column 130a and the right pixel column 130b, which are arranged in the microlenses of one row of the microlens array 120, Since there is no depth difference, pixel mismatch does not occur. Accordingly, the depth value of the second subject 220 becomes zero.

4C, an image 230 'of a third subject 230 further away from the second subject 220 is formed in front of the microlens array 120. The microlens array 120 can not accurately capture the image 230 'of the third object 230 on the image sensor 130 and the third object 230 on the image output by the image sensor 130 It appears dim. Referring to FIG. 7, a depth difference is generated between image signals generated by the left pixel column 130a and the right pixel column 130b, which are arranged in the microlenses of one row of the microlens array 120 . 5, the direction of the depth difference between the left pixel column 130a and the right pixel column 130b in FIG. 7 is opposite to that in FIG. For example, in the case of FIG. 5, the depth value is defined as negative (-), and in FIG. 7, the depth value may be defined as positive (+). Therefore, if the depth value is negative, it can be seen that the subject is closer and the depth value is greater, the subject is farther away. The greater the defocus, the greater the parallax and the greater the pixel mismatch. Therefore, the distance of the subject can be roughly known from the depth value.

In the above-described manner, the image capturing apparatus 100 according to the present embodiment can generate a depth map for a subject and obtain an image having depth information. The depth value of the subject that is focused on the depth map is 0 and the depth value of the subject positioned ahead of the focused subject is negative and the depth value of the subject positioned behind the focused subject The value can have a positive value. In addition, as the distance from the focused object increases, the depth value may increase.

8A to 8D illustrate depth maps of an image obtained by photographing the subjects 210, 220 and 230 shown in FIG. For example, FIG. 8A shows a depth map 300 obtained with the image acquisition apparatus 100 focused on a distance D0, i.e., a focal length. In this case, the depth values 211, 221, and 231 for each of the subjects 210, 220, and 230 in the depth map 300 all have negative depth values. 8B shows a depth map 310 obtained by focusing on the third subject 230 at the distance D3. The depth value 231 of the third subject 230 becomes 0 in the depth map 310 and the depth values 211 and 221 of the remaining subjects 210 and 220 become negative values. Also, FIG. 8C shows the depth map 320 obtained while focusing on the second object 220 at the distance D2. The depth value 221 of the second subject 220 becomes 0 in the depth map 320 and the depth value 211 of the first subject 210 has a negative value and the depth value 221 of the third subject 230 The value 231 has a positive value. Finally, FIG. 8D shows a depth map 330 obtained with focus on the first subject 210 at distance D1. The depth value 211 of the first subject 210 is 0 in the depth map 330 and the depth values 221 and 231 of the remaining subjects 220 and 230 have a positive value.

As described above, it can be determined that a subject having a depth value of 0 in the depth map is focused. In addition, the magnitude and sign of the depth value can be used to know how far away the subject is out of focus from the front or back of the focused subject. Accordingly, it is possible to display an image focused on a subject selected by the user by obtaining a plurality of images each focused on subjects at different distances.

It is called refocusing that a sharp image that focuses only on a specific subject selected by the user from a plurality of subjects at different distances is processed and a blurred image which is not focused on the remaining subject is processed. In order to perform re-focusing, basically two pieces of information are required: a plurality of images each focused on a plurality of subjects at different distances, and a depth map including distance information between each of the subjects . In the case where there is no depth map, even if a plurality of images focused on a plurality of subjects at different distances are acquired, an image focused on a specific user selected by the user is automatically processed It is impossible to choose. Without a depth map, you can not know which subject was in focus. Therefore, even if a plurality of images having different subject depths are obtained by using a conventional camera, there is no depth map, so it is impossible to perform re-focusing. In this respect, the re-focusing function is a typical function of the light field camera. In general, a light field camera acquires a plurality of images at different times at one time in order to perform a re-focusing function, thereby reducing the resolution of each image.

The image acquiring apparatus 100 according to the present embodiment acquires a plurality of images having different viewpoints at different time points with different focus depths through focus bracketing instead of acquiring a plurality of images having different viewpoints at the same time . For example, FIG. 9 illustrates an exemplary process of performing focus bracketing by moving a position of a main lens 110 according to an image acquisition method according to an embodiment, and sequentially changing a subject depth. 9, first, the position of the main lens 110 is moved so as to focus on the distance indicated by (1), and then the position of the main lens 110 is moved again to focus on the distance indicated by And then the main lens 110 is moved so as to correspond to the distances indicated by ③, ④ and ⑤, respectively, and the photographing can be performed respectively. Although focus bracketing is shown as being performed from the far side to the near side as shown by arrows in FIG. 9, it is also possible to perform focus bracketing from the near side to the far side.

When the focus bracketing step is completed by capturing an image by adjusting the subject depth, the controller 140 can generate a color image and calculate a depth map from the captured image through image processing. After calculating the depth map, a plurality of regions can be distinguished according to the depth value in the image. In particular, a region having a depth value of 0 or a region having a minimum depth value can be identified and stored in a memory . In this manner, the color image and the depth map can be obtained for each focus bracketing step until the focus bracketing operation is completely terminated, and an area in which the depth value is zero or a region having the minimum depth value can be stored.

After the focus bracketing operation is completely terminated, the user can perform a re-focusing command to obtain a focused image in the region of interest. The criterion for selecting an image focused on the area selected by the user is the depth map. For example, when the user selects an object at an arbitrary position in the image, the depth value of the selected object area is compared with all depth maps obtained through focus bracketing. Thereafter, a depth map in which the depth value is 0 or minimum for the object region selected by the user is selected, and a color image corresponding to the selected depth map is selected and output or displayed on the screen to perform the re-focusing function . The color image selected in this way can be stored separately according to the user's command. It is also possible to further perform a re-focusing operation for another area of interest.

Another function of the re-focusing operation is blurring enhancement. That is, after a desired color image is selected through the above-described re-focusing operation, a blur scale factor is adjusted so that a blurring effect is obtained in proportion to the magnitude of the depth value with respect to other subjects, . For example, if it is assumed that a color image corresponding to the depth map 320 shown in FIG. 8C is selected, the image of the second subject 220 with the depth value of 0 is left as it is, The blurring effect can be increased or decreased by a blur scale factor through image processing for images of other objects 210 and 230 disposed at the rear side and the rear side. This operation is possible because each color image has corresponding depth information for each color image. If the blur scale factor is multiplied by the depth values in the depth map, the blur scale factor is always 0 regardless of the value of the blur scale factor because the depth value is 0 for the focused image, The depth value can be changed in proportion to the depth of the film. Therefore, the blurring effect can be enhanced simply by applying an algorithm that interlocks the amount of blur in the color image according to the scaled depth value of the depth map. It is also possible to adjust the minimum depth value to 0 if there is no depth value of 0 in the depth map.

As described above, according to the image capturing apparatus 100 according to the present embodiment, by performing a plurality of photographs while adjusting the focus by changing the position of the main lens 100, a plurality of images It is not necessary to increase the size of the individual microlenses 121 in order to increase the number of images having different viewpoints. Accordingly, it is possible to minimize the size of the microlens 121 to about twice or four times the pixel size of the image sensor 130, thereby suppressing the resolution degradation of the image, and sufficiently securing the number of images having different viewpoints.

On the other hand, in performing the focus bracketing operation, it is important to determine the bracketing step optimally since it is necessary to acquire a plurality of images each focused on each of all subjects. If the bracketing step is too short, a number of surplus images that are focused on the same subject are calculated and the number of times of shooting is too large. If the bracketing step is too large, it may not be possible to obtain an image that is focused accurately on some subjects.

The relationship between the circle of confusion (CoC) and the depth of focus (DOF) is first discussed to derive the optimal bracketing step. It is practically impossible to obtain the theoretical size of the light spot in consideration of the aberration of the main lens 110 and the assembly deviation because the size of the light spot that theoretically indicates the resolving power of the lens reaches the diffraction limit. Therefore, the allowable confinement circle is usually used as a scale that is larger than the diffraction limit but is recognized as being in focus when the user judges by eye. For example, when the size of the light spot formed on the image sensor 130 is smaller than the allowable confusion circle, the user determines that the image is focused.

Figs. 10 and 11 show the relationship between the allowable confusion circle and the depth of focus. Fig. 10 schematically shows a region A where the light spot is formed by the main lens 110, Fig. 11 is a cross- The area A shown in Fig. Referring to Figs. 10 and 11, S represents a light spot that is the theoretical diffraction limit, CoC represents a permitted confusion circle recognized as focused, and DOF represents the interval during which the permitted confusion circle is maintained, i.e., Which indicates the depth of focus, which is an acceptable section. As shown in Fig. 11, the depth of focus is determined by the F number (aperture ratio) of the main lens 110 and the size of the allowable confusion circle (i.e., DOF = 2 x F x CoC).

For example, assuming that the focal length of the main lens 110 is 4.2 mm, the F number of the main lens 110 is 2.2, and the pitch of the microlens 121 is 2.24 micrometers, the theoretical value for green light having a wavelength of 540 nm The size of the optical spot is 1.45um (= 1.22xF number x wavelength) which is the diffraction limit. Further, if the size of the CoC is set to correspond to one pitch of the microlenses 121 (that is, the size of one unit pixel of the color image), the DOF can be 2 x 2.2 x 2.24 um = 9.856 um, If the size of the CoC is set to correspond to two pitches of the microlenses 121 (i.e., the size of two unit pixels of the color image), the DOF can be 2 x 2.2 x 4.48 um = 19.712 um.

In order to acquire an image focused on the image sensor 130 with respect to a subject located at an arbitrary distance, the imaging position of the subject may be within the DOF section. Therefore, in order to obtain images that are focused on all the objects located from the infinite distance to the closest focusing distance, all the objects focused within the DOF section are focused on the main lens 110 The drive interval for focus adjustment can be optimized in DOF units. For example, a point at which focus can be focused with respect to a distance over the focal length is set as an initial position for driving the focus of the main lens 110, and the focus bracketing is performed every week The position of the lens 110 can be changed. In this way, each step of focus bracketing can be performed in DOF units to the point where the focus bracketing operation is completely terminated (that is, the point where the focus can be focused on the shortest focal distance).

Then the total number of steps for focus bracketing can be defined as (total trajectory) / (DOF). For example, assuming that the focal length of the main lens 110 is 4.2 mm, the F number of the main lens 110 is 2.2, and the pitch of the micro lens 121 is 2.24 um, And the image sensor 130 is 4.2 mm, which is the same as the focal distance. Further, assuming that the shortest focal distance is 10 cm, the distance between the image sensor 130 and the main lens 110 when focusing on the shortest focal distance is calculated as 4.3841 mm by the lens equation. Therefore, the total trajectory of the focus drive for the focus bracketing operation to obtain the focused image for all subjects from the infinite distance to the shortest focal distance is 184.1 um which is a difference of 4.3841 mm and 4.2 mm.

Now, since the total trajectory and the DOF of the focus drive for the focus bracketing operation of the main lens 110 illustrated above are known, the total number of steps for the focus bracketing operation can be obtained. For example, when the size of the allowable confocal circle CoC is set to one pitch of the microlenses 121, the DOF is 9.856 um, so that the total number of steps for the focus bracketing operation is 184.1 um / 9.856 um = 18.7 It can be calculated and total 19 steps including initial position. The distance between the main lens 110 and the image sensor 130 when focusing on the focal length is Dh and the distance between the main lens 110 and the image sensor 130 when focusing on the shortest focal distance, The distance between the main lens 110 and the image sensor 130 can be changed by 19 steps in units of DOF between Dh and Dc. In the same manner as above, when the size of the allowable confusion circle CoC is set to two pitches of the microlens 121, the DOF is 19.712 um, so that the total number of steps of the focus bracketing operation is calculated as 184.1 um / 19.712 um = 9.4 And can be a total of 10 steps including the initial position. That is, the distance between the main lens 110 and the image sensor 130 can be changed by 10 steps in DOF between Dh and Dc.

12 is a graph showing the size of the light spot when the subject is formed on the image sensor 130 according to the distance of the subject when the size of the allowable confusion circle is set to one pitch of the micro lenses 121. FIG. In the graph of FIG. 12, the vertical axis represents the size of the light spot as the unit pixel size of the color image (the pitch of the microlenses 121), and when the ideal light without considering the diffraction limit is condensed at the focus position, Is equal to the size of pixel 0. Also, the graph of FIG. 12 is obtained by performing a focus bracketing operation of 10 steps for a distance of 5 m to a shortest focal distance of 20 cm.

Referring to the graph of FIG. 12, at the initial position of the focus bracketing, the size of the light spot produced by the subject located from the infinite distance to the focal distance enters into one unit pixel of the color image. And the size of a light spot formed by a subject located at a distance shorter than the focal distance becomes greater than the size of one unit pixel of the color image, resulting in blur. Therefore, at the initial position of the focus bracketing, it is possible to acquire an image focused on a subject located farther than the over focal distance.

The second focus bracket offset (1 DOF Bracket offset) is performed at a position where the main lens 110 is shifted by one DOF at the initial focus bracketing position. Referring to FIG. 12, it can be seen that the depth of a subject in which the size of a light spot enters a unit pixel of a color image is shifted. In the same manner, when the position of the main lens 110 is sequentially moved by one DOF unit up to the "9 DOF Bracket offset", the distance of the focused object is continuously maintained up to an object distance of 20 cm. Therefore, when focus bracketing is performed in units of DOF, it is possible to obtain a focused image for all objects located from infinite distance to the shortest focal distance.

The over focal distance, which is the initial position of the focus bracketing, is (focal length x focal length) / (F x CoC). For example, when the focal distance of the main lens 110 is 4.2 mm and the F number is 2.2, if the size of the allowable confusion circle is set to one pitch of the microlens 121, the over focal distance can be 3.6 m. Further, when the size of the allowable confusion circle is set to two pitches of the microlenses 121, the over focal distance can be 1.8 m. Table 1 below shows the depth of the subject focused on each focus bracketing step based on the graph shown in FIG. As shown in Table 1, when focus bracketing is performed while sequentially changing the position of the main lens 110 in units of DOF, it is possible to obtain a focused image from an infinite distance to a shortest focal distance.

13 is a graph showing the size of the light spot when the subject is formed on the image sensor 130 according to the distance of the subject when the size of the allowable confusion circle is set to two pitches of the micro lenses 121. [ The graph of FIG. 13 is obtained in the same manner as the graph of FIG. 12, and only the case where the size of the allowable confusion circle is two pitches of the microlenses 121 is considered. Referring to FIG. 13, when the size of the allowable confusion circle is doubled, DOF increases twice as compared with the case of FIG. 12, which shows that the subject depth of each step increases during focus bracketing.

Table 2 below shows the depth of the subject focused on each focus bracketing step based on the graph shown in FIG. As shown in Table 2, when focus bracketing is performed while sequentially changing the position of the main lens 110 in units of DOF, it is possible to obtain a focused image from an infinite distance to a shortest focal distance.

The numerical values shown in Tables 1 and 2 are obtained assuming that the focal length of the main lens 110 is 4.2 mm, the F number of the main lens 110 is 2.2, and the pitch of the micro lenses 121 is 2.24. However, these numerical values are merely illustrative examples, and the present embodiment is not limited to the values exemplified above. Depending on the design, the focal length of the main lens 110, the F number, and the pitch of the microlenses 121 may be different. Regardless of such design changes, the positional change of the main lens 110 in each focus bracketing step can be done in DOF when performing the focus bracketing operation from the over focal distance to the shortest focal distance.

In Table 1 and Table 2, to optimize the number of focus bracketing steps, we set the range in which the color image is considered to be in focus based on the allowable confusion circle, and calculate the depth of the object that forms an image in the DOF in each focus bracketing step Respectively. However, the distance information on the subject in the depth map described in FIGS. 8A to 8D can be obtained more accurately than the depth of field shown in Tables 1 and 2. For example, the depth value for an arbitrary subject can be calculated on a sub-pixel basis using interpolation or the like based on the difference between depth value information obtained from adjacent pixels.

The sequential focus bracketing operation described with reference to FIGS. 9 to 13 can be applied when it is difficult to identify the subjects in the field of view of the image capturing apparatus 100 or when the number of identifiable subjects is very large. If the number of identifiable subjects is small in the field of view of the image acquisition apparatus 100, the focus bracketing operation may be performed by adjusting the focus bracketing step only for a distance at which the subject exists. For example, FIG. 14 exemplarily shows a process of performing a focus bracketing operation only on a subject depth where a subject exists in accordance with an image acquisition method according to another embodiment. 15A to 15C illustrate depth maps of an image obtained by photographing the subjects shown in FIG. 14. FIG.

As shown in FIG. 14, it is assumed that a first subject 210 is disposed at a distance D1 from the image capturing apparatus 100, and a third subject 230 is disposed at a distance D3. In Fig. 14, D0 represents the focal length. 15A also shows a depth map 340 obtained with focus on the focal length. In this case, the depth values 211 and 231 for each of the subjects 210 and 230 in the depth map 340 have negative values, and the absolute value of the depth value increases as the distance from the focal length increases. do. FIG. 15B shows a depth map 350 obtained by focusing on the third object 230. FIG. The depth value 231 of the third subject 230 in the depth map 350 is 0 and the depth values 211 and 221 of the first subject 210 are negative values. Fig. 15C shows a depth map 360 obtained by focusing on the first object 210. Fig. The depth value 211 of the first object 210 in the depth map 360 is 0 and the depth value 231 of the third object 230 is a positive value.

In the image acquiring method according to the present embodiment, a focus bracketing operation is performed by measuring a distance in which an object of interest exists in advance, and skipping the focus bracketing step with the subject depth at which the subject exists. To do this, first, a color image is acquired by adjusting the focal point of the image capturing apparatus 100 to a focal length, and a depth map 340 as shown in FIG. 15A is generated from the color image. The control unit 140 analyzes the depth map 340 to calculate the distances of the two subjects 210 and 230 and determines whether the two subjects 210 and 230 are within one DOF interval that can focus at one time Review.

If the two subjects 210 and 230 are separated by a distance that can not be focused at one time, the focus bracketing step is performed on the two subjects 210 and 230, respectively. For example, the focus bracketing step is performed with the subject depth that can focus on the third subject 230 to obtain the color image and the depth map 350 shown in FIG. 15B. Then, a focus bracketing step is performed with the subject depth at which the first subject 210 can be focused, thereby obtaining the color image and the depth map 360 shown in FIG. 15C. In this manner, the focus bracketing operation can be terminated after the focus bracketing step is performed on all of the subjects 210 and 230.

Unlike the method shown in Fig. 9 in which the focus bracketing step is performed sequentially in units of DOF for an object from an infinite distance to a shortest focal distance, the method shown in Fig. 14 performs the focus bracketing step for the over focal distance first Perform the focus bracketing step only when there is an actual subject. However, in the case of the method shown in Fig. 14, each step of focus bracketing is performed in DOF units as in the method shown in Fig. Therefore, the focus position of the main lens 110 can be moved by an interval of DOF or an integral multiple of the DOF while proceeding with each focus bracketing step. For example, assuming that the distance between the image capturing apparatus 100 and the first subject 210 is 50 cm and the distance between the third subject 230 and the image capturing apparatus 100 is 1.5 m, according to the method shown in FIG. 14, , It is possible to perform only the focus bracketing step, the second focus bracketing step and the third focus bracketing step for the over-focus distance.

According to the method shown in FIG. 14, the efficiency of the focus bracketing operation can be improved when the number of subjects is small, or when the subject does not exist in a part of the subject depth range. For example, when the object closest to the image capturing apparatus 100 is 50 cm away from the image capturing apparatus 100, the focus bracketing operation can be performed only up to the fifth step in the example of Table 1 and terminated. Particularly, as can be seen from Tables 1 and 2, as the depth of field of a subject becomes narrower toward the image capturing apparatus 100, the number of focus bracketing steps for obtaining a focused image increases. Accordingly, the presence or absence of the subject can be determined in advance at the closest distance, and the focus bracketing operation can be improved by omitting the focus bracketing steps for the closest distance when the subject is not present at the closest distance.

The focus bracketing operation described so far can be performed by the control unit 140 controlling the driving unit 141 and the actuator 115 to move the main lens 110. In addition, after performing the initial focus bracketing step on the over focal length, the control unit 140 determines whether to proceed all the focus bracketing steps in sequence according to the method shown in Fig. 9, or in accordance with the method shown in Fig. 14, It is possible to perform only the bracketing step and judge whether to skip the remaining focus bracketing step.

For example, if the color image and the depth map obtained by performing the initial focus bracketing step are analyzed and it is difficult to identify or identify the subject in all of the subject depth ranges shown in Table 1 or Table 2 All focus bracketing steps can be sequentially performed according to the method shown in FIG. In the case where the subject exists only in some object depth regions illustrated in Table 1 or Table 2, only a part of focus bracketing steps may be performed according to the method shown in FIG. 14, and the remaining focus bracketing steps may be skipped.

In the embodiment described with reference to FIGS. 9 to 13, all focus bracketing steps are sequentially performed regardless of the presence or absence of a subject. In the embodiment described with reference to FIG. 14 and FIGS. 15A to 15C, in order to improve the efficiency of the focus bracketing operation, the focus bracketing step is performed only on the subject depth range in which the subject exists. However, in the embodiment described with reference to FIG. 14 and FIGS. 15A to 15C, since the focus bracketing step is performed on all subject depth ranges in which the subject exists, the number of focus bracketing steps to be performed may be increased in some cases.

In order to further improve the efficiency of the focus bracketing operation, the focus bracketing step may be performed only on a subject depth range in which some subjects exist. For example, if the user selects a subject of interest in the color image obtained by performing the initial focus bracketing step, the focus bracketing step may be performed according to the method shown in FIG. 14 only for the selected subject.

It is also possible to automatically perform the process of selecting a subject. For example, FIG. 16 exemplarily shows a depth map of an image with two central subjects obtained by adjusting the focus of the image capturing apparatus 100 to the focal length. As can be seen from FIG. 16, the field corresponding to the background has a relatively uniform depth value, and the field corresponding to the bottom has a gradually changing depth value. And, the area where the two central subjects exist has a depth value that is obviously different from the depth value of the background area. A subject having a depth value equal to or similar to the depth value of the background area may exist in the image but if there is a subject having a depth value that is clearly different from the depth value of the background area, And a subject having an apparently different depth value. It is also possible to judge that the region in which the depth value gradually changes is also not generally the region of interest of the photographer.

Based on this point, it is possible to minimize the number of focus bracketing steps by determining a subject (hereinafter referred to as an interested subject) that the photographer is interested in and performing a focus bracketing operation on the depth range of the subject of interest. Also, in order to further reduce the number of focus bracketing steps, a focus bracketing operation may be performed on a selected object of interest by selecting only a part of the object of interest according to the user's preference among a plurality of objects of interest. Thus, the number of operations of the processor due to the focus bracketing operation of the image capturing apparatus 100, and power consumption and memory consumption can be reduced.

Specifically, FIG. 17 is an exemplary flowchart illustrating a process of automatically performing a focus bracketing operation only on a subject depth in which an object of interest is selected by automatically selecting an object of interest. Referring to FIG. 17, in the object identifying step (S10), the background and the candidate object of interest are distinguished through the depth map of the initial image obtained by matching the focus of the image capturing apparatus 100 with the focal length. For example, it can be identified that there is an object of interest in an area that has a depth value that is distinctly different from the depth value of the background area in the initial image and in which the depth value does not change progressively. For example, a saliency object detection algorithm or the like can be used to identify a subject of interest by extracting a prominent or prominent outline compared to the surroundings.

Next, in the object selecting step S11, an interested object to be photographed through the focus bracketing operation can be selected from the candidate objects of interest identified in the object identifying step S10. For this, the controller 140 may analyze the history of the images taken by the user stored in the memory (not shown) in advance to determine the intention or preference of the user, and store the result in a memory or the like. For example, a salient object detection algorithm may be applied to photographed images to store the objects that are most frequently displayed in photographed images in a memory. For example, various types of objects such as a person, flower, insect, bird, etc. can be extracted and classified within photographed images. Then, the control unit 140 can select a subject matching the user's favorite objects stored in the memory among the candidate interesting subjects identified in the subject identifying step S10. The number of the objects of interest to be finally selected may be predetermined, for example, within five, depending on the setting of the user. Alternatively, it is possible to select all the objects satisfying the condition preset by the user, such as the position or size in the initial image, as the object of interest.

If there is no object of interest that matches the user's preferred object, for example, a plurality of objects of interest may be selected in the order closest to the center of the initial image, or a plurality of interest objects arranged in the vicinity of the center of the initial image A plurality of interesting objects may be selected in the order of the largest size among the objects. Further, for example, as shown in FIG. 18, if a plurality of objects of interest are uniformly distributed over the entire area of the initial image, all the objects of interest may be selected.

If the number of the objects of interest identified in the object identifying step S10 is only one or less than the number of objects of interest preset by the user, the object selecting step S11 is skipped and the object identifying step S10 All of the objects of interest can be selected.

Then, in the focus bracketing step S12, it is possible to perform the focus bracketing step only for the subject depth range in which the finally selected interesting subject exists, using the depth value of each of the finally selected interesting subjects. Then, the images photographed in the respective focus bracketing steps can be stored in the memory (S13).

On the other hand, if a plurality of focus bracketing steps are performed, power consumption and memory consumption of the image capturing apparatus 100 increase, and the number of processor operations of the control unit 140 increases. Therefore, the remaining amount of the battery and the memory may be insufficient to perform the focus bracketing operation. Therefore, it is possible to limit the number of focus bracketing steps when it is determined that the battery and the memory are insufficient by checking the battery and the memory before performing the focus bracketing operation.

For example, FIG. 19 is an exemplary flowchart illustrating a process of automatically performing focus bracketing only on a subject depth where a selected core subject exists by automatically selecting only a core subject in order to minimize a focus bracketing step. Referring to FIG. 19, the battery and memory remaining amount of the image capturing apparatus 100 are checked in a battery and memory check step (S20). If it is determined that the battery and the remaining amount of memory are sufficient to perform the focus bracketing operation, the focus bracketing operation can be performed according to the procedure shown in FIG. However, if it is determined that the battery and the remaining amount of memory are insufficient, the focus bracketing operation shown in Fig. 19, that is, steps S21 to S24 may be performed.

The subject identifying step S21 may be performed in the same manner as the subject identifying step S10 described with reference to FIG. The object selection minimizing step S22 is performed according to the same algorithm as the object selecting step S11 described with reference to FIG. 17, but the number of the objects of interest finally selected may be one or two, for example, . For example, when there are a plurality of candidate interesting objects, one or two interesting objects closest to the center of the initial image may be selected, or one or two largest objects may be selected according to the user's setting, You can also select objects of interest. If the number of objects of interest identified in the object identifying step S10 is only one or two, the object selecting step S11 may be omitted and all objects of interest identified in the object identifying step S10 may be selected.

Then, in the focus bracketing step S23, the focus bracketing step may be performed only on the subject depth range in which the finally selected interesting subject exists, using the depth values of the finally selected interesting subjects. Then, the images photographed in the respective focus bracketing steps can be stored in the memory (S24).

Up to now, an exemplary embodiment of an image acquiring apparatus and an image acquiring method for generating an image having depth information in order to facilitate understanding of the present invention has been described and shown in the accompanying drawings. It should be understood, however, that such embodiments are merely illustrative of the present invention and not limiting thereof. And it is to be understood that the invention is not limited to the details shown and described. Since various other modifications may occur to those of ordinary skill in the art.

100 ..... Image acquisition device 110 ..... Main lens
115 ..... Actuator 120 ..... Micro lens array
121 ..... Micro lens 130 ..... Image sensor
131, 132, 133, 134 ..... pixel 140 ..... control unit
141 ..... drive

Claims (27)

A main lens for condensing incident light;
An image sensor having a plurality of two-dimensionally arranged pixels for detecting an incident light to form an image;
A microlens array disposed between the main lens and the image sensor, the microlens array having a plurality of microlenses arranged two-dimensionally; And
And a controller for receiving an image signal from the image sensor and generating an image,
Wherein the control unit is configured to acquire a plurality of images having different subject depths by varying a distance between the main lens and the image sensor and to generate at least one depth map from at least one of the acquired images. The method according to claim 1,
Wherein each of the microlenses is arranged to correspond to at least two pixels of the image sensor. 3. The method of claim 2,
Wherein the control unit is configured to generate a depth map by using outputs of at least two pixels arranged corresponding to the same microlens and to generate an image by summing outputs of at least two pixels arranged corresponding to the same microlens, Acquisition device. The method according to claim 1,
Wherein the control unit is configured to change the distance between the main lens and the image sensor by using a depth of focus (DOF) as a basic step unit. 5. The method of claim 4,
Wherein the control unit acquires an image through the image sensor and generates a depth map every time the distance between the main lens and the image sensor is changed in increments of depth of focus, And acquires a matching image. 6. The method of claim 5,
Wherein the control unit is configured to store a subject area having a minimum depth value in each depth map generated each time the distance between the main lens and the image sensor is changed in units of depth of focus. The method according to claim 6,
Wherein the control unit selects a depth map having a minimum depth value with respect to the selected subject area and outputs an image corresponding to the selected depth map when a specific subject area is selected in the plurality of images. 5. The method of claim 4,
And a distance between the main lens and the image sensor when focusing on the shortest focal length is Dh and a distance between the main lens and the image sensor when the focal point is focused on the shortest focal distance is Dc, And a distance between the main lens and the image sensor is changed in units of depth of focus between Dh and Dc. 9. The method of claim 8,
Wherein the control unit initially selects the distance Dh between the main lens and the image sensor and sequentially sets the distance between the main lens and the image sensor to the depth of focus until the distance between the main lens and the image sensor becomes Dc The image capturing device configured to change the image capturing device. 9. The method of claim 8,
The control unit initially obtains an image and a depth map by selecting the distance between the main lens and the image sensor as Dh and analyzes the depth map to obtain an image only for a subject depth in which the subject exists, To the depth of focus unit. 5. The method of claim 4,
Wherein the depth of focus is determined by 2 占 (aperture ratio of the main lens) 占 allowable confusion circle (CoC), and the size of the allowable confusion circle is one pitch or two pitches of the microlenses. 5. The method of claim 4,
The control unit may identify the background of the initial image and the candidate object of interest through the depth map of the initial image obtained by focusing on the focal length, select an object of interest according to a predefined condition among the identified objects of interest, And the depth value of each of the plurality of subject depths is used. 13. The method of claim 12,
Wherein the control unit is configured to adjust the number of objects of interest selected according to a battery and a remaining amount of memory. A method for acquiring an image of an image acquisition apparatus including a main lens, an image sensor, and a microlens disposed between the main lens and the image sensor and including a plurality of microlenses arranged two-
Changing a distance between the main lens and the image sensor;
Acquiring a plurality of images having different object depths by acquiring an image every time the distance between the main lens and the image sensor changes; And
And generating at least one depth map from at least one of the plurality of acquired images. 15. The method of claim 14,
Wherein each of the microlenses is arranged to correspond to at least two pixels of the image sensor. 16. The method of claim 15,
Generating a depth map using the outputs of at least two pixels arranged corresponding to the same microlens, respectively; And
And summing outputs of at least two pixels arranged corresponding to the same microlens to generate an image. 15. The method of claim 14,
Wherein the distance between the main lens and the image sensor changes with a depth of focus (DOF) as a unit of a basic step. 18. The method of claim 17,
Acquiring an image every time the distance between the main lens and the image sensor is changed in units of depth of focus, and generating a depth map, thereby acquiring an image focused on all objects located from an infinite distance to a shortest focal distance . 19. The method of claim 18,
Further comprising the step of storing a subject area having a minimum depth value in each generated depth map each time the distance between the main lens and the image sensor is changed in units of depth of focus. 20. The method of claim 19,
Selecting a specific subject area within the plurality of images;
Selecting a depth map having a minimum depth value for the selected subject area; And
And outputting an image corresponding to the selected depth map. 18. The method of claim 17,
And the distance between the main lens and the image sensor when focusing on the shortest focal distance is Dh and the distance between the main lens and the image sensor when the focal point is focused on the shortest focal distance is Dc, And the image sensor is changed in units of depth of focus between Dh and Dc. 22. The method of claim 21,
Wherein changing the distance between the main lens and the image sensor comprises:
An initial step of selecting a distance Dh between the main lens and the image sensor; And
And sequentially changing the distance between the main lens and the image sensor in units of depth of focus until the distance between the main lens and the image sensor becomes Dc. 22. The method of claim 21,
Wherein changing the distance between the main lens and the image sensor comprises:
An initial step of selecting a distance Dh between the main lens and the image sensor; And
And changing a distance between the main lens and the image sensor in units of depth of focus to analyze the depth map obtained in the initial step to obtain an image only for the depth of the subject in which the subject exists. 18. The method of claim 17,
Wherein the depth of focus is determined by 2 占 (aperture ratio of the main lens) 占 allowable confusion circle (CoC), and the size of the allowable confusion circle is one pitch or two pitches of the microlenses. 18. The method of claim 17,
And acquiring an initial image by focusing on a focal distance;
Identifying a background in the initial image and a candidate object of interest through a depth map of the acquired initial image;
Selecting an object of interest according to a predetermined condition among the identified objects of interest; And
And photographing the subject depth range in which the selected subject matter of interest is present using the depth values of the selected objects of interest. 26. The method of claim 25,
Inspecting the remaining capacity of the battery and the memory; And
And adjusting the number of objects of interest selected according to the battery and the remaining amount of memory. Acquiring a plurality of images each having a different subject depth;
Generating a depth map from each of the acquired images;
Storing a subject area having a minimum depth value in each of the depth maps;
Selecting a specific subject area within the plurality of images;
Selecting a depth map having a minimum depth value for the selected subject area; And
And outputting an image corresponding to the selected depth map.

Images