KR20160065742A - Apparatus and method for taking lightfield image - Google Patents

Abstract

A method and an apparatus for imaging a light field image are provided. The light field imaging apparatus comprises a mirror array or a microlens array. An activated mirror among a plurality of mirrors of the mirror array provides light to a plurality of elements of an image sensor. An activated lens among a plurality of lenses of the microlens array provides light to the elements of the image sensor. A selected mirror is changed as the activated mirror among the mirrors, and a selected lens is changed as the activated lens among the lenses by time-division

Description

FIELD OF THE INVENTION [0001] The present invention relates to a light field photographing apparatus,

The following embodiments relate to image processing, and more particularly, to a light field photographing apparatus and method.

An image capture apparatus, such as a camera module, obtains color values representing the image of the subject.

For acquisition of color values, the imaging device includes an image sensor. The image sensor of the photographing apparatus has a width and a height and has a predetermined resolution. The image sensor generates color values for each of the coordinates of the image through shooting. Here, the coordinates represent positions in the image and are expressed by (x, y) which are two-dimensional coordinate values. For example, the color values may be red, green, and blue red (RGB) values.

Through shooting of the photographing apparatus, the user can finally obtain a two-dimensional color image. Conventional imaging devices, so-called digital cameras, produce two-dimensional color images as described above.

As technology advances, plenoptic technology has been developed. Plenoptic technology acquires the color values for the coordinates (x, y) of the image sensor and for (u, v) at the above coordinates. Here, (u, v) may be information on angular resolution. That is to say, the plane optical technique obtains (R, G, B) color values for (x, y, u, v).

According to Planen-Optic technology, a microlens array is disposed between the main lens and the image sensor, and (R, G, B) color values for (x, y, u, v) are obtained through the microlens array.

Information related to the angle of light can be obtained by planen optical technology. Due to the nature of the planenote technology that acquires information related to the angle of the light, the planenote technique is also called the lighten field technique.

Conventional imaging devices, such as camera modules, acquire images after focusing on the subject. On the other hand, a light field photographing apparatus using a light field technique can perform optical re-focus on an acquired image after acquiring an image. That is to say, when an image is acquired by a conventional photographing apparatus, among the images, there may be an image which is out of focus and therefore not suitable for use or can not be used. On the other hand, the light field photographing apparatus can arbitrarily readjust the focus of the obtained image. Therefore, with respect to the image obtained by the light field photographing apparatus, the case where the image can not be used because the focus is not adjusted does not occur. In addition, the light field photographing apparatus can generate an all-in-focus image that can be focused on each object of all objects in the image.

Another feature of the light field photographing apparatus is refocusing. The light field photographing apparatus records light information in all directions. Therefore, the light field photographing apparatus can generate an image at another optically proper point within the limit of the information of the recorded light. Since the images can be generated at various points in time, the light-field photographing apparatus can be suitable for multi-view stereoscopic display.

The light field photographing apparatus must use the information space of the sensor by dividing it into (u, v) information corresponding to each resolution and (x, y) information corresponding to the spatial resolution in order to generate an image . For example, when a large amount of information for each resolution is recorded for an image, information on a valid spatial resolution can not be recorded much. On the contrary, if a large amount of information on the effective spatial resolution is recorded for the image, information on each resolution can not be recorded much. Therefore, in the design of the light-fill photographing apparatus, how to divide and store the information about each resolution and the spatial resolution is a problem.

With respect to a light field photographing apparatus, Korean Patent Laid-Open No. 10-2010-0088326 discloses a multi-view light field data capturing apparatus using a variable modulator.

One embodiment can provide an apparatus and method for providing information on more resolution or more spatial resolution without increasing the resolution of the image sensor.

One embodiment may provide an apparatus and method for generating image data using a mirror of some of a plurality of mirrors or some of a plurality of lenses at one instant through time sharing.

One embodiment can provide an apparatus and method in which one mirror or one lens provides light to a plurality of elements of an image sensor at a moment.

At one side, a mirror array; And an image sensor, wherein a mirror of a part of the plurality of mirrors of the mirror array provides light to a plurality of elements of the image sensor.

The remaining mirror of the part of the plurality of mirrors other than the mirror may not provide light to the image sensor.

In two adjacent mirrors of the plurality of mirrors, if one of the two mirrors provides light to the plurality of elements of the image sensor as the part of the mirror, the other of the two mirrors One mirror may not provide light to the image sensor as the remaining mirror.

The angle of rotation of each mirror of the plurality of mirrors may vary.

Each of the mirrors may operate as a mirror of one of the mirror and the remaining mirror according to the rotation angle.

The light field photographing apparatus may further include a controller for controlling the rotation angle.

Each mirror of the plurality of mirrors may be selected as the mirror of the part at least one of the plurality of views according to the time division.

The light field photographing apparatus can select part of the plurality of mirrors as the part of mirrors in accordance with the time division and allows the selected part of the mirrors to provide light to the plurality of elements of the image sensor, It is possible to generate image data of a larger size than the resolution.

Wherein the light field photographing apparatus is capable of generating the image data through n time division and wherein each mirror of the plurality of mirrors provides light to a plurality of elements of the image sensor at least one time division of the n time division can do.

The n may be an integer of 2 or more.

The light field photographing apparatus may further include a main lens.

The mirror may reflect light passing through the main lens to provide light to a plurality of elements of the image sensor.

On another side, a mirror of some of the plurality of mirrors of the mirror array provides light to a plurality of elements of the image sensor; And a light field photographing method including the step of the image sensor generating image data according to the light provided to the plurality of elements.

On another side, a micro lens array; And an image sensor, wherein some lenses of the plurality of lenses of the microlens array provide light to a plurality of elements of the image sensor.

The remaining lenses other than the part of the plurality of lenses may not provide light to the image sensor.

In two adjacent lenses among the plurality of lenses, when one of the two lenses provides light to the plurality of elements of the image sensor as the part of the lens, the other of the two lenses One lens may not provide light to the image sensor as the remaining lens.

The light field photographing apparatus may further include a plurality of regions, and may further include a mask for controlling transmission of light to each region of the plurality of regions.

And each lens of the plurality of lenses can be operated by the mask as one of the lens and the remaining lens.

Wherein a lens adjacent to a region through which light passes is operable as a part of the lens among the plurality of regions, and a lens adjacent to an area not passing light among the plurality of regions includes a lens And can operate as the remaining lens.

The light field photographing apparatus may further include a controller for electronically controlling the mask.

Each lens of the plurality of lenses may be selected as the partial lens at at least one of a plurality of viewpoints according to the time division.

Wherein the light field photographing apparatus is capable of selecting the plurality of lenses as the partial lens in accordance with the time division and comparing the resolution of the image sensor by causing the selected lens to provide light to a plurality of elements of the image sensor It is possible to generate image data of a larger size.

Wherein the light field photographing apparatus is capable of generating the image data through n times of division and each lens of the plurality of lenses provides light to a plurality of elements of the image sensor in at least one time division of the n time division can do.

The n may be an integer of 2 or more.

The light field photographing apparatus may further include a main lens.

The partial lens may provide light to a plurality of elements of the image sensor by focusing light passing through the main lens to each element of the plurality of elements.

There is provided an apparatus and method for providing information on more resolution or more spatial resolution without increasing the resolution of the image sensor.

An apparatus and method for generating image data using a lens of some of a plurality of mirrors or a plurality of lenses of a plurality of mirrors at a moment through time division is provided.

An apparatus and method are provided in which one mirror or one lens provides light to a plurality of elements of an image sensor at a moment.

1 shows a photographing method of planen optical 1.0.
2 shows a photographing method of planen optical 2.0.
3 shows a light field photographing apparatus using a mirror according to an embodiment.
4 is a flowchart of a light field photographing method according to an embodiment.
5 shows a light field photographing apparatus using a mirror according to an embodiment.
6 is a flowchart of a light field photographing method according to an embodiment.

In the following, embodiments will be described in detail with reference to the accompanying drawings. It should be understood that the embodiments are different, but need not be mutually exclusive.

The terms used in the embodiments can be interpreted based on the actual meaning of the terms that are not the names of simple terms and the contents throughout the specification.

In embodiments, the connection relationship for a particular portion and the other portion may include an indirect connection relationship that is connected via another portion therebetween in addition to the direct connection relationship between the specific portion and the other portion. Like reference numerals in the drawings denote like elements.

1 shows a photographing method of planen optical 1.0.

The photographing apparatus of Planen-Optic 1.0 may include a main lens 110, a microlens array 120, and an image sensor 130.

When the light passes through the center lens 110, light passing through the main lens 110 may be focused on the microlens array 120. The lens L _i of the microlens array 120 and the region I _i of the image sensor may correspond to each other. For example, light passing through the lens L _i may be projected onto an area I _i of the image sensor, and information about light passing through the lens L _i may be recorded in an area I _i of the image sensor.

If there are n number of image sensor elements in region I _i of the image sensor, there are three items of information it may be recorded with respect to the light passing through the lens L _i. n may be an integer of 1 or more. For example, if the area I ₃ of the image sensor are three image sensor elements I _3-1, I _3-2 and I _3-3, the image sensor element with respect to the light passing through the lens L ₃ I _{3- 1} , I _3-2, and I _3-3 , and all three pieces of information can be generated.

2 shows a photographing method of planen optical 2.0.

The photographing apparatus of Planen-Optic 2.0 may include a main lens 210, a microlens array 220, and an image sensor 230.

When the light passes through the center lens 210, the light passing through the main lens 210 can be focused in front of the microlens array 220. In addition, the lenses of the microlens array 220 may be focused on a focal plane.

Light passing through the lens of the microlens array 220 can be projected onto the image sensor 230 on the sensor plane. Further, the light passing through the lens can be recorded in the area of the image sensor.

In both Planen-Optic 1.0 and Planen-Optic 2.0 described above, the image sensor is located behind the microlens array. In other words, the main lens, the microlens array, and the image sensor are arranged in the order of the main lens, the microlens array, and the image sensor. In addition, in both of Planen-Optic 1.0 and Planen-Optic 2.0 described above, one lens of the microlens array may correspond to one area of the image sensor. Here, the area of the image sensor may be a unit of information recording or a unit of information generation.

3 shows a light field photographing apparatus using a mirror according to an embodiment.

The light field photographing apparatus 300 may include a main lens 310, a mirror array 320, a controller 330, and an image sensor 340.

Mirror array 320 may include mirrors. Mirrors M ₀ through M ₆ are illustrated by way of example in FIG. 3 as mirrors of mirror array 320. Each mirror of the mirror array 320 may be a micro mirror. For example, the mirror array 320 may operate in the same manner as a mirror array in a Digital Light Processing (DLP) projector.

The plurality of mirrors may be arranged in a two-dimensional form in the mirror array 320.

The image sensor 340 may include elements. In FIG. 3, elements I ₀ through I ₆ are illustrated by way of example as elements of the image sensor 340. Each element of the image sensor 340 may generate information about the light projected on the element.

The light field photographing apparatus 300 can operate in time division. Depending on the time division, at one point in time, the mirrors of some of the plurality of mirrors of the mirror array 320 may be activated and the remaining mirrors of the plurality of mirrors may be deactivated. Further, at other times, some previously activated mirrors may be deactivated and at least some of the previously deactivated remaining mirrors may be activated. That is to say, as time passes, the plurality of mirrors of the mirror array 320 may be activated sequentially.

The activated mirror may be a mirror that provides light to the image sensor 340 or a mirror that is controlled to provide light to the image sensor 340. The deactivated mirror may be a mirror that does not provide light to the image sensor 340 or a mirror that is controlled to not provide light to the image sensor 340.

An activated mirror of some of the plurality of mirrors of the mirror array 320 may provide light to the image sensor 340. Some of the activated mirrors may record information in the area of the image sensor 340 by providing light to the image sensor 340. Here, the object to which light is provided may be a plurality of sensors instead of only one sensor of the image sensor 340. In other words, an activated mirror of some of the plurality of mirrors of the mirror array 320 may reflect light transmitted through the main lens 310 to provide light to a plurality of elements of the image sensor 340. An activated mirror of some of the plurality of mirrors of the mirror array 320 may focus on each of the plurality of elements of the image sensor 340 and may reflect light focused on each of the plurality of elements .

Some of the activated mirrors may provide light to a plurality of elements of the image sensor 340 by reflecting light that has passed through the main lens 310. That is to say, some activated mirrors can provide light in a wider area of the image sensor 340 and record information in a wider area of the image sensor 340 compared to the conventional planar optic method.

In addition, the remaining deactivated mirrors of the plurality of mirrors of the mirror array 320 may not provide light to the image sensor 340. In other words, the deactivated mirror of the rest of the plurality of mirrors may not reflect the light transmitted through the main lens 310. Alternatively, the remaining deactivated mirror of the plurality of mirrors may reflect the light transmitted through the main lens 310 to an area not affecting the image sensor 340.

Each mirror of the plurality of mirrors of the mirror array 320 may be selected as a part of the activated mirror at at least one of the plurality of views according to the time division. For example, the plurality of mirrors of the mirror array 320 may be classified into a plurality of groups, and one group of the plurality of groups may be selected at a moment according to the time division. The mirror of the selected group can be some active mirror. In addition, the mirror of the remaining group other than the selected one of the plurality of groups may become the remaining inactivated mirror. Depending on the time division, a plurality of groups may be selected one by one by the controller 330 sequentially.

For example, at the time T ₀ , the mirror M ₃ may be selected as the activated mirror, and the mirror M ₂ and the mirror M ₄ may be selected as the inactivated mirror. The activated mirror M ₃ may record information on all of the plurality of elements I ₂ , I _3, and I ₄ of the image sensor 340. Mirrors M ₂ and M _{4 that} are deactivated mirrors may not record information to any element of the image sensor 340.

Further, at the next point in time T ₁ , mirrors M ₂ and M ₄ can be selected as activated mirrors, and mirror M ₃ can be selected as inactivated mirrors. M ₂ may write information to all of the plurality of elements I ₀ , I _1, and I ₂ of the image sensor 340 and M ₄ may write information to a plurality of elements I ₄ , I ₅ , I ₆ < / RTI > A disabled mirror the mirror M ₃ can not write information to any element of the image sensor 340.

The light field photographing apparatus 300 selects some of the plurality of mirrors of the mirror array 320 as a part of the activated mirrors in accordance with the time division and a selected part of the activated mirrors is reflected by the plurality of elements of the image sensor 340 Image data of a larger size than that of the image sensor 340 can be generated. That is to say, the resolution of the image data generated by the light field photographing apparatus 300 may be larger than the resolution of the image sensor 340. Here, the resolution of the image sensor 340 may correspond to the number of elements of the image sensor 340.

The light field photographing apparatus 300 may combine the information obtained at the image sensor 340 at a plurality of points of view to generate image data. For example, the light field image pickup apparatus 300 is the image data by using the information obtained by the device of the image sensor 340 in the information and the time T ₁ obtained by the elements of the image sensor 340 at the time T ₀ Lt; / RTI > As described above, when one activated mirror records information in three elements at one time, the light field photographing apparatus 300 has three times the resolution of the image sensor 340 Image data can be generated. Further, in some cases, the light field photographing apparatus 300 can generate image data through three times of division.

The light field photographing apparatus 300 can generate image data through n time division. For example, a plurality of mirrors of the mirror array 320 may be classified into n groups. Each mirror of the plurality of mirrors of mirror array 320 may provide light to a plurality of elements of image sensor 340 in at least one time division of n time division. For example, each mirror of the plurality of mirrors of mirror array 320 may belong to at least one group of n groups. Here, n may be an integer of 2 or more.

For example, for a plurality of mirrors M ₀ to M _k of the mirror array 320, when the value of j% n is p, the mirror M _j may belong to the group p. The group p may be a group of mirrors activated at the (p + 1) th time point of n time division. k may be an integer of 1 or more. j may be an integer equal to or greater than 0 and equal to or less than k. p may be an integer of 0 or more and n-1 or less. For example, at the first point in time, M ₀ , M _n , M _2n, and M _3n among the plurality of mirrors of the mirror array 320 may be activated. At the second point in time, M ₁ , M _{n + 1} , M _{2n + 1,} and M _{3n + 1} of the plurality of mirrors of the mirror array 320 can be activated. M _n-1 , M _2n-1 , M _3n-1, and M _4n-1 among the plurality of mirrors of the mirror array 320 may be activated at the (n-1) th time point.

An activated mirror of some of the plurality of mirrors of the mirror array 320 may provide light to a plurality of adjacent elements of the image sensor 340. Here, a plurality of adjacent elements may have a one-dimensional (e.g., linear) or two-dimensional (e.g., rectangular or circular) shape. While an activated mirror provides light for a plurality of adjacent elements, a malfunction may occur where multiple mirrors provide light to a single element if the mirror adjacent to the activated mirror also provides light to the element. In two adjacent mirrors of the plurality of mirrors of the mirror array 320, if one of the two mirrors provides light for a plurality of elements of the image sensor as a part of the activated mirror, The other mirror of the mirrors may not provide light to the image sensor 340 as the remaining deactivated mirror.

The activation of each mirror of the plurality of mirrors of the mirror array 320 can be determined by the rotation angle. For example, a mirror can be activated to provide light to a plurality of elements of the image sensor 340 when the angle of rotation of the mirror corresponds to a predetermined value or a value within a predetermined range. Alternatively, if the rotation angle of the mirror does not correspond to the predetermined value or the value within the predetermined range, the mirror may be deactivated so as not to provide light to the image sensor 340.

The angle of rotation of each mirror of the plurality of mirrors of mirror array 320 may vary. Each mirror can operate as a mirror of one of the activated mirrors and the rest of the activated mirrors depending on the angle of rotation. The controller 330 can control the rotation angle of each mirror of the plurality of mirrors of the mirror array 320. [ Alternatively, the controller 330 may control the rotation angle of the mirror array 320 and may rotate the mirror of each mirror of the mirror array 320 as the rotation angle of the mirror array 320 is changed by control Angles can vary.

Each mirror of the plurality of mirrors of mirror array 320 or mirror array 320 may comprise a rotational axis and may rotate about a rotational axis.

The controller 330 can control each mirror of the plurality of mirrors of the mirror array 320 or the mirror array 320 at different angles from each other at different points in time according to the time division.

4 is a flowchart of a light field photographing method according to an embodiment.

At step 410, a mirror of some of the plurality of mirrors of mirror array 320 may be selected as the activated mirror. The remaining mirrors other than the activated mirror among the plurality of mirrors of the mirror array 320 can be selected as inactivated mirrors.

At step 420, an activated mirror of some of the plurality of mirrors of the mirror array 320 may provide light to a plurality of elements of the image sensor 340.

In addition, other deactivated mirrors other than the activated mirrors of some of the plurality of mirrors of the mirror array 320 may not provide light to the image sensor 340.

In step 430, the image sensor 340 may generate image data according to light provided to a plurality of elements provided with light.

When light is provided to an element of the image sensor 340, the element can generate data corresponding to the provided light.

In step 440, the controller 330 may determine whether to terminate the light-fill shooting. Step 410 can be repeated if the controller 330 determines that the light-fill shooting is not to end. If the controller 330 determines to terminate the light-fill shooting, the procedure may end.

If step 410 is repeated, a mirror other than the active mirror of some of the now active mirrors may be selected as the activated mirror.

For example, by n time division, steps 410, 420 and 430 may be repeated n times. For a plurality of mirrors M ₀ to M _k of the mirror array 320, when the value of j% n is p, the mirror M _j can belong to the group p. The group p may be a group of mirrors activated in the (p + 1) th iteration of the iterations by n time division. k may be an integer of 1 or more. j may be an integer equal to or greater than 0 and equal to or less than k. p may be an integer of 0 or more and n-1 or less.

In the first iteration of step 410, M ₀ , M _n , M _2n, and M _3n , among the plurality of mirrors of the mirror array 320, may be activated. In the second iteration of step 410, M ₁ , M _{n + 1} , M _{2n + 1,} and M _{3n + 1,} etc. among the plurality of mirrors of mirror array 320 may be activated. In the (n-1) th iteration of step 410, M _n-1 , M _2n-1 , M _3n-1, and M _4n-1 among the plurality of mirrors of the mirror array 320 may be activated.

5 shows a light field photographing apparatus using a mirror according to an embodiment.

The light field photographing apparatus 500 may include a main lens 510, a microlens array 520, a mask 530, a controller 535, and an image sensor 540.

The microlens array 520 may include lenses. In FIG. 5, lenses L ₀ through L ₆ are shown as exemplary lenses as the lenses of the microlens array 520. Each lens of the microlens array 520 may be a microlens.

The plurality of lenses may be arranged in a two-dimensional form in the microlens array 520.

The image sensor 540 may include elements. In FIG. 5, elements I ₀ through I ₆ are illustrated by way of example as elements of the image sensor 540. Each element of the image sensor 540 can generate information about the light projected on the element.

The light field photographing apparatus 500 can operate in time division. Depending on the time division, at one point in time, some lenses of the plurality of lenses of the microlens array 520 can be activated, and the remaining lenses of the plurality of lenses can be deactivated. Also, at other times, some previously activated lenses may be deactivated and at least some of the previously deactivated lenses may be activated. That is to say, as time passes, the plurality of lenses of the microlens array 520 may be activated sequentially.

The activated lens can be a lens that provides light to the image sensor 540 or a lens that is controlled to provide light to the image sensor 540. The inactivated lens may be a lens not providing light to the image sensor 540 or a lens controlled not to provide light to the image sensor 540. [

Some of the activated lenses of the plurality of lenses of the microlens array 520 may provide light to the image sensor 540. Some of the activated lenses may record information in the area of the image sensor 540 by providing light to the image sensor 540. [ Here, the object to which the light is provided may be a plurality of sensors instead of only one sensor of the image sensor 540. In other words, the activated lens of some of the plurality of lenses of the microlens array 520 focuses the light transmitted through the main lens 510 to a plurality of elements of the image sensor 540, Lt; RTI ID = 0.0 > light. ≪ / RTI > The activated lens of some of the plurality of lenses of the microlens array 520 can focus on each of the plurality of elements of the image sensor 540 and can project light focused on each of the plurality of elements have.

Some activated lenses may provide light to a plurality of elements of the image sensor 540 by focusing light passing through the main lens 510 to a plurality of elements of the image sensor 540. That is to say, some activated lenses can provide light over a wider area of the image sensor 540 and record information over a wider area of the image sensor 540 than conventional planar optics.

In addition, the remaining deactivated lenses of the plurality of lenses of the microlens array 520 may not provide light to the image sensor 540. That is to say, the remaining inactive lenses of the plurality of lenses may not reflect the light transmitted through the main lens 510. Alternatively, the remaining inactive lenses of the plurality of lenses can focus the light transmitted through the main lens 510 to an area that does not affect the image sensor 540.

Each lens of the plurality of lenses of the microlens array 520 can be selected as a part of the activated lens at at least one of the plurality of viewpoints according to the time division. For example, the plurality of lenses of the microlens array 520 may be classified into a plurality of groups, and one group of the plurality of groups may be selected at a time according to the time division. The selected group of lenses can be some activated lenses. In addition, the remaining group of lenses other than the selected one of the plurality of groups may be the remaining inactive lenses. Depending on the time division, a plurality of groups may be selected one by one by the controller 535 sequentially.

For example, at the time T _0, it can be selected by a lens L ₃ active lens, a lens L ₂ and the lens L ₄ may be selected in a disabled lens. L _{3, which} is an activated lens, can record information on all of a plurality of elements I ₂ , I _3, and I ₄ of the image sensor 540. The lenses L ₂ and L _{4, which} are inactivated lenses, may not record information to any element of the image sensor 540.

Further, at the next time point T ₁ , the lenses L ₂ and L ₄ can be selected as the activated lens, and the lens L ₃ can be selected as the inactivated lens. L ₂ may write information to all of the plurality of elements I ₀ , I _1, and I ₂ of the image sensor 540 and L ₄ may write information to a plurality of elements I ₄ , I ₅ , I ₆ < / RTI > The lens L _{3, which} is an inactive lens, may not record information to any element of the image sensor 540.

The light field photographing apparatus 500 selects a part of the plurality of lenses of the microlens array 520 as a part of the activated lenses according to the time division and the selected part of the activated lenses is moved to a plurality of elements of the image sensor 540 Image data of a larger size than the resolution of the image sensor 540 can be generated. That is to say, the resolution of the image data generated by the light field photographing apparatus 500 may be larger than the resolution of the image sensor 540. Here, the resolution of the image sensor 540 may correspond to the number of elements of the image sensor 540.

The light field photographing apparatus 500 may combine the information obtained at the image sensor 540 at a plurality of points of view to generate image data. For example, the light field image pickup apparatus 500 includes an image data by using the information obtained by the device of the image sensor 540 in the information and the time T ₁ obtained by the device of the image sensor 540 at the time T ₀ Lt; / RTI > As described above, when one activated lens at one time writes information to three elements, the light field photographing apparatus 500 has a resolution three times that of the image sensor 540 Image data can be generated. Further, in some cases, the light field photographing apparatus 500 can generate image data through three times of division.

The light field photographing apparatus 500 can generate image data through n times of time division. For example, the plurality of lenses of the microlens array 520 may be classified into n groups. Each lens of the plurality of lenses of the microlens array 520 may provide light to a plurality of elements of the image sensor 540 in at least one time division of n time division. For example, each lens of the plurality of lenses of the microlens array 520 may belong to at least one group of n groups. Here, n may be an integer of 2 or more.

For example, in a plurality of lenses L ₀ to L _k of the microlens array 520, when the value of j% n is p, the lens L _j can belong to the group p. The group p may be a group of lenses activated at the (p + 1) th time point of n time division. k may be an integer of 1 or more. j may be an integer equal to or greater than 0 and equal to or less than k. p may be an integer of 0 or more and n-1 or less. For example, at the first point in time, L ₀ , L _n , L _2n , L _3n, etc. among the plurality of lenses of the microlens array 520 can be activated. At the second time point, among the plurality of lenses of the microlens array 520, L ₁ , L _{n + 1} , L _{2n + 1} , L _{3n + 1,} and so on can be activated. ₁ , L _2n-1 , L _3n-1 and L _4n-1 among the plurality of lenses of the microlens array 520 can be activated at the (n-1) th time point.

The activated lens of some of the plurality of lenses of the microlens array 520 may provide light to a plurality of adjacent elements of the image sensor 540. [ Here, a plurality of adjacent elements may have a one-dimensional (e.g., linear) or two-dimensional (e.g., rectangular or circular) shape. While an activated lens provides light to a plurality of adjacent elements, if an activated lens and a neighboring lens also provide light to the element, malfunction may occur in which a plurality of lenses provide light to one element. In two adjacent lenses among the plurality of lenses of the microlens array 520, when one of the two lenses provides light to a plurality of elements of the image sensor as a part of the activated lens, And the other one of the lenses may not provide light to the image sensor 540 as the other deactivated lens.

A mask 530 may be used to select some of the activated lenses and the remaining deactivated lenses. The controller 535 may electronically control the mask 530. That is to say, the mask 530 may be an electronic mask.

The mask 530 may include a plurality of regions. The mask 530 may control the passage of light for each region of the plurality of regions of the mask 530.

A plurality of regions of the mask 530 and a plurality of lenses of the microlens array 520 may correspond to each other. The plurality of regions of the mask 530 and the plurality of lenses of the microlens array 520 closest to each other and the lenses may correspond to each other. Alternatively, an area disposed in front of the specific lens of the microlens array 520 among the plurality of areas of the mask 530 may correspond to the specific lens.

Each lens of the plurality of lenses of the microlens array 520 may be operated by the mask 530 as one lens of one of the activated lens and the other deactivated lens. The activation of each lens of the plurality of lenses of the microlens array 520 can be determined by the mask 530. [ For example, if the area of the mask 530 corresponding to the lens of the microlens array 520 is configured to allow light to pass, the lens can act as an activated lens. Alternatively, if the area of the mask 530 corresponding to the lens of the microlens array 520 is configured not to allow light to pass, the lens can act as an inactivated lens. That is to say, the mask 530 can act as a barrier.

The controller 535 can control the area of some of the plurality of areas of the mask 530 to be the activated area (i.e., the area through which light passes), and the remaining area can control the inactive area It is possible to control the area to be controlled. Of the plurality of lenses of the microlens array 520, the lens corresponding to the activated region may be an activated lens, and the lens corresponding to the inactivated region may be an inactive lens. In other words, in the plurality of lenses of the microlens array 520, the lens adjacent to the light-transmitting area among the plurality of areas of the mask 530 operates as a part of the activated lens, The lens adjacent to the area not passing through the lens can operate as the remaining inactive lens.

For each region of the plurality of regions of the mask 530, the controller 535 may control whether each of the regions passes light. The controller 530 may change the activated region of the mask 530 at different points in time according to the time division.

6 is a flowchart of a light field photographing method according to an embodiment.

In step 610, a portion of the plurality of lenses of the microlens array 520 may be selected as the activated lens. The remaining lenses other than the activated lens among the plurality of lenses of the microlens array 520 can be selected as inactivated lenses.

In step 620, an activated lens of some of the plurality of lenses of the microlens array 520 may provide light to a plurality of elements of the image sensor 540.

In addition, some of the plurality of lenses of the microlens array 520, other than the activated lens, may not provide light to the image sensor 540. [

In step 630, the image sensor 540 may generate image data according to light provided to a plurality of elements provided with light.

When light is provided to an element of the image sensor 540, the element can generate data corresponding to the provided light.

In step 640, the controller 635 may determine whether to terminate the light-fill shooting. If the controller 635 determines not to end the light-fill shooting, step 610 may be repeated. If the controller 635 determines to terminate the light-fill shooting, the procedure may end.

If step 610 is repeated, a lens other than the active lens of some of the now active lenses may be selected as the activated lens.

For example, by n time division, step 610, step 620 and step 630 may be repeated n times. For a plurality of lenses L ₀ to L _k of the microlens array 520, when the value of j% n is p, the lens L _j can belong to the group p. The group p may be a group of lenses activated in the (p + 1) th iteration of the iterations by n time division. k may be an integer of 1 or more. j may be an integer equal to or greater than 0 and equal to or less than k. p may be an integer of 0 or more and n-1 or less.

In the first iteration of step 610, L ₀ , L _n , L _2n , L _3n, etc. among the plurality of lenses of the microlens array 520 may be activated. In the second iteration of step 610, L ₁ , L _{n + 1} , L _{2n + 1} , L _{3n + 1,} etc. among the plurality of lenses of the microlens array 520 may be activated. ₁ , L _2n-1 , L _3n-1 and L _4n-1 among the plurality of lenses of the microlens array 520 can be activated in the (n-1) th iteration of step 610.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

300: Light field shooting device
310: main lens
320: mirror array
330:
340: Image sensor
500: Light field shooting device
510: main lens
520: microlens array
530: mask
535:
540: Image sensor

Claims (20)

Mirror array; And
Comprising an image sensor,
Wherein a mirror of some of the plurality of mirrors of the mirror array provides light to a plurality of elements of the image sensor. The method according to claim 1,
Wherein the mirror of the part of the plurality of mirrors other than the mirror does not provide light to the image sensor. 3. The method of claim 2,
In two adjacent mirrors of the plurality of mirrors, if one of the two mirrors provides light to the plurality of elements of the image sensor as the part of the mirror, the other of the two mirrors And one mirror does not provide light to the image sensor as the remaining mirror. 3. The method of claim 2,
Wherein a rotation angle of each mirror of the plurality of mirrors changes,
Wherein each mirror acts as a mirror of one of the mirror and the remaining mirror according to the rotation angle. 5. The method of claim 4,
A controller for controlling the rotation angle
Further comprising: The method according to claim 1,
Wherein each mirror of said plurality of mirrors is selected as said part of said mirror at at least one of a plurality of time points according to time division. The method according to claim 6,
The light field photographing apparatus selects a part of the plurality of mirrors as the part of the mirrors according to a time division and causes the selected part of the mirrors to provide light to the plurality of elements of the image sensor, The image data of a larger size is generated. 8. The method of claim 7,
The light field photographing apparatus generates the image data through n time division, and each mirror of the plurality of mirrors provides light to a plurality of elements of the image sensor in at least one time division of the n time division, And n is an integer of 2 or more. The method according to claim 1,
Main lens
Further comprising:
Wherein the mirror reflects light passing through the main lens to provide light to a plurality of elements of the image sensor. The mirror of some of the plurality of mirrors of the mirror array providing light to a plurality of elements of the image sensor; And
Wherein the image sensor generates image data according to the light provided to the plurality of elements
And a light source for emitting light. A microlens array; And
Comprising an image sensor,
Wherein some of the plurality of lenses of the microlens array provide light to a plurality of elements of the image sensor. 12. The method of claim 11,
And the remaining lenses other than the part of the plurality of lenses do not provide light to the image sensor. 13. The method of claim 12,
In two adjacent lenses among the plurality of lenses, when one of the two lenses provides light to the plurality of elements of the image sensor as the part of the lens, the other of the two lenses And one lens does not provide light to the image sensor as the remaining lens. 13. The method of claim 12,
A mask for controlling the transmission of light to each region of the plurality of regions,
Further comprising:
Wherein each lens of the plurality of lenses is operated by the mask as one of the lens and the remaining lens. 15. The method of claim 14,
The lens of the plurality of lenses, which is adjacent to the area through which light passes, acts as the lens of the part of the plurality of areas, and the lens adjacent to the area of the plurality of areas, A light field photographing device operating as a lens. 16. The method of claim 15,
A controller for electronically controlling the mask
Further comprising: The method according to claim 1,
Wherein each lens of the plurality of lenses is selected as the partial lens at at least one of a plurality of time points according to a time division. 18. The method of claim 17,
Wherein the light field photographing apparatus selects the plurality of lenses as the partial lens in accordance with the time division and causes the selected lens to provide light to the plurality of elements of the image sensor, A light field photographing apparatus that generates image data of a large size. 19. The method of claim 18,
The light field photographing apparatus generates the image data through n time division, each lens of the plurality of lenses providing light to a plurality of elements of the image sensor in at least one time division of the n time division, And n is an integer of 2 or more. 12. The method of claim 11,
Main lens
Further comprising:
Wherein the plurality of lenses provide light to a plurality of elements of the image sensor by focusing light passing through the main lens to each element of the plurality of elements.

Images