JP2012147087A - Apparatus and method for generating image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily generate a composite image with high resolution without performing a pixel interpolation processing.SOLUTION: A multi-eye camera includes: a plurality of photographic lenses which are two-dimensionally arranged; imaging elements which individually image subject images which are image-formed by the respective photographic lenses; and a composite processing section 25 compositing the plurality of images obtained from the respective imaging elements. The composite processing section 25 includes: a relative position detecting section 40; a superposition section 41; and a pixel arranging section 42. The relative position detecting section 40 detects relative positions of a reference image with respect to remaining images. The superposition section 41 superposes the plurality of images so that deviation based on the relative position becomes zero on a virtual image region in which pixels are increased. The pixel arranging section 42 generates a single composite image by selecting and arranging a pixel whose center is the nearest to centers of respective pixels constituting the virtual image region from the plurality of superposed images.

Description

  The present invention relates to an image generation apparatus and method for generating a single high-resolution image using a plurality of images having different parallaxes.

  2. Description of the Related Art Conventionally, there is known an image reconstruction device that generates a single high-resolution object image from a plurality of low-resolution object reduced images formed on a light-receiving surface of a single light-receiving element by a microlens array ( Patent Document 1). The microlens array is formed by integrating a plurality of microlenses in a two-dimensional manner. This device calculates the relative position between the reduced object images using a plurality of reduced object images, the relative position of all the reduced object images with respect to the reference reduced object image, and reduces each object based on the calculated relative position. Pixels of the image are rearranged on the same enlarged area (predetermined image memory area) to generate a single object image with multiple pixels, and then the rearranged pixels of the single object image deleted After performing the interpolation processing, sharpening processing such as edge enhancement is performed to generate a single high-definition composite image.

JP 2003-141529 A

  However, in the invention described in Patent Document 1, the pixels of the reference reduced object image are arranged in the same enlarged area in accordance with the scale, and then the pixels of the other reduced object image are placed in the same area based on the relative positions. Since a single object image is generated by disposing the pixel, the deleted pixel is included depending on the object distance between the object and the microlens array. As a result, an interpolation process for interpolating the missing pixel using the surrounding pixel values is required, and the process may take time.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image generation apparatus and method capable of simply performing a synthesis process without performing a troublesome process such as pixel interpolation.

  An aspect of the image generating apparatus exemplifying the present invention includes: a relative position detecting unit that detects a relative position of a remaining image with respect to a reference image of a plurality of images; Superimposing means for superimposing on the image area based on the detected relative position so that the deviation is zero; a plurality of pixels in which the pixel closest to the center is superimposed on the center of each pixel constituting the virtual image area And a pixel arrangement unit that selects and arranges from among the images.

  According to the image generating apparatus of the present invention, a pixel having the closest center is selected from a plurality of superimposed images and arranged at the center of each pixel constituting a virtual image area having multiple pixels. Therefore, pixels that are deleted in the virtual image region never occur. Therefore, since no interpolation process is required, the synthesis process can be simplified.

It is a front perspective view which shows the multi-lens camera which employ | adopted this invention. It is a rear perspective view of a multi-view camera. It is a block diagram explaining the electrical structure of a multi-view camera. It is a block diagram which shows the structure of a synthetic | combination process part. It is a flowchart explaining operation | movement of a multi-view camera. It is a flowchart explaining a synthetic | combination process. It is explanatory drawing which illustrates typically the condition which arranges a photographic lens and an image sensor with 2x2, for example, and picturizes a photographic subject. FIG. 8 is an explanatory diagram schematically illustrating four image data captured by the image sensor described in FIG. 7. It is explanatory drawing explaining the state displayed by the attitude | position based on the to-be-photographed image, calculating | requiring the relative position of a to-be-photographed object and an image pick-up element. It is explanatory drawing explaining the state which overlap | superposed each image data on the virtual image area. It is explanatory drawing explaining a mode that the pixel with the nearest center is selected from the several superimposed image and arrange | positioned in the center of each pixel which comprises a virtual image area | region. It is explanatory drawing for demonstrating the symbol used for the formula of a horizontal direction optical resolution and a horizontal direction Nyquist resolution. It is a flowchart explaining the operation | movement of camera shake correction.

  As shown in FIG. 1, the multi-lens camera 10 has 16 shooting openings 11 provided on the front surface of the camera body 12. A photographing lens and an imaging element are arranged in the back of each photographing opening 11, and the photographing opening, the photographing lens, and the imaging element constitute a single-eye imaging unit.

  As each imaging device, for example, an imaging device with a low pixel of 5M to 10M and low power consumption is used. Further, the optical axes of the photographing lenses are substantially parallel. Each imaging element is arranged so that the imaging surface is perpendicular to the optical axis of the imaging lens.

  The multi-lens camera 10 is provided with a shutter button 13 and a power switch 14 on the upper surface of the camera body 12, and acquires, for example, 16 pieces of image data by one shutter release, and synthesizes these image data. One high-definition image data is generated.

  The imaging openings 11 are arranged in a two-dimensional manner. The interval in the x and y directions of the imaging aperture 11 is set to a predetermined periodic pattern according to the M sequence that is a pseudo-random sequence. In this case, the autocorrelation function of the M sequence is close to the delta (δ) function, and has a feature that the correlation function value is constant except for the peak.

  In addition, as long as the space | interval of the imaging | photography opening 11 is a binary pseudorandom series, things other than M series, such as Q series (square remainder series), Gold series, Walsh code, etc., can also be used. The number of imaging apertures 11 is not limited to 16 as long as the interval in the two-dimensional direction is set to a predetermined periodic pattern that is a pseudo-random sequence.

  An LCD 15 is arranged behind the camera body 12 as shown in FIG. A touch panel operation unit is incorporated in the LCD 15. The scaling operation is performed using the touch panel operation unit. Of course, the LCD 15 displays a menu character, a picture such as an operation button, and the like, making it easy to operate.

  As shown in FIG. 3, each image sensor 20 is connected to an AFE 21 and a frame memory 22, picks up a subject image formed by the photographing lens 23, and outputs an image signal to the AFE 21.

  The AFE 21 includes a well-known CDS (correlated double sampling) / AGC (gain control amplifier circuit), A / D, and a signal processing circuit, and operates in synchronization with a pulse supplied from the TG together with the CPU 24. (Not shown). The signal processing circuit captures digital image data and performs corrections such as pixel defect correction, white balance correction, and gamma correction.

  Various operation signals are input to the CPU 24 from a touch panel operation unit provided on the LCD 15 (not shown). The CPU 24 simultaneously loads a plurality of image data into each frame memory 22 by one release operation. Each image data is reduced-resolution image data with low resolution, and is individually taken into the frame memory 22 and then read out to the synthesis processing unit 25. The composition processing unit 25 generates high-resolution composite image data using a plurality of image data having different parallaxes.

  In addition, the multi-lens camera 10 includes a distance measuring unit (ranging unit) 26, an angular velocity sensor (shake detecting unit) 27, a movement amount calculating unit (movement amount calculating unit) 28, and a moving mechanism (moving unit) 29. Has been. A focusing mechanism 30 for individually focusing the photographing lens 23 is provided for each photographing lens 23. The angular velocity sensor 27 detects a shake of the multi-lens camera 10 and sends the detected shake information to the movement amount calculation unit 28.

  The distance measuring unit 26 captures at least two of the plurality of image data, calculates a relative shift amount between the two images based on the two image data, and calculates a subject distance from the calculated shift amount. All the focusing mechanisms 30 are controlled, and all the photographing lenses 23 are individually moved to a focusing position according to the subject distance.

  For example, in the triangulation method using the principle of triangulation, if the center distance (base line length) d of the image sensor 20 that captured two image data, the focal length f of the photographing lens 23, and the parallax δ, the subject distance L is It is obtained by using an equation “L = f × d / δ” that is formed by a proportional relationship of triangles. The obtained subject distance information is sent to the movement amount calculation unit 28. Note that it is preferable to use image data obtained from the image sensor 20 having the longest baseline length d as the two image data.

  The movement amount calculation unit 28 calculates the movement amount for each image sensor 20 by the movement mechanism 29 based on the shake detected by the angular velocity sensor 27. Information on the calculated movement amount is sent to the movement mechanism 29. These pieces of movement information include movement direction information in addition to the movement amount.

  The moving mechanism 29 is arranged for each image pickup device 20, and includes an actuator such as a voice coil or a stepping motor, for example. The moving mechanism 29 uses the actuator to light the image pickup device 20 based on the movement amount calculated by the movement amount calculation unit 28. Instantly move in a direction perpendicular to the axis.

  The ROM 31 stores various programs and setting values necessary for executing the programs in advance. The RAM 32 is used as a work memory for the CPU 24 and as a temporary memory for each unit. The CPU 24, ROM 31, RAM 32, LCD driver 33, I / F 34, and composition processing unit 25 are connected by a bus 35.

  The photographing lens 23 may be a zoom lens or a focal length switching type lens. In this case, all the photographing lenses may be configured to change magnification in synchronization with the magnification operation.

  The CPU 24 records the composite image data generated by the image composition unit 25 in the recording unit 36 via the I / F 34. Note that a compression unit may be provided to record the composite image data in a format compressed by, for example, the JPEG method. Further, the CPU 24 generates display image data based on the composite image data, sends the display image data to the LCD driver 33, and displays the display image data on the LCD 15 as a through image under the control of the LCD driver 33.

  The CPU 24 controls the charge accumulation time (electronic shutter) of the image sensor 20, measures the luminance of the subject based on the image data obtained from the specific image sensor 20, and performs all imaging based on the measurement result. The exposure is controlled by changing the value of the electronic shutter of the element 20.

  As illustrated in FIG. 4, the composition processing unit 25 includes a relative position detection unit 40, a superposition unit 41, a pixel arrangement unit 42, a blur removal processing unit 43, and a reduction processing unit 44. The relative position detection unit 40 calculates a relative position of each of the plurality of images with respect to the remaining image with respect to the reference image in order to align the plurality of images. The superimposing unit 41 superimposes a plurality of images on a virtual image area having multiple pixels so that the deviation is zero based on each relative position. The pixel arrangement unit 42 selects and arranges a pixel having the closest center at the center of each pixel constituting the virtual image area from a plurality of superimposed images, thereby providing a single high resolution. Simple composite image data is generated. The blur removal processing unit 43 performs, for example, deconvolution processing on the composite image data to remove the influence of blur. The reduction processing unit 44 performs reduction processing on the composite image data that has been subjected to blur removal processing, for example, by a bilinear method, and outputs final composite image data.

  Next, the operation of the above configuration will be briefly described with reference to FIG. When the multi-eye camera 10 turns on the power switch 14, the moving mechanism 29 sets the image sensor 20 to the initial position (S-1). The initial position is a position where the center of the imaging surface coincides with the optical axis. The camera shake correction process is started (S-2). A detailed description of the camera shake correction process will be described later.

  In response to the half-pressing operation of the shutter button 13 (S-3), the distance measuring unit 26 calculates the subject distance (S-4), and the focusing mechanism 30 aligns all the photographing lenses 23 based on the subject distance. Move to the focal position (S-5). Then, in response to the full-pressing operation of the shutter button 13 (S-6), the CPU 24 controls to drive all the image sensors 20 and capture a plurality of image data having different parallaxes into the frame memory 22 (S-). 7). The acquired plurality of image data is sent to the composition processing unit 25, where a single high-quality composite image data is generated (S-8). The composite image data is sent to the recording unit 36 and recorded (S-9). In addition, after the shutter button 13 is fully pressed, the focusing operation of each photographing lens 23 may be performed.

  Here, the processing of the synthesis processing unit 25 will be described with reference to FIG. First, when the photographing lenses 23a to 23d are arranged in 2 rows (x direction) × 2 columns (y direction) as shown in FIG. 7, digital image data A to D acquired from the respective image sensors 20A to 20D. As shown in FIG. 8, reduced images 50a to 50d of the subject 50 are shown.

  Four squares A1 to A4, B1 to B4, C1 to C4, and D1 to D4 of each image data A to D represent pixels (pixels) corresponding to 2 × 2 = 4 pixels of the imaging elements 20A to 20D. ing. The image data A to D shown in FIG. 8 display reduced images 50a to 50d based on the orientations of the image sensors 20A to 20D. The reduced images 50a to 50d are parallax between the photographing lenses 23a to 23d and photographing. It is displayed in a posture including alignment errors of the lenses 23a to 23d and the image sensors 20A to 20D.

  The relative position detection unit 40 takes in a plurality of image data A to D, and performs a correlation operation of the reduced images 50a to 50d between certain reference image data A to D and any one of the image data A to D, thereby reducing the reduced images 50a to 50d. The relative positions between the reduced images 50a to 50d of the remaining image data A to D with respect to the reduced images 50a to 50d of the reference image data A to D are calculated by obtaining the relative positions between 50d (S- 10). The shift amount (shift amount) of all the image data A to D with respect to the reference image data A to D may be obtained in units of pixels of the image sensors 20A to 20D.

  Detection of the relative position between the reduced images 50a to 50d uses an average of edge coordinates, a gravity center of luminance, or the like. As a result, the relative position between the reduced images 50a to 50d and the image sensors A to D, that is, the posture deviation (position deviation, rotation deviation, magnification deviation) of each of the image data A to D with reference to the reduced images 50a to 50d. Can be detected. When the image data A to D are displayed based on the postures of the reduced images 50a to 50d, the postures of the image data A to D are shifted as shown in FIG.

  As shown in FIG. 10, the superposition unit 41 rotates, parallels, enlarges and reduces each of the image data A to D so that the shift amount of the reduced images 50 a to 50 d becomes zero. Processing is performed to superimpose the image on the virtual image area (memory) 51 having multiple pixels (S-11). The pixels 51a to 51p in the virtual image area 51 are set to a size that is 1/4 of the pixels B1 to B4, C1 to C4, and D1 to D4 of each image data A to D.

  Here, what is stored in the frame memory 22 is data in pixel units, and the matrix of the virtual image area 51 does not necessarily correspond to each pixel of the frame memory 22 on a one-to-one basis. Conventionally, since one pixel of the virtual image area 51 and one pixel of the frame memory 22 are shifted, one pixel of the virtual image area 51 corresponds to a combination of a plurality of pixels in the frame memory 22. Therefore, pixel interpolation of the value of one pixel in the virtual image area 51 is performed by a plurality of pixels in the corresponding frame memory 22.

  However, in the present embodiment, as shown in FIG. 11, the pixel arrangement unit 42 is a reduced image 50a in which the centers A1a to D1a are closest to the centers 51aa to 51pa of the pixels 51a to 51p constituting the virtual image region 51. The pixels A1 to A4, B1 to A4, B1 to B4, C1 to C4, and D1 to D4 of ˜50d are selected and arranged from the plurality of superimposed image data A to D (S-12).

  In FIG. 11, the distance from the center A1a of the pixel A1 of the image data A1 is H1, the distance from the center B1a of the pixel B1 of the image data B1 is H2, the distance from the center C1a of the pixel C1 of the image data C1 is H3, and the image If the distance from the center D1a of the pixel D1 of the data D1 is H4 and the relationship of the linear distance to the center 51aa is H3> H1> H2> H4, the center closest to the pixel 51a in the virtual image region 51 is the center. Since it is D1a, the value of the pixel D1 of the image data D having the center D1a is set. Subsequently, the pixel of the image data A to D having the center closest to the pixel 51b is selected from the pixels A2, B2, C2, and D2. Similarly, the remaining pixels 51c to 51p are selected. The value of each pixel in the virtual image area 51 obtained in this way is sequentially written in the buffer memory provided in the pixel arrangement unit 42. Thereby, since pixel interpolation is not required, a single high-resolution composite image data can be simply generated.

  When the relative position between the reduced image and the imaging position is reproduced well, the relative positional relationship between the reduced image and the image sensor is measured and stored in advance for each subject distance, and the relative position according to the subject distance is stored. May be read and used.

  The blur removal processing unit 43 performs, for example, a deconvolution process on the composite image data read from the buffer memory to remove the influence of the blur (S-13). Then, the reduction processing unit 44 performs reduction processing by, for example, the bilinear method on the composite image data that has been subjected to the blur removal processing (S-14). The composition processing unit 25 outputs the reduced data as final composite image data (S-15).

  In the above embodiment, the pixel having the closest center to the pixel of the virtual image region is selected from a plurality of pieces of image data and applied, but when applying a pixel that is too far from the center Problems may arise. In this case, a threshold value may be determined in advance, and if the distance exceeds the threshold value, the pixels in the virtual image area may be complemented based on the surrounding pixels.

  As the multi-lens camera 10, it is preferable to obtain a high-resolution image using a low-pixel imaging device 20. In order to prevent the sharpness of the composite image data from being lowered, no low-pass filter (low-pass filter) is provided between the photographing lens 23 and the image sensor 20. That is, the image data is synthesized using image data captured without using a low-pass filter.

  The optical resolution of the photographic lens 23 is, for example, twice or more higher than the Nyquist resolution determined by the image sensor 20, and lower than the resolution of the composite image data. Therefore, the image sensor 20 captures an image without the low-pass filter, and the composition processing unit 25 synthesizes the image using image data without the low-pass filter. A low pass filter may be provided. In that case, the bandwidth is set as described above.

  With the configuration as described above, a plurality of images having a Nyquist frequency or lower determined by the image sensor 20 are combined to draw out the resolving power of the image pickup lens 23 and the image sensor 20 with a higher resolution than that of the image sensor 20. Data can be obtained.

  As for the hand touch correction process, as shown in FIG. 13, the angular velocity sensor 27 is constantly monitored in response to turning on the power switch 14 (S-16). In response to the angular velocity sensor 27 detecting the shake (S-17), the movement amount calculation unit 28 calculates the movement amount for each image sensor 20 by the movement mechanism 29 based on the detected shake (S-18). ). The moving mechanism 29 moves the image sensor 20 according to the calculated movement amount, and corrects camera shake (S-19).

  In the above embodiment, the image sensor 20 is moved. However, instead of the image sensor 20, the photographic lens 23 may be moved, or both may be moved.

  As the image sensor described in the above embodiment, an image sensor such as a CCD or a CMOS can be used. In the above embodiment, a multi-lens digital camera has been described as an example of the imaging device. However, the present invention is not limited to this, and the present invention is applied to, for example, a camera-equipped mobile phone or a video camera. May be.

  There is a type of CMOS that can capture a plurality of frame image data at a high speed for one shutter release. When this type is used, a plurality of image data can be acquired for each image sensor in a time series with one shutter release. In this case, a plurality of frame averaging means for averaging the composite image data which is the frame image data is provided after the pixel arrangement unit 42. The multi-frame averaging means generates a next composition generated based on the next image data captured from the image sensor with respect to the previous image data generated based on the image data previously captured from the plurality of imaging elements. Image data is averaged. This process is repeated in accordance with the number of image data per image sensor obtained by one shutter release. That is, a plurality of composite image data generated based on image data per image sensor obtained by one shutter release may be averaged. Note that the number of the average frame image data is reduced and more of the plurality of frame image data is applied to the coordinates of the reference pixel than when the composite image data is generated and averaged by dividing it into a plurality of frame image data. You may comprise.

  The relative position detection means of the present invention calculates the relative position between the images by performing a correlation calculation on the shift amount between the subject images of the two images. By repeating the same procedure, the relative positions of all the remaining images with respect to a certain reference image are calculated.

  The distance from the photographic lens to the subject can be obtained based on the relative position detected by the relative position detecting means. In this case, the subject distance can be obtained by the correlation calculation of at least two images. The calculated subject distance reflects the alignment error of the photographing lens and the image sensor.

  The plurality of images having different parallaxes may be, for example, images captured from other digital cameras and recorded in a recording unit, or may be configured to be captured and captured by the apparatus itself. In this case, a plurality of photographing lenses arranged two-dimensionally and a plurality of image sensors that individually capture subject images formed by the photographing lenses may be provided. It is preferable that the reference image is an image acquired from an image pickup element arranged at the approximate center of the plurality of image pickup elements.

  If camera shake is prevented during shooting, a higher resolution image can be generated. In this case, shake detecting means for detecting shake of the photographic lens or the image sensor, moving means for relatively moving each photographic lens and the corresponding image sensor in a direction perpendicular to the photographic optical axis, What is necessary is just to provide the movement amount calculation means for calculating the movement amount of the movement means for relatively moving the photographic lens and the imaging element based on the detected shake.

  As an imaging device, it is preferable to obtain an image with a high resolution using a low-pixel imaging device. In order to prevent the sharpness of the composite image from being lowered, it is preferable to synthesize using an acquired image because a low-pass filter (low-pass filter) is not used between the photographing lens and the image sensor. The optical resolution of the photographic lens is, for example, 2 / √m times or less of the pixel pitch of the imaging device, where m is the number of imaging units (cameras) that are one imaging system including the photographic lens and the imaging device. (Two times the pixel pitch is the Nyquist wavelength). Further, the number of pixels of the composite image is K to mK pixels, where K is the number of pixels of the image sensor of each imaging unit. The optical resolution is F × λ where the F number of the lens is F and the wavelength is λ. For example, when λ = 550 nm and F = 2.8, the optical resolution is 1.54 μm. The optical resolution of a recent small camera module is about 2.5 μm with a 1/9 inch optical size VGA image pickup device (output pixel 640 × 480). For example, if the number of image pickup units m = 16, 2 × 2. If 5/4 / 0.55 = 2.2, that is, if the F number is 2.2 or less, the combined pixel output can be multiplied by m. 640 × 480 × 16 = 4915200, which is about 5 million pixel output. Usually, since the resolving power cannot be obtained as desired due to the influence of aberration, it is better to allow a margin of about 1/2 to 1 / 2.5 times the F number. In that case, F = 1.1 to 0.88. More strictly, the MTF of the optical system at the Nyquist wavelength of the following synthesized output is designed to be equal to or greater than the target value. A general design goal is a minimum of 10%.

  It is preferable that the image pickup device acquires an image without an optical low-pass filter, and the combining processing unit combines the images using an image without a low-pass filter. The attenuation characteristics when using an optical low-pass filter for each lens are such that the Nyquist wavelength of the composite output = 2 × pixel pitch of the image sensor × √ (number of image sensors / number of output pixels) or more is sufficient. Use it.

  In front of each photographing lens, a photographing aperture is provided. The photographing apertures are two-dimensionally arranged in the same manner as the photographing lens and the image sensor. When the photographing apertures are arranged at equal intervals in the two-dimensional direction (x, y direction), a blurred image (moire or the like) that periodically changes is generated in a part of the generated image. Therefore, it is preferable to set the intervals in the x and y directions of the photographing apertures to a predetermined periodic pattern that is a binary pseudo-random sequence because a natural blurred image can be obtained.

  That is, the array pattern in the x and y directions of each imaging aperture corresponding to the binary sequence of 0 and 1 in which the point sequence for arranging each imaging aperture in the x and y directions is such that the autocorrelation function is a substantially delta function. A periodic pseudo-random sequence is formed with the number of columns as a period.

  The intervals in the x and y directions of the photographing apertures are preferably set to a predetermined periodic pattern according to the M series. In this case, the number of point sequences in the x direction and y direction in which the imaging aperture is arranged is (2n−1), and the intervals in the x direction and y direction of the imaging aperture form an M series.

  As long as it is a binary pseudo-random sequence, in addition to the M sequence, it is also possible to use a Q sequence (square remainder sequence), Gold sequence, Walsh code, or the like.

  Each image sensor simultaneously captures a plurality of images having different parallaxes in response to one shutter. The generated high resolution image is preferably compressed and recorded in the recording unit.

  In this embodiment, the camera arrangement is a random arrangement. However, as another embodiment, the arrangement may be a conventional grid arrangement, and only camera modules that are electrically random arrangement may be selected. In that case, the M sequence may be used for the random array, or the camera module to be activated is switched every time shooting is performed by a random number generating means for generating “0” and “1” using a predetermined probability density function. May be. The predetermined probability density function is, for example, a uniform function.

  When electrical switching is possible, for example, when using M-sequence random numbers, the period to be used and the number of cameras may be switched. By doing so, when the object and the camera are close to each other, such as macro photography, the period length and the number of cameras can be reduced, the blur can be reduced, and the blur can be adjusted to be too large.

  When switching electrically, the number of cameras used for each pixel of the output image may be switched. By doing so, it is possible to create an optimum blur for each subject by combining the use of image processing.

  In the case of electrical switching, a large number of cameras may be used as the cameras in focus to increase the number of pixels, and the number of unfocused cameras in a blurred area may be reduced.

DESCRIPTION OF SYMBOLS 10 Multi-lens camera 11 Image | photographing opening 13 Shutter button 20 Image pick-up element 23 Shooting lens 25 Composition processing part 28 Movement amount calculation part 29 Movement mechanism 40 Relative position detection part 41 Superposition part 42 Pixel arrangement part

Claims (9)

In an image generation apparatus that generates a single high-resolution image using a plurality of images having different parallaxes,
A relative position detecting means for detecting a relative position of the remaining image with respect to a reference image among the plurality of images;
A superimposing means for superimposing the plurality of images on a virtual image area having a plurality of pixels so that the deviation is zero based on the detected relative position;
A pixel arrangement unit that selects and arranges a pixel having the closest center at the center of each pixel constituting the virtual image region from a plurality of superimposed images;
An image generation apparatus comprising: The image generation apparatus according to claim 1.
An image generating apparatus comprising distance measuring means for calculating a subject distance by a correlation calculation of at least two images of the plurality of images. The image generation apparatus according to claim 1 or 2,
A plurality of taking lenses arranged two-dimensionally;
A plurality of image sensors that individually image subject images formed by the photographing lens;
An image generation apparatus comprising: The image generation apparatus according to claim 3.
Moving means for relatively moving the photographing lens and the image sensor corresponding thereto in a direction perpendicular to the photographing optical axis;
Shake detecting means for detecting shake of the device;
The movement amount calculating means for calculating the movement amount of the moving means for relatively moving the photographic lens and the imaging element based on the detected shake;
An image generation apparatus comprising: In the image generation device according to claim 3 or 4,
Each of the image sensors outputs an image without a low-pass filter. The image generation device according to any one of claims 3 to 5,
Having a photographic aperture in front of each photographic lens;
An interval in the x and y directions of each imaging aperture is set to a predetermined periodic pattern that is a binary pseudo-random sequence. The image generation apparatus according to claim 6.
An image generating apparatus characterized in that a predetermined periodic pattern forms an M series with respect to an interval in the x and y directions of the photographing aperture. In an image generation method for generating a single high-resolution image using a plurality of images having different parallaxes,
A relative position detecting step of calculating a relative position of the remaining image with respect to a reference image among the plurality of images;
A superposition step of superimposing the plurality of images on a virtual image area having a plurality of pixels so that the deviation is zero based on the detected relative position;
A pixel arrangement step of selecting and arranging a pixel having the closest center at the center of each pixel constituting the virtual image region from a plurality of superimposed images;
An image generation method comprising: The image generation method according to claim 8.
An image generation method comprising a distance measuring step of calculating a subject distance by a correlation calculation of at least two images of the plurality of images.

Images