JP2013145985A - Imaging device, control method therefor, and imaging system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize an imaging device for generating image data with an appropriate image size and reading speed in accordance with optical characteristics of a used photographing lens.SOLUTION: In an imaging device using an image sensor having a plurality of pieces of photoelectric conversion means sharing a microlens, the number of pieces of photoelectric conversion means to be read is switched in accordance with the F-number of a photographing lens, and when the F-number is high, only pieces of photoelectric conversion means around the center of the microlens are read using a smaller number of pieces of photoelectric conversion means than when the F-number is low.

Description

  The present invention relates to an imaging device that captures, records, and reproduces still images and moving images, a control method thereof, and an imaging system, and in particular, an imaging device having a microlens array on the front surface of an imaging element that is a component of the imaging device, The present invention relates to a control method and an imaging system.

  2. Description of the Related Art Conventionally, there are many imaging devices such as electronic cameras that record and reproduce still images and moving images captured by a solid-state imaging device such as a CCD or CMOS using a memory card having a solid-state memory device as a recording medium.

  As an example of a technology related to these imaging devices, an imaging device having a configuration in which a microlens array arranged at a ratio of one pixel to a plurality of pixels (photoelectric conversion means) is arranged in front of a solid-state imaging device is proposed in Non-Patent Document 1 Has been. With such a configuration, it is possible to obtain information on the incident direction of a light beam incident on the image sensor.

  If such an imaging device is used, in addition to generating a normal captured image based on the output signal from each pixel, the captured image is subjected to predetermined image processing to an arbitrary focal length. It is also possible to reconstruct a focused image.

Ren.Ng and 7 others, “Light Field Photography with a Hand-Held Plenoptic Camera”, Stanford Tech Report CTSR 2005-02

  However, shooting in the imaging apparatus having the microlens array as described above has the following problems.

  In the configuration of the imaging device described above, it is an efficient arrangement to arrange the microlens array so that the F number of the photographing lens and the F number of the microlens coincide. More specifically, when the F number of the photographing lens is smaller than the F number of the microlens, the subject image that has passed through one microlens and the subject image that has passed through another microlens overlap on the imaging element surface. As a result, image reconstruction cannot be realized. On the other hand, when the F number of the photographing lens is larger than the F number of the microlens, useless pixels are generated in which the subject light that has passed through the microlens is not incident. For the above reasons, the arrangement position of the microlens array is determined by the F number of the photographing lens.

  However, the F number of the photographing lens is fixed only when the photographing lens is a single-focus lens. For example, in the case of a zoom lens, the F number is often different depending on the focal length. In addition, when the imaging device is a device capable of exchanging lenses, such as a single-lens reflex camera, the F number is generally different depending on the lens used.

  As a countermeasure against this problem, a method of moving the position of the microlens array in accordance with the F number of the photographing lens can be considered. However, making the microlens array movable is not preferable because it requires very high accuracy for position adjustment and leads to an increase in cost and size.

Further, the following problems are encountered in shooting with an imaging apparatus having a microlens array as described above.
The pixel unit in the image generated by the imaging device is one pixel per microlens. Since a normal image sensor has a configuration of one pixel per microlens, it is only necessary to read out pixel signals (photoelectric conversion signals) of approximately the same number as the number of image data to be generated. However, the imaging apparatus according to the prior art has a configuration having several tens of pixels (photoelectric conversion means) per microlens. For this reason, it is necessary to read out several tens of times more pixels than a normal imaging apparatus. Don't be. This is a big problem in terms of increase in image size and increase in readout time.

  In order to solve the above-described problems, according to the present invention, an imaging apparatus using an imaging device having a plurality of photoelectric conversion means sharing a microlens can read the number of photoelectric conversion means that is read in accordance with the F number of the imaging lens. When the F number is large, only photoelectric conversion means with a smaller number than when the F number is small are read.

  According to the present invention, in an imaging apparatus having a microlens array, it is possible to realize an imaging apparatus that generates image data with an appropriate image size and readout speed in accordance with the optical characteristics of a photographing lens to be used.

1 is a block diagram of an imaging apparatus according to a first embodiment of the present invention. It is a block diagram which shows the structure of the optical system periphery in the imaging device concerning 1st Example of this invention. It is a figure which shows pixel arrangement | positioning of the image pick-up element in the Example concerning the 1st Example of this invention. FIG. 4 is an enlarged view of recording pixels in the pixel array shown in FIG. 3. It is a figure which shows notionally the locus | trajectory of the light ray from the to-be-photographed object in the imaging system of FIG. It is a figure which shows notionally the locus | trajectory of incident light in case the F number of the imaging lens in the imaging device concerning 1st Example of this invention is small. It is a figure which shows arrangement | positioning of the pixel used for reading of an imaging signal when the F number of the imaging lens in the imaging device concerning 1st Example of this invention is small. It is a figure which shows notionally the locus | trajectory of incident light when the F number of the imaging lens in the imaging device concerning 1st Example of this invention is medium. It is a figure which shows arrangement | positioning of the pixel used for read-out of an image pick-up signal, when the F number of an imaging lens in the imaging device concerning 1st Example of this invention is medium. It is a figure which shows notionally the locus | trajectory of an incident light in case the F number of the imaging lens in the imaging device concerning 1st Example of this invention is large. It is a figure which shows arrangement | positioning of the pixel used for reading of an imaging signal when the F number of the imaging lens in the imaging device concerning 1st Example of this invention is large. It is a figure which shows the flowchart of the imaging operation of the imaging device concerning 1st Example of this invention. 1 is a block diagram illustrating a configuration around an optical system in an imaging apparatus according to a first embodiment of the present invention. It is a figure which shows notionally the locus | trajectory of an incident light in case the open F number of the interchangeable lens in the imaging device concerning 2nd Example of this invention is small. It is a figure which shows arrangement | positioning of the pixel used for read-out of the image pick-up signal in case the open F number of the interchangeable lens is the minimum in the image pick-up device concerning the 2nd Example of this invention. It is a figure which shows notionally the locus | trajectory of an incident light in case the open F number of the interchangeable lens in the imaging device concerning 2nd Example of this invention is larger than minimum value. It is a figure which shows arrangement | positioning of the pixel used for reading of an imaging signal when the open F number of the interchangeable lens in the imaging device concerning 2nd Example of this invention is a magnitude | size of FIG. It is a figure which shows the flowchart of the imaging operation of the imaging device concerning 2nd Example of this invention.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of an image pickup apparatus according to the first embodiment of the present invention.
In FIG. 1, 101 is an optical system such as an interchangeable photographic lens and diaphragm, 102 is a mechanical shutter (shown as a mechanical shutter), and 103 is an image sensor that outputs an electrical signal obtained by photoelectrically converting an optical image of a subject. is there. Here, reference numeral 104 denotes a photoelectric conversion unit that actually converts incident light into an electric signal in the image sensor 103. The photoelectric conversion unit 104 has a sensor array in which a plurality of pixels as photoelectric conversion means are arranged two-dimensionally.

Reference numeral 105 denotes a signal amplification circuit that amplifies an electric signal in the image sensor 103.
Reference numeral 106 denotes an analog signal processing circuit that performs analog signal processing on the image signal output from the image sensor 103, and 107 denotes a CDS circuit that performs correlated double sampling in the analog signal processing circuit 106.
In the analog signal processing circuit 106, 108 is a signal amplifier that amplifies the analog signal, 109 is a clamp circuit that performs horizontal OB clamping, and 110 is an A / D converter that converts the analog signal into a digital signal.

  Reference numeral 111 denotes a timing signal generation circuit that generates signals for operating the image sensor 103 and the analog signal processing circuit 106, and 112 denotes a drive circuit for the optical system 101 and the mechanical shutter 102. Reference numeral 113 denotes a digital signal processing circuit that performs digital signal processing necessary for captured image data. In the digital signal processing circuit 113, 114 is an image correction circuit that performs necessary correction processing on image data, 115 is a signal amplification circuit that amplifies the digital signal, and 116 is an image processing circuit that performs necessary image processing on the image data. is there.

  Reference numeral 117 denotes an image memory for storing signal-processed image data, 118 denotes an image recording medium (shown as a recording medium) that is removable from the imaging device, and 119 records signal-processed image data to the image recording medium 118. Recording circuit. Reference numeral 120 denotes an image display device that displays signal-processed image data, and 121 denotes a display circuit that displays an image on the image display device 120.

  Reference numeral 122 denotes a system control unit that controls the entire imaging apparatus. 123 is a nonvolatile memory that stores a program describing a control method executed by the system control unit 122, control data such as parameters and tables used when executing the program, and correction data such as a scratch address (ROM). Reference numeral 124 denotes a volatile memory (RAM) that is used to transfer and store the program, control data, and correction data stored in the nonvolatile memory 123 and that is used when the system control unit 122 controls the imaging apparatus.

  Reference numeral 125 denotes shooting mode setting means for setting shooting conditions such as ISO sensitivity setting and switching between still image shooting (first shooting mode) and live view driving (second shooting mode).

  Hereinafter, the shooting operation of the imaging apparatus configured as described above will be described. Prior to the photographing operation, necessary programs, control data, and correction data are transferred from the nonvolatile memory 123 to the volatile memory 124 and stored at the start of the operation of the system control unit 122 such as when the imaging apparatus is turned on. Shall. These programs and data are used when the system control unit 122 controls the imaging apparatus. Further, as necessary, additional programs and data are transferred from the nonvolatile memory 123 to the volatile memory 124, or the system control unit 122 directly reads and uses the data in the nonvolatile memory 123. .

  First, the optical system 101 drives the optical system 101 such as a lens in accordance with a control signal from the system control unit 122 to form a subject image set to an appropriate brightness on the image sensor 103. Next, during still image shooting, the mechanical shutter 102 is driven by the control signal from the system control unit 122 so as to shield the image sensor 103 in accordance with the operation of the image sensor 103 so that a necessary exposure time is obtained. Is done. At this time, when the image sensor 103 has an electronic shutter function, it may be used together with the mechanical shutter 102 to secure a necessary exposure time. The mechanical shutter 102 is maintained in the open state so that the image pickup device 103 is always exposed during shooting by a control signal from the system control unit 122 during moving image shooting and live view driving.

  The image sensor 103 is driven with a drive pulse based on the operation pulse generated by the timing signal generation circuit 111 controlled by the system control unit 122. With this drive control, a photoelectric conversion signal is read from the photoelectric conversion unit 104. The photoelectric conversion unit 104 converts the subject image into an electrical signal by photoelectric conversion, and the signal amplification circuit 10 multiplies the electrical gain from the photoelectric conversion unit 104 by the gain of the amplification factor set according to the amount of incident light as an analog image signal. Output. Note that a specific configuration of reading out photoelectric conversion signals from each pixel of the sensor array included in the photoelectric conversion unit 104 is a well-known technique in a CMOS sensor or the like, and thus description thereof is omitted here.

  The analog image signal output from the image sensor 103 is subjected to signal processing by the analog signal processing circuit 106 by an operation pulse generated by the timing signal generation circuit 111 controlled by the system control unit 122. First, the clock synchronization noise is removed by the CDS circuit 107, and the gain of the amplification factor set according to the amount of incident light is applied by the PGA circuit. Next, the clamp circuit 109 clamps the signal output in the horizontal OB region as a reference voltage, and the A / D converter 110 converts it into a digital image signal.

  Next, with respect to the digital image signal output from the analog signal processing circuit 106, the digital signal processing circuit 113 controlled by the system control unit 122 performs image processing such as color conversion, white balance, and gamma correction, resolution conversion processing, Perform image compression processing. First, the image correction circuit 114 performs various image correction processes such as scratch correction and dark shading correction. Subsequently, the signal amplification circuit 115 multiplies the gain set according to the amount of incident light, and the image processing circuit 116 performs various image processing such as color conversion, white balance, and gamma correction, resolution conversion processing, and image compression processing. Perform image processing. The image processing circuit 116 also performs the refocus processing described above.

  The image memory 117 is used for temporarily storing a digital image signal during signal processing or for storing image data which is a digital image signal subjected to signal processing. The image data processed by the digital signal processing circuit 113 and the image data stored in the image memory 117 are converted into data suitable for the image recording medium 118 (for example, file system data having a hierarchical structure) by the recording circuit 119. Are recorded on the image recording medium 118. Alternatively, after the resolution conversion processing is performed by the digital signal processing circuit 113, the display circuit 121 converts the signal to a signal suitable for the image display device 120 (for example, an NTSC analog signal) and displays the image display device 120. .

  Here, the digital signal processing circuit 113 may output the digital image signal as it is as image data to the image memory 117 or the recording circuit 119 without performing signal processing by the control signal from the system control unit 122. Further, the digital signal processing circuit 113 outputs information on digital image signals and image data generated in the signal processing process to the system control unit 122 when requested by the system control unit 122. For example, information such as the spatial frequency of the image, the average value of the designated area, the data amount of the compressed image, or information extracted from the information is output to the system control unit 122. Further, the recording circuit 119 outputs information such as the type and free capacity of the image recording medium 118 to the system control unit 122 when requested by the system control unit 122.

  Further, a reproduction operation when image data is recorded on the image recording medium 118 will be described.

  The recording circuit 119 reads image data from the image recording medium 118 in accordance with a control signal from the system control unit 122. Similarly, when the image data is a compressed image, the digital signal processing circuit 113 performs an image expansion process according to a control signal from the system control unit 122 and stores it in the image memory 117. The image data stored in the image memory 117 is subjected to resolution conversion processing by the digital signal processing circuit 113, converted to a signal suitable for the image display device 120 by the display circuit 121, and displayed on the image display device 120. The

FIG. 2 is a block diagram for explaining the periphery of the optical system in the embodiment of the imaging apparatus according to the present invention.
In FIG. 2, 201 is a photographic lens, 202 is a microlens array disposed between the image sensor 103 and the photographic lens 201, and 203 is a sensor array that is also a component of the image sensor 103, as described above. It is a two-dimensional array of pixels (photoelectric conversion means). Reference numeral 204 denotes an aperture, and reference numeral 205 denotes a subject. The other components described in FIG. 2 are the same as those described with reference to FIG.
Hereinafter, the imaging operation in the embodiment of the imaging apparatus according to the present invention will be described with reference to FIG.

  In a state where the mechanical shutter 102 and the diaphragm 204 are opened by the drive circuit 112, an image of the subject 204 is formed on the image sensor 103 by the photographing lens 201. The optical signal incident on the image sensor 103 is further condensed by each microlens of the microlens array 202 and incident on each pixel of the sensor array 203. The configurations of the microlens array 202 and the sensor array 203 will be described later with reference to FIG. The optical signal incident on the sensor array 203 is photoelectrically converted in each pixel and output as an electrical signal. Subsequent processing is as described with reference to FIG.

FIG. 3 is a layout diagram illustrating a pixel array of the image sensor according to the embodiment of the present invention.
FIG. 3 is a diagram of the image sensor 103 as viewed from the subject side. Reference numeral 301 denotes a recording pixel which is a unit pixel corresponding to one pixel of the reconstructed image. In this embodiment, the recording pixel 301 includes 6 rows and 6 columns of photoelectric conversion means. In this case, each photoelectric conversion unit is referred to as a divided pixel 302. Reference numeral 303 denotes a microlens arranged for each recording pixel 301. That is, when the recording pixel is a pixel block that is a two-dimensional array of photoelectric conversion means, each microlens of the microlens array is associated with a plurality of photoelectric conversion means of one pixel block.

  In the description of this embodiment, for the sake of convenience, the operation of the image pickup apparatus is performed using a sensor array 203 in which recording pixels (pixel blocks) made up of 6 × 6 (predetermined number) divided pixels shown in FIG. explain.

FIG. 4 is an enlarged view of the recording pixel 301.
As described with reference to FIG. 3, the recording pixel 301 includes a 6 × 6 two-dimensional array of the divided pixels 302. For later explanation, in this embodiment, 36 divided pixels are named a11 to a66 as shown in FIG.

FIG. 5 is a diagram conceptually showing the trajectory of light rays that are incident on the imaging system of the imaging apparatus according to the present embodiment from subjects at various distances.
The subject 501 a is a subject placed at a position where the image is formed on the surface A including the microlens array 202 by the photographing lens 101. Of the light rays from the subject 501a, the light rays that pass through the outermost periphery of the photographing lens and enter the sensor array 203 via the microlenses on the optical axis are indicated by solid lines.

  The subject 501b is a subject located farther from the subject 501a when viewed from the photographing lens 101. The image formed by the photographing lens 101 of the subject 501b is formed on the surface B closer to the photographing lens than the surface A including the microlens array 202. Of the light rays from the subject 501b, the light rays that pass through the outermost periphery of the photographing lens and enter the sensor array 203 via the microlenses on the optical axis are indicated by broken lines.

  The subject 501c is a subject closer to the subject 501a when viewed from the photographing lens 101. An image formed by the photographing lens 101 of the subject 501c is formed on a surface C farther from the photographing lens than the surface A including the microlens array 202. Of the light rays from the subject 501c, the light rays that pass through the outermost periphery of the photographing lens and enter the sensor array 203 via the microlenses on the optical axis are indicated by alternate long and short dash lines.

  As shown by the locus of each light ray shown in FIG. 5, the divided pixels incident in the sensor array 203 differ depending on the distance from the photographing lens 101 to the subject 501. By using this, the image pickup apparatus of the present embodiment can generate images focused on subjects at various distances by reconstructing an image from the image signal after shooting.

  Next, the operation corresponding to the F number of the photographing lens in the image pickup apparatus according to the present embodiment will be described with reference to FIGS. In the following drawings, the same components as those shown in FIGS. 2 to 4 are denoted by the same reference numerals, and description thereof will be omitted unless particularly required.

FIG. 6 is a diagram conceptually showing the locus of incident light when the F number of the photographic lens in the image pickup apparatus according to the present embodiment is minimum.
In FIG. 6, reference numeral 601 denotes a subject. In the imaging apparatus according to the present embodiment, the microlens array 202 is arranged as follows. In other words, the microlens array 202 is arranged so that the F-number of the microlens is the same as the minimum F-number in various optical systems that can be used including a photographing lens and a diaphragm.
The state of the optical system in FIG. 6 is a state in which the F number is minimized. In this case, light rays that enter the microlens through the photographing lens 201 and the diaphragm 204 are incident on all 6 × 6 divided pixels.

FIG. 7 is a diagram illustrating an arrangement of pixels used for reading pixel signals when the F number of the photographing lens is minimum in the imaging apparatus according to the present embodiment. FIG. 7A is a layout diagram of the sensor array 203 described with reference to FIG. 3, and FIG. 7B shows an arrangement of the divided pixels 302 included in the recording pixel 301 that is a component of the sensor array 203. FIG.
Here, in FIG. 7 and the subsequent drawings, the divided pixels used for reading out the pixel signals are kept white, and the divided pixels not used are shown in gray.

  When the F number of the optical system described with reference to FIG. 6 is the minimum, the signal light is incident on all the 6 × 6 divided pixels. Therefore, the signals of all the divided pixels must be read and recorded independently. I must. Therefore, in the pixel arrangement diagram showing the pixels used for reading out the pixel signal when the F number of the photographing lens shown in FIG. 7 is the minimum, all the divided pixels are shown as white.

FIG. 8 is a diagram conceptually illustrating the locus of incident light when the F number of the photographing lens is medium in the imaging apparatus according to the present embodiment.
In the state of FIG. 8, light rays from the subject that enter the microlens through the photographing lens 201 and the diaphragm 204 are incident on a part of the 6 × 6 divided pixels, and the outermost divided pixels and the like Not incident. That is, only the output signal of the 4 × 4 divided pixels at the positions a2 to a5 of the recording pixel block 301 shown in FIG.

  FIG. 9 is a diagram illustrating an arrangement of pixels used for reading pixel signals when the F number of the photographing lens is medium in the imaging apparatus according to the present embodiment. FIG. 9A is a layout diagram of the sensor array 203 described with reference to FIG. 3, and FIG. 9B is a diagram illustrating an arrangement of the divided pixels 302 in the recording pixel 301. As described above, the divided pixels used for pixel readout remain white, and the unused divided pixels are shown in gray.

  In the case where the F number of the optical system is medium as described with reference to FIG. 8, only some of the divided pixels 302 into which the signal light from the subject is incident out of the plurality of divided pixels 302 constituting each recording pixel 301. The drive to read out is appropriate. Therefore, in FIG. 9 showing the arrangement of pixels to be read when the F number of the optical system is medium, only the 4 × 4 divided pixels a22 to a55 shown in FIG. 9B are shown in white, and the other divided pixels are Shown in gray.

FIG. 10 is a diagram conceptually showing the locus of incident light when the F number of the photographing lens in the image pickup apparatus according to the present embodiment is maximum.
In the state of FIG. 10, the light rays from the subject that enter the microlens through the photographing lens 201 and the diaphragm 204 are incident on a part of the 6 × 6 divided pixels of each recording pixel, and the divided pixels near the outer periphery. It does not enter. In other words, only the output signals from the 2 × 2 divided pixels located at a3 to a4 shown in FIG.

  FIG. 11 is a diagram illustrating an arrangement of pixels used for reading out pixel signals when the F number of the photographing lens in the imaging apparatus according to the present embodiment is maximum. FIG. 11A is a layout diagram of the sensor array 203 described with reference to FIG. 3, and FIG. 11B shows an arrangement of the divided pixels 302 in the recording pixel 301 that is a component of the sensor array 203. FIG. As described above, also in FIG. 11, the used divided pixels remain white, and the unused divided pixels are shown in gray.

  In the case where the F number of the optical system is minimum as described with reference to FIG. 10, only some of the divided pixels on which the signal light is incident out of the plurality of divided pixels 302 constituting each recording pixel (pixel block) 301. The drive to read out is appropriate. Therefore, in the figure showing the arrangement of the pixels to be read when the F number of the optical system shown in FIG. 11 is the minimum, only the 2 × 2 divided pixels a33 to a44 shown in FIG. Pixels are shown in gray.

  In the description using FIGS. 8 to 11 above, the relationship between the F number value and the readout pixel is shown for the three F number values. However, the present invention is not limited to this, and the F number value is appropriately set. The readout pixel can be changed accordingly. For example, a plurality of threshold values are set, and the change of the readout pixel may be controlled based on the magnitude relationship between them and the F number. Further, although the vicinity of the center of the recording pixel 301 is shown as the range of divided pixels for reading out pixel signals corresponding to different F numbers of the optical system, the present invention is not limited to this. When the microlens 303 is shifted in the direction perpendicular to the optical axis of the photographing lens due to light shading countermeasures, the vicinity of the central position of the recording pixel may not necessarily be an appropriate selection range. In this case, for example, a configuration in which a plurality of divided pixels are selected centering on the position on the sensor array where the light beam passing through the center of the photographing lens 201 and the center of the microlens 303 arrives may be employed.

  FIG. 12 is a flowchart illustrating the imaging operation of the imaging apparatus according to the present embodiment. The control of this operation is achieved by the system control unit 122 controlling the timing signal generation circuit 111 and driving the image sensor 103 in accordance with a program, control data, etc. read from the nonvolatile memory 123.

  In this imaging apparatus, first, the setting value set by the system control unit 122 is acquired via the drive circuit 112 for the photographing lens 201 and the diaphragm 204 that are used, and the current diaphragm value (F) of the optical system 101 is acquired. Number) is acquired (S1201). Subsequently, according to the acquired F number, the area of the divided pixel 302 used for reading is determined and set as described above (S1202). Thereafter, photographing is executed (S1203). When shooting is completed, the driving of the image sensor is controlled to read out signals from the divided pixels 302 in the previously set area, and the series of operations ends. Note that the size of the acquired F number may be determined by setting a plurality of threshold values in advance and comparing the acquired F number with the acquired F number as described above.

  As described above, according to the imaging apparatus according to the present embodiment, the pixel used for signal readout is changed according to the F number of the photographing optical system, and only the divided pixels on which the light rays from the subject are incident are read. It becomes possible. Therefore, it is possible to shorten the readout time and reduce the size of the generated image data.

  Next, a second embodiment of the present invention will be described. The imaging apparatus according to the present embodiment is an imaging system in which a photographic lens can be separated and replaced from the imaging apparatus.

FIG. 13 is a block diagram showing the periphery of the optical system in the image pickup apparatus according to the present embodiment.
In FIG. 13, reference numeral 1301 denotes a removable interchangeable lens, and 1302 denotes an image pickup apparatus main body excluding a photographing lens. In the interchangeable lens 1301, a lens control circuit 1303 receives a control signal from the imaging apparatus body and controls the photographing lens, and 1304 is a connector that is electrically connected to the imaging apparatus body. Similarly, reference numeral 1305 denotes a connector that is electrically connected to the interchangeable lens 1301 in the imaging apparatus main body 1302. The other components described in FIG. 13 are the same as those described in FIG. 1 and FIG. In this embodiment, since the diaphragm 204 is not used, it is not shown in FIG.
Hereinafter, the imaging operation in the imaging apparatus according to the present embodiment will be described with reference to FIG.

  The system control unit 122 in the imaging apparatus main body 1302 detects the state of the connector 1305 and determines whether the interchangeable lens 1301 is attached. When it is determined that the interchangeable lens 1301 is attached, information on the attached interchangeable lens 1301 is acquired through communication with the lens control circuit 1303 in the interchangeable lens 1301. Information to be acquired includes the type of the interchangeable lens 1301, the focal length, the aperture value, the open F number, and the like.

  In a state where the mechanical shutter 102 is opened by the drive circuit 112, an image of the subject 204 is formed on the image sensor 103 by the photographing lens 201. Light rays from the subject incident on the image sensor 103 are further condensed by each microlens of the microlens array 202 and are incident on each pixel (divided pixel) of the sensor array 203. The light beam incident on the sensor array 203 is photoelectrically converted in each pixel and output as an electrical signal. Subsequent processing is as described with reference to FIG.

  Next, an operation according to the open F number of the interchangeable lens in the imaging apparatus according to the present embodiment will be described with reference to FIGS. The open F number represents the F number when the diaphragm is open in each interchangeable lens. In the imaging apparatus according to the present embodiment, since there is no diaphragm, the open F number is an F number determined by the photographing lens 201 and its peripheral members.

  FIG. 14 is a diagram conceptually illustrating the locus of incident light when the open F number of the interchangeable lens in the imaging apparatus according to the present embodiment is small.

  Also in the imaging apparatus according to the present embodiment, the microlens array 202 is arranged so that the minimum value of the open F-number of various interchangeable lenses that can be used is the same as the F-number of the microlens.

  The state of the optical system shown in FIG. 14 is a state in which an interchangeable lens having the smallest open F number is attached. In this case, light rays that enter the microlens through the photographing lens 201 are incident on all 6 × 6 divided pixels.

  FIG. 15 is a diagram illustrating an arrangement of pixels used for pixel signal readout when the open F number of the interchangeable lens in the imaging apparatus according to the example is the minimum. FIG. 15A is a layout diagram of the sensor array 203 described with reference to FIG. 3, and FIG. 15B is a divided pixel 302 in a recording pixel (pixel block) 301 that is a component of the sensor array 203. It is a figure which shows arrangement | positioning. In FIG. 15, as in the first embodiment, the divided pixels used for pixel readout remain white and the unused divided pixels are shown in gray.

  In the case where the open F-number of the interchangeable lens is the minimum described with reference to FIG. 14, the signal light is incident on all the divided pixels of the 6 × 6 recording pixel, so that the signals of all the divided pixels are read out independently. Recorded. Therefore, in the pixel arrangement diagram showing the pixels used when the open F number of the interchangeable lens shown in FIG. 15 is minimum, all the divided pixels are shown in white.

  FIG. 16 is a diagram conceptually illustrating the locus of incident light when the open F number of the interchangeable lens is greater than the minimum value in the imaging apparatus according to the present embodiment.

  In the state of FIG. 16, the light beam from the subject that enters the microlens through the photographing lens 201 enters a part of the 6 × 6 divided pixels in each recording pixel, and the outermost divided pixel or the like. It does not enter. That is, only the output signals from the 4 × 4 divided pixels located at a2 to a5 in the recording pixel 301 (pixel block) may be read from the image sensor 103 and recorded.

  FIG. 17 is a diagram illustrating an arrangement of pixels used for reading pixel signals when the open F number of the interchangeable lens in the imaging apparatus according to the present embodiment is FIG. FIG. 17A is a layout diagram of the sensor array 203 described with reference to FIG. 3, and FIG. 17B is a diagram illustrating an arrangement of the divided pixels 302 in the recording pixel 301 that is a component of the sensor array 203. It is. Also in FIG. 17, as in FIG. 15, the used divided pixels remain white and the unused divided pixels are shown in gray.

  When the open F number of the interchangeable lens shown in FIG. 16 is large, it is appropriate to read out only some of the divided pixels on which the signal light is incident from among the plurality of divided pixels 302 constituting each recording pixel 301. Therefore, in FIG. 17 showing the arrangement of pixels to be read when the F number of the optical system is medium, only the 4 × 4 divided pixels a22 to a55 described in FIG. 17B are shown in white, and the other divided pixels Is shown in gray.

  In the description using FIG. 16 and FIG. 17, the relationship between the F number value and the readout pixel is shown for the two F number values. However, the present invention is not limited to this, and depending on the F number value as appropriate. The readout pixel can be changed. For example, a plurality of threshold values are set, and the change of the readout pixel may be controlled based on the magnitude relationship between them and the F number. Further, although the vicinity of the center of the recording pixel 301 is selected as the divided pixel range for reading out the pixel signal in accordance with the F number of the attached interchangeable lens, the present invention is not limited to this. When the microlens 303 is shifted in the direction perpendicular to the optical axis of the photographing lens due to light shading countermeasures, the vicinity of the center of the recording pixel may not necessarily be an appropriate selection range. In this case, for example, a configuration in which a plurality of divided pixels are selected centering on the position on the sensor array where the light beam passing through the center of the photographing lens 201 and the center of the microlens 303 arrives may be employed.

  FIG. 18 is a diagram illustrating a flowchart of the imaging operation of the imaging apparatus according to the present embodiment. As in the first embodiment, the control of this operation is performed by the system control unit 122 controlling the timing signal generation circuit 111 and driving the image sensor 103 in accordance with a program, control data, and the like read from the nonvolatile memory 123. Is achieved.

  In this imaging apparatus, first, the system control unit 122 detects the type of the interchangeable lens 1301 being used via the drive circuit 112, and acquires the open F number of the photographing lens 201 (S1201). Subsequently, the area of the divided pixel 302 to be read is determined according to the acquired F number (S1202). Thereafter, photographing is executed (S1203). When the photographing is completed, a signal is read from the divided pixels 302 in the previously determined region, and a series of operations is finished.

  As described above, according to the imaging apparatus according to the present embodiment, the divided pixels for reading the imaging signal of the subject are selected according to the interchangeable lens to be used. As a result, the pixel signal readout time can be reduced and the size of the generated image data can be reduced.

  As mentioned above, although embodiment of the imaging device concerning embodiment of this invention was described using FIGS. 1-18, this invention is not limited to this and can take various forms. .

  For example, in the pixel configuration of the embodiment of the imaging device according to the present invention, the divided pixels under the same microlens are configured in 6 × 6 divisions for easy understanding of the pixel structure, but the present invention is not limited to this. Instead, it may have a configuration having divided pixels of various numbers and shapes.

  Moreover, although the image pickup apparatus according to the second embodiment of the present invention described with reference to FIGS. 13 to 18 has been described as a configuration without an aperture in the optical system, the present invention is not limited to this. That is, this premise is that the image pickup apparatus which is the background art of the present invention is an image pickup apparatus capable of reconstructing images with various focal lengths, and will be photographed under an optical condition with a small depth of field. However, it is not necessary to be limited to this. In the embodiment of the present invention described with reference to FIG. 1, the image processing such as image reconstruction has been described as being performed by the digital signal processing circuit 113 which is one of the components of the imaging apparatus. The image processing is not necessarily performed inside the imaging apparatus. Specifically, the image processing means is provided in a device different from the imaging device, for example, a PC (Personal Computer), and the imaging data obtained by the imaging device is transferred to the PC. It is also possible to perform processing.

  Each means constituting the imaging apparatus according to the embodiment of the present invention and each step of the arrangement control method can be realized by operating a program stored in a RAM or a ROM of a computer. This program and a computer-readable storage medium storing the program are included in the present invention.

  In addition, the present invention can be implemented as, for example, a system, apparatus, method, program, storage medium, or the like. Specifically, the present invention may be applied to a system including a plurality of devices. The present invention may be applied to an apparatus composed of a single device.

  Note that the present invention includes a case where a software program that realizes the functions of the above-described embodiments is supplied directly or remotely to a system or apparatus. This includes the case where the system or the computer of the apparatus is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention. In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, and the like.

  Examples of the storage medium for supplying the program include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk. Further, there are MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, there is a method of connecting to a homepage on the Internet using a browser of a client computer. It can also be supplied by downloading the computer program itself of the present invention or a compressed file including an automatic installation function from a homepage to a storage medium such as a hard disk.

  It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  As another method, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and encrypted from a homepage via the Internet to users who have cleared predetermined conditions. Download the key information to be solved. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. Furthermore, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can be realized by the processing.

  As another method, a program read from a storage medium is first written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Then, based on the instructions of the program, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are also realized by the processing.

Claims (12)

An imaging element having a two-dimensional array of a photographing lens, a plurality of photoelectric conversion means for converting an optical image formed by the photographing lens into an electric signal, and disposed between the photographing lens and the imaging element A microlens array, and each microlens of the microlens array is associated with one pixel block when the two-dimensional array of the plurality of photoelectric conversion means is divided by a predetermined number of pixel blocks of the photoelectric conversion means. In the imaging device
Obtaining means for obtaining an F number of the photographing lens;
Driving means for driving the image sensor to read out an electrical signal from the image sensor;
Control means for controlling the driving means,
The control means changes and sets the photoelectric conversion means used for reading out an electrical signal in each pixel block according to the acquired F number of the photographing lens, and the driving means according to the setting of the photoelectric conversion means An imaging device characterized by controlling the above.   2. The imaging apparatus according to claim 1, wherein the control unit sets a smaller number of photoelectric conversion units used for reading out an electric signal in each pixel block as the F number of the acquired photographing lens is larger. .   When the control means sets a small number of photoelectric conversion means used for reading out electrical signals in each pixel block, it is close to the position where the light beam passing through the center of the photographing lens and the center of the microlens reaches. The imaging apparatus according to claim 2, wherein the photoelectric conversion unit is set as a photoelectric conversion unit used for reading.   The control means sets, in each pixel block, a photoelectric conversion means in a range in which a light beam passing through the photographing lens is incident as a photoelectric conversion means used for reading an electric signal. The imaging device according to any one of the above.   5. The imaging apparatus according to claim 1, wherein the control unit controls the driving unit to read out an electric signal from only the set photoelectric conversion unit. 6.   The imaging apparatus according to claim 1, wherein the photographing lens has an aperture, and the acquisition unit acquires an F number based on information on the imaging lens and the aperture.   The photographing lens is a replaceable lens, and the imaging device includes communication means for communicating with the replaceable lens, and the acquisition means acquires an F number from the replaceable lens via the communication means. The imaging apparatus according to claim 1, wherein: An imaging element having a two-dimensional array of a photographing lens, a plurality of photoelectric conversion means for converting an optical image formed by the photographing lens into an electric signal, and disposed between the photographing lens and the imaging element A microlens array, and each microlens of the microlens array is associated with one pixel block when the two-dimensional array of the plurality of photoelectric conversion means is divided by a predetermined number of pixel blocks of the photoelectric conversion means. In the control method of the imaging device
An acquisition step of acquiring an F number of the photographing lens;
A driving step of driving the image sensor to read out an electrical signal from the image sensor;
A control step for controlling the driving means,
In the control step, the photoelectric conversion means used for reading out an electrical signal in each pixel block is changed and set according to the acquired F number of the photographing lens, and the driving step is performed according to the setting of the photoelectric conversion means. The control method characterized by controlling. Computer
An imaging element having a two-dimensional array of a photographing lens, a plurality of photoelectric conversion means for converting an optical image formed by the photographing lens into an electric signal, and disposed between the photographing lens and the imaging element A microlens array, and each microlens of the microlens array is associated with one pixel block when the two-dimensional array of the plurality of photoelectric conversion means is divided by a predetermined number of pixel blocks of the photoelectric conversion means. In the control method of the imaging device
Obtaining means for obtaining an F number of the photographing lens;
Driving means for driving the image sensor to read out an electrical signal from the image sensor;
A control means for controlling the driving means, wherein the photoelectric conversion means used for reading out an electric signal in each pixel block is set by changing according to the F number of the acquired photographing lens, and the photoelectric conversion means A program that functions as control means for controlling the drive means in accordance with the setting.   A computer-readable storage medium storing the program according to claim 9.   A program that causes a computer to function as each unit of the imaging device according to any one of claims 1 to 7.   A storage medium storing a program that causes a computer to function as each unit of the imaging apparatus according to claim 1.

Images