JP2016046773A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a small-sized inexpensive imaging device capable of simultaneously acquiring, by using a single imaging unit, a wide-angle image and a telescopic image whose optical axes agree with each other, and capable of securely capturing an object of a tracking target by using the telescopic image.SOLUTION: An imaging device comprises irises individually arranged to a first light path of a light flux incident on a directional sensor from a central optical system and a second light path of a light flux incident on the directional sensor from an annular optical system, where the first light path and the second light path do not overlap with each other, making it possible to simultaneously acquire, by using a single imaging unit, a wide-angle image and a telescopic image whose optical axes agree with each other and securely capture an imaging target.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an imaging apparatus, and more particularly to an imaging apparatus that can simultaneously acquire a wide-angle image and a telephoto image.

  Conventionally, a double focal length optical system has been proposed in which a wide-angle lens is disposed at the center on the same optical axis and an annular telephoto lens is disposed at the periphery thereof (FIG. 1 of Patent Document 1). ). The annular telephoto lens of this optical system has a reflecting mirror type lens configuration including two reflecting mirrors, and the telephoto lens having a long focal length has a compact configuration. In addition, the imaging positions of the wide-angle lens and the telephoto lens of this optical system are designed to be different positions in the optical axis direction, and separate imaging elements are arranged at the respective imaging positions.

  In addition, any one of the subject light captured by the plurality of objective optical systems has a wide-angle objective optical system, a telephoto objective optical system, and a common optical system through which the light beams that have passed through each objective optical system pass in common An optical device including a reflecting member that selectively guides the light to a common optical system has been proposed (Patent Document 2).

  Then, by moving the reflecting member, the subject light captured by either the wide-angle objective optical system or the telephoto objective optical system is guided to the common image sensor via the common optical system. ing.

  In addition, subject light passing through different areas of the imaging lens is pupil-separated and incident on each pixel of the image sensor corresponding to the different areas of the imaging lens, and multiple images corresponding to the subject light separated from the pupil are simultaneously captured. An imaging apparatus has been proposed (Patent Document 3).

Special table 2011-5005022 gazette JP 2009-122379 A JP 2012-88696 A

  The optical system described in Patent Document 1 acquires a wide-angle image and a telephoto image captured by a wide-angle lens and a telephoto lens, respectively, from separate image sensors arranged at different positions in the optical axis direction. Therefore, there is a problem that the apparatus cannot be reduced in size and cost.

  The optical device described in Patent Document 2 selects either one of a wide-angle objective optical system and a telephoto objective optical system by moving a reflecting member (reflecting mirror). There is a problem that a telephoto image cannot be taken and a mechanism for moving the reflecting member is required, resulting in an increase in the size of the apparatus.

  The imaging device described in Patent Literature 3 pupil-separates subject light that passes through different regions of the imaging lens, and simultaneously captures a plurality of images corresponding to the subject light that has undergone pupil separation using a single imaging device. However, Patent Document 3 does not describe a specific configuration for favorably capturing a wide-angle image and a telephoto image.

  The present invention has been made in view of such circumstances, and can simultaneously acquire a first image and a second image respectively formed by a central optical system and an annular optical system having a common optical axis, An object of the present invention is to provide a compact and inexpensive imaging apparatus.

  According to a first aspect of the present invention, there is provided a photographing optical system comprising a central optical system at the center and an annular optical system at the periphery thereof arranged on the same optical axis, and photoelectric conversion arranged in a two-dimensional manner. A directional sensor having a plurality of pixels each composed of an element, the directional sensor including a plurality of pixels that selectively receive light by dividing a light beam incident through a central optical system and an annular optical system, respectively. An image reading device for acquiring an image signal indicating the first image received through the central optical system and an image signal indicating the second image received through the annular optical system from the directivity sensor, and from the central optical system. The first optical path of the light beam incident on the directional sensor and the second optical path of the light beam incident on the directional sensor from the annular optical system, where the first optical path and the second optical path do not overlap each other. Individually arranged Litho, provides an imaging device comprising a.

  The second aspect of the present invention is an optical path of a light beam incident on the directional sensor from the central optical system and a third optical path of a light beam incident on the directional sensor from the annular optical system, the first optical path and the second optical path. The imaging apparatus according to Item 1, further comprising a light amount adjusting unit arranged in a third optical path that overlaps the optical path.

  In the third aspect of the present invention, the annular optical system has a reflective optical system that reflects the light beam twice or more, and the light amount adjusting means is reflected twice or more by the reflective optical system of the annular optical system and is applied to the directional sensor. An imaging apparatus according to item 2, which is disposed in an optical path of an incident light beam.

  A fourth aspect of the present invention provides the imaging apparatus according to Item 2 or 3, wherein the light amount adjusting means is a variable ND filter.

  The fifth aspect of the present invention is the imaging apparatus according to item 1, further comprising a first light amount adjusting unit disposed in the first optical path and a second light amount adjusting unit disposed in the second optical path. provide.

  According to a sixth aspect of the present invention, there is provided the imaging apparatus according to Item 5, wherein the first light amount adjusting means and the second light amount adjusting means are variable ND filters.

  Item 7 is the seventh aspect of the present invention, wherein the directivity sensor has a directivity characteristic corresponding to the pupil shape of the central optical system and the annular optical system at the image plane position. An imaging apparatus is provided.

  In an eighth aspect of the present invention, the directional sensor has a plurality of microlenses that function as pupil dividing means, and the number and / or positions of pixels allocated per microlens of the plurality of microlenses is the center. The imaging apparatus according to claim 7, wherein the number and / or position of the optical system and the annular optical system is the number and / or position corresponding to the pupil position and shape.

  According to a ninth aspect of the present invention, the directivity sensor includes a microlens that functions as a pupil dividing unit, and the number of pixels per microlens for separating and entering the pupil image by the microlens is as follows. Item 9. The imaging device according to Item 7 or 8, wherein the imaging device is allocated so as to decrease as the image height increases.

  The tenth aspect of the present invention further includes a shading correction unit that performs shading correction of the first image and the second image according to a distance based on a center point corresponding to the optical axis of the photographing optical system. The imaging device of any one of -9 is provided.

  In an eleventh aspect of the present invention, the annular optical system has a reflection optical system that reflects the light beam twice or more, and the stop is an optical path of the light beam before being reflected by the reflection optical system of the ring optical system more than once. Item 11. The imaging device according to any one of Items 1 to 10 disposed in

  A twelfth aspect of the present invention provides the imaging apparatus according to any one of Items 1 to 11, wherein the central optical system has a wider angle than the annular optical system.

  The thirteenth aspect of the present invention is the display device according to any one of items 1 to 11, further comprising a display control unit that causes the display device to display the first image and the second image selectively, in a superimposed manner, or in parallel. An imaging apparatus is provided.

  The imaging apparatus according to the present invention includes a first optical path of a light beam incident on the directional sensor from the central optical system and a second optical path of a light beam incident on the directional sensor from the annular optical system, the first optical path and the second optical path. The apertures are individually arranged in each of the optical paths that do not overlap with each other, and a wide-angle image and a telephoto image in which the optical axes coincide with each other can be acquired simultaneously by a single imaging unit. I can capture it reliably.

1 is an external perspective view of an imaging apparatus according to the present invention. Sectional drawing which shows 1st Embodiment of the imaging part of an imaging device FIG. 2 is an enlarged view of a main part of the microlens array and the image sensor shown in FIG. The figure which shows the number of the light reception cells which one micro lens 16a takes charge of The figure which shows the color filter arrangement etc. which are arranged in the image sensor The block diagram which shows embodiment of the internal structure of an imaging device The figure which shows an example of the wide-angle image (photographing object exists in the peripheral part of a wide-angle image) and a telephoto image imaged with an imaging device The figure which shows an example of the wide-angle image (photographing object exists in the center part of a wide-angle image) and a telephoto image imaged with an imaging device The figure used to explain an example of a moving object detection method for detecting a moving object as an object to be imaged 6 is a flowchart showing an example of an imaging control method by the imaging apparatus according to the present invention. Side view showing another embodiment of a directional sensor Sectional drawing which shows other embodiment of the imaging part applicable to an imaging device Sectional drawing which shows other embodiment of the imaging part of an imaging device

  Hereinafter, embodiments of an imaging apparatus according to the present invention will be described with reference to the accompanying drawings.

<Appearance of imaging device>
FIG. 1 is an external perspective view of an imaging apparatus according to the present invention.

  FIG. 1 is an external perspective view of the imaging apparatus according to the first embodiment of the present invention. As shown in FIG. 1, a photographing optical system 12, a flash light emitting unit 21, and the like are arranged on the front surface of the imaging apparatus 10, a shutter button 38-1 on the top surface, and a variable ND filter around the lens of the photographing optical system 12. An adjustment button 38-2 and a zoom button 38-3 are provided on the side surface. L represents the optical axis of the photographing optical system 12.

[Configuration of imaging unit]
FIG. 2 is a cross-sectional view illustrating the first embodiment of the imaging unit 11 of the imaging device 10.

  As illustrated in FIG. 2, the imaging unit 11 includes a photographing optical system 12, a directivity sensor 17, a diaphragm 63, and a variable ND filter 64.

<Photographing optical system>
The photographing optical system 12 is arranged on the same optical axis, and a central optical system 13 as a first optical system and an annular optical system as a concentric second optical system at the periphery thereof. 14.

  The central optical system 13 is a wide-angle optical system (wide-angle lens) that includes a first lens 13 a, a second lens 13 b, a third lens 13 c, a fourth lens 13 d, and a common lens 15. A wide-angle image is formed on the microlens array 16.

  The annular optical system 14 is a telephoto optical system (telephoto lens) including a first lens 14a, a second lens 14b, a first reflection mirror 14c as a reflection optical system, a second reflection mirror 14d, and a common lens 15. A telephoto image is formed on the microlens array 16. The light beam incident through the first lens 14a and the second lens 14b is reflected twice by the first reflection mirror 14c and the second reflection mirror 14d, and then passes through the common lens 15. The light beam is folded back by the first reflecting mirror 14c and the second reflecting mirror 14d, so that the length in the optical axis direction of the telephoto optical system (telephoto lens) having a long focal length is shortened.

  The diaphragm 63 includes a diaphragm 63 a corresponding to the central optical system 13 and a diaphragm 63 b corresponding to the annular optical system 14. As illustrated in FIG. 4B, the diaphragm 63a changes the incident amount of the light beam that forms an image on the microlens array 16 from the central optical system 13 by expanding and contracting the outer diameter of the hole formed in the central portion. Let In addition, as illustrated in FIG. 4C, the stop 63 b expands and contracts the inner diameter of the annular hole formed in the peripheral portion, so that the incident amount of the light beam that forms an image on the microlens array 16 from the annular optical system 14. To change.

  The variable ND filter 64 provided between the common lens 15 and the image sensor 18 has a plurality of continuous or stepwise transmittances, and the applied voltage is changed or mechanically rotated. Then, an arbitrary transmittance is selected from them, and the received light amount of the subject image formed on the image sensor 18 from the photographing optical system 12 is adjusted.

  Since the central optical system 13 and the annular optical system 14 are arranged on the same optical axis, no parallax occurs in the wide-angle image and the telephoto image that are formed.

<Directivity sensor>
The directivity sensor 17 includes a microlens array 16 and an image sensor 18.

  FIG. 3 is an enlarged view of a main part of the microlens array 16 and the image sensor 18.

  The microlens array 16 includes a plurality of microlenses (pupil imaging lenses) 16 a arranged in a two-dimensional manner, and the horizontal and vertical intervals between the microlenses are the photoelectric conversion elements of the image sensor 18. This corresponds to the interval of three light receiving cells 18a. That is, each microlens of the microlens array 16 is formed corresponding to the position of every two light receiving cells in each of the horizontal direction and the vertical direction.

  Each microlens 16a of the microlens array 16 has a circular central pupil image (first pupil image) 17a and an annular pupil image (first pupil image) corresponding to the central optical system 13 and the annular optical system 14 of the photographing optical system 12. 2b) 17b is formed on the light receiving cell 18a in the corresponding light receiving region of the image sensor 18.

  According to the microlens array 16 and the image sensor 18 shown in FIG. 3, 3 × 3 light receiving cells 18 a having a lattice shape (square lattice shape) are allocated per microlens 16 a of the microlens array 16. . Hereinafter, one microlens 16a and a light receiving cell group (3 × 3 light receiving cells 18a) corresponding to one microlens 16a are referred to as a unit block.

  The central pupil image 17a is formed only on the light receiving cell 18a at the center of the unit block, and the annular pupil image 17b is formed on the eight light receiving cells 18a around the unit block.

  According to the imaging unit 11 configured as described above, a wide-angle image corresponding to the central optical system 13 and a telephoto image corresponding to the annular optical system 14 can be simultaneously captured as described later.

  FIG. 4 shows an arrangement example of the light receiving cells 18a. The light receiving cell 18a has a structure in which the number of light receiving cells handled by one microlens 16a decreases from the center toward the periphery (as the image height increases). In the example shown in FIG. 4, the number of light receiving cells 18a per microlens 16a (the number in the vertical direction of the paper surface) is three at the center (incident angle 0 °) and at the middle (incident angle 3.2 °). Two and one at the periphery (incident angle 14.4 °).

  As shown in FIG. 4, when the size of the light receiving cell 18a is the same, the microlens 16a having a smaller size and a shorter focal length is used as it goes to the periphery. As a result, higher density sampling is possible in the periphery.

[Embodiment of Image Sensor]
FIG. 5 is a diagram showing a color filter array and the like disposed in the image sensor 18. In FIG. 4, the microlens array 16 is omitted. However, the region indicated by a circle includes 3 × 3 light receiving cells on which pupil images are formed by the microlenses 16 a of the microlens array 16. The unit block to be included is shown.

  As shown in FIG. 5A, on the imaging surface of the image sensor 18, a color filter array including color filters arranged on each light receiving cell is provided.

  This color filter array is constituted by three primary color filters (hereinafter referred to as R filter, G filter, and B filter) that transmit light in each wavelength band of red (R), green (G), and blue (B). Has been. One of the RGB filters is disposed on each light receiving cell. Hereinafter, the light receiving cell in which the R filter is disposed is referred to as “R light receiving cell”, the light receiving cell in which the G filter is disposed is referred to as “G light receiving cell”, and the light receiving cell in which the B filter is disposed is referred to as “B light receiving cell”.

  In the color filter array shown in FIG. 5A, 6 × 6 light receiving cells are defined as a basic block B (see a block indicated by a thick frame in FIG. 5A and FIG. 5B). Are repeatedly arranged in the horizontal direction and the vertical direction.

  As shown in FIG. 5B, the basic block B includes four unit blocks B1 to B4.

  5 (c1) and 5 (c2) show a group of light receiving cells (light receiving cells into which a light beam that has passed through the central optical system 13 shown in FIG. 3) of the four unit blocks B1 to B4, and the surroundings. 8 groups of light receiving cells (light receiving cells into which the speed of light having passed through the annular optical system 14 shown in FIG. 3 is incident).

  As shown in FIG. 5C1, the image of the group of light receiving cells in the center is a Bayer array mosaic image. Accordingly, a color image can be obtained by demosaicing the mosaic image of the Bayer array.

  On the other hand, as shown in FIG. 5 (c2), a group of eight light receiving cells around each central light receiving cell of the unit blocks B1 to B4 is composed of all the RGB light receiving cells (R RGB light receiving cells are arranged in the same pattern regardless of the unit blocks B1 to B4.

  Specifically, G light receiving cells are arranged in the four light receiving cells at the four corners of each of the unit blocks B1 to B4, and R light receiving cells are arranged in the two upper and lower light receiving cells across the center light receiving cell. The B light receiving cells are arranged in the two left and right light receiving cells with the central light receiving cell interposed therebetween.

  In addition, the R light receiving cell, the G light receiving cell, and the B light receiving cell are arranged at symmetrical positions with respect to the center light receiving cell (center) of the unit block. Thereby, using the output signal of the RGB light receiving cell in the unit block, one pixel (RGB pixel value) constituting the image after the demosaic process (synchronization process) for each unit block is generated. be able to.

  That is, by obtaining the average value of the output signals (pixel values) of the four G light receiving cells in the unit block, the pixel value of the G pixel at the center position of the unit block (1 microlens) can be obtained. Similarly, by calculating the average value of the pixel values of the two R light receiving cells in the unit block and the average value of the pixel values of the two B light receiving cells, the R pixel and the B pixel at the center position of the unit block, respectively. Pixel values can be acquired.

  Thereby, for the telephoto image corresponding to the annular optical system 14 (telephoto optical system) generated by the group of eight light receiving cells around the unit block, the pixel values of the RGB light receiving cells in the unit block are used. Therefore, it is not necessary to generate pixel values of pixels in a specific wavelength range by interpolating the pixel values of the light receiving cells of the surrounding unit blocks, and the resolution of the output image (substantial number of pixels) ) Is not reduced.

<Internal configuration of imaging device>
FIG. 6 is a block diagram illustrating an embodiment of the internal configuration of the imaging apparatus 10.

  As shown in FIG. 6, the imaging apparatus 10 includes the photographing optical system 12 having the central optical system 13 and the annular optical system 14 described with reference to FIG. 2, the microlens array 16 and the image sensor 18 described with reference to FIGS. An imaging unit 11 including a directional sensor 17 is provided.

  The imaging unit 11 preferably includes a focus adjustment unit 19 that performs focus adjustment of the telephoto optical system (annular optical system 14). The focus adjustment unit 19 can be configured by, for example, a voice coil motor or the like that moves the entire or part of the annular optical system 14 in the optical axis direction. In addition, the focus of the telephoto image can be determined based on the contrast of the focus detection area of the telephoto image, but the focus adjustment method is not limited to this. For the wide-angle optical system (central optical system 13), a focus adjustment unit may be provided separately or pan focus may be used.

  The imaging unit 11 captures time-series wide-angle images and telephoto images via the photographing optical system 12 and the directivity sensor 17, and each of the directivity sensors 17 (image sensors 18) via the photographing optical system 12. The subject image formed on the light receiving surface of the light receiving cell (photoelectric conversion element) is converted into a signal voltage (or charge) in an amount corresponding to the amount of incident light.

  The lens CPU 65 determines the target density (dimming rate) of the variable ND filter 64 that works in conjunction with the increase in the aperture diameter of the diaphragm 63 and cancels the exposure amount (exposure amount increment ΔEV) that increases accordingly. The lens CPU 65 applies a signal corresponding to the determined density to the drive system of the variable ND filter 64. Thereby, the density of the variable ND filter 64 is adjusted. That is, the lens CPU 65 compares the current density of the variable ND filter 64 with the target density, determines again the target density (dimming rate) so as to eliminate the difference, and a control signal corresponding to the determined density. Is applied to the drive system of the variable ND filter 64. The drive system further selects the transmittance corresponding to the control signal, so that the density of the variable ND filter 64 is adjusted and the received light amount of the image sensor is adjusted before and after the aperture 63 is opened. Is done.

  The transmittance of the variable ND filter 64 can be changed by an input to the variable ND filter adjustment button 38-2 configured by operation means such as a switch, a button, and a touch sensor. Preferably, the variable ND filter adjustment button 38-2 is provided around the lens.

  The signal voltage (or charge) accumulated in the image sensor 18 is accumulated in the light receiving cell itself or an attached capacitor. The stored signal voltage (or charge) is read out together with the selection of the light receiving cell position by using a method of a MOS type image pickup device (so-called CMOS sensor) using an XY address method.

  Thereby, the pixel signal indicating the wide-angle image of the group of the light receiving cells in the center corresponding to the central optical system 13 from the image sensor 18 and the pixel indicating the telephoto image of the group of eight light receiving cells corresponding to the annular optical system 14. Signal can be read out. Note that pixel signals indicating a wide-angle image and a telephoto image are continuously read from the image sensor 18 at a predetermined frame rate (for example, the number of frames per second is 24p, 30p, or 60p).

  The pixel signal (voltage signal) read from the image sensor 18 is converted into an output signal for each light receiving cell for the purpose of reducing correlated double sampling processing (noise (particularly thermal noise) included in the sensor output signal). The pixel signal for each light-receiving cell is sampled and held by a process of obtaining accurate pixel data by taking the difference between the included feedthrough component level and the signal component level, and is amplified and applied to the A / D converter 20 . The A / D converter 20 converts sequentially input pixel signals into digital signals and outputs them to the image acquisition unit 22. Note that some MOS type sensors include an A / D converter. In this case, a digital signal is directly output from the image sensor 18.

  The image acquisition unit 22 can acquire the pixel signal indicating the wide-angle image and the pixel signal indicating the telephoto image simultaneously or selectively by selecting the light receiving cell position of the image sensor 18 and reading out the pixel signal. .

  That is, by selectively reading out the pixel signal of the light receiving cell on which the central pupil image 17a of the image sensor 18 enters, the wide angle of one light receiving cell (light receiving cell in the center of 3 × 3 light receiving cells) per microlens. A pixel signal indicating an image (a pixel signal indicating a mosaic image of a Bayer array) can be acquired. On the other hand, by selectively reading out a pixel signal of a light receiving cell on which the annular pupil image 17b of the image sensor 18 is incident, 1 A pixel signal indicating a telephoto image of eight light receiving cells (light receiving cells around a 3 × 3 light receiving cell) per microlens can be acquired.

  Note that all pixel signals are read from the image sensor 18 and temporarily stored in the buffer memory, and the pixel signals of the two images of the wide-angle image and the telephoto image are grouped from the pixel signals stored in the buffer memory. Also good.

  Pixel signals indicating the wide-angle image and the telephoto image acquired by the image acquisition unit 22 are output to the digital signal processing unit 40 and the object detection unit 50, respectively.

  The digital signal processing unit 40 performs offset processing, gamma correction processing, and demosaicing processing on RGB mosaic image signals on input digital pixel signals (RGB dot-sequential R signal, G signal, B signal). Then, predetermined signal processing such as white balance correction processing based on the color temperature of the light source individually estimated from each of the wide-angle image and the telephoto image is performed.

  Here, the demosaic process is a process of calculating all color information for each pixel from the RGB mosaic image corresponding to the color filter array of the single-plate image sensor 18, and is also referred to as a synchronization process. For example, in the case of the image sensor 18 composed of RGB color filters, this is a process of calculating color information of all RGB for each pixel from a mosaic image composed of RGB.

  The white balance correction processing based on the color temperature of the light source individually estimated from each of the wide-angle image and the telephoto image is the light receiving cell in the center of FIG. 4 (c1) and the eight in the periphery of FIG. 4 (c2). The light source type (the color temperature of the object field) is automatically obtained based on the R, G, B image signals corresponding to the wide-angle image and the telephoto image obtained from each of the light receiving cells, and the light source type is determined in advance. R, G, and B white balance gains (white balance correction values) set in the table are read out corresponding white balance gains, and R, G, and B image signals corresponding to the wide-angle image and the telephoto image, respectively. And the corresponding white balance gain.

  That is, the demosaic processing unit included in the digital signal processing unit 40 has no R light receiving cell and B light receiving cell at the position of the G light receiving cell in the wide-angle image (Bayer array mosaic image). R signal and B signal of the R light receiving cell and B light receiving cell are respectively interpolated to generate R signal and B signal at the position of the G light receiving cell. Similarly, since there are no G light receiving cell and B light receiving cell at the position of the R light receiving cell in the mosaic image, the G light receiving cell around the R light receiving cell, the G signal of the B light receiving cell, and the B signal are respectively interpolated. G signal and B signal are generated at the position of the R light receiving cell, and since there are no G light receiving cell and R light receiving cell at the position of the B light receiving cell of the mosaic image, the G light receiving cells around the B light receiving cell, The G signal and R signal at the position of the B light receiving cell are generated by interpolating the G signal and R signal of the R light receiving cell, respectively.

  On the other hand, as shown in FIG. 5 (c2), the telephoto image is composed of 8 mosaic images per microlens 16a (8 around the 3 × 3 unit block), and in the 8 light receiving cells. Since all the RGB color information (R light receiving cell, G light receiving cell, B light receiving cell) is included, the demosaic processing unit uses the output signals of the eight light receiving cells in the unit block. It is possible to generate one pixel (RGB pixel value) constituting an image, demosaiced every time.

  Specifically, the demosaic processing unit that performs demosaic processing of the mosaic image of the telephoto image obtains the average value of the pixel values of the four G light receiving cells in the unit block, thereby obtaining the center position of the unit block (1 microlens). By calculating the G pixel value of the pixel at the same time, and similarly calculating the average value of the pixel values of the two R light receiving cells and the average value of the pixel values of the two B light receiving cells in the unit block, respectively. The R pixel value and B pixel value of the pixel at the center position of the block are calculated.

  The demosaic image of the telephoto image of the two demosaic images of the wide-angle image and the telephoto image generated by the demosaic processing unit is demosaiced using the output signals of the eight light receiving cells in the unit block. The resolution is substantially higher than the demosaic image of the wide-angle image in which the demosaic process is performed using (interpolating) the output signals of the light receiving cells of the surrounding unit blocks.

  Further, the digital signal processing unit 40 performs RGB / YC conversion for generating the luminance signal Y and the color difference signals Cb, Cr from the RGB color information (R signal, G signal, B signal) demosaiced by the demosaic processing unit. And generating image signals for recording and displaying moving images indicating wide-angle images and telephoto images at a predetermined frame rate.

  Image signals indicating the wide-angle image and the telephoto image processed by the digital signal processing unit 40 are output to the recording unit 42 and the display control unit 45, respectively. The recording unit 42 records a moving image recording image signal indicating the wide-angle image and / or the telephoto image processed by the digital signal processing unit 40 on a recording medium (hard disk, memory card, or the like). The recording unit 42 may record the telephoto image when an object is detected from the angle of view corresponding to the telephoto image in the wide-angle image, and may record only the wide-angle image otherwise.

  The display unit 44 displays a wide-angle image and a telephoto image by an image signal for moving image display indicating a wide-angle image or a telephoto image selectively output from the display control unit 45. The display unit 44 can also reproduce a wide-angle image and / or a telephoto image based on the image signal recorded in the recording unit 42.

  On the other hand, the object detection unit 50 detects an object to be imaged based on the pixel signal indicating the wide-angle image acquired by the image acquisition unit 22, and outputs position information of the detected object in the image to the display control unit 45. It is.

  As a method for detecting an object in the object detection unit 50, a technique for performing human face recognition, a method for detecting a specific object by an object recognition technique represented by number recognition of a license plate of a vehicle, or a moving object is a tracking target. There is a moving object detection method for detecting an object.

  The object detection method based on object recognition is based on the characteristics of how a particular object looks (such as the shape of a person's face or the number of a car license plate) as an object dictionary. This is a method of recognizing an object by comparing an image cut out while changing its size and an object dictionary.

  The shading correction unit 70 strengthens concentric shading correction as the distance from the center of each image increases with respect to the wide-angle image and the telephoto image with reference to the center point corresponding to the optical axis of the imaging unit 11. This is because, at the center of the light receiving surface of the image sensor 18, the light beam enters in parallel to the optical axis of the microlens 16, so that a signal with uniform brightness is output between the photodiodes of the image sensor 18. At the peripheral edge of the light receiving surface of the image sensor 18, the light beam is incident obliquely with respect to the optical axis of the microlens 16. This is because the pixel density increases as the edge of the screen decreases.

  7 and 8 are diagrams illustrating examples of the wide-angle image and the telephoto image that are captured, respectively. In addition, the area | region shown with the broken line in a wide angle image has shown the imaging range of the telephoto image.

  Now, a wide-angle image I1 and a telephoto image I2 (corresponding to a range R in the wide-angle image I1) shown in FIGS. 7A and 7B are captured, and a person's face is a tracking target, and object (face) recognition is performed. When a human face is detected by technology, a face can be detected in a wide-angle image, but a face cannot be detected in a telephoto image. This is because the telephoto image contains only a part of the human face and cannot recognize the face.

  Therefore, in this case, the position information of the person's face (object) in the wide-angle image is output to the display control unit 45. Further, the display control unit 45 selectively outputs a wide-angle image to the display unit 44. That is, when only a part of the tracking target is shown in the telephoto image, the wide-angle image is selectively displayed on the display unit 44 to make the tracking target easy to see.

  On the other hand, as shown in FIGS. 8A and 8B, when an object is detected in a range R corresponding to the telephoto image I2 in the wide-angle image I1, position information of the detected object in the telephoto image is displayed. In addition to outputting to the control unit 45, the display control unit 45 selectively outputs a telephoto image to the display unit 44. This is because when the tracking target appears in the telephoto image and an object is detected, the telephoto image is easier to see than the wide-angle image.

  Next, as an object detection method in the object detection unit 50, an example of a moving object detection method for detecting a moving object as a tracking target object will be described.

  In this case, as shown in FIG. 9, the object detection unit 50 includes two time-series wide-angle images (a wide-angle image acquired previously (FIG. 9A) and a wide-angle image acquired this time (FIG. 9B)). A difference image (FIG. 9C) obtained from the difference is detected.

  In the example shown in FIGS. 9A and 9B, the object A out of the objects A and B moves and the object B stops.

Therefore, as shown in FIG. 9C, the difference images A ₁ and A ₂ are images generated by the movement of the object A.

Here, the barycentric positions of the difference images A ₁ and A ₂ are calculated and set as positions P ₁ and P ₂ , respectively, and the midpoint of the line segment connecting these positions P ₁ and P ₂ is set as a position G. The position G is set as a position in the wide-angle image of the moving object (object A).

  The object A is repeatedly controlled by the imaging unit 11 so that the position G of the object A in the wide-angle image calculated in this way moves to the center position (position on the optical axis) of the wide-angle image. Moves (converges) to the center position of the wide-angle image.

  When the imaging unit 11 moves, the background between the time-series images also moves. In this case, the images are shifted so that the background between the time-series images matches, and between the images after the shift By taking the difference image, it is possible to detect an object moving in the real space regardless of the movement of the imaging unit 11. Furthermore, the moving object detection method is not limited to the above embodiment.

<Imaging control method>
FIG. 10 is a flowchart illustrating an example of an imaging control method by the imaging apparatus according to the present invention.

  In FIG. 10, the object detection unit 50 acquires a wide-angle image from the image acquisition unit 22 (step S20), and detects a tracking target object from the acquired wide-angle image (step S22).

  Subsequently, in step S22, the object detection unit 50 determines whether an object is detected from a range corresponding to the angle of view of the telephoto image in the wide-angle image (step S24). When an object is detected from this range (in the case of “Yes”), the object detection unit 50 calculates position information of the detected object in the wide-angle image (step S26). The display control unit 45 selectively outputs the telephoto image among the telephoto image and the wide-angle image output from the digital signal processing unit 40 to the display unit 44 (step S27). As a result, a telephoto image is displayed on the display unit 44. The recording unit 42 records an image signal for recording a moving image indicating the telephoto image processed by the digital signal processing unit 40 on a recording medium (step S27). Thereby, for example, when a person to be photographed is detected, recording of a telephoto image in which the person is captured within the angle of view is started.

  On the other hand, when an object is not detected from this range (in the case of “No”), the process proceeds to step S28. Here, in the processing from step S28 to step S32, the object detection unit 50 acquires a wide-angle image from the image acquisition unit 22 (step S28), detects the tracking target object from the acquired wide-angle image (step S30), and Position information in the wide-angle image of the detected object is calculated (step S32). The display control unit 45 selectively outputs a wide-angle image among the telephoto image and the wide-angle image output from the digital signal processing unit 40 to the display unit 44 (step S33). As a result, a wide-angle image is displayed on the display unit 44.

  Next, the display control unit 45 outputs the position information of the object in the wide-angle image detected in step S26 or step S32 (step S34).

  Next, it is determined whether or not imaging has ended (step S36). If it is determined that imaging has not ended, the process proceeds to step S20. Thereby, the processing from step S20 to step S36 is repeated, and imaging is performed by tracking the object. On the other hand, when it is determined that the imaging is finished, the imaging is finished.

<Other Embodiments of Directional Sensor>
FIG. 11 is a side view showing another embodiment of the directivity sensor.

  The directivity sensor 117 includes a microlens array 118 serving as a pupil dividing unit, a light shielding member 120 functioning as a light shielding mask, and an image sensor 116 in which a part of the light receiving cells 116 a and 116 b is shielded from light by the light shielding member 120. ing. The light receiving cells 116a and the light receiving cells 116b partially shielded by the light shielding member 120 are provided alternately (checker flag shape) in the horizontal direction and the vertical direction of the image sensor 116.

  The microlens array 118 includes microlenses 118 a that correspond one-to-one with the light receiving cells 116 a and 116 b of the image sensor 116.

  The light shielding member 120 regulates the opening of the light receiving cells 116a and 116b of the image sensor 116, and has an opening shape corresponding to the central optical system 13 and the annular optical system 14 of the photographing optical system 12 shown in FIG. ing. Note that red (R), green (G), and blue (B) color filters are disposed below each lens of the microlens array 118.

  In the light receiving cell 116a, the periphery of the opening is shielded by the light shielding portion 120a of the light shielding member 120, while in the light receiving cell 116b, the central portion of the opening is shielded by the light shielding portion 120b of the light shielding member 120. As a result, the light beam that has passed through the central optical system 13 of the photographic optical system 12 is pupil-divided by the microlens array 118 and the light shielding portion 120a of the light shielding member 120 and enters the light receiving cell 116a. The light beam that has passed through the optical system 14 is divided into pupils by the microlens array 118 and the light shielding portion 120b of the light shielding member 120, and enters the light receiving cell 116b.

  Thereby, the pixel signal of the wide-angle image can be read from each light receiving cell 116a of the image sensor 116, and the pixel signal of the telephoto image can be read from each light receiving cell 116b of the image sensor 116.

<Other Embodiments of Imaging Unit>
Next, another embodiment of the imaging unit applied to the imaging apparatus according to the present invention will be described.

  FIG. 12 is a cross-sectional view illustrating another embodiment of an imaging unit applicable to the imaging device 10.

  The imaging unit includes a photographic optical system 112 and a directivity sensor 17. Since the directivity sensor 17 is the same as that shown in FIGS. 2 and 3, the photographing optical system 112 will be described below.

  The photographing optical system 112 includes a central optical system 113 at the center and an annular optical system 114 at the periphery thereof arranged on the same optical axis.

  The central optical system 113 is a telephoto optical system including a first lens 113a, a second lens 113b, and a common lens 115, and has an angle of view α.

  The annular optical system 114 is a wide-angle optical system including a lens 114 a and a common lens 115, has an angle of view β (β> α), and is wider than the central optical system 113.

  Compared with the photographing optical system 12 shown in FIG. 2, the photographing optical system 112 does not use a reflecting mirror, the central optical system 113 is a telephoto optical system, and the annular optical system 114 is a wide-angle optical system. It is different in that.

<Other arrangement examples of variable ND filter>
As shown in FIG. 13, a wide-angle variable ND filter 64 a is provided on the light source side of the central optical system 13, and a telephoto variable ND filter 64 b is provided on the light source side of the annular optical system 14. Thus, the amount of light received by the subject image incident on the photographic optical system 12 may be individually adjusted by controlling each of them individually.

[Others]
In the photographing optical system of the present embodiment, the second optical system provided in the peripheral portion of the first optical system is an annular optical system, but is not limited to this, and is arranged on a concentric circle with the optical axis as the center. It may be configured by a plurality of optical systems provided.

  2 is not limited to a concave mirror or a convex mirror, and may be a plane mirror, and the number of reflection mirrors is not limited to two. You may make it provide more sheets.

  Further, the focus adjustment unit may move the common lens of the central optical system and the annular optical system or the image sensor in the optical axis direction.

  Furthermore, it goes without saying that the present invention is not limited to the embodiments described above, and various modifications can be made without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 10 ... Imaging device, 11 ... Image pick-up part, 12, 112 ... Shooting optical system, 13, 113 ... Central optical system, 14, 114 ... Annular optical system, 16, 118 ... Micro lens array, 16a, 118a ... Micro lens, 17 DESCRIPTION OF SYMBOLS 117 ... Directional sensor 18, 116 ... Image sensor, 18a, 116a, 116b ... Light receiving cell, 22 ... Image acquisition part, 42 ... Recording part, 50 ... Object detection part, 120 ... Light shielding member

Claims (13)

An imaging optical system composed of a central optical system at the center and an annular optical system at the periphery thereof, each disposed on the same optical axis,
A directional sensor having a plurality of pixels configured by two-dimensionally arranged photoelectric conversion elements, wherein light beams incident through the central optical system and the annular optical system are selectively divided by pupils. A directional sensor including a plurality of pixels for receiving light;
An image reading device for acquiring from the directivity sensor an image signal indicating a first image received via the central optical system and an image signal indicating a second image received via the annular optical system;
A first optical path of a light beam incident on the directional sensor from the central optical system and a second optical path of a light beam incident on the directional sensor from the annular optical system, the first optical path and the second optical path. An aperture individually disposed in each of the optical paths that do not overlap with each other;
An imaging apparatus comprising:   An optical path of a light beam incident on the directional sensor from the central optical system and a third optical path of a light beam incident on the directional sensor from the annular optical system, wherein the first optical path and the second optical path are The imaging apparatus according to claim 1, further comprising a light amount adjusting unit disposed in the overlapping third optical path. The annular optical system has a reflection optical system that reflects a light beam twice or more,
The imaging apparatus according to claim 2, wherein the light amount adjusting unit is disposed in an optical path of a light beam that is reflected twice or more by the reflection optical system of the annular optical system and enters the directivity sensor.   The imaging apparatus according to claim 2, wherein the light amount adjusting unit is a variable ND filter.   The imaging apparatus according to claim 1, further comprising a first light amount adjusting unit disposed in the first optical path and a second light amount adjusting unit disposed in the second optical path.   The imaging apparatus according to claim 5, wherein the first light amount adjusting unit and the second light amount adjusting unit are variable ND filters.   The imaging device according to claim 1, wherein the directional sensor has a directional characteristic corresponding to a pupil shape of the central optical system and the annular optical system at an image plane position. The directivity sensor has a plurality of microlenses that function as pupil dividing means,
The number and / or position of pixels allocated per microlens of the plurality of microlenses is a number and / or position corresponding to the pupil position and shape of the central optical system and the annular optical system. The imaging device described.   The directivity sensor has a microlens that functions as pupil dividing means, and separates the pupil image by the microlens and makes each incident, as the image height increases as the number of pixels per microlens increases. The imaging device according to claim 7 or 8, wherein the imaging device is allocated so as to decrease.   The shading correction part which performs the shading correction | amendment of the said 1st image and the said 2nd image according to the distance on the basis of the center point corresponding to the optical axis of the said imaging | photography optical system is further provided. The imaging apparatus according to item 1. The annular optical system has a reflection optical system that reflects a light beam twice or more,
The imaging device according to any one of claims 1 to 10, wherein the diaphragm is disposed in an optical path of a light beam before being reflected twice or more by the reflection optical system of the annular optical system.   The imaging device according to claim 1, wherein the central optical system has a wider angle than the annular optical system.   The imaging device according to claim 1, further comprising: a display control unit that causes the display device to display the first image and the second image selectively, in a superimposed manner, or in parallel.

Images