JP2018056943A - Imaging device and imaging element - Google Patents

Abstract

PROBLEM TO BE SOLVED: To increase the accuracy in focus adjustment.SOLUTION: An imaging device comprises: an imaging element including a plurality of imaging areas in which a plurality of pixels are arranged that generate signals with an electric charge photoelectrically converted through a lens movable in an optical axis direction of an optical system, and signal lines that are arranged respectively in the imaging areas and output the signals generated by the pixels; a generation part that generates a picked-up image of a subject from the pixels arranged in a first imaging area of the imaging areas by using signals from first pixels output to the signal lines in the first imaging area; and a control part that controls a movement of the lens by using the signals from second pixels that are arranged in a second imaging area different from the first imaging area of the imaging areas and in a larger number than the first pixels, the signals output to the signal lines in the second imaging area.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging device and an imaging element.

An imaging apparatus equipped with a technique for adjusting the focus of a lens by a signal from an imaging element is known (see Patent Document 1).
Conventionally, improvement in focus adjustment accuracy has been required.

JP 2006-197192 A

According to the first aspect, the imaging apparatus includes: a plurality of imaging regions in which a plurality of pixels that generate signals using electric charges photoelectrically converted through a lens movable in the optical axis direction of the optical system; An image sensor having a signal line arranged for each region and outputting a signal generated by the pixel; and from the pixels arranged in the first imaging region among the imaging regions, the first imaging region A generation unit that generates an image of a captured subject using the signal of the first pixel output to the signal line, and the second imaging area that is different from the first imaging area among the imaging areas. And a control unit that controls movement of the lens using a signal of a second pixel that is output from the pixel to the signal line of the second imaging region and is larger in number than the first pixel.
According to the second aspect, the imaging device includes: a plurality of imaging regions in which a plurality of pixels that generate signals by charges that are photoelectrically converted through a lens that is movable in the optical axis direction of the optical system; A signal line that is arranged for each region and that outputs a signal generated by the pixel, and the imaging region includes a first pixel that is used to generate an image of the captured subject. And a second imaging region having a second pixel larger than the number of the first pixels, which is used to control the movement of the lens.
According to the third aspect, the imaging device includes a first photoelectric conversion unit that converts light into electric charge in an imaging region that images a subject via a lens that is movable in the optical axis direction of the optical system, and the light as electric charge. A second imaging region in which a first imaging region in which a second photoelectric conversion unit for conversion is arranged, a third photoelectric conversion unit in which light is converted into electric charge, and a fourth photoelectric conversion unit in which light is converted into electric charge are arranged And an image sensor of the subject imaged using the first signal generated by the charge converted by the first photoelectric conversion unit and the second photoelectric conversion unit in the first imaging region. A generating unit that generates an image; and a signal generated by the charge converted by the third photoelectric conversion unit and a signal generated by the charge converted by the fourth photoelectric conversion unit in the second imaging region. And a control unit for controlling the movement of the lens. .

It is a block diagram which illustrates the composition of a camera. It is sectional drawing of a laminated type image pick-up element. It is a figure explaining the pixel arrangement | sequence and unit area | region of an imaging chip. It is a figure explaining the circuit in a unit area. It is a block diagram which shows the functional structure of an image pick-up element. FIG. 6A to FIG. 6C are diagrams illustrating the arrangement of the first imaging region and the second imaging region set on the imaging surface of the imaging device. FIG. 7A is a diagram illustrating a display image, and FIG. 7B is a diagram illustrating a detection image. FIG. 8A to FIG. 8D are diagrams illustrating the setting of the operation rate of the blocks included in the first imaging area and the operation rate of the blocks included in the second imaging area. It is a figure which illustrates the to-be-photographed object area | region in an imaging screen. It is a figure which illustrates a setting screen. It is a figure which illustrates a kernel. It is a figure which illustrates the position of the pixel for focus detection in an imaging surface. It is the figure which expanded the one part area | region of the focus detection pixel line. It is the figure which expanded the focus point. FIG. 15A is a diagram illustrating a template image representing an object to be detected, and FIG. 15B is a diagram illustrating a live view image and a search range. It is a flowchart explaining the flow of a process. FIG. 17A to FIG. 17D are diagrams for explaining setting of the operation rate of the blocks included in the first imaging area and the operation rate of the blocks included in the second imaging area according to the first modification. FIG. 18A to FIG. 18D are diagrams for explaining setting of the operation rate of the blocks included in the first imaging area and the operation rate of the blocks included in the second imaging area according to the second modification. FIGS. 19A and 19B are diagrams illustrating display images according to the third modification.

  An image sensor according to an embodiment and an electronic device in which the image sensor is mounted will be described. A digital camera 1 in FIG. 1 (hereinafter referred to as a camera 1) is an example of an electronic device. The camera 1 is equipped with an image sensor 32a as an example of an image sensor. The image sensor 32a is configured to be able to capture images under different conditions for each region of the imaging surface. The image processing unit 33 of the camera 1 performs appropriate processing on regions with different imaging conditions. Details of the camera 1 will be described with reference to the drawings.

<Explanation of camera>
FIG. 1 is a block diagram illustrating the configuration of a camera 1 according to an embodiment. In FIG. 1, the camera 1 includes an imaging optical system 31, an imaging unit 32, an image processing unit 33, a control unit 34, a display unit 35, an operation member 36, and a recording unit 37.

  The imaging optical system 31 guides the light beam from the object scene to the imaging unit 32. The imaging unit 32 includes an imaging element 32a and a driving unit 32b, and photoelectrically converts an object image formed by the imaging optical system 31. The imaging unit 32 can capture images under the same conditions over the entire imaging surface of the imaging element 32a, or can perform imaging under different conditions for each region of the imaging surface. Details of the imaging unit 32 will be described later. The drive unit 32b generates a drive signal necessary for causing the image sensor 32a to perform accumulation control. Imaging instructions such as charge accumulation time, ISO sensitivity (gain), and frame rate for the drive unit 32b are transmitted from the control unit 34 to the drive unit 32b.

  The image processing unit 33 includes an input unit 33a, a correction unit 33b, and a generation unit 33c. Image data acquired by the imaging unit 32 is input to the input unit 33a. The correction unit 33b corrects the input image data. The correction includes, for example, processing of matching levels of data (signal) from the first imaging area B1 described later and data (signal) from the second imaging area B2 described later. The generation unit 33c performs image processing on the input image data and the corrected image data, and generates an image. Image processing includes, for example, color interpolation processing, pixel defect correction processing, contour enhancement processing, noise reduction processing, white balance adjustment processing, gamma correction processing, display luminance adjustment processing, saturation adjustment processing, and the like. . Further, the generation unit 33 c generates an image to be displayed by the display unit 35.

  The control unit 34 is configured by a CPU, for example, and controls the overall operation of the camera 1. For example, the control unit 34 performs a predetermined exposure calculation based on the photoelectric conversion signal acquired by the imaging unit 32, the charge accumulation time (exposure time) of the imaging element 32a necessary for proper exposure, and the aperture of the imaging optical system 31. The exposure conditions such as the value and ISO sensitivity are determined and instructed to the drive unit 32b. In addition, image processing conditions for adjusting saturation, contrast, sharpness, and the like are determined and instructed to the image processing unit 33 according to the imaging scene mode set in the camera 1 and the type of the detected subject element. The detection of the subject element will be described later.

  The control unit 34 includes an object detection unit 34a, a setting unit 34b, an imaging control unit 34c, and a lens movement control unit 34d. These are realized as software by the control unit 34 executing a program stored in a nonvolatile memory (not shown). However, these may be configured by an ASIC or the like.

  The object detection unit 34a performs a known object recognition process, and from the image data acquired by the imaging unit 32, a person (person's face), an animal such as a dog or a cat (animal face), a plant, a bicycle, A subject element such as a vehicle, a vehicle such as a train, a building, a stationary object, a landscape such as a mountain or a cloud, or a predetermined specific object is detected. The setting unit 34b divides the image data acquired by the imaging unit 32 into a plurality of regions including the subject element detected as described above.

  The setting unit 34b further sets imaging conditions for a plurality of areas. Imaging conditions include the exposure conditions (charge accumulation time, gain, ISO sensitivity, frame rate, etc.) and the image processing conditions (for example, white balance adjustment parameters, gamma correction curves, display brightness adjustment parameters, saturation adjustment parameters, etc.) ). As the imaging conditions, the same imaging conditions can be set for a plurality of areas, or different imaging conditions can be set for a plurality of areas.

  The imaging control unit 34c controls the imaging unit 32 (imaging element 32a) and the image processing unit 33 by applying the imaging conditions set for each region by the setting unit 34b. Thereby, it is possible to cause the imaging unit 32 to perform imaging under different exposure conditions for each of the plurality of regions, and for the image processing unit 33, images with different image processing conditions for each of the plurality of regions. Processing can be performed. Any number of pixels may be included in the region, for example, 1000 pixels or 1 pixel. Further, the number of pixels may be different between regions.

  The lens movement control unit 34d controls an automatic focus adjustment (autofocus: AF) operation for focusing on a corresponding subject at a predetermined position (called a focus point) on the imaging screen. When the focus is adjusted, the sharpness of the subject image increases. That is, the image by the imaging optical system 31 is adjusted by moving the focus lens of the imaging optical system 31 in the optical axis direction. The lens movement control unit 34d is a drive signal for moving the focus lens of the imaging optical system 31 to the in-focus position based on the calculation result, for example, a signal for adjusting the subject image with the focus lens of the imaging optical system 31. Is sent to the lens driving mechanism 31m of the imaging optical system 31. In this way, the lens movement control unit 34d functions as a moving unit that moves the focus lens of the imaging optical system 31 in the optical axis direction based on the calculation result. The process performed by the lens movement control unit 34d for the AF operation is also referred to as a focus detection process. Details of the focus detection process will be described later.

  The display unit 35 reproduces and displays the image generated by the image processing unit 33, the image processed image, the image read by the recording unit 37, and the like. The display unit 35 also displays an operation menu screen, a setting screen for setting imaging conditions, and the like.

  The operation member 36 includes various operation members such as a release button and a menu button. The operation member 36 sends an operation signal corresponding to each operation to the control unit 34. The operation member 36 includes a touch operation member provided on the display surface of the display unit 35.

  In response to an instruction from the control unit 34, the recording unit 37 records image data or the like on a recording medium including a memory card (not shown). The recording unit 37 reads image data recorded on the recording medium in response to an instruction from the control unit 34.

<Description of Laminated Image Sensor>
A laminated image sensor 100 will be described as an example of the image sensor 32a described above. FIG. 2 is a cross-sectional view of the image sensor 100. The imaging element 100 includes an imaging chip 111, a signal processing chip 112, and a memory chip 113. The imaging chip 111 is stacked on the signal processing chip 112. The signal processing chip 112 is stacked on the memory chip 113. The imaging chip 111, the signal processing chip 112, the signal processing chip 112, and the memory chip 113 are electrically connected by a connection unit 109. The connection unit 109 is, for example, a bump or an electrode. The imaging chip 111 captures a light image from a subject and generates image data. The imaging chip 111 outputs image data from the imaging chip 111 to the signal processing chip 112. The signal processing chip 112 performs signal processing on the image data output from the imaging chip 111. The memory chip 113 has a plurality of memories and stores image data. Note that the image sensor 100 may include an image pickup chip and a signal processing chip. When the imaging device 100 is configured by an imaging chip and a signal processing chip, a storage unit for storing image data may be provided in the signal processing chip or may be provided separately from the imaging device 100. .

  As shown in FIG. 2, the incident light is incident mainly in the positive direction of the Z-axis indicated by a white arrow. Further, as shown in the coordinate axes, the left direction of the paper orthogonal to the Z axis is the X axis plus direction, and the front side of the paper orthogonal to the Z axis and X axis is the Y axis plus direction. In the following several figures, the coordinate axes are displayed so that the orientation of each figure can be understood with reference to the coordinate axes in FIG.

  The imaging chip 111 is a CMOS image sensor, for example. Specifically, the imaging chip 111 is a backside illumination type CMOS image sensor. The imaging chip 111 includes a microlens layer 101, a color filter layer 102, a passivation layer 103, a semiconductor layer 106, and a wiring layer 108. The imaging chip 111 is arranged in the order of the microlens layer 101, the color filter layer 102, the passivation layer 103, the semiconductor layer 106, and the wiring layer 108 in the positive Z-axis direction.

  The microlens layer 101 has a plurality of microlenses L. The microlens L condenses incident light on the photoelectric conversion unit 104 described later. The color filter layer 102 has a plurality of types of color filters F having different spectral characteristics. Specifically, the color filter layer 102 includes a first filter (R) having a spectral characteristic that mainly transmits red component light and a second filter (Gb, Gr) that has a spectral characteristic that mainly transmits green component light. ) And a third filter (B) having a spectral characteristic that mainly transmits blue component light. In the color filter layer 102, for example, a first filter, a second filter, and a third filter are arranged in a Bayer arrangement. The passivation layer 103 is made of a nitride film or an oxide film, and protects the semiconductor layer 106.

  The semiconductor layer 106 includes a photoelectric conversion unit 104 and a readout circuit 105. The semiconductor layer 106 includes a plurality of photoelectric conversion units 104 between a first surface 106a that is a light incident surface and a second surface 106b opposite to the first surface 106a. The semiconductor layer 106 includes a plurality of photoelectric conversion units 104 arranged in the X-axis direction and the Y-axis direction. The photoelectric conversion unit 104 has a photoelectric conversion function of converting light into electric charge. In addition, the photoelectric conversion unit 104 accumulates charges based on the photoelectric conversion signal. The photoelectric conversion unit 104 is, for example, a photodiode. The semiconductor layer 106 includes a readout circuit 105 on the second surface 106b side of the photoelectric conversion unit 104. In the semiconductor layer 106, a plurality of readout circuits 105 are arranged in the X-axis direction and the Y-axis direction. The readout circuit 105 includes a plurality of transistors, reads out image data generated by the electric charges photoelectrically converted by the photoelectric conversion unit 104, and outputs the image data to the wiring layer 108.

  The wiring layer 108 has a plurality of metal layers. The metal layer is, for example, an Al wiring, a Cu wiring, or the like. The wiring layer 108 outputs the image data read by the reading circuit 105. The image data is output from the wiring layer 108 to the signal processing chip 112 via the connection unit 109.

  Note that the connection unit 109 may be provided for each photoelectric conversion unit 104. Further, the connection unit 109 may be provided for each of the plurality of photoelectric conversion units 104. When the connection unit 109 is provided for each of the plurality of photoelectric conversion units 104, the pitch of the connection units 109 may be larger than the pitch of the photoelectric conversion units 104. In addition, the connection unit 109 may be provided in a peripheral region of the region where the photoelectric conversion unit 104 is disposed.

  The signal processing chip 112 has a plurality of signal processing circuits. The signal processing circuit performs signal processing on the image data output from the imaging chip 111. The signal processing circuit includes, for example, an amplifier circuit that amplifies the signal value of the image data, a correlated double sampling circuit that performs noise reduction processing of the image data, and analog / digital (A / D) conversion that converts the analog signal into a digital signal. Circuit etc. A signal processing circuit may be provided for each photoelectric conversion unit 104.

  In addition, the signal processing circuit may be provided for each of the plurality of photoelectric conversion units 104. The signal processing chip 112 has a plurality of through electrodes 110. The through electrode 110 is, for example, a silicon through electrode. The through electrode 110 connects circuits provided in the signal processing chip 112 to each other. The through electrode 110 may also be provided in the peripheral region of the imaging chip 111 and the memory chip 113. Note that some elements constituting the signal processing circuit may be provided in the imaging chip 111. For example, in the case of an analog / digital conversion circuit, a comparator that compares an input voltage with a reference voltage may be provided in the imaging chip 111, and a circuit such as a counter circuit or a latch circuit may be provided in the signal processing chip 112.

  The memory chip 113 has a plurality of storage units. The storage unit stores image data that has been subjected to signal processing by the signal processing chip 112. The storage unit is a volatile memory such as a DRAM, for example. A storage unit may be provided for each photoelectric conversion unit 104. In addition, the storage unit may be provided for each of the plurality of photoelectric conversion units 104. The image data stored in the storage unit is output to the subsequent image processing unit.

  FIG. 3 is a diagram for explaining the pixel array and the unit region 131 of the imaging chip 111. In particular, a state where the imaging chip 111 is observed from the back surface (imaging surface) side is shown. For example, 20 million or more pixels are arranged in a matrix in the pixel region. In the example of FIG. 3, four adjacent pixels of 2 pixels × 2 pixels form one unit region 131. The grid lines in the figure indicate the concept that adjacent pixels are grouped to form a unit region 131. The number of pixels forming the unit region 131 is not limited to this, and may be about 1000, for example, 32 pixels × 32 pixels, more or less, or one pixel.

  As shown in the partially enlarged view of the pixel region, the unit region 131 in FIG. 3 includes a so-called Bayer array composed of four pixels of the green pixels Gb and Gr, the blue pixel B, and the red pixel R. The green pixels Gb and Gr are pixels having a green filter as the color filter F, and receive light in the green wavelength band of incident light. Similarly, the blue pixel B is a pixel having a blue filter as the color filter F and receives light in the blue wavelength band, and the red pixel R is a pixel having a red filter as the color filter F and having a red wavelength band. Receives light.

  In the present embodiment, a plurality of blocks are defined so as to include at least one unit region 131 per block. That is, the minimum unit of one block is one unit area 131. As described above, of the possible values for the number of pixels forming one unit region 131, the smallest number of pixels is one pixel. Therefore, when one block is defined in units of pixels, the minimum number of pixels among the number of pixels that can define one block is one pixel. Each block can control pixels included in each block with different control parameters. In each block, all the unit areas 131 in the block, that is, all the pixels in the block are controlled under the same imaging condition. That is, photoelectric conversion signals having different imaging conditions can be acquired between a pixel group included in a certain block and a pixel group included in another block. Examples of the control parameters include a frame rate, a gain, a thinning rate, the number of addition rows or addition columns to which photoelectric conversion signals are added, a charge accumulation time or accumulation count, a digitization bit number (word length), and the like. The imaging device 100 can freely perform not only thinning in the row direction (X-axis direction of the imaging chip 111) but also thinning in the column direction (Y-axis direction of the imaging chip 111). Furthermore, the control parameter may be a parameter in image processing.

  FIG. 4 is a diagram for explaining a circuit in the unit region 131. In the example of FIG. 4, one unit region 131 is formed by four adjacent pixels of 2 pixels × 2 pixels. As described above, the number of pixels included in the unit region 131 is not limited to this, and may be 1000 pixels or more, or may be a minimum of 1 pixel. The two-dimensional position of the unit region 131 is indicated by reference signs A to D.

  The reset transistor (RST) of the pixel included in the unit region 131 is configured to be turned on / off individually for each pixel. In FIG. 4, a reset wiring 300 for turning on / off the reset transistor of the pixel A is provided, and a reset wiring 310 for turning on / off the reset transistor of the pixel B is provided separately from the reset wiring 300. Similarly, a reset line 320 for turning on and off the reset transistor of the pixel C is provided separately from the reset lines 300 and 310. A dedicated reset wiring 330 for turning on and off the reset transistor is also provided for the other pixels D.

  The pixel transfer transistor (TX) included in the unit region 131 is also configured to be turned on and off individually for each pixel. In FIG. 4, a transfer wiring 302 for turning on / off the transfer transistor of the pixel A, a transfer wiring 312 for turning on / off the transfer transistor of the pixel B, and a transfer wiring 322 for turning on / off the transfer transistor of the pixel C are separately provided. Also for the other pixels D, a dedicated transfer wiring 332 for turning on / off the transfer transistor is provided.

  Further, the pixel selection transistor (SEL) included in the unit region 131 is also configured to be turned on and off individually for each pixel. In FIG. 4, a selection wiring 306 for turning on / off the selection transistor of the pixel A, a selection wiring 316 for turning on / off the selection transistor of the pixel B, and a selection wiring 326 for turning on / off the selection transistor of the pixel C are separately provided. Also for the other pixels D, a dedicated selection wiring 336 for turning on and off the selection transistor is provided.

  The power supply wiring 304 is commonly connected to the pixels A to D included in the unit region 131. Similarly, the output wiring 308 is commonly connected to the pixel D from the pixel A included in the unit region 131. Further, the power supply wiring 304 is commonly connected between a plurality of unit regions, but the output wiring 308 is provided for each unit region 131 individually. The load current source 309 supplies current to the output wiring 308. The load current source 309 may be provided on the imaging chip 111 side or may be provided on the signal processing chip 112 side.

  By individually turning on and off the reset transistor and the transfer transistor in the unit region 131, the charge accumulation including the charge accumulation start time, the accumulation end time, and the transfer timing is controlled from the pixel A to the pixel D included in the unit region 131. can do. In addition, by individually turning on and off the selection transistors in the unit region 131, the photoelectric conversion signals of the pixels A to D can be output via the common output wiring 308.

  Here, a so-called rolling shutter system is known in which charge accumulation is controlled in a regular order with respect to rows and columns for pixels A to D included in the unit region 131. When a column is designated after selecting a pixel for each row by the rolling shutter method, photoelectric conversion signals are output in the order of “ABCD” in the example of FIG.

  Thus, by configuring the circuit with the unit region 131 as a reference, the charge accumulation time can be controlled for each unit region 131. For example, photoelectric conversion signals with different frame rates between the unit regions 131 can be output. In addition, the unit area 131 included in another block is paused while the unit areas 131 included in some blocks in the imaging chip 111 perform charge accumulation (imaging). Only the imaging can be performed, and the photoelectric conversion signal can be output. Furthermore, it is also possible to switch a block (accumulation control target block) where charge accumulation (imaging) is performed between frames, sequentially perform imaging with different blocks of the imaging chip 111, and output a photoelectric conversion signal.

  FIG. 5 is a block diagram illustrating a functional configuration of the image sensor 100 corresponding to the circuit illustrated in FIG. 4. The multiplexer 411 sequentially selects the four PDs 104 that form the unit region 131 and outputs each pixel signal to the output wiring 308 provided corresponding to the unit region 131. The multiplexer 411 is formed in the imaging chip 111 together with the PD 104.

  The pixel signal output through the multiplexer 411 is supplied to the signal processing chip 112 by a signal processing circuit 412 that performs correlated double sampling (CDS) / analog / digital (A / D) conversion. D conversion is performed. The A / D converted pixel signal is transferred to the demultiplexer 413 and stored in the pixel memory 414 corresponding to each pixel. The demultiplexer 413 and the pixel memory 414 are formed in the memory chip 113.

The arithmetic circuit 415 formed in the memory chip 113 processes the pixel signal stored in the pixel memory 414 and passes it to the subsequent image processing unit. The arithmetic circuit 415 may be provided in the signal processing chip 112. Note that FIG. 5 shows connections for one unit region 131, but actually these exist for each unit region 131 and operate in parallel. However, the arithmetic circuit 415 does not have to exist for each unit region 131. For example, one arithmetic circuit 415 may perform sequential processing while sequentially referring to the values of the pixel memory 414 corresponding to each unit region 131. Good.
The arithmetic circuit 415 may be configured to include functions of a control unit, an image processing unit, and the like at the subsequent stage.

  As described above, the output wiring 308 is provided corresponding to each of the unit regions 131. Since the image pickup device 100 includes the image pickup chip 111, the signal processing chip 112, and the memory chip 113, each chip is arranged in the surface direction by using the electrical connection between the chips using the connection portion 109 for the output wiring 308. The wiring can be routed without increasing the size.

<Block control of image sensor>
In the present embodiment, an imaging condition can be set for each of a plurality of blocks in the imaging device 32a. The imaging control unit 34c of the control unit 34 causes the plurality of regions to correspond to the block and performs imaging under imaging conditions set for each block. Any number of pixels may be included in the block, for example, 1000 pixels or 1 pixel.

  For example, the camera 1 acquires a live view image by performing photoelectric conversion on the subject image before an imaging instruction is given or while a moving image is being captured. The live view image refers to a monitor image that is repeatedly imaged at a predetermined frame rate (for example, 60 fps). FIG. 6A is a diagram illustrating the arrangement of the first imaging region B1 and the second imaging region B2 set on the imaging surface of the imaging device 32a when a live view image is acquired.

<First imaging region and second imaging region>
According to FIG. 6 (a), the first imaging region B1 is composed of blocks of even rows in odd columns of blocks and blocks of odd rows in even columns of blocks. The second imaging region B2 is composed of even-numbered blocks in even-numbered columns of blocks and odd-numbered blocks in odd-numbered columns of blocks. As described above, the imaging surface of the imaging element 32a is divided into a checkered pattern by a plurality of blocks belonging to the first imaging region B1 and a plurality of blocks belonging to the second imaging region B2.

<Display image and detection image>
FIGS. 7A and 7B are diagrams illustrating a display image 81 and a detection image 82, which will be described in detail later. The control unit 34 generates the display image 81 based on the photoelectric conversion signal read from the first imaging region B1 of the imaging element 32a that has captured one frame. Further, the control unit 34 generates a detection image 82 based on the photoelectric conversion signal read from the second imaging region B2 of the imaging element 32a.
In the following description, the photoelectric conversion signal is referred to as image data or luminance data.

  The control unit 34 uses the display image 81 for live view display, and uses the detection image 82 for subject detection, focus adjustment, and imaging condition setting. In the present embodiment, an image displayed on the display unit 35 as a live view image is referred to as a display image 81. Further, the control unit 34 detects the subject element by the object detection unit 34 a based on the detection image 82, performs the focus detection process by the lens movement control unit 34 d based on the detection image 82, and based on the detection image 82. Then, an exposure calculation process is performed by the setting unit 34b. In the present embodiment, an image used for subject element detection, focus detection, and exposure calculation is referred to as a detection image 82.

<Number of read signals per block>
The control unit 34 includes a first imaging region B1 of the imaging element 32a that reads image data for the display image 81 and a second imaging region B2 of the imaging element 32a that reads image data for the detection image 82. The number of read signals per block can be varied. For example, the density of the image data read per predetermined number of pixels may be different between the first imaging area B1 and the second imaging area B2 of the imaging element 32a.
When the number of pixels constituting the block is 1, the number of readout signals per block is zero or one.

  In general, the display unit 35 of the camera 1 has a lower display resolution than the number of pixels of the image sensor 22. For this reason, the control unit 34 reads image data from a smaller number of pixels than all the pixels included in the first imaging region B1 of the imaging element 32a. For example, the control unit 34 reads out the number of image data corresponding to the display resolution of the display unit 35 from the first imaging region B1 by thinning and reading out every two pixels or every four pixels in the first imaging region B1. read out.

  Note that the control unit 34 does not simply perform thinning-out reading, but performs pixel addition processing in which data of a predetermined number of pixels among the pixels included in the first imaging region B1 are added and handled as one pixel signal. Thus, a smaller number of data than the number of pixels included in the first imaging region B1 may be read out. Pixel addition processing is performed by, for example, the arithmetic circuit 415 provided in the signal processing chip 112 or the memory chip 113.

On the other hand, it is desirable that the detection image 82 used for subject element detection, focus detection, and exposure calculation can refer to information obtained by the image sensor 22 as much as possible. For this purpose, for example, the control unit 34 reads image data from all pixels included in the second imaging region B2 of the imaging element 32a.
Note that when reading from all pixels is not necessary, image data may be read from a smaller number of pixels than all the pixels included in the second imaging region B2.

  As described above, the control unit 34 generates the display image 81 and the detection image 82 based on the image data read from the imaging element 32a that has captured one frame. According to FIGS. 7A and 7B, the display image 81 and the detection image 82 are captured at the same angle of view and include a common subject image. Acquisition of the display image 81 and acquisition of the detection image 82 can be performed in parallel.

<Another example of division of the first imaging area and the second imaging area>
Note that the first imaging area B1 and the second imaging area B2 in the imaging area may be divided as shown in FIG. 6B or FIG. According to the example of FIG. 6B, the first imaging region B1 is configured by even columns of blocks. Further, the second imaging region B2 is configured by an odd number of blocks. Further, according to the example of FIG. 6C, the first imaging region B1 is configured by odd-numbered rows of blocks. Further, the second imaging region B2 is configured by even rows of blocks.

  In both cases of FIG. 6B and FIG. 6C, the control unit 34 displays based on the image data read from the first imaging region B1 of the imaging device 32a that has captured one frame. A work image 81 is generated. Further, the control unit 34 generates the detection image 82 based on the image data read from the second imaging region B2 of the imaging element 32a.

  Note that the display image 81 may be used for focus detection processing or the like for uses other than live view display. Instead of displaying the live view image on the display unit 35, the live view image may be transmitted from the camera 1 to a monitor outside the camera 1 and displayed on the external monitor.

  In the present embodiment, the imaging condition set in the first imaging area B1 for capturing the display image 81 is referred to as the first imaging condition, and the imaging condition set in the second imaging area B2 for capturing the detection image 82 is the first. It will be referred to as two imaging conditions. The control unit 34 may set the first imaging condition and the second imaging condition to the same condition or different conditions.

  As an example, the control unit 34 sets the first imaging condition set in the first imaging region B1 to a condition suitable for the live view display of the display unit 35. At this time, the same first imaging condition is set uniformly throughout the first imaging area B1 of the imaging screen. On the other hand, the control unit 34 sets the second imaging condition set in the second imaging region B2 to a condition suitable for the above-described detection or the like. The condition set as the second imaging condition may be the same second imaging condition for the entire second imaging area B2 of the imaging screen, or different for each area in the second imaging area B2. Two imaging conditions may be set.

  For example, when the conditions suitable for subject element detection, focus detection, and exposure calculation processing are different, the control unit 34 sets the second imaging condition to be set in the second imaging area B2 in the second imaging area B2. For each of these areas, a condition suitable for subject element detection, a condition suitable for focus detection processing, and a condition suitable for exposure calculation processing are set.

  Further, the control unit 34 may change the second imaging condition set in the second imaging region B2 for each frame. For example, the second imaging condition set in the second imaging area B2 in the first frame is set as a condition suitable for detection of the subject element, and the second imaging condition set in the second imaging area B2 in the second frame is used in the focus detection process. The second imaging condition set in the second imaging area B2 in the third frame is set as a condition suitable for the exposure calculation process. In these cases, the same second imaging condition may be set for the entire second imaging area B2 of the imaging screen in each frame, or different second imaging conditions may be set for each area.

<Area ratio between first imaging region and second imaging region>
Furthermore, in FIG. 6A to FIG. 6C, the area ratio between the first imaging region B1 and the second imaging region B2 may be different. For example, the control unit 34 sets the ratio of the first imaging region B1 in the imaging screen to be higher than the ratio of the second imaging region B2 based on the operation by the user or the determination of the control unit 34, or sets the first ratio on the imaging screen. The ratio occupied by the first imaging area B1 and the second imaging area B2 is set to be equal as illustrated in FIGS. 6A to 6C, or the ratio occupied by the first imaging area B1 on the imaging screen is set to the first. 2 is set lower than the ratio occupied by the imaging region B2.

<Occupancy rate of first imaging area and second imaging area>
Moreover, the control part 34 can set the operation rate of the block contained in 1st imaging area B1, and the operation rate of the block contained in 2nd imaging area B2, respectively. For example, when both the operation rate of the blocks included in the first imaging region B1 and the operation rate of the blocks included in the second imaging region B2 are set to 100%, the example is illustrated in FIGS. 6 (a) to 6 (c). Imaging is performed with all blocks included in the first imaging area B1, and imaging is performed with all blocks included in the second imaging area B2.

  On the other hand, for example, when the operating rates of the blocks included in the first imaging area B1 and the second imaging area B2 are set to 50% and 100%, respectively, FIG. 6 (a) to FIG. 6 (c). In addition to performing imaging with half of the blocks included in the first imaging area B1 illustrated in FIG. 5, imaging is performed with all blocks included in the second imaging area B2. When imaging is performed with half of the blocks included in the first imaging area B1, for example, driving every other block is performed to drive half of the blocks and stop driving the remaining half of the blocks.

  FIG. 8A is a diagram illustrating an example in which the same operation rate is set for the blocks included in the first imaging region B1 and the blocks included in the second imaging region B2 in the entire imaging screen. The hatched area 92 indicates that the operating rate of the blocks included in the second imaging area B2 is higher than the operating rate of the blocks included in the first imaging area B1.

  For example, at the time of capturing the first frame after starting the acquisition of the live view image, the control unit 34 is not sure where the subject is present on the imaging screen. Therefore, in order to obtain the detection image 82 used for subject detection, 2 The operation rate of the blocks included in the imaging area B2 is set to 100% over the entire imaging screen. At this time, when the control unit 34 sets the operation rate of the blocks included in the first imaging area B1 to 50% over the entire imaging screen, the operation rate of the blocks included in the second imaging area B2 is The operation rate of the blocks included in the first imaging region B1 is higher. Thereby, relatively more data (signals) from the second imaging region B2 can be obtained over the entire imaging screen than the data (signals) from the first imaging region B1.

  FIG. 8B is a diagram illustrating an example of setting the operating rate of the blocks included in the first imaging area B1 and the operating rate of the blocks included in the second imaging area B2 for each area of the imaging screen. The white area 91 indicates that the operation rate of the blocks included in the first imaging region B1 is higher than the operation rate of the blocks included in the second imaging region B2. A hatched area 92 indicates that the operating rate of the blocks included in the second imaging area B2 is higher than the operating rate of the blocks included in the first imaging area B1.

  For example, when the person T as the main subject has already been detected by the object detection unit 34a at the time of imaging from the second frame after starting the acquisition of the live view image, the control unit 34 includes an area including the person T. The operation rate of the blocks included in the second imaging area B2 of 91 is set to 40%. The reason why the operating rate of the block included in the second imaging area B2 of the area 91 where the person T is detected is lower than that for the first frame is for detection (data) from the second imaging area B2. Based on the idea that the need to obtain the image 82 is low. At this time, when the operation rate of the block included in the first imaging region B1 is set to 50% with respect to the region 91, the control unit 34 operates the block included in the first imaging region B1 of the region 91. The rate is higher than the operating rate of the blocks included in the second imaging region B2 of the region 91. As a result, relatively much data (signal) from the first imaging region B1 can be obtained for the region 91 more than data (signal) from the second imaging region B2.

  In addition, the control unit 34 performs setting for a case where a subject existing outside the imaging screen moves into the imaging screen for the left and right regions 92 of the imaging screen. Specifically, in order to obtain a detection image 82 for accurately detecting a subject moving into the shooting screen in the left and right areas 92 of the imaging screen, the operation rate of the blocks included in the second imaging area B2 of the area 92 Is set to 100%. At this time, if the operation rate of the blocks included in the first imaging area B1 is set to 50% with respect to the left and right areas 92 of the imaging screen, the control unit 34 sets the second imaging area B2 in the area 92. The operation rate of the included block is higher than the operation rate of the block included in the first imaging region B1 of the region 92. Thereby, with respect to the region 92, relatively more data (signal) from the second imaging region B2 can be obtained than data (signal) from the first imaging region B1.

  FIG. 8C illustrates an example in which the main subject moves within the imaging screen. The hatched area 92 indicates that the operating rate of the blocks included in the second imaging area B2 is higher than the operating rate of the blocks included in the first imaging area B1. The white area 91 indicates that the operating rate of the blocks included in the first imaging area B1 is higher than the operating rate of the blocks included in the second imaging area B2.

  For example, when a moving ball T has already been detected by the object detection unit 34a at the time of imaging from the second frame after starting the acquisition of the live view image, the control unit 34 controls the region 92 including the ball T. The operating rate of the blocks included in the second imaging area B2 is set to 100%. The reason why the operation rate of the blocks included in the second imaging area B2 of the area 92 including the moving ball T is not lowered is the detection image that is data (signal) from the second imaging area B2 in order to track the ball T. Based on the idea that there is a high need to obtain 82.

  The control unit 34 further sets a region 92 that is widened in the moving direction of the ball T that moves from the right to the left of the imaging screen (in this example, the left direction of the ball). The reason why the area 92 having a large area is set in the moving direction of the ball T is to make it difficult for the moving ball T to come off the area 92. At this time, when the operation rate of the block included in the first imaging region B1 in the region 92 is set to 50% by the control unit 34, the operation rate of the block included in the second imaging region B2 in the region 92 is set. The operating rate of the blocks included in the first imaging area B1 of the area 92 is higher. Thereby, with respect to the region 92, relatively more data (signal) from the second imaging region B2 can be obtained than data (signal) from the first imaging region B1.

  Further, the control unit 34 is based on the idea that for the background region 91 other than the region 92, the necessity for obtaining the detection image 82 that is data (signal) from the second imaging region B2 is lower than that of the region 92. The operating rate of the blocks included in the second imaging area B2 of the background area 91 is set to 40%. At this time, when the control unit 34 sets the operation rate of the blocks included in the first imaging area B1 to 50% with respect to the background area 91, the blocks included in the first imaging area B1 of the background area 91 Is higher than the operation rate of the blocks included in the second imaging region B2 of the background region 91. As a result, relatively much data (signal) from the first imaging region B1 can be obtained for the region 91 more than data (signal) from the second imaging region B2.

In the above description, in the first frame and the second and subsequent frames, the region 92 in which the operation rate of the blocks included in the second imaging region B2 is higher than the operation rate of the blocks included in the first imaging region B1, The example which changes the setting of the area | region 91 which makes the operation rate of the block contained in 1 imaging region B1 higher than the operation rate of the block contained in 2nd imaging region B2 was demonstrated. The movement of the subject can be detected based on the detection image 82 of the preceding and following frames. Therefore, when the moving subject is detected, the settings of the area 92 and the area 91 may be reviewed for each frame. .
In addition, instead of reviewing the settings of the area 92 and the area 91 for each frame, the settings of the area 92 and the area 91 are made with the passage of time since the start of capturing the live view image (moving image). You may make it review.

  FIG. 8D is a diagram illustrating an example in which the main subject moves in the depth direction (for example, the automobile T approaches the camera 1). The hatched area 92 indicates that the operating rate of the blocks included in the second imaging area B2 is higher than the operating rate of the blocks included in the first imaging area B1. The white area 91 indicates that the operating rate of the blocks included in the first imaging area B1 is higher than the operating rate of the blocks included in the second imaging area B2.

For example, the control unit 34 detects the distance from the camera 1 to the automobile T by using the focus adjustment result by the lens movement control unit 34d. In general, the distance from the camera 1 to the car T corresponds to the position of the focus lens after the focus adjustment. Therefore, if the position of the focus lens in focus on the car T is determined, the camera 1 to the car T corresponding to the position is determined. Can be obtained. That is, distance information (which may be a lookup table or a function) based on the design data of the imaging optical system 31 is prepared for each position of the focus lens and stored in advance in a nonvolatile memory (not shown) in the control unit 34. Thus, the distance to the automobile T corresponding to the position of the focus lens can be obtained.
In the case of a zoom lens having a variable focal length, distance information may be prepared for each focal length and stored in the nonvolatile memory of the control unit 34.

  For example, the control unit 34 calculates the difference between the distances of the automobile T (distance from the camera 1 to the automobile T) obtained by the lens movement control unit 34d based on the detection images 82 of different frames between the frames of the live view image. The moving speed of the automobile T is calculated by dividing by the difference of the imaging start times. The control unit 34 sets the operation rate of the blocks included in the second imaging region B2 of the region 92 to 100% in order to obtain the detection image 82 for accurately detecting the moving vehicle T in the region 92 including the vehicle T. To do.

  The control unit 34 further sets the area of the region 92 including the automobile T wider as the moving speed of the automobile T approaching the camera 1 is higher. The reason why the controller 34 sets the area 92 having a larger area as the moving speed of the automobile T is faster is to make it difficult for the approaching automobile T to protrude from the area 92. At this time, when the operation rate of the block included in the first imaging region B1 in the region 92 is set to 50% by the control unit 34, the operation rate of the block included in the second imaging region B2 in the region 92 is set. The operating rate of the blocks included in the first imaging area B1 of the area 92 is higher. Thereby, with respect to the region 92, relatively more data (signal) from the second imaging region B2 can be obtained than data (signal) from the first imaging region B1.

  Further, the control unit 34 is based on the idea that for the background region 91 other than the region 92, the necessity for obtaining the detection image 82 that is data (signal) from the second imaging region B2 is lower than that of the region 92. The operating rate of the blocks included in the second imaging area B2 of the background area 91 is set to 40%. At this time, when the control unit 34 sets the operation rate of the blocks included in the first imaging area B1 to 50% with respect to the background area 91, the blocks included in the first imaging area B1 of the background area 91 Is higher than the operation rate of the blocks included in the second imaging region B2 of the background region 91. As a result, relatively much data (signal) from the first imaging region B1 can be obtained for the region 91 more than data (signal) from the second imaging region B2.

<Example of subject>
FIG. 9 is a diagram illustrating a subject area on the imaging screen of the camera 1. In FIG. 9, the subject area includes a person 61, a car 62, a bag 63, a mountain 64, a cloud 65, and a cloud 66. The person 61 holds the bag 63 with both hands. The automobile 62 stops at the right rear side of the person 61. When the first imaging area B1 and the second imaging area B2 are determined as shown in FIG. 6A, FIG. 6B, or FIG. 6C, the image data read from the first imaging area B1 is used as shown in FIG. ) As shown in FIG. Further, a detection image 82 as shown in FIG. 7B is obtained from the image data read from the second imaging region B2.

As an example of setting different imaging conditions for the blocks of the image sensor 32a in correspondence with the subject, the imaging conditions can be set for each block corresponding to each subject area. For example, a block included in the area of the person 61, a block included in the car 62, a block included in the bag 63, a block included in the mountain 64, a block included in the cloud 65, and a block included in the cloud 66. The imaging conditions are set for each. In order to appropriately set the imaging conditions for each of these blocks, it is necessary to capture the above-described detection image 82 under appropriate exposure conditions, and to obtain an image without overexposure or blackout. . Overexposure means that the gradation of data in a high-luminance portion of an image is lost due to overexposure. Further, blackout means that the gradation of data in the low-luminance portion of the image is lost due to underexposure.
This is because in the detection image 82 in which whiteout or blackout has occurred, for example, the outline (edge) of each subject area does not appear, and it is difficult to detect the subject element based on the detection image 82.

<Imaging conditions set in the second imaging area>
Therefore, in the present embodiment, subject detection processing based on the detection image 82 is performed by appropriately setting the second imaging condition for acquiring the detection image 82 for the second imaging region B2 of the imaging element 32a. The focus detection process and the exposure calculation process can be appropriately performed. In the present embodiment, the second imaging condition is set as follows.

  If it is assumed that the detection image 82 is a dark image, it is difficult to extract edges, color differences, and the like in the imaging screen, and it is difficult to detect the subject. On the other hand, if the detection image 82 is obtained with appropriate brightness without overexposure or underexposure, it becomes easier to extract edges and color differences in the screen, and the accuracy of subject detection and subject recognition is improved. Can be expected. For example, the control unit 34 makes the second gain as the second imaging condition for obtaining the detection image 82 higher than the first gain as the first imaging condition for obtaining the display image 81. Since the second gain, which is higher than the first gain of the display image 81, is set for the detection image 82, the object detection unit 34a can detect the bright detection image 82 even when the display image 81 is a dark image. It is possible to accurately detect the subject based on the above. If the subject detection can be accurately performed, the image can be appropriately divided for each subject.

  In addition, although the example mentioned above is a case where the display image 81 for live view display is imaged, the case where a moving image is imaged while displaying live view is also included. That is, it is a concept including a case where an image displayed in live view is recorded as a moving image.

<Detection of subject element based on detection image>
For example, when the main switch of the camera 1 is turned on, the control unit 34 sets the first imaging area B1 and the second imaging area B2 in the imaging element 32a. As described above, the first imaging region B1 is a region for capturing the display image 81 for live view display. The second imaging region B2 is a region where a detection image 82 for detection is captured.

  Next, the control unit 34 sets a first gain for live view display in the first imaging region B1 of the imaging device 32a, and the second gain higher than the first gain in the second imaging region B2 of the imaging device 32a. Set the gain. The control unit 34 that has performed the gain setting performs imaging for the detection image 82 in the second imaging region B2 of the imaging device 32a, and performs detection based on the image data read from the second imaging region B2 of the imaging device 32a. An image 82 is generated. The control unit 34 causes the object detection unit 34 a to detect the subject element based on the detection image 82. The setting unit 34b divides the image (for example, the display image 81) acquired by the imaging unit 32 based on the subject element detected by the object detection unit 34a.

<Setting imaging conditions for each area>
Referring to FIG. 9, the image acquired by the imaging unit 32 includes, for example, a person 61 area, an automobile 62 area, a bag 63 area, a mountain 64 area, and a cloud 65 area. The area is divided into a cloud 66 area and other areas. When the subject detection process is performed by the object detection unit 34a and the image is automatically divided by the setting unit 34b, the control unit 34 causes the display unit 35 to display a setting screen illustrated in FIG. In FIG. 10, a live view image 60a based on the display image 81 is displayed on the display unit 35, and an imaging condition setting screen 70 is displayed on the right side of the live view image 60a.

  The setting screen 70 lists frame rate, shutter speed (TV), and gain (ISO sensitivity) in order from the top as an example of setting items for imaging conditions. The frame rate is the number of frames of a live view image acquired per second or a moving image recorded by the camera 1. The shutter speed corresponds to the exposure time. The gain corresponds to the ISO sensitivity. The setting items for the imaging conditions may be added as appropriate in addition to those illustrated in FIG. When all the setting items do not fit in the setting screen 70, other setting items may be displayed by scrolling the setting items up and down.

  In the present embodiment, the control unit 34 can set an area selected by a user operation among the areas divided by the setting unit 34b as a target for setting (changing) imaging conditions. For example, in the camera 1 that can be touched, the user touches the display position of the subject for which the imaging condition is to be set (changed) on the display surface of the display unit 35 on which the live view image 60a is displayed. For example, when the display position of the person 61 is touched, the control unit 34 sets an area corresponding to the person 61 in the live view image 60 a as an imaging condition setting (change) target area and an area corresponding to the person 61. The outline is highlighted.

In FIG. 10, an area to be displayed with an emphasized outline represents an area that is a target for setting (changing) imaging conditions. The highlighted display is, for example, a thick display, a bright display, a display with a different color, a broken line display, a blinking display, or the like. In the example of FIG. 10, it is assumed that a live view image 60 a in which the outline of the area corresponding to the person 61 is emphasized is displayed. In this case, the highlighted area is a target for setting (changing) the imaging condition. For example, in the camera 1 capable of touch operation, when the user performs a touch operation on the shutter speed (TV) display 71, the control unit 34 displays the current shutter speed for the highlighted area (person 61). The set value is displayed on the screen (reference numeral 68).
In the following description, the camera 1 is described on the premise of a touch operation. However, the imaging condition may be set (changed) by operating a button or the like constituting the operation member 36.

  When the upper icon 71a or the lower icon 71b of the shutter speed (TV) is touched by the user, the setting unit 34b increases or decreases the shutter speed display 68 from the current setting value according to the touch operation. An instruction is sent to the imaging unit 32 (FIG. 1) so as to change the imaging condition of the unit area 131 (FIG. 3) of the imaging element 32a corresponding to the displayed area (person 61) in accordance with the touch operation. The decision icon 72 is an operation icon for confirming the set imaging condition. The setting unit 34b performs the setting (change) of the frame rate and gain (ISO) in the same manner as the setting (change) of the shutter speed (TV).

Although the setting unit 34b has been described as setting the imaging condition based on the user's operation, the setting unit 34b is not limited to this. The setting unit 34b may set the imaging conditions based on the determination of the control unit 34 without being based on a user operation. For example, when an overexposure or underexposure occurs in an area including a subject having the maximum luminance or the minimum luminance in the image, the setting unit 34b cancels the overexposure or underexposure based on the determination of the control unit 34. Imaging conditions may be set.
For the area that is not highlighted (area other than the person 61), the set imaging conditions are maintained.

  Instead of highlighting the outline of the area for which the imaging condition is set (changed), the control unit 34 displays the entire target area brightly, increases the contrast of the entire target area, or displays the entire target area. May be displayed blinking. Further, the target area may be surrounded by a frame. The display of the frame surrounding the target area may be a double frame or a single frame, and the display mode such as the line type, color, and brightness of the surrounding frame may be appropriately changed. In addition, the control unit 34 may display an indication of an area for which an imaging condition is set, such as an arrow, in the vicinity of the target area. The control unit 34 may darkly display a region other than the target region for which the imaging condition is set (changed), or may display a low contrast other than the target region.

When the release button 36B constituting the operation member 36 or a display for instructing the start of imaging (for example, the release icon 74 in FIG. 10) is operated, the control unit 34 controls the imaging unit 32 to control the first of the imaging element 32a. The settings of the first imaging area B1 and the second imaging area B2 are canceled, and, for example, imaging is performed using all the pixels in the pixel area on the imaging surface (main imaging). As the imaging conditions of the main imaging, different imaging conditions can be applied to each of the divided areas (person 61, car 62, bag 63, mountain 64, cloud 65, cloud 66). The image processing unit 33 performs image processing on the image data acquired by the imaging unit 32. Image processing can also be performed under different image processing conditions for each of the divided areas.
Note that when the release button 36B or the release icon 74 is operated, imaging other than main imaging may be performed in which imaging is performed using all the pixels in the pixel area of the imaging surface. For example, imaging may be performed only with the first imaging area B1 of the imaging element 32a, or imaging may be performed with only the second imaging area B2 of the imaging element 32a. Further, as described above, a moving image may be captured in the first imaging region B1.

  After the image processing by the image processing unit 33, the recording unit 37 that receives an instruction from the control unit 34 records the image data after the image processing on a recording medium including a memory card (not shown). Thereby, a series of imaging processes is completed.

<Example of image processing>
An example of image processing performed by the image processing unit 33 (generation unit 33c) will be described. For example, the generation unit 33c uses a kernel having a predetermined size centered on the pixel of interest P (processing target pixel) in the image data. FIG. 11 is a diagram illustrating a kernel. In FIG. 11, pixels around the target pixel P (eight pixels in this example) included in the target region 90 (for example, 3 × 3 pixels) centered on the target pixel P are set as reference pixels Pr1 to Pr8. The position of the target pixel P is the target position, and the positions of the reference pixels Pr1 to Pr8 surrounding the target pixel P are reference positions.

(1) Pixel Defect Correction Process The pixel defect correction process is one of image processes performed on a captured image. In general, the image pickup element 32a, which is a solid-state image pickup element, may produce pixel defects in the manufacturing process or after manufacturing, and output abnormal level image data. Therefore, the generation unit 33c of the image processing unit 33 corrects the image data output from the pixel in which the pixel defect has occurred, thereby making the image data in the pixel position in which the pixel defect has occurred inconspicuous.

  The generation unit 33c of the image processing unit 33 uses, for example, a pixel at the position of a pixel defect recorded in advance in a non-illustrated nonvolatile memory in an image of one frame as a target pixel P (processing target pixel), Pixels around the pixel of interest P included in the central region of interest 90 are referred to as reference pixels Pr1 to Pr8.

  The generation unit 33c of the image processing unit 33 calculates the maximum value and the minimum value of the image data in the reference pixels Pr1 to Pr8. When the image data output from the target pixel P exceeds these maximum value or minimum value, the target pixel Max and Min filter processing for replacing the image data output from P with the maximum value or the minimum value is performed. Such a process is performed for all pixel defects whose position information is recorded in a non-volatile memory (not shown).

  In the present embodiment, the generation unit 33 c of the image processing unit 33 can perform the above-described Max and Min filter processing on the image data of the detection image 82. Thereby, it is possible to prevent the influence of the pixel defect in the subject detection process, the focus detection process, and the exposure calculation process based on the detection image 82. The generation unit 33 c of the image processing unit 33 can perform the above-described Max and Min filter processing on the image data of the display image 81. Thereby, it is possible to prevent the influence of pixel defects on the live view image based on the display image 81.

(2) Color Interpolation Processing Color interpolation processing is one of image processing performed on a captured image. As illustrated in FIG. 3, in the imaging chip 111 of the imaging device 100, green pixels Gb and Gr, a blue pixel B, and a red pixel R are arranged in a Bayer array. Since the generation unit 33c of the image processing unit 33 lacks image data of a color component different from the color component of the color filter F arranged at each pixel position, by performing a known color interpolation process, The image data of the missing color component is generated with reference to the image data.

  In the present embodiment, the generation unit 33 c of the image processing unit 33 can perform color interpolation processing on the image data of the detection image 82. This makes it possible to appropriately perform subject detection processing, focus detection processing, and exposure calculation processing based on the color-interpolated detection image 82. Further, the generation unit 33 c of the image processing unit 33 can perform the above-described color interpolation processing on the image data of the display image 81. Accordingly, the live view image can be appropriately displayed in color based on the display image 81 subjected to color interpolation.

(3) Outline Enhancement Process The outline enhancement process is one of image processes performed on a captured image. The generation unit 33c of the image processing unit 33 performs, for example, a known linear filter calculation using a kernel of a predetermined size centered on the pixel of interest P (processing target pixel) in an image of one frame. When the kernel size of the sharpening filter which is an example of the linear filter is N × N pixels, the position of the target pixel P is the target position, and the positions of (N ² −1) reference pixels Pr surrounding the target pixel P are Reference position.
The kernel size may be N × M pixels.

  The generation unit 33c of the image processing unit 33 performs a filter process for replacing the image data in the target pixel P with a linear filter calculation result on each horizontal line, for example, from the upper horizontal line to the lower horizontal line of the frame image. This is done while shifting the pixels from left to right.

In the present embodiment, the generation unit 33 c of the image processing unit 33 can perform the above-described linear filter processing on the image data of the detection image 82. Accordingly, the subject detection process, the focus detection process, and the exposure calculation process can be appropriately performed based on the detection image 82 in which the contour is emphasized. Further, the generation unit 33 c of the image processing unit 33 can perform the above-described linear filter processing on the image data of the display image 81. Accordingly, the live view image can be appropriately displayed based on the display image 81 in which the contour is emphasized.
Note that the strength of contour enhancement may be different between the case of performing the image data of the detection image 82 and the case of performing the image data of the display image 81.

(4) Noise reduction processing Noise reduction processing is one type of image processing performed on a captured image. The generation unit 33c of the image processing unit 33 performs, for example, a known linear filter calculation using a kernel of a predetermined size centered on the pixel of interest P (processing target pixel) in an image of one frame. When the kernel size of the smoothing filter as an example of the linear filter is N × N pixels, the position of the target pixel P is the target position, and the positions of the (N ² −1) reference pixels Pr surrounding the target pixel P are Reference position.
The kernel size may be N × M pixels.

  The generation unit 33c of the image processing unit 33 performs a filter process for replacing the image data in the target pixel P with a linear filter calculation result on each horizontal line, for example, from the upper horizontal line to the lower horizontal line of the frame image. This is done while shifting the pixels from left to right.

In the present embodiment, the generation unit 33 c of the image processing unit 33 can perform the above-described linear filter processing on the image data of the detection image 82. Thereby, it is possible to prevent the influence of noise in the subject detection process, the focus detection process, and the exposure calculation process based on the detection image 82. Further, the generation unit 33 c of the image processing unit 33 can perform the above-described linear filter processing on the image data of the display image 81. Thereby, it is possible to prevent the influence of noise on the live view image based on the display image 81.
Note that the degree of noise reduction may be different between the case of performing the image data of the detection image 82 and the case of performing the image data of the display image 81.

<Example of focus detection processing>
An example of focus detection processing performed by the control unit 34 (lens movement control unit 34d) will be described. The lens movement control unit 34d of the control unit 34 performs focus detection processing using signal data (image data) corresponding to a predetermined position (focus point) on the imaging screen. In the AF operation of the present embodiment, for example, the subject corresponding to the focus point selected by the user from a plurality of focus points on the imaging screen is focused. The lens movement control unit 34 d (generation unit) of the control unit 34 detects image shift amounts (phase differences) of a plurality of subject images due to light beams that have passed through different pupil regions of the imaging optical system 31, whereby the imaging optical system 31. The defocus amount of is calculated. The lens movement control unit 34d of the control unit 34 adjusts the focus of the imaging optical system 31 by moving the focus lens of the imaging optical system 31 to a position where the defocus amount is zero (allowable value or less), that is, a focus position. .

  FIG. 12 is a diagram illustrating the position of the focus detection pixel on the imaging surface of the imaging element 32a. In the present embodiment, focus detection pixels are discretely arranged along the X-axis direction (horizontal direction) of the imaging chip 111. In the example of FIG. 12, 15 focus detection pixel lines 160 are provided at predetermined intervals. The focus detection pixels constituting the focus detection pixel line 160 output image data for focus detection. In the imaging chip 111, normal imaging pixels are provided at pixel positions other than the focus detection pixel line 160. The imaging pixels output a display image 81, a detection image 82, and image data at the time of actual imaging.

  FIG. 13 is an enlarged view of a part of the focus detection pixel line 160 corresponding to the focus point 80A shown in FIG. In FIG. 13, a red pixel R, a green pixel G (Gb, Gr), and a blue pixel B, a focus detection pixel S1, and a focus detection pixel S2 are illustrated. The red pixel R, the green pixel G (Gb, Gr), and the blue pixel B are arranged according to the rules of the Bayer arrangement described above.

The square area illustrated for the red pixel R, the green pixel G (Gb, Gr), and the blue pixel B indicates a light receiving area of the imaging pixel. Each imaging pixel receives a light beam passing through the exit pupil of the imaging optical system 31 (FIG. 1). That is, the red pixel R, the green pixel G (Gb, Gr), and the blue pixel B each have a square-shaped mask opening, and light passing through these mask openings reaches the light-receiving portion of the imaging pixel. .
In addition, the shape of the light receiving region (mask opening) of the red pixel R, the green pixel G (Gb, Gr), and the blue pixel B is not limited to a quadrangle, and may be, for example, a circle.

  The semicircular region exemplified for the focus detection pixel S1 and the focus detection pixel S2 indicates a light receiving region of the focus detection pixel. That is, the focus detection pixel S1 has a semicircular mask opening on the left side of the pixel position in FIG. 13, and the light passing through the mask opening reaches the light receiving portion of the focus detection pixel S1. On the other hand, the focus detection pixel S2 has a semicircular mask opening on the right side of the pixel position in FIG. 13, and light passing through the mask opening reaches the light receiving portion of the focus detection pixel S2. As described above, the focus detection pixel S1 and the focus detection pixel S2 respectively receive a pair of light beams passing through different areas of the exit pupil of the imaging optical system 31 (FIG. 1).

  Note that the position of the focus detection pixel line 160 in the imaging chip 111 is not limited to the position illustrated in FIG. Also, the number of focus detection pixel lines 160 is not limited to the example of FIG. Further, the shape of the mask opening in the focus detection pixel S1 and the focus detection pixel S2 is not limited to a semicircular shape. For example, a rectangular light receiving region (mask opening) in the imaging pixel R, the imaging pixel G, and the imaging pixel B is used. Part) may be a rectangular shape divided in the horizontal direction.

  Further, the focus detection pixel line 160 in the imaging chip 111 may be a line in which focus detection pixels are arranged along the Y-axis direction (vertical direction) of the imaging chip 111. An imaging element in which imaging pixels and focus detection pixels are two-dimensionally arranged as shown in FIG. 13 is known, and detailed illustration and description of these pixels are omitted.

  In the example of FIG. 13, the configuration in which the focus detection pixels S <b> 1 and S <b> 2 each receive one of a pair of focus detection light beams, the so-called 1PD structure, has been described. Instead of this, the focus detection pixels may be configured to receive both of a pair of light beams for focus detection, that is, a so-called 2PD structure. With the 2PD structure, the photoelectric conversion signal obtained by the focus detection pixels can be used as the display image 81, the detection image 82, and the image data at the time of actual imaging.

  The lens movement control unit 34d of the control unit 34 passes through different regions of the imaging optical system 31 (FIG. 1) based on the focus detection photoelectric conversion signals output from the focus detection pixel S1 and the focus detection pixel S2. An image shift amount (phase difference) between the pair of images by the pair of light beams is detected. Then, the defocus amount is calculated based on the image shift amount (phase difference). Such defocus amount calculation by the pupil division phase difference method is well known in the field of cameras, and thus detailed description thereof is omitted.

  It is assumed that the focus point 80A (FIG. 12) is selected by the user in the live view image 60a illustrated in FIG. FIG. 14 is an enlarged view of the focus point 80A. In FIG. 14, the position surrounded by the frame 170 corresponds to the focus detection pixel line 160 (FIG. 12).

  In the present embodiment, the lens movement control unit 34d of the control unit 34 can perform focus detection processing using signal data from the focus detection pixels indicated by the frame 170 in the detection image 82. For example, the focus detection processing is appropriately performed by using the signal data of the focus detection pixels of the detection image 82 in which the gain is set high or the image processing suitable for detection of the image shift amount (phase difference) is performed. Can be done.

  In the above description, the focus detection process using the pupil division phase difference method is exemplified. However, in the case of the contrast detection method in which the focus lens of the imaging optical system 31 is moved to the in-focus position based on the contrast of the subject image. Can be done as follows.

When the contrast detection method is used, the control unit 34 moves the focus lens of the imaging optical system 31 and outputs signal data output from the imaging pixels of the imaging element 32a corresponding to the focus point at each position of the focus lens. Based on this, a known focus evaluation value calculation is performed. Then, the position of the focus lens that maximizes the focus evaluation value is obtained as the focus position.
That is, the lens movement control unit 34d of the control unit 34 performs a focus evaluation value calculation using signal data from the imaging pixels corresponding to the focus point 80A in the detection image 82. Thereby, for example, the focus detection process is appropriately performed by using the signal data of the detection image 82 in which the gain is set high or the image process suitable for the detection of the image shift amount (phase difference) is performed. be able to.

<Subject detection processing>
FIG. 15A is a diagram illustrating a template image representing an object to be detected, and FIG. 15B is a diagram illustrating a live view image 60 (a) and a search range 190. The object detection unit 34a of the control unit 34 detects an object (for example, the bag 63 which is one of the subject elements in FIG. 9) from the live view image. The object detection unit 34a of the control unit 34 may set the range in which the object is detected as the entire range of the live view image 60a. However, in order to reduce the detection process, a part of the live view image 60a may be used as the search range 190. Good.

  A case where the bag 63 that is the belonging of the person 61 is detected in the live view image 60a illustrated in FIG. 10 will be described. The object detection unit 34 a of the control unit 34 sets the search range 190 in the vicinity of the region including the person 61. Note that the region 61 including the person 61 may be set as the search range.

  In the present embodiment, the object detection unit 34a of the control unit 34 can perform subject detection processing using image data constituting the search range 190 in the detection image 82. For example, the subject detection process can be appropriately performed by using the image data of the detection image 82 in which the gain is set high or the image process suitable for the detection of the subject element is performed.

<Processing for setting imaging conditions>
For example, when the setting unit 34b of the control unit 34 divides the area of the imaging screen and sets different imaging conditions between the divided areas and newly determines the exposure condition by performing re-photometry, Exposure calculation processing is performed using image data constituting the photometric range. The setting unit 34b sets the imaging condition based on the exposure calculation result as follows. For example, when an overexposure or underexposure occurs in an area including a subject having the maximum luminance or the minimum luminance in the image, the setting unit 34b sets an imaging condition so as to eliminate overexposure or underexposure.

  In the present embodiment, the setting unit 34b of the control unit 34 can perform an exposure calculation process using image data constituting the photometric range in the detection image 82. For example, by using the image data of the detection image 82 set with a low gain, the arithmetic processing can be appropriately performed.

  Not only the photometric range when performing the exposure calculation process described above, but also the photometric (colorimetric) range used when determining the white balance adjustment value and the necessity of emission of the auxiliary photographing light by the light source that emits the auxiliary photographing light are determined. The same applies to the photometric range performed at the time, and further to the photometric range performed at the time of determining the light emission amount of the photographing auxiliary light by the light source.

<Description of flowchart>
FIG. 16 is a flowchart for explaining the flow of processing of the camera 1 that captures an image by setting an imaging condition for each region. For example, when the main switch of the camera 1 is turned on, the control unit 34 activates a program that executes the process shown in FIG. In step S10, the control unit 34 sets the first imaging area B1 and the second imaging area B2 in the imaging element 32a, and proceeds to step S20. As described above, the first imaging region B1 is a region for capturing the display image 81, and the second imaging region B2 is a region for capturing the detection image 82.

  In step S20, the control unit 34 causes the display unit 35 to start live view display, and causes the object detection unit 34a to start processing for detecting a subject based on the detection image 82 captured in the second imaging region B2. Then, the process proceeds to step S30. Further, as shown in FIG. 8A, the control unit 34 sets the operation rate of the blocks included in the second imaging region B2 over the entire imaging screen as the initial setting, and the operation of the blocks included in the first imaging region B1. Set higher than rate. In addition, the same imaging condition is set for the entire imaging screen for each of the first imaging area B1 and the second imaging area B2.

With the start of live view display, live view images based on the display image 81 imaged in the first imaging area B1 are sequentially displayed on the display unit 35. When the setting for performing the AF operation is performed during live view display, the lens movement control unit 34d of the control unit 34 performs focus detection processing based on the detection image 82 captured in the second imaging region B2. Thus, the AF operation for focusing on the subject element corresponding to the predetermined focus point is controlled.
If the setting for performing the AF operation is not performed during live view display, the lens movement control unit 34d of the control unit 34 performs the AF operation when the AF operation is instructed later.

  In step S30, the setting unit 34b of the control unit 34 divides the imaging screen by the imaging element 32a into a plurality of regions including the subject element, and blocks included in the first imaging region B1 and the second imaging region B2, respectively. The operating rate is set and the process proceeds to step S40. The operating rate of the block may be set higher than the operating rate of the block included in the first imaging region B1 according to the position and movement of the subject on the imaging screen (FIG. 8). The operation rate of the blocks included in the first imaging area B1 is set to be higher than the operation rate of the blocks included in the second imaging area B2 (corresponding to the white background in FIG. 8).

For example, in the case of a stationary object (the position of the object does not change between frames), as shown in FIG. 8B, the area 91 including the person T is included in the first imaging area B1. The operation rate of the blocks to be set is set higher than the operation rate of the blocks included in the second imaging region B2.
In the case of a moving subject (when the position and size of the subject are different between frames), as shown in FIG. 8C and FIG. On the other hand, the operating rate of the blocks included in the second imaging area B2 is set higher than the operating rate of the blocks included in the first imaging area B1.

In step S <b> 40, the control unit 34 displays an area on the display unit 35. As illustrated in FIG. 10, the control unit 34 highlights a region that is a target for setting (changing) imaging conditions in the live view image 60 a. The control unit 34 determines a region to be highlighted based on the touch operation position by the user. In addition, the control unit 34 displays the imaging condition setting screen 70 on the display unit 35 and proceeds to step S50.
Note that when the display position of another subject on the display screen is touched with the user's finger, the control unit 34 changes the region including the subject to a region for which the imaging condition is set (changed). To highlight it.

  In step S50, the control unit 34 determines whether an AF operation is necessary. The control unit 34, for example, when the focus adjustment state changes due to the movement of the subject, when the position of the focus point is changed by a user operation, or when execution of an AF operation is instructed by a user operation, An affirmative decision is made in step S50 and the process proceeds to step S70. If the focus adjustment state does not change, the position of the focus point is not changed by the user operation, and the execution of the AF operation is not instructed by the user operation, the control unit 34 makes a negative determination in step S50 and proceeds to step 60. .

  In step S70, the controller 34 performs an AF operation and returns to step S40. The lens movement control unit 34d of the control unit 34 performs an AF operation for focusing on a subject corresponding to a predetermined focus point by performing a focus detection process based on the detection image 82 imaged in the second imaging region B2. Control. In addition, the control unit 34 that has returned to step S40 performs the above-described processing based on the display image 81 captured in the first imaging region B1 and the detection image 82 captured in the second imaging region B2 after the AF operation. Repeat the same process.

In step S60, the setting unit 34b of the control unit 34 sets (changes) the imaging condition for the highlighted area in response to a user operation, and the process proceeds to step S80. For example, the setting unit 34b sets (changes) an imaging condition for the first imaging region B1. Further, the setting unit 34b may set (change) the imaging condition for the second imaging region B2.
When the user operation for setting (changing) the imaging condition is not performed, the setting unit 34b sets the same imaging condition in the entire imaging screen for each of the first imaging area B1 and the second imaging area B2 ( Do not set (change) each area).
The setting unit 34b performs exposure calculation based on the detection image 82 imaged in the second imaging region B2 when a new metering is performed again with the imaging condition set (changed) to determine the imaging condition for the main imaging. Process.

  In step S80, the control unit 34 determines the presence / absence of an imaging instruction. When a release button (not shown) constituting the operation member 36 or a display icon for instructing imaging is operated, the control unit 34 makes a positive determination in step S80 and proceeds to step S90. When the imaging instruction is not performed, the control unit 34 makes a negative determination in step S80 and returns to step S50.

  In step S90, the control unit 34 performs predetermined imaging processing. That is, the imaging control unit 34c controls the imaging element 32a so as to perform the actual imaging under the imaging conditions set (changed) for each area, and the process proceeds to step S100. In the main imaging, the setting of the first imaging area B1 and the second imaging area B2 of the imaging element 32a can be canceled and all the blocks (pixels) of the imaging element 32a can be used.

  In step S100, the imaging control unit 34c of the control unit 34 sends an instruction to the image processing unit 33, performs predetermined image processing on the image data obtained by the imaging, and proceeds to step S110. Image processing includes the pixel defect correction processing, color interpolation processing, contour enhancement processing, and noise reduction processing.

  In step S110, the control unit 34 sends an instruction to the recording unit 37, records the image data after image processing on a recording medium (not shown), and proceeds to step S120.

  In step S120, the control unit 34 determines whether an end operation has been performed. When the end operation is performed, the control unit 34 makes a positive determination in step S120 and ends the process of FIG. When the end operation is not performed, the control unit 34 makes a negative determination in step S120 and returns to step S20. When returning to step S20, the control unit 34 repeats the above-described processing.

  In the above description, the multilayer image sensor 100 is illustrated as the image sensor 32a. However, if the imaging condition can be set for each of a plurality of blocks in the image sensor (imaging chip 111), the image sensor 32a is not necessarily configured as a multilayer image sensor. do not have to.

According to the embodiment described above, the following operational effects can be obtained.
(1) The image pickup device 32a mounted on the camera 1 picks up a subject, picks up the subject, and picks up the subject. And a second imaging area B2 that generates a detection image 82 (also referred to as luminance data) for obtaining information on the data, and the number of data read from the first imaging area B1 and the number of data read from the second imaging area B2 And made it different. Thereby, the image sensor 32a can generate luminance data for obtaining information about the subject while generating image data for display. In addition, for example, by increasing the number of read data from the second imaging area B2 of the imaging element 32a, information about the subject can be increased, so that the accuracy of subject detection can be increased.

(2) Since the image sensor 32a of (1) above reads out the image data generated by each of the plurality of pixels in the first imaging area B1, the number of data read from the second imaging area B2 is the first imaging area. More than the number of data read from B1. As a result, the accuracy of subject detection can be improved as compared to the case where subject detection is performed based on the image data from the first imaging region B1.

(3) In the imaging element 32a of (2), the luminance data generated by each of the plurality of pixels in the second imaging area B2 is read out, and the image data generated in the first imaging area B1 is More data than the luminance data generated in the two imaging areas B2 are thinned out. As a result, the number of data read from the second imaging area B2 is larger than the number of data read from the first imaging area B1. As a result, the accuracy of subject detection can be improved as compared to the case where subject detection is performed based on the image data from the first imaging region B1.

(4) Since the imaging element 32a of (1) reads out data obtained by adding image data generated by each of the plurality of pixels in the first imaging area B1, the number of data read from the second imaging area B2 is the first. More than the number of data read from the imaging area B1. As a result, the accuracy of subject detection can be improved as compared to the case where subject detection is performed based on the image data from the first imaging region B1.

(5) The image sensor 32a of (4) above reads out data obtained by adding the luminance data generated by each of the plurality of pixels in the second imaging area B2, and the image data generated in the first imaging area B1 is A larger amount of data than the luminance data generated in the second imaging region B2 is added. As a result, the number of data read from the second imaging area B2 is larger than the number of data read from the first imaging area B1. As a result, the accuracy of subject detection can be improved as compared to the case where subject detection is performed based on the image data from the first imaging region B1.

(6) The imaging element 32a of (1) to (5) above images the same subject in the first imaging area B1 and the second imaging area B2, and the first imaging area for each of the areas 91 and 92 on the imaging screen. The ratio between the number of pixels (block) that generates image data in B1 and the number of pixels (block) that generates luminance data in the second imaging region B2 is set. Thereby, for example, power consumption and heat generation can be suppressed as compared with a case where the blocks included in the first imaging area B1 and the second imaging area B2 are operated 100% in both the area 91 and the area 92.

(7) Since the area 91 of the imaging screen in (6) is set based on the position of the person T as the subject (FIG. 8B), the area 91 where the subject has already been detected and the imaging In an area 92 where it is desired to detect a subject moving from the outside of the screen into the imaging screen, the number of pixels (blocks) for generating image data in the first imaging area B1 and the pixels for generating luminance data in the second imaging area B2. The ratio with the number (block) can be set appropriately.

(8) Since the setting of the area 91 of the imaging screen (6) or (7) is changed by the movement of the ball T as a subject (FIG. 8 (c)), the area is adjusted according to the movement of the ball T. Thus, the ratio between the number of pixels (block) that generates image data in the first imaging region B1 and the number of pixels (block) that generates luminance data in the second imaging region B2 can be changed appropriately.

(9) Since the setting of the area 91 of the imaging screen (6) to (8) is changed as time passes, image data is generated in the first imaging area B1 as time elapses, for example. It is possible to appropriately change the ratio between the number of pixels (block) to be generated and the number of pixels (block) to generate luminance data in the second imaging region B2.

(10) Since the imaging element 32a of (1) to (9) above has different imaging conditions for the first imaging area B1 and the second imaging area B2, for example, the first imaging area B1 and the second imaging area By applying different gains to the region B2, it is possible to obtain image data suitable for display and luminance data suitable for detection of information related to the subject.

(11) In the first imaging area B1 of the imaging element 32a of (10) above, the setting of imaging conditions is changed based on the luminance data read from the second imaging area B2. Thereby, while outputting the image data of the display image 81 from the first imaging area B1 of the imaging element 32a, the imaging of the first imaging area B1 is performed based on the luminance data output from the second imaging area B2 of the imaging element 32a. A condition is set.

(12) The camera 1 provided with the subject detection device captures a subject, captures the subject, captures the subject, displays a subject image for display 81 (also referred to as image data), and captures the subject. Image data of the first data number and second data different from the first data number from the imaging unit 32 having a second imaging region B2 for generating a detection image 82 (also referred to as luminance data) for obtaining information. The image processing unit 33 and the control unit 34 for inputting a number of luminance data, and the control unit 34 for detecting a subject imaged by the imaging unit 34 based on the inputted luminance data. Thereby, the camera 1 can perform subject detection based on the detection image 82 while displaying the display image 81. In addition, for example, by increasing the number of read data from the second imaging area B2 of the imaging element 32a, information about the subject can be increased, so that the accuracy of subject detection can be increased.

(13) The camera 1 including the subject detection device captures an image of the subject, generates a display image 81 (also referred to as image data) of the subject for display, images the subject, and relates to the subject. Image data of the first data number and second data different from the first data number from the imaging unit 32 having a second imaging region B2 for generating a detection image 82 (also referred to as luminance data) for obtaining information. The image processing unit 33 and the control unit 34 that input a number of luminance data, and the control unit 34 that generates a signal for detecting a subject imaged by the imaging unit 34 based on the input luminance data. Thereby, the camera 1 can perform subject detection based on the detection image 82 while displaying the display image 81. In addition, for example, by increasing the number of read data from the second imaging area B2 of the imaging element 32a, information about the subject can be increased, so that the accuracy of subject detection can be increased.

(14) The imaging device 32a mounted on the camera 1 captures the light from the imaging optical system 31, and generates the image signal based on the subject for display, and the light from the imaging optical system 31. And a second region for generating a luminance signal for obtaining information relating to the adjustment of the light image, and the number of signals read from the first imaging region B1 and the number of signals read from the second imaging region B2 Was made different. Thereby, the image sensor 32a can generate a luminance signal for obtaining information relating to the adjustment of the light image while generating an image signal for display. In addition, for example, by increasing the number of readout signals from the second imaging region B2 of the imaging element 32a, information regarding the adjustment of the light image can be increased, so that the accuracy of the focus adjustment can be increased.

(15) In the imaging element 32a of (14), the image signals generated by the plurality of pixels in the first imaging area B1 are read out, so that the number of signals read from the second imaging area B2 is the first imaging area. More than the number of signals read from B1. As a result, since the focus detection accuracy is improved as compared with the case where focus detection is performed based on the image signal from the first imaging region B1, appropriate focusing can be performed.

(16) In the imaging element 32a of (15), the luminance signals generated by the plurality of pixels in the second imaging region B2 are read out, and the image signal generated in the first imaging region B1 is More signals than the luminance signal generated in the two imaging areas B2 are thinned out. Thereby, the number of signals read from the second imaging area B2 is larger than the number of signals read from the first imaging area B1. As a result, since the focus detection accuracy is improved as compared with the case where focus detection is performed based on the image signal from the first imaging region B1, appropriate focusing can be performed.

(17) Since the imaging element 32a of (14) reads out a signal obtained by adding the image signals generated by the plurality of pixels in the first imaging region B1, the number of signals read out from the second imaging region B2 is the first. More than the number of signals read from the imaging region B1. As a result, since the focus detection accuracy is improved as compared with the case where focus detection is performed based on the image signal from the first imaging region B1, appropriate focusing can be performed.

(18) The imaging device 32a of (17) reads out a signal obtained by adding the luminance signals respectively generated by the plurality of pixels in the second imaging region B2, and the image signal generated in the first imaging region B1 is More signals are added than the luminance signal generated in the second imaging region B2. Thereby, the number of signals read from the second imaging area B2 is larger than the number of signals read from the first imaging area B1. As a result, since the focus detection accuracy is improved as compared with the case where focus detection is performed based on the image signal from the first imaging region B1, appropriate focusing can be performed.

(19) The imaging element 32a of (14) to (18) captures the same subject in the first imaging area B1 and the second imaging area B2, and the first imaging area for each of the areas 91 and 92 on the imaging screen. The ratio of the number of pixels (block) that generates an image signal in B1 to the number of pixels (block) that generates a luminance signal in the second imaging region B2 is set. Thereby, for example, power consumption and heat generation can be suppressed as compared with a case where the blocks included in the first imaging area B1 and the second imaging area B2 are operated 100% in both the area 91 and the area 92.

(20) Since the area 91 of the imaging screen of (19) is set based on the position of the person T as the subject (FIG. 8B), the area 91 where the subject has already been detected and the imaging In the area 92 where it is desired to detect a subject moving from the outside of the screen into the imaging screen, the number of pixels (block) that generates an image signal in the first imaging area B1 and the pixel that generates a luminance signal in the second imaging area B2. The ratio with the number (block) can be set appropriately.

(21) Since the setting of the area 91 of the imaging screen (19) or (20) is changed by the movement of the ball T as the subject (FIG. 8 (c)), the area 91 of the imaging screen is adapted to the movement of the ball T. Thus, the ratio between the number of pixels (block) that generates an image signal in the first imaging region B1 and the number of pixels (block) that generates a luminance signal in the second imaging region B2 can be appropriately changed.

(22) Since the setting of the area 91 of the imaging screen from (19) to (21) is changed as time elapses, for example, an image signal is generated in the first imaging area B1 as time elapses. The ratio of the number of pixels (block) to be generated and the number of pixels (block) to generate a luminance signal in the second imaging region B2 can be appropriately changed.

(23) Since the imaging element 32a of (14) to (22) above has different imaging conditions for the first imaging area B1 and the second imaging area B2, for example, the first imaging area B1 and the second imaging area By applying different gains to the region B2, it is possible to obtain an image signal suitable for display and a luminance signal suitable for detection of information related to adjustment of the light image.

(24) In the first imaging area B1 of the imaging element 32a of (23) above, the setting of imaging conditions is changed based on the luminance signal read from the second imaging area B2. As a result, while the image signal of the display image 81 is output from the first imaging area B1 of the imaging element 32a, the imaging of the first imaging area B1 is performed based on the luminance signal output from the second imaging area B2 of the imaging element 32a. A condition is set.

(25) The camera 1 provided with the lens adjusting device images the light from the imaging optical system 31 and generates the image signal based on the subject for display, and the light from the imaging optical system 31. The image signal of the first signal number and the second signal number different from the first signal number are obtained from the image pickup unit 32 having the second image pickup region B2 that picks up an image and generates a luminance signal for obtaining information relating to the adjustment of the light image. An image processing unit 33 and a control unit 34 for inputting a luminance signal of the number of signals, and a control unit 34 for moving the imaging optical system 31 so as to adjust an image of light incident on the imaging unit 32 by the input luminance signal. . Thus, the camera 1 can perform focusing based on the luminance signal while displaying based on the image signal. In addition, for example, by increasing the number of readout signals from the second imaging region B2 of the imaging element 32a, information regarding the adjustment of the light image can be increased, so that the accuracy of the focus adjustment can be increased.

(26) The camera 1 including the lens adjustment device captures the light from the imaging optical system 31, captures the light from the imaging optical system 31, and the first imaging region B1 that generates an image signal based on the subject for display. The image signal of the first signal number and the second signal number different from the first signal number are obtained from the image pickup unit 32 having the second image pickup region B2 that picks up an image and generates a luminance signal for obtaining information relating to the adjustment of the light image. An image processing unit 33 and a control unit 34 for inputting a luminance signal of the number of signals, and a control for generating a signal for adjusting the image of light incident on the imaging unit 32 by the imaging optical system 31 based on the input luminance signal. Unit 34. Thus, the camera 1 can perform focusing based on the luminance signal while displaying based on the image signal. In addition, for example, by increasing the number of readout signals from the second imaging region B2 of the imaging element 32a, information regarding the adjustment of the light image can be increased, so that the accuracy of the focus adjustment can be increased.

(27) The image sensor 32a mounted on the camera 1 captures the subject, captures the subject, and captures information about the first imaging region B1 that generates an image signal based on the subject for display, and captures the subject. And a second imaging region B2 for generating a luminance signal for the purpose, and the number of signals read from the first imaging region B1 is different from the number of signals read from the second imaging region B2. Thereby, the image sensor 32a can generate a luminance signal for obtaining information relating to setting of the imaging condition while generating an image signal for display. In addition, for example, by increasing the number of readout signals from the second imaging region B2 of the imaging element 32a, it is possible to increase information related to the setting of the imaging condition, so that it is possible to increase the setting accuracy of the exposure condition, for example.

(28) Since the image pickup device 32a of (27) reads out the image signals generated by the plurality of pixels in the first image pickup area B1, the number of signals read from the second image pickup area B2 is the first image pickup area. More than the number of signals read from B1. As a result, compared with the case where it performs based on the image signal from 1st imaging area B1, the setting precision of exposure conditions can be improved, for example.

(29) The imaging device 32a of (28) above reads out the luminance signal generated by each of the plurality of pixels in the second imaging region B2, and the image signal generated in the first imaging region B1 is More signals than the luminance signal generated in the two imaging areas B2 are thinned out. Thereby, the number of signals read from the second imaging area B2 is larger than the number of signals read from the first imaging area B1. As a result, compared with the case where it performs based on the image signal from 1st imaging area B1, the setting precision of exposure conditions can be improved, for example.

(30) Since the image pickup device 32a of (27) reads a signal obtained by adding the image signals respectively generated by the plurality of pixels in the first image pickup region B1, the number of signals read from the second image pickup region B2 is the first. More than the number of signals read from the imaging region B1. As a result, compared with the case where it performs based on the image signal from 1st imaging area B1, the setting precision of exposure conditions can be improved, for example.

(31) The image pickup device 32a of (30) reads out a signal obtained by adding the luminance signals respectively generated by the plurality of pixels in the second image pickup area B2, and the image signal generated in the first image pickup area B1 is More signals are added than the luminance signal generated in the second imaging region B2. Thereby, the number of signals read from the second imaging area B2 is larger than the number of signals read from the first imaging area B1. As a result, compared with the case where it performs based on the image signal from 1st imaging area B1, the setting precision of exposure conditions can be improved, for example.

(32) The imaging element 32a of (27) to (31) captures the same subject in the first imaging area B1 and the second imaging area B2, and the first imaging area for each of the areas 91 and 92 on the imaging screen. The ratio of the number of pixels (block) that generates an image signal in B1 to the number of pixels (block) that generates a luminance signal in the second imaging region B2 is set. Thereby, for example, power consumption and heat generation can be suppressed as compared with a case where the blocks included in the first imaging area B1 and the second imaging area B2 are operated 100% in both the area 91 and the area 92.

(33) Since the region 91 of the imaging screen of (32) is set based on the position of the person T as the subject (FIG. 8B), the region 91 where the subject has already been detected and the imaging In the area 92 where it is desired to detect a subject moving from the outside of the screen into the imaging screen, the number of pixels (block) that generates an image signal in the first imaging area B1 and the pixel that generates a luminance signal in the second imaging area B2. The ratio with the number (block) can be set appropriately.

(34) Since the setting of the area 91 of the imaging screen of (32) or (33) is changed by the movement of the ball T as the subject (FIG. 8 (c)), the setting is made in accordance with the movement of the ball T. Thus, the ratio between the number of pixels (block) that generates an image signal in the first imaging region B1 and the number of pixels (block) that generates a luminance signal in the second imaging region B2 can be appropriately changed.

(35) Since the setting of the area 91 of the imaging screen from (32) to (34) is changed with the passage of time, for example, an image signal is generated in the first imaging area B1 with the passage of time. The ratio of the number of pixels (block) to be generated and the number of pixels (block) to generate a luminance signal in the second imaging region B2 can be appropriately changed.

(36) Since the imaging elements 32a of (27) to (35) are set with different imaging conditions for the first imaging area B1 and the second imaging area B2, for example, the first imaging area B1 and the second imaging area By applying different gains to the region B2, it is possible to obtain an image signal suitable for display and a luminance signal suitable for detection of information related to setting of imaging conditions.

(37) In the first imaging area B1 of the imaging element 32a of (36), the setting of imaging conditions is changed based on the luminance signal read from the second imaging area B2. As a result, while the image signal of the display image 81 is output from the first imaging area B1 of the imaging element 32a, the imaging of the first imaging area B1 is performed based on the luminance signal output from the second imaging area B2 of the imaging element 32a. A condition is set.

(38) The camera 1 images a subject and generates a display image 81 (also referred to as an image signal) based on the subject for display, and information regarding the setting of the imaging condition by capturing the subject. The image signal of the first signal number and the second signal number different from the first signal number from the image pickup unit 32 having the second image pickup region B2 for generating the detection image 82 (also referred to as luminance signal) for obtaining Are provided with an image processing unit 33 and a control unit 34 for inputting the luminance signal, and a control unit 34 for setting an imaging condition of the imaging unit 32 based on the input luminance signal. Accordingly, the camera 1 can set the imaging condition based on the luminance signal while displaying based on the image signal. In addition, for example, by increasing the number of readout signals from the second imaging region B2 of the imaging element 32a, it is possible to increase information related to the setting of the imaging conditions, so that it is possible to increase the accuracy of setting the exposure conditions, for example.

(39) The camera 1 images a subject and generates a display image 81 (also referred to as an image signal) based on the subject for display, and information regarding the setting of the imaging condition by capturing the subject. The image signal of the first signal number and the second signal number different from the first signal number from the image pickup unit 32 having the second image pickup region B2 for generating the detection image 82 (also referred to as luminance signal) for obtaining The image processing unit 33 and the control unit 34 for inputting the luminance signal, and the control unit 34 for generating a signal for setting the imaging condition of the imaging unit 32 based on the input luminance signal. Accordingly, the camera 1 can set the imaging condition based on the luminance signal while displaying based on the image signal. In addition, for example, by increasing the number of readout signals from the second imaging region B2 of the imaging element 32a, it is possible to increase information related to the setting of the imaging condition, so that it is possible to increase the setting accuracy of the exposure condition, for example.

--- Modification of the above embodiment ---
The following modifications are also within the scope of the present invention, and one or a plurality of modifications can be combined with the above-described embodiment.
(Modification 1)
In the above embodiment, the region 92 and the region 91 in the case where one moving subject is detected have been described with reference to FIG. In the first modification, the control unit 34 sets the region 92 and the region 91 as follows when a plurality of moving subjects are detected.

  FIG. 17A is a diagram illustrating an example in which the main subject TA moves in the left direction and the main subject TB moves in the right direction. The hatched regions 92A and 92B indicate that the operation rate of the blocks included in the second imaging region B2 is higher than the operation rate of the blocks included in the first imaging region B1. The white area 91 indicates that the operating rate of the blocks included in the first imaging area B1 is higher than the operating rate of the blocks included in the second imaging area B2.

  The control unit 34 sets the area 92A in the direction of the movement destination of the person TA moving from the center of the imaging screen to the left. The reason why the area 92A is set in the moving direction of the person TA is that it is highly necessary to obtain the detection image 82 that is data (signal) from the second imaging area B2 in order to track the moving person TA. .

  Similarly, the control unit 34 sets the area 92B in the direction of the movement destination of the person TB moving from the center of the imaging screen to the right. The reason why the area 92B is set in the moving direction of the person TB is that it is highly necessary to obtain the detection image 82 that is data (signal) from the second imaging area B2 in order to track the moving person TB. .

  FIG. 17B is a diagram illustrating an example in which the main subject TA moves rightward toward the center of the imaging screen, and the main subject TB moves leftward toward the center of the imaging screen. The hatched area 92A and area 92B indicate that the operation rate of the blocks included in the second imaging region B2 is higher than the operation rate of the blocks included in the first imaging region B1. The white area 91 indicates that the operating rate of the blocks included in the first imaging area B1 is higher than the operating rate of the blocks included in the second imaging area B2.

  The control unit 34 sets the area 92A in the movement destination direction of the moving person TA and sets the area 92B in the movement destination direction of the moving person TB. This is because, as in the case of FIG. 17A, in order to track the moving person TA and person TB, it is highly necessary to obtain the detection image 82 that is data (signal) from the second imaging region B2. .

  FIG. 17C is a diagram showing the integrated area 92. The hatched area 92 indicates that the operating rate of the blocks included in the second imaging area B2 is higher than the operating rate of the blocks included in the first imaging area B1. The white area 91 indicates that the operating rate of the blocks included in the first imaging area B1 is higher than the operating rate of the blocks included in the second imaging area B2.

  When the distance between the main subject TA and the main subject TB in FIG. 17B approaches a predetermined distance, the control unit 34 integrates the region 92A and the region 92B into one region 92. The area 92 in this case can be used to track both the moving person TA and the person TB.

  FIG. 17D is a diagram illustrating an example in which the main subject TA moves in the lower left direction of the imaging screen and the main subject TB moves in the upper right direction of the imaging screen. The hatched regions 92A and 92B indicate that the operation rate of the blocks included in the second imaging region B2 is higher than the operation rate of the blocks included in the first imaging region B1. The white area 91 indicates that the operating rate of the blocks included in the first imaging area B1 is higher than the operating rate of the blocks included in the second imaging area B2.

  The control unit 34 sets the area 92A in the movement destination direction of the moving person TA and sets the area 92B in the movement destination direction of the moving person TB. As in the case of FIGS. 17A to 17C, it is necessary to obtain the detection image 82 that is data (signal) from the second imaging region B2 in order to track the moving person TA and the person TB. Is based on the notion of high. If the direction of movement is changed so that the person TA moves in the upper right direction of the imaging screen, the control unit 34 detects data (signal) from the second imaging area B2 obtained based on the area 92B. The person TA may be tracked using the image 82. Similarly, when the direction of movement is changed so that the person TB moves in the lower left direction of the imaging screen, the control unit 34 detects data (signal) from the second imaging area B2 obtained based on the area 92A. The person TB may be tracked using the work image 82.

  As described above, when a plurality of main subjects move, the region 92 and the region 91 may be set corresponding to each subject. Then, the setting of the area 92 and the area 91 may be reviewed for each frame or over time.

(Modification 2)
The control unit 34 may set the area 92 and the area 91 as shown in FIG. FIG. 18A is a diagram illustrating an example of setting the area 91 and the area 92 when the main subject moves in the left-right direction. Each of the plurality of hatched areas 92 having a vertical belt shape indicates that the operation rate of the blocks included in the second imaging region B2 is higher than the operation rate of the blocks included in the first imaging region B1. Also, the plurality of vertical strip-shaped areas 91 indicate that the operation rate of the blocks included in the first imaging region B1 is higher than the operation rate of the blocks included in the second imaging region B2.

  When it is known in advance that the main subject moves in the left-right direction, or when the control unit 34 detects that the main subject moves in the left-right direction, the control unit 34 displays the region 92 and the region shown in FIG. 91 is set. A plurality of vertical belt-like regions 92 are set because a subject moving in the left-right direction is frequently imaged in the region 92 and moves to data (signal) from the second imaging region B2 necessary for tracking. This is because subject information is likely to be included.

  FIG. 18B is a diagram illustrating a setting example of the area 91 and the area 92 when the main subject moves in the vertical direction. Each of the plurality of hatched areas 92 having a horizontal band indicates that the operation rate of the blocks included in the second imaging region B2 is higher than the operation rate of the blocks included in the first imaging region B1. A plurality of white areas 91 in a horizontal band shape indicate that the operation rate of the blocks included in the first imaging region B1 is higher than the operation rate of the blocks included in the second imaging region B2.

  When it is known in advance that the main subject moves in the vertical direction, or when the control unit 34 detects that the main subject is moved in the vertical direction, the control unit 34 displays the region 92 and the region shown in FIG. 91 is set. The plurality of horizontal band-shaped regions 92 are set because the subject moving in the vertical direction is frequently imaged in the region 92 and moves to the data (signal) from the second imaging region B2 necessary for tracking. This is because subject information is likely to be included.

  FIG. 18C is a diagram illustrating a setting example of the area 91 and the area 92 when the camera 1 is used for monitoring. A frame-shaped hatched area 92 provided on the outer edge of the imaging screen indicates that the operating rate of the blocks included in the second imaging area B2 is higher than the operating rate of the blocks included in the first imaging area B1. Show. A white area 91 in the center of the imaging screen indicates that the operating rate of the blocks included in the first imaging area B1 is higher than the operating rate of the blocks included in the second imaging area B2.

  For example, when the camera 1 is used for monitoring purposes, the control unit 34 performs setting for a case where a subject existing outside the imaging screen moves into the imaging screen. Specifically, an area 92 is set on the outer edge of the imaging screen in order to obtain a detection image 82 that accurately detects a subject entering the imaging screen.

  FIG. 18D is a diagram illustrating a setting example of the region 91 and the region 92 when the camera 1 is held in the horizontal position and a horizontally long image is captured. The hatched areas 92 provided on the left and right of the imaging screen indicate that the operating rate of the blocks included in the second imaging area B2 is higher than the operating rate of the blocks included in the first imaging area B1. A white area 91 in the center of the imaging screen indicates that the operating rate of the blocks included in the first imaging area B1 is higher than the operating rate of the blocks included in the second imaging area B2.

  When the camera 1 is held in the horizontal position, the control unit 34 detects this by using, for example, an attitude sensor, and performs settings corresponding to the horizontally long imaging screen. Specifically, a setting is prepared for a case where a subject existing outside the imaging screen moves from the left and right into the imaging screen. In general, since it is unlikely that the subject will fall into the imaging screen from above during hand-held shooting, the upper and lower sides of the imaging screen are set to be included in the area 91 instead of the area 92.

(Modification 3)
The control unit 34 may generate a high definition image based on the detection image 82. As shown in FIG. 19A, the control unit 34 causes the display unit 35 to display a live view image 81A based on the display image 81 and to display a high-definition image 82A based on the detection image 82 on a picture-in-picture (PIP). ) Display. When the user performs a touch operation on the display position of the window in which picture-in-picture is displayed, the control unit 34 displays a high-definition image 82A on the display unit 35 as shown in FIG. 81A is displayed in picture-in-picture.

  As described above, the detection image 82 read from the second imaging region B2 of the image sensor 32a has a larger number of data read than the display image 81 read from the first imaging region B1 of the image sensor 32a (image data). Therefore, it is a high-definition image. For this reason, it can be displayed as a high-definition image by displaying it largely on the display unit 35. When the user touches the display position of the window in which picture-in-picture is displayed in FIG. 19B, the control unit 34 returns to the display shown in FIG.

(Modification 4)
In the embodiment described above, the setting unit 34b of the control unit 34 detects the subject element based on the live view image, and divides the screen of the live view image into regions including the subject element. In the fourth modification, when the control unit 34 includes a photometric sensor in addition to the image sensor 32a, the control unit 34 may divide the region based on an output signal from the photometric sensor.

  The control unit 34 divides the foreground and the background based on the output signal from the photometric sensor. Specifically, the live view image acquired by the image sensor 32b is a foreground area corresponding to an area determined as a foreground from an output signal from a photometric sensor, and an area determined as a background from an output signal from the photometric sensor. Is divided into background areas corresponding to.

  Further, as illustrated in FIG. 6A to FIG. 6C, the control unit 34 further controls the first imaging region B1 and the second imaging region with respect to the position corresponding to the foreground region on the imaging surface of the imaging device 32a. B2 is arranged. On the other hand, the control unit 34 arranges only the first imaging region B1 on the imaging surface of the imaging device 32a with respect to the position corresponding to the background region of the imaging surface of the imaging device 32a. The control unit 34 uses the display image 81 for live view display, and uses the detection image 82 for subject element detection, focus detection, and exposure calculation.

  According to the modified example 4, by using the output signal from the photometric sensor, it is possible to perform region division of the live view image acquired by the imaging device 32b. In addition, the display image 81 and the detection image 82 can be obtained for the foreground area, and only the display image 81 can be obtained for the background area.

(Modification 5)
In the above description, the camera 1 has been described as an example. However, the camera 1 may be configured by a mobile electronic device such as a high-function mobile phone, a tablet terminal, or a wearable device having a camera function such as a smartphone including the image sensor 32a.

The imaging optical system 31 described above may include a zoom lens or tilt lens. The lens movement control unit 34d adjusts the angle of view by the imaging optical system 31 by moving the zoom lens in the optical axis direction. That is, the image by the imaging optical system 31 can be adjusted by moving the zoom lens, such as obtaining an image of a wide range of subjects or obtaining a large image of a far subject.
Further, the lens movement control unit 34d can adjust the distortion of the image by the imaging optical system 31 by moving the tilt lens in a direction orthogonal to the optical axis.
Based on the idea that it is preferable to use the preprocessed image data as described above in order to adjust the state of the image by the imaging optical system 31 (for example, the state of the angle of view or the state of distortion of the image). The pre-processing described above may be performed.

The above-described embodiments and modifications include the following imaging device and imaging element.
(1) It has a plurality of imaging regions in which a plurality of pixels that generate signals by photoelectrically converted charges are arranged, and a signal line that is arranged for each of the imaging regions and that outputs signals generated by the pixels. An image of the imaged subject is captured using the first pixel signal output to the signal line of the first imaging area from the imaging element and the pixels arranged in the first imaging area of the imaging area. A generating unit configured to generate the first pixel output to the signal line of the second imaging region from a pixel arranged in a second imaging region different from the first imaging region out of the imaging region; An imaging apparatus comprising: a detection unit that detects a subject using signals of a second pixel that is larger than the number of second pixels.
(2) In the imaging device as in (1), the generation unit thins out the pixels arranged in the first imaging region and outputs the signal of the first pixel output to the signal line in the first imaging region. Is used to generate a captured image of the subject.
(3) In the imaging device as in (2), the detection unit thins out the pixels arranged in the second imaging region and outputs the first pixel output to the signal line in the second imaging region. A subject is detected using signals of the second pixels larger than the number.
(4) The imaging device according to any one of (1) to (3) includes a setting unit that sets an area to be used as the second imaging area among the imaging areas by the subject detected by the detection unit.
(5) In the imaging device as in (4), the setting unit sets an area to be used as the second imaging area among the imaging areas, according to the moving direction of the subject detected by the detection unit.
(6) In the imaging device as in (4) or (5), the setting unit may move the subject in the moving direction side of the subject with respect to the subject detected by the detection unit in the imaging region. More areas to be used as the second imaging area than the direction opposite to the moving direction are set.
(7) In the imaging device according to any one of (4) to (6), the setting unit changes a setting of an area used as the second imaging area in the imaging area as the imaging time elapses.

(8) In the imaging device according to any one of (4) to (7), the setting unit sets the first imaging area to an imaging condition different from the imaging condition of the second imaging area.
(9) In the imaging device as in (8), the setting unit sets the imaging condition of the first imaging area based on the signal of the second pixel read from the second imaging area.
(10) A plurality of imaging regions in which a plurality of pixels that generate signals by photoelectrically converted charges are arranged, and a signal line that is arranged for each of the imaging regions and that outputs signals generated by the pixels. The imaging area includes a first imaging area having a first pixel used for generating an image of the captured subject, and a second number larger than the number of the first pixels used for detecting the subject. A second imaging region having pixels;
(11) A first imaging region in which a first photoelectric conversion unit that converts light into electric charge and a second photoelectric conversion unit that converts light into electric charge are arranged, and light is converted into electric charge. An imaging device having a third photoelectric conversion unit for converting and a second imaging region in which a fourth photoelectric conversion unit for converting light into charges is disposed; and in the first imaging region, the first photoelectric conversion unit and A generation unit that generates an image of a captured subject using the first signal generated by the electric charge converted by the second photoelectric conversion unit, and the third photoelectric conversion unit in the second imaging region. An imaging apparatus comprising: a detection unit that detects a subject using a signal generated by the converted charge and a signal generated by the charge converted by the fourth photoelectric conversion unit.

Moreover, the following embodiments and modifications include the following imaging device and imaging element.
(1) A plurality of imaging regions in which a plurality of pixels that generate signals by charges photoelectrically converted through a lens that is movable in the optical axis direction of the optical system are arranged, and the pixels are arranged for each of the imaging regions. An image sensor that outputs a signal generated in step (1), and a first pixel output to the signal line in the first imaging region from a pixel arranged in the first imaging region among the imaging regions. The second imaging region is generated from a generation unit that generates an image of a captured subject using a signal of the pixel and pixels arranged in a second imaging region different from the first imaging region in the imaging region. And a control unit that controls movement of the lens using a signal of the second pixel that is output to the signal line and is larger than the number of the first pixels.
(2) In the imaging device as in (1), the generation unit thins out the pixels arranged in the first imaging region and outputs the signal of the first pixel output to the signal line in the first imaging region. Is used to generate a captured image of the subject.
(3) In the imaging device as in (2), the control unit thins out the pixels arranged in the second imaging region and outputs the first pixel output to the signal line in the second imaging region. The movement of the lens is controlled using signals of the second pixels larger than the number.
(4) In the imaging device according to any one of (1) to (3), a detection unit that detects a subject using the signal of the second pixel and a subject detected by the detection unit A setting unit configured to set an area to be used as the second imaging area.
(5) In the imaging device as in (4), the setting unit sets an area to be used as the second imaging area among the imaging areas, according to the moving direction of the subject detected by the detection unit.
(6) In the imaging device as in (4) or (5), the setting unit may move the subject in the moving direction side of the subject with respect to the subject detected by the detection unit in the imaging region. More areas to be used as the second imaging area than the direction opposite to the moving direction are set.
(7) In the imaging device according to any one of (4) to (6), the setting unit changes a setting of an area used as the second imaging area in the imaging area as the imaging time elapses.
(8) In the imaging device according to any one of (4) to (7), the setting unit sets the first imaging area to an imaging condition different from the imaging condition of the second imaging area.
(9) In the imaging device as in (8), the setting unit sets the imaging condition of the first imaging area based on the signal of the second pixel read from the second imaging area.
(10) A plurality of imaging regions in which a plurality of pixels that generate signals by charges photoelectrically converted through a lens that is movable in the optical axis direction of the optical system are arranged, and the pixels are arranged for each of the imaging regions. And a signal line for outputting the signal generated in step (b), wherein the imaging region controls a movement of the lens and a first imaging region having a first pixel used for generating a captured image of the subject. And a second imaging region having a second pixel that is larger than the number of the first pixels.
(11) A first photoelectric conversion unit that converts light into electric charge and a second photoelectric conversion unit that converts light into electric charge in an imaging region for imaging an object via a lens movable in the optical axis direction of the optical system An imaging device having a first imaging region in which is arranged, a second photoelectric conversion unit in which a third photoelectric conversion unit that converts light into electric charge and a fourth photoelectric conversion unit that converts light into electric charge are arranged, A generating unit configured to generate an image of the imaged subject using the first signal generated by the charge converted by the first photoelectric conversion unit and the second photoelectric conversion unit in the first imaging region; In the second imaging region, the movement of the lens is performed using a signal generated by the charge converted by the third photoelectric conversion unit and a signal generated by the charge converted by the fourth photoelectric conversion unit. A control unit for controlling the imaging device.

Furthermore, the above-described embodiments and modifications include the following imaging device and imaging element.
(1) It has a plurality of imaging regions in which a plurality of pixels that generate signals by photoelectrically converted charges are arranged, and a signal line that is arranged for each of the imaging regions and that outputs signals generated by the pixels. An image of the imaged subject is captured using the first pixel signal output to the signal line of the first imaging area from the imaging element and the pixels arranged in the first imaging area of the imaging area. A generating unit configured to generate the first pixel output to the signal line of the second imaging region from a pixel arranged in a second imaging region different from the first imaging region out of the imaging region; An imaging apparatus comprising: a setting unit configured to set an imaging condition of the first imaging region by using more second pixel signals.
(2) In the imaging device as in (1), the generation unit thins out the pixels arranged in the first imaging region and outputs the signal of the first pixel output to the signal line in the first imaging region. Is used to generate a captured image of the subject.
(3) In the imaging device as in (2), the setting unit thins out the pixels arranged in the second imaging region and outputs the first pixel output to the signal line in the second imaging region. The imaging condition of the first imaging area is set using signals of the second pixels larger than the number.
(4) The imaging device according to any one of (1) to (3), further including a detection unit that detects a subject using the signal of the second pixel, wherein the setting unit detects the subject detected by the detection unit Thus, an area to be used as the second imaging area among the imaging areas is set.
(5) In the imaging device as in (4), the setting unit sets an area to be used as the second imaging area among the imaging areas, according to the moving direction of the subject detected by the detection unit.
(6) In the imaging device as in (4) or (5), the setting unit may move the subject in the moving direction side of the subject with respect to the subject detected by the detection unit in the imaging region. More areas to be used as the second imaging area than the direction opposite to the moving direction are set.
(7) In the imaging device according to any one of (4) to (6), the setting unit changes a setting of an area used as the second imaging area in the imaging area as the imaging time elapses.
(8) In the imaging device of any one of (1) to (7), the setting unit sets the first imaging area to an imaging condition different from the imaging condition of the second imaging area.
(9) A plurality of imaging regions in which a plurality of pixels that generate signals by photoelectrically converted charges are arranged, and a signal line that is arranged for each of the imaging regions and outputs a signal generated by the pixels. The imaging area includes a first imaging area having a first pixel that is used to generate an image of a captured subject, and the first pixel that is used to set an imaging condition of the first imaging area. And a second imaging region having more second pixels than the number of the imaging elements.
(10) A first imaging region in which a first photoelectric conversion unit that converts light into electric charge and a second photoelectric conversion unit that converts light into electric charge are arranged, and the light is converted into electric charge. An imaging device having a third photoelectric conversion unit for converting and a second imaging region in which a fourth photoelectric conversion unit for converting light into charges is disposed; and in the first imaging region, the first photoelectric conversion unit and A generation unit that generates an image of a captured subject using the first signal generated by the electric charge converted by the second photoelectric conversion unit, and the third photoelectric conversion unit in the second imaging region. An imaging device comprising: a setting unit that sets an imaging condition of the first imaging region using a signal generated by the converted charge and a signal generated by the charge converted by the fourth photoelectric conversion unit .

  Although various embodiments and modifications have been described above, the present invention is not limited to these contents. Other embodiments conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Camera 31 ... Imaging optical system 32 ... Imaging part 32a, 100 ... Imaging element 33 ... Image processing part 33a ... Input part 33b ... Correction | amendment part 33c ... Generation | occurrence | production part 34 ... Control part 34a ... Object detection part 34b ... Setting part 34c ... Imaging control unit 34d ... Lens movement control unit 35 ... Display unit

Claims (11)

A plurality of imaging regions in which a plurality of pixels that generate signals by charges that are photoelectrically converted through a lens that is movable in the optical axis direction of the optical system are arranged, and are arranged for each of the imaging regions and generated by the pixels. An image sensor having a signal line for outputting the received signal;
A generation unit that generates an image of a captured subject using a signal of the first pixel output to the signal line of the first imaging area from pixels arranged in the first imaging area of the imaging area When,
A second number larger than the number of the first pixels output from the pixels arranged in a second imaging area different from the first imaging area to the signal line in the second imaging area. A control unit for controlling movement of the lens using a pixel signal;
An imaging apparatus comprising: The imaging device according to claim 1,
The generation unit generates an image of a captured subject using a signal of the first pixel that is thinned out from pixels arranged in the first imaging region and output to the signal line of the first imaging region. Imaging device. The imaging device according to claim 2,
The control unit uses signals of second pixels larger than the number of the first pixels, which are thinned out from the pixels arranged in the second imaging region and output to the signal line in the second imaging region, An imaging device that controls movement of the lens. In the imaging device according to any one of claims 1 to 3,
A detection unit for detecting a subject using the signal of the second pixel;
A setting unit configured to set an area to be used as the second imaging area among the imaging areas based on the subject detected by the detection unit;
An imaging apparatus comprising: The imaging apparatus according to claim 4,
The setting unit is an imaging device that sets an area to be used as the second imaging area among the imaging areas according to the moving direction of the subject detected by the detection unit. In the imaging device according to claim 4 or 5,
The setting unit includes, as the second imaging region, the number of the second imaging region in the moving direction side of the subject is larger than that in the direction opposite to the moving direction of the subject with respect to the subject detected by the detecting unit. An imaging device that sets an area to be used. In the imaging device according to any one of claims 4 to 6,
The said setting part is an imaging device which changes the setting of the area | region used as said 2nd imaging area among the said imaging areas with progress of imaging time. In the imaging device according to any one of claims 4 to 7,
The setting unit is an imaging apparatus that sets the first imaging region to an imaging condition different from the imaging condition of the second imaging region. The imaging device according to claim 8,
The imaging unit is configured to set an imaging condition of the first imaging region based on a signal of the second pixel read from the second imaging region. A plurality of imaging regions in which a plurality of pixels that generate signals by charges that are photoelectrically converted through a lens that is movable in the optical axis direction of the optical system are arranged, and are arranged for each of the imaging regions and generated by the pixels. A signal line for outputting
The imaging area is greater than the number of the first imaging area having a first pixel used to generate an image of the imaged subject and the first pixel used to control the movement of the lens. A second imaging region having a second pixel. A first photoelectric conversion unit that converts light into electric charge and a second photoelectric conversion unit that converts light into electric charge are arranged in an imaging region for imaging an object via a lens movable in the optical axis direction of the optical system. An imaging device having a first imaging region, a second imaging region in which a third photoelectric conversion unit that converts light into electric charge and a fourth photoelectric conversion unit that converts light into electric charge are disposed,
A generating unit configured to generate an image of the imaged subject using the first signal generated by the charge converted by the first photoelectric conversion unit and the second photoelectric conversion unit in the first imaging region;
In the second imaging region, the movement of the lens is performed using a signal generated by the charge converted by the third photoelectric conversion unit and a signal generated by the charge converted by the fourth photoelectric conversion unit. A control unit to control;
An imaging apparatus comprising:

Images