JP7318631B2 - Image sensor and electronic equipment - Google Patents

Description

The present invention relates to image sensors and electronic devices.

2. Description of the Related Art An imaging device having an imaging device in which a back-illuminated imaging chip and a signal processing chip are stacked (hereinafter, this imaging device is referred to as a stacked imaging device) has been proposed (see Patent Document 1, for example). In the stacked imaging device, a back-illuminated imaging chip and a signal processing chip are stacked such that they are connected via microbumps for each block unit in which a plurality of pixels are grouped together.

JP 2006-49361 A

However, there are not many proposals for obtaining an image by capturing an image in units of a plurality of blocks in a conventional imaging device equipped with a stacked imaging device, and the usability of an imaging device equipped with a stacked imaging device has not been sufficient.

An object of the present invention is to prevent a flicker phenomenon occurring in an image by detecting flickering of a light source in a partial area of an image sensor.

An object of the present invention is to generate an image (partial moving image) in which a portion of the image is a moving image and the other portion is a still image while capturing a moving image using an imaging device.

An object of an aspect of the present invention is to set a region according to a main subject and perform imaging under imaging conditions corresponding to the region.

An object of an aspect of the present invention is to obtain an image with a wide dynamic range by varying the imaging conditions of the imaging device for each of a plurality of regions.

An object of the present invention is to perform processing related to exposure from saturation level exposure to proper exposure in a short period of time.

An object of an aspect of the present invention is to reduce the power consumption required to drive an imaging device.

An object of the present invention is to reliably transfer image data from an imaging unit to an image processing unit even when the frame rate of a predetermined area is increased.

An object of the present invention is to photograph or avoid photographing a subject that has unexpectedly jumped into the angle of view.

An imaging device according to a first aspect of the present invention includes a plurality of stacked semiconductor chips. The plurality of semiconductor chips includes a first semiconductor in which a plurality of pixels having a photoelectric conversion portion for converting light into electric charge and a transfer portion for transferring the electric charge converted by the photoelectric conversion portion are arranged in a row direction . have a tip. The plurality of semiconductor chips include first pixels among the plurality of pixels that output a first signal used for flicker detection, and second pixels among the plurality of pixels that output a second signal used for generating an image of a subject . It has a second semiconductor chip having a drive unit that drives the two pixels under different imaging conditions. A transfer unit included in the first pixel is connected to a first transfer control wiring through which a first transfer control signal is output from the second semiconductor chip. A transfer section included in the second pixel is connected to a second transfer control wiring through which a second transfer control signal is output from the second semiconductor chip. An imaging device according to a second aspect of the present invention is an imaging device including a plurality of stacked semiconductor chips. The plurality of semiconductor chips includes a first semiconductor in which a plurality of pixels having a photoelectric conversion portion for converting light into electric charge and a reset portion for discharging the electric charge converted by the photoelectric conversion portion are arranged in a row direction. have a tip. The plurality of semiconductor chips includes first pixels among the plurality of pixels that output a first signal used for flicker detection, and second pixels among the plurality of pixels that output a second signal used for generating an image of a subject. and a second semiconductor chip having a driving unit for driving under different imaging conditions. A reset unit included in the first pixel is connected to a first reset control wiring through which a first reset control signal is output from the second semiconductor chip. A reset unit included in the second pixel is connected to a second reset control wiring through which a second reset control signal is output from the second semiconductor chip. An electronic device according to a third aspect of the invention includes the imaging device according to the first aspect or the second aspect of the invention .

According to the aspect of the present invention, it is possible to prevent the flicker phenomenon that occurs in the image by detecting blinking of the light source in a partial area of the image sensor.

According to the aspect of the present invention, it is possible to generate an image (partial moving image) in which a portion of the image is a moving image and the other portion is a still image while capturing a moving image using an imaging device.

According to the aspect of the present invention, it is possible to set an area according to the main subject and perform imaging under imaging conditions corresponding to the area.

According to the aspect of the present invention, it is possible to obtain an image with a wide dynamic range by varying the imaging conditions of the imaging element for each of the plurality of regions.

According to the aspect of the present invention, exposure processing can be performed from saturation level exposure to proper exposure in a short period of time.

According to the aspect of the present invention, it is possible to reduce the power consumption required for driving the imaging device.

According to the aspect of the present invention, it is possible to reliably transfer image data from the imaging unit to the image processing unit even when the frame rate of the predetermined area is increased.

According to the aspect of the present invention, it is possible to photograph or not to photograph a subject that unexpectedly jumps into the angle of view.

1 is a cross-sectional view of an imaging device according to a first embodiment; FIG. It is a figure explaining the pixel arrangement|sequence of an imaging chip, and a unit group. FIG. 4 is a circuit diagram corresponding to a unit group of imaging chips; 3 is a block diagram showing the functional configuration of an imaging device; FIG. It is a figure which shows the 1st area|region and the 2nd area|region in the image pick-up element of 1st Embodiment. 1 is a cross-sectional view showing a schematic configuration of a digital camera, which is an example of an imaging device; FIG. 1 is a block diagram showing the configuration of a digital camera according to a first embodiment; FIG. 4 is a flowchart for explaining a shooting operation executed by the system control unit of the first embodiment; 4 is a timing chart showing the blinking period of the light source, the exposure time of the first area, and the signal output; 4 is a timing chart showing the blinking period of the light source, the exposure time of the second area, and the signal output; It is a figure which shows the 1st area|region, the 3rd area|region, and the 4th area|region in the image pick-up element of 2nd Embodiment. 4 is a timing chart showing the blinking cycle of the light source, the exposure time and signal output of the third area, and the exposure time and signal output of the fourth area; FIG. 11 is a flowchart for explaining a shooting operation executed by a system control unit according to the third embodiment; FIG. FIG. 12 is a diagram showing the arrangement of two imaging elements according to the fifth embodiment; FIG. 14 is a block diagram showing the configuration of an imaging device and an electronic device according to a sixth embodiment; FIG. 4 is a diagram for explaining an outline of a partial moving image; FIG. FIG. 12 is a block diagram showing the configuration of a digital camera according to a seventh embodiment; FIG. FIG. 21 is a diagram showing a display unit of a digital camera according to a seventh embodiment; FIG. FIG. 16 is a flowchart for explaining a partial moving image shooting operation executed by a system control unit according to the seventh embodiment; FIG. FIG. 20 is a flowchart for explaining setting processing shown in FIG. 19; FIG. FIG. 10 is a diagram showing a display example of the display panel when the user sets the moving image area; FIG. 4 is a diagram showing a moving image area in a pixel area of an imaging device; FIG. 10 is a diagram showing a display example of the display panel when the user sets the frame rate; FIG. 10 is a diagram showing a display example of the display panel when the user sets a still image area; FIG. 4 is a diagram showing a still image area in a pixel area of an imaging device; FIG. 10 is a diagram showing a display example of the display panel when setting a moving subject area as a moving image area; FIG. 22 is a flowchart showing imaging processing in the eighth embodiment; FIG. FIG. 10 is a diagram for explaining complementation of an image accompanying movement of a moving subject; It is a figure which shows the arrangement|sequence pattern in each block. FIG. 21 is a block diagram showing the configuration of an imaging device and an electronic device according to a ninth embodiment; FIG. 21 is a block diagram showing the configuration of a digital camera according to a tenth embodiment; FIG. FIG. 14 is a flowchart for explaining a photographing operation executed by a system control unit according to the tenth embodiment; FIG. FIG. 14 is a flowchart for explaining a photographing operation executed by a system control unit according to the tenth embodiment; FIG. FIG. 21 is a diagram showing a main subject area, a partial area, and a non-main subject area in an imaging area of an image sensor according to a tenth embodiment; FIG. 22 is a diagram showing a main subject area, a partial area, and a non-main subject area on the display surface of the display unit according to the tenth embodiment; It is a figure which shows an example of the image of a smile, and the image which is not a smile. FIG. 21 is a flowchart for explaining region setting processing in the eleventh embodiment; FIG. FIG. 21 is a diagram showing an arrangement pattern of AF areas and AE areas according to the eleventh embodiment; FIG. 10 is a diagram showing a non-main subject area when the main subject is not detected by the detection unit; FIG. 10 is a diagram showing a main subject area, a partial area, and a non-main subject area when the main subject is detected by the detection unit; FIG. 22 is a diagram showing a modification of the arrangement pattern of AF areas and AE areas according to the eleventh embodiment; FIG. 21 is a flowchart for explaining shooting mode setting processing executed by a system control unit according to a twelfth embodiment; FIG. FIG. 22 is a diagram showing an arrangement pattern of still image areas and moving image areas according to the twelfth embodiment; FIG. 22 is a diagram showing a main subject area, a partial area, and a non-main subject area in an imaging area of an image sensor according to a twelfth embodiment; FIG. 22 is a diagram showing a first display example of the display section of the twelfth embodiment; FIG. 22 is a diagram showing a second display example of the display section of the twelfth embodiment; FIG. 22 is a block diagram showing the configuration of an imaging device and an electronic device according to a thirteenth embodiment; FIG. 22 is a block diagram showing the configuration of a digital camera according to a fourteenth embodiment; FIG. FIG. 16 is a flow chart for explaining a photographing operation executed by a system control unit according to a fourteenth embodiment; FIG. FIG. 10 is a diagram showing a display example of an image made up of a plurality of areas according to luminance levels; FIG. 51 is a graph showing the relationship between the brightness of the subject shown in FIG. 50 and the luminance level of the imaging element; FIG. 51 is a waveform diagram showing changes in luminance level between points A1 and A2 in FIG. 50; FIG. 53 is a waveform diagram showing luminance levels when area division and imaging conditions are set so that the luminance levels shown in FIG. 52 fall within a specific range; FIG. 10 is a diagram showing a band-shaped boundary area set at the boundary between a level (1) area and a level (2) area; FIG. 51 is a partially enlarged view showing a level (3) area and a level (4) area shown in FIG. 50; FIG. 56 is a waveform diagram showing changes in luminance level between points D1 and D2 shown in FIG. 55; FIG. 57 is a waveform diagram showing luminance levels when area division and imaging conditions are set so that the luminance levels shown in FIG. 56 fall within a specific range; 4 is a timing chart showing imaging timings in each region; FIG. 21 is a flow chart for explaining image processing executed by an image processing unit according to a fourteenth embodiment; FIG. FIG. 21 is a block diagram showing the configuration of an imaging device and an electronic device according to a fifteenth embodiment; FIG. 22 is a block diagram showing the configuration of a digital camera according to a sixteenth embodiment; FIG. FIG. 21 is a diagram showing a first area, a second area, and a third area in an imaging device according to a sixteenth embodiment; FIG. 22 is a diagram for explaining AE processing in the sixteenth embodiment; FIG. FIG. 32 is a diagram showing a state in which the first region is saturated in the seventeenth embodiment; FIG. 22 is a diagram for explaining moving image HDR in the eighteenth embodiment; FIG. FIG. 20 is a block diagram showing the configuration of a smartphone according to a twentieth embodiment; FIG. FIG. 20 is a diagram showing an imaging region in an imaging element of a twentieth embodiment; It is a figure which shows the example of a display of the display part of a smart phone. It is a figure which shows the example of a display of the display part of a smart phone. It is a figure which shows the example of a display of the display part of a smart phone. FIG. 21 is a diagram showing an imaging region in an imaging element of a twenty-first embodiment; FIG. FIG. 20 is a block diagram showing the configurations of an imaging unit, an image processing unit, and a recording unit according to a twenty-first embodiment; 4 is a diagram showing transfer timing of image data from an imaging unit to an image processing unit; FIG. FIG. 4 is a block diagram showing configurations of an imaging unit, an image processing unit, and a recording unit when image data is read from the left half area at a high frame rate and image data is not read from the right half area; FIG. 10 is a diagram showing the transfer timing of image data from the imaging unit to the image processing unit when image data is read from the left half area at a high frame rate and image data is not read from the right half area. FIG. 14 is a flow chart showing processing in an image processing unit of the twenty-first embodiment; FIG. FIG. 22 is a flow chart showing processing in an imaging unit of the twenty-first embodiment; FIG. FIG. 4 is a diagram showing the relationship between frame rate and area; FIG. 20 is a block diagram showing the configuration of an imaging system according to a twenty-second embodiment; FIG. FIG. 22 is a diagram showing switching timings of imaging conditions of the first imaging area and the second imaging area in the twenty-third embodiment;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to this. In addition, in the drawings, in order to describe the embodiments, the scale may be changed as appropriate, such as by enlarging or emphasizing a portion. In each of the following embodiments, an interchangeable lens type digital camera will be described as an example of an imaging device.

<First embodiment>
FIG. 1 is a cross-sectional view of the imaging device 100 of the first embodiment. The imaging device 100 is described in Japanese Patent Application No. 2012-139026 previously filed by the applicant of the present application. The imaging device 100 includes an imaging chip 113 that outputs pixel signals corresponding to incident light, a signal processing chip 111 that processes the pixel signals output from the imaging chip 113, and a memory that stores the pixel signals processed by the signal processing chip 111. and a chip 112 . The imaging chip 113, the signal processing chip 111, and the memory chip 112 are stacked, and the imaging chip 113 and the signal processing chip 111 are electrically connected to each other by conductive bumps 109 such as Cu. Also, the signal processing chip 111 and the memory chip 112 are electrically connected to each other by conductive bumps 109 such as Cu.

As indicated by the illustrated coordinate axes, the incident light is mainly incident in the positive Z-axis direction. In this embodiment, the surface of the imaging chip 113 on which incident light is incident is referred to as the rear surface. Also, as indicated by the coordinate axes, the leftward direction on the paper perpendicular to the Z-axis is the positive X-axis direction, and the frontward direction on the paper perpendicular to the Z-axis and the X-axis is the positive Y-axis direction. In the following several figures, the coordinate axes are displayed with reference to the coordinate axes of FIG. 1 so that the direction of each figure can be understood.

An example of the imaging chip 113 is a back-illuminated MOS image sensor. The PD layer 106 is arranged on the back side of the wiring layer 108 . The PD layer 106 has a plurality of photodiodes (hereinafter referred to as PDs) 104 arranged two-dimensionally and storing charges according to incident light, and transistors 105 provided corresponding to the PDs 104 . .

A color filter 102 is provided on the incident light side of the PD layer 106 with a passivation film 103 interposed therebetween. The color filter 102 is a filter that passes a specific wavelength region of visible light. This color filter 102 has a plurality of types that transmit different wavelength regions, and has a specific arrangement corresponding to each of the PDs 104 . The arrangement of the color filters 102 will be described later. A set of color filter 102, PD 104, and transistor 105 forms one pixel.

A microlens 101 is provided on the incident light side of the color filter 102 so as to correspond to each pixel. The microlens 101 collects incident light toward the corresponding PD 104 .

The wiring layer 108 has wiring 107 for transmitting pixel signals from the PD layer 106 to the signal processing chip 111 . The wiring 107 may be multi-layered and may be provided with passive and active elements. A plurality of bumps 109 are arranged on the surface of the wiring layer 108 . These bumps 109 are aligned with the bumps 109 provided on the opposite surface of the signal processing chip 111 . By pressurizing the imaging chip 113 and the signal processing chip 111, the aligned bumps 109 are bonded and electrically connected.

Similarly, a plurality of bumps 109 are arranged on surfaces of the signal processing chip 111 and the memory chip 112 facing each other. These bumps 109 are aligned with each other. By applying pressure to the signal processing chip 111 and the memory chip 112, the aligned bumps 109 are bonded and electrically connected.

The bonding between the bumps 109 is not limited to Cu bump bonding by solid phase diffusion, and may be microbump bonding by solder melting. Also, about one bump 109 may be provided for one unit group, which will be described later. Therefore, the size of the bumps 109 may be larger than the pitch of the PDs 104 . Also, bumps larger than the bumps 109 corresponding to the pixel regions may be provided in peripheral regions other than the pixel region (pixel region 113A shown in FIG. 2) in which pixels are arranged.

The signal processing chip 111 has a TSV (Through-Silicon Via) 110 that connects circuits provided on the front surface and the back surface. The TSV 110 is provided in the peripheral area. Also, the TSV 110 may be provided in the peripheral area of the imaging chip 113 or in the memory chip 112 .

FIG. 2 is a diagram for explaining the pixel array and unit groups of the imaging chip 113. As shown in FIG. FIG. 2 particularly shows a state in which the imaging chip 113 is observed from the back side. A region in which pixels are arranged in the imaging chip 113 is called a pixel region (imaging region) 113A. More than 20 million pixels are arranged in a matrix in the pixel area 113A. In the example shown in FIG. 2, 16 pixels of adjacent 4×4 pixels form one unit group 131 . The grid lines in FIG. 2 illustrate the concept that adjacent pixels are grouped together to form unit groups 131 . The number of pixels forming the unit group 131 is not limited to this, and may be about 1000, for example, 32 pixels×64 pixels, or may be more or less.

As shown in the partially enlarged view of the pixel region 113A, the unit group 131 contains four so-called Bayer arrays each including four pixels of green pixels Gb and Gr, blue pixels B, and red pixels R, vertically and horizontally. A green pixel is a pixel having a green filter as the color filter 102, and receives light in the green wavelength band among incident light. Similarly, a blue pixel is a pixel having a blue filter as the color filter 102 and receives light in the blue wavelength band. A red pixel is a pixel having a red filter as the color filter 102 and receives light in the red wavelength band.

FIG. 3 is a circuit diagram corresponding to a unit group of the imaging chip 113. As shown in FIG. In FIG. 3, a rectangle encircled by dotted lines typically represents a circuit corresponding to one pixel. Note that at least part of each transistor described below corresponds to the transistor 105 in FIG.

As described above, the unit group 131 is formed from 16 pixels. The 16 PDs 104 corresponding to each pixel are connected to transfer transistors 302 respectively. A gate of each transfer transistor 302 is connected to a TX wiring 307 to which a transfer pulse is supplied. In this embodiment, the TX wiring 307 is commonly connected to 16 transfer transistors 302 .

The drain of each transfer transistor 302 is connected to the source of each corresponding reset transistor 303 , and the so-called floating diffusion FD (charge detector) between the drain of the transfer transistor 302 and the source of each reset transistor 303 is connected to the amplification transistor 304 . connected to the gate. A drain of each reset transistor 303 is connected to a Vdd wiring 310 to which a power supply voltage is supplied. A gate of each reset transistor 303 is connected to a reset wiring 306 to which a reset pulse is supplied. In this embodiment, the reset wiring 306 is commonly connected to the 16 reset transistors 303 .

A drain of each amplifying transistor 304 is connected to a Vdd wiring 310 supplied with a power supply voltage. Also, the source of each amplification transistor 304 is connected to the drain of each corresponding selection transistor 305 . A gate of each selection transistor 305 is connected to a decoder wiring 308 to which a selection pulse is supplied. In this embodiment, the decoder wiring 308 is provided independently for each of the 16 select transistors 305 . A source of each selection transistor 305 is connected to a common output wiring 309 . A load current source 311 supplies current to the output wiring 309 . That is, the output wiring 309 for the selection transistor 305 is formed by a source follower. The load current source 311 may be provided on the imaging chip 113 side or may be provided on the signal processing chip 111 side.

Here, the flow from the start of charge accumulation to the pixel output after the end of charge accumulation will be described. A reset pulse is applied to the reset transistor 303 through the reset wiring 306 . At the same time, a transfer pulse is applied to the transfer transistor 302 through the TX wiring 307 . This resets the potentials of the PD 104 and the floating diffusion FD.

When the application of the transfer pulse is released, the PD 104 converts the incident light it receives into charges and accumulates them. After that, when the transfer pulse is applied again while the reset pulse is not applied, the charges accumulated in the PD 104 are transferred to the floating diffusion FD. As a result, the potential of the floating diffusion FD changes from the reset potential to the signal potential after charge accumulation. Then, when a selection pulse is applied to the selection transistor 305 through the decoder wiring 308 , fluctuations in the signal potential of the floating diffusion FD are transmitted to the output wiring 309 via the amplification transistor 304 and the selection transistor 305 . By such circuit operation, a pixel signal corresponding to the reset potential and the signal potential is output from the unit pixel to the output wiring 309 .

As shown in FIG. 3, in this embodiment, the reset wiring 306 and the TX wiring 307 are common to the 16 pixels forming the unit group 131 . That is, the reset pulse and the transfer pulse are each applied simultaneously to all 16 pixels. Therefore, all the pixels forming the unit group 131 start charge accumulation at the same timing and finish charge accumulation at the same timing. However, pixel signals corresponding to the accumulated charges are selectively output to the output wiring 309 by sequentially applying selection pulses to the respective selection transistors 305 . Also, the reset wiring 306 , the TX wiring 307 , and the output wiring 309 are provided separately for each unit group 131 .

By configuring the circuit on the basis of the unit group 131 in this way, the charge accumulation time can be controlled for each unit group 131 . In other words, it is possible to output pixel signals with different charge accumulation times between the unit groups 131 . Furthermore, while one unit group 131 is caused to perform one charge accumulation, the other unit group 131 is caused to repeat charge accumulation many times and to output a pixel signal each time. It is also possible to output each moving image frame at a different frame rate between the unit groups 131 .

FIG. 4 is a block diagram showing the functional configuration of the imaging device 100. As shown in FIG. Analog multiplexer 411 sequentially selects 16 PDs 104 forming unit group 131 . Then, the multiplexer 411 outputs the pixel signal of each of the 16 PDs 104 to the output wiring 309 provided corresponding to the unit group 131 . A multiplexer 411 is formed in the imaging chip 113 together with the PD 104 .

The analog pixel signal output via the multiplexer 411 is amplified by the amplifier 412 formed in the signal processing chip 111 . Then, the pixel signal amplified by the amplifier 412 is processed by a signal processing circuit 413 formed in the signal processing chip 111 that performs correlated double sampling (CDS) and analog/digital conversion. Signal processing of correlated double sampling is performed, and A/D conversion (conversion from analog signal to digital signal) is performed. Noise in the pixel signal is reduced by subjecting the pixel signal to signal processing of correlated double sampling in the signal processing circuit 413 . A/D-converted pixel signals are transferred to a demultiplexer 414 and stored in pixel memories 415 corresponding to respective pixels. Demultiplexer 414 and pixel memory 415 are formed in memory chip 112 .

An arithmetic circuit (signal processing unit) 416 processes the pixel signals stored in the pixel memory 415 and transfers them to the subsequent image processing unit. In this embodiment, the arithmetic circuit 416 executes processing for averaging pixel signals from a plurality of pixels included in each block that constitutes a predetermined area (the second area 210 shown in FIG. 5). The arithmetic circuit 416 may be provided in the signal processing chip 111 or may be provided in the memory chip 112 . Although FIG. 4 shows connections for one unit group 131, in reality, these units exist for each unit group 131 and operate in parallel. However, the arithmetic circuit 416 may not exist for each unit group 131 . For example, one arithmetic circuit 416 may sequentially refer to the values of the pixel memory 415 corresponding to each unit group 131 and process them sequentially.

As described above, the output wiring 309 is provided corresponding to each unit group 131 . The imaging device 100 has an imaging chip 113, a signal processing chip 111, and a memory chip 112 stacked. Therefore, by using the bumps 109 as the output wirings 309 for electrical connection between the chips, the wiring can be routed without increasing the size of each chip in the planar direction.

Next, blocks set in the pixel area 113A (see FIG. 2) of the image sensor 100 will be described. In this embodiment, the pixel area 113A of the image sensor 100 is divided into a plurality of blocks. A plurality of blocks are defined to include at least one unit group 131 per block. Pixels included in each block are controlled with different control parameters. That is, pixel signals with different control parameters are obtained for a pixel group included in a certain block and a pixel group included in another block. Control parameters include, for example, charge accumulation time or number of accumulations, frame rate, gain, thinning rate (pixel thinning rate), number of addition rows or columns for adding pixel signals (pixel addition number), digitization bit number, etc. Furthermore, the control parameter may be a parameter in image processing after acquiring image signals from pixels.

Here, the charge accumulation time is the time from when the PD 104 starts accumulating charges until it ends. This charge accumulation time is also called exposure time or shutter speed. The number of charge accumulations refers to the number of times the PD 104 accumulates charges per unit time. A frame rate is a value representing the number of frames processed (displayed or recorded) per unit time in a moving image. The unit of frame rate is fps (Frames Per Second). The higher the frame rate, the smoother the movement of the subject (that is, the object being imaged) in the moving image.

A gain is a gain factor (amplification factor) of the amplifier 412 . By changing this gain, the ISO sensitivity can be changed. The ISO sensitivity is a standard for photographic film established by ISO, and indicates how weak light the photographic film can record. However, in general, the ISO sensitivity is also used when expressing the sensitivity of the imaging device 100 . In this case, the ISO sensitivity is a value representing the ability of the image sensor 100 to capture light. Increasing the gain also improves the ISO sensitivity. For example, if the gain is doubled, the electric signal (pixel signal) is also doubled, and even if the amount of incident light is halved, appropriate brightness can be obtained. However, increasing the gain also amplifies the noise contained in the electrical signal, resulting in increased noise.

Also, the thinning rate refers to the ratio of the number of pixels whose pixel signals are not read out to all the number of pixels in a predetermined area. For example, when the thinning rate of a predetermined area is 0, it means that pixel signals are read out from all pixels in the predetermined area. Further, when the thinning rate of a predetermined area is 0.5, it means that pixel signals are read out from half of the pixels in the predetermined area. Specifically, when the unit group 131 has a Bayer array, every other unit of the Bayer array in the vertical direction, that is, pixels from which pixel signals are alternately read out in units of two pixels (two rows each) and read out. Pixels that are not displayed are set. It should be noted that the resolution of the image decreases when the readout of the pixel signals is thinned out. However, since 20 million or more pixels are arranged in the imaging device 100, even if thinning is performed at a thinning rate of 0.5, an image can be displayed with 10 million or more pixels. Therefore, it is considered that the user (photographer) does not mind the decrease in resolution.

The number of rows to be added refers to the number of pixels in the vertical direction (the number of rows) to be added when pixel signals of pixels adjacent to each other in the vertical direction are added. The number of columns to be added refers to the number of pixels in the horizontal direction (the number of columns) to be added when pixel signals of horizontally adjacent pixels are added. Such addition processing is performed in the arithmetic circuit 416, for example. The arithmetic circuit 416 adds the pixel signals of a predetermined number of vertically or horizontally adjacent pixels, thereby providing the same effect as reading out pixel signals by thinning at a predetermined thinning rate. In addition, in the above-described addition processing, the average value may be calculated by dividing the added value by the number of rows or columns added by the arithmetic circuit 416 .

Also, the number of bits for digitization refers to the number of bits when the signal processing circuit 413 converts an analog signal into a digital signal in A/D conversion. As the number of bits of the digital signal increases, luminance, color change, etc. are expressed in more detail.

In the present embodiment, the accumulation condition refers to a condition regarding charge accumulation in the image sensor 100 . Specifically, the accumulation condition refers to the charge accumulation time or the number of accumulations, the frame rate, and the gain among the control parameters described above. The frame rate is included in the accumulation conditions because the frame rate can change according to the charge accumulation time and the number of accumulations. In addition, the amount of light for proper exposure changes according to the gain, and the charge accumulation time or number of times of accumulation may also change according to the amount of light for proper exposure. Therefore, gain is included in the accumulation condition.

In addition, in the present embodiment, the imaging conditions refer to conditions related to imaging of a subject. Specifically, the imaging conditions refer to control parameters including the accumulation conditions described above. The imaging conditions include control parameters for controlling the image sensor 100 (for example, charge accumulation time or number of times, frame rate, gain) and control parameters for controlling readout of signals from the image sensor 100 ( For example, a thinning rate, the number of rows or columns to be added for adding pixel signals), a control parameter for processing the signal from the image sensor 100 (for example, the number of bits for digitization, the image processing unit 30 to be described later) control parameters for executing) are also included.

FIG. 5 is a diagram showing the first region 200 and the second region 210 in the imaging device 100 of the first embodiment. As shown in FIG. 5, a pixel region 113A of the image sensor 100 (imaging chip 113) is divided into a first region 200 and a second region 210. As shown in FIG. The first area 200 is a rectangular area made up of a plurality of blocks (blocks indicated by "A" in FIG. 5) in the central portion of the pixel area 113A. The second area 210 is a frame-shaped area made up of a plurality of blocks (blocks indicated by "B" in FIG. 5) around the pixel area 113A. That is, the second area 210 is an area composed of a plurality of blocks arranged so as to surround the first area 200 . Each block forming the first area 200 and the second area 210 is made up of, for example, 120×60 pixels.

In FIG. 5, a small number of blocks are set in the pixel region 113A in order to make the arrangement of blocks in each region easier to see. 113A may be set.

In this embodiment, the first area 200 is a normal imaging area (imaging area). The second area 210 is a flicker detection area used to detect blinking (flicker) of a light source such as a fluorescent lamp or a mercury lamp (see light source L in FIG. 6), as will be described later. The commercial power frequency is 50 Hz in eastern Japan and 60 Hz in western Japan. A fluorescent lamp, for example, emits at twice that frequency. Therefore, a fluorescent lamp blinks 100 times per second in eastern Japan and 120 times per second in western Japan. The blinking of a light source such as a fluorescent lamp is called a flicker phenomenon.

When the digital camera 1 takes an image with a short exposure time (that is, a fast shutter speed), the image may become bright depending on the timing of the start of imaging (that is, the timing of pressing the shutter) when the fluorescent light blinks. , the image may become dark. Moreover, in the case of a fluorescent lamp, in addition to the change in brightness at 100 Hz or 120 Hz, the position where light is emitted also moves 50 or 60 times per second. Therefore, the way the image is shaded also shifts. In the case of the image sensor 100, the timing of reading out signals from pixels (green pixels Gb and Gr, blue pixels B, and red pixels R) differs for each color. Therefore, the difference in the amount of light for each color may appear as a color shift. In this embodiment, as shown in FIG. 5, the imaging device 100 is provided with a second area 210 in addition to the first area 200 for normal imaging. Then, the digital camera 1 of this embodiment uses the signal read out from the second area 210 to detect the blinking timing of the light source. Then, the digital camera 1 of the present embodiment takes an image at an optimal exposure start timing corresponding to the detected blinking timing.

FIG. 6 is a cross-sectional view showing a schematic configuration of a digital camera 1, which is an example of an imaging device. As shown in FIG. 6, it is assumed that the digital camera 1 of the present embodiment takes pictures in a room where a light source L such as a fluorescent lamp is installed.

A digital camera 1 of this embodiment includes a lens unit 10 and a camera body 2 . The lens unit 10 is an interchangeable lens. The camera body 2 is provided with a body-side mount section 80A for mounting the lens section 10 thereon. Further, the lens portion 10 is provided with a lens side mount portion 80B corresponding to the body side mount portion 80A. The lens section 10 is attached to the camera body 2 by the user joining the body-side mount section 80A and the lens-side mount section 80B. When the lens unit 10 is attached to the camera body 2, the electric contact 81A provided on the body side mount portion 80A and the electric contact 81B provided on the lens side mount portion 80B are electrically connected.

The lens unit 10 includes a photographing optical system 11 , a diaphragm 14 and a lens driving control device 15 . The imaging optical system 11 includes a lens 11a, a zooming lens 11b and a focusing lens 11c. The lens drive control device 15 has a lens-side CPU (Central Processing Unit), a memory, and a drive control circuit. The lens drive control device 15 is electrically connected to the system control section 70 on the side of the camera body 2 via electrical contacts 81A and 81B, and transmits lens information regarding the optical characteristics of the photographic optical system 11 provided in the lens section 10. and reception of control information for driving the zooming lens 11b, the focusing lens 11c, and the diaphragm .

The lens-side CPU of the lens drive control device 15 causes the drive control circuit to execute drive control of the focusing lens 11c based on the control information transmitted from the system control unit 70 in order to adjust the focus of the photographing optical system 11. . The lens-side CPU of the lens drive control device 15 causes the drive control circuit to execute drive control of the zooming lens 11b based on control information transmitted from the system control unit 70 in order to perform zooming adjustment. A diaphragm 14 is arranged along the optical axis of the imaging optical system 11 . The diaphragm 14 forms an aperture with a variable aperture diameter at the center of the optical axis for adjusting the amount of light and the amount of blurring. The lens-side CPU of the lens drive control device 15 causes the drive control circuit to control the drive of the diaphragm 14 based on the control information transmitted from the system control unit 70 in order to adjust the aperture diameter of the diaphragm 14 .

The camera body 2 includes an imaging section 20 , an image processing section 30 , a display section 50 , a storage section 60 and a system control section 70 . The imaging unit 20 has an imaging device 100 . A light flux emitted from the imaging optical system 11 of the lens unit 10 is incident on the imaging device 100 . The imaging device 100 photoelectrically converts an incident light beam to generate a pixel signal (a pixel signal is included in image data) for each pixel of the imaging device. RAW data (RAW data is also included in image data) consisting of pixel signals of pixels is sent from the imaging section 20 to the image processing section 30 . The image processing unit 30 performs various image processing on RAW data composed of pixel signals of pixels to generate image data in a predetermined file format (eg, JPEG format, etc.). The display unit 50 displays image data generated by the image processing unit 30 . The storage unit 60 stores image data generated by the image processing unit 30 .

Note that "image data" is sometimes referred to as "image signal". Images include still images, moving images, and live view images. A live view image is an image displayed on the display unit 50 by sequentially outputting image data generated by the image processing unit 30 to the display unit 50 . The live view image is used by the user to check the image of the subject captured by the imaging unit 20 . A live view image is also called a through image or a preview image.

A system control unit 70 controls the overall processing and operation of the digital camera 1 . Details of the processing and operation of the system control unit 70 and details of the internal configuration of the camera body 2 will be described with reference to FIG. 7 below.

FIG. 7 is a block diagram showing the configuration of the digital camera 1 according to the first embodiment. As shown in FIG. 7, the digital camera 1 includes a camera body 2 and a lens section 10. As shown in FIG. As described above, the lens unit 10 is an interchangeable lens that can be attached to and detached from the camera body 2 . Therefore, the digital camera 1 does not have to include the lens unit 10 . However, the lens unit 10 may be integrated with the digital camera 1 . The lens unit 10 guides the luminous flux from the subject to the imaging unit 20 while being connected to the camera body 2 .

The lens unit 10 has the lens driving control device 15 as described above (see FIG. 6). Also, the lens unit 10 has a plurality of lens groups as a photographing optical system 11, that is, a lens 11a, a zooming lens 11b, and a focusing lens 11c. The lens drive control device 15 transmits the lens information stored in the memory to the system control section 70 of the camera body 2 when the lens section 10 is connected to the camera body 2 . Further, the lens drive control device 15 receives control information transmitted from the system control section 70 when the lens section 10 is connected to the camera body 2 . The lens drive control device 15 performs drive control of the zooming lens 11b, the focusing lens 11c, and the diaphragm 14 based on the control information.

As shown in FIG. 7, the camera body 2 includes an imaging section 20, an image processing section 30, a work memory 40, a display section 50, an operation section 55, a recording section 60, a system control section 70 and a strobe 90.

The imaging unit 20 has an imaging device 100 and a driving unit 21 . The driving unit 21 is a control circuit that controls the driving of the imaging device 100 according to instructions from the system control unit 70 . Here, the driving unit 21 controls the timing (or timing cycle) of applying the reset pulse and the transfer pulse to the reset transistor 303 and the transfer transistor 302, respectively, thereby setting the charge accumulation time or the number of times of accumulation, which are control parameters. Control. Further, the drive unit 21 controls the frame rate by controlling the timing (or timing cycle) of applying the reset pulse, the transfer pulse, and the selection pulse to the reset transistor 303, the transfer transistor 302, and the selection transistor 305, respectively. do. Further, the drive unit 21 controls the thinning rate by setting the pixels to which the reset pulse, transfer pulse, and selection pulse are applied.

Further, the drive unit 21 controls the ISO sensitivity of the image sensor 100 by controlling the gain (also referred to as gain factor or amplification factor) of the amplifier 412 . Further, the drive unit 21 sets the number of addition rows or the number of addition columns for adding the pixel signals by sending an instruction to the arithmetic circuit 416 . Further, the drive unit 21 sets the number of bits for digitization by sending an instruction to the signal processing circuit 413 . Further, the driving section 21 sets the area in the pixel area (imaging area) 113A of the image sensor 100 in units of blocks. In this manner, the drive unit 21 functions as an image pickup device control unit that causes the image pickup device 100 to pick up images under different image pickup conditions for each of a plurality of blocks and output pixel signals. The system control unit 70 instructs the drive unit 21 about the position, shape, range, and the like of the block. In addition, the drive unit 21 sends an instruction to the arithmetic circuit 416 to cause each block of the second region 210 to execute a process of averaging pixel signals from a plurality of pixels included in each block. Arithmetic circuit 416 outputs an averaged signal obtained by averaging pixel signals from a plurality of pixels included in each block to image processing unit 30 .

The imaging device 100 transfers pixel signals from the imaging device 100 to the image processing unit 30 . Using the work memory 40 as a workspace, the image processing unit 30 performs various image processing on RAW data composed of pixel signals of pixels, and generates image data in a predetermined file format (for example, JPEG format). . The image processing unit 30 executes the following image processing. For example, the image processing unit 30 generates RGB image signals by performing color signal processing (color tone correction) on the signals obtained in the Bayer array. The image processing unit 30 also performs image processing such as white balance adjustment, sharpness adjustment, gamma correction, and gradation adjustment on the RGB image signal. Further, the image processing unit 30 performs processing for compressing in a predetermined compression format (JPEG format, MPEG format, etc.) as necessary. The image processing section 30 outputs the generated image data to the recording section 60 . The image processing unit 30 also outputs the generated image data to the display unit 50 .

Control parameters (imaging conditions) also include parameters referred to when the image processing unit 30 performs image processing. For example, the control parameters include parameters such as color signal processing (color tone correction), white balance adjustment, gradation adjustment, and compression rate. A signal read out from the imaging device 100 changes according to the charge accumulation time and the like, and the parameters referred to when image processing is performed also change according to the change of the signal. The image processing unit 30 sets different control parameters for each block and executes image processing such as color signal processing based on these control parameters.

Note that in the present embodiment, the image processing unit 30 determines the signal level of the averaged signal based on the averaged signal for each block of the second region 210 output from the arithmetic circuit 416 . The image processing unit 30 then outputs the determination result of the signal level of the averaged signal to the system control unit 70 .

The work memory 40 temporarily stores image data and the like when image processing is performed by the image processing unit 30 . The display unit 50 displays images (still images, moving images, live view images) captured by the imaging unit 20 and various information. The display unit 50 has a display panel 51 such as a liquid crystal display panel. A touch panel 52 is formed on the display panel 51 of the display unit 50 . The touch panel 52 outputs a signal indicating the position touched by the user to the system control unit 70 when the user performs an operation such as menu selection.

The operation unit 55 includes a release switch (switch pressed when shooting a still image), a moving image switch (switch pressed when shooting an action), and various operation switches operated by the user. The operation unit 55 outputs a signal to the system control unit 70 according to the user's operation. The recording unit 60 has a card slot into which a storage medium such as a memory card can be inserted. The recording unit 60 stores image data generated by the image processing unit 30 and various data in a recording medium loaded in the card slot. Also, the recording unit 60 has an internal memory. The recording unit 60 can also record image data and various data generated by the image processing unit 30 in the internal memory.

A system control unit 70 controls the overall processing and operation of the digital camera 1 . The system control unit 70 has a body-side CPU (Central Processing Unit). In this embodiment, the system control unit 70 divides the imaging surface (pixel region 113A) of the imaging element 100 (imaging chip 113) into a plurality of blocks, and sets different charge accumulation times (or charge accumulation times) and frame rates between the blocks. , to acquire an image with gain. Therefore, the system control unit 70 instructs the drive unit 21 about the position, shape, range of blocks, and storage conditions for each block.

In addition, the system control unit 70 acquires an image with a thinning rate different between blocks, the number of addition rows or addition columns for adding pixel signals, and the number of bits for digitization. For this reason, the system control unit 70 instructs the driving unit 21 on imaging conditions for each block (thinning rate, number of rows or columns to be added for adding pixel signals, and number of bits for digitization). Further, the image processing unit 30 executes image processing under different imaging conditions (color signal processing, white balance adjustment, gradation adjustment, control parameters such as compression ratio) between blocks. Therefore, the system control unit 70 instructs the image processing unit 30 on imaging conditions for each block (control parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate).

Also, the system control unit 70 causes the recording unit 60 to record the image data generated by the image processing unit 30 . Further, the system control unit 70 causes the display unit 50 to display an image by outputting the image data generated by the image processing unit 30 to the display unit 50 . The system control unit 70 also causes the display unit 50 to display an image by reading the image data recorded in the recording unit 60 and outputting it to the display unit 50 . Images displayed on the display unit 50 include still images, moving images, and live view images.

Further, the system control unit 70 outputs control information to the lens drive control device 15 to cause the lens drive control device 15 to execute drive control of the focusing lens 11 c and drive control of the diaphragm 14 .

In this embodiment, the system control section 70 includes a division section 71, a drive control section 72, a control section 73, and a blink detection section 74, as shown in FIG. The dividing unit 71 divides the pixel region 113A of the image sensor 100 into a first region 200 and a second region 210 in units of blocks as shown in FIG.

The drive control unit 72 sets imaging conditions for the first area 200 and sets imaging conditions for the second area 210 . Further, the drive control unit 72 executes drive control of the imaging device 100 according to the user's operation of the release switch and the moving image switch. The drive control unit 72 also executes drive control of the imaging element 100 when capturing a live view image (that is, after starting the shooting operation after power-on). The control unit 73 controls the display unit 50 to output the image data generated by the image processing unit 30 and display the image on the display panel 51 of the display unit 50 . The blink detection unit 74 detects the frequency and phase of blinking (flicker) of the light source L based on the signal read from the second area 210 .

In the system control unit 70, the division unit 71, the drive control unit 72, the control unit 73, and the blink detection unit 74 are implemented by the body-side CPU executing processing based on the control program.

Next, still image shooting operation of the digital camera 1 according to the first embodiment will be described. FIG. 8 is a flowchart for explaining the shooting operation executed by the system control unit 70 of the first embodiment. In the process shown in FIG. 8, when the user operates the operation unit 55 or the like to start shooting after the power of the digital camera 1 is turned on, the division unit 71 performs , divided into a first area 200 and a second area 210 as shown in FIG. Further, the drive control unit 72 sets imaging conditions for the first region 200 and the second region 210 divided by the dividing unit 71 (step S1).

For example, the drive control unit 72 sets, for example, 60 fps (the standard frame rate when capturing a live view image) as the frame rate of the first area 200 when capturing a live view image. In addition, the drive control unit 72 sets the frame rate of the second area 210 when capturing the live view image to be higher than the frame rate of the first area 200 (for example, a frame rate that is four times or more the frequency of the light source L). set. As an example, the drive control unit 72 drives and controls the second region 210 at a frame rate (eg, four times 400 fps) that is an integral multiple of the blinking frequency of the light source L (eg, 100 Hz).

In addition, the drive control unit 72 sets normal imaging conditions (for example, appropriate (imaging conditions that provide the correct exposure). In addition, the drive control unit 72 sets imaging conditions suitable for detecting flicker of the light source L as the imaging conditions (exposure time, gain, etc.) other than the frame rate of the second area 210 when capturing the live view image. .

In step S<b>1 , the dividing section 71 outputs an instruction signal to the driving section 21 to instruct the positions of blocks corresponding to the first area 200 and the second area 210 . Further, the drive control section 72 outputs an instruction signal for instructing the imaging conditions of the first area 200 and the second area 210 to the drive section 21 .

Next, the system control unit 70 starts photographing operation (step S2). That is, the drive control unit 72 outputs an instruction signal to the drive unit 21 to execute drive control of the imaging device 100 under the imaging conditions instructed in step S1. As a result, pixel signals read from each pixel in the first region 200 of the image sensor 100 are output to the image processing section 30 . In addition, pixel signals read from each pixel in the second region 210 of the image sensor 100 are averaged by the arithmetic circuit 416 . The arithmetic circuit 416 then outputs the averaged signal to the image processing section 30 . The image processing section 30 determines the signal level of the averaged signal output from the arithmetic circuit 416 and outputs the determination result to the system control section 70 .

When shooting is started, the control unit 73 causes the display panel 51 of the display unit 50 to display a live view image captured by the first area 200 of the image sensor 100 (step S3). FIG. 9 is a timing chart showing the blinking period of the light source L, the exposure time of the first area 200, and the signal output. In FIG. 9, the waveform shown at the top indicates the blinking cycle of the light source L. In FIG. The vertical axis represents the amount of light from the light source L, and the horizontal axis represents time. In the example shown in FIG. 9, the frequency of the light source L is 100 Hz (cycle is 10 ms).

FIG. 9 also shows the relationship between the exposure time A of the live view image of the first area 200 and the signal output A when the exposure time A is used for exposure. In the example shown in FIG. 9, the frame rate of the first area 200 is 60 fps (cycle is 16.667 ms), and the exposure time A is short (fast shutter speed). FIG. 9 also shows the relationship between the exposure time B of the live view image of the first area 200 and the signal output B when the exposure time B is used for exposure. In the example shown in FIG. 9, the frame rate of the first area 200 is 60 fps (cycle 16.667 ms), and the exposure time B is set to a long time (slow shutter speed).

As shown in FIG. 9, in the case of a short exposure time A, the signal output A (the pixel read from the first region 200 The level of the signal output A) changes significantly. Therefore, a flicker phenomenon occurs in the live-view image, and the flicker is an eyesore for the user.

As shown in FIG. 9, in the case of a long exposure time B, the influence of the blinking phase of the light source L at the exposure start timing is reduced, and the signal output B (output B of the pixel signal read from the first region 200) is reduced. Level changes are reduced. In addition, the occurrence of the flicker phenomenon can be suppressed by setting the exposure time to an integral multiple of the blinking period of the light source. In the example shown in FIG. 9, the exposure time B is set to the same time (one time) as the blinking cycle of the light source L, so no flicker phenomenon occurs in the live view image. Thus, in the live view image, setting a long exposure time reduces the effect of the flicker phenomenon, and setting the exposure time to an integer multiple of the flickering cycle of the light source suppresses the occurrence of the flicker phenomenon. or both. However, in capturing a still image based on the user's operation of the release switch, the exposure time is set according to the shooting environment. Therefore, if the exposure time when capturing a still image is short or if the exposure time is not an integral multiple of the blink cycle, the flicker phenomenon may occur.

Returning to the description of FIG. 8, the blinking detection unit 74 detects the blinking phase of the light source L based on the determination result of the signal level of the averaged signal from the image processing unit 30 during display of the live view image. (Step S4). FIG. 10 is a timing chart showing the blinking period of the light source L, the exposure time of the second area 210, and the signal output. As shown in FIG. 10 , the drive control unit 72 sets a higher frame rate than the frame rate of the first area 200 as the frame rate of the second area 210 . Further, the drive control section 72 sets a short exposure time as the exposure time of the second region 210 . In this case, as shown in FIG. 10, the signal output of the second region 210 appears at short intervals. Also, the level of the signal output from the second region 210 changes greatly according to the blinking phase of the light source L. FIG. That is, the signal output level of the second region 210 periodically repeats a change of increasing and decreasing. Therefore, the blink detection unit 74 can detect the blink frequency (blink cycle) and blink phase of the light source L based on the change in the signal output level of the second area 210 . Note that the higher the frame rate of the second region 210, the higher the accuracy of detection of the blinking frequency and blinking phase of the light source L by the blinking detection unit 74. For example, if the frame rate of the second area 210 is 1000 fps or more, the blink detection unit 74 can detect the blink cycle of the light source L's clear waveform.

The processing from steps S1 to S4 as described above is repeatedly executed until the release switch of the operation unit 55 is half-pressed (step S5) by the user. In this embodiment, the drive control unit 72 drives and controls the second area 210 at a frame rate changed in accordance with the blinking frequency of the light source L detected by the blinking detection unit 74 (see steps S1 and S2). For example, immediately after power-on, the drive control unit 72 sets the frame rate of the second area 210 to 1000 fps in step S1. After that, when the blink detection unit 74 detects that the blink frequency of the light source L is 100 Hz in step S4, the drive control unit 72 sets the frame rate of the second area 210 to 400 fps in step S1. According to such a configuration, immediately after the power is turned on, it is not possible to know what kind of waveform the flickering waveform of the light source L will have. A detection of the blinking frequency and blinking phase of the light source L is performed. On the other hand, after the blinking detection unit 74 detects the blinking frequency and blinking phase of the light source L, if the frame rate is an integral multiple of the blinking frequency of the light source L, the blinking detection unit 74 detects the blinking frequency of the light source L with high accuracy. and blink phase detection can be performed. Therefore, the frame rate of the second area 210 is lowered to reduce power consumption.

The system control unit 70 determines whether or not the release switch has been half-pressed (operation of SW1) (step S5). A half-press operation is used as an instruction to start shooting preparation. Although not shown in FIG. 8, the system control unit 70 performs autofocus (AF) control for automatic focus adjustment, automatic exposure (AE) control for automatic exposure adjustment, and the like. do.

Next, the system control unit 70 determines whether or not the release switch has been fully pressed (operation of SW2) (step S6). If it is determined that the full-press operation has been performed, the drive control unit 72 determines the optimum timing (best (referred to as "timing" in FIG. 10). In this embodiment, the best timing for starting exposure is set so that the exposure time is synchronized with the period in which the amount of light from the light source L is high. In the example shown in FIG. 10, the timing is before the peak of the amount of light from the light source L. However, the best timing for starting exposure changes depending on the exposure time (shutter speed). As the exposure time becomes shorter, the timing closer to the peak of the amount of light from the light source L becomes the best timing for starting the exposure. If the exposure time is longer than a certain amount, the influence of the exposure start timing on the still image is reduced. The drive control unit 72 adjusts the exposure start timing based on the determined best exposure start timing (step S7). Then, at the timing adjusted in step S7, the drive control unit 72 performs an imaging process that causes the imaging device 100 to perform imaging (step S8). Note that imaging based on the full-press operation in step S10 is referred to as main imaging.

In this embodiment, the imaging unit 20 performs main imaging under imaging conditions that provide proper exposure at the focal position based on autofocus and autoexposure control. At this time, the imaging unit 20 performs main imaging on the pixel area (imaging area) 113A of the imaging element 100 . That is, the imaging unit 20 causes not only the first region 200 in which the live view image was captured, but also the second region 210 used in detecting blinking of the light source L to capture the image. It should be noted that the imaging unit 20 may be configured to perform main imaging only in the first area 200 .

As described above, in the first embodiment, the drive control unit 72 that drives and controls the imaging device 100, the dividing unit 71 that divides the imaging device 100 into the first region 200 and the second region 210, and the drive control a flickering detection unit 74 that detects flickering of the light source L based on the signal read from the second area 210 by the unit 72 . According to such a configuration, by detecting blinking of the light source L in the partial second region 210 of the image sensor 100, it is possible to prevent the flicker phenomenon occurring in the image.

Further, in the first embodiment, the drive control unit 72 changes the imaging start timing based on blinking of the light source L detected by the blinking detection unit 74 . According to such a configuration, still images can be captured at optimum timing. Further, in the first embodiment, the drive control unit 72 includes the control unit 73 that causes the display unit 50 to display the live view image based on the signal read from the first area 200, and the flickering detection unit 74 controls the display of the live view image. During the operation, blinking of the light source L is detected. According to such a configuration, it is possible to take an image at an optimum timing that matches the flickering of the light source L at all times. In other words, a configuration in which the system control unit detects flickering of the light source L only immediately after the power is turned on and captures an image based on the detected flickering of the light source L is also conceivable. However, in such a configuration, the blinking phase of the light source L may deviate from the optimal imaging timing during display of the live view image. In the configuration as described above, the flickering detection unit 74 constantly detects the flickering phase of the light source L even while the live view image is being displayed. never. In addition, according to such a configuration, it is possible to perform image capturing at optimum timing not only for single shots of still images but also for continuous shots.

Further, in the first embodiment, the drive control unit 72 drives and controls the second area 210 at a frame rate higher than that of the first area 200 . With such a configuration, the flickering detection unit 74 can detect flickering of the light source L with high accuracy. Further, in the first embodiment, the drive control unit 72 drives and controls the second region 210 at a frame rate that is an integral multiple of the flickering frequency of the light source L. FIG. According to such a configuration, the blinking detection section 74 can accurately detect blinking of the light source L without increasing the frame rate more than necessary. Further, in the first embodiment, the drive control unit 72 drives and controls the second area 210 at a frame rate changed according to the blinking frequency of the light source L. FIG. According to such a configuration, it is possible to ensure accuracy in detecting the flickering of the light source L by the flickering detection unit 74 and reduce power consumption.

Further, in the first embodiment, the second area 210 is composed of a plurality of blocks and includes the signal processing section 416 that averages signals from a plurality of pixels included in each block. According to such a configuration, the blinking detection unit 74 can detect blinking of the light source L based on a stable signal. That is, the signal level of the pixel signal in each block of the second region 210 varies from pixel to pixel. Therefore, it is necessary for the blinking detector 74 to detect blinking of the light source L based on a signal whose signal level stably changes according to the blinking of the light source L. In the configuration as described above, by averaging the pixel signal of each pixel, variations in the signal level are reduced, and blinking of the light source L can be detected based on a stable signal. In addition, by averaging the signals in each block, one block has one data, so the amount of data can be reduced. Therefore, the burden on the image processing section 30 and the system control section 70 is reduced. In addition, since the dividing unit 71 sets the second area 210 around the first area 200, blinking of the light source L can be detected with high accuracy. That is, since it is not clear where the light source L has a strong light intensity (because there is unevenness), even if the light intensity from the light source L is detected only in a biased specific area of the image sensor 100, the flickering of the light source L can be accurately detected. may not be possible. With the configuration as described above, blinking can be detected regardless of the intensity of light from the light source L. FIG.

As shown in FIG. 7, in the first embodiment, the digital camera 1 has been described as an example of the imaging device. However, the imaging device is not limited to the digital camera 1, and may be composed of a device such as a smart phone, a mobile phone, or a personal computer having an imaging function. Further, in the digital camera 1 according to the first embodiment shown in FIG. 7, the display section 50 may be arranged outside the imaging device. In this case, each of the system control unit 70 and the display unit 50 is provided with a communication unit that transmits and receives signals (image data, control signals, etc.) in a wired or wireless manner. Further, the image processing section 30 and the system control section 70 may be integrated. In this case, the system control unit having one body-side CPU performs the functions of the image processing unit 30 and the system control unit 70 by executing processing based on the control program.

Further, in the above-described first embodiment, the signal processing unit 416 for averaging the pixel signals of each block of the second region 210 is provided in the imaging unit 20, but it may be provided in the image processing unit 30. Further, although the flickering detection section 74 is provided in the system control section 70 , it may be provided in the imaging section 20 or the image processing section 30 . That is, the signal processing unit (arithmetic circuit) 416 of the imaging unit 20 may have a function of detecting flickering of the light source L, or the image processing unit 30 may have a function of detecting flickering of the light source L. In this case, the blink detection unit in the imaging unit 20 or the image processing unit 30 detects the blink frequency and blink phase of the light source L and outputs a detection signal to the system control unit 70 .

Further, in the above-described first embodiment, the drive control unit 72 adjusts the timing of starting to capture a still image based on the blinking of the light source L detected by the blinking detection unit 74 (step S4, S7). However, without being limited to such a configuration, the drive control section 72 may adjust the timing of starting imaging of the live view image based on blinking of the light source L detected by the blinking detection section 74 . In this manner, the drive control unit 72 can expose the live view image at the best timing, so that the sensitivity of the live view image can be increased, and a live view image with an appropriate white balance can be obtained.

<Second embodiment>
In the above-described first embodiment, the dividing section 71 divides the imaging device 100 into two regions, the first region 200 and the second region 210 . On the other hand, in the second embodiment, the dividing unit 71 divides the imaging device 100 into three regions, the first area 200, the area 210A, and the area 210B. In the second embodiment, the area 210A is called the third area, and the area 210B is called the fourth area. The third area 210A and the fourth area 210B correspond to the second area 210 of the first embodiment. Therefore, in the second embodiment, the dividing section 71 further divides the second area 210 into the third area 210A and the fourth area 210B.

FIG. 11 is a diagram showing the first area 200, the third area 210, and the fourth area 211 in the imaging device 100 of the second embodiment. As shown in FIG. 11, the pixel area 113A of the image sensor 100 is divided into a first area 200, a third area 210A and a fourth area 210B. The first area 200 is a rectangular area made up of a plurality of blocks (blocks indicated by "A" in FIG. 11) in the central portion of the pixel area 113A. The third area 210A is an area composed of a plurality of blocks (blocks indicated by "B" in FIG. 5) in the peripheral portion of the pixel area 113A. That is, the third area 210A is an area composed of a plurality of blocks arranged so as to surround the first area 200. As shown in FIG. The fourth area 210B is an area composed of a plurality of blocks (blocks indicated by "C" in FIG. 5) in the peripheral portion of the pixel area 113A. That is, the fourth area 210B is an area composed of a plurality of blocks arranged so as to surround the first area 200. As shown in FIG. As described above, the third area 210A and the fourth area 210B correspond to the second area 210. FIG. Each block constituting the first area 200, the third area 210A, and the fourth area 210B is composed of, for example, 120×60 pixels.

In FIG. 11, a small number of blocks are set in the pixel region 113A in order to make the arrangement of blocks in each region easier to see. 113A may be set.

In this embodiment, the first area 200 is a normal imaging area (imaging area). The third area 210A and the fourth area 210B are both flicker detection areas used to detect blinking (flicker) of the light source (see light source L in FIG. 6). However, different frame rates are set for the third area 210A and the fourth area 210B.

FIG. 12 is a timing chart showing the blinking cycle of the light source L, the exposure time and signal output of the third area 210A, and the exposure time and signal output of the fourth area 210B. As shown in FIG. 12, the drive control unit 72 sets a frame rate higher than the frame rate of the first area 200 (for example, a frame rate four times 100 Hz) as the frame rate of the third area 210A. Further, the drive control unit 72 sets the frame rate of the fourth area 210B to be higher than the frame rate of the first area 200 and different from the frame rate of the third area 210A (for example, a frame rate four times as high as 120 Hz). ). According to such a configuration, the flickering detection unit 74 reliably detects the flickering frequency and flickering cycle of the light source L for different flickering frequencies (100 Hz and 120 Hz) corresponding to different commercial power frequencies (50 Hz and 60 Hz). can be detected.

Note that the frame rate of the third area 210A and the frame rate of the fourth area 210B are not limited to the frame rate corresponding to the frequency of the commercial power supply. For example, the drive control unit 72 sets the frame rate of the third area 210A and the frame rate of the fourth area 210B to the same frame rate, but sets the phases of these frame rates to be shifted by 90 degrees. With such a configuration, the blinking phase of the light source L can be detected even more easily.

As described above, in the second embodiment, the division unit 71 divides the second area 210 into the third area 210A and the fourth area 210B, and the drive control unit 72 divides the third area 210A and the fourth area 210B into the third area 210A and the fourth area 210B. are driven and controlled at different frame rates. According to such a configuration, blinking of the light source L at various frequencies can be detected with high accuracy.

<Third Embodiment>
In the above-described first embodiment, the blink detection unit 74 executes blink detection during display of the live view image (see step S4 in FIG. 8). On the other hand, in the third embodiment, the blink detection unit 74 executes blink detection in response to the user's half-pressing of the release switch.

FIG. 13 is a flowchart for explaining the shooting operation executed by the system control unit 70 of the third embodiment. As shown in FIG. 13, when the system control unit 70 determines that the release switch has been half-pressed by the user (YES in step S5), the blinking detection unit 74 detects blinking of the light source L ( step S11). Note that the other processes are the same as those described with reference to FIG. 8, so description thereof is omitted. According to such a configuration, there is no need to perform drive control in the second area 210 (the third area 210A and the fourth area 210B in the second embodiment) during display of the live view image, thereby reducing power consumption. be. As in the first embodiment, the blinking detection unit 74 detects blinking of the light source L during display of the live view image, and also detects blinking of the light source L when the release switch is half-pressed. It may be configured to detect

Various timings are conceivable as the timing (opportunity) at which the flickering detection unit 74 executes the process of detecting flickering of the light source L. FIG. For example, when the user moves between rooms, the light source L may change. This is the case when the light source moves from a room with fluorescent lights to a room with LED lights. In this case, the image processing unit 30 may detect that the white balance of the image has changed, thereby determining that the room has been moved, and the flickering detection unit 74 may detect flickering of the light source L. The digital camera 1 may be provided with a GPS function (Global Positioning System), and the flickering detection unit 74 may detect flickering of the light source L in response to detection of movement of a predetermined distance or more by GPS. Also, while the live view image is being displayed, the frame rate of the second area 210 may be set to four times the frequency of the commercial power supply, and the frame rate of the second area 210 may be set to eight times the frequency of the commercial power supply in response to the half-press operation. good.

<Fourth Embodiment>
In the first embodiment described above, the arithmetic circuit 416 averages the pixel signals of each block in the second area 210 . On the other hand, in the fourth embodiment, an arithmetic circuit (block determination unit) 416 determines a block with a high blink contrast of the light source L based on the pixel signal of each block in the second area 210 . Then, the blink detection unit 74 detects blinking of the light source L based on the signal level from the block determined by the arithmetic circuit 416 .

Specifically, a pixel signal read from each pixel in the second region 210 of the image sensor 100 is output to the arithmetic circuit 416 . The arithmetic circuit 416 detects the contrast (high and low) of blinking of the light source L based on the signal level of each pixel signal. Arithmetic circuit 416 determines the block with the highest blinking contrast of light source L. FIG. Arithmetic circuit 416 averages the pixel signals of the pixels in the block in which the blinking contrast of light source L is highest. The arithmetic circuit 416 then outputs the averaged signal to the image processing section 30 . The image processing section 30 determines the signal level of the averaged signal output from the arithmetic circuit 416 and outputs the determination result to the system control section 70 . According to such a configuration, the blinking detection accuracy of the blinking detection unit 74 is further improved.

<Fifth Embodiment>
In the above-described first embodiment, as shown in FIG. 6 and the like, one image sensor 100 is divided into a shooting area (first area 200) and a blinking detection area (second area 210) of the light source L. had split. In contrast, in the fifth embodiment, an imaging device 400 for detecting flickering of the light source L is provided separately from the imaging device 100 for photographing.

FIG. 14 is a diagram showing the arrangement of two imaging elements 100 and 400 according to the fifth embodiment. As shown in FIG. 14 , the light beam guided from the imaging optical system 11 passes through the half mirror 300 , part of which reaches the image sensor 100 , and the other part is reflected by the half mirror 300 to reach the image sensor 400 . to reach The imaging element 400 is an imaging element for detecting blinking of the light source L. FIG. The blinking detection unit 74 detects the blinking frequency and blinking phase of the light source L based on the signal from the imaging device 400 . In this fifth embodiment, the image pickup device 100 is called a first image pickup device, and the image pickup device 400 is called a second image pickup device. With such a configuration as well, by detecting blinking of the light source L by the image sensor 400 different from the image sensor 100, it is possible to prevent the flicker phenomenon occurring in the image.

<Sixth embodiment>
In the sixth embodiment, the digital camera 1 in the first embodiment is separated into an imaging device 1A and an electronic device 1B.

FIG. 15 is a block diagram showing configurations of an imaging device 1A and an electronic device 1B according to the sixth embodiment. In the configuration shown in FIG. 15, an imaging device 1A is a device that takes an image of a subject. This imaging apparatus 1A includes a lens section 10, an imaging section 20, an image processing section 30, a work memory 40, an operation section 55, a recording section 60, and a first system control section 70A. The configuration of the lens unit 10, the imaging unit 20, the image processing unit 30, the work memory 40, the operation unit 55, and the recording unit 60 in the imaging apparatus 1A is the same as the configuration shown in FIG. Therefore, the same configurations are denoted by the same reference numerals, and overlapping descriptions are omitted.

The electronic device 1B is a device that displays images (still images, moving images, live view images). The electronic device 1B includes a display section 50 and a second system control section (control section) 70B. Note that the configuration of the display unit 50 in the electronic device 1B is the same as the configuration shown in FIG. Therefore, the same configurations are denoted by the same reference numerals, and overlapping descriptions are omitted.

The first system control section 70A has a first communication section 75A. The second system control section 70B also has a second communication section 75B. The first communication section 75A and the second communication section 75B transmit and receive signals to and from each other by wire or wirelessly. In such a configuration, the first system control unit 70A transmits image data (image data processed by the image processing unit 30, image data recorded in the recording unit 60) to the second system via the first communication unit 75A. It is transmitted to the communication section 75B. The second system control unit 70B causes the display unit 50 to display the image data received by the second communication unit 75B. The second system control unit 70B also causes the second display unit 53 to display a preset menu image.

Further, the second system control unit 70B performs control to change the imaging conditions (including accumulation conditions) according to the user's touch operation on the touch panel 52 or automatically. Further, the second system control unit 70B performs control to select each display area in the image display area 510 according to the touch operation of the touch panel 52 by the user or automatically. In addition, the first system control unit 70A executes imaging control in accordance with the user's operation of the operation unit 55 (an operation unit provided on the electronic device 1B side for instructing the start of imaging of a still image or a moving image) 55 .

The configuration shown in FIG. 7 (dividing section 71, drive control section 72, control section 73, and blink detection section 74) may be provided in either the first system control section 70A or the second system control section 70B. All the configuration shown in FIG. 7 may be provided in the first system control unit 70A or the second system control unit 70B, and part of the configuration shown in FIG. 7 may be provided in the first system control unit 70A, 7 may be provided in the second system control unit 70B.

The imaging device 1A is composed of, for example, a digital camera, a smart phone, a mobile phone, a personal computer, etc., having an imaging function and a communication function, and the electronic device 1B is, for example, a smart phone, a mobile phone, a portable personal computer, etc. It consists of portable terminals such as computers.

The first system control unit 70A shown in FIG. 15 is implemented by the body-side CPU executing processing based on a control program. Also, the second system control unit 70B shown in FIG. 15 is implemented by the body-side CPU executing processing based on the control program.

As described above, in the sixth embodiment, in addition to the effects described in the first embodiment, blinking of the light source is achieved using a live view image captured by the imaging device 1A using a portable terminal such as a smartphone. can be detected.

In the configuration shown in FIG. 15, the image processing section 30 and the first system control section 70A may be integrated. In this case, a system control unit having one body-side CPU performs processing based on a control program to perform the function of the image processing unit 30 and the function of the first system control unit 70A.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various modifications and improvements can be made to the above embodiments without departing from the scope of the present invention. Also, one or more of the requirements described in the above embodiments may be omitted. Such modifications, improvements, and omissions are also included in the technical scope of the present invention. Moreover, it is also possible to appropriately combine the configurations of the above-described embodiments and modifications and apply them.

For example, although the arrangement of the color filters 102 is the Bayer arrangement in each of the above-described embodiments, an arrangement other than this arrangement may be used. Also, the number of pixels forming the unit group 131 should include at least one pixel. Also, it is sufficient that each block includes at least one pixel. Therefore, it is possible to perform imaging under different imaging conditions for each pixel.

Further, in each of the above-described embodiments, the drive unit 21 may be partially or wholly mounted on the imaging chip 113 , or may be partially or wholly mounted on the signal processing chip 111 . Also, part of the configuration of the image processing unit 30 may be mounted on the imaging chip 113 or the signal processing chip 111 . Also, part of the configuration of the system control unit 70 may be mounted on the imaging chip 113 or the signal processing chip 111 .

Further, in each of the above-described embodiments, a still image is captured at an optimum timing based on the blinking frequency and blinking phase of the light source L detected by the blinking detection unit 74 . However, a frame rate that does not cause the flicker phenomenon may be determined based on the blinking frequency and blinking phase of the light source L detected by the blinking detection unit 74, and a moving image may be captured at the determined frame rate.

Further, in the above-described first embodiment, the imaging device 100 is divided into two regions, the first region 200 and the second region 210. In the second embodiment, the imaging device 100 is divided into the first region 200, the third region 210A, and the fourth region 210A. Although the region 210B is divided into three, it may be divided into four or more. Moreover, although the second area 210 (the third area 210A and the fourth area 210B) is arranged around the first area 200, the position is not limited to such. For example, it may be arranged in the central portion, or may be arranged evenly over the entire surface of the imaging device 100 .

<Seventh embodiment>
In the seventh embodiment, images include still images, moving images, live view images, and partial moving images. First, the partial moving image will be described. FIG. 16 is a diagram for explaining an outline of a partial moving image. An image 1500 shown in FIG. 16 includes a person O10 as a subject and an earring O20 as a subject worn on the ear of the person O10. In the image 1500, the earring O20 and its surrounding area (a slightly wider area than the earring O20) 1501 are composed of moving images. In the example shown in FIG. 16, the moving image of the area 1501 is the moving image of the earring O20 swinging. On the other hand, the area of the image 1500 other than the area 1501 (that is, the area in which the person O10 is shown) is composed of a still image. Therefore, the image of the area other than the area 1501 is stopped. It appears to the user that only the earring O20 is moving in the image 1500 . An image in which part of the image is a moving image and the other part is a still image is called a "partial moving image". A "partial moving image" is also called a "cinema photo" or a "cinemagraph." A partial moving image is a method of partially moving a photograph (still image) to draw attention to the moving part. For the user, in a picture where time has stopped, only the parts that are moving appear to be moving forward. In this embodiment, a partial moving image is generated using the image sensor 1100 described above.

FIG. 17 is a block diagram showing the configuration of a digital camera 1001 according to the seventh embodiment. 17, a digital camera 1001 as an example of electronic equipment corresponds to the digital camera 1 in FIG. 7, a lens unit 1010 corresponds to the lens unit 10 in FIG. 7, and an imaging unit 1020 corresponds to the imaging unit 20 in FIG. , the image sensor 1100 corresponds to the image sensor 100 in FIG. 7, the driving unit 1021 corresponds to the driving unit 21 in FIG. 7, the image processing unit 1030 corresponds to the image processing unit 30 in FIG. 7, the display unit 1050 corresponds to the display unit 50 in FIG. 7, the display panel 1051 corresponds to the display panel 51 in FIG. 7, and the touch panel 1052 corresponds to the touch panel 52 in FIG. 7, the recording unit 1060 corresponds to the storage unit 60 in FIG. 7, and the system control unit 1070 corresponds to the system control unit 70 in FIG. Of these configurations, the configurations of the lens unit 1010, imaging unit 1020, imaging device 1100, driving unit 1021, work memory 1040, display unit 1050, operation unit 1055, and recording unit 1060 are omitted.

The image processing unit 1030 includes an image generation unit 1031 and a detection unit 1032, as shown in FIG. The image generation unit 1031 generates image data by performing various image processing on RAW data composed of pixel signals of pixels output from the imaging unit 1020 . In this embodiment, the image generation unit 1031 can generate not only image data of moving images (including live view images) and still images, but also image data of the partial moving images described above. A detection unit 1032 detects a subject from the image data generated by the image generation unit 1031 . In this embodiment, the detection unit 1032 has a function of comparing a plurality of image data (frames) obtained in time series from the image generation unit 1031 and detecting a moving subject (moving subject). Further, the detection unit 1032 has a function of detecting the subject by specifying the boundary of the subject based on the contrast and color change between the bright and dark areas in the image data. Note that the detection unit 1032 may detect a human body included in the image data as a subject, as described in Japanese Patent Application Laid-Open No. 2010-16621 (US2010/0002940), for example.

Here, the subject, which is the object to be imaged by the imaging unit 1020, is not limited to the case where there is only one in the image data, and there may be a plurality of subjects. Among subjects, a subject that attracts the user's (photographer's) attention or a subject that is presumed to attract the user's attention is referred to as a "main subject". The main subject is also not limited to the case where there is only one in the image data, and there may be a plurality of main subjects.

In this embodiment, by performing a touch operation on the touch panel 1052 formed on the display panel 1051, focus adjustment (AF: Automatic Focusing) is performed on the touched subject.

A system control unit 1070 controls the overall processing and operation of the digital camera 1001 . The system control unit 1070 has a body-side CPU (Central Processing Unit). In this embodiment, the system control unit 1070 divides the imaging surface (pixel region 113A) of the imaging element 1100 (imaging chip 113) into a plurality of blocks, and sets different charge accumulation times (or charge accumulation times) and frame rates between the blocks. , to acquire an image with gain. Therefore, the system control unit 1070 instructs the driving unit 1021 about the position, shape, range of blocks, and storage conditions for each block.

In addition, the system control unit 1070 acquires an image with a thinning rate that differs between blocks, the number of addition rows or addition columns for adding pixel signals, and the number of bits for digitization. Therefore, the system control unit 1070 instructs the driving unit 1021 about imaging conditions for each block (decimation rate, number of rows or columns to be added for adding pixel signals, and number of bits for digitization). In addition, the image processing unit 1030 executes image processing under different imaging conditions (control parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression ratio) between blocks. Therefore, the system control unit 1070 instructs the image processing unit 1030 on imaging conditions for each block (control parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate).

Also, the system control unit 1070 causes the recording unit 1060 to record the image data generated by the image processing unit 1030 . Further, the system control unit 1070 causes the display unit 1050 to display an image by outputting the image data generated by the image processing unit 1030 to the display unit 1050 . Further, the system control unit 1070 reads the image data recorded in the recording unit 1060 and causes the display unit 1050 to output the image data, thereby causing the display unit 1050 to display an image. Images displayed on the display unit 1050 include still images, moving images, live view images, and partial moving images (see FIG. 16).

Further, the system control unit 1070 outputs control information to the lens drive control device 15 to cause the lens drive control device 15 to execute drive control of the focusing lens 11 c and drive control of the diaphragm 14 .

In this embodiment, the system control unit 1070 includes a display control unit 1071, a setting unit (frame rate setting unit) 1072, an imaging control unit 1073, and an AF processing unit 1074, as shown in FIG. The display control unit 1071 outputs the image data generated by the image processing unit 1030 to the display unit 1050, and displays an image (live view image, still image, moving image, partial moving image) on the display panel 1051 of the display unit 1050. control. Further, the display control unit 1071 performs control to display a preset menu image (see FIG. 18) on the display panel 1051 of the display unit 1050 .

Based on the user's touch operation on touch panel 1052, setting unit 1072 sets a region of pixel region 113A corresponding to a region corresponding to a subject detected by detection unit 1032 (hereinafter referred to as a subject region) as a moving image capturing region. (hereinafter referred to as a moving image area) or an area for capturing a still image (hereinafter referred to as a still image area). Further, the setting unit 1072 controls to set the area of the pixel area 113A corresponding to the area selected by the user's touch operation on the touch panel 1052 (hereinafter referred to as the selected area) as a moving image area or a still image area in units of blocks. I do. In addition, the setting unit 1072 performs control for setting imaging conditions including a frame rate in accordance with a user's touch operation on the touch panel 1052 or automatically.

The imaging control unit 1073 drives and controls the imaging element 1100 so as to perform imaging under the imaging conditions set by the setting unit 1072 . Here, the imaging control unit 1073 drives and controls the imaging device 1100 so as to capture a live view image at the start of the imaging operation. In addition, the imaging control unit 1073 drives and controls the imaging device 1100 so as to capture a moving image in the moving image area set by the setting unit 1072 . Further, the imaging control unit 1073 performs drive control of the imaging device 1100 so as to capture a still image in the still image area set by the setting unit 1072 . The imaging control unit 1073 also performs control to record image data of moving images, still images, and partial moving images generated by the image generating unit 1031 in the recording unit 1060 .

The AF processing unit 1074 executes focus adjustment processing (AF processing) called contrast AF in response to a user's touch operation on the touch panel 1052 so as to maximize the contrast of the subject. Specifically, the AF processing unit 1074 causes the lens drive control device 15 to move the focusing lens 11c by transmitting control information to the lens drive control device 15 via the electrical contacts 81A and 81B. Further, the AF processing unit 1074 outputs a contrast signal representing the contrast of the subject image captured by the imaging unit 1020 (in this embodiment, the contrast of the subject image selected by the user: see steps S1029 and S1030 in FIG. 20). is acquired from the image processing unit 1030 . The AF processing unit 1074 moves the focusing lens 11c, and based on the contrast signal from the image processing unit 1030, detects the position of the focusing lens 11c at which the contrast of the object image is highest as the focal position. The AF processing unit 1074 transmits control information to the lens driving control device 15 so as to move the focusing lens 11c to the detected focal position.

In the system control unit 1070, the display control unit 1071, the setting unit 1072, the imaging control unit 1073, and the AF processing unit 1074 are implemented by the body-side CPU executing processing based on the control program.

FIG. 18 is a diagram showing a display section according to the digital camera of the seventh embodiment. As shown in FIG. 18, the display panel 1051 of the display unit 1050 is provided with an image display area 1510 and an operation button display area 1520 . The image display area 1510 is an area for displaying images captured by the imaging unit 1020, that is, still images, moving images, live view images, and partial moving images. Further, the operation button display area 1520 is an area for displaying a menu image 1520M for the user to set imaging conditions and the like.

A plurality of blocks B(i, j) are set in the image display area 1510 . In the example shown in FIG. 18, in the image display area 1510, eight blocks (i=1 to 8) are set in the image display area 1510 (horizontal direction in FIG. 18), and eight blocks (i=1 to 8) are set in the vertical direction ( Six blocks (j=1 to 6) are set in the vertical direction of FIG. That is, the image display area 1510 is divided into 8×6 (=48) blocks. Also, a touch panel 1052 is provided on the image display area 1510 . In touch panel 1052, a plurality of touch areas P(i, j) are formed so as to overlap each of a plurality of blocks B(i, j). When each touch area P(i, j) detects that it has been pressed (touched) by the user, it sends a detection signal indicating the pressed position (which touch area) to the system control unit 1070. Output.

A plurality of blocks B(i, j) set in the image display area 1510 correspond to a plurality of blocks set in the above-described pixel area 113A (see FIG. 2). However, the number of blocks set in the pixel area 113A may be greater than or less than the number of blocks set in the image display area 1510. FIG. In FIG. 18, a small number of blocks are set in the image display area 1510 in order to make the arrangement of the blocks in the image display area 1510 easier to see. A block may be set in the image display area 1510 .

Operation button display area 1520 is provided near image display area 1510 . Within the operation button display area 1520, a menu image 1520M is displayed for the user to set the shooting mode, areas (moving image area, still image area), imaging conditions, and execution of AF processing. The menu image 1520M includes a shooting mode button image 1521, a touch moving image button image 1522, a touch still image button image 1523, a frame rate button image 1524, and an AF button image 1525. The shooting mode button image 1521 is used when the user selects a mode for shooting normal moving images and still images (normal mode) and a mode for shooting partial moving images (partial moving image mode). The touch moving image button image 1522 is used when the user sets a moving image area (eg, area 1511 in FIG. 21 described later) in the image display area 1510 . The touch still image button image 1523 is used when the user sets a still image area (eg, area 1512 in FIG. 24 described later) in the image display area 1510 . A frame rate button image 1524 is used when the user sets the frame rate of the moving image. The AF button image 1525 is used when the user executes AF processing on a subject specified by a touch operation on the touch panel 1052 .

On the touch panel 1052 , a touch area 1521 a is formed so as to overlap the shooting mode button image 1521 . A touch area 1522 a is formed so as to overlap the touch moving image button image 1522 . A touch area 1523 a is formed so as to overlap the touch still image button image 1523 . A touch area 1524 a is formed so as to overlap the frame rate button image 1524 . A touch area 1525 a is formed so as to overlap the AF button image 1525 . When the touch area 1521a, the touch area 1521a, the touch area 1521a, and the touch area 1525a detect that they have been pressed (touched) by the user, they generate a detection signal indicating the pressed position (which touch area). is output to the system control unit 1070 .

In the embodiments (seventh embodiment, eighth and ninth embodiments to be described later), the “first region” means that a setting unit is automatically set in response to a user's operation in the image display region 1510 of the display panel 1051. It means an area within the pixel area 113A of the image sensor 1100 corresponding to the area (moving image area or still image area) set by 1072 . In addition, the “second area” is an area (moving image area or still image area) set by the setting unit 1072 automatically or according to the user's operation in the image display area 1510 of the display panel 1051. It refers to a region within the pixel region 113A of the element 1100. FIG. Both the first area and the second area are not limited to one area, and may be a plurality of areas.

Next, the partial moving image capturing operation of the digital camera 1001 according to the seventh embodiment will be described. FIG. 19 is a flow chart for explaining the partial moving image shooting operation executed by the system control unit 1070 of the seventh embodiment.

When the user selects the partial moving image mode as the shooting mode, the partial moving image shooting operation (processing of steps S1001 to S1012) shown in FIG. 19 is started. Selection of the shooting mode by the user is performed, for example, as follows. When the digital camera 1001 is powered on, the display control unit 1071 displays the screen shown in FIG. 18 on the display panel 1051 of the display unit 1050 . The user touches the shooting mode button image 1521 (that is, the touch area 1521a). Touch panel 1052 outputs a detection signal corresponding to touch area 1521a touched by the user to system control section 1070 . Based on the detection signal corresponding to the touch area 1521a, the display control unit 1071 displays "normal mode" and "partial moving image mode" as images of types of shooting modes next to the imaging mode button image 1521. FIG. The setting unit 1072 newly sets a touch area on the touch panel 1052 so as to overlap the area of the image of the shooting mode type. The user touches "partial moving image mode" as an image of the type of shooting mode. Touch panel 1052 outputs to system control section 1070 a detection signal corresponding to the touch area on the “partial moving image mode” touched by the user. The setting unit 1072 sets “partial moving image mode” as the type of shooting mode. After that, the processing shown in FIG. 19 is started.

In the process shown in FIG. 19, first, the imaging control unit 1073 starts the operation of capturing a live view image (step S1001). When the live view image capturing is started, the display control unit 1071 displays the live view image captured by the image capturing unit 1020 in the image display area 1510 of the display panel 1051 (step S1002). For example, a live view image of a waterfall is displayed in the image display area 1510 (see FIGS. 21, 23, and 24, which will be described later).

Next, the setting unit 1072 determines whether or not the user has performed a touch operation on the touch moving image button image 1522 or the touch still image button image 1523 (step S1003). When setting the moving image area in the image display area 1510, the user touches the touch moving image button image 1522 (that is, the touch area 1522a) with a finger. Further, when the user sets a still image area in the image display area 1510, the user touches the touch still image button image 1523 (that is, the touch area 1523a) with a finger. When the user touches touch moving image button image 1522 or touch still image button image 1523 , touch panel 1052 outputs a detection signal corresponding to touch area 1522 a or 1523 a touched by the user to system control unit 1070 . If the setting unit 1072 determines that the user has performed a touch operation on the touch video button image 1522 or the touch still image button image 1523, the setting unit 1072 performs setting processing for setting an area (moving image area, still image area), imaging conditions, and the like. (step S1004). Note that AF processing by the AF processing unit 1074 is also executed in the setting processing in step S1004.

(1) When the user touches the touch video button image 1522:
FIG. 20 is a flowchart for explaining the setting process (step S1004) shown in FIG. FIG. 21 is a diagram showing a display example of the display panel 1051 when the user sets the moving image area 1511. As shown in FIG. Also, FIG. 22 is a diagram showing a moving image area 1211 in the pixel area 113A of the image sensor 1100. As shown in FIG. Also, FIG. 23 is a diagram showing a display example of the display panel 1051 when the user sets the frame rate.

As shown in FIG. 21, the display control unit 1071 displays “auto” and “manual” as the area setting mode image 1522v next to the touch moving image button image 1522 based on the detection signal corresponding to the touch area 1522a. Let Here, “auto” is an area setting mode (automatic mode) in which the setting unit 1072 automatically sets the moving image area. "Manual" is an area setting mode (manual mode) in which the user manually sets the moving image area. The setting unit 1072 newly sets a touch area on the touch panel 1052 so as to overlap with the area of the image 1522v in the area setting mode (“auto” and “manual” areas). The user touches "auto" or "manual" as the area setting mode image 1522v. The touch panel 1052 outputs a detection signal corresponding to the “auto” or “manual” touch area touched by the user to the system control unit 1070 .

In the process shown in FIG. 20, the setting unit 1072 determines whether the user has selected "auto" or "manual" as the area setting mode based on the detection signal corresponding to the "auto" or "manual" touch area. It is determined whether or not it has been done (step S1021). If the setting unit 1072 determines that the user has selected “auto” as the area setting mode (YES in step S1021), the area within the image display area 1510 is selectable, that is, the touch panel 1052 is touch operated. to be discoverable. Thereafter, the user selects a subject displayed in image display area 1510 by touching block B(i, j) of the subject displayed in image display area 1510 with a finger. Touch panel 1052 outputs a detection signal corresponding to touch area P(i, j) touched by the user to system control section 1070 .

Setting unit 1072 recognizes block B(i, j) selected by the user based on the detection signal from touch panel 1052 . In the example shown in FIG. 21, it is assumed that the user has selected a predetermined block B(i, j) within area 1511 (the area with the line drawn in FIG. 21). The setting unit 1072 outputs a position signal indicating the position of the predetermined block B(i,j) selected by the user to the image processing unit 1030, thereby causing the detection unit 1032 to detect the subject (step S1022). ). The detector 1032 recognizes the block at the position indicated by the position signal from the system controller 1070 . Then, the detection unit 1032 detects a subject including the recognized block. Specifically, the detection unit 1032 compares a plurality of image data obtained in time series from the image generation unit 1031 to detect a moving subject (waterfall in FIG. 21). Alternatively, the detection unit 1032 detects the subject by identifying the boundary of the subject (the waterfall in FIG. 21) based on the contrast between the bright and dark areas and the color change in the image data. The detection unit 1032 outputs a subject detection signal indicating the position and range of the detected subject to the system control unit 1070 .

The setting unit 1072 recognizes the position and range of the object detected by the detection unit 1032 based on the object detection signal from the image processing unit 1030 . Then, the setting unit 1072 sets a subject area (area 1211 in FIG. 22) in the pixel area 113A corresponding to the subject area (area 1511 in FIG. 21) according to the recognized subject position and range (step S1023). Specifically, the setting unit 1072 outputs an instruction signal to the driving unit 1021 to instruct the positions of the blocks in the subject area 1211 and the like. Note that, as shown in FIG. 21, the setting unit 1072 sets the subject detected by the detection unit 1032 (the waterfall in FIG. 21) so that a moving image of the subject can be captured even when the subject moves irregularly. ) is set as a subject area 1511. In FIG.

Next, the setting unit 1072 determines whether or not the touch moving image has been operated in step S1003, that is, whether or not the user has touched the touch moving image button image 1522 in step S1003 (step S1025). If the setting unit 1072 determines that the user has touched the touch moving image button image 1522, the setting unit 1072 sets the subject area 1211 (the subject area 1511 in FIG. 21) as the moving image area (step S1026).

Then, the setting unit 1072 sets the frame rate of the moving image area based on the user's touch operation on the frame rate button image 1524 (step S1028). Specifically, the user touches the frame rate button image 1524 (that is, the touch area 1524a). Touch panel 1052 outputs a detection signal corresponding to touch area 1524 a touched by the user to system control section 1070 . As shown in FIG. 23, the display control unit 1071 displays images 1524v of a plurality of frame rate values next to the frame rate button image 1524 based on the detection signal corresponding to the touch area 1524a. In the example shown in FIG. 23, "30", "60", "90", "120", and "240" are displayed as frame rate values. The setting unit 1072 newly sets a touch area on the touch panel 1052 so as to overlap with the area of the image 1524v having a plurality of frame rate values. The user touches one of the multiple frame rate value images 1524v. Touch panel 1052 outputs to system control unit 1070 a detection signal corresponding to the touch area on the image of the frame rate value touched by the user. The setting unit 1072 sets a frame rate value (for example, 240 fps) according to the detection signal from the touch panel 1052 .

The setting unit 1072 also sets imaging conditions other than the frame rate in step S1028. For example, the setting unit 1072 automatically sets the charge accumulation time (exposure time), the gain, and the like for proper exposure.

After that, the AF processing unit 1074 determines whether or not the user has performed a touch operation on the AF button image 1525 (step S1029). When the AF processing unit 1074 determines that the user has performed a touch operation on the AF button image 1525, the AF processing unit 1074 executes AF processing (step S1030). Specifically, the user touches the AF button image 1525 (that is, the touch area 1525a). Touch panel 1052 outputs a detection signal corresponding to touch area 1525 a touched by the user to system control section 1070 . Next, the user touches a subject within image display area 1510 in order to select a subject whose focus position is to be adjusted. For example, the user touches the block B(i, j) within the subject of the waterfall shown in FIGS. 21 and 23 as the subject whose focal position is to be adjusted. Touch panel 1052 outputs a detection signal corresponding to touch area P(i, j) touched by the user to system control section 1070 . Based on the detection signal from the touch panel 1052, the AF processing unit 1074 executes focus adjustment processing (AF processing) for the object in block B(i, j) selected by the user.

That is, while moving the focusing lens 11c, the AF processing unit 1074 detects the position of the focusing lens 11c at which the subject image of the waterfall has the highest contrast as the focus position based on the contrast signal from the image processing unit 1030. do. The AF processing unit 1074 transmits control information to the lens driving control device 15 so as to move the focusing lens 11c to the detected focal position.

In step S1021 of FIG. 20, when the setting unit 1072 determines that the user has selected “manual” as the region setting mode (NO in step S1021), the region within the image display region 1510 is selectable, That is, touch panel 1052 is set to a state in which touch operation can be detected. After that, the user touches (or traces) the block B(i, j) set in the image display area 1510 with a finger, thereby selecting an area within the image display area 1510 block by block. . Touch panel 1052 outputs a detection signal corresponding to touch area P(i, j) touched by the user to system control section 1070 .

Setting unit 1072 recognizes the selection area selected by the user based on the detection signal from touch panel 1052 . In the example shown in FIG. 21, it is assumed that the selection area 1511 has been selected by the user. The setting unit 1072 sets a selected area (area 1211 in FIG. 22) in the pixel area 113A corresponding to the recognized selected area 1512 (step S1024). Specifically, the setting unit 1072 outputs an instruction signal to the driving unit 1021 to instruct the positions of the blocks in the selection area 1211 and the like. Note that, as shown in FIG. 21, the setting unit 1072 selects an area that is slightly wider than the area selected by the user so that a still image of the subject can be captured even if the subject moves irregularly. may be set as the selection area 1511 .

Next, the setting unit 1072 determines whether or not the touch moving image has been operated in step S1003, that is, whether or not the user has touched the touch moving image button image 1522 in step S1003 (step S1025). When the setting unit 1072 determines that the user has touched the touch moving image button image 1522, the setting unit 1072 sets the selection area 1211 (the selection area 1511 in FIG. 21) as the moving image area (step S1026).

Then, similarly to the case described above, the setting unit 1072 sets the frame rate of the moving image area based on the user's touch operation on the frame rate button image 1524 (step S1028). The setting unit 1072 also sets imaging conditions other than the frame rate in step S1028. For example, the setting unit 1072 automatically sets the charge accumulation time (exposure time), the gain, and the like for proper exposure. Thereafter, the AF processing unit 1074 executes AF processing (step S1030) when the user touches the AF button image 1525 (YES in step S1029).

(2) When the user touches the touch still image button image 1523:
FIG. 24 is a diagram showing a display example of the display panel 1051 when the user sets the still image area 1512. As shown in FIG. Also, FIG. 25 is a diagram showing a still image area 1212 in the pixel area 113A of the image sensor 1100. As shown in FIG. Also when the user touches the touch still image button image 1523, the setting process (step S1004) shown in FIG. 20 is executed.

As shown in FIG. 24, the display control unit 1071 displays “auto” and “manual” as an area setting mode image 1523v next to the touch still image button image 1523 based on the detection signal corresponding to the touch area 1523a. display. The setting unit 1072 newly sets a touch area on the touch panel 1052 so as to overlap the area of the image 1523v in the area setting mode (“auto” and “manual” areas). The user touches "auto" or "manual" as the area setting mode image 1523v. The touch panel 1052 outputs a detection signal corresponding to the “auto” or “manual” touch area touched by the user to the system control unit 1070 .

In the process shown in FIG. 20, the setting unit 1072 determines whether the user has selected "auto" or "manual" as the area setting mode based on the detection signal corresponding to the "auto" or "manual" touch area. It is determined whether it has been done (step S1021). If the setting unit 1072 determines that the user has selected “auto” as the area setting mode (YES in step S1021), the area within the image display area 1510 is selectable, that is, the touch panel 1052 is touch operated. to be discoverable. Thereafter, the user selects a subject displayed in image display area 1510 by touching block B(i, j) of the subject displayed in image display area 1510 with a finger. Touch panel 1052 outputs a detection signal corresponding to touch area P(i, j) touched by the user to system control section 1070 .

Setting unit 1072 recognizes block B(i, j) selected by the user based on the detection signal from touch panel 1052 . In the example shown in FIG. 24, it is assumed that the user has selected a predetermined block B(i, j) within area 1512 (the area with white lines in FIG. 24). The setting unit 1072 outputs a position signal indicating the position of the predetermined block B(i,j) selected by the user to the image processing unit 1030, thereby causing the detection unit 1032 to detect the object (step S1022). ). The detector 1032 recognizes the block at the position indicated by the position signal from the system controller 1070 . Then, the detection unit 1032 detects a subject including the recognized block. Specifically, the detection unit 1032 detects the subject by identifying the boundary of the subject (the background behind the waterfall in FIG. 24) based on the contrast and color change between the bright and dark areas in the image data. The detection unit 1032 outputs a subject detection signal indicating the position and range of the detected subject to the system control unit 1070 .

The setting unit 1072 recognizes the position and range of the object detected by the detection unit 1032 based on the object detection signal from the image processing unit 1030 . Then, the setting unit 1072 sets a subject area (area 1212 in FIG. 25) within the pixel area 113A corresponding to the subject area (area 1512 in FIG. 24) according to the recognized subject position and range (step S1023). Specifically, the setting unit 1072 outputs an instruction signal to the driving unit 1021 to instruct the positions of the blocks in the subject region 1212 and the like.

Next, the setting unit 1072 determines whether or not the touch moving image has been operated in step S1003, that is, whether or not the user has touched the touch moving image button image 1522 in step S1003 (step S1025). If the setting unit 1072 determines that the user touches the touch still image button image 1523 instead of the touch moving image button image 1522, the setting unit 1072 sets the subject region 1212 (the subject region 1512 in FIG. 24) to the still image. A region is set (step S1027).

In step S1028, the setting unit 1072 sets imaging conditions other than the frame rate (step S1028). For example, the setting unit 1072 automatically sets a charge accumulation time (exposure time), a gain, and the like that provide proper exposure. Note that when the setting unit 1072 sets the still image area in step S1027, the frame rate setting process is not executed.

After that, the AF processing unit 1074 determines whether or not the user has performed a touch operation on the AF button image 1525 (step S1029). When the AF processing unit 1074 determines that the user has performed a touch operation on the AF button image 1525, the AF processing unit 1074 executes AF processing (step S1030).

In step S1021 of FIG. 20, when the setting unit 1072 determines that the user has selected “manual” as the region setting mode (NO in step S1021), the region within the image display region 1510 is selectable, That is, touch panel 1052 is set to a state in which touch operation can be detected. After that, the user touches (or traces) the block B(i, j) set in the image display area 1510 with a finger, thereby selecting an area within the image display area 1510 block by block. . Touch panel 1052 outputs a detection signal corresponding to touch area P(i, j) touched by the user to system control section 1070 .

Setting unit 1072 recognizes the selection area selected by the user based on the detection signal from touch panel 1052 . In the example shown in FIG. 24, it is assumed that the selection area 1512 has been selected by the user. The setting unit 1072 sets a selected area (area 1212 in FIG. 25) within the pixel area 113A corresponding to the recognized selected area 1512 (step S1024). Specifically, the setting unit 1072 outputs an instruction signal to the driving unit 1021 to instruct the positions of the blocks in the selection area 1212 and the like.

Next, the setting unit 1072 determines whether or not the touch moving image has been operated in step S1003, that is, whether or not the user has touched the touch moving image button image 1522 in step S1003 (step S1025). If the setting unit 1072 determines that the user touched the touch still image button image 1523 instead of the touch moving image button image 1522, the setting unit 1072 sets the selection area 1212 (selection area 1512 in FIG. 24) to the still image. Set as an area (step S1028).

Then, the setting unit 1072 also sets imaging conditions other than the frame rate, as in the case described above (step S1028). For example, the setting unit 1072 automatically sets the charge accumulation time (exposure time), the gain, and the like for proper exposure. Thereafter, the AF processing unit 1074 executes AF processing (step S1030) when the user touches the AF button image 1525 (YES in step S1029).

Note that in the AF processing (step S1030), the subject (region) selected by the user is focused. When the user selects the subject of the waterfall (moving image area), the focus of the subject of the waterfall is adjusted. Also, when the user selects the background subject (still image area) behind the waterfall, the subject behind the waterfall is focused. In general, the subject targeted for focus adjustment is the subject that the user is paying attention to. The AF processing unit 1074 may perform AF processing for a still image area only when a still image is captured in the still image area.

The processing of steps S1001 to S1004 in FIG. 19 as described above is repeatedly executed until the recording switch of the operation unit 1055 is pressed by the user. Note that the image sensor 1100 can capture moving images or still images in units of blocks. Therefore, the setting unit 1072 can set a plurality of moving image areas according to the user's touch operation on the touch panel 1052 . The setting unit 1072 can also set a plurality of still image areas according to the user's touch operation on the touch panel 1052 . In addition, the setting unit 1072 can reset an area set as a moving image area to a still image area according to a user's touch operation on the touch panel 1052, and can set an area set as a still image area as a moving image area. It is also possible to reset. The setting unit 1072 can also set a still image area within the area set as the moving image area. The setting unit 1072 can also set a moving image area within the area set as the still image area.

In the setting process in step S1004, if the setting unit 1072 sets a moving image area or a still image area, in step S1001, the imaging control unit 1073 drives and controls the imaging device 1100 to capture a moving image in the moving image area. is performed, and a still image is captured in the still image area. In step S<b>2 , the display control unit 1071 causes the image display area 1510 of the display panel 1051 to display the partial moving image (live view image of the partial moving image) captured by the imaging device 1100 . This allows the user to confirm in advance what kind of partial moving image is to be captured.

When the system control unit 1070 determines that the moving image switch has been operated by the user (YES in step S1005), the imaging control unit 1073 executes imaging processing (step S1006). At this time, if the setting unit 1072 sets a moving image area or a still image area in the setting process in step S1004, the imaging control unit 1073 drives and controls the imaging device 1100 in step S1006 to A moving image is captured, and a still image is captured in the still image area. Further, the display control unit 1071 displays the image captured by the image sensor 1100 in the image display area 1510 of the display panel 1051 (step S1007). At this time, if the setting unit 1072 sets a moving image area or a still image area in the setting process in step S1004, the display control unit 1071 displays the partial moving image captured by the image sensor 1100 on the display panel 1051 in step S1007. is displayed in the image display area 1510 of . Thereby, the user can confirm what kind of partial moving image is captured. An image (moving image, partial moving image) captured by the imaging device 1100 is recorded in the recording unit 1060 by the imaging control unit 1073 .

Next, the setting unit 1072 determines whether or not the user has performed a touch operation on the touch moving image button image 1522 or the touch still image button image 1523 (step S1008). If the setting unit 1072 determines that the user has performed a touch operation on the touch moving image button image 1522 or the touch still image button image 1523, the setting unit 1072 performs setting processing for setting an area (moving image area, still image area), imaging conditions, and the like. is executed (step S1009). The setting process in step S1009 is the same as the setting process in step S1004 shown in FIG. That is, the setting unit 1072 not only performs the setting process (step S1004) while displaying the live view image on the display panel 1051 of the display unit 1050, but also performs the setting process during capturing of moving images (including partial moving images). Processing (step S1009) is performed.

Even in this case, the setting unit 1072 can reset the area set as the moving image area to the still image area in accordance with the user's touch operation on the touch panel 1052. It is also possible to reset it as a moving image area. The setting unit 1072 can also set a still image area within the area set as the moving image area. The setting unit 1072 can also set a moving image area within the area set as the still image area.

The processing of steps S1006 to S1009 in FIG. 19 is repeatedly executed until the user presses the recording switch of the operation unit 1055 to end recording. When the system control unit 1070 determines that the moving image switch has been operated by the user (YES in step S1010), the imaging control unit 1073 controls the moving image data (including partial moving image data) recorded in the recording unit 1060. A file is created (step S1011). For example, the imaging control unit 1073 names files of moving image data (including data of partial moving images), and adds information such as shooting date and time. Then, the imaging control unit 1073 saves a file of moving image data (including partial moving image data) in the recording unit 1060 (step S1012).

Note that if the setting process (FIG. 20) in step S1004 or step S1009 in FIG. 19 is not executed, a normal moving image will be captured.

As described above, in the seventh embodiment, the imaging unit 1020 has the imaging element 1100 and can capture at least one of a moving image and a still image in the imaging area 113A of the imaging element 1100, and the imaging unit 1020 A setting unit 1072 that sets a first area (for example, the area 1211 in FIG. 22 or the area 1212 in FIG. 25) in the imaging area 113A while a moving image is being captured in the imaging area 113A by the setting unit 1072. Then, a moving image is generated for one of the first area and a second area different from the first area (for example, the area 1211 in FIG. 22 or the area 1212 in FIG. 25), and a still image is generated for the other area. and a generation unit 1031 . According to such a configuration, it is possible to generate an image (partial moving image) in which a portion of the image is a moving image and the other portion is a still image while capturing a moving image using the imaging device 1100 .

Moreover, according to the configuration as described above, the recording capacity of the recording unit 1060 can be reduced. That is, conventionally, after the user captures a moving image with the digital camera 1001, the partial moving image is created by editing the moving image. However, such a configuration requires a capacity for moving image data captured in all imaging areas of the image sensor. Also, if it is desired to keep the moving image data before editing, the data of the partial moving image after editing and the moving image data before editing are recorded in the recording unit, and the recording unit requires a large recording capacity. On the other hand, in the configuration as described above, since a partial moving image can be generated while capturing a moving image using the image sensor 1100, the recording unit 1060 only needs a capacity for the data of the partial moving image. Therefore, the recording capacity of the recording unit 1060 can be reduced.

Moreover, according to the configuration as described above, since a moving image is captured only in a partial area of the image sensor 1100, power consumption can be reduced. In addition, since the setting unit 1072 can set the first area in real time while the imaging unit 1020 is capturing the moving image, the user can edit the partial moving image while capturing the moving image.

Further, in the seventh embodiment, the detection unit 1032 detects the subject from at least one of the moving image and the still image captured by the imaging unit 1020, and the setting unit 1072 detects one or more A plurality of subject areas is set as a first area. With such a configuration, the setting unit 1072 can set an area according to the shape and range of the subject as the first area. Therefore, for example, even if the subject has a complicated shape or the range of the subject is small, the setting unit 1072 can easily set the subject area as the first area. In addition, the work of creating partial moving images by the user is also simplified.

In the seventh embodiment, the display control unit 1071 displays at least one of the moving image and the still image captured by the imaging unit 1020 on the display unit 1050, and at least one of the moving image and the still image displayed on the display unit 1050 Either one of them further includes a selection unit 1052 capable of selecting the first area. According to such a configuration, the user can select any area as the first area. Therefore, it is possible to create a partial moving image according to the user's preference. Further, in the seventh embodiment, since the selection unit 1052 has the touch panel 1052 formed on the display unit 1050, the convenience of operation by the user is improved.

Further, in the seventh embodiment, the setting unit 1072 sets the first area while displaying the live view image captured by the imaging unit 1020 on the display unit 1050 . According to such a configuration, the user can confirm in advance what kind of partial moving image is to be captured. The seventh embodiment also includes a frame rate setting unit 1072 that sets the frame rate of moving images. According to such a configuration, it is possible to appropriately change the smoothness of motion of moving images. It is also possible to record in the recording unit 1060 as a slow-motion moving image or as a high-speed moving image in the recording unit 1060 according to the frame rate. Further, in the seventh embodiment, when there are a plurality of first regions or second regions in which moving images are generated, the frame rate setting unit 1072 makes the frame rate of moving images different for each region. According to such a configuration, it is possible to change the smoothness of motion of the moving image for each area. Therefore, a more interesting partial moving image can be created.

In the seventh embodiment described above, when the setting unit 1072 sets the moving image area according to the user's touch operation, the area other than the moving image area in the pixel area 113A is automatically set as the still image area. may Further, when the setting unit 1072 sets the still image area according to the user's touch operation, the area other than the still image area in the pixel area 113A may be automatically set as the moving image area.

In the above-described seventh embodiment, the digital camera 1001 is used as an example of an electronic device, but the electronic device is not limited to this. good too. Further, the image processing unit 1030 and the system control unit 1070 shown in FIG. 17 may be integrated. In this case, the system control unit having one body-side CPU performs the functions of the image processing unit 1030 and the system control unit 1070 by executing processing based on the control program.

<Eighth embodiment>
In the above-described seventh embodiment, the case where the imaging unit 1020 captures an image of a non-moving subject (a non-moving subject such as the waterfall shown in FIG. 21 and the background behind it) has been described as an example. It can also be applied to imaging a moving subject such as a person. However, when the imaging unit 1020 captures an image of a moving subject, image data (image data of a still image) may be lost as the moving subject moves. Therefore, the eighth embodiment proposes a technique for complementing image data lost due to movement of a moving subject.

FIG. 26 is a diagram showing a display example of the display panel 1051 when setting the area 1513 of the moving object O30 as the moving image area. In the setting process in step S1004 or step S1009 in FIG. 19, the setting unit 1072 determines that the user has selected “auto” as the region setting mode (see step S1021 in FIG. 20), and the movement detected by the detection unit 1032 is Assume that a subject area 1513 corresponding to the person O30 who is the subject is set (see steps S1022 and S1023 in FIG. 20), and the subject area 1513 is set as a moving image area (see step S1026 in FIG. 20). Further, the setting unit 1072 determines that the user has selected “auto” as the area setting mode (see step S1021 in FIG. 20), and sets an area other than the subject area 1513 detected by the detection unit 1032 (see FIG. 20). (see steps S1022 and S1023 in FIG. 20), and an area other than the subject area 1513 is set as the still image area (see step S1027 in FIG. 20).

As described above, in the image capturing process in step S1006 in FIG. 19, the image capturing control unit 1073 captures a moving image in the moving image area (subject area 1513 in the example illustrated in FIG. 26) and still image area (in the example illustrated in FIG. A still image is captured in a region other than the subject region 1513). In this embodiment, the setting unit 1072 also moves the subject area 1513 when the person O30, which is the moving subject, moves as shown in FIG. That is, the setting unit 1072 sets the subject area 1513 so as to track the moving subject. If the subject region 1513 moves after the imaging control unit 1073 captures a still image in the still image region, the image data of the subject region 1513 portion of the image data of the captured still image is lost.

FIG. 27 is a flowchart showing imaging processing in the eighth embodiment. Also, FIG. 28 is a diagram for explaining complementation of an image accompanying movement of a moving subject. The imaging process shown in FIG. 27 corresponds to the process of step S1006 in FIG. In the process shown in FIG. 26, if a moving image area is set in the setting process of step S1004 or step S1009 (YES in step S31), the imaging control unit 1073 captures a moving image in the moving image area (step S32). In the example shown in FIG. 28, the imaging control unit 1073 causes the moving image to be captured in the subject area 1513a as the moving image area. Further, when a still image area is set in the setting process of step S1004 or step S1009 (YES in step S33), the image capturing control unit 1073 captures a still image in the still image area (step S34). In the example shown in FIG. 28, the image capturing control unit 1073 captures a still image in an area other than the subject area 1513a.

Next, the imaging control unit 1073 determines whether or not the moving image area has moved a certain distance or more (step S35). In the example shown in FIG. 28, the person O30 has moved and the subject area 1513a has moved to the subject area 1513b. At this time, the imaging control unit 1073 determines that the moving image area has moved a distance equal to or greater than a certain distance. When the imaging control unit 1073 determines that the moving image area has moved a certain distance, the imaging control unit 1073 moves the area other than the moving image area. ), a still image is captured again (step S36). In the example shown in FIG. 28, the image capture control unit 1073 captures a still image in an area other than the subject area 1513b. Then, the image capturing control unit 1073 complements the missing portion of the image data of the still image captured in step S34 with the image data of the still image in the moving image area before the movement among the image data of the still image recaptured in step S36. (step S37). In the example shown in FIG. 28, the imaging control unit 1073 determines that the image data of the still image captured in step S34 is missing in the image data of the still image of the subject area 1513a among the image data of the still image recaptured in step S36. The part (the part of the subject area 1513a) is complemented.

According to such a configuration, even if the image data is lost due to the movement of the moving subject, the missing image data can be complemented. Therefore, it is possible to prevent generation of an unnatural partial moving image with missing image data.

Note that in FIG. 19 described above, if the setting process (FIG. 20) in step S1004 or step S1009 is not executed, a normal moving image is captured. In this case, as in the conventional case, after the moving image is captured by the imaging device 1100, the user edits the moving image to create a partial moving image. However, if it is possible to capture two moving images of the same subject at the same time, it is considered that the work of editing the partial moving image after capturing the moving image will become easier. Therefore, with the configuration shown in FIG. 29, two moving images of the same subject are simultaneously captured.

FIG. 29 is a diagram showing an arrangement pattern in each block. The array pattern shown in FIG. 29 is an array pattern in which the pixel area 113A is divided into two areas, one area and the other area. In this arrangement pattern, in the pixel region 113A, blocks in odd rows (2n−1) in odd columns (2m−1) and blocks in even rows (2n) in even columns (2m) are one block. A region is configured. Another area consists of blocks of odd rows (2n-1) in even columns (2m) and blocks of even rows (2n) in odd columns (2m-1). That is, each block in the pixel region 113A is divided into a checkered pattern. m and n are positive integers (m = 1, 2, 3, ..., n = 1, 2, 3, ...). According to such a configuration, the imaging device 1100 captures a moving image in one region and also captures a moving image in another region, thereby simultaneously capturing two moving images of the same subject. can be done. By using two moving images of the same subject, the user can easily create various variations of partial moving images.

<Ninth Embodiment>
In the ninth embodiment, the digital camera 1001 in the seventh embodiment is separated into an imaging device 1001A and an electronic device 1001B.

FIG. 30 is a block diagram showing configurations of an imaging device 1001A and an electronic device 1001B according to the ninth embodiment. In the configuration shown in FIG. 30, an imaging device 1001A is a device for imaging a subject. This imaging apparatus 1001A includes a lens unit 1010, an imaging unit 1020, an image processing unit 1030, a work memory 1040, an operation unit 1055, a recording unit 1060, and a first system control unit 1070A. The configurations of the lens unit 1010, the imaging unit 1020, the image processing unit 1030, the work memory 1040, the operation unit 1055, and the recording unit 1060 in the imaging device 1001A are the same as those shown in FIG. Therefore, the same components are denoted by the same reference numerals, and overlapping descriptions are omitted.

The electronic device 1001B is a device that displays images (still images, moving images, live view images, partial moving images). This electronic device 1001B includes a display section 1050 and a second system control section 1070B. Note that the configuration of the display unit 1050 in the electronic device 1001B is the same as the configuration shown in FIG. Therefore, the same configurations are denoted by the same reference numerals, and overlapping descriptions are omitted.

The first system control section 1070A has a first communication section 1075A. Also, the second system control unit 1070B has a second communication unit 1075B. The first communication unit 1075A and the second communication unit 1075B transmit and receive signals to and from each other in a wired or wireless manner. In such a configuration, the first system control unit 1070A transfers image data (image data processed by the image processing unit 1030, image data recorded in the recording unit 1060) to the second system via the first communication unit 1075A. It is transmitted to the communication section 1075B. Second system control unit 1070B causes display unit 1050 to display the image data received by second communication unit 1075B. Second system control unit 1070B also causes second display unit 53 to display a preset menu image.

In addition, the second system control unit 1070B performs control to change the imaging conditions (including accumulation conditions) according to the user's touch operation on the touch panel 1052 or automatically. Further, the second system control unit 1070B performs control to select each display area in the image display area 1510 according to the touch operation of the touch panel 1052 by the user or automatically. In addition, the first system control unit 1070A executes imaging control according to the user's operation of the operation unit 1055 (an operation unit provided on the electronic device 1001B side for instructing the start of imaging of a still image or moving image) 1055 .

The configuration shown in FIG. 17 (display control unit 1071, setting unit 1072, imaging control unit 1073, and AF processing unit 1074) may be provided in either the first system control unit 1070A or the second system control unit 1070B. All the configuration shown in FIG. 17 may be provided in the first system control unit 1070A or the second system control unit 1070B, and part of the configuration shown in FIG. 17 may be provided in the first system control unit 1070A, 17 may be provided in the second system control unit 1070B.

Note that the imaging device 1001A is configured by, for example, a digital camera, a smart phone, a mobile phone, a personal computer, or the like having an imaging function and a communication function, and the electronic device 1001B is, for example, a smart phone, a mobile phone, or a portable personal It consists of portable terminals such as computers.

The first system control unit 1070A shown in FIG. 30 is implemented by the CPU executing processing based on the control program. Also, the second system control unit 1070B shown in FIG. 30 is implemented by the CPU executing processing based on the control program.

As described above, in the ninth embodiment, in addition to the effects described in the seventh embodiment, a mobile terminal such as a smartphone is used to check a partial moving image captured by the imaging device 1001A, and then capture the image. be able to.

In the configuration shown in FIG. 30, image processing section 1030 and first system control section 1070A may be integrated. In this case, a system control unit having one CPU performs processing based on a control program to perform the function of the image processing unit 1030 and the function of the first system control unit 1070A.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various modifications and improvements can be made to the above embodiments without departing from the scope of the present invention. Also, one or more of the requirements described in the above embodiments may be omitted. Such modifications, improvements, and omissions are also included in the technical scope of the present invention. Moreover, it is also possible to appropriately combine the configurations of the above-described embodiments and modifications and apply them.

For example, although the arrangement of the color filters 102 is the Bayer arrangement in each of the above-described embodiments, an arrangement other than this arrangement may be used. Also, the number of pixels forming the unit group 131 should include at least one pixel. Also, it is sufficient that each block includes at least one pixel. Therefore, it is possible to perform imaging under different imaging conditions for each pixel.

Further, in each of the above-described embodiments, the drive unit 1021 may be partially or wholly mounted on the imaging chip 113 , or partially or wholly mounted on the signal processing chip 111 . Also, part of the configuration of the image processing unit 1030 may be mounted on the imaging chip 113 or the signal processing chip 111 . Also, part of the configuration of the system control unit 1070 may be mounted on the imaging chip 113 or the signal processing chip 111 .

Further, in each of the above-described embodiments, only the moving image area may be set without setting the still image area in the pixel area 113A. In this case, a plurality of moving image areas are set and the frame rate is changed for each of the plurality of moving image areas. An image including a plurality of moving images having different frame rates in this way may also be included in the partial moving images. According to such a configuration, it is possible to generate an image in which a slow-moving moving image and a fast-moving moving image are mixed, so that a more interesting image can be generated.

Also, in the arrangement pattern of the blocks shown in FIG. 29, the pixel region 113A is divided into two, but it may be divided into three or more. If the pixel area 113A is divided into three or more, three or more moving images of the same subject can be captured simultaneously. Therefore, it becomes possible to create a more interesting partial moving image.

Further, in the seventh embodiment described above, the subject that can be detected by the detection unit 1032 is not limited to a person or the like, and may be an object such as a car as long as its characteristics can be detected. Also, the imaging control unit 1073 may record the still image and the moving image separately in the recording unit 1060 . Also, the imaging control unit 1073 links the timelines of the still image and the moving image and records them in the recording unit 1060 . In addition, the detection unit 1032 registers a specific person in advance, and automatically tracks the specific person when detecting through pattern matching that the registered specific person is captured as a subject. It is also possible to shoot a moving image by pressing the button. Also, the touch operation of the touch panel 1052 may be replaced with the operation of switches, dials, or the like in the operation unit 1055 .

Also, in each of the above-described embodiments, it is desirable that the digital camera 1001 is fixed on a tripod or the like to take an image. This is because if the angle of view changes during imaging, it may become impossible to synthesize a moving image and a still image well.

<Tenth Embodiment>
FIG. 31 is a block diagram showing the configuration of a digital camera 2001 according to the tenth embodiment. 31, a digital camera 2001 as an example of electronic equipment corresponds to the digital camera 1 in FIG. 7, a lens unit 2010 corresponds to the lens unit 10 in FIG. 7, and an imaging unit 2020 corresponds to the imaging unit 20 in FIG. , the image sensor 2100 corresponds to the image sensor 100 in FIG. 7, the driving unit 2021 corresponds to the driving unit 21 in FIG. 7, the image processing unit 2030 corresponds to the image processing unit 30 in FIG. 7, the display unit 2050 corresponds to the display unit 50 in FIG. 7, the display panel 2051 corresponds to the display panel 51 in FIG. 7, and the touch panel 2052 corresponds to the touch panel 52 in FIG. 7, the recording unit 2060 corresponds to the storage unit 60 in FIG. 7, and the system control unit 2070 corresponds to the system control unit 70 in FIG. Of these configurations, the configurations of the lens unit 2010, imaging unit 2020, imaging element 2100, driving unit 2021, work memory 2040, display unit 2050, operation unit 2055, and recording unit 2060 are omitted.

In this embodiment, drive control of the focusing lens 11c is control for performing focus adjustment of the focusing lens 11c, and is called AF processing (automatic focusing). Further, drive control of the diaphragm 14 is control for adjusting the aperture diameter of the diaphragm 14, and is called AE processing (automatic exposure).

The image processing unit 2030 includes a detection unit (mode detection unit) 2031 as shown in FIG. A detection unit 2031 detects a main subject from the image data generated by the image processing unit 2030 . In this embodiment, the detection unit 2031 has a function of detecting a human body included in image data as a main subject, as described in, for example, Japanese Patent Laying-Open No. 2010-16621 (US2010/0002940). The detection unit 2031 also has a function of detecting a person's face using a known face detection function, as described, for example, in Japanese Unexamined Patent Application Publication No. 2010-16621 (US2010/0002940). The detection unit 2031 also has a function of detecting the facial expression of a person as a main subject and determining whether the detected facial expression is a specific facial expression (for example, a smile).

Here, the subject, which is the object to be imaged by the imaging unit 2020, is not limited to the case where there is only one in the image data, and there may be a plurality of subjects. Among subjects, a subject that attracts the user's (photographer's) attention or a subject that is presumed to attract the user's attention is referred to as a "main subject". The main subject is also not limited to the case where there is only one in the image data, and there may be a plurality of main subjects.

A system control unit 2070 controls the overall processing and operation of the digital camera 2001 . The system control unit 2070 has a body-side CPU (Central Processing Unit). In this embodiment, the system control unit 2070 divides the imaging surface (pixel region 113A) of the imaging element 2100 (imaging chip 113) into a plurality of blocks, and sets different charge accumulation times (or charge accumulation times) and frame rates between the blocks. , to acquire an image with gain. Therefore, the system control unit 2070 instructs the driving unit 2021 about the position, shape, range of blocks, and accumulation conditions for each block.

In addition, the system control unit 2070 acquires an image with a thinning rate that differs between blocks, the number of addition rows or addition columns for adding pixel signals, and the number of bits for digitization. Therefore, the system control unit 2070 instructs the driving unit 2021 about imaging conditions for each block (thinning rate, number of rows or columns to be added for adding pixel signals, and number of bits for digitization). Further, the image processing unit 2030 executes image processing under different imaging conditions (control parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression ratio) between blocks. Therefore, the system control unit 2070 instructs the image processing unit 2030 on imaging conditions for each block (control parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate).

Also, the system control unit 2070 causes the recording unit 2060 to record the image data generated by the image processing unit 2030 . Further, the system control unit 2070 causes the display unit 2050 to display an image by outputting the image data generated by the image processing unit 2030 to the display unit 2050 . Further, the system control unit 2070 reads the image data recorded in the recording unit 2060 and causes the display unit 2050 to output the image data, thereby causing the display unit 2050 to display an image. Images displayed on the display unit 2050 include still images, moving images, and live view images.

In this embodiment, the system control unit 2070 includes a setting unit 2071, a control unit 2072, a display control unit 2073, an AF processing unit (focus detection unit) 2074, and an AE processing unit 2075, as shown in FIG. . The setting unit 2071 sets a main subject area (for example, a main subject area 2201 including a person O110 shown in FIG. 34) including the main subject (for example, a person) detected by the detection unit 2031 in the pixel area (imaging area) 113A of the image sensor 2100. , and a non-main subject area that does not include the main subject (for example, the non-main subject area 2203 shown in FIG. 34) are set in units of blocks. In addition, the setting unit 2071 selects a partial area (for example, a partial area 2202 shown in FIG. 34) including a part of the main subject (for example, a person's face) detected by the detection unit 2031 in the pixel area 113A of the image sensor 2100 in block units. set. Incidentally, as shown in FIG. 34, which will be described later, the partial area is an area included in the main subject area. Also, the setting unit 2071 changes the positions and sizes of the main subject area, the non-main subject area, and the partial area according to the movement of the main subject detected by the detection unit 2031 . In addition, the setting unit 2071 sets different imaging conditions for each area in the pixel area 113A. That is, the setting unit 2071 sets different imaging conditions for the main subject area, the partial area, and the non-main subject area.

The control unit 2072 controls the driving of the imaging element 2100 so that the main subject area, the partial area, and the non-main subject area are imaged under different imaging conditions. In addition, the control unit 2072 directs the image sensor 2100 to take the main image (step S2017 in FIG. 32 to be described later) on condition that the expression of the person as the main subject detected by the detection unit 2031 is a specific expression (for example, a smile). (i.e., imaging based on the full-press operation of the release switch) is executed. Further, the control unit 2072 performs control to record image data of still images and moving images generated by the image processing unit 2030 in the recording unit 2060 . The display control unit 2073 outputs the image data generated by the image processing unit 2030 to the display unit 2050, and performs control to display an image (live view image, still image, moving image) on the display surface 2051 of the display unit 2050. .

The AF processing unit 2074 executes focus adjustment processing (AF processing) called contrast AF during live view image shooting so that the contrast of the subject is maximized (see step S2007 in FIG. 32). The AF processing unit 2074 also executes focus adjustment processing (AF processing) when the release switch is half-pressed or when the moving image switch is operated (see step S2013 in FIG. 32; 33 step S2027).

Specifically, the AF processing unit 2074 causes the lens drive control device 15 to move the focusing lens 11c by transmitting control information to the lens drive control device 15 via the electrical contacts 81A and 81B. Further, the AF processing unit 2074 outputs a contrast signal representing the contrast of the image of the subject captured by the imaging unit 2020 (here, when the detection unit 2031 detects the main subject, the evaluation value of the contrast of the image of the main subject). is acquired from the image processing unit 2030 . The AF processing unit 2074 moves the focusing lens 11c, and based on the contrast signal from the image processing unit 2030, detects the position of the focusing lens 11c where the contrast of the subject image is highest as the focal position. The AF processing unit 2074 transmits a control signal to the lens driving control device 15 so as to move the focusing lens 11c to the detected focal position.

The AE processing unit 2075 executes exposure adjustment processing (AE processing) during live view image shooting (see step S2008 in FIG. 32). The AE processing unit 2075 also executes exposure adjustment processing (AE processing) when the release switch is half-pressed or when the moving image switch is operated (see step S2014 in FIG. 32; 33 step S2028). Specifically, the AE processing unit 2075 acquires luminance distribution data representing the luminance distribution of the image from the image processing unit 2030 . Then, the AE processing unit 2075 performs photometry using the luminance distribution data. Also, the AE processing unit 2075 determines imaging conditions (shutter speed (exposure time, charge accumulation time), frame rate, and gain (ISO sensitivity)) and aperture value based on the result of photometry. At this time, the AE processing unit 2075 can determine imaging conditions that provide proper exposure for each block in the pixel area 113A of the image sensor 2100. FIG. Note that the aperture value cannot be set for each block, and is set for the entire pixel region 113A.

The AE processing unit 2075 causes drive control for adjustment of the aperture diameter of the diaphragm 14 to be executed by transmitting control information indicating an aperture value corresponding to proper exposure to the lens drive control device 15 . The AE processing unit 2075 also outputs an instruction signal to the driving unit 2021 to instruct shutter speed (charge accumulation time), frame rate, and gain (ISO sensitivity) according to proper exposure. The drive unit 2021 drives and controls the imaging element 2100 with the shutter speed, frame rate, and gain indicated by the instruction signal.

In the system control unit 2070, the setting unit 2071, the control unit 2072, the display control unit 2073, the AF processing unit 2074, and the AE processing unit 2075 are each executed by the body side CPU based on the control program. Realized.

Next, the shooting operation of the digital camera 2001 according to the tenth embodiment will be described. 32 and 33 are flowcharts for explaining the shooting operation executed by the system control unit 2070 of the tenth embodiment. In the processing shown in FIGS. 32 and 33, when the user operates the operation unit 2055 or the like to start photographing after the digital camera 2001 is powered on, the control unit 2072 causes the live view image to be photographed. is started (step S2001). That is, the control unit 2072 outputs an instruction signal to the driving unit 2021 to execute drive control of the imaging device 2100 . Also, the display control unit 2073 displays the live view image captured by the image sensor 2100 on the display surface 2051 of the display unit 2050 (step S2002).

Next, the setting unit 2071 outputs an instruction signal to the image processing unit 2030 to cause the detection unit 2031 to detect the main subject (step S2003). Upon receiving the instruction signal from the setting unit 2071, the detection unit 2031 detects a person (human body) included in the image data as the main subject. The detection unit 2031 then outputs a signal indicating the detected position and size of the person to the system control unit 2070 . The setting unit 2071 outputs an instruction signal to the image processing unit 2030 to cause the detection unit 2031 to detect the face of a person who is part of the main subject (step S2004). Upon receiving the instruction signal from the setting unit 2071, the detection unit 2031 detects the face of a person who is part of the main subject using a known face detection function based on the image data. Then, the detection unit 2031 outputs a signal indicating the detected position and size of the person's face to the system control unit 2070 .

Next, the setting unit 2071 sets an area in the pixel area 113A (step S2005). Specifically, the setting unit 2071 sets the main subject area including the person in units of blocks based on the signal indicating the position and size of the person from the detection unit 2031 . Also, the setting unit 2071 sets a partial area including a face in the main subject area on a block-by-block basis based on the signal indicating the position and size of the person's face from the detection unit 2031 . The setting unit 2071 also sets areas other than the main subject area in the pixel area 113A as non-main subject areas.

FIG. 34 is a diagram showing a main subject area 2201, a partial area 2202, and a non-main subject area 2203 in the imaging area 113A of the image sensor 2100 of the tenth embodiment. In the example shown in FIG. 34, in a pixel area (imaging area) 113A of the imaging device 2100, a walking person O110, a road O120, and a plurality of trees O130 are imaged. A person O110 is the main subject, and a road O120 and a tree O130 are background subjects. In this case, the detection unit 2031 detects the person O110 as the main subject. The detection unit 2031 also detects the face of the person O110 using the face detection function. The setting unit 2071 sets the area including the person O110 detected by the detection unit 2031 as the main subject area 2201 . The setting unit 2071 also sets the area including the face of the person O110 detected by the detection unit 2031 as the partial area 2202 in the main subject area 2201 . Furthermore, the setting unit 2071 sets an area other than the main subject area 2201 in the pixel area 113A as a non-main subject area 2203 . Note that as shown in FIG. 34, the setting unit 2071 sets a main subject area 2201 that is slightly wider than the area of the main subject detected by the detection unit 2031 . Also, the setting unit 2071 sets an area slightly wider than the partial area of the main subject detected by the detection unit 2031 as a partial area 2202 .

FIG. 35 is a diagram showing a main subject area 2511, a partial area 2512, and a non-main subject area 2513 on the display surface 2051 of the display unit 2050 of the tenth embodiment. When the imaging device 2100 images the subjects O110, O120, and O130 shown in FIG. 34, the image of the person O110 walking and the image of the road O120 are displayed on the display surface 2051 of the display unit 2050 as shown in FIG. , and images of a plurality of trees O130 are displayed. A main subject area 2511 including the person O110 shown in FIG. 35 is an area corresponding to the main subject area 2201 in the pixel area 113A shown in FIG. A partial area 2512 including the face of the person O110 shown in FIG. 35 is an area corresponding to the partial area 2202 in the pixel area 113A shown in FIG. A non-main subject area 2513 shown in FIG. 35 is an area corresponding to the non-main subject area 2203 in the pixel area 113A shown in FIG.

In this embodiment, as shown in FIG. 35, when the detection unit 2031 detects the main subject (person O110), the display control unit 2073 creates a frame 2601 surrounding the main subject and a part of the main subject (person O110). A frame 2602 surrounding O 110's face) is displayed on the display surface 2051 of the display unit 2050 . A frame 2601 is displayed to inform the user that the detection unit 2031 has captured the main subject and to inform the user of the position, size, and the like of the main subject. A frame 2602 is displayed to inform the user that the detection unit 2031 has captured a portion of the main subject and to inform the user of the position and size of the portion of the main subject. In the example shown in FIG. 35, the frame 2601 is the frame around the main subject area 2511 and the frame 2602 is the frame around the partial area 2512 .

In addition, in this embodiment, as shown in FIG. 35, a sub-screen 2051S of a small area is provided on the upper right of the display surface 2051. In FIG. When the detection unit 2031 detects the main subject (person O110), the display control unit 2073 displays the image of the partial area 2202 on the sub-screen 2051S (see step S2009 described later).

Returning to the description of FIG. 32, the setting unit 2071 sets imaging conditions for each region set in step S2005 (step S2006). Here, the setting unit 2071 sets different imaging conditions for the main subject area 2201 , the partial area 2202 and the non-main subject area 2203 . Specifically, the setting unit 2071 sets a higher frame rate and higher gain in the main subject area 2201 than in the non-main subject area 2203 . As a result, the detection unit 2031 can detect the main subject in the main subject area 2201 at high speed with higher accuracy than in the non-main subject area 2203 . Therefore, the detection response of the main subject by the detection unit 2031 is increased. Also, the setting unit 2071 sets a higher frame rate and a higher gain in the partial area 2202 than in the main subject area 2201 . This allows the detection unit 2031 to quickly and accurately determine the facial expression of the person O110 who is the main subject.

Note that the setting unit 2071 automatically sets the imaging conditions other than the frame rate and gain that are suitable for the subject in each of the areas 2201, 2202, and 203, or sets the imaging conditions according to the user's operation of the operation unit 2055 or the like. set accordingly.

Next, the AF processing unit 2074 executes AF processing for automatically adjusting the focus (step S2007). Specifically, the AF processing unit 2074 acquires from the image processing unit 2030 a contrast signal representing the contrast of the image of the person O110 who is the main subject. In addition, the AF processing unit 2074 transmits control information instructing movement of the focusing lens 11c to the lens driving control device 15. FIG. Accordingly, the lens drive control device 15 performs control to move the focusing lens 11c based on the control information from the AF processing section 2074. FIG. Then, the AF processing unit 2074 detects the lens position of the focusing lens 11c at which the contrast of the image of the person O110 is highest while moving the focusing lens 11c as the focal position. The AF processor 2074 moves the focusing lens 11c to the detected focal position. As described above, the setting unit 2071 sets a high frame rate and a high gain for the main subject area 2201 . Therefore, the image processing unit 2030 can quickly calculate the contrast evaluation value of the image of the main subject. As a result, the AF processing section 2074 can quickly detect the focal position.

In this embodiment, the AF processing unit 2074 calculates the distance to the main subject based on the focal position detected by the AF processing in step S2007. The AF processing unit 2074 also stores information on the focus position of the main subject and information on the distance to the main subject. Then, when the main subject is moving, the AF processing unit 2074 predicts the position of the main subject based on the history of information on the focus position of the main subject and the history of information on the distance to the main subject. The AF processing unit 2074 predicts the focal position of the main subject based on the predicted position of the main subject, and moves the focusing lens 11c to the predicted focal position. With such a configuration, the AF processing section 2074 can always focus on the main subject moving at high speed.

In this embodiment, when the main subject is moving, in step S2005, the setting unit 2071 sets the main subject based on the history of the focal position information of the main subject and the history of the distance information to the main subject. Predict the position and size of A setting unit 2071 predicts the position and size of the main subject area based on the predicted position and size of the main subject. Then, the setting unit 2071 sets the main subject area of the predicted position and size. According to such a configuration, even when the main subject is moving at high speed, the main subject area can be set according to the movement of the main subject.

Note that, as shown in FIG. 34, when the main subject is the person O110, the AF processing unit 2074 focuses on the face of the person O110. However, the AF processing unit 2074 may focus on the closest position of the person O110. Also, the main subject is not limited to a person, and may be an object such as a car. Also, when a plurality of main subjects are detected by the detection unit 2031, the AF processing unit 2074 may, for example, preferentially focus on a person among the plurality of main subjects. In addition, when a plurality of persons are detected by the detection unit 2031, the AF processing unit 2074 may focus on, for example, the centralmost person among the plurality of persons. Also, when a plurality of main subjects are detected by the detection unit 2031, the setting unit 2071 sets a main subject area for each of the plurality of main subjects. However, the setting unit 2071 may set the main subject area only for some of the main subjects (for example, the centralmost main subject) among the plurality of main subjects.

Also, the AE processing unit 2075 executes AE processing for controlling exposure so as to obtain proper exposure (step S2008). Specifically, the AE processing unit 2075 acquires from the image processing unit 2030 luminance distribution data representing the luminance distribution of the image data generated by the image processing unit 2030 . Based on the brightness distribution data, the AE processing unit 2075 then determines imaging conditions (frame rate, shutter speed, gain (ISO sensitivity), etc.) and aperture value for proper exposure. The AE processing unit 2075 causes drive control for adjusting the aperture diameter of the diaphragm 14 to be executed by transmitting control information indicating the aperture value corresponding to the determined appropriate exposure to the lens drive control device 15 . The AE processing unit 2075 also outputs an instruction signal to the driving unit 2021 to instruct imaging conditions (frame rate, shutter speed, gain, etc.) according to proper exposure. The drive unit 2021 sets the imaging conditions instructed by the instruction signal.

In step S2006, the setting unit 2071 sets imaging conditions such as the frame rate and gain, and in step S2008, the AE processing unit 2075 sets the aperture value according to the imaging conditions set in step S2006. good too. Also, when the AE processing unit 2075 sets imaging conditions such as frame rate and gain in addition to the aperture value in step S2008, a higher frame rate and higher gain are set in the main subject area 2201 than in the non-main subject area 2203. , a higher frame rate and a higher gain than in the main subject area 2201 are set in the partial area 2202 . As a result, the detection response of the person who is the main subject by the detection unit 2031 can be maintained, and the accuracy of facial expression determination by the detection unit 2031 can be maintained.

When the detection unit 2031 detects the main subject in step S2003, the display control unit 2073 displays the image of the partial area 202 (the face of the person O110 who is the main subject) on the sub-screen 2051S of the display surface 2051 as shown in FIG. ) is enlarged and displayed (step S2009). As a result, even when the size of the display surface 2051 is small, the user can easily check the face (and its facial expression) of the person O110. Note that the image displayed on the sub-screen 2051S is not limited to the image of part of the main subject (the face of the person O110), and may be the image of the entire main subject. Further, the partial image of the main subject displayed on the sub-screen 2051S is not limited to the image of the face of the person O110, and may be an image of the upper body of the person O110.

After that, the system control unit 2070 determines whether or not the release switch has been half-pressed (operation of SW1) (step S2010). If the system control unit 2070 determines that the release switch has not been pressed halfway, it determines whether the moving image switch has been operated (step S2020). The system control unit 2070 repeats the processing from step S2001 to step S2009 until the release switch is half-pressed or the moving image switch is operated. Therefore, detection of the main subject (step S2003) and detection of the face (step S2004) by the detection unit 2031 are performed as needed. That is, the main subject is tracked by the detection unit 2031 . Further, according to the motion of the main subject detected by the detection unit 2031, the setting unit 2071 sets the regions (main subject region 2201, partial region 202, non-main subject region 2203) (step S2005) as needed. The setting of imaging conditions for each area by 2071 is also performed as needed. As a result, the detection unit 2031 can detect the movement of the main subject with high accuracy and high speed, and the tracking response of the main subject by the detection unit 2031 is improved.

When the system control unit 2070 determines that the release switch has been half-pressed (YES in step S2010), the control unit 2072 captures a live view image (step S2011), and the display control unit 2073 A live view image captured by the imaging element 2100 is displayed on the display surface 2051 of the display unit 2050 (step S2012). Also, the AF processing unit 2074 executes AF processing (step S2013) in the same manner as described in step S2007, and the AE processing unit 2075 executes AE processing in the same manner as in step S2008 (step S2014). ). Note that the AF processing unit 2074 may be configured to execute AF processing only when the release switch is half-pressed. That is, the AF processing unit 2074 may be configured not to execute AF processing while capturing a live view image. Similarly, the AE processing unit 2075 may be configured to perform AE processing only when the release switch is half-pressed. That is, the AE processing unit 2075 may be configured not to execute AE processing while the live view image is being captured.

After that, the system control unit 2070 determines whether or not the release switch has been fully pressed (operation of SW2) (step S2015). When the system control unit 2070 determines that the release switch has not been fully pressed (NO in step S2015), the processes from steps S2010 to S2014 are repeated. When the system control unit 2070 determines that the release switch has been fully pressed (YES in step S2015), the control unit 2072 determines whether the facial expression of the main subject is smiling (step S2016). In this embodiment, the detection unit 2031 uses a face detection function to detect the facial expression of a person as the main subject, and outputs information indicating the detected facial expression to the system control unit 2070 . The control unit 2072 determines whether or not the facial expression of the person is a specific facial expression (smile) based on the information indicating the facial expression of the person output from the detection unit 2031 .

FIG. 36 is a diagram showing an example of a smiling image and a non-smiling image. FIG. 36A shows a case where the facial expression of the person is smiling, and FIG. 36B shows a case where the facial expression of the person is not smiling. The detection unit 2031 determines whether or not the facial expression of the person detected using the face detection function is a smile. As shown in FIG. 36A , when determining that the facial expression of the person is smiling, the detecting unit 2031 outputs information indicating that the facial expression of the person is smiling to the system control unit 2070 . Further, as shown in FIG. 36B, when the detection unit 2031 determines that the facial expression of the person is not smiling, it outputs information indicating that the facial expression of the person is not smiling to the system control unit 2070 . The control unit 2072 determines whether or not the facial expression of the person is a smile based on the information indicating the facial expression of the person output from the detection unit 2031 .

If the control unit 2072 determines that the facial expression of the person who is the main subject is a specific facial expression (smile) based on the information indicating the facial expression of the person output from the detection unit 2031, the control unit 2072 executes imaging processing (step S2017). As described above, in the AE processing in step S2014, the AE processing unit 2075 determines imaging conditions suitable for the subject based on the brightness level of proper exposure. In the imaging process of step S2017, the control unit 2072 outputs to the drive unit 2021 an instruction signal that instructs the imaging conditions determined in the AE process of step S2014. Then, the control unit 2072 outputs an instruction signal instructing imaging to the driving unit 2021 . The drive unit 2021 drives and controls the image sensor 2100 under the imaging conditions instructed by the control unit 2072 to perform imaging. In each pixel of the pixel region 113A, imaging may be performed under the same imaging conditions, or imaging may be performed under different imaging conditions for each subject.

Note that the control unit 2072 determines whether or not the facial expression of the person is smiling in step S2016, and executes the imaging process in step S2017 on the condition that it is determined to be smiling. However, the configuration is not limited to this, and the control unit 2072 determines whether or not the person's eyes are closed, and executes the imaging process in step S2017 on the condition that it is determined that the person's eyes are not closed. good too. That is, the condition for executing the imaging process is not limited to the condition related to the facial expression of the person, and may be the condition that the person is performing a specific action.

If the system control unit 2070 determines in step S2020 that the moving image switch has been operated (YES in step S2020), the control unit 2072 starts capturing moving images (step S2021 in FIG. 33). Also, the display control unit 2073 displays the moving image captured by the image sensor 2100 on the display surface 2051 of the display unit 2050 (step S2022). The setting unit 2071 outputs an instruction signal to the image processing unit 2030 to cause the detection unit 2031 to detect the main subject (step S2023). Upon receiving the instruction signal from the setting unit 2071, the detection unit 2031 detects a person (human body) included in the image data as the main subject. Further, the setting unit 2071 outputs an instruction signal to the image processing unit 2030 to cause the detection unit 2031 to detect the face of a person who is part of the main subject, as in the case described in step S2004. (Step S2024). Upon receiving the instruction signal from the setting unit 2071, the detection unit 2031 detects the face of a person who is part of the main subject using a known face detection function based on the image data.

Next, the setting unit 2071 sets a region in the pixel region 113A (step S2025) in the same manner as described in step S2005. That is, when the detection unit 2031 detects the main subject, the setting unit 2071 sets the main subject area and the partial area including the main subject in units of blocks. The setting unit 2071 also sets areas other than the main subject area in the pixel area 113A as non-main subject areas.

Also, the setting unit 2071 sets different imaging conditions for the main subject area, the partial area, and the non-main subject area (step S2026), as in the case described in step S2006. That is, the setting unit 2071 sets a higher frame rate and a higher gain in the main subject area than in the non-main subject area. Also, the setting unit 2071 sets a higher frame rate and a higher gain in the partial area than in the main subject area. Also, the AF processing unit 2074 executes AF processing in the same manner as described in step S2007 (step S2027), and the AE processing unit 2075 executes AE processing in the same manner as in step S2008 (step S2028). ). Then, the display control unit 2073 enlarges and displays the image of the partial area (for example, the image of the person's face) on the sub-screen 2051S of the display surface 2051 (step S2029) in the same manner as described in step S2009.

The control unit 2072 performs control to record the moving image data generated by the image processing unit 2030 in the recording unit 2060 (step S2030). After that, the system control unit 2070 determines whether or not the user has operated the moving image switch (operation for ending the shooting of the moving image) (step S2031). If the system control unit 2070 determines that the moving image switch has not been operated, the system control unit 2070 repeats the processing from step S2021 to step S2031 described above. If the system control unit 2070 determines that the moving image switch has been operated, the process ends.

As described above, in the tenth embodiment, the imaging unit 2020 having the imaging device 2100, the detecting unit 2031 that detects the main subject (for example, the person O110 shown in FIG. 34), and the imaging region 113A of the imaging device 2100 , a setting unit 2071 that sets a first area (for example, the main subject area 2201) containing the main subject detected by the detection unit 2031 and a second area (for example, the non-main subject area 2203) different from the first area; and a control unit 2072 for imaging the area and the second area under different imaging conditions. According to such a configuration, it is possible to set an area according to the main subject and perform imaging under imaging conditions corresponding to the area. Therefore, the detection unit 2031 can highly accurately track the main subject based on the image data captured in the first area.

Also, in the tenth embodiment, the setting unit 2071 changes the first area according to the movement of the main subject. With such a configuration, the detection unit 2031 can detect the movement of the main subject with high accuracy and high speed based on the image data of the first area. Further, in the tenth embodiment, a focus detection unit 2074 that detects the focus position of the main subject is provided, and the setting unit 2071 predicts movement of the main subject based on information regarding the focus position detected by the focus detection unit 2074. According to such a configuration, even when the main subject is moving, the main subject area can be set according to the movement of the main subject.

Further, in the tenth embodiment, the control unit 2072 sets at least one of the gain and the frame rate as imaging conditions for the first region to be higher than that for the second region. According to such a configuration, the tracking response of the main subject by the detection unit 2031 is improved.

Further, in the tenth embodiment, the display control unit 2073 that displays the first area on the display unit 2050 is provided. According to such a configuration, it is possible to inform the user that the detection unit 2031 has detected the main subject, and to inform the user of the position, size, etc. of the first area. Also, the display control unit 2073 enlarges and displays at least part of the main subject (for example, the face of the person O110) in a predetermined position (for example, the sub-screen 2051S) in the second area. According to such a configuration, even when the size of the display surface 2051 of the display unit 2050 is small, the user can easily confirm the main subject or part of the main subject.

Further, in the tenth embodiment, a mode detection unit 2031 is provided for detecting a mode including facial expressions and actions (for example, blinking) of a person as a main subject. In the case of a state (for example, a smiling face, a state in which eyes are not closed), the imaging unit 2020 is caused to perform main imaging. According to such a configuration, it is possible to capture an image at the optimum timing when the person's facial expression, motion, and the like are performed.

In the tenth embodiment described above, the digital camera 2001 is used as an example of an electronic device, but the electronic device is not limited to this, and may be configured by a device such as a smartphone, a mobile phone, or a personal computer equipped with an imaging function. good too. Also, the image processing unit 2030 and the system control unit 2070 shown in FIG. 31 may be integrated. In this case, the system control unit having one body-side CPU performs the functions of the image processing unit 2030 and the system control unit 2070 by executing processing based on the control program.

<Eleventh Embodiment>
In the tenth embodiment described above, the non-main subject area does not change between when the main subject is detected by the detection unit 2031 and when the main subject is not detected. On the other hand, in the eleventh embodiment, the non-main subject area changes depending on whether the detection unit 2031 detects the main subject or not.

FIG. 37 is a flowchart for explaining the area setting process (step S2005 in FIG. 32 and step S2025 in FIG. 33) in the eleventh embodiment. , the setting unit 2071 determines whether or not the detection unit 2031 has detected the main subject (step S2041).If the setting unit 2071 determines that the detection unit 2031 has detected the main subject, A main subject area and a non-main subject area are set in the pixel area (imaging area) 113A based on the signal indicating the position and size of the main subject from the detection unit 2031 (step S2042). A partial area is set within the main subject area based on a signal indicating the position and size of a part of the main subject from (step S2043) Such processing from steps S2041 to S2043 will be described in the tenth embodiment. It is the same as the processing that was done.

When the detection unit 2031 determines that the main subject is not detected, the setting unit 2071 sets the AF area and the AE area in the pixel area (imaging area) 113A (step S2044). The AF area is an area where an image (object) to be referred to when the AF processing unit 2074 executes AF processing (steps S2007, S2013, S2027) is captured. The AE area is an area where an image (object) to be referred to when the AE processing unit 2075 executes AE processing (steps S2008, S2014, S2028) is captured.

FIG. 38 is a diagram showing an arrangement pattern of AF areas 2114 and AE areas 2115 according to the eleventh embodiment. The array pattern shown in FIG. 38 is an array pattern in which the pixel area 113A is divided into two areas, the AF area 2114 and the AE area 2115. The array pattern shown in FIG. In this arrangement pattern, in the pixel region 113A, the AF area is composed of a block of an odd row (2n−1) in an odd column (2m−1) and a block of an even row (2n) in an even column (2m). 2114 is configured. Also, an AE area 2115 is composed of blocks in odd rows (2n−1) in even columns (2m) and blocks in even rows (2n) in odd columns (2m−1). That is, each block in the pixel region 113A is divided into a checkered pattern. m and n are positive integers (m = 1, 2, 3, ..., n = 1, 2, 3, ...). In such a configuration, since the AF area 2114 and the AE area 2115 are dispersedly arranged over the entire pixel area 113A of the image sensor 2100, the image sensor 2100 can simultaneously capture images in the AF area 2114 and the AE area 2115. can be done.

In FIG. 38, a small number of blocks are set in the pixel region 113A in order to make the arrangement of the blocks easier to see, but a larger number of blocks than those shown in FIG. 38 are set in the pixel region 113A. may be

FIG. 39 is a diagram showing a non-main subject area 2213 when the main subject is not detected by the detection unit 2031. FIG. As shown in FIG. 39, when the detection unit 2031 does not detect the main subject (for example, the person O110 shown in FIG. 34), the setting unit 2071 detects the An AF area 2114 and an AE area 2115 are set (see step S2044). In this case, the setting unit 2071 sets a higher frame rate and higher gain for the AF area 2114 than for the AE area 2115 as the imaging conditions (see steps S2006 and S2026).

In the AF processing (see steps S2007, S2013, and S2027), the AF processing unit 2074 acquires from the image processing unit 2030 a contrast signal representing the contrast of the images of the background subjects O120 and O130 captured in the AF area 2114. . Then, while moving the focusing lens 11c, the AF processing unit 2074, based on the contrast signal from the image processing unit 2030, sets the position of the focusing lens 11c at which the contrast of the images of the objects O120 and O130 is the highest. Detect as The AF processing unit 2074 transmits a control signal to the lens driving control device 15 so as to move the focusing lens 11c to the detected focal position.

In the AE processing (see steps S2008, S2014, and S2028), the AE processing unit 2075 sends luminance distribution data representing the luminance distribution of the image data of the background subjects O120 and O130 captured in the AE area 2115 to the image processing unit 2035. Get from Based on the brightness distribution data, the AE processing unit 2075 then determines imaging conditions (frame rate, shutter speed, gain, etc.) and aperture value that provide proper exposure. The AE processing unit 2075 causes drive control for adjusting the aperture diameter of the diaphragm 14 to be executed by transmitting control information indicating the aperture value corresponding to the determined appropriate exposure to the lens drive control device 15 . The AE processing unit 2075 also outputs an instruction signal to the driving unit 2021 to instruct imaging conditions (frame rate, shutter speed, gain, etc.) according to proper exposure. The drive unit 2021 sets the imaging conditions instructed by the instruction signal.

In this way, when the detection unit 2031 does not detect the main subject, the setting unit 2071 sets the AF area 2114 and the AE area 2115 in the pixel area 113A. Set a high frame rate and high gain. With such a configuration, the image processing unit 2030 can quickly calculate the contrast evaluation value of the images of the subjects O120 and O130 in the non-main subject area 2213 . As a result, the AF processing unit 2074 can quickly detect the focal position in AF processing. On the other hand, the image data used by the AE processing unit 2075 to perform photometry in AE processing does not need to have a high frame rate or high gain. Therefore, by setting the frame rate of the AE area 2115 lower than that of the AF area 2114 and setting the gain of the AE area 2115 lower than that of the AF area 2114, the power consumption is suppressed and the noise of the image data of the AE area 2115 is reduced. ing.

As described above, in the eleventh embodiment, when the detection unit 2031 has not detected the main subject, the setting unit 2071 dispersively sets the third area (for example, the AF area 2114) in the imaging area 113A. A fourth area (for example, the AE area 2115) different from the third area is dispersively arranged, and the control unit 2072 causes the third area to be imaged under the imaging conditions related to focus detection, and the fourth area to be related to exposure detection. Take an image under the imaging conditions. With such a configuration, the focus detection section 2074 can quickly detect the focus position.

Next, a modification of the eleventh embodiment will be described. FIG. 40 is a diagram showing a main subject area 2211, a partial area 2212, and a non-main subject area 2213 when the main subject is detected by the detection unit 2031. FIG. In the eleventh embodiment described above, when the detection unit 2031 does not detect the main subject (see NO in step S2041 in FIG. 37), the setting unit 2071 sets the AF area 2114 and the AE area 2115 in the pixel area 113A. has been set (see step S2044 in FIG. 37). On the other hand, in the modification of the eleventh embodiment, when the detection unit 2031 detects the main subject (see YES in step S2041 in FIG. 37), the setting unit 2071 sets the non-main subject as shown in FIG. An AF area 2114 and an AE area 2115 are set in the subject area 2213 . Thus, in the modification of the eleventh embodiment, when the detection unit 2031 detects the main subject, the setting unit 2071 sets the third area (for example, the AF area 2114) in the second area (for example, the non-main subject area 2213). ) are dispersively arranged and a fourth region (for example, the AE region 2115) different from the third region is dispersively arranged. The area is imaged under imaging conditions related to exposure detection. According to such a configuration, when a new main subject (for example, a main subject different from the person O110) enters the second area (non-main subject area 2213), the image processing unit 2030 detects the second area (non-main subject area 2213). It is possible to quickly calculate the contrast evaluation value of the image of the main subject in the subject area 2213).

FIG. 41 is a diagram showing a modification of the arrangement pattern of AF areas 2116 and AE areas 2117 according to the eleventh embodiment. In the eleventh embodiment described above, as shown in FIG. 38, the array pattern is such that each block in the pixel region 113A is arranged in a checkered pattern. On the other hand, in the modification of the eleventh embodiment, as shown in FIG. 41, a plurality of AF regions 2116 are discretely and evenly formed in the pixel region 113A. The AF area 2116 is formed as a plurality of small rectangular areas. Also, in the pixel area 113A, the AE area 2117 is an area other than the AF area 2116. FIG. In the example shown in FIG. 41, the area of the AE area 2117 is wider than the area of the AF area 2116. In the example shown in FIG. That is, the AF area 2116 and the AE area 2117 have different area ratios.

In FIG. 41, a small number of blocks are set in the pixel region 113A in order to make the arrangement of the blocks easier to see, but a larger number of blocks than the blocks shown in FIG. 41 are set in the pixel region 113A. may be

With such a configuration, the AE processing unit 2075 can perform photometry based on the luminance distribution of image data generated in the AE area 2117 over a wide range. As a result, the accuracy of photometry by the AE processing unit 2075 is improved.

<Twelfth Embodiment>
In the eleventh embodiment described above, the AF area and the AE area are set in the pixel area 113A (see FIGS. 38 and 41). In contrast, in the twelfth embodiment, a still image area for capturing a still image and a moving image area for capturing a moving image are set in the pixel area 113A.

FIG. 42 is a flowchart for explaining shooting mode setting processing executed by the system control unit 2070 of the twelfth embodiment. The shooting mode setting process shown in FIG. 42 is executed before the shooting operation shown in FIGS. 32 and 33 is performed. As shown in FIG. 42, first, the system control unit 2070 confirms the shooting mode set by the user by operating the operation unit 2055 (step S2051). Then, the system control unit 2070 determines whether or not the shooting mode confirmed in step S2051 is the normal shooting mode (step S2052). Here, the normal shooting mode means that the setting unit 2071 shoots a still image or a moving image of a subject using the pixel area 113A as one area without dividing the pixel area (imaging area) 113A into a still image area and a moving image area. shooting mode. This normal shooting mode is a shooting mode in which normal still images or moving images are generally shot. That is, it is a shooting mode for shooting still images or moving images described in the tenth and eleventh embodiments.

If the system control unit 2070 determines that the shooting mode confirmed in step S2051 is the normal shooting mode, it sets the shooting mode to the normal shooting mode (step S2053). On the other hand, when the system control unit 2070 determines that the shooting mode confirmed in step S2051 is not the normal shooting mode, it sets the shooting mode to the mix mode (step S2054). Here, the mix mode is a shooting mode in which the setting unit 2071 divides the pixel area 113A into a still image area and a moving image area, shoots a still image in the still image area, and shoots a moving image in the moving image area. In this mix mode, still images and moving images can be shot at the same time. Note that the live view image is shot in the still image area.

FIG. 43 is a diagram showing an arrangement pattern of still image areas 2118 and moving image areas 2119 according to the twelfth embodiment. The arrangement pattern of still image areas 2118 and moving image areas 2119 shown in FIG. 43 is the same as the arrangement pattern of AF areas 2114 and AE areas 2115 shown in FIG. That is, the still image area 2118 and the moving image area 2119 are arranged in a checkered pattern in the pixel area 113A. In FIG. 43, a small number of blocks are set in the pixel region 113A in order to make the arrangement of the blocks easier to see, but a larger number of blocks than the blocks shown in FIG. 43 are set in the pixel region 113A. may be

FIG. 44 is a diagram showing a main subject area 2221, a partial area 2222, and a non-main subject area 2223 in the imaging area 113A of the image sensor 2100 of the twelfth embodiment. As shown in FIG. 44, the setting unit 2071 sets a main subject area 2221 including the person O110, which is the main subject detected by the detection unit 2031. As shown in FIG. The setting unit 2071 also sets a partial area 2222 including the face of the person O110, which is the main subject detected by the detection unit 2031. FIG. Also, the setting unit 2071 sets an area other than the main subject area 2221 and the partial area 2222 in the pixel area 113A as a non-main subject area 2223 (see step S2005). In this embodiment, the display control unit 2073 displays the image captured in the still image area 2118 as a live view image on the display surface 2051 of the display unit 2050 (see step S2002). Further, the detection unit 2031 detects the main subject (person O110) and a part of the main subject (person O110's face) based on the image data of the subject shot in the still image area 2118 (steps S2003 and S2004). reference).

The setting unit 2071 sets different imaging conditions for the main subject area 2221, the partial area 2222, and the non-main subject area 2223, as described in the tenth embodiment (see step S2006). Specifically, the setting unit 2071 sets a higher frame rate and higher gain for the main subject area 2221 than for the non-main subject area 2223 . Also, the setting unit 2071 sets a higher frame rate and higher gain for the partial area 2222 than for the main subject area 2221 . Furthermore, the setting unit 2071 sets a lower frame rate for the still image area 2118 than for the moving image area 2119 in the main subject area 2221 . Similarly, the setting unit 2071 sets a lower frame rate for the still image area 2118 than for the moving image area 2119 in the partial area 2222 . Similarly, the setting unit 2071 sets a lower frame rate for the still image area 2118 than for the moving image area 2119 in the non-main subject area 2223 . The reason why the frame rate of the still image area 2118 is set lower than the frame rate of the moving image area 2119 is that the live view image does not need to be an image in which the subject moves smoothly. Note that the setting unit 2071 automatically sets the imaging conditions other than the frame rate and gain that are suitable for the subject in each region, or sets them according to the user's operation of the operation unit 2055 or the like. .

After that, the AF processing unit 2074 executes AF processing based on the image data of the subject captured in the still image area 2118 (see step S2007). Also, the AE processing unit 2075 executes AE processing based on the image data of the subject shot in the still image area 2118 (see step S2008).

After that, the system control unit 2070 performs the processing from steps S2011 to S2014 shown in FIG. to run. Further, the system control unit 2070 performs the processing of steps S2016 and S2017 shown in FIG. Execute. Further, the system control unit 2070 executes the processing from steps S2021 to S2030 shown in FIG. 33 based on the image data generated in the moving image area 2119 in response to the user's operation of the moving image switch (step S2020). .

FIG. 45 is a diagram showing a first display example of the display section 2050A of the twelfth embodiment. As shown in FIG. 45, a display unit 2050A according to the twelfth embodiment includes a first display area 2051A for displaying a still image captured in a still image area 2118 (a live view image before capturing a still image), A second display area 2051B for displaying the moving image captured in the moving image area 2119 is formed. As shown in FIG. 45, in the first display area 2051A, similarly to the display example shown in FIG. there is With such a configuration, the user can simultaneously check a still image (or a live view image) and a moving image.

FIG. 46 is a diagram showing a second display example of the display section 2050B of the twelfth embodiment. As shown in FIG. 46, the display unit 2050B of the twelfth embodiment includes a first display area 2051A for displaying a still image captured in the still image area 2118 (a live view image before capturing the still image), A second display area 2051B for displaying a moving image captured in the moving image area 2119, a third display area 2051C for displaying an enlarged view of the person O110 (that is, the main subject area 2511) as the main subject, and a portion of the main subject. A fourth display area 2051D is formed to enlarge and display the face of the person O110 (that is, the partial area 2512). As shown in FIG. 46, in the first display area 2051A and the third display area 2051C, similarly to the display example shown in FIG. 2602 are displayed. A frame 2602 surrounding the face of the person O110 is displayed in the fourth display area 2051D. With such a configuration, the user can simultaneously check a still image (or a live view image) and a moving image. In addition, the user can easily confirm the person O110 and his face (facial expression).

As described above, in the twelfth embodiment, the setting unit 2071 sets the fifth area (for example, the still image area 2118) for capturing a still image and the sixth area for capturing a moving image in the imaging area 113A. A region (eg, moving image region 2119) is set, and in the fifth region, a first region (eg, main subject region 2221) and a second region (eg, non-main subject region 2223) are set. According to such a configuration, in addition to the effects of the tenth embodiment described above, it is possible to simultaneously capture a still image and a moving image.

Further, in the twelfth embodiment, the control unit 2072 causes the fifth area (eg, still image area 2118) and the sixth area (eg, moving image area 2119) to be imaged under different imaging conditions. According to such a configuration, it is possible to capture a still image under an imaging condition suitable for capturing a still image, and to capture a moving image under an imaging condition suitable for capturing a moving image. In the twelfth embodiment, the setting unit 2071 dispersively arranges the fifth area (for example, the still image area 2118) and dispersively arranges the sixth area (for example, the moving image area 2119) in the imaging area 113A. . According to such a configuration, it is possible to simultaneously capture a still image and a moving image of the same subject (strictly speaking, a subject that looks the same to the user).

In the twelfth embodiment described above, the frame rate of the still image area 2118 is lower than the frame rate of the moving image area 2119. may Further, when performing the main imaging of the still image (that is, when performing the imaging process of step S17), the control unit 2072 does not perform imaging in the still image area 2118, but performs imaging with all the pixels in the pixel area 113A. may Further, in FIG. 46, when the detection unit 2031 does not detect the main subject (person O110), only the first display area 2051A may be enlarged on the display unit 2050B. For example, the AF area and the AE area shown in FIG. 41 may be set in each block forming the still image area 2118 . For example, the AF area and the AE area shown in FIG. 41 may be set in each block forming the moving image area 2119 .

<Thirteenth Embodiment>
In the thirteenth embodiment, the digital camera 2001 in the tenth embodiment is separated into an imaging device 2001A and an electronic device 2001B.

FIG. 47 is a block diagram showing configurations of an imaging device 2001A and an electronic device 2001B according to the thirteenth embodiment. In the configuration shown in FIG. 47, the imaging device 2001A is a device for imaging a subject. This imaging apparatus 2001A includes a lens unit 2010, an imaging unit 2020, an image processing unit 2030, a work memory 2040, an operation unit 2055, a recording unit 2060, and a first system control unit 2070A. The configuration of the lens unit 2010, the image capturing unit 2020, the image processing unit 2030, the work memory 2040, the operation unit 2055, and the recording unit 2060 in the imaging device 2001A is the same as the configuration shown in FIG. Therefore, the same configurations are denoted by the same reference numerals, and overlapping descriptions are omitted.

The electronic device 2001B is a device that displays images (still images, moving images, live view images). This electronic device 2001B includes a display section 2050 and a second system control section 2070B. Note that the configuration of the display unit 2050 in the electronic device 2001B is the same as the configuration shown in FIG. Therefore, the same components are denoted by the same reference numerals, and overlapping descriptions are omitted.

The first system control section 2070A has a first communication section 2075A. Also, the second system control unit 2070B has a second communication unit 2075B. The first communication unit 2075A and the second communication unit 2075B transmit and receive signals to and from each other in a wired or wireless manner. In such a configuration, the first system control unit 2070A transfers image data (image data processed by the image processing unit 2030, image data recorded in the recording unit 2060) to the second system via the first communication unit 2075A. It is transmitted to the communication section 2075B. Second system control unit 2070B causes display unit 2050 to display the image data received by second communication unit 2075B.

The configuration shown in FIG. 31 (the setting unit 2071, the control unit 2072, the display control unit 2073, the AF processing unit 2074, and the AE processing unit 2075) is provided in either the first system control unit 2070A or the second system control unit 2070B. good too. All the configuration shown in FIG. 31 may be provided in the first system control unit 2070A or the second system control unit 2070B, and part of the configuration shown in FIG. 31 may be provided in the first system control unit 2070A, 31 may be provided in the second system control section 2070B.

Note that the imaging device 2001A is configured by, for example, a digital camera, smartphone, mobile phone, or personal computer having an imaging function and a communication function, and the electronic device 2001B is, for example, a smartphone, a mobile phone, or a portable personal It consists of portable terminals such as computers.

The first system control unit 2070A shown in FIG. 47 is implemented by the CPU executing processing based on the control program. Also, the second system control unit 2070B shown in FIG. 47 is implemented by the CPU executing processing based on the control program.

As described above, in the thirteenth embodiment, in addition to the effects described in the tenth embodiment, an image captured by the imaging device 2001A using a mobile terminal such as a smartphone is displayed on the display unit 2050 of the electronic device 2001B. can be displayed.

In the configuration shown in FIG. 47, image processing section 2030 and first system control section 2070A may be integrated. In this case, a system control unit having one CPU performs processing based on a control program to perform the function of the image processing unit 2030 and the function of the first system control unit 2070A.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various modifications and improvements can be made to the above embodiments without departing from the scope of the present invention. Also, one or more of the requirements described in the above embodiments may be omitted. Such modifications, improvements, and omissions are also included in the technical scope of the present invention. Moreover, it is also possible to appropriately combine the configurations of the above-described embodiments and modifications and apply them.

For example, although the arrangement of the color filters 102 is the Bayer arrangement in each of the above-described embodiments, an arrangement other than this arrangement may be used. Also, the number of pixels forming the unit group 131 should include at least one pixel. Also, it is sufficient that each block includes at least one pixel. Therefore, it is possible to perform imaging under different imaging conditions for each pixel.

Further, in each of the above-described embodiments, the drive unit 2021 may be partially or wholly mounted on the imaging chip 113 , or partially or wholly mounted on the signal processing chip 111 . Also, part of the configuration of the image processing unit 2030 may be mounted on the imaging chip 113 or the signal processing chip 111 . Also, part of the configuration of the system control unit 2070 may be mounted on the imaging chip 113 or the signal processing chip 111 .

In addition, in the arrangement pattern shown in FIG. 38, the AF regions 2114 and the AE regions 2115 are arranged in a checkered pattern, but the lines of the AF regions 2114 and the lines of the AE regions 2115 are arranged in the row direction or the column direction. may be alternately arranged. In addition, in the arrangement pattern shown in FIG. 43, the still image area 2118 and the moving image area 2119 are arranged in a checkered pattern. They may be arranged alternately in the column direction.

Further, in the tenth embodiment described above, the detection unit 2031 detects a person or face, which is the main subject, using a known face detection function. However, it is not limited to such a configuration, and the detection unit 2031 compares a plurality of image data (frames) obtained in time series from the image processing unit 2030, and selects a moving subject (moving subject) as the main subject. may be detected. Further, the detection unit 2031 may detect the main subject by identifying the boundary of the main subject based on the contrast between the bright and dark areas in the image data and the color change.

Further, in each of the above-described embodiments, the setting unit 2071 sets a high frame rate and a high gain in a predetermined area, so that the detection unit 2031 detects the main subject with high accuracy in the predetermined area, and the AF processing unit 2074 It is configured to detect the focal position at high speed in a predetermined area. However, it is not limited to such a configuration, and the setting unit 2071 sets either a high frame rate or a high gain in a predetermined area so that the detection unit 2031 detects the main subject with high accuracy in the predetermined area, The AF processing unit 2074 may be configured to detect the focal position at high speed in a predetermined area.

Further, although the sub-screen 2051S shown in FIG. 35 is arranged on the upper right of the display surface 2051, it is not limited to such a position. For example, it may be arranged at an arbitrary position near the main subject detected by the detection unit 2031 . However, it is desirable that the sub-screen 2051S be positioned so as not to overlap the main subject. Also, the sub-screen 2051S may be fixed, or may be movable according to the user's operation (for example, the user's operation of the touch panel 2052). Also, the size of the sub-screen 2051S may be fixed, or may be changeable according to the user's operation (for example, the user's operation of the touch panel 2052). Note that when the size of the sub-screen 2051S is changed according to the user's operation, the size of the image displayed in the sub-screen 2051S may also be changed in accordance with the change in the size of the sub-screen 2051S. Also, the size of the image displayed in the sub-screen 2051S may be fixed regardless of the change in the size of the sub-screen 2051S.

In the configuration shown in FIG. 37, the setting unit 2071 sets the AF region 2114 and the AE region 2115 in the pixel region 113A based on the detection of the main subject by the detection unit 2031. It is configured to switch between a state in which the area 2114 and the AE area 2115 are not set. That is, the detection unit 2031 detects a change in the state of the subject, and the state change detected by the detection unit 2031 (detection of the main subject) is used as a trigger to detect the focal position of the subject in the imaging region 113A of the image sensor 2100. A first setting state in which an area for performing exposure detection (for example, an AF area 2114) and an area for performing exposure detection (for example, an AE area 2115) are set, and a second setting state (AF area 2114 and AE and a switching unit (setting unit 2071) for switching between the state in which the area 2115 is not set). However, the switching unit (setting unit 2071) is not limited to switching between the first setting state and the second setting state when the detection unit 2031 detects the main subject (target). Triggered by a change in the focal position detected by 2074, the first setting state and the second setting state may be switched.

<14th Embodiment>
FIG. 48 is a block diagram showing the configuration of a digital camera 3001 according to the fourteenth embodiment. 48, a digital camera 3001 as an example of electronic equipment corresponds to the digital camera 1 in FIG. 7, a lens unit 3010 corresponds to the lens unit 10 in FIG. 7, and an imaging unit 3020 corresponds to the imaging unit 20 in FIG. , the image sensor 3100 corresponds to the image sensor 100 in FIG. 7, the driving unit 3021 corresponds to the driving unit 21 in FIG. 7, the image processing unit 3030 corresponds to the image processing unit 30 in FIG. 7, the display unit 3050 corresponds to the display unit 50 in FIG. 7, the display panel 3051 corresponds to the display panel 51 in FIG. 7, and the touch panel 3052 corresponds to the touch panel 52 in FIG. 7, the recording unit 3060 corresponds to the storage unit 60 in FIG. 7, and the system control unit 3070 corresponds to the system control unit 70 in FIG. Of these configurations, the configuration of the lens unit 3010, imaging unit 3020, imaging device 3100, driving unit 3021, work memory 3040, display unit 3050, operation unit 3055, and recording unit 3060 will be omitted.

In this embodiment, the image processing unit 3030 generates luminance distribution data representing the luminance distribution of the subject image based on the gradation value of each pixel of the RAW data, in addition to the processing executed by the image processing unit 30 in FIG. Generate. The brightness distribution data is also called brightness mapping information.

Moreover, in this embodiment, the image processing unit 3030 includes a conversion unit 3031 as shown in FIG. The conversion unit 3031 converts each region in the RAW data when images are captured under different imaging conditions (for example, charge accumulation time, frame rate, gain) for each of a plurality of regions in the image region (imaging region) 113A of the image sensor 3100. The luminance level is converted into a luminance level when each region is imaged under the same or substantially the same imaging conditions. An image processing unit 3030 performs various image processing on the RAW data converted by the conversion unit 3031 .

A system control unit 3070 controls the overall processing and operation of the digital camera 3001 . This system control unit 3070 has a body-side CPU (Central Processing Unit). In this embodiment, the system control unit 3070 divides the imaging surface (pixel region 113A) of the imaging element 3100 (imaging chip 113) into a plurality of blocks, and sets different charge accumulation times (or charge accumulation times) and frame rates between the blocks. , to acquire an image with gain. For this reason, the system control unit 3070 instructs the driving unit 3021 on the position, shape, range of blocks, and accumulation conditions for each block.

In addition, the system control unit 3070 acquires an image with a thinning rate that differs between blocks, the number of addition rows or addition columns for adding pixel signals, and the number of bits for digitization. Therefore, the system control unit 3070 instructs the driving unit 3021 about imaging conditions for each block (decimation rate, number of rows or columns to be added for adding pixel signals, and number of bits for digitization). Further, the image processing unit 3030 executes image processing under different imaging conditions (control parameters such as color signal processing, white balance adjustment, gradation adjustment, compression rate, etc.) between blocks. Therefore, the system control unit 3070 instructs the image processing unit 3030 on imaging conditions for each block (control parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate).

Also, the system control unit 3070 causes the recording unit 3060 to record the image data generated by the image processing unit 3030 . Further, the system control unit 3070 causes the display unit 3050 to display an image by outputting the image data generated by the image processing unit 3030 to the display unit 3050 . Further, the system control unit 3070 reads the image data recorded in the recording unit 3060 and causes the display unit 3050 to output the image data, thereby causing the display unit 3050 to display an image. Images displayed on the display unit 3050 include still images, moving images, and live view images.

In this embodiment, the system control unit 3070 includes an area dividing unit 3071 and a control unit 3072, as shown in FIG. The area division unit 3071 acquires luminance distribution data of the subject image generated by the image processing unit 3030 . Then, the area division unit 3071 recognizes the luminance level of each pixel in the subject image based on the acquired luminance distribution data, and divides the imaging area 113A of the imaging element 3100 into a plurality of areas according to the luminance level in units of blocks. .

The control unit 3072 controls the image sensor so that the luminance level of each region divided by the region dividing unit 3071 falls within a specific range (for example, a range of 35% to 80% of the saturation level of the image sensor 3100; see FIG. 51). 3100 imaging conditions (for example, charge accumulation time, frame rate, gain) are set for each region. Further, the control unit 3072 controls the driving of the imaging device 3100 so as to perform imaging under imaging conditions set for each region. Further, the control unit 3072 performs control to record image data of still images and moving images generated by the image processing unit 3030 in the recording unit 3060 . The control unit 3072 outputs the image data generated by the image processing unit 3030 to the display unit 3050 and controls the display screen 3051 of the display unit 3050 to display an image (live view image, still image, moving image).

In the system control unit 3070, the area dividing unit 3071 and the control unit 3072 respectively correspond to processing and control executed by the body side CPU based on the control program.

Next, the photographing operation of the digital camera 3001 according to the fourteenth embodiment will be described. FIG. 49 is a flow chart for explaining the imaging operation executed by the system control unit 3070 of the fourteenth embodiment. In the processing shown in FIG. 49, when the user operates the operation unit 3055 or the like to start shooting after the digital camera 3001 is powered on, the control unit 3072 starts shooting a live view image. (Step S3001). That is, the control unit 3072 outputs an instruction signal to the driving unit 3021 to execute drive control of the imaging device 3100 . At this time, the control unit 3072 causes the entire surface of the pixel region 113A of the image sensor 3100 to perform imaging under the same live view image imaging conditions (for example, predetermined initial imaging conditions). Also, the control unit 3072 displays the live view image captured by the imaging device 3100 on the display screen 3051 of the display unit 3050 (step S3002).

FIG. 50 is a diagram showing a display example of an image 3500 consisting of a plurality of areas a11, a12, a21, a22, a31, and a41 corresponding to luminance levels. For example, assume that an image 3500 as shown in FIG. 50 is captured in step S3001 and displayed as a live view image on the display screen 3051 of the display unit 3050 in step S3002.

An image 3500 shown in FIG. 50 includes a house O210, the ground O220, the sky O230, and the sun O240 as subjects. As shown in FIG. 50, in the image 3500, the area a11 on the side of the house O210 that is not exposed to the light from the sun O240 is the area with the lowest luminance level. The average luminance level of this area a11 is called level (1). Also, the area a12 of the roof of the house O210 has a luminance level higher than level (1). The average luminance level of this area a12 is called level (2).

Also, the region a21 of the shadow of the house O210 on the ground O220 has the same luminance level (that is, level (1)) as the region a11. An area a22 on the ground O220 other than the area a21 (an area on the ground O220 illuminated by the sun O240) also has the same brightness level as the area a21 (that is, level (2)). Also, the area a31 of the sky O230 has a luminance level higher than the level (2). The average luminance level of this area a31 is called level (3). Also, the sun O240 and the area a41 around the sun O240 are areas with the highest luminance level. The average luminance level of this area a41 is called level (4). In FIG. 50, the line connecting the point A1 in the area a12 and the point A2 in the area a22 indicates a normal line perpendicular to the boundary line between the areas a12 and a22.

Thereafter, the system control unit 3070 determines whether or not the release switch has been half-pressed (operation of SW1) (step S3003). If the system control unit 3070 determines that the release switch has not been half-pressed, it repeats the processes of steps S3001 and S3002 described above.

When the system control unit 3070 determines that the release switch has been half-pressed (YES in step S3003), the system control unit 3070 executes shooting preparation processing such as AF processing. Also, the area dividing unit 3071 acquires the luminance distribution data generated by the image processing unit 3030 (step S3004). Specifically, the region division unit 3071 outputs an instruction signal to the image processing unit 3030 to instruct generation of luminance distribution data. When the image processing unit 3030 receives the instruction signal from the image processing unit 3030, the image processing unit 3030 generates a luminance distribution representing the distribution of luminance values (luminance levels) of each pixel based on the gradation value of each pixel in the RAW data of the live view image. Generate data. For example, the image processing unit 3030 converts the RGB gradation values (for example, values of 0 to 255) of each pixel in the RAW data into luminance values of each pixel based on a predetermined conversion formula to generate luminance distribution data. do. In this luminance distribution data, the luminance value of each pixel is expressed like a contour line. Image processing section 3030 then outputs the generated brightness distribution data to system control section 3070 . The area division unit 3071 acquires the luminance distribution data output from the image processing unit 3030. FIG.

Next, the area dividing unit 3071 recognizes the luminance level of each pixel in the subject image based on the acquired luminance distribution data, and divides the imaging area 113A of the image sensor 3100 into a plurality of pixels according to the luminance level based on the luminance level threshold. area in units of blocks (step S3005).

FIG. 51 is a graph showing the relationship between the brightness of subjects O210 to O240 shown in FIG. In FIG. 51, the horizontal axis indicates the actual brightness of each subject O210-O240 in the image 3500 shown in FIG. The vertical axis indicates the luminance level of the image signal output by the imaging device 3100 according to the actual brightness of each of the subjects O210 to O240 (that is, the output level of the image signal).

In the imaging device 3100, the brightness level of the image signal changes according to the brightness of the subject. However, when the brightness of the object exceeds a certain level, the brightness level of the image signal does not increase and remains constant. The luminance level of such an image signal is the saturation level. When the luminance level of the image signal exceeds the saturation level, overexposure occurs in the image. Further, when the brightness of the subject falls below a certain luminance level, the output level of the image signal does not decrease and becomes constant. In this case, blackout occurs in the image. In the image sensor 3100, the range of luminance levels in which neither blown-out highlights nor blocked-up shadows occur is called an allowable range.

In the example shown in FIG. 51, the actual brightness of the area a41 (the sun O240 and the surroundings of the sun O240) is a certain brightness l2 or higher, and the brightness level of the image signal of the image sensor 3100 is the saturation level. Therefore, the image of the area a41 has whiteout. Also, the actual brightness of the areas a11 and a12 (the sides of the house O210 and the shadow of the house O210) is less than or equal to a certain brightness l1, and the images of the areas a11 and a12 have blocked up shadows.

In addition, in the image pickup device 3100, the brightness level of the image signal of the image pickup device 3100 does not rise in proportion to the increase in the brightness of the object in all the allowable ranges of the brightness level. In the image pickup device 3100, there is linearity between changes in the brightness of the object and changes in the brightness level within a certain range of the allowable range of the brightness level, and in other ranges, changes in the brightness and the brightness of the object are linear. No linearity with level changes. A luminance level range in which there is linearity between a change in brightness of an object and a change in luminance level is called a specific range. In this embodiment, as shown in FIG. 51, when the saturation level of the imaging element 3100 is 100%, the specific range is a luminance level range of 35% to 80% of the saturation level.

In the present embodiment, as shown in FIG. 51, as the luminance level thresholds, a luminance level of 35% of the saturation level is set as a threshold t11, a luminance level of 80% of the saturation level is set as a threshold t12, and a saturation level (100%) is set. is set as the threshold value t13. Further, as shown in FIG. 51, the luminance levels (level (1)) of the areas a11 and a12 are less than the threshold t11, and the luminance levels (level (2)) of the areas a21 and a22 are equal to or greater than the threshold t11 and less than the threshold t12. , the luminance level (level (3)) of the area a31 is greater than or equal to the threshold t12 and less than the threshold t13, and the luminance level (level (3)) of the area a41 is greater than or equal to the threshold t13. Therefore, in step S3005, the region dividing unit 3071 divides the pixel region 113A of the image sensor 3100 into regions a11 and a12 whose luminance level is less than the threshold value t11, and regions a11 and a12 whose luminance level is less than the threshold value t11, based on the luminance level thresholds t11, t12, and t13. The area is divided into areas a21 and 22 having a luminance level equal to or higher than the threshold t11 and less than the threshold t12, an area a31 having a luminance level equal to or higher than the threshold t12 and less than the threshold t13, and an area a41 having a luminance level equal to or higher than the threshold t13. Note that the specific range is the range from the threshold t11 to the threshold t12 as a result of the region dividing unit 3071 setting the threshold in this way.

Next, the control unit 3072 sets the imaging conditions of the image sensor 3100 for each region such that the luminance level of each region divided by the region division unit 3071 falls within a specific range (step S3006). For example, the control unit 3072 raises the luminance levels of the regions a11 and a12 by increasing the gain in the regions a11 and a12 and lengthening the charge accumulation time. Also, the control unit 3072 reduces the luminance levels of the areas a31 and a41 by shortening the charge accumulation times of the areas a31 and a41.

Even if the control unit 3072 sets the imaging conditions for each region in step S3006, if the difference between the bright portion and the dark portion is large, the luminance level of each region may not belong to the specific range. obtain. For example, if the control unit 3072 increases the gain of the area a12 so that the luminance level of the area a12 falls within the specific range, the luminance level near the boundary between the areas a12 and a22 becomes too high and does not belong to the specific range. If the control unit 3072 shortens the charge accumulation time of the area a41 so that the luminance level of the area a41 falls within the specific range, the luminance level near the boundary between the areas a41 and a31 becomes too low and does not belong to the specific range. .

Therefore, the control unit 3072 determines whether or not the luminance level of each region after setting the imaging conditions in step S3006 belongs to the specific range (step S3007). Then, if the control unit 3072 determines that the luminance level does not belong to the specific range yet, the region dividing unit 3071 again performs region division for the portion (region) that does not belong to the specific range (step S3005). Also, the control unit 3072 sets the imaging condition for each area so that the brightness level of each area divided by the area division unit 3071 belongs to the specific range (step S3006). The region division unit 3071 and the control unit 3072 repeatedly execute the processing of steps S3005 to S3007 as described above until the luminance levels of all the pixel regions 113A of the image sensor 3100 belong to the specific range.

FIG. 52 is a waveform diagram showing changes in luminance level between points A1 and A2 in FIG. In FIG. 52, the horizontal axis indicates the distance between points A1 and A2, and the vertical axis indicates the luminance levels of areas a12 and a22. As described above, point A1 exists in area a12 and point A2 exists in area a22. Also, the line connecting the points A1 and A2 is a normal line perpendicular to the boundary line between the areas a12 and a22.

As shown in FIG. 52, in the vicinity of the boundary between the areas a12 and a22, the brightness level (level (2)) of the area a22 sharply drops to the brightness level (level (1)) of the area a12. A threshold t11 exists between the luminance level of the area a22 and the luminance level of the area a12. In the present embodiment, a boundary region r10 is defined as a region extending from the brightness level threshold t11 to the brightness level of the region a12. In FIG. 52, it is assumed that the gain of the area a12 and the area a22 is 1 (for example, the ISO sensitivity is 100).

FIG. 53 is a waveform diagram showing luminance levels when the area division and imaging conditions are set so that the luminance levels shown in FIG. 52 fall within a specific range. In FIG. 53, the horizontal axis indicates the distance between the points A1 and A2, and the vertical axis indicates the brightness levels of the area a12, the area a22, and the boundary area r10. In step S3006, the control unit 3072 increases the gain of the boundary region r10 and the region a12 so that the luminance level of the boundary region r10 and the region a12 is increased to a luminance level slightly below the threshold value t12. For example, the controller 3072 doubles the gains of the boundary region r10 and the region a12. In this case, as shown in FIG. 53A, the brightness levels of the boundary region r10 and the region a12 are doubled.

In step S3007, as shown in FIG. 53A, the control unit 3072 determines that the luminance levels of the boundary region r10 and region a12 do not yet belong to the specific range (range from threshold t11 to threshold t12). In this case, in step S3005, the region dividing unit 3071 divides the boundary region r10 into a boundary region r11 with a luminance level higher than the threshold t11 and a boundary region r12 with a luminance level lower than the threshold t11. Then, in step S3006, the control unit 3072 increases the gains of the boundary region r12 and the region a12 so that the luminance level of the boundary region r12 and the region a12 is increased to a luminance level slightly below the threshold value t12. For example, the control unit 3072 sets the gains of the boundary regions r12 and a12 to four times the gain of the region a22 (twice the gain of the boundary region r11). In this case, as shown in FIG. 53B, the brightness levels of the boundary region r12 and the region a12 increase by four times the gain of the region a22. In step S3007, the control unit 3072 determines that the luminance levels of the boundary area r12 and the area a12 belong to the specific range, as shown in FIG. 53(B).

FIG. 54 is a diagram showing strip-shaped boundary areas r11 and r12 set at the boundary between the level (1) area a12 and the level (2) area a22. As described above, the region division unit 3071 and the control unit 3072 execute the processes of steps S3005 to S3007, thereby forming a plurality of strip-shaped boundary regions between the adjacent regions a12 and a22 as shown in FIG. r11 and r12 are set. Note that the larger the difference between the bright portion and the dark portion between adjacent regions, the greater the number of strip-shaped boundary regions.

FIG. 55 is a partially enlarged view showing the level (3) area a31 and the level (4) area a41 shown in FIG. In FIG. 55, the point D1 exists in the area a31 and the point D2 exists in the area a41. Also, the line connecting the point D1 in the area a31 and the point D2 in the area a41 is a normal line perpendicular to the boundary line between the areas a31 and a41.

FIG. 56 is a waveform diagram showing changes in luminance level between points D1 and D2 in FIG. In FIG. 56, the horizontal axis indicates the distance between the points D1 and D2, and the vertical axis indicates the luminance levels of the areas a31 and a41. Note that the image sensor 3100 outputs an image signal having a saturation level as the luminance level of the area a41, but as shown in FIG. is a higher luminance level than In this embodiment, as shown in FIG. 56, the area from the point where the brightness level of area a41 starts to decrease to the point where the brightness level of area a31 is reached is called boundary area r20. In FIG. 56, it is assumed that the charge accumulation time (shutter speed) of the areas a31 and a41 is 1/100 seconds.

FIG. 57 is a waveform diagram showing luminance levels when area division and imaging condition setting are performed so that the luminance levels shown in FIG. 56 fall within a specific range. In FIG. 57, the horizontal axis indicates the distance between the points D1 and D2, and the vertical axis indicates the brightness levels of the area a31, the boundary area r20, and the area a41. In step S3006, the control unit 3072 accumulates charges in the area a31, the boundary area r20, and the area a41 so as to lower the luminance levels of the area a31, the boundary area r20, and the area a41 to a luminance level slightly lower than the threshold value t12. shorten the time. For example, the control unit 3072 sets the charge accumulation times of the area a31, the boundary area r20, and the area a41 to 1/10 (1/1000 second). In this case, the brightness levels of the area a31, the boundary area r20, and the area a41 are reduced to 1/10.

In step S3007, the control unit 3072 determines that the luminance levels of the area a31 and the boundary area r20 do not yet belong to the specific range (range from threshold t11 to threshold t12). In this case, in step S3005, the region dividing unit 3071 divides the boundary region r20 into a boundary region r21 with a luminance level higher than the threshold t11 and a boundary region r22 with a luminance level lower than the threshold t11. Then, in step S3006, the control unit 3072 lengthens the charge accumulation time of the boundary region r22 and the region a31 so as to increase the luminance level of the boundary region r22 and the region a31 to a luminance level slightly below the threshold value t12. For example, the control unit 3072 sets the charge accumulation time of the boundary region r22 and the region a31 to double (1/500 second). In this case, the brightness levels of the boundary region r22 and the region a31 are doubled.

In step S3007, the control unit 3072 determines that the luminance levels of the boundary region r22 and the region a31 do not yet belong to the specific range (range from threshold t11 to threshold t12). In this case, in step S3005, the region dividing unit 3071 divides the boundary region r22 into a boundary region r22a with a luminance level higher than the threshold t11 and a boundary region r22b with a luminance level lower than the threshold t11. Then, in step S3006, the control unit 3072 lengthens the charge accumulation time of the boundary region r22b and the region a31 so as to raise the luminance level of the boundary region r22b and the region a31 to a luminance level slightly below the threshold value t12. For example, the control unit 3072 sets the charge accumulation time of the boundary region r22b and the region a31 to double (1/250 second). In this case, the brightness levels of the boundary region r22b and the region a31 are doubled. In step S3007, the control unit 3072 determines that the luminance levels of the boundary region r22b and the region a31 belong to the specific range, as shown in FIG.

Note that FIG. 52 and FIG. 53 describe the area division on the line connecting the points A1 and A2 and the setting of the imaging conditions (processing of steps S3005 to S3007). The region division on the line connecting the points D1 and D2 and the setting of the imaging conditions (processing of steps S3005 to S3007) have been described. However, the system control unit 3070 (region dividing unit 3071 and control unit 3072) divides regions and sets imaging conditions (processing in steps S3005 to S3007) on all straight lines in the normal direction to the boundary between adjacent regions. to run.

Returning to the description of FIG. 49, when the system control unit 3070 determines in step S3007 that the luminance level belongs to the specific range, whether the user has fully pressed the release switch (operation of SW2). It is determined whether or not (step S3008). When the system control unit 3070 determines that the release switch has not been fully pressed (NO in step S3008), the processes from steps S3001 to S3007 are repeated.

When the system control unit 3070 determines that the release switch has been fully pressed (YES in step S3008), the control unit 3072 executes still image capturing processing (step S3009). That is, the control unit 3072 drives and controls the imaging element 3100 so as to perform imaging under the imaging conditions (charge accumulation time, gain, etc.) set for each region in step S3006. Specifically, the control unit 3072 outputs an instruction signal to the driving unit 3021 to instruct setting of the plurality of areas set in step S3005 in units of blocks. Further, the control unit 3072 outputs to the driving unit 3021 an instruction signal for instructing the imaging conditions set for each area in step S3006. The driving unit 3021 executes imaging under imaging conditions for each region based on the instruction signal from the control unit 3072 .

FIG. 58 is a timing chart showing imaging timings in the regions a41, r21, r22a, r22b, and a31. As shown in FIG. 58, in step S3006, the control unit 3072 sets the charge accumulation time T1 (for example, 1/1000 second) for the region a41 and the boundary region r21, and sets the charge accumulation time T2 (for example, 1/100 second) for the boundary region r22a. 500 seconds) is set, and the charge accumulation time T3 (for example, 1/250 seconds) of the boundary region r22b and the region a31 is set. In this case, in step S3009, the control unit 3072 performs image capturing for the region a41 and the boundary region r21 for the charge accumulation time T1, image capturing for the boundary region r22a for the charge accumulation time T2, and charge accumulation for the boundary region r22b and the region a31. Imaging is performed at time T3. In this way, the control unit 3072 controls the image pickup device 3100 so that the image pickup timings based on the image pickup conditions for each of the plurality of areas partially overlap.

Generally, HDR (High Dynamic Range) imaging is widely known as an image synthesizing technique for recording and displaying images with a wide dynamic range. In HDR imaging, a plurality of images are captured while changing imaging conditions (for example, exposure), and an image with little blown-out highlights or blocked-up shadows is generated by synthesizing the images. However, in the conventional HDR, for example, since the imaging times for capturing two images with different imaging conditions are different, the subject may move or the user (photographer) may move the digital camera 3001. There is In this case, since the multiple images are not images of the same subject, it is difficult to synthesize the images. On the other hand, in the fourteenth embodiment, the imaging time of a plurality of images with different imaging conditions can be the same time (or substantially the same time) (see FIG. 58). Therefore, the configuration of the fourteenth embodiment can solve the problems of conventional HDR imaging. An image with a wide dynamic range can be generated by the image processing unit 3030 synthesizing the images for each region.

Next, image processing by the image processing unit 3030 will be described. FIG. 59 is a flowchart for explaining image processing executed by the image processing unit 3030 of the fourteenth embodiment. When the image processing unit 3030 receives the RAW data output from the image sensor 3100, the image processing unit 3030 executes the image processing shown in FIG. In the process shown in FIG. 59, the conversion unit 3031 performs the imaging conditions (charge accumulation time, gain, etc.) that are different for each of a plurality of areas in the image area 113A of the imaging device 3100. The luminance level of the region is converted into a luminance level when imaging is performed under the same or substantially the same imaging conditions (for example, imaging conditions such as charge accumulation time and gain that provide proper exposure) in each region (step S3011). ).

Specifically, in step S3006 of FIG. 49, the control unit 3072 sets different gains for each area (for example, the ISO sensitivity of the area a12 is 100, the ISO sensitivity of the boundary area r11 is 200, and the ISO sensitivity of the boundary area r12 and the area a22 is It is assumed that the sensitivity is set to 400). In this case, in step S3011, the conversion unit 3031 converts the brightness level of each region into a brightness level when the image is captured with a gain (for example, ISO sensitivity of 150) that provides proper exposure. Further, in step S3006 of FIG. 49, the control unit 3072 sets the charge accumulation time different for each region (for example, the charge accumulation time of the region a41 and the boundary region r21 is 1/1000 second, the charge accumulation time of the boundary region r22a is 1/1000 second). 500 seconds, and the charge accumulation time of the boundary region r22b and the region a31 is 1/250 seconds). In this case, in step S3011, the conversion unit 3031 converts the luminance level of each region into a luminance level when imaging is performed with a charge accumulation time (for example, 1/300 second) that provides proper exposure. .

The conversion unit 3031 performs conversion processing of the luminance level of each area as described above, so that the luminance level of each area can be returned (aligned) to a luminance level that provides proper exposure. However, the image processing unit 3030 records the RAW data before the conversion processing by the conversion unit 3031 in the work memory 3040 and the recording unit 3060 .

Next, the image processing unit 3030 synthesizes the RAW data of each area whose luminance level has been converted in step S3011 by the conversion unit 3031 (step S3012). Then, the image processing unit 3030 performs various image processing such as color signal processing (color tone correction), white balance adjustment, sharpness adjustment, gamma correction, and gradation adjustment on the RAW data synthesized in step S3012 (step S3013). ). At this time, the image processing unit 3030 performs various image processing using not only the RAW data synthesized in step S3012, but also the RAW data before the conversion processing by the conversion unit 3031, so that a bright subject (for example, the sun O240) Even when image data with a brightness level suitable for the subject or image data with a brightness level suitable for a dark subject (for example, the side of the house O210) is generated, the brightness resolution of the image data is ensured. The image processing unit 3030 then converts the image data into image data in a predetermined format such as JPEG.

After that, the control unit 3072 performs control to record the image data of the still image generated by the image processing unit 3030 in the recording unit 3060 . Further, the control unit 3072 causes the display unit 3050 to output the image data generated by the image processing unit 3030 and display a still image on the display screen 3051 of the display unit 3050 .

In the fourteenth embodiment described above, the operation of the digital camera 3001 for capturing a still image has been described with reference to FIGS. The configuration of the 14th embodiment can also be applied to the operation. However, in this case, the control unit 3072 sets the frame rate for each region instead of setting the charge accumulation time for each region in step S3006. It should be noted that as the frame rate increases, the charge accumulation time per frame becomes shorter.

In the fourteenth embodiment described above, the control unit 3072 changes the luminance level of each region by setting the gain or the charge accumulation time for each region in step S3006. and the charge accumulation time may be set to change the brightness level of each area. In the case of moving images, the control unit 3072 may change the luminance level of each region by setting the gain or frame rate for each region in step S3006, and set both the gain and the frame rate in step S3006. may be used to change the brightness level of each region.

As described above, in the fourteenth embodiment, the imaging region 113A of the imaging device 3100 is divided into a plurality of regions according to the luminance levels based on the luminance distribution data of the subject image captured by the imaging device 3100. A dividing unit 3071 and a control unit 3072 that sets the imaging conditions of the imaging device 3100 for each region so that the luminance level of each region divided by the region dividing unit 3071 falls within a specific range. According to such a configuration, it is possible to obtain an image with a wide dynamic range by varying the imaging conditions of the imaging element for each of the plurality of regions.

Further, in the fourteenth embodiment, the area division unit 3071 divides the imaging area 113A into a plurality of areas based on luminance level thresholds (for example, thresholds t11, t12, and t13 shown in FIG. 51). According to such a configuration, the region dividing section 3071 can set the threshold value of the luminance level based on the allowable range of the imaging element 3100 and the like. Therefore, the region dividing unit 3071 can divide into regions having appropriate luminance levels according to the performance of the imaging element 3100 and the like. In addition, the area division unit 3071 can set the threshold value of the brightness level based on the actual brightness of the subject. Therefore, the area division unit 3071 can divide the area into areas having appropriate luminance levels according to the subject.

Further, in the fourteenth embodiment, even if the imaging conditions are set by the control unit 3072, if the brightness level of a predetermined area out of the plurality of areas does not fall within the specific range, the area division unit 3071 sets the imaging conditions. The predetermined area is further divided into a plurality of areas based on the threshold of the brightness level of the predetermined area later. According to such a configuration, even if the specific range of the imaging element 3100 is narrower than the luminance difference of the subject, the luminance levels of all the imaging areas 113A of the imaging element 3100 are kept within the specific range. be able to.

In addition, in the fourteenth embodiment, the control unit 3072 determines the specific range based on the saturation level of the image sensor 3100. Therefore, the range of luminance levels in which the image sensor 3100 does not saturate can be determined as the specific range. Various ranges such as a range in which the change in the amount of light from the subject and the change in the luminance level are linear can also be determined as the specific range. Further, in the fourteenth embodiment, the control unit 3072 sets at least one of gain, exposure time, and frame rate as imaging conditions, so that the control unit 3072 ensures that the luminance level of each region falls within the specific range. can be controlled as follows.

Further, in the fourteenth embodiment, the image processing unit 3030 that executes image processing according to the imaging conditions for each of a plurality of areas is provided. , and in this case, the brightness resolution of the image data is ensured.

Further, in the fourteenth embodiment, based on the imaging conditions set by the control unit 3072, the conversion unit converts the luminance level of each area into the luminance level when the area is imaged under the same or substantially the same imaging condition. 3031. With such a configuration, it is possible to restore an image that matches the actual brightness of the subject. In addition, in the fourteenth embodiment, the control unit 3072 controls the image sensor 3100 so that the image capturing timings based on the image capturing conditions for each of the plurality of areas partially overlap, so that the image processing unit 3030 can easily synthesize images. It is possible to display images with a wide dynamic range.

In addition, in the fourteenth embodiment described above, the conversion unit 3031 is provided in the image processing unit, but may be provided in the imaging device 3100 . For example, the arithmetic circuit 416 of the imaging element 3100 may convert the luminance level of each region into a luminance level when each region is imaged under the same or substantially the same imaging conditions.

In addition, in the fourteenth embodiment described above, the digital camera 3001 was given as an example of an electronic device, but it is not limited to this. good too. Also, the image processing unit 3030 and the system control unit 3070 shown in FIG. 48 may be integrated. In this case, the system control unit having one body-side CPU performs the functions of the image processing unit 3030 and the system control unit 3070 by executing processing based on the control program.

<Fifteenth embodiment>
In the fifteenth embodiment, the digital camera 3001 in the fourteenth embodiment is separated into an imaging device 3001A and an electronic device 3001B.

FIG. 60 is a block diagram showing configurations of an imaging device 3001A and an electronic device 3001B according to the fifteenth embodiment. In the configuration shown in FIG. 60, an imaging device 3001A is a device for imaging a subject. This imaging device 3001A includes a lens unit 3010, an imaging unit 3020, an image processing unit 3030, a work memory 3040, an operation unit 3055, a recording unit 3060, and a first system control unit 3070A. The configuration of the lens unit 3010, the image capturing unit 3020, the image processing unit 3030, the work memory 3040, the operation unit 3055, and the recording unit 3060 in the imaging device 3001A is the same as the configuration shown in FIG. Therefore, the same configurations are denoted by the same reference numerals, and overlapping descriptions are omitted.

The electronic device 3001B is a device that displays images (still images, moving images, live view images). This electronic device 3001B includes a display section 3050 and a second system control section 3070B. Note that the configuration of the display unit 3050 in the electronic device 3001B is the same as the configuration shown in FIG. Therefore, the same configurations are denoted by the same reference numerals, and overlapping descriptions are omitted.

The first system control section 3070A has a first communication section 3075A. Also, the second system control unit 3070B has a second communication unit 3075B. The first communication unit 3075A and the second communication unit 3075B transmit and receive signals to and from each other in a wired or wireless manner. In such a configuration, the first system control unit 3070A transfers image data (image data processed by the image processing unit 3030, image data recorded in the recording unit 3060) to the second system via the first communication unit 3075A. It is transmitted to the communication section 3075B. Second system control unit 3070B causes display unit 3050 to display the image data received by second communication unit 3075B.

The configuration shown in FIG. 48 (area dividing section 3071 and control section 3072) may be provided in either the first system control section 3070A or the second system control section 3070B. All the configuration shown in FIG. 48 may be provided in the first system control unit 3070A or the second system control unit 3070B, and part of the configuration shown in FIG. 48 may be provided in the first system control unit 3070A, A configuration other than a part of the configuration shown in 48 may be provided in the second system control section 3070B.

Note that the imaging device 3001A is configured by, for example, a digital camera, a smartphone, a mobile phone, or a personal computer having an imaging function and a communication function, and the electronic device 3001B is, for example, a smartphone, a mobile phone, or a portable personal It consists of portable terminals such as computers.

In the first system control unit 3070A shown in FIG. 60, the processing executed by the CPU based on the control program corresponds to the area dividing unit 3071 and the control unit 3072. FIG. In addition, in the second system control unit 3070B shown in FIG. 60, the processing executed by the CPU based on the control program corresponds to the area dividing unit 3071 and the control unit 3072. FIG.

As described above, in the fifteenth embodiment, in addition to the effects described in the fourteenth embodiment, an image captured by the imaging device 3001A using a mobile terminal such as a smartphone is displayed on the display unit 3050 of the electronic device 3001B. can be displayed.

In the configuration shown in FIG. 60, image processing section 3030 and first system control section 3070A may be integrated. In this case, a system control unit having one body-side CPU performs processing based on a control program to perform the function of the image processing unit 3030 and the function of the first system control unit 3070A.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various modifications and improvements can be made to the above embodiments without departing from the scope of the present invention. Also, one or more of the requirements described in the above embodiments may be omitted. Such modifications, improvements, and omissions are also included in the technical scope of the present invention. Moreover, it is also possible to appropriately combine the configurations of the above-described embodiments and modifications and apply them.

For example, although the arrangement of the color filters 102 is the Bayer arrangement in each of the above-described embodiments, an arrangement other than this arrangement may be used. Also, the number of pixels forming the unit group 131 should include at least one pixel. Also, it is sufficient that each block includes at least one pixel. Therefore, it is possible to perform imaging under different imaging conditions for each pixel.

Further, in each of the above-described embodiments, the drive unit 3021 may be partially or wholly mounted on the imaging chip 113 , or partially or wholly mounted on the signal processing chip 111 . Also, part of the configuration of the image processing unit 3030 may be mounted on the imaging chip 113 or the signal processing chip 111 . Also, part of the configuration of the system control unit 3070 may be mounted on the imaging chip 113 or the signal processing chip 111 .

In the fourteenth embodiment described above, the processing of steps S3004 to S3007 shown in FIG. 49 was executed based on the user's half-pressing of the release switch. may That is, the system control unit 3070 may divide the live view image into regions and set the imaging conditions for each region based on the luminance distribution data. Also, the system control unit 3070 may execute the processing of steps S3004 to S3007 based on the full-press operation of the release switch. In other words, the processing of steps S3004 to S3007 may be executed immediately before performing the imaging processing of step S3009.

In the fourteenth embodiment described above, the image processing unit 3030 generates luminance distribution data by calculating luminance values based on the RGB tone values of each pixel in the RAW data. The luminance distribution data may be generated by calculating the luminance value based on any of the RGB gradation values of each pixel.

In the fourteenth embodiment described above, the system control unit 3070 (region dividing unit 3071 and control unit 3072) divides regions and sets imaging conditions ( The processing of steps S3005 to S3007) was executed. That is, the system control unit 3070 executes area division and setting of imaging conditions in all boundary areas between adjacent areas. However, the present invention is not limited to such a configuration. Region division and imaging condition setting are executed in a certain boundary region between adjacent regions, and in another boundary region between adjacent regions, a certain portion of The area division and imaging condition setting may be performed in the same manner as the area division and imaging condition setting performed in the boundary area. With such a configuration, the processing load on the system control unit 3070 can be reduced.

Further, in the fourteenth embodiment described above, the specific range is the range of 35% to 80% of the saturation level, but it is not limited to such a range. It may be wider or narrower than the range of 35% to 80% of the saturation level. For example, the specified range may be the allowable range (range of 0% to 100% of the saturation level). However, the specific range may be a range in which the brightness and luminance level of the subject, which is less susceptible to noise, have a linear relationship. Further, although the luminance level threshold values are set to 35%, 80%, and 100% of the saturation level, these values are also examples and are not limited to such values. Also, although the threshold value of the brightness level is matched with the lower limit value and the upper limit value of the specific range, the threshold value of the brightness level does not have to match the lower limit value and the upper limit value of the specific range.

In the fourteenth embodiment described above, the same specific range (35% to 80% of the saturation level) is used when the system control unit 3070 lowers the high luminance level so that it falls within the specific range and when it increases the low luminance level. range). However, the specific range may be different between when the system control unit 3070 lowers the high luminance level so as to enter the specific range and when the low luminance level is increased.

In the fourteenth embodiment described above, at least one of the charge accumulation time, frame rate, and gain may be changed as the imaging condition set by the control unit 3072 .

<Sixteenth Embodiment>
FIG. 61 is a block diagram showing the configuration of a digital camera 4001 according to the sixteenth embodiment. 61, a digital camera 4001 as an example of electronic equipment corresponds to the digital camera 1 in FIG. 7, a lens unit 4010 corresponds to the lens unit 10 in FIG. 7, and an imaging unit 4020 corresponds to the imaging unit 20 in FIG. , the image pickup device 4100 corresponds to the image pickup device 100 in FIG. 7, the driving unit 4021 corresponds to the driving unit 21 in FIG. 7, the image processing unit 4030 corresponds to the image processing unit 30 in FIG. 7, the display unit 4050 corresponds to the display unit 50 in FIG. 7, the display panel 4051 corresponds to the display panel 51 in FIG. 7, and the touch panel 4052 corresponds to the touch panel 52 in FIG. 7, the recording unit 4060 corresponds to the storage unit 60 in FIG. 7, and the system control unit 4070 corresponds to the system control unit (control unit) 70 in FIG. Of these configurations, the lens unit 4010, imaging unit 4020, imaging element 4100, driving unit 4021, image processing unit 4030, work memory 4040, display unit 4050, operation unit 4055, and recording unit 4060 will not be described. .

A system control unit (control unit) 4070 controls the overall processing and operation of the digital camera 4001 . The system control unit 4070 has a body-side CPU (Central Processing Unit). In this embodiment, the system control unit 4070 divides the imaging surface (pixel region 113A) of the imaging device 100 (imaging chip 113) into a plurality of regions (for example, the first region 4200, the second region 4210, and the third region 4220 in FIG. 62). ), and acquire images with different shutter speeds (charge accumulation time or number of charge accumulations), frame rates, and gains between regions. Therefore, the system control unit 4070 instructs the driving unit 21 about the position, shape, range of each area, and imaging conditions for each area.

In addition, the system control unit 4070 obtains an image with a thinning rate, the number of addition rows or addition columns for adding pixel signals, and the number of bits for digitization, which are different for each region. Therefore, the system control unit 4070 instructs the driving unit 4021 about imaging conditions for each region (thinning rate, number of rows or columns to be added for adding pixel signals, and number of bits for digitization). In addition, the image processing unit 4030 executes image processing under imaging conditions (control parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate) that differ for each region. Therefore, the system control unit 4070 instructs the image processing unit 4030 on imaging conditions for each area (control parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate).

Also, the system control unit 4070 causes the recording unit 4060 to record the image data generated by the image processing unit 4030 . Further, the system control unit 4070 causes the display unit 4050 to display an image by outputting the image data generated by the image processing unit 4030 to the display unit 4050 . Further, the system control unit 4070 reads the image data recorded in the recording unit 4060 and causes the display unit 4050 to output the image data, thereby causing the display unit 4050 to display an image. Images displayed on the display unit 4050 include still images, moving images, and live view images.

In addition, the system control unit 4070 executes exposure adjustment processing (AE processing) during shooting of live view images. Also, the system control unit 4070 executes AE processing even during shooting of moving images. Furthermore, AE processing is executed based on the fact that the release switch is half-pressed. Specifically, the system control unit 4070 acquires luminance distribution data representing the luminance distribution of the image from the image processing unit 4030 . At this time, the system control 4070 controls the luminance distribution data of the image corresponding to the first area (first area 4200 in FIG. 62), the luminance distribution data of the image corresponding to the second area (second area 4210 in FIG. 6), and the luminance distribution data of the image corresponding to the third area (the third area 4220 in FIG. 62). Then, the system control unit 4070 performs photometry using these luminance distribution data. Also, the system control unit 4070 determines imaging conditions (shutter speed (exposure time, charge accumulation time), frame rate, and gain (ISO sensitivity)) and aperture value based on the results of photometry. At this time, the system control unit 4070 can determine imaging conditions that provide proper exposure for each region in the imaging region 113A of the imaging device 4100. FIG. Note that the aperture value cannot be set for each area, and is set for the entire imaging area 113A.

The system control unit 4070 causes drive control for adjustment of the aperture diameter of the diaphragm 14 to be executed by transmitting control information indicating an aperture value corresponding to proper exposure to the lens drive control device 15 . The system control unit 4070 also outputs an instruction signal to the driving unit 21 to instruct the shutter speed (charge accumulation time), frame rate, and gain (ISO sensitivity) according to proper exposure. The driving unit 21 drives and controls the imaging element 4100 at the shutter speed, frame rate, and gain indicated by the instruction signal.

Note that the system control unit 4070 corresponds to processing and control executed by the body-side CPU based on the control program.

FIG. 62 is a diagram showing a first area 4200, a second area 4210, and a third area 4220 in an imaging device 4100 of the sixteenth embodiment. As shown in FIG. 62, each of the first region 4200, the second region 4210, and the third region 4220 is formed by using a rectangular region consisting of four blocks (two blocks each in length and width) as one unit region. , a plurality of unit areas are evenly arranged. Also, the first region 4200, the second region 4210 and the third region 4220 are arranged so that the ratio of their areas is 4:3:1. However, each of the first region 4200, the second region 4210, and the third region 4220 may be formed by forming a region consisting of less than four blocks as one unit region, and by forming a region consisting of five or more blocks as one unit region. It may be formed as one unit area. Also, the first region 4200, the second region 4210, and the third region 4220 may be arranged over the entire surface of the pixel region 113A, and may not be arranged uniformly in the pixel region 113A. Also, the ratio of the areas of the first region 4200, the second region 4210 and the third region 4220 does not have to be 4:3:1, and for example, they may have the same area.

The system control unit 4070 sets a normal frame rate (for example, 60 fps) and a normal shutter speed (exposure time) when shooting live view images and moving images as imaging conditions for the first area 4200, and also sets the normal shutter speed (exposure time) for the live view images. or set the normal gain (ISO sensitivity) for shooting movies. In addition, the system control unit 4070 sets the normal frame rate and normal shutter speed as imaging conditions for the second region 4210, and sets a gain lower than the normal gain set for the first region 4200. do. In addition, the system control unit 4070 sets the above-described normal frame rate and fast shutter speed (that is, short exposure time) as the imaging conditions for the third region 4220, and the normal gain set for the first region 4200. Set the gain too low.

FIG. 63 is a diagram for explaining AE processing in the sixteenth embodiment. In FIGS. 63A and 63B, the horizontal axis indicates the frame of the live view image or moving image, and the vertical axis indicates the brightness level of a predetermined area (here, the area of the high-brightness subject) in the frame of the live view image or moving image. is shown. FIG. 63A shows AE processing in the conventional example, and FIG. 63B shows AE processing in this embodiment.

As shown in FIG. 63A, when the system control unit 4070 captures a high-brightness subject such as the sun or lighting with a normal shutter speed and a normal gain, the brightness level is saturated in the area of the high-brightness subject. level (brightness level at which whiteout occurs) is exceeded. In this case, the system control unit 4070 cannot determine how high the brightness level of the region of the high-brightness subject is above the saturation level. Therefore, in the AE process, the system control unit 4070 increases the shutter speed or lowers the gain to gradually lower the luminance level of the high-brightness object area to an appropriate level (appropriate exposure level). In the example shown in FIG. 63(A), the system control unit 4070 lowers the brightness level of the high-brightness subject area to the appropriate level using five frames.

As shown in FIG. 63B, when the system control unit 4070 captures an image of a bright subject such as the sun or lighting under the imaging conditions (normal shutter speed, normal gain) of the first area 4200, ), the brightness level exceeds the saturation level in the region of the high-brightness object. Further, as shown in FIG. 63B, even when the system control unit 4070 captures an image of a high-brightness subject under the imaging conditions of the second area 4210 (normal shutter speed, low gain), in the area of the high-brightness subject The brightness level exceeds the saturation level.

On the other hand, when the system control unit 4070 captures the high-brightness subject under the imaging conditions of the third area 4220 (high shutter speed, low gain), the brightness level does not exceed the saturation level in the area of the high-brightness subject. In this case, system control section 4070 estimates the luminance level of first area 4200 from the luminance level of third area 4220 that does not exceed the saturation level in the AE process. Then, system control section 4070 changes either one or both of the shutter speed and the gain so that the estimated luminance level of first region 4200 becomes an appropriate level. As a result, as shown in FIG. 63B, the system control section 4070 can lower the brightness level of the third area 4220 in the high-brightness subject area to the appropriate level in one frame. Therefore, AE processing can be performed from overexposure to proper exposure in a short period of time. In the example shown in FIG. 63B, system control section 4070 also changes the luminance levels of second area 4210 and third area 4220 in accordance with the ratio of changing the luminance level of first area 4200 to the appropriate level. However, the luminance levels of the second area 4210 and the third area 4220 may also be changed to appropriate levels according to the subject.

In the sixteenth embodiment described above, the brightness level of the third region 4220 in the high-brightness subject region is prevented from exceeding the saturation level by varying the shutter speed and gain of each of the regions 4200 to 4220. However, by varying one or more of the frame rate, shutter speed, and gain of each of the areas 4200 to 4220, the brightness level of any of the areas 4200 to 4220 in the high-brightness subject area exceeds the saturation level. may be omitted.

<Seventeenth Embodiment>
FIG. 64 is a diagram showing a saturated state of the first region 4200 in the seventeenth embodiment. In the seventeenth embodiment, similarly to the case described in the sixteenth embodiment, the system control unit 4070 sets the imaging condition of the first area 4200 to the normal frame rate (for example, 60 fps) and a normal shutter speed (exposure time), and a normal gain (ISO sensitivity) for capturing live view images and moving images. In addition, the system control unit 4070 sets the normal frame rate and normal shutter speed as imaging conditions for the second region 4210, and sets a gain lower than the normal gain set for the first region 4200. do. In addition, the system control unit 4070 sets the above-described normal frame rate and fast shutter speed (that is, short exposure time) as the imaging conditions for the third region 4220, and the normal gain set for the first region 4200. set a lower gain.

In the example shown in FIG. 64, the luminance level exceeds the saturation level (the luminance level at which overexposed highlights occur) in the first area 4200 (white-filled area in FIG. 64) in the high-brightness subject area 4300 . However, in the second area 4210 and the third area 4220 in the high-brightness subject area 4300, the brightness level does not exceed the saturation level. In other words, the entire image of the high-brightness subject area 4300 is not overexposed. In this manner, the system control unit 4070 captures images with different imaging conditions for the areas 4200 to 4220, thereby preventing the entire image of the high-brightness subject area 4300 from being overexposed.

<Eighteenth Embodiment>
FIG. 65 is a diagram for explaining moving image HDR in the eighteenth embodiment. In the eighteenth embodiment, as in the case of the sixteenth embodiment, the system control unit 4070 sets the imaging condition of the first area 4200 to the normal frame rate (for example, 60 fps) and a normal shutter speed (exposure time), and a normal gain (ISO sensitivity) for capturing live view images and moving images. In addition, the system control unit 4070 sets the normal frame rate and normal shutter speed as imaging conditions for the second region 4210, and sets a gain lower than the normal gain set for the first region 4200. do. In addition, the system control unit 4070 sets the above-described normal frame rate and fast shutter speed (that is, short exposure time) as the imaging conditions for the third region 4220, and the normal gain set for the first region 4200. set a lower gain.

In the example shown in FIG. 65, in the first area 4200 (the area marked with * in FIG. 65) in the high-brightness subject area 4300, the brightness level exceeds the saturation level (brightness level at which whiteout occurs). there is On the other hand, in the second area 4210 and the third area 4220 in the high-luminance subject area 4300, the luminance level does not exceed the saturation level. In this case, the system control unit 4070 lowers the brightness level of the first area 4200 by lowering the gain of the first area 4200 in the high-brightness subject area 4300 or increasing the shutter speed so as not to exceed the saturation level. to control. Then, system control section 4070 synthesizes the image (moving image) of first area 4200 , the image (moving image) of second area 4210 and the image (moving image) of third area 4220 . With such a configuration, it is possible to obtain a moving image without blown-out highlights even when shooting a high-brightness subject.

HDR (High Dynamic Range) imaging is generally known as an image synthesis technique for recording and displaying images with a wide dynamic range. In HDR imaging, a plurality of images are captured while changing imaging conditions (for example, exposure), and an image with little blown-out highlights or blocked-up shadows is generated by synthesizing the images. However, in the conventional HDR, for example, since the imaging times for capturing two images with different imaging conditions are different, the subject may move or the user (photographer) may move the digital camera 4001. There is In this case, since the two images are not images of the same subject, it is difficult to synthesize the images. On the other hand, in the eighteenth embodiment, the imaging time of a plurality of images with different imaging conditions can be the same time (or substantially the same time). Therefore, the configuration of the eighteenth embodiment can solve the problems of conventional HDR imaging.

The image processing unit 4030 detects a high-brightness subject area from a plurality of image data obtained in time series. Based on the area of the high-brightness subject detected by the image processing unit 4030, the system control unit 4070 determines the gain and shutter speed in the area of the high-brightness subject and its surrounding area. With such a configuration, it is possible to acquire a moving image with a wide dynamic range. Note that the configuration of the eighteenth embodiment described above can be applied not only to shooting moving images but also to shooting still images.

<Nineteenth Embodiment>
In the above-described first embodiment, the division unit 71 divides the area of the image sensor 100 into the first area 200 and the second area 210, and the blink detection unit 74 detects the signal read from the second area 210 of the image sensor 100. The drive control unit 72 changes the imaging start timing based on the blinking of the light source L detected by the blinking detection unit 74 . On the other hand, in the nineteenth embodiment, the system control unit 4070 divides the area of the imaging device 4100 into a first area 4200, a second area 4210 and a third area 4220 as shown in FIG. Then, the drive control unit 72 sets the shutter speed (exposure time) of the first region 4200 to an integral multiple (eg, 1 time) of the blinking cycle (1/100 second) of the 50 Hz light source, and sets the shutter speed of the second region 4210. The speed (exposure time) is set to an integer multiple (for example, 1 time) of the blinking cycle (1/120 second) of the light source of 60 Hz, and the shutter speed (exposure time) of the third area 4220 is set to a high shutter speed (short exposure time ). Then, the system control unit 4070 drives and controls the imaging device 4100 so as to capture a moving image under the imaging conditions set in each of the areas 4200-4220.

According to such a configuration, under natural light or a non-blinking light source, no flicker phenomenon occurs in any of the areas 4200 to 4220, and flicker-free moving images can be captured in each of the areas 4200 to 4220. can. In addition, under the light source of 50 Hz, the shutter speed of the first area 4200 is an integer multiple of the flickering cycle (1/100 second) of the light source of 50 Hz, so it is possible to shoot a moving image without flickering in the first area 4200. can be done. In addition, under the light source of 60 Hz, the shutter speed of the second region 4210 is an integral multiple of the blinking cycle (1/120 second) of the light source of 60 Hz, so that moving images without flickering can be captured in the second region 4210. can be done. Further, under a light source with a frequency other than 50 Hz or 60 Hz, the system control unit 4070 detects the blinking period of the light source based on the signal read out from the second area 210, and either the first area 4200 or the second area 4210 One or both shutter speeds are set to integer multiples of the blinking period of the sensed light source. As a result, a flicker-free moving image can be captured in one or both of the first area 4200 and the second area 4210 . Further, as in the first embodiment described above, by changing the timing of starting to capture a still image based on the blinking period of the light source detected by the system control unit 4070, the still image can be captured without being affected by the flicker phenomenon. You can shoot.

Since the shutter speed of the second area 4210 is set to 1/120 second, the exposure amount is smaller than that of the first area 4200 whose shutter speed is 1/100 second. Therefore, system control section 4070 sets the gain of second region 4210 to 1.2 times the gain of first region 4200 . Thereby, the exposure amounts of the first region 4200 and the second region 4210 are made uniform. In addition, since the shutter speed of the third area 4220 is set to a high shutter speed, the exposure amount is smaller than that of the first area 4200 where the shutter speed is 1/100 second. Therefore, system control section 4070 sets the gain of third region 4220 higher than the gain of first region 4200 . As a result, the exposure amounts of the regions 4200 to 4220 are made uniform.

<Twentieth Embodiment>
FIG. 66 is a block diagram showing the configuration of a smart phone 5001 according to the twentieth embodiment. Note that in FIG. 66, a smart phone 5001, which is an example of an electronic device, has a configuration corresponding to the digital camera 1 in FIG. A lens unit 5010 corresponds to the lens unit 10 in FIG. 7, an imaging unit 5020 corresponds to the imaging unit 20 in FIG. 7, an imaging device 5100 corresponds to the imaging device 100 in FIG. , the image processing unit 5030 corresponds to the image processing unit 30 in FIG. 7, the work memory 5040 corresponds to the work memory 40 in FIG. 7, and the display unit 5050 corresponds to the display unit 50 in FIG. A display panel 5051 corresponds to the display panel 51 in FIG. 7, a touch panel 5052 corresponds to the touch panel 52 in FIG. 7, an operation unit 5055 corresponds to the operation unit 55 in FIG. 60, and the system control unit 5070 corresponds to the system control unit 70 in FIG. Of these configurations, the lens unit 5010, imaging unit 5020, imaging element 5100, driving unit 5021, image processing unit 5030, work memory 5040, display unit 5050, operation unit 5055, and recording unit 5060 will not be described. . Note that the smartphone 5001 is used as an example of the electronic device, but a tablet terminal, a digital camera with a liquid crystal panel, or the like may also be used.

As shown in FIG. 66, the system controller 5070 has an operation detector 5071 and a drive controller 5072 . The operation detection unit 5071 detects at least one operation of enlarging, reducing, and moving an image. The drive control unit 5072 outputs an instruction signal to the imaging unit 5020 to cause the imaging unit 5020 to acquire an image (still image, moving image, live view image). In addition, the drive control section 5072 selects a region to drive and control the imaging element 5100 according to the operation detected by the operation detection section 5071 . Further, the drive control unit 5072 controls the area in the imaging area (pixel area) 113A corresponding to the image displayed on the display unit 5050 to the active state, and puts the area other than the area into the sleep state (inactive state). to control.

Here, "active state" means that each pixel in the area is in an operable state (imaging possible state). In this active state region, each pixel is driven and controlled by the driving unit 5021, and a pixel signal is read out from each pixel. Also, the "sleep state" means that each pixel in the area is in a state of stopping operation (imaging stop state). The operation stop state includes a state in which drive control by the driving unit 5021 is not performed in each pixel in the region. In other words, it includes a state in which the drive unit 5021 does not apply pulses to the reset transistor 303 , the transfer transistor 302 , and the selection transistor 305 . In addition, the operation stop state includes a state in which pixel signals are not read from each pixel in the region. That is, it includes a state in which the driving unit 5021 does not apply a selection pulse to the selection transistor 305 . Further, the operation stop state includes a state in which pixel signals are read from each pixel in the region but the read pixel signals are not used. For example, it includes a state in which the signal processing chip 111 or the like discards the pixel signal read from each pixel in the area.

Note that the operation detection unit 5071 and the drive control unit 5072 correspond to processing and control executed by the body-side CPU based on the control program.

The operation of system control section 5070 will be described with reference to FIGS. 67 to 70. FIG. FIG. 67 is a diagram showing an imaging region 113A in the imaging element 5100 of the twentieth embodiment. 68 to 70 are diagrams showing display examples of the display unit 5050 of the smart phone 5001. FIG.

The drive control unit 5072 outputs an instruction signal to the imaging unit 5020 to cause the imaging unit 5020 to acquire an image (here, a moving image or a live view image) as shown in FIG. 67, for example. In the example shown in FIG. 67, the image acquired by the imaging unit 5020 includes a person O310 and a house O320 as subjects.

A region 5201 within the imaging region 113A shown in FIG. 67 is a region corresponding to the image displayed on the display panel 5051 of the display unit 5050 (see FIG. 68). At this time, the drive control unit 5072 controls the area 5201 in the imaging area 113A to the active state, and controls the areas other than the area 5201 in the imaging area 113A to the sleep state.

Next, as shown in FIG. 68, the user performs an operation of pinching the display panel 5051 with the fingers and then moving the fingers away (pinch-out operation). Touch panel 5052 outputs a detection signal corresponding to the user's pinch-out operation to system control unit 5070 . Based on a detection signal from the touch panel 5052, the operation detection unit 5071 detects a user's pinch-out operation for enlarging an image. The drive control unit 5072 controls to enlarge and display an image according to the pinch-out operation detected by the operation detection unit 5071 (see FIG. 69). Further, the drive control unit 5072 controls the area 5202 within the imaging area 113A corresponding to the image enlarged and displayed on the display panel 5051 to the active state, and controls the area other than the area 5202 within the imaging area 113A to the sleep state. .

Also, the user performs a pinch-out operation on the display panel 5051, as shown in FIG. Touch panel 5052 outputs a detection signal corresponding to the user's pinch-out operation to system control unit 5070 . Based on a detection signal from the touch panel 5052, the operation detection unit 5071 detects a user's pinch-out operation for enlarging an image. The drive control unit 5072 controls to enlarge and display an image according to the pinch-out operation detected by the operation detection unit 5071 (see FIG. 70). Further, the drive control unit 5072 controls the area 5203 in the imaging area 113A corresponding to the image enlarged and displayed on the display panel 5051 to the active state, and controls the areas other than the area 5203 in the imaging area 113A to the sleep state. .

With such a configuration, the drive control unit 5072 controls the region other than the region corresponding to the image displayed on the display panel 5051 to the sleep state. can be reduced. In the above configuration, the user's pinch-out operation has been described, but it is also applicable to the user's pinch-in operation (an operation of pinching the display panel 5051 with the fingers and then bringing the fingers closer together) or a scrolling operation. It is possible. That is, the drive control unit 5072 controls the area within the imaging area 113A corresponding to the image reduced and displayed on the display panel 5051 in accordance with the user's pinch-in operation to the active state, and places the area other than that area in the sleep state. Control. Further, the drive control unit 5072 controls the area within the imaging area 113A corresponding to the image moved and displayed on the display panel 5051 according to the user's scrolling operation to the active state, and puts the other areas to the sleep state. Control.

In the above configuration, the drive control unit 5072 controls the area of the image displayed on the display unit 5050 to the active state, and controls other areas to the sleep state. However, the drive control unit 5072 may control an area including the image area displayed on the display unit 5050 to the active state, and control other areas to the sleep state. With such a configuration, it is possible to prevent the sleep state area from being displayed on the display unit 5050 even when camera shake occurs while the smartphone 5001 is shooting an image (moving image or live view image). . Further, since the drive control unit 5072 controls the area including the area of the image displayed on the display unit 5050 to the active state, it is possible to correct the camera shake when camera shake occurs.

Further, the drive control section 5072 may change the thinning rate of the area for driving and controlling the imaging element 5100 according to the operation detected by the operation detection section 5071 . Specifically, it is assumed that the number of pixels in the region 5201 of the image sensor 5100 is 2560 pixels in the vertical direction×1920 pixels in the horizontal direction. It is also assumed that the number of recording pixels of the recording unit 5060 (the number of pixels of an image (still image or moving image) recorded in the recording unit 5060) is set in advance to 640 vertical pixels×480 horizontal pixels. When capturing an image in the area 5201, the drive control unit 5072 reads signals at a thinning rate of 3/4 both vertically and horizontally. That is, the drive control unit 5072 reads out signals from ¼ of all the pixels in the region 5201 . Further, when imaging in the area 5202 (this area 5202 is assumed to be half the size of the area 5201 vertically and horizontally), the driving control unit 5072 sets the vertical and horizontal thinning rate to 1/2. Read out the signal. That is, the drive control unit 5072 reads out signals from half of all the pixels in the region 5202 . In addition, the drive control unit 5072, when imaging in the area 5203 (this area 5203 is assumed to be 1/4 of the size of the area 5201 vertically and horizontally), reduces the signal with a thinning rate of 0 both vertically and horizontally. Do a read. That is, the drive control unit 5072 reads signals from all pixels in the region 5202 .

According to such a configuration, signals are read out only from the pixels in the area corresponding to the preset number of recording pixels, so that the power consumption of the driving section 5021 that drives the imaging device 5100 is further reduced. be able to.

Further, when the number of pixels recorded in the recording unit 5060 exceeds the number of pixels in the area in which the imaging device 5100 is driven and controlled, the image processing unit 5030 may perform interpolation processing of image data. For example, when capturing an image in a region smaller than the region 5203 (half the size of the region 5203), the drive control unit 5072 reads signals from all pixels in the region. Then, the image processing unit 5030 generates image data between pixels in the region based on the signals (image data) read out from all pixels in the region, and records it in the recording unit 5060 . With such a configuration, there is no need to limit the size of the region in which the imaging element 5100 is driven and controlled.

<21st embodiment>
FIG. 71 is a diagram showing an imaging region 113A in the imaging element of the twenty-first embodiment. As shown in FIG. 71, the imaging region 113A is divided into 16 rectangular unit regions A1 to A16. Each of the unit areas A1 to A16 is an area composed of a plurality of blocks. Note that each unit area may be composed of one or several blocks. In this case, the number of unit areas is greater than the number shown in FIG.

FIG. 72 is a block diagram showing the configuration of the imaging unit 6020, image processing unit 6030 and recording unit 6060 of the twenty-first embodiment. 72, an imaging unit 6020 corresponds to the imaging unit 20 in FIG. 7, an image processing unit 6030 corresponds to the image processing unit 30 in FIG. 7, and a recording unit 6060 corresponds to the recording unit 60 in FIG.

The imaging unit 6020 can capture an image at a different frame rate for each area in the imaging area 113A. The imaging unit 6020 includes an imaging area 113A of the imaging device, an imaging device control unit 6021, a buffer 6022, and a selector (selection unit) 6023. In FIG. 72, the unit areas A1 to A16 in the image pickup area 113A of the image sensor are arranged horizontally in a row, but actually they are arranged vertically and horizontally as shown in FIG. As shown in FIG. 72, the image data obtained from each of the unit areas A1 to A16 are output to the selector 6023.

The image pickup device control unit 6021 sets the frame rate of each region in the image pickup region 113A based on the control signal from the readout control unit 6031 of the image processing unit 6030, and outputs a selection signal to the selector 6023, so that the unit The buffers B1 to B16 that store the image data obtained in the areas A1 to A16 are switched to buffers corresponding to the frame rate of each area. A buffer 6022 (a plurality of buffers B1 to B16) is a storage area for temporarily storing image data to be transmitted (transferred) to the image processing unit 6030. FIG. Buffers 6023 (a plurality of buffers B1 to B16) are provided corresponding to a plurality of unit areas A1 to A16 of the imaging area 113A. The buffers 6023 (a plurality of buffers B1 to B16) are connected to the interface section 6032 of the image processing section 6030 via a plurality of transmission lines (lanes) L1 to L16, respectively. In this embodiment, the image data transfer capability (the amount of image data that can be transferred per unit time) of each of the plurality of transmission lines L1 to L16 is determined by the frame rate. In this embodiment, the image data transfer capacity of each of the transmission lines L1 to L16 is, for example, 60 fps. The selector 6023 switches buffers B1 to B16 for storing the image data obtained from the unit areas A1 to A16 by switching based on the selection signal from the image sensor control unit 6021 .

The image processing unit 6030 receives the image data transmitted from the imaging unit 6020 and performs image processing on the received image data. The image processing unit 6030 has a read control unit 6031 , an interface unit 6032 and a memory controller 6033 . Based on the detection signal from the sensor 6050, the readout control unit 6031 determines which area of the imaging area 113A should be set to a high frame rate (a frame rate higher than the frame rate when shooting moving images or live view images). Area/frame rate determination processing to be determined is executed. Then, the reading control unit 6031 outputs a control signal instructing the frame rate of each region in the imaging region 113A to the imaging element control unit 6021 based on the result of the area/frame rate determination processing. Further, the read control unit 6031 rearranges the image data for each of the unit areas A1 to A16 stored in the interface unit 6032 based on the result of the area/frame rate determination processing.

The interface unit 6032 is connected to a plurality of buffers B1 to B16 via a plurality of transmission lines L1 to L16, receives image data transmitted from each buffer B1 to B16, and stores the received image data in a storage area. The memory controller 6033 reads the image data stored in the storage area of the interface section 6032 and outputs it to the recording section 6060 . Sensor 6050 is an acceleration sensor that detects horizontal acceleration or angular acceleration. A recording unit 6060 is a memory that records image data output from the image processing unit 6030 .

FIG. 73 is a diagram showing transfer timing of image data from the imaging unit 6020 to the image processing unit 6030. As shown in FIG. It is assumed that the frame rates of the unit areas A1 to A16 are all the same frame rate and not a high frame rate. As shown in FIG. 73, the image data D1-1 to D16-1 obtained from the unit areas A1 to A16 in the first frame are respectively stored in the buffers B1 to B16 via the selector 6023 at the same timing. be. The image data D1-1 to D16-1 stored in the buffers B1 to B16 are transferred to the image processing section 6030 via the transmission lines L1 to L16 at the same timing.

The image data D1-2 to D16-2 obtained from the unit areas A1 to A16 in the second frame are also stored in the buffers B1 to B16 via the selector 6023 at the same timing. The image data D1-2 to D16-2 stored in the buffers B1 to B16 are transferred to the image processing section 6030 via the transmission lines L1 to L16 at the same timing. The image data D1-3 to D16-3 obtained from the unit areas A1 to A16 in the third frame are also stored in the buffers B1 to B16 via the selector 6023 at the same timing. The image data D1-3 to D16-3 stored in the buffers B1 to B16 are transferred to the image processing section 6030 via the transmission lines L1 to L16 at the same timing.

As shown in FIG. 73, when the image data obtained in each of the unit areas A1 to A16 of one frame are transferred from the imaging unit 6020 to the image processing unit 6030 at the same timing, the reading control unit 6031 controls the interface unit 6032 is not rearranged for each of the unit areas A1 to A16 stored in the storage area of .

Next, image data is read at a high frame rate from the left half areas (unit areas A1, A2, A5, A6, A9, A10, A13, A14) in the imaging area 113A, and the right half area (unit areas) in the imaging area 113A is read out. A case where image data is not read out from the areas A3, A4, A7, A8, A11, A12, A15 and A16 will be described.

In FIG. 74, image data is read from the left half area (unit areas A1, A2, A5, A6, A9, A10, A13, A14) at a high frame rate, and the right half area (unit areas A3, A4, A7, A8, A11, A12, A15, A16) is a block diagram showing the configuration of the imaging unit 6020, the image processing unit 6030, and the recording unit 6060 when image data is not read out. Also, in FIG. 75, image data is read at a high frame rate from the left half area (unit areas A1, A2, A5, A6, A9, A10, A13, A14), and the right half area (unit areas A3, A4, A14) is read out. 60 is a diagram showing the transfer timing of image data from the imaging unit 6020 to the image processing unit 6030 when image data is not read from A7, A8, A11, A12, A15, A16).

In this case, the readout control unit 6031 outputs a control signal to the imaging device control unit 6021 to set the frame rate of the left half area to the high frame rate and set the frame rate of the right half area to zero. Based on the control signal from the reading control unit 6031, the imaging device control unit 6021 sets the frame rate of the left half area to a high frame rate and sets the frame rate of the right half area to 0. In addition, the image sensor control unit 6021 outputs a selection signal to the selector 6023 to switch the buffers B1 to B16 storing the image data obtained in the unit areas A1 to A16 to buffers corresponding to the frame rate of each area. Let

Specifically, as shown in FIGS. 74 and 75, image data D1-1 and D2-1 obtained from unit areas A1, A2, A5, A6, A9, A10, A13, and A14 in the first frame , D5-1, D6-1, D9-1, D10-1, D13-1, and D14-1 are supplied to buffers B1, B2, B5, B6, B9, B10, It is stored in B13 and B14. Image data D1-1, D2-1, D5-1, D6-1, D9-1, D10-1, D13-1, D10-1, D13-1, D14-1 is transferred to image processing section 6030 via transmission lines L1, L2, L5, L6, L9, L10, L13, and L14 at the same timing.

Image data D1-2, D2-2, D5-2, D6-2, D9-2, D10- obtained in unit areas A1, A2, A5, A6, A9, A10, A13, A14 in the second frame 2, D13-2 and D14-2 are stored in buffers B3, B4, B7, B8, B11, B12, B15 and B16 at the same timing via selector 6023, respectively. Image data D1-2, D2-2, D5-2, D6-2, D9-2, D10-2, D13-2, D10-2, D13-2, D1-2, D2-2, D5-2, D6-2, D9-2, D14-2 is transferred to image processing section 6030 via transmission lines L3, L4, L7, L8, L11, L12, L15 and L16 at the same timing.

Image data D1-2, D2-2, D5-2, D6-2, D9-2, D10-2, D13-2, and D14-2 of the second frame are image data D1-1 of the first frame. , D2-1, D5-1, D6-1, D9-1, D10-1, D13-1, and D14-1 are not yet completed. Therefore, even if the frame rate of the left half area is high, the buffers and transmission paths corresponding to each unit area in the left half area and the buffers and transmission paths corresponding to each unit area in the right half area By alternately using the transmission lines, image data can be transferred at a frame rate exceeding the transfer capacity of each transmission line.

In the example described above, the imaging unit 6020 can read image data at a high frame rate from the left half areas (unit areas A1, A2, A5, A6, A9, A10, A13, and A14). Image data cannot be read from the right half areas (unit areas A3, A4, A7, A8, A11, A12, A15, and A16). Therefore, the area from which image data is read at a high frame rate (hereinafter referred to as the first area) is set to 1/x (x=3, 4, 5, . . . ) of the entire imaging area 113A. Then, every time x frames of image data are read from the first area, the imaging unit 6020 reads out one frame of image data from an area other than the first area in the imaging area 113A (hereinafter, this area is referred to as a second area). read out. As a result, the imaging unit 6020 can read out the image of the entire imaging region 113A, and transfer image data without exceeding the transfer capacity of each of the transmission lines L1 to L16.

Next, the operation of the imaging device of this embodiment will be described. In the following description, it is assumed that the user (photographer) is shooting a moving image in front of the vehicle using the imaging device. FIG. 76 is a flow chart showing processing in the image processing unit 6030 of the twenty-first embodiment. First, based on the detection signal from the sensor 6050, the readout control unit 6031 executes area/frame rate determination processing to determine which region of the imaging region 113A is set to a high frame rate (step S6001). For example, when the sensor 6050 detects acceleration or angular acceleration when the automobile is turning to the left, the readout control unit 6031 controls the left area (first area) is set to a high frame rate. Also, based on the magnitude of the acceleration or angular acceleration detected by the sensor 6050, the read control unit 6031 determines the area of the region (first region) set to the high frame rate.

Next, the reading control unit 6031 outputs a control signal instructing the frame rate of each area in the imaging area 113A to the image sensor control unit 6021 based on the result of the area/frame rate determination processing (step S6002). For example, when the sensor 6050 detects the acceleration or angular acceleration when the automobile is turning to the left, the readout control unit 6031 detects the first area (the area specified by the unit area) on the left side of the imaging area 113A. ) is set to a high frame rate, and the second area (area specified by the unit area) of the imaging area 113A other than the first area is set to a low frame rate. Output.

After that, the read control unit 6031 receives the image data transferred from the imaging unit 6020 and stores it in the storage area of the interface unit 6032 (step S6003). Then, based on the result of the area/frame rate determination processing, the read control unit 6031 rearranges the image data for each of the unit areas A1 to A16 stored in the storage area of the interface unit 6032 (step S6004). For example, the read control unit 6031, based on the areas of the first region and the second region determined by the area/frame rate determination process and the frame rate of the first region and the second region, via the transmission paths L1 to L16 It identifies which unit area the image data received at each timing belongs to. Then, based on the position of the specified unit area of the image data, the read control unit 6031 rearranges the image data so that the position of the address of the image data in the storage area matches the position of the unit area in the imaging area 113A.

When the area of the image data updated in one frame is only the first area, the memory controller 6033 causes the recording unit 6060 to rewrite and record only the image data in the first area, and updates the image data in one frame. If the image data areas are the first area and the second area (that is, the entire area), the image data of the first area and the second area are rewritten and recorded in the recording unit 6060 (step S6005). Next, the reading control unit 6031 determines whether or not the moving image shooting has ended (step S6006). When the reading control unit 6031 determines that the moving image shooting has not ended (NO in step S6006), the process proceeds to step S6001. If the read control unit 6031 determines that the moving image shooting has ended (YES in step S6006), the process ends.

FIG. 77 is a flow chart showing processing in the imaging unit 6020 of the twenty-first embodiment. In the process shown in FIG. 77, the image sensor control unit 6021 sets the frame rate for each region (first region, second region) based on the control signal output from the read control unit 6031, and switches the selector 6023. (step S6011). That is, the image pickup device control unit 6021 sets the readout control unit 6031 to set the first area to a high frame rate and the second area to a low frame rate. In addition, the image sensor control unit 6021 outputs a selection signal to the selector 6023, and selects the buffers B1 to B16 for storing the image data obtained in the unit areas A1 to A16 according to the frame rate of each area. let me switch.

Next, the image sensor control unit 6021 stores the image data obtained from the unit areas A1 to A16 in the buffers B1 to B16 via the selector 6023 (step S6012), and stores the image data stored in the buffers B1 to B16. The data is transmitted to the image processing unit 6030 through the transmission lines L1 to L16 (step S6013).

In the present embodiment, in the area/frame rate determination process, the read control unit 6031 controls the first area and the second area so that the ratio of the high frame rate in the first area and the low frame rate in the second area is an integer multiple. Determine the frame rate of the two regions. This is to synchronize the readout timing (transfer timing) of image data from the first area and the readout timing (transfer timing) of image data from the second area. Further, in the above configuration, when the entire imaging region 113A is imaged at the standard frame rate, and when the first frame rate is imaged at a high frame rate and the second frame rate is imaged at a low frame rate, the power consumption is can be made the same.

Next, the relationship between the areas of the first and second regions and the frame rates of the first and second regions will be described. FIG. 78 is a diagram showing the relationship between the frame rate and areas 5201 and 5202. FIG. In the example shown in the left diagram of FIG. 78, the imaging unit 6020 reads image data from the entire imaging area 113A at a standard frame rate (here, 60 fps). The readout control unit 6031 sets the frame rate of the first region 5201 on the left side of the imaging region 113A to a high frame rate (here, 120 fps), and sets the frame rate of the second region 5202 on the right side of the imaging region 113A to a low frame rate (here, 120 fps). 30 fps), the area of the first area 5021 is ⅓ of the imaging area 113A.

Specifically, when the standard frame rate is A (fps), the high frame rate is B (fps), and the low frame rate is C (fps), {(1/C)/(1/A)- 1}/{(1/C)/(1/B)-1}, the area of the first region can be obtained. In the example on the left side of FIG. 78, {(1/30)/(1/60)−1}/{(1/30)/(1/120)−1}=1/3, the first region 5021 is calculated to be 1/3 of the area of the imaging region 113A.

In the example shown in the right diagram of FIG. 78, the imaging unit 6020 reads image data from the entire imaging area 113A at a standard frame rate (here, 60 fps). The readout control unit 6031 sets the frame rate of the first region 5201 on the left side of the imaging region 113A to a high frame rate (here, 120 fps), and sets the frame rate of the second region 5202 on the right side of the imaging region 113A to a low frame rate (here, 120 fps). 20 fps), the area of the first area 5021 is 2/5 of the imaging area 113A. That is, in the example on the right side of FIG. 78, {(1/20)/(1/60)−1}/{(1/20)/(1/120)−1}=2/5, and The area of the region 5021 is calculated to be 2/5 of the area of the imaging region 113A.

As described above, in the twenty-first embodiment, even when the frame rate of the predetermined area of the imaging area 113A is increased, the frame rate of areas other than the predetermined area is decreased, and the buffer and transmission path for transferring image data are switched. Accordingly, the image data can be reliably transferred from the imaging unit 6020 to the image processing unit 6030 .

In the twenty-first embodiment described above, the read control unit 6031 executes area/frame rate determination processing based on the detection signal from the sensor 6050 that detects acceleration or angular acceleration. The area/frame rate determination process may be executed by recognizing the traveling direction and speed of the automobile based on the direction and strength of the . In the twenty-first embodiment described above, a case in which an image is captured from an automobile using an imaging device has been described as an example, but it is also applicable to a case in which an image is captured by an imaging device attached to a member such as a bicycle or a snowboard.

<22nd Embodiment>
When shooting with an action camera, for example, when attaching a digital camera to a bicycle or a radio-controller and shooting a moving image with a sense of realism, the subject suddenly jumps into the angle of view while shooting the moving image. Sometimes it comes. When you want to shoot a subject that suddenly jumps into the angle of view like this, the shooting conditions (exposure conditions: shutter speed, ISO sensitivity, etc., also called shooting conditions) are not properly set for that subject. , the image desired by the photographer cannot be captured. Conversely, even if you can set the shooting conditions for the subject, you may not want to shoot a subject that suddenly jumps into the angle of view, so set the appropriate shooting conditions for the subject. There are some things you shouldn't do. Therefore, in the 22nd embodiment, the area of the image sensor is divided into an area photographed under predetermined photographing conditions and an area photographed under randomly changed photographing conditions, and photographing is performed in each area.

FIG. 79 is a block diagram showing the configuration of an imaging system according to the twenty-second embodiment. The imaging system shown in FIG. 79 includes an imaging device (device to be set) 7001 , a smartphone (setting device, external device) 7100 , and a bicycle (device to be installed) 7200 . A photographing device 7001 is attached to a bicycle 7300 . This imaging device 7001 includes a lens unit 7010, an imaging unit 7020, an image processing unit 7030, a work memory 7040, an operation unit 7055, a recording unit 7060, a control unit 7070, a positioning unit 7081, an orientation detection unit 7082, a measurement unit 7083, and a communication unit. 7090.

79, an imaging device 7001 corresponds to the digital camera 1 in FIG. 7, a lens unit 7010 corresponds to the lens unit 10 in FIG. 7, an imaging unit 7020 corresponds to the imaging unit 20 in FIG. 7, a drive unit 7021 corresponds to the drive unit 21 in FIG. 7, an image processing unit 7030 corresponds to the image processing unit 30 in FIG. 7, and a work memory 7040 stores the work in FIG. 7, the recording unit 7060 corresponds to the storage unit 60 in FIG. 7, and the control unit 7070 corresponds to the system control unit 70 in FIG. Of these configurations, the configuration of the lens unit 7010, imaging unit 7020, imaging device 7100, driving unit 7021, image processing unit 7030, work memory 7040, operation unit 7055, and recording unit 7060 will be omitted.

A control unit 7070 controls the overall processing and operation of the imaging device 7001 . As shown in FIG. 79 , the control section 7070 has a setting section 7071 . The setting unit 7071 divides the area of the imaging device 7100 into a first imaging area and a second imaging area, as shown in FIG. 29, for example. Further, the setting unit 7071 sets normal shooting conditions for shooting a moving image as shooting conditions for the first imaging area (hereinafter referred to as first shooting conditions), and sets shooting conditions for the second imaging area (hereinafter referred to as second shooting conditions). 2 photographing conditions) are set to photographing conditions in which the shutter speed, ISO sensitivity, etc., change irregularly.

The positioning unit 7081 is composed of a GPS (Global Positioning System) for positioning the current position. Positioning section 7081 outputs information on the measured current position to control section 7070 . The orientation detection unit 7082 is a sensor that detects rotational shake about three mutually orthogonal axes and translational shake in three axial directions. For example, the posture detection unit 7082 is composed of a 3-axis gyro sensor that detects angular acceleration around 3 axes of the sensor itself, and a 3-axis acceleration sensor that detects translational acceleration of the sensor itself in 3-axis directions. Posture detection section 7082 outputs acceleration information indicating the detected acceleration (angular acceleration, linear acceleration) to control section 7070 . The measurement unit 7083 is a sensor that measures information related to the grounded device 7300 such as the rotation speed of the gear of the bicycle 7300 . The measurement unit 7083 outputs information indicating the measured number of revolutions of the gear to the control unit 7070 . Communication unit 7090 transmits and receives information to and from smartphone 7200 and bicycle 7300 through a communication line (wireless line).

A smartphone (setting device) 7200 has a display unit 7201 , an operation unit 7202 , a control unit 7203 , and a communication unit 7204 . The display unit 7201 is composed of, for example, a liquid crystal display panel. An operation unit 7202 includes operation buttons, a touch panel formed on the display unit 7201, and the like. A control unit 7203 controls the overall processing and operation of the smartphone 7200 . A communication unit 7204 transmits and receives information to and from the imaging device 7001 and the bicycle 7300 through a communication line (wireless line).

A measurement unit 7301 , a control unit 7302 , and a communication unit 7303 are attached to a bicycle (installed device) 7300 . The measurement unit 7301 is a sensor that measures the number of revolutions of gears of the bicycle 7300 and the like. The measurement unit 7301 outputs information indicating the measured number of revolutions of the gear to the control unit 7302 . The control unit 7302 controls operations of the measurement unit 7301 and the communication unit 7303 . A communication unit 7303 transmits and receives information to and from the imaging device 7001 and the smartphone 7200 through a communication line (wireless line).

The communication units 7090, 7204, 7303 of the devices 7001, 7200, 7300 establish communication with each device according to instructions from the control units 7070, 7203, 7302 and exchange various information. The information is, for example, a live view image sent from the imaging device 7001 to the smartphone 7200, and includes imaging conditions and region determination information (first imaging region and second imaging region) set by the operation unit 7202 sent from the smartphone 7200 to the imaging device 7001 information indicating the range of

Not only the control unit 7070 of the imaging device 7001 but also the control unit 7203 of the smartphone 7200 may instruct the control unit 7070 to divide the area of the image sensor 7100 into a first imaging area and a second imaging area. It is possible. As a specific method, the control unit 7070 of the imaging device 7001 transmits the live view image to the smartphone 7200 via the communication unit 7090. A control unit 7203 of the smartphone 7200 receives the live view image transmitted from the imaging device 7001 via the communication unit 7204 and displays the live view image on the display unit 7201 . When an area is determined by the user's operation on the live view image displayed on the display unit 7201 , the control unit 7203 of the smartphone 7200 transmits area determination information to the imaging device 7001 via the communication unit 7204 . The control unit 7070 of the imaging device 7001 divides the area of the image sensor 710 based on the area determination information transmitted from the smart phone 7200 . The area determination information includes coordinates on the imaging device 7100 for specifying an area, information obtained by extracting a person by face recognition or the like, and the like.

(1) When shooting a subject that suddenly jumps into the angle of view;
A setting unit 7071 of the imaging device 7001 divides the area in the imaging element 7100 into a first imaging area that is photographed under predetermined imaging conditions and a second imaging area that is photographed under randomly changed imaging conditions. In the first imaging area, representative imaging conditions that the user (photographer) wishes to photograph are set. In the second imaging area, shooting conditions such as frame rate, shutter speed, gain, and exposure compensation are randomly changed for each frame. By randomly changing the shooting conditions in the second imaging area, the setting unit 7071 can produce an image ( video) can be taken.

(2) When shooting in the sun and shade;
The sun and shadows are subjects that suddenly pop into the angle of view. When the bicycle 7300 on which the imaging device 7001 is installed enters the sun or when the sun enters the angle of view within the imaging device 7001, whiteout occurs. Collapse is likely to occur. In addition, a photographing situation in which the non-installed device 7300 in which the photographing device 7001 is installed enters the shade and immediately exits from the shade is repeated (for example, photographing in a mountain overgrown with trees on a sunny day). ), the setting unit 7071 does not randomly change the imaging conditions of the second imaging area, but uses the imaging result of the area to be imaged at random, so that the imaging conditions are not changed too frequently. do.

The control unit 7070 also acquires information on the current position measured by the positioning unit 701 and information (for example, weather forecast and weather map) received from an external device (not shown) via the communication unit 7090 . Also, when the photographing device 7001 is attached to the bicycle 7300 , the control section 7070 acquires acceleration information detected by the posture detection section 7082 . Also, when the photographing device 7001 is attached to the bicycle 7300 , the control unit 7070 acquires information indicating the rotation speed of the gear of the bicycle 7300 detected by the measurement unit 7083 . Control unit 7070 also acquires information indicating the number of rotations of the gears of bicycle 7300 measured by measurement unit 7301 of bicycle 7300 via communication unit 7090 .

Based on the acquired information, the control unit 7070 estimates the position (orientation) and altitude (angle from the ground) of the sun, the direction of shadow formation, and the like. In addition, the control unit 7070 controls the mounting position (or/and angle) of the photographing device 7001 on the bicycle 7300, the movement distance of the bicycle 7300 based on information such as the number of revolutions, the location information of the building based on the information on the current location, the position of the sun, and so on. and altitude to estimate how many seconds it will take to enter the shade, and how many seconds it will take to move from the shade to the sun. When the bicycle 7300 on which the image capturing device is installed enters the shade and the control unit 7070 tries to change the image capturing conditions abruptly, the load on the driving unit 7021 in the image capturing unit 7020 is large. Therefore, the control unit 7070 predicts in advance that the bicycle 7300 on which the imaging device is installed will enter the shade, and in the second imaging region where the imaging conditions are randomly changed, before (immediately before) the imaging conditions (for example, (higher gain). In addition, the control unit 7070 sets the shooting conditions (for example, lowering the gain) for the second imaging area from a predetermined time before leaving the shade, because there is a risk that overexposure may occur the moment the shade leaves, that is, when moving from the shade to the sun. ). The predetermined time may be determined based on the movement amount and speed of the imaging device 7001, the date and time, the weather, and the like.

(3) When setting the shooting conditions remotely;
The control unit 7070 of the imaging device (device to be set) 7001 connects to the control unit 7203 of the smartphone (setting device) 7200 via the communication units 7090 and 7204 via wireless lines. The control unit 7203 of the smart phone 7200 transmits area determination information indicating an imaging area and information indicating imaging conditions for that area to the control unit 7070 in response to the user's operation of the operation unit 7202 . As a result, the user can set imaging conditions using an image (live view image) captured by the imaging device 7001 from the screen of the display unit 7201 of the smartphone 7200 . Shooting conditions that can be set include frame rate, shutter speed, gain, and exposure compensation. By operating the operation unit 7202, the user can also set whether or not to shoot a subject that suddenly jumps into the angle of view.

According to such a configuration, it is possible to photograph a subject that has unexpectedly entered the angle of view, or it is possible not to photograph a subject that has unexpectedly entered the angle of view.

<Twenty-third embodiment>
FIG. 80 is a diagram showing switching timings of imaging conditions of the first imaging region and the second imaging region in the twenty-third embodiment. In the twenty-second embodiment, the first imaging area is an area in which representative imaging conditions that the user (photographer) wants to shoot are set, and the second imaging area is an area in which the imaging conditions are randomly changed. However, the configuration is not limited to this, and as shown in FIG. 80, the control unit 7070 alternately switches between the imaging conditions for the first imaging region and the imaging conditions for the second imaging region at predetermined time intervals. may be configured. That is, the control unit 7070 changes the state in which the representative shooting conditions that the user wishes to shoot in the first imaging region is set to the state in which the shooting conditions are randomly changed, and also randomly changes the shooting conditions in the second imaging region. , to a state in which representative photographing conditions that the user wishes to photograph are set. After a predetermined period of time has passed, the control unit 7070 changes the state in which representative imaging conditions that the user wishes to photograph in the second imaging area are set to the state in which the imaging conditions are changed randomly, and also changes the imaging conditions in the first imaging area. The state in which the conditions are changed at random is changed to the state in which representative photographing conditions that the user wants to photograph are set. Such switching of imaging conditions is performed at predetermined time intervals.

In the area where the imaging conditions are randomly changed, the control unit 7070 and the driving unit 7021 are heavily loaded because the imaging conditions are frequently changed, and heat tends to accumulate in the area where the imaging conditions are randomly changed in the image sensor 7100 . On the other hand, since the other region (the region in which the imaging conditions are not randomly changed) has constant imaging conditions, the load on the control unit 7070 and the driving unit 7021 is less than in the region in which the imaging conditions are changed randomly. , heat is less likely to accumulate in that area. In this way, the control unit 7070 can prevent the load from increasing only in a partial area of the image sensor 7100 by appropriately switching the area where the imaging conditions are changed at random. In addition, in the case of shooting situations where the user (photographer) moves in and out of the shadow, instead of switching at predetermined time intervals, for example, when shooting in the sun, the first imaging area is the representative area that the user (photographer) wants to shoot. and the second imaging area is an area where the imaging conditions are randomly changed, and when shooting in the shade, the second imaging area is the representative shot that the user (photographer) wants to shoot. The area in which the conditions are set, the first imaging area, may be changed to an area in which the imaging conditions are randomly changed. In addition, the control unit 7070 monitors the heat of the driving unit 7021 instead of randomly switching the area for changing the imaging condition every predetermined time, and randomly changes the area when the image sensor 7100 reaches a predetermined temperature or higher. The regions may be switched so as to change the imaging conditions immediately.

In the twenty-second and twenty-third embodiments described above, the control unit 7070 (that is, the setting unit 7071) sets the area (the number of pixels) of the area for which the imaging conditions are changed randomly to the area for which the representative imaging conditions are set. It may be set to be smaller than the area. Also, the control unit 7070 may change the area ratio. For example, in a situation where the shooting conditions should be changed frequently, for example, in a shooting situation in a forested mountain on a sunny day, the control unit 7070 randomly increases the area of the area where the shooting conditions are changed. may Alternatively, a region in which the imaging conditions are changed randomly may be arranged around the imaging device 7100 . This makes it easier to detect a subject that suddenly jumps into the angle of view. Further, by operating the smart phone 7200, the position of the area in which the imaging conditions are to be changed at random may be set. When the user (photographer) guesses from which direction the subject will come into the angle of view with respect to the photographing device 7001, pixels in that direction are randomly set as an area for changing the photographing conditions, The number of pixels may be increased.

Further, in the twenty-second and twenty-third embodiments, when the imaging conditions are randomly changed in a predetermined area, for example, the control unit 7070 extracts random numbers from the random number generator and determines the imaging conditions based on the extracted random numbers. You may Also, in the twenty-second and twenty-third embodiments, the shooting conditions are randomly changed every frame, but the shooting conditions may be changed randomly every several frames. Moreover, although the imaging conditions are randomly changed in a predetermined area, the imaging conditions may be changed regularly (according to a predetermined rule). Also, although the device to be installed is the bicycle 7300, it may be a radio control device.

Further, in the twenty-second and twenty-third embodiments, the control unit 7070 may optimize the randomly changed imaging conditions according to the imaging situation. For example, the control unit 7070 may determine the shooting conditions based on images captured for a predetermined period of time, and narrow the range of shooting conditions that are randomly changed based on the determined shooting conditions. By providing the control unit 7070 with a learning function, the processing load on the control unit 7070 and the imaging unit 7020 can be reduced.

DESCRIPTION OF SYMBOLS 1, 1B... Digital camera 1A... Imaging device 20... Imaging part 30, 30... Image processing part 50... Display part 70... System control part 70A... First system control part 70B... Second system control Part 71... Division part 72... Drive control part 73... Control part 74... Blink detection part 100... Imaging element 113A... Pixel area (imaging area) 416... Arithmetic circuit (signal processing part, block determination part ), 1001, 1001B... Electronic equipment, 1001A... Imaging device, 1020... Imaging unit, 1030... Image processing unit, 1031... Image generating unit, 1032... Detecting unit, 1050... Display unit, 1051... Display panel, 1052... Touch panel ( selection unit), 1070... system control unit, 1070A... first system control unit, 1070B... second system control unit, 1071... display control unit, 1072... setting unit (frame rate setting unit), 1073... imaging control unit, 1074 AF processing unit 1100 image sensor 2001, 2001B electronic device 2001A imaging device 2020 imaging unit 2030 image processing unit 2031 detection unit (mode detection unit) 2050, 2050A, 2050B display Unit 2051 Display surface 2052 Touch panel 2070 System control unit 2070A First system control unit 2070B Second system control unit 2071 Setting unit 2072 Control unit 2073 Display control unit 2074 ... AF processing section (focus detection section) 2075 ... AE processing section 2100 ... imaging element 3001, 3001B ... electronic device 3001A ... imaging device 3020 ... imaging section 3030 ... image processing section 3031 ... conversion section 3050 Display unit 3051 Display screen 3052 Touch panel 3070 System control unit 3070A First system control unit 3070B Second system control unit 3071 Area division unit 3072 Control unit 3100 Image sensor , 4001...Digital camera (electronic device) 4100...Image sensor 4070...System control unit (control unit) 5001...Smartphone (electronic device) 5030...Image processing unit 5070...System control unit 5071...Operation detection unit , 5072 drive control unit 6020 imaging unit 6030 image processing unit 6021 selector (selection unit) 6023 buffer 6032 readout control unit (control unit) 6050 sensor A1 to A16 unit area , L1 to L16... transmission path, 7001... imaging device (device to be set), 7020... imaging unit, 7070... control unit, 7071... setting unit, 7200... smartphone (setting device, external device), 7300... bicycle (to be installed device), 7090...Communication unit

Claims (31)

An imaging device comprising a plurality of stacked semiconductor chips,
The plurality of semiconductor chips are
a first semiconductor chip in which a plurality of pixels are arranged in a row direction and have a photoelectric conversion portion that converts light into an electric charge; and a transfer portion that transfers the electric charge converted by the photoelectric conversion portion;
A first pixel outputting a first signal used for flicker detection and a second pixel outputting a second signal used for generating an image of a subject among the plurality of pixels are imaged differently. a second semiconductor chip having a driving unit driven under a condition;
has
the transfer unit included in the first pixel is connected to a first transfer control wiring for outputting a first transfer control signal from the second semiconductor chip;
The transfer unit included in the second pixel is an imaging device connected to a second transfer control wiring through which a second transfer control signal is output from the second semiconductor chip. In the imaging device according to claim 1,
the plurality of pixels have a reset unit for discharging the charges converted by the photoelectric conversion unit;
the reset unit included in the first pixel is connected to a first reset control wiring for outputting a first reset control signal from the second semiconductor chip;
The reset section included in the second pixel is an imaging device connected to a second reset control wiring through which a second reset control signal is output from the second semiconductor chip. In the imaging device according to claim 1,
The drive unit drives the first pixel and a third pixel among the plurality of pixels that outputs a third signal used for flicker detection under different imaging conditions,
The transfer unit included in the third pixel is an imaging device connected to a third transfer control wiring through which a third transfer control signal from the second semiconductor chip is output. In the imaging device according to claim 3,
the plurality of pixels have a reset unit for discharging the charges converted by the photoelectric conversion unit;
the reset unit included in the first pixel is connected to a first reset control wiring for outputting a first reset control signal from the second semiconductor chip;
the reset unit included in the second pixel is connected to a second reset control wiring for outputting a second reset control signal from the second semiconductor chip;
The reset section included in the third pixel is an imaging device connected to a third reset control wiring through which a third reset control signal from the second semiconductor chip is output. In the imaging device according to claim 3 or claim 4,
A said 2nd pixel is an image pick-up element arrange|positioned between a said 1st pixel and a said 3rd pixel in the said row direction. An imaging device comprising a plurality of stacked semiconductor chips,
The plurality of semiconductor chips are
a first semiconductor chip in which a plurality of pixels are arranged in a row direction and have a photoelectric conversion unit that converts light into an electric charge; and a reset unit that discharges the electric charge converted by the photoelectric conversion unit;
A first pixel outputting a first signal used for flicker detection and a second pixel outputting a second signal used for generating an image of a subject among the plurality of pixels are imaged differently. a second semiconductor chip having a driving unit driven under a condition;
has
the reset unit included in the first pixel is connected to a first reset control wiring for outputting a first reset control signal from the second semiconductor chip;
The reset section included in the second pixel is an imaging device connected to a second reset control wiring through which a second reset control signal is output from the second semiconductor chip. In the imaging device according to claim 6,
The drive unit drives the first pixel and a third pixel among the plurality of pixels that outputs a third signal used for flicker detection under different imaging conditions,
The reset section included in the third pixel is an imaging device connected to a third reset control wiring through which a third reset control signal from the second semiconductor chip is output. In the imaging device according to claim 7,
A said 2nd pixel is an image pick-up element arrange|positioned between a said 1st pixel and a said 3rd pixel in the said row direction. In the imaging device according to any one of claims 1 to 8,
The driving section drives the first pixel and the second pixel such that signal readout is performed at different frame rates. In the imaging device according to claim 9,
The driving unit is an imaging device that drives the frame rate of the first pixels to be higher than the frame rate of the second pixels. In the imaging device according to any one of claims 1 to 10,
The driving section drives the photoelectric conversion section of the first pixel and the photoelectric conversion section of the second pixel such that charge accumulation is performed in different charge accumulation times. In the imaging device according to claim 11,
The driving section drives the imaging device so that the charge accumulation time of the photoelectric conversion section of the first pixel is shorter than the charge accumulation time of the photoelectric conversion section of the second pixel. In the imaging device according to any one of claims 1 to 12,
A plurality of the first pixels are arranged in the first semiconductor chip,
Said 2nd pixel is an image pick-up element with which multiple arrangement|positioning is carried out at said 1st semiconductor chip. In the imaging device according to claim 13,
The imaging device, wherein the number of the first pixels is smaller than the number of the second pixels. In the imaging device according to claim 13 or 14,
The second semiconductor chip has an arithmetic circuit that performs an averaging process on the first signals output from two or more of the first pixels among the plurality of first pixels. In the imaging device according to claim 15,
The arithmetic circuit performs the averaging process on the first signals output from two or more of the first pixels selected from among the plurality of first pixels. In the imaging device according to claim 16,
The arithmetic circuit selects the first signal to be subjected to the averaging process based on the signal values of the first signals output from the plurality of first pixels. In the imaging device according to any one of claims 15 to 17,
The arithmetic circuit is an imaging device that detects flicker based on the signal subjected to the averaging process. In the imaging device according to claim 18,
The arithmetic circuit detects flicker based on the averaged signal while the image generated based on the second signal is displayed on the display unit. In the imaging device according to claim 18 or 19,
The driving unit is an imaging device that adjusts the timing of driving the second pixels based on the result of flicker detection by the arithmetic circuit. In the imaging device according to claim 20,
The drive section adjusts a frame rate of the second pixels based on a flicker detection result by the arithmetic circuit. In the imaging device according to claim 20 or 21,
The driving section adjusts the accumulation time of the electric charges converted by the photoelectric conversion section of the second pixel based on the result of flicker detection by the arithmetic circuit. In the imaging device according to claim 13 or 14,
The plurality of semiconductor chips includes an imaging device having a third semiconductor chip having an arithmetic circuit for averaging the first signals output from two or more of the first pixels among the plurality of first pixels. . 24. The imaging device according to claim 23,
The arithmetic circuit performs the averaging process on the first signals output from two or more of the first pixels selected from among the plurality of first pixels. 25. The imaging device according to claim 24,
The arithmetic circuit selects the first signal to be subjected to the averaging process based on the signal values of the first signals output from the plurality of first pixels. In the imaging device according to any one of claims 23 to 25,
The arithmetic circuit is an imaging device that detects flicker based on the signal subjected to the averaging process. The imaging device according to claim 26,
The arithmetic circuit detects flicker based on the averaged signal while the image generated based on the second signal is displayed on the display unit. In the imaging device according to claim 26 or 27,
The driving unit is an imaging device that adjusts the timing of driving the second pixels based on the result of flicker detection by the arithmetic circuit. 29. The imaging device according to claim 28,
The drive section adjusts a frame rate of the second pixels based on a flicker detection result by the arithmetic circuit. In the imaging device according to claim 28 or 29,
The driving section adjusts the accumulation time of the electric charges converted by the photoelectric conversion section of the second pixel based on the result of flicker detection by the arithmetic circuit. An electronic device comprising the imaging device according to any one of claims 1 to 30.

Images