JP7176591B2 - Electronics - Google Patents

Description

The present invention relates to electronic equipment .

2. Description of the Related Art There has been proposed an electronic device provided with an image pickup device in which a back-illuminated image pickup chip and a signal processing chip are stacked (hereinafter, this image pickup device will be referred to as a stacked image pickup device) (see Patent Document 1, for example). In the stacked imaging device, a back-illuminated imaging chip and a signal processing chip are stacked such that they are connected via microbumps for each block unit in which a plurality of pixels are grouped together.

JP-A-2006-49361

However, there are not many proposals for obtaining an image by capturing an image in units of a plurality of blocks in electronic devices equipped with conventional stacked imaging devices, and the usability of electronic devices equipped with stacked imaging devices has not been sufficient.

According to the aspect of the present invention, the accumulation condition for the first area of the live view image is different from the accumulation condition for the second area, so that the change in the image corresponding to the change in the accumulation condition for each area can be confirmed in the live view image. for the purpose.

An aspect of the present invention provides an electronic device. An electronic device includes a plurality of photoelectric conversion units that are arranged side by side in row and column directions and convert light into charge. The electronic device stores the number of accumulations of charges converted by a first photoelectric conversion unit among the plurality of photoelectric conversion units at a position in the row direction from the first photoelectric conversion unit among the plurality of photoelectric conversion units. 2 An operation unit for receiving an operation for setting the number of times of accumulation of charges converted by the photoelectric conversion unit to be different from the number of times of accumulation. The electronic device generates a first signal based on the charge of the first photoelectric conversion unit controlled by the control unit and a second signal based on the charge of the second photoelectric conversion unit controlled by the control unit. and a display unit for displaying an image generated using the display unit. The plurality of photoelectric conversion units are arranged on a first semiconductor substrate on which light is incident. The controller is arranged on a second semiconductor substrate connected to the first semiconductor substrate.

According to the aspect of the present invention, by differentiating the accumulation condition for the first area of the live-view image from the accumulation condition for the second area, the change in the image corresponding to the change in the accumulation condition for each area can be displayed in the live-view image. can be confirmed.

1 is a cross-sectional view of a stacked imaging device; FIG. It is a figure explaining the pixel arrangement|sequence of an imaging chip, and a unit group. FIG. 4 is a circuit diagram corresponding to a unit group of imaging chips; 3 is a block diagram showing the functional configuration of an imaging device; FIG. 1 is a block diagram showing the configuration of an electronic device according to a first embodiment; FIG. It is a figure which shows an example of the display screen in a display part. 3 is a functional block diagram of an image processing section and a system control section in the first embodiment; FIG. 4 is a flowchart for explaining a shooting operation performed by a system control unit according to the first embodiment; FIG. 10 is a diagram showing a display example of a display screen when a small number of blocks are set in an image display area; FIG. 10 is a diagram showing a display example of a display screen when a small number of blocks are set in an image display area; FIG. 10 is a diagram showing a display example of a display screen when a small number of blocks are set in an image display area; FIG. 10 is a diagram showing a display example of a display screen when a small number of blocks are set in an image display area; 4 is a flowchart showing an example of imaging processing according to the first embodiment; FIG. 10 is a diagram showing a display example of a display screen when a large number of blocks are set in an image display area; FIG. 10 is a diagram showing a display example of a display screen when a large number of blocks are set in an image display area; FIG. 10 is a diagram showing a display example of a display screen when a large number of blocks are set in an image display area; 8 is a functional block diagram of an image processing unit and a system control unit in the second embodiment; FIG. 9 is a flowchart for explaining a shooting operation performed by a system control unit according to the second embodiment; 7 is a flow chart showing an example of automatic setting processing of an area and imaging conditions; 9 is a flowchart showing an example of imaging processing according to the second embodiment; FIG. 10 is a diagram showing a display example of a display screen when only a selected region is imaged; FIG. 11 is a block diagram showing the configuration of an imaging device and an electronic device according to a third embodiment; It is a figure which shows the example of a display of the display screen in 4th Embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to this. In addition, in the drawings, in order to describe the embodiments, the scale is changed as appropriate, such as by enlarging or emphasizing a portion.

<First embodiment>
FIG. 1 is a cross-sectional view of a stacked imaging device. The stacked imaging device 100 is described in Japanese Patent Application No. 2012-139026 previously filed by the applicant of the present application. The imaging device 100 includes an imaging chip 113 that outputs pixel signals corresponding to incident light, a signal processing chip 111 that processes the pixel signals, and a memory chip 112 that stores the pixel signals. These imaging chip 113, signal processing chip 111, and memory chip 112 are stacked and electrically connected to each other by conductive bumps 109 such as Cu.

Incidentally, as shown in the figure, the incident light is mainly incident in the Z-axis plus direction indicated by the white arrow. In the present embodiment, the surface of the imaging chip 113 on which incident light is incident is referred to as the rear surface. Also, as shown in the coordinate axes, the leftward direction of the paper perpendicular to the Z-axis is the positive X-axis direction, and the frontward direction of the paper perpendicular to the Z-axis and the X-axis is the positive Y-axis direction. In the following several figures, the coordinate axes are displayed with reference to the coordinate axes of FIG. 1 so that the direction of each figure can be understood.

An example of the imaging chip 113 is a back-illuminated MOS image sensor. The PD layer 106 is arranged on the back side of the wiring layer 108 . The PD layer 106 has a plurality of photodiodes (hereinafter referred to as PDs) 104 arranged two-dimensionally and storing charges according to incident light, and transistors 105 provided corresponding to the PDs 104 . .

A color filter 102 is provided on the incident light side of the PD layer 106 with a passivation film 103 interposed therebetween. The color filter 102 is a filter that passes a specific wavelength region of visible light. This color filter 102 has a plurality of types that transmit different wavelength regions, and has a specific arrangement corresponding to each of the PDs 104 . The arrangement of the color filters 102 will be described later. A set of color filter 102, PD 104, and transistor 105 forms one pixel.

A microlens 101 is provided on the incident light side of the color filter 102 so as to correspond to each pixel. The microlens 101 collects incident light toward the corresponding PD 104 .

The wiring layer 108 has wiring 107 for transmitting pixel signals from the PD layer 106 to the signal processing chip 111 . The wiring 107 may be multi-layered and may be provided with passive and active elements. A plurality of bumps 109 are arranged on the surface of the wiring layer 108 . These bumps 109 are aligned with the bumps 109 provided on the opposite surface of the signal processing chip 111 . By pressurizing the imaging chip 113 and the signal processing chip 111, the aligned bumps 109 are bonded and electrically connected.

Similarly, a plurality of bumps 109 are arranged on surfaces of the signal processing chip 111 and the memory chip 112 facing each other. These bumps 109 are aligned with each other. By applying pressure to the signal processing chip 111 and the memory chip 112, the aligned bumps 109 are bonded and electrically connected.

The bonding between the bumps 109 is not limited to Cu bump bonding by solid phase diffusion, and may be microbump bonding by solder melting. Also, about one bump 109 may be provided for one unit group, which will be described later. Therefore, the size of the bumps 109 may be larger than the pitch of the PDs 104 . Also, bumps larger than the bumps 109 corresponding to the pixel regions may be provided in peripheral regions other than the pixel region (pixel region 113A shown in FIG. 2) in which pixels are arranged.

The signal processing chip 111 has a TSV (Through-Silicon Via) 110 that connects circuits provided on the front surface and the back surface. The TSV 110 is provided in the peripheral area. Also, the TSV 110 may be provided in the peripheral area of the imaging chip 113 or in the memory chip 112 .

FIG. 2 is a diagram for explaining the pixel array and unit groups of the imaging chip. FIG. 2 particularly shows a state in which the imaging chip 113 is observed from the back side. A region in which pixels are arranged in the imaging chip 113 is called a pixel region 113A. More than 20 million pixels are arranged in a matrix in the pixel area 113A. In the example shown in FIG. 2, 16 pixels of adjacent 4×4 pixels form one unit group 131 . The grid lines in FIG. 2 illustrate the concept that adjacent pixels are grouped together to form unit groups 131 . The number of pixels forming the unit group 131 is not limited to this, and may be about 1000, for example, 32 pixels×64 pixels, or may be more or less.

As shown in the partially enlarged view of the pixel region 113A, the unit group 131 contains four so-called Bayer arrays each including four pixels of green pixels Gb and Gr, blue pixels B, and red pixels R, vertically and horizontally. A green pixel is a pixel having a green filter as the color filter 102, and receives light in the green wavelength band among incident light. Similarly, a blue pixel is a pixel having a blue filter as the color filter 102 and receives light in the blue wavelength band. A red pixel is a pixel having a red filter as the color filter 102 and receives light in the red wavelength band.

FIG. 3 is a circuit diagram corresponding to a unit group of imaging chips. In FIG. 3, a rectangle encircled by dotted lines typically represents a circuit corresponding to one pixel. Note that at least part of each transistor described below corresponds to the transistor 105 in FIG.

As described above, the unit group 131 is formed from 16 pixels. The 16 PDs 104 corresponding to each pixel are connected to transfer transistors 302 respectively. A gate of each transfer transistor 302 is connected to a TX wiring 307 to which a transfer pulse is supplied. In this embodiment, the TX wiring 307 is commonly connected to 16 transfer transistors 302 .

The drain of each transfer transistor 302 is connected to the source of each corresponding reset transistor 303 , and the so-called floating diffusion FD (charge detector) between the drain of the transfer transistor 302 and the source of each reset transistor 303 is connected to the amplification transistor 304 . connected to the gate. A drain of each reset transistor 303 is connected to a Vdd wiring 310 to which a power supply voltage is supplied. A gate of each reset transistor 303 is connected to a reset wiring 306 to which a reset pulse is supplied. In this embodiment, the reset wiring 306 is commonly connected to the 16 reset transistors 303 .

A drain of each amplifying transistor 304 is connected to a Vdd wiring 310 supplied with a power supply voltage. Also, the source of each amplification transistor 304 is connected to the drain of each corresponding selection transistor 305 . A gate of each selection transistor 305 is connected to a decoder wiring 308 to which a selection pulse is supplied. In this embodiment, the decoder wiring 308 is provided independently for each of the 16 selection transistors 305 . A source of each selection transistor 305 is connected to a common output wiring 309 . A load current source 311 supplies current to the output wiring 309 . That is, the output wiring 309 for the selection transistor 305 is formed by a source follower. The load current source 311 may be provided on the imaging chip 113 side or may be provided on the signal processing chip 111 side.

Here, the flow from the start of charge accumulation to the pixel output after the end of charge accumulation will be described. A reset pulse is applied to the reset transistor 303 through the reset wiring 306 . At the same time, a transfer pulse is applied to the transfer transistor 302 through the TX wiring 307 . This resets the potentials of the PD 104 and the floating diffusion FD.

When the application of the transfer pulse is released, the PD 104 converts the incident light it receives into charges and accumulates them. After that, when the transfer pulse is applied again while the reset pulse is not applied, the charges accumulated in the PD 104 are transferred to the floating diffusion FD. As a result, the potential of the floating diffusion FD changes from the reset potential to the signal potential after charge accumulation. Then, when a selection pulse is applied to the selection transistor 305 through the decoder wiring 308 , fluctuations in the signal potential of the floating diffusion FD are transmitted to the output wiring 309 via the amplification transistor 304 and the selection transistor 305 . By such circuit operation, a pixel signal corresponding to the reset potential and the signal potential is output from the unit pixel to the output wiring 309 .

As shown in FIG. 3, in this embodiment, the reset wiring 306 and the TX wiring 307 are common to the 16 pixels forming the unit group 131 . That is, the reset pulse and the transfer pulse are each applied simultaneously to all 16 pixels. Therefore, all the pixels forming the unit group 131 start charge accumulation at the same timing and finish charge accumulation at the same timing. However, pixel signals corresponding to the accumulated charges are selectively output to the output wiring 309 by sequentially applying selection pulses to the respective selection transistors 305 . Also, the reset wiring 306 , the TX wiring 307 , and the output wiring 309 are provided separately for each unit group 131 .

By configuring the circuit on the basis of the unit group 131 in this way, the charge accumulation time can be controlled for each unit group 131 . In other words, it is possible to output pixel signals with different charge accumulation times between the unit groups 131 . Furthermore, while one unit group 131 is caused to perform one charge accumulation, the other unit group 131 is caused to repeat charge accumulation many times and to output a pixel signal each time. It is also possible to output each moving image frame at a different frame rate between the unit groups 131 .

FIG. 4 is a block diagram showing the functional configuration of the imaging device. Analog multiplexer 411 sequentially selects 16 PDs 104 forming unit group 131 . Then, the multiplexer 411 outputs the pixel signal of each of the 16 PDs 104 to the output wiring 309 provided corresponding to the unit group 131 . A multiplexer 411 is formed in the imaging chip 113 together with the PD 104 .

The analog pixel signal output via the multiplexer 411 is amplified by the amplifier 412 formed in the signal processing chip 111 . Then, the pixel signal amplified by the amplifier 412 is processed by a signal processing circuit 413 formed in the signal processing chip 111 that performs correlated double sampling (CDS)/analog/digital conversion. Signal processing of correlated double sampling is performed, and A/D conversion (conversion from analog signal to digital signal) is performed. Noise in the pixel signal is reduced by subjecting the pixel signal to signal processing of correlated double sampling in the signal processing circuit 413 . A/D-converted pixel signals are transferred to a demultiplexer 414 and stored in pixel memories 415 corresponding to respective pixels. Demultiplexer 414 and pixel memory 415 are formed in memory chip 112 .

The arithmetic circuit 416 processes the pixel signal stored in the pixel memory 415 and transfers it to the subsequent image processing unit. The arithmetic circuit 416 may be provided in the signal processing chip 111 or may be provided in the memory chip 112 . Although FIG. 4 shows connections for one unit group 131, in reality, these units exist for each unit group 131 and operate in parallel. However, the arithmetic circuit 416 may not exist for each unit group 131 . For example, one arithmetic circuit 416 may sequentially refer to the values of the pixel memory 415 corresponding to each unit group 131 and process them sequentially.

As described above, the output wiring 309 is provided corresponding to each unit group 131 . The imaging device 100 has an imaging chip 113, a signal processing chip 111, and a memory chip 112 stacked. Therefore, by using the bumps 109 as the output wirings 309 for electrical connection between the chips, the wiring can be routed without increasing the size of each chip in the planar direction.

Next, blocks set in the pixel area 113A (see FIG. 2) of the image sensor 100 will be described. In this embodiment, the pixel area 113A of the image sensor 100 is divided into a plurality of blocks. A plurality of blocks are defined to include at least one unit group 131 per block. Pixels included in each block are controlled with different control parameters. That is, pixel signals with different control parameters are obtained for a pixel group included in a certain block and a pixel group included in another block. Control parameters include, for example, charge accumulation time or number of accumulations, frame rate, gain, thinning rate, number of addition rows or addition columns for adding pixel signals, number of bits for digitization, and the like. Furthermore, the control parameter may be a parameter in image processing after acquiring image signals from pixels.

Here, the charge accumulation time is the time from when the PD 104 starts accumulating charges until it ends. The number of charge accumulations refers to the number of times the PD 104 accumulates charges per unit time. A frame rate is a value representing the number of frames processed (displayed or recorded) per unit time in a moving image. The unit of frame rate is fps (Frames Per Second). The higher the frame rate, the smoother the movement of the subject in the moving image.

A gain is a gain factor (amplification factor) of the amplifier 412 . By changing this gain, the ISO sensitivity can be changed. The ISO sensitivity is a standard for photographic film established by ISO, and indicates how weak light the photographic film can record. However, in general, the ISO sensitivity is also used when expressing the sensitivity of the imaging device 100 . In this case, the ISO sensitivity is a value representing the ability of the image sensor 100 to capture light. Increasing the gain also improves the ISO sensitivity. For example, if the gain is doubled, the electric signal (pixel signal) is also doubled, and even if the amount of incident light is halved, appropriate brightness can be obtained. However, increasing the gain also amplifies the noise contained in the electrical signal, resulting in increased noise.

Also, the thinning rate refers to the ratio of the number of pixels whose pixel signals are not read out to all the number of pixels in a predetermined area. For example, when the thinning rate of a predetermined area is 0, it means that pixel signals are read out from all pixels in the predetermined area. Further, when the thinning rate of a predetermined area is 0.5, it means that pixel signals are read out from half of the pixels in the predetermined area. Specifically, when the unit group 131 has a Bayer array, every other unit of the Bayer array in the vertical direction, that is, pixels from which pixel signals are alternately read out by two pixels (by two rows) in a pixel unit and read out. Pixels that are not displayed are set. It should be noted that the resolution of the image decreases when the readout of the pixel signals is thinned out. However, since 20 million or more pixels are arranged in the imaging device 100, even if thinning is performed at a thinning rate of 0.5, an image can be displayed with 10 million or more pixels. For this reason, it is considered that the user does not mind the decrease in resolution.

The number of rows to be added refers to the number of pixels in the vertical direction (the number of rows) to be added when pixel signals of pixels adjacent to each other in the vertical direction are added. The number of columns to be added refers to the number of pixels in the horizontal direction (the number of columns) to be added when pixel signals of horizontally adjacent pixels are added. Such addition processing is performed in the arithmetic circuit 416, for example. The arithmetic circuit 416 adds the pixel signals of a predetermined number of vertically or horizontally adjacent pixels, thereby providing the same effect as reading out pixel signals by thinning at a predetermined thinning rate. In addition, in the above-described addition processing, the average value may be calculated by dividing the added value by the number of rows or columns added by the arithmetic circuit 416 .

Also, the number of bits for digitization refers to the number of bits when the signal processing circuit 413 converts an analog signal into a digital signal in A/D conversion. As the number of bits of the digital signal increases, luminance, color change, etc. are expressed in more detail.

In the present embodiment, the accumulation condition refers to a condition regarding charge accumulation in the image sensor 100 . Specifically, the accumulation condition refers to the charge accumulation time or the number of accumulations, the frame rate, and the gain among the control parameters described above. The frame rate is included in the accumulation conditions because the frame rate can change according to the charge accumulation time and the number of accumulations. In addition, the amount of light for proper exposure changes according to the gain, and the charge accumulation time or number of times of accumulation may also change according to the amount of light for proper exposure. Therefore, gain is included in the accumulation condition.

Also, the imaging conditions refer to conditions related to imaging of a subject. Specifically, the imaging conditions refer to control parameters including the accumulation conditions described above. The imaging conditions include control parameters for controlling the image sensor 100 (for example, charge accumulation time or number of times, frame rate, gain) and control parameters for controlling readout of signals from the image sensor 100 ( For example, a thinning rate), control parameters for processing the signal from the image sensor 100 (for example, the number of addition rows or addition columns for adding pixel signals, the number of bits for digitization, the image processing unit 30 described later image processing control parameters for executing) are also included.

FIG. 5 is a block diagram showing the configuration of the electronic device according to the first embodiment. As shown in FIG. 5, the electronic device 1 includes a lens unit 10, an imaging unit 20, an image processing unit 30, a work memory 40, a display unit 50, an operation unit 55, a recording unit 60, and a system control unit . The lens unit 10 is an imaging optical system configured from a plurality of lens groups. The lens unit 10 guides the light flux from the subject to the imaging unit 20 . The lens unit 10 may be integrated with the electronic device 1 or may be an interchangeable lens detachable from the electronic device 1 . Further, the lens unit 10 may incorporate a focus lens or may incorporate a zoom lens.

The imaging unit 20 has an imaging device 100 and a driving unit 21 . The driving unit 21 is a control circuit that controls the driving of the imaging device 100 according to instructions from the system control unit 70 . Here, the driving unit 21 controls the timing (or timing cycle) of applying the reset pulse and the transfer pulse to the reset transistor 303 and the transfer transistor 302, respectively, thereby setting the charge accumulation time or the number of times of accumulation, which are control parameters. Control. Further, the drive unit 21 controls the frame rate by controlling the timing (or timing cycle) of applying the reset pulse, the transfer pulse, and the selection pulse to the reset transistor 303, the transfer transistor 302, and the selection transistor 305, respectively. do. Further, the drive unit 21 controls the thinning rate by setting the pixels to which the reset pulse, transfer pulse, and selection pulse are applied.

Further, the drive unit 21 controls the ISO sensitivity of the image sensor 100 by controlling the gain (also referred to as gain factor or amplification factor) of the amplifier 412 . Further, the drive unit 21 sets the number of addition rows or the number of addition columns for adding the pixel signals by sending an instruction to the arithmetic circuit 416 . Further, the drive unit 21 sets the number of bits for digitization by sending an instruction to the signal processing circuit 413 . Further, the driving unit 21 sets blocks in the pixel area (imaging area) 113A of the image sensor 100 . In this way, the drive unit 21 functions as an image pickup device control unit that causes the image pickup device 100 to pick up images under different image pickup conditions for each of a plurality of blocks and output pixel signals. The system control unit 70 instructs the drive unit 21 about the position, shape, range, and the like of the block.

The imaging device 100 transfers pixel signals from the imaging device 100 to the image processing unit 30 . Using the work memory 40 as a work space, the image processing unit 30 performs various image processing on raw image data composed of pixel signals of pixels to generate image data. The image processing section 30 has a CPU, and the CPU executes various image processing. For example, the image processing unit 30 generates RGB image signals by performing color signal processing on the signals obtained in the Bayer array. The image processing unit 30 also performs image processing such as white balance adjustment, sharpness adjustment, gamma correction, and gradation adjustment on the RGB image signal. Further, the image processing unit 30 performs processing for compressing in a predetermined compression format (JPEG format, MPEG format, etc.) as necessary. The image data generated by the image processing section 30 is recorded in the recording section 60 via the system control section 70 . The image data generated by the image processing section 30 is output to the display section 50 via the system control section 70 and displayed on the display section 50 .

In this embodiment, the image processing unit 30 performs, in addition to the above-described processing, a processing of synthesizing image data based on images that have been successively captured a plurality of times. Noise is removed by such processing.

Control parameters (imaging conditions) also include parameters referred to when the image processing unit 30 performs image processing. For example, control parameters include parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate. A signal read out from the image pickup device 100 changes according to the charge accumulation time or the like, and a parameter referred to when image processing is performed changes according to the change of the signal. The image processing unit 30 sets different control parameters for each block and executes image processing such as color signal processing based on these control parameters.

The image processing unit 30 extracts frames at predetermined timings from among the plurality of frames obtained in time series from the imaging unit 20 . Alternatively, the image processing unit 30 discards frames at predetermined timings among the plurality of frames obtained in time series from the imaging unit 20 . As a result, the amount of data can be reduced, and the load of post-processing can be reduced. Also, the image processing unit 30 calculates one or more frames to be interpolated between each frame based on a plurality of frames obtained in time series from the imaging unit 20 . Then, the image processing unit 30 adds one or more calculated frames between each frame. As a result, it is possible to reproduce moving images with smoother movements during moving image reproduction. Further, although the drive unit 21 is configured to control the thinning rate, the configuration is not limited to this. For example, although the driving unit 21 reads out pixel signals from all pixels, the image processing unit 30 or the arithmetic circuit 416 may control the thinning rate by discarding predetermined pixel signals among the read out pixel signals. good.

The work memory 40 temporarily stores image data and the like when image processing is performed by the image processing unit 30 . The display unit 50 is configured by, for example, a liquid crystal display panel. As shown in FIG. 5, the display unit 50 includes a first display unit 51, a first touch panel (selection unit, first operation unit) 52, a second display unit 53, and a second touch panel (second touch panel). 2 operating part) 54 .

The first display unit 51 displays images (still images, moving images, live view images) captured by the imaging unit 20 and various information. The first touch panel 52 is formed on the display screen of the first display section 51 . The first touch panel 52 outputs a signal indicating the position touched by the user to the system control unit 70 when the user designates an area. The second display unit 53 displays a menu for the user to set imaging conditions (control parameters). The second touch panel 54 is formed on the display screen of the second display section 53 . The second touch panel 54 outputs a signal indicating the position touched by the user to the system control unit 70 when the user sets imaging conditions.

The operation unit 55 includes a shutter button, a recording start button, various operation switches, and the like, which are operated by the user. The operation unit 55 outputs a signal to the system control unit 70 according to the user's operation. The recording unit 60 has a card slot into which a storage medium such as a memory card can be inserted. The recording unit 60 stores image data generated by the image processing unit 30 and various data in a recording medium loaded in the card slot. Also, the recording unit 60 has an internal memory. The recording unit 60 may be configured to record image data and various data generated by the image processing unit 30 in an internal memory.

The system control unit 70 controls the overall processing and operation of the electronic device 1 . The system control unit 70 has a CPU (Central Processing Unit). In this embodiment, the system control unit 70 divides the imaging surface (pixel region 113A) of the imaging device 100 (imaging chip 113) into a plurality of blocks, and sets different charge accumulation times (or charge accumulation times) and frame rates between the blocks. , to acquire an image with gain. For this reason, the system control unit 70 instructs the drive unit 21 about the position, shape, range, and storage conditions for each block. In addition, the system control unit 70 acquires an image with a thinning rate different between blocks, the number of addition rows or addition columns for adding pixel signals, and the number of bits for digitization. For this reason, the system control unit 70 instructs the driving unit 21 on imaging conditions for each block (thinning rate, number of rows or columns to be added for adding pixel signals, and number of bits for digitization). Further, the image processing unit 30 executes image processing under different imaging conditions (color signal processing, white balance adjustment, gradation adjustment, control parameters such as compression ratio) between blocks. Therefore, the system control unit 70 instructs the image processing unit 30 on imaging conditions for each block (control parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate).

Also, the system control unit 70 causes the recording unit 60 to record the image data generated by the image processing unit 30 . Further, the system control unit 70 outputs the image data generated by the image processing unit 30 to the display unit 50 to display an image on the first display unit 51 of the display unit 50 . Alternatively, the system control unit 70 reads the image data recorded in the recording unit 60 and outputs it to the display unit 50 to display an image on the first display unit 51 of the display unit 50 . Images displayed on the first display unit 51 include still images, moving images, and live view images. Here, the live view image is an image displayed on the first display unit 51 by sequentially outputting the image data generated by the image processing unit 30 to the display unit 50 . The live view image is used by the user to check the image of the subject being imaged by the imaging unit 20 . A live view image is also called a through image or a preview image.

Note that the image processing unit 30 is implemented by the CPU executing processing based on a control program. Moreover, the system control unit 70 is implemented by the CPU executing processing based on a control program.

FIG. 6 is a diagram showing an example of a display screen on the display unit. As shown in FIG. 6, the display screen 500 of the display unit 50 is provided with an image display area 510 and an operation button display area 520 . The image display area 510 is an area for displaying images captured by the imaging unit 20, that is, still images, moving images, and live view images. Also, the operation button display area 520 is an area for displaying a menu for the user to set imaging conditions and the like. In this embodiment, the image display area 510 and the operation button display area 520 are provided within one display screen 500 . The image display area 510 corresponds to the first display section 51 , and the operation button display area 520 corresponds to the second display section 53 .

A plurality of blocks B(i, j) are set in the image display area 510 . In the example shown in FIG. 6, in the image display area 510, eight blocks (i=1 to 8) are set in the image display area 510 (horizontal direction in FIG. 6), and eight blocks (i=1 to 8) are set in the vertical direction ( Six blocks (j=1 to 6) are set in the vertical direction of FIG. That is, the image display area 510 is divided into 8×6 (=48) blocks. Also, the first touch panel 52 is provided on the image display area 510 . In the first touch panel 52, a plurality of touch areas P(i, j) are formed so as to overlap each of the plurality of blocks B(i, j). When each touch area P(i, j) detects that it has been pressed (touched) by the user, it sends a detection signal indicating the pressed position (which touch area) to the system control unit 70. Output.

The operation button display area 520 is provided near the image display area 510 . A menu (menu image) is displayed in the operation button display area 520 for the user to set the area, imaging conditions, and imaging mode. A setting button 521 is a menu for the user to set the area. The menu for the user to set imaging conditions includes an ISO sensitivity button 522 (this is also referred to as a gain-related menu), a shutter speed button 523 (this is also referred to as a charge accumulation time menu), and a frame A rate button 524 (this is also referred to as a frame rate menu). A shooting mode button 525 is a menu for the user to set the shooting mode. Hereinafter, the setting button 521, the ISO sensitivity button 522, the shutter speed button 523, the frame rate button 524, and the shooting mode button 525 are simply referred to as "setting 521," "ISO sensitivity 522," respectively. They are described as "shutter 523", "frame rate 524", and "shooting mode 525".

A setting 521 is a button that the user presses when setting (selecting) an area within the image display area 510 in units of blocks. The ISO sensitivity 522 is a button that the user presses when setting the ISO sensitivity (that is, gain). The shutter 523 is a button that the user presses when setting the shutter speed (that is, exposure time). Note that the shutter speed corresponds to the charge accumulation time. A frame rate 524 is a button that the user presses when setting the frame rate of moving images and live view images. A shooting mode 525 is a button that the user presses when selecting whether to manually set the imaging conditions or to automatically set the imaging conditions.

Also, the second touch panel 54 is provided on the operation button display area 520 . On the second touch panel 54, a touch area 521a is formed so as to overlap a setting 521. As shown in FIG. Also, the touch area 522 a is formed so as to overlap the ISO sensitivity 522 . Also, a touch area 523 a is formed so as to overlap the shutter 523 . Also, a touch area 524 a is formed so as to overlap the frame rate 524 . Also, a touch area 525 a is formed so as to overlap the shooting mode 525 . When each of the touch areas 521a to 525a detects that it has been pressed (touched) by the user, it outputs a detection signal indicating the pressed position (which touch area) to the system control unit . In the present embodiment, the first touch panel 52 and the second touch panel 54 may be composed of separate touch panels, or may be composed of one touch panel. When the touch panel is composed of one touch panel, the touch area P(i, j) of the touch panel corresponds to the first touch panel 52, and the touch areas 521a to 525a correspond to the second touch panel .

FIG. 7 is a functional block diagram of an image processing unit and a system control unit according to the first embodiment; As shown in FIG. 7 , the image processing section 30 includes a composition processing section 31 . The synthesizing unit 31 performs a process of synthesizing images captured multiple times at high speed. Such processing reduces noise contained in the image. That is, this processing serves the function of noise reduction.

The system control unit 70 also includes a display control unit (control unit) 71 , a change unit 72 , a selection unit 73 , and an imaging control unit (image synthesizing unit) 74 . The display control unit 71 outputs image data to the display unit 50 to perform control to display an image (still image, moving image, live view image) on the first display unit 51 . Further, the display control unit 71 performs control to display a preset menu image on the second display unit 53 . The changing unit 72 performs control to change the imaging conditions (including the accumulation conditions) according to the user's touch operation on the second touch panel 54 or automatically. Further, the selection unit 73 performs control to select an area within the image display area 510 in units of blocks according to a touch operation of the first touch panel 52 by the user. In addition, the imaging control section (image synthesizing section) 74 controls imaging according to the operation of the shutter button (operation section 55) by the user.

In the embodiments (first embodiment, second and third embodiments to be described later), the “first region” refers to an image display region 510 of the display screen 500 in accordance with a user's operation or automatically. A region in the pixel region 113A of the image sensor 100 corresponding to the region selected by 73. In addition, the “second region” refers to a region within the pixel region 113A of the image sensor 100 corresponding to the region selected automatically by the selection unit 73 according to the user's operation or automatically in the image display region 510 of the display screen 500. refers to the area. The first region and the second region may be one and the other of two divided regions, or one and the other of three or more divided regions.

Next, a shooting operation according to the first embodiment will be described. FIG. 8 is a flowchart for explaining a photographing operation executed by a system control unit according to the first embodiment; 9 to 12 are diagrams showing display examples of the display screen when a small number of blocks are set in the image display area.

In the process shown in FIG. 8, when the user operates the operation unit 55 or the like to start photographing after the electronic device 1 is powered on, the system control unit 70 starts the photographing operation (step S1). ). Note that the user performs an operation in advance to select whether to shoot a still image or to shoot a moving image.

When shooting is started, the display control unit 71 displays the live view image captured by the imaging unit 20 on the first display unit 51 (image display area 510) (step S2). In the example shown in FIG. 9, a live view image of a waterfall is displayed in the image display area 510 . When the user selects an area within the image display area 510, the user touches the setting 521 (that is, the touch area 521a) with a finger. As a result, the area within the image display area 510 becomes selectable, that is, the first touch panel 52 becomes capable of detecting the touch operation. After that, the user touches (or traces) the block B(i, j) set in the image display area 510 with a finger to select the area in the image display area 510 in units of blocks. . The first touch panel 52 outputs a detection signal corresponding to the touch area P(i, j) touched by the user to the system control section 70 .

The selection unit 73 recognizes the area selected by the user based on the detection signal from the first touch panel 52 . In the example shown in FIG. 10, an area 511 is selected by the user. This region 511 is the background region behind the waterfall (the white-lined region in FIG. 10). Specifically, the area 511 includes blocks B(3,1) to B(8,1), B(4,2) to B(8,2), B(5,3) to B(8,3 ), B(6,4) to B(8,4).

The selection unit 73 determines whether or not the user has selected an area (step S3). If it is determined that the user has selected an area, the selector 73 sets the area 511 selected by the user (step S4). Specifically, the selection unit 73 outputs an instruction signal for instructing the position of the block selected by the user to the drive unit 21 .

The changing unit 72 determines whether or not the user has selected the auto mode as the shooting mode (step S5). Here, the user touches the shooting mode 525 (that is, the touch area 525a) when setting the shooting mode. Then, the user selects a manual mode (manual mode) or an auto mode (automatic mode) from among the photographing modes. If the changing unit 72 determines that the auto mode is not selected as the shooting mode, that is, if it determines that the manual mode is selected as the shooting mode, the changing unit 72 changes the shooting conditions according to the user's operation of the second touch panel 54. (including accumulation conditions) are set (step S6).

For example, the user touches the ISO sensitivity 522 (that is, the touch area 522a) when setting the ISO sensitivity as the imaging condition. The second touch panel 54 outputs a detection signal corresponding to the touch area 522a touched by the user to the system control section 70 . The display control unit 71 displays a plurality of ISO sensitivity values next to the ISO sensitivity 522 as shown in FIG. In the example shown in FIG. 11, "100", "200", "400", "800", and "1600" are displayed as the ISO sensitivity. The changing unit 71 newly sets a touch area on the second touch panel 54 so as to overlap with each value area of the ISO sensitivity. The user touches one of the ISO sensitivity values. The second touch panel 54 outputs a detection signal corresponding to the touch area touched by the user to the system control section 70 . The changing unit 72 sets the ISO sensitivity value touched by the user. Specifically, the changing unit 72 outputs to the driving unit 21 an instruction signal that instructs the gain according to the ISO sensitivity selected by the user.

When the user sets the shutter speed (charge accumulation time) and frame rate, the same operation as for setting the ISO sensitivity is performed. That is, the user touches the shutter 523 (that is, the touch area 523a) or the frame rate 524 (that is, the touch area 524a) when setting the shutter speed or frame rate as the imaging condition. The second touch panel 54 outputs a detection signal corresponding to the touch area 523 a or 524 a touched by the user to the system control section 70 . The display control unit 71 displays a plurality of shutter speed or frame rate values next to the shutter 523 or frame rate 524 . The user touches either the shutter speed or frame rate value. The changing unit 72 sets the value of the shutter speed or frame rate touched by the user. Specifically, the changing unit 72 outputs to the driving unit 21 an instruction signal that instructs the shutter speed or frame rate selected by the user.

The drive unit 21 receives an instruction signal that instructs, in units of blocks, the area of the pixel area 113A corresponding to the area 511 selected by the user. The drive unit 21 also receives an instruction signal that instructs the imaging conditions selected by the user. In response, the drive unit 21 drives the imaging unit 20 so as to capture an image in the area of the pixel area 113A corresponding to the area 511 under the instructed imaging conditions (shutter speed, ISO sensitivity, frame rate). Note that generally, the shutter speed is set when capturing a still image, and the frame rate is set when capturing a moving image.

When the driving unit 21 changes the imaging conditions of the area of the pixel area 113A corresponding to the area 511 in this way, the image displayed in the area 511 in the image display area 510 (live view image in the area 511) changes. . For example, if the ISO sensitivity is increased, the subject can be imaged brightly even with a small amount of light. Also, the dark portions of the live view image become brighter. Higher shutter speeds also reduce the blurring of moving subjects. Also, when the frame rate is increased, the movement of the subject in the moving image becomes smoother.

In this way, by selecting an area and setting the imaging conditions for the selected area, the user can confirm the image of the selected area that changes according to the change in the imaging conditions in the live view image. . In this case, the imaging conditions selected by the user may not be suitable for proper exposure. However, the user can also recognize that overexposure or underexposure will occur due to changes in imaging conditions. Also, the user can recognize how the image changes when the imaging conditions are changed. Therefore, the user can create an image by changing the imaging conditions before photographing, and then photograph the image.

The processing of steps S1 to S6 as described above is repeatedly executed until the user half-presses the shutter (recording start button when shooting a moving image) of the operation unit 55 (step S8). In the example shown in FIG. 12, an area 512 other than the area 511 is selected in the image display area 510 . The live view image displayed in the area 512 within the image display area 510 changes according to the user's selection of the values of the ISO sensitivity 522, the shutter 523, and the frame rate 524. FIG.

A region 512 shown in FIG. 12 is the waterfall region (the region where the white line is drawn in FIG. 12). Specifically, area 512 includes blocks B(1,1) to B(2,1), B(1,2) to B(3,2), B(1,3) to B(4,3 ), B(1,4) to B(5,4), B(1,5) to B(8,5), and B(1,6) to B(8,6).

When determining that the auto mode is selected as the imaging mode (YES in step S5), the changing unit 72 automatically sets imaging conditions for the region 511 selected by the user (step S7). At this time, the changing unit 72 changes the imaging conditions based on at least one of contrast and color change of the image within the area 511 in the live view image. For example, the changing unit 72 changes the imaging conditions (ISO sensitivity, shutter speed, frame rate) so that the contrast between the bright and dark portions of the image within the region 511 is maximized. Further, the changing unit 72 changes the imaging conditions (ISO sensitivity, shutter speed, frame rate) so that the color change of the image in the area 511 becomes the most vivid. Further, the changing unit 72 changes the imaging conditions so that the contrast of the image within the area 511 is maximized and the color change of the image within the area 511 is most vivid.

The processing (steps S1 to S5, S7) when the auto mode is selected as the photographing mode is repeatedly executed until the user half-presses the shutter of the operation unit 55 (step S8). For example, in the area 512 as well, the changing unit 72 automatically sets the imaging conditions based on at least one of contrast and color change of the image in the area 512 (step S7). The live view images displayed in areas 511 and 512 in the image display area 510 change according to the values of the ISO sensitivity, shutter speed, or frame rate changed by the change unit 72 .

Although not shown in FIGS. 9 to 12, imaging conditions for each region that can be set by the user are not limited to ISO sensitivity, shutter speed, and frame rate. For example, the thinning rate described above, the number of addition rows or addition columns for adding pixel signals, the number of bits for digitization, and the like may be set as imaging conditions for each region. Also, parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate may be configured to be settable as imaging conditions for each area.

The system control unit 70 determines whether or not the shutter or the recording start button has been half-pressed (SW1 operation) (step S8). A half-press operation is used as an instruction to start shooting preparation. Although not shown in FIG. 8, the system control unit 70 executes autofocus (AF) control for automatic focus adjustment.

Next, the system control unit 70 determines whether or not the shutter or recording start button has been fully pressed (SW2 operation) (step S9). When determining that the full-press operation has been performed, the imaging control unit 74 performs imaging processing for causing the imaging device 100 to perform imaging (step S10). FIG. 13 is a flowchart illustrating an example of imaging processing according to the first embodiment. The imaging control unit 74 first determines whether or not the ISO sensitivity is set to a predetermined value or higher by the user or automatically (step S11). When the imaging control unit 74 determines that the ISO sensitivity is not set to the predetermined value or higher, it causes the imaging unit 20 to perform normal imaging (step S12). That is, a still image is acquired by one imaging as usual. At this time, imaging is performed under the imaging conditions for each region selected as described above.

On the other hand, when the imaging control unit 74 determines that the ISO sensitivity is set to the predetermined value or more, it causes the imaging unit 20 to perform continuous imaging a plurality of times (step S13). For example, the imaging control unit 74 outputs an instruction signal to the driving unit 21 to cause the imaging unit 20 to perform imaging five times at high speed. Also in this case, imaging is performed under the imaging conditions for each region selected as described above. For example, multiple times of imaging are performed during the time of the already set shutter speed.

The imaging control unit 74 also instructs the image processing unit 30 to synthesize a plurality of images that have been continuously imaged a plurality of times (step S14). In the image processing unit 30 , the synthesis processing unit 31 synthesizes a plurality of images to generate a still image based on an instruction from the imaging control unit 74 . When the ISO sensitivity is increased, noise tends to occur. However, as described above, noise is reduced by synthesizing a plurality of images captured at high speed.

In addition, in the imaging process of step S10, when capturing a moving image, the imaging control unit 74 performs normal imaging without performing the process shown in step S11 of FIG. 13 (step S12). That is, in the case of capturing a moving image, continuous capturing of a plurality of times and composition of a plurality of images shown in step S13 are not performed.

In the display examples of the display screen 500 shown in FIGS. 9 to 12, each block set in the image display area 510 is a large area block. However, blocks smaller than the blocks shown in FIGS. 9 to 12 may be set in the image display area 510. FIG. 14 to 16 are diagrams showing display examples of the display screen when a large number of blocks are set in the image display area. In the examples shown in FIGS. 14 to 16, the block is the same area as the unit group or an area containing several unit groups.

When selecting an area, the user touches the setting 521 and then traces the finger along the boundary of the area to be selected as shown in FIG. With such a selection method, the area can be selected more finely. The operation for changing the imaging conditions is the same as that described with reference to FIGS. 10 to 12. FIG. That is, as shown in FIG. 14, the user selects area 513 . This area 513 is the background area behind the waterfall (the area with white lines in FIG. 14). When the user touches the ISO sensitivity 522, multiple ISO sensitivity values appear next to the ISO sensitivity 522 as shown in FIG. The user sets the ISO sensitivity by touching one of the ISO sensitivity values. The user also sets the shutter speed and frame rate by touching the shutter 523 and frame rate 524 .

Further, after touching the setting 521, the user selects the area 514 by tracing the finger along the boundary of the area to be selected as shown in FIG. Then, in the same manner as described above, the user touches the ISO sensitivity 522 , shutter speed 523 and frame rate 524 to set the ISO sensitivity, shutter speed and frame rate in the area 514 . 9 to 12, by touching the shooting mode 525 to change the shooting mode, it is possible to set the shooting condition to be automatically changed for each area. .

As described above, in the first embodiment, the imaging unit 20 has the imaging element 100 and is capable of imaging the first area and the second area, and the first area and the second area imaged by the imaging unit 20. a control unit 71 that causes the display unit 50 to display a live view image corresponding to the area; According to such a configuration, it is possible to check the change of the image corresponding to the change of the accumulation condition for each area in the live view image. Therefore, the user can create an image by changing the imaging conditions for each area before starting to capture a still image or moving image. It can be said that such processing is performed before the image is captured.

Further, the changing unit 72 is configured to change the accumulation time of the first area and the accumulation time of the second area. Therefore, the user can edit the image by changing the accumulation time (shutter speed) for each area, and perform imaging after confirming the edited image. Also, the changing unit 72 is configured to change the frame rate of the first area and the frame rate of the second area. Therefore, the user can edit the image by changing the frame rate for each area, and perform imaging after confirming the edited image. Further, the changing unit 72 is configured to change the gain of the first area and the gain of the second area. Therefore, the user can change the gain (ISO sensitivity) for each area, edit the image, and check the edited image before shooting.

Further, the changing unit 72 is configured to change the accumulation condition for the first area and the accumulation condition for the second area based on at least one of contrast and color change of the live view image. Therefore, optimum accumulation conditions can be automatically set for each area based on at least one of contrast and color change.

It also has a selection unit 73 capable of selecting the first area and the second area of the live view image. Therefore, the user can select any area. Alternatively, the selection unit 73 can automatically select an area corresponding to the subject of the live view image and provide the selected area to the user. In addition, since the image synthesizing unit 74 is provided for capturing images of at least part of the region selected by the selecting unit 73 a plurality of times and synthesizing the images, noise can be reduced. This configuration works effectively especially when the gain is increased.

Moreover, since the selection unit 73 has the first touch panel 52 for selecting a part of the live view image, the user can select the area while viewing the live view image. Also, since the first touch panel 52 has a touch panel formed on the display unit 50, the user can select an area by a touch operation on the touch panel. Therefore, the area can be easily selected by a simple operation.

In addition, since the second touch panel 54 is provided for selecting menus (setting 521, ISO sensitivity 522, shutter 523, frame rate 524, shooting mode 525 in the operation button display area 520), the user can select imaging conditions, etc. by touching the touch panel. can be selected. Therefore, it is possible to easily select imaging conditions and the like with a simple operation. Further, since the control unit 71 displays at least one of the menu regarding the gain (ISO sensitivity 522), the menu regarding the accumulation time (shutter 523), and the menu regarding the frame rate (frame rate 524), the gain and the accumulation time are displayed. , and frame rate can be selected by a touch operation on the touch panel. In addition, since the control unit 71 displays the menu near the live view image, it is possible to change the imaging condition while confirming the change in the live view image corresponding to the change in the imaging condition. Therefore, operability by the user is improved.

Note that the electronic device 1 according to the first embodiment shown in FIG. 5 is composed of a device such as a digital camera, a smart phone, a mobile phone, a personal computer, etc., having an imaging function. Further, in the electronic device 1 according to the first embodiment shown in FIG. 5, the display unit 50 may be configured to be provided outside the electronic device. In this case, each of the system control unit 70 and the display unit 50 is provided with a communication unit that transmits and receives signals (image data, control signals, etc.) in a wired or wireless manner. Further, the image processing section 30 and the system control section 70 may be integrated. In this case, the system control unit having one CPU performs the functions of the image processing unit 30 and the system control unit 70 by executing processing based on the control program.

<Second embodiment>
In the above-described first embodiment, the selection of the area in the image display area 510 is configured to be manually performed, but in the second embodiment, the selection of the area can also be performed automatically. Further, in the second embodiment, the area and imaging conditions (including accumulation conditions) are set according to the movement of the moving subject. In addition, the second embodiment also has a function of imaging only a region selected by the user or automatically selected.

FIG. 17 is a functional block diagram of an image processing section and a system control section in the second embodiment. As shown in FIG. 17, the image processing section 30A includes a composition processing section 31 and a detection section 32. As shown in FIG. The synthesizing unit 31 has the same configuration as that described with reference to FIG. 7, so description thereof will be omitted. The detection unit 32 performs processing for detecting a moving subject (for example, a waterfall shown in FIG. 9). Also, the configuration of the system control unit 70 is the same as the configuration described with reference to FIG. 7, so description thereof will be omitted. Configurations other than the image processing unit 30 and system control unit 70 (lens unit 10, imaging unit 20, work memory 40, display unit 50, recording unit 60) are the same as those shown in FIG.

Next, a shooting operation according to the second embodiment will be described. FIG. 18 is a flowchart for explaining a photographing operation executed by a system control unit according to the second embodiment; In the process shown in FIG. 18, when the user operates the operation unit 55 or the like to start photographing after the electronic device 1 is powered on, the system control unit 70 starts the photographing operation (step S1). ).

When shooting is started, the display control unit 71 displays the live view image captured by the imaging unit 20 on the first display unit 51 (image display area 510) (step S2). The user sets the shooting mode. In this case, the shooting mode 525 (that is, the touch area 525a) is touched. Then, a manual mode (manual mode) or an auto mode (automatic mode) is selected from among the shooting modes.

The system control unit 70 determines whether or not auto has been selected as the shooting mode 525 (step S21). If it is determined that the auto mode is not selected as the shooting mode, that is, if it is determined that the manual mode is selected as the shooting mode, the selector 73 performs the same processing as steps S3 and S4 shown in FIG. . Subsequently, the changing unit 72 performs the same processing as in step S6 shown in FIG.

On the other hand, when determining that the auto mode is selected as the shooting mode, the selection unit 73 automatically sets the area and the shooting conditions based on the live view image displayed in the image display area 510 (step S22). .

FIG. 19 is a flow chart showing an example of the automatic setting process (step S22) of the area and imaging conditions. As shown in FIG. 19, the system control unit 70 instructs the image processing unit 30 to detect a moving object (step S22a). The detection unit 32 compares a plurality of pieces of image data obtained in time series from the live view image to detect a moving subject and a non-moving subject (not moving subject). The detection unit 32 then outputs the detection result to the system control unit 70 together with the image data. The selection unit 73 sets the area of the moving subject and the area of the non-moving subject based on the detection result of the detection unit 32 (step S22b).

The changing unit 72 also sets imaging conditions for the moving subject area and the non-moving subject area based on the detection result of the detecting unit 32 (step S22c). For example, for moving subject areas, the ISO sensitivity is increased and the shutter speed is increased. Also, when a moving image is being captured, the frame rate for the area of the moving subject is set higher than the frame rate for the area of the non-moving subject. For the non-moving subject area, the ISO sensitivity is set lower than the ISO sensitivity for the moving subject area, and the shutter speed is set slower than the shutter speed for the moving subject area. Also, when capturing a moving image, the frame rate for the area of the non-moving subject is set lower than the frame rate for the area of the moving subject.

Note that the automatic setting process of the region and imaging conditions shown in FIG. 19 is an example, and the region and imaging conditions may be set by other methods. For example, if the detection unit 32 does not detect a moving subject, the selection unit 73 specifies the boundary of the area based on the contrast and color change between the bright and dark areas in the live view image and sets the area. may Also, as described in the first embodiment, the changing unit 72 may change the imaging conditions based on at least one of contrast and color change of the image within the area 511 in the live view image. The processing for setting such regions and imaging conditions and the processing shown in FIG. 19 may be combined and applied.

As in the first embodiment, the system control unit 70 determines whether or not the shutter or recording start button has been half-pressed (SW1 operation) (step S8). If the half-press operation has been performed, the system control unit 70 determines whether or not the full-press operation (SW2 operation) of the shutter or recording start button has been performed (step S9). When determining that the full-press operation has been performed, the imaging control unit 74 performs imaging processing for causing the imaging element 100 to perform imaging (step S23).

FIG. 20 is a flowchart illustrating an example of imaging processing according to the second embodiment. The imaging control unit 74 determines whether or not the area imaging mode is set by the user (step S31). Here, the area imaging mode is a mode in which only an area selected by the user or an area automatically set is imaged. The user sets the area imaging mode by operating the operation unit 55 or the second touch panel 54 (for example, touch operation of the imaging mode 525) before executing the imaging process.

When the imaging control unit 74 determines that the area imaging mode is not set, imaging is performed for all areas of the image display area 510 (step S32). In this case, for example, processing similar to steps S11 to S14 in FIG. 13 is performed. On the other hand, when the imaging control unit 74 determines that the area imaging mode is set, imaging is performed only for the area selected in the image display area 510 (step S33). At this time, imaging is performed under the imaging conditions set for the selected region.

FIG. 21 is a diagram showing a display example of the display screen when only the selected area is imaged. As shown in FIG. 21, when a waterfall region 513 is selected, only that region 513 is imaged, and other regions are not imaged.

As described above, in the second embodiment, the detection unit 32 that detects a moving subject is provided, and the change unit 72 changes the accumulation conditions for the first area and the second area according to the detection result of the detection unit 32. is configured to be different from the accumulation condition of Therefore, it is possible to set imaging conditions suitable for the movement of the moving subject. The selection unit 73 is also configured to set the first area and the second area according to the detection result of the detection unit 32 . Therefore, it is possible to reliably extract the area of the moving subject. Also, in the second embodiment, the same effects as those described in the first embodiment are obtained.

In addition, the second embodiment has a function of imaging only the area selected by the user or the area automatically selected. Therefore, it is possible to image only the area that the user wants to image. According to such a configuration, the user can acquire, for example, various material images with high utility value.

Also in the second embodiment, the imaging conditions for each region that can be set by the user are not limited to ISO sensitivity, shutter speed, and frame rate. For example, the thinning rate described above, the number of addition rows or addition columns for adding pixel signals, the number of bits for digitization, and the like may be set as imaging conditions for each region. Also, parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate may be configured to be settable as imaging conditions for each area.

Further, in the second embodiment, the image processing section 30A is implemented by the CPU executing processing based on a control program.

<Third Embodiment>
In the third embodiment, the electronic device 1 in the first embodiment described above is separated into an imaging device 1A and an electronic device 1B.

FIG. 22 is a block diagram showing configurations of an imaging device and an electronic device according to the third embodiment. In the configuration shown in FIG. 22, the imaging device 1A is a device for imaging a subject. This imaging apparatus 1A includes a lens section 10, an imaging section 20, an image processing section 30, a work memory 40, an operation section 55, a recording section 60, and a first system control section 70A. In addition, the configuration of 10, the imaging unit 20, the image processing unit 30, the work memory 40, the operation unit 55, and the recording unit 60 in the imaging apparatus 1A is the same as the configuration shown in FIG. Therefore, the same configurations are denoted by the same reference numerals, and overlapping descriptions are omitted.

Further, the electronic device 1B is a device that displays images (still images, moving images, live view images). The electronic device 1B includes a display section 50 and a second system control section (control section) 70B. Note that the configuration of the display unit 50 in the electronic device 1B is the same as the configuration shown in FIG. Therefore, the same configurations are denoted by the same reference numerals, and overlapping descriptions are omitted.

The first system control section 70A has a first communication section 75A. The second system control section 70B also has a second communication section 75B. The first communication section 75A and the second communication section 75B transmit and receive signals to and from each other by wire or wirelessly. Further, the first system control section 70A has only a configuration corresponding to, for example, the imaging control section 74 among the configurations shown in FIG. The second system control unit 70B has a configuration corresponding to, for example, the display control unit 71, the change unit 72, and the selection unit 73 in the configuration shown in FIG.

In such a configuration, the first system control unit 70A transmits image data (image data processed by the image processing unit 30, image data recorded in the recording unit 60) to the second system via the first communication unit 75A. It is transmitted to the communication section 75B. The second system control unit 70B causes the display unit 50 to display the image data received by the second communication unit 75B. The second system control unit 70B also causes the second display unit 53 to display a preset menu image.

In addition, the second system control unit 70B performs control to change the imaging conditions (including accumulation conditions) according to the user's touch operation on the second touch panel 54 or automatically. In addition, the second system control unit 70B performs control to select an area within the image display area 510 in units of blocks in accordance with a user's touch operation on the first touch panel 52 or automatically. In addition, the first system control unit 70A executes imaging control according to the user's operation of the shutter button (an operation unit provided on the electronic device 1B side for instructing the start of imaging of a still image or a moving image).

The configuration shown in FIG. 7 (the display control unit 71, the change unit 72, the selection unit 73, and the imaging control unit 74) may be provided in either the first system control unit 70A or the second system control unit 70B. All the configuration shown in FIG. 7 may be provided in the first system control unit 70A or the second system control unit 70B, and part of the configuration shown in FIG. 7 may be provided in the first system control unit 70A, 7 may be provided in the second system control unit 70B.

The imaging device 1A is composed of, for example, a digital camera, a smart phone, a mobile phone, a personal computer, etc., each having an imaging function and a communication function, and the electronic device 1B is, for example, a smart phone, a mobile phone, a portable personal computer, etc. It consists of portable terminals such as computers.

The image processing unit 30 shown in FIG. 22 is implemented by the CPU executing processing based on a control program. Also, the first system control unit 70A shown in FIG. 22 is implemented by the CPU executing processing based on the control program. Also, the second system control unit 70B shown in FIG. 22 is implemented by the CPU executing processing based on the control program.

As described above, in the third embodiment, the display unit 50 has the image sensor 100 and can display a live view image captured by the image capturing unit 20 that captures images of the first region and the second region; Control to display on the display unit 50 a menu for setting imaging conditions of the live view image when the live view image corresponding to the first region and the second region captured by the unit 20 is displayed on the display unit 50. and a portion 75B. According to such a configuration, in addition to the effects described in the first embodiment, the live view image captured by the imaging device 1A is edited using a mobile terminal such as a smartphone, and the edited live view image is confirmed. Imaging can be performed on the

In the configuration shown in FIG. 22, the image processing section 30 and the first system control section 70A may be integrated. In this case, a system control unit having one CPU performs processing based on a control program to perform the function of the image processing unit 30 and the function of the first system control unit 70A.

<Fourth Embodiment>
In the operation button display area 520 shown in FIGS. 6, 9 to 12, 14 to 16, and 21, buttons 521 to 525 are arranged in a predetermined order. That is, setting 521 is placed at the top, ISO sensitivity 522 is placed second from the top (i.e., below setting 521), shutter 523 is placed third from the top (i.e., below ISO speed 522), The frame rate 524 was placed fourth from the top (that is, below the shutter 523), and the shooting mode 525 was placed at the bottom. On the other hand, in the fourth embodiment, the display control unit 71 changes the arrangement of the buttons 521 to 525 according to the subject in the area and displays them. Further, in the fourth embodiment, the display control unit 71 displays a button with a high priority in a display mode different from the display mode of other buttons.

FIG. 23 is a diagram showing a display example of the display screen in the fourth embodiment. In the example shown in FIG. 23, setting 521 is placed at the top, shutter 523 is placed second from the top (i.e., below setting 521), and frame rate 524 is placed third from the top (i.e., below shutter 523). , the ISO sensitivity 522 is the fourth from the top (that is, below the frame rate 524), and the shooting mode 525 is located at the bottom. Also, as shown in FIG. 23, the shutter 523 has a larger area than the other buttons (setting 521, ISO sensitivity 522, frame rate 524, shooting mode 525).

Also, in the second touch panel 54, a touch area 521a is formed so as to overlap the setting 521. As shown in FIG. Also, a touch area 523 a is formed so as to overlap the shutter 523 . Also, a touch area 524 a is formed so as to overlap the frame rate 524 . Also, the touch area 522 a is formed so as to overlap the ISO sensitivity 522 . Also, a touch area 525 a is formed so as to overlap the shooting mode 525 . It should be noted that the touch area 523a is larger than the other button areas in accordance with the size of the shutter 523 area.

Next, a process of changing the arrangement of the buttons 512 to 525 by the image processing section 30 and the system control section 70 will be described. When executing step S6 in FIGS. 8 and 18, the system control unit 70 instructs the image processing unit 30 to detect a moving subject. The detection unit 32 detects a moving subject and a non-moving subject by comparing a plurality of pieces of image data obtained in time series from the live view image. The detection unit 32 then outputs the detection result to the system control unit 70 together with the image data.

The display control unit 71 determines whether or not the region selected by the selection unit 73 includes a moving subject based on the detection result of the detection unit 32 . When the display control unit 71 determines that the moving subject is included in the area selected by the selection unit 73, as shown in FIG. to the second position from . In addition, the display control unit 71 displays the shutter 523 larger than the other buttons.

Changing the shutter speed changes how the moving subject is captured. For example, when the moving subject is a waterfall, by increasing (fastening) the shutter speed, an image appears as if the flow of water in the waterfall has stopped momentarily. On the other hand, if the shutter speed is set low (slow), an image in which the flowing water of the waterfall looks like threads is created. As described above, for a moving subject, the shutter speed has a greater effect on how the subject is captured than other imaging conditions. Therefore, the display control unit 71 determines that the priority of the shutter speed is higher than the other imaging conditions when the moving subject is included in the area. Then, the display control unit 71 moves the position of the shutter 523 upward and displays the shutter 523 as a button with a large area. As a result, the buttons with high priority are arranged at positions that are easy for the user to operate, and the operability for the user is improved. Also, a button with a high priority (that is, an imaging condition) becomes more conspicuous than other buttons (that is, an imaging condition), and the user can be prompted to operate the button with a high priority.

In the case of moving images, the frame rate greatly affects the smoothness of motion of a moving subject. Therefore, the display control unit 71 determines that the priority of the frame rate is higher than other imaging conditions when capturing a moving image of a moving subject. Then, the display control unit 71 moves the position of the frame rate 524 upward and displays the frame rate 524 as a button in a large area.

In the example described above, the display control unit 71 changes the layout and display mode (size) of the buttons depending on whether the subject in the area is a moving subject, but the configuration is not limited to this. . For example, when executing step S6 in FIGS. 8 and 18, the system control unit 70 instructs the image processing unit 30 to detect the brightness of the image. The image processing unit 30 detects the brightness of the image within the area based on the live view image. The image processing unit 30 then outputs the detection result to the system control unit 70 together with the image data.

The display control unit 71 determines whether the brightness of the image in the region selected by the selection unit 73 is within a predetermined range based on the brightness detection result by the image processing unit 30 . When the display control unit 71 determines that the brightness of the image in the area is not within the predetermined range, it displays the ISO sensitivity 522, which is a button for setting the ISO sensitivity, at the second position from the top. Also, the display control unit 71 displays the ISO sensitivity 522 larger than the other buttons.

By changing the ISO sensitivity, the dark portions of the live view image become brighter and the bright portions become darker. If the lightness in the area is not within the predetermined range, that is, if the image in the area is too bright or too dark, the ISO sensitivity can be changed to bring the exposure closer to the optimal exposure. Therefore, when the image in the area does not have a value within the predetermined range, the display control unit 71 determines that the priority of the ISO sensitivity is higher than the other imaging conditions. Then, the display control unit 71 moves the position of the ISO sensitivity 522 upward and displays the ISO sensitivity 522 as a button in a large area. As a result, the buttons with high priority are arranged at positions that are easy for the user to operate, and the operability for the user is improved. Also, a button with a high priority (that is, an imaging condition) becomes more conspicuous than other buttons (that is, an imaging condition), and the user can be prompted to operate the button with a high priority.

In addition to enlarging the buttons, a display mode such as changing the color of the buttons or blinking the buttons may be used as a display mode to make the buttons with high priority stand out. Also, before a button with a high priority is pressed by the user, a plurality of imaging condition values may be displayed next to the button so that the button with the high priority is conspicuous. Alternatively, the configuration may be such that only the layout of the buttons is changed, or the configuration that only the display mode of the buttons is changed.

Further, the layout of the buttons may be changed and the display mode of the buttons may be changed according to the order in which the user operates the buttons. For example, the user selects a region after pressing Settings 521 . Next, the user presses shooting mode 525 to select manual mode as the shooting mode. After that, the user changes the imaging conditions for each area. In such a case, the display control unit 71 arranges the setting 521 at the top and increases the setting 521 . Next, the display control unit 71 arranges the photographing mode 525 at the top and enlarges the photographing mode 525 . After that, the display control unit 71 arranges any one of the ISO sensitivity 522, the shutter 523, and the frame rate 524 at the top according to the subject (image) for each area, and furthermore, presses the top button. Enlarge. In this way, by changing the layout of the buttons and the display mode in accordance with the user's operation procedure, the convenience of the user's operation is improved.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various modifications and improvements can be made to the above embodiments without departing from the scope of the present invention. Also, one or more of the requirements described in the above embodiments may be omitted. Such modifications, improvements, and omissions are also included in the technical scope of the present invention. Moreover, it is also possible to appropriately combine the configurations of the above-described embodiments and modifications and apply them.

For example, in the above-described first and second embodiments, if the electronic device 1 includes the imaging unit 20 and the system control unit 70 including the control unit 71 and the change unit 72, the lens unit 10 and the recording The unit 60 and the like may not be provided. That is, these configurations may be configured separately from the electronic device 1 . Further, in the above-described third embodiment, the lens unit 10, the recording unit 60, and the like may be configured separately in the imaging device 1A as well.

Further, in each of the above-described embodiments, the arrangement of the color filters 102 is the Bayer arrangement, but an arrangement other than this arrangement may be used. Also, the number of pixels forming the unit group 131 should include at least one pixel. Also, it is sufficient that each block includes at least one pixel. Therefore, it is possible to perform imaging under different imaging conditions for each pixel.

Further, in each of the above-described embodiments, the drive unit 21 may be partially or wholly mounted on the imaging chip 113 , or may be partially or wholly mounted on the signal processing chip 111 . Also, part of the configuration of the image processing unit 30 may be mounted on the imaging chip 113 or the signal processing chip 111 . Also, part of the configuration of the system control unit 70 may be mounted on the imaging chip 113 or the signal processing chip 111 .

In each of the above-described embodiments, the ISO sensitivity, shutter speed, and frame rate are all changeable as imaging conditions, but at least one of them may be changeable. Further, when the imaging conditions are automatically set, the best mode imaging conditions that provide the optimum exposure may be presented to the user. You may make it present to a user.

Further, although the operation button display area 520 is provided near the image display area 510 in each of the above-described embodiments, it may be provided at a position other than the image display area 510 . Also, the operation button display area 520 may be arranged at a position overlapping the image display area 510 . Alternatively, the user may operate the operation unit 55 instead of performing the touch operation on the second touch panel 54 .

Further, in each of the above-described embodiments, the user operates the first touch panel 52 to select an area, and operates the second touch panel 54 to set imaging conditions and the like. However, the configuration is not limited to such a configuration, and the configuration may be such that the user operates the operation unit 55 to select an area and set imaging conditions and the like.

Further, in the above-described first embodiment, the user manually sets the area and selects whether the imaging condition should be set manually or automatically by setting the imaging mode. Further, in the second embodiment described above, the user selects whether to set the area and imaging conditions manually or automatically by setting the imaging mode. However, the user may set all the areas and imaging conditions only manually. Alternatively, the area may be set automatically, and whether the imaging condition should be set manually or automatically may be selected by setting the imaging mode.

Further, in each of the above-described embodiments, the case where the size of the block area is set in advance has been described, but the user may set the size of the block area. Further, in the above-described first embodiment, when the number of blocks set in the image display area 510 is small (FIGS. 9 to 12), and when the number of blocks set in the image display area 510 is large ( 14 to 16) are exemplified. However, the system control unit 70 (selection unit 73) may automatically set large area blocks and small area blocks. For example, the system control unit 70 (selection unit 73) automatically recognizes the boundary of the area. Next, the system control unit 70 (selection unit 73) sets the small area block within the large area block when the boundary of the area crosses the large area block. Then, the system control unit 70 sets the area along the boundary of the area in block units of the small area.

Further, in each of the above-described embodiments, it is assumed that the imaging condition for the live view image displayed on the first display unit 51 and the imaging condition for imaging in response to the shutter operation are the same conditions. . This is so that the image that is actually captured can be confirmed in the live view image. However, the imaging conditions for the live view image and the imaging conditions for the actually captured image may be different. In this case, the imaging conditions are changed to such an extent that the user can recognize the image that is actually captured based on the live view image.

DESCRIPTION OF SYMBOLS 1, 1B... Electronic equipment 1A... Imaging device 20... Imaging part 30, 30A... Image processing part 31... Combining process part 32... Detection part 50... Display part 51... First display part 52... First touch panel (selection unit, first operation unit) 53 Second display unit 54 Second touch panel (second operation unit) 70 System control unit 70A First system control unit 70B Second System control unit (control unit) 71 Display control unit (control unit) 72 Change unit 73 Selection unit 74 Imaging control unit (image synthesizing unit) 100 Imaging device

Claims (15)

an imaging device having a plurality of photoelectric conversion units arranged side by side in the row direction and the column direction and converting light into electric charge;
The number of times the charges converted by the first photoelectric conversion unit among the plurality of photoelectric conversion units are accumulated is set to the first photoelectric conversion unit arranged at the position in the row direction from the first photoelectric conversion unit among the plurality of photoelectric conversion units. 2 an operation unit that receives an operation for setting the number of times of accumulation of charges converted by the photoelectric conversion unit to be different from the number of times of accumulation;
a control unit that controls the number of times the charge converted by the first photoelectric conversion unit is accumulated and the number of times the charge converted by the second photoelectric conversion unit is accumulated, based on the operation received by the operation unit;
A first signal based on the charge of the first photoelectric conversion unit whose charge accumulation count is controlled by the control unit, and a second signal based on the charge of the second photoelectric conversion unit whose charge accumulation count is controlled by the control unit. and a display on which an image generated using the signal is displayed. In the electronic device according to claim 1,
The operation unit is an electronic device formed on a display screen of the display unit. In the electronic device according to claim 2,
The operation unit is an electronic device having a touch panel that detects a touch operation on the display screen. In the electronic device according to any one of claims 1 to 3,
The display unit is an electronic device on which an image used for the operation is displayed. In the electronic device according to any one of claims 1 to 4,
The imaging element is
a first storage unit that stores the first signal;
a second storage unit that stores the second signal;
The display section is an electronic device on which an image generated using the first signal stored in the first storage section and the second signal stored in the second storage section is displayed. In the electronic device according to claim 5,
The imaging element has a plurality of semiconductor substrates,
The electronic device, wherein the first photoelectric conversion unit and the second photoelectric conversion unit are arranged on a semiconductor substrate different from that of the first storage unit and the second storage unit. In the electronic device according to claim 5,
The imaging device includes an arithmetic circuit that performs arithmetic processing on at least one of the first signal stored in the first storage unit and the second signal stored in the second storage unit. electronic equipment. In the electronic device according to claim 7,
The imaging element has a plurality of semiconductor substrates,
the first photoelectric conversion unit and the second photoelectric conversion unit are arranged on a semiconductor substrate different from that of the first storage unit and the second storage unit;
The electronic device, wherein the arithmetic circuit is arranged on a semiconductor substrate different from that of the first storage section and the second storage section. In the electronic device according to claim 7,
The imaging element has a plurality of semiconductor substrates,
the first photoelectric conversion unit and the second photoelectric conversion unit are arranged on a semiconductor substrate different from that of the first storage unit and the second storage unit;
The electronic device, wherein the arithmetic circuit is arranged on the same semiconductor substrate as the first storage section and the second storage section. In the electronic device according to any one of claims 1 to 9,
The imaging element is
a first transfer unit that transfers the charge converted by the first photoelectric conversion unit;
a second transfer unit that transfers the charge converted by the second photoelectric conversion unit;
a first control line connected to the first transfer unit and outputting a first control signal for controlling the first transfer unit;
and a second control line connected to the second transfer section and outputting a second control signal for controlling the second transfer section. In the electronic device according to claim 10,
The control unit outputs the first control signal such that an accumulation time of charges converted by the first photoelectric conversion unit is different from an accumulation time of charges converted by the second photoelectric conversion unit. An electronic device that controls timing to output the first control line and timing to output the second control signal to the second control line. In the electronic device according to claim 10 or claim 11,
The imaging element is
a first reset unit that resets the potential of a first floating diffusion to which the charge accumulated in the first photoelectric conversion unit is transferred by the first transfer unit;
a second reset unit that resets the potential of a second floating diffusion to which the charge accumulated in the second photoelectric conversion unit is transferred by the second transfer unit;
a third control line connected to the first reset section and outputting a third control signal for controlling the first reset section;
and a fourth control line connected to the second reset section and outputting a fourth control signal for controlling the second reset section. In the electronic device according to claim 12,
The control section outputs the third control signal such that the accumulation time of the charges converted by the first photoelectric conversion section is different from the accumulation time of the charges converted by the second photoelectric conversion section. An electronic device that controls timing to output the third control line and timing to output the fourth control signal to the fourth control line. In the electronic device according to claims 1 to 9,
The imaging element is
a first reset unit that resets the potential of a first floating diffusion to which the charge accumulated in the first photoelectric conversion unit is transferred;
a second reset unit that resets the potential of a second floating diffusion to which charges accumulated in the second photoelectric conversion unit are transferred;
a first control line connected to the first reset unit and outputting a first control signal for controlling the first reset unit;
and a second control line connected to the second reset section and outputting a second control signal for controlling the second reset section. In the electronic device according to claim 14,
The control unit outputs the first control signal such that an accumulation time of charges converted by the first photoelectric conversion unit is different from an accumulation time of charges converted by the second photoelectric conversion unit. An electronic device that controls timing to output the first control line and timing to output the second control signal to the second control line.

Images