JP6885374B2 - Electronics - Google Patents

Description

The present invention relates to electronic devices.

An electronic device including an image pickup device in which a back-illuminated image pickup chip and a signal processing chip are laminated (hereinafter, this image pickup device is referred to as a stacked image pickup device) has been proposed (see, for example, Patent Document 1). In the stacked image sensor, the back-illuminated image sensor and the signal processing chip are laminated so as to be connected via microbumps for each block in which a plurality of pixels are grouped.

Japanese Unexamined Patent Publication No. 2006-49361

However, in the conventional electronic device equipped with a stacked image sensor, there are not many proposals for acquiring an image by taking an image for each of a plurality of blocks, and the usability of the electronic device provided with the stacked image sensor is not sufficient.

An object of the present invention is to suppress power consumption when the display magnification of an image is changed.

According to the aspect of the present invention, a display unit having a first display area in which the first image is displayed and a second display area in which the second image is displayed, and first image data showing the first image are generated. An imaging unit having a first imaging region from which the first pixel signal for the purpose is read and a second imaging region from which the second image data for generating the second image data indicating the second image is read, and an optical zoom. , The magnification changing unit that changes the display magnification of the first image and the second image displayed on the display unit, and when the display magnification is changed by the magnification changing unit, the first imaging region and the second imaging region are different from each other. An electronic device including a control unit that controls an image pickup unit so as to read out a first pixel signal and a second pixel signal under charge accumulation conditions is provided.
According to the aspect of the present invention, a display unit having a first display area in which the first image is displayed and a second display area in which the second image is displayed, and first image data showing the first image are generated. An image pickup unit having a first image pickup region from which the first pixel signal for the purpose is read and a second image pickup region from which the second image data for generating the second image data indicating the second image is read out, and a display unit. When the display magnification is changed by the magnification changing unit that changes the display magnification of the displayed first image and the second image and the magnification changing unit, the first imaging region and the second imaging region are subjected to different charge accumulation conditions. An electronic device including a control unit that controls an image pickup unit so as to read out a first pixel signal and a second pixel signal is provided.
According to the aspect of the present invention, the image sensor having an image pickup region for capturing the subject, the first image of the subject captured in the first imaging region in the imaging region, and the first imaging region in the imaging region are different. In the display unit where the second image of the subject captured in the second image pickup region is displayed, the magnification changing unit for changing the display magnification of at least one of the first image and the second image, and the magnification When the display magnification is changed by the changing unit, the charge accumulation condition of at least one of the imaging regions is changed in the first imaging region and the second imaging region, and the image sensor is controlled so as to image the subject. An electronic device including a control unit is provided.
According to the aspect of the present invention, a display unit having a first display area in which the first image is displayed and a second display area in which the second image is displayed, and first image data showing the first image are generated. An imaging unit having a first imaging region and a second imaging region for generating second image data indicating a second image, and a magnification changing unit for changing the display magnification of the first image and the second image displayed on the display unit. An electronic device including an image pickup control unit that changes charge accumulation conditions in the first image pickup region and the second image pickup region when the display magnification is changed by the magnification change unit is provided.
According to the aspect of the present invention, the image sensor having an image pickup region for photographing the subject, the first image of the subject captured in the first image pickup region in the image pickup region, and the first image pickup region in the image pickup region are different. In the display unit where the second image of the subject captured in the second image pickup region is displayed, the magnification changing unit for changing the display magnification of at least one of the first image and the second image, and the magnification When the display magnification is changed by the changing unit, an electronic device including an imaging control unit that changes the charge accumulation condition of at least one of the first imaging region and the second imaging region is provided. ..
According to the first aspect of the present invention, a display unit having a first display area on which the first image is displayed and a second display area on which the second image is displayed, and first image data showing the first image are displayed. An imaging unit having a first imaging region to be generated and a second imaging region for generating second image data indicating a second image, and a magnification for changing the display magnification of the first image and the second image displayed on the display unit. An electronic device including a changing unit and an imaging control unit that changes a charge accumulation condition or a reading condition in a first imaging region and a second imaging region when the display magnification is changed by the magnification changing unit is provided.

According to the second aspect of the present invention, a display unit having a first display area on which the first image is displayed and a second display area on which the second image is displayed, and first image data showing the first image are displayed. A control method for an electronic device including an imaging unit having a first imaging region to be generated and a second imaging region for generating second image data indicating a second image, and a first image displayed on the display unit. And control of electronic devices including changing the display magnification of the second image and changing the charge accumulation condition or the read condition in the first imaging region and the second imaging region when the display magnification is changed. A method is provided.

According to the third aspect of the present invention, a display unit having a first display area on which the first image is displayed and a second display area on which the second image is displayed, and first image data showing the first image are displayed. A first image and a first image displayed on a control device of an electronic device including an imaging unit having a first imaging region to be generated and a second imaging region for generating second image data indicating a second image. 2 Magnification change processing for changing the display magnification of an image, and imaging control processing for changing the charge accumulation condition or reading condition in the first imaging region and the second imaging region when the display magnification is changed by the magnification change processing. A control program is provided to execute.

According to the aspect of the present invention, power consumption can be suppressed when the display magnification of the image is changed.

It is sectional drawing which shows the schematic structure of the digital camera as an electronic device. It is sectional drawing of the laminated type image sensor. It is a figure explaining the pixel arrangement and the unit group of an image pickup chip. It is a circuit diagram corresponding to the unit group of the image pickup chip. It is a block diagram which shows the functional structure of an image sensor. It is a block diagram which shows the structure of the digital camera as an electronic device. It is a functional block diagram of the image processing unit and the system control unit shown in FIG. It is a figure which shows the display example at the time of performing the optical zoom. It is a figure which shows the display example at the time of performing electronic zoom. It is a flowchart which shows the imaging process which concerns on 1st Embodiment. It is a flowchart which shows the control corresponding to the optical zoom shown in FIG. It is a figure which shows the central region and the peripheral region in the imaging region. It is a flowchart which shows the control corresponding to the electronic zoom shown in FIG. It is a figure which shows the enlargement target area in the imaging area. It is a flowchart which shows the imaging process which concerns on 2nd Embodiment. It is a flowchart which shows the control corresponding to the optical zoom shown in FIG. It is a figure which shows the main subject area and the feature point area set in the image pickup area at the time of an optical zoom. It is a flowchart which shows the control corresponding to the electronic zoom shown in FIG. It is a figure which shows the main subject area and the feature point area set in the image pickup area at the time of electronic zoom. It is a flowchart which shows the control corresponding to the electronic zoom which concerns on 3rd Embodiment. It is a figure which shows the display example at the time of performing the electronic zoom in 3rd Embodiment. It is a figure which shows the other display example at the time of performing the electronic zoom in 3rd Embodiment. It is a block diagram which shows the structure of the image pickup apparatus and the electronic apparatus which concerns on 4th Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to this. Further, in the drawings, in order to explain the embodiment, the scale may be changed as appropriate, such as drawing a part in a large or emphasized manner. In each of the following embodiments, an interchangeable lens digital camera will be described as an example of an electronic device.

<First Embodiment>
FIG. 1 is a cross-sectional view showing a schematic configuration of a digital camera as an electronic device. The digital camera (electronic device) 1 of the present embodiment has a lens unit 10 as an interchangeable lens and a camera body 2 as a camera body. The lens portion 10 is attached to the camera body 2 via the mount portion 80. A lens unit 10 having various photographing optical systems can be attached to the camera body 2 via a mount unit 80. The photographing optical system (lens 11a, zooming lens 11b, focusing lens 11c) in the lens unit 10 is referred to as a photographing optical system 11.

The lens unit 10 includes a lens 11a, a zooming lens 11b, a focusing lens 11c, an aperture 11d, a lens drive control device 15, and the like. The lens drive control device 15 includes a CPU (Central Processing Unit), a memory, a drive control circuit, and the like. The lens drive control device 15 transmits lens information regarding the characteristics (lens size, etc.) of the lens 11a and controls information (zooming lens) by communicating with the system control unit 70 on the camera body 2 side via the electrical contact 80A. The movement amount of 11b, the movement amount of the focusing lens 11c, the aperture value of the aperture 11d, etc.) are received.

Further, the lens drive control device 15 causes the drive control circuit to execute drive control of the focusing lens 11c in order to adjust the focus of the photographing optical system 11 based on the control information transmitted from the system control unit 70. Further, the lens drive control device 15 causes the drive control circuit to execute the drive control of the zooming lens 11b in order to perform the zooming adjustment based on the control information transmitted from the system control unit 70. The diaphragm 11d forms an aperture having a variable aperture diameter at the center of the optical axis for adjusting the amount of light and the amount of blur. The lens drive control device 15 causes the drive control circuit to execute the drive control of the diaphragm 11d in order to adjust the aperture diameter of the diaphragm 11d based on the control information transmitted from the system control unit 70.

When the drive control circuit executes the drive control of the zooming lens 11b, the focal length changes without changing the focal position, and the display magnification of the image displayed on the display unit 50 is changed (enlarged or reduced). To. Changing the display magnification of an image by moving the zooming lens 11b in this way is called optical zoom.

The camera body 2 includes an image pickup unit 20 including a stacked image sensor, an image processing unit 30, a display unit 50, a storage unit 60, a system control unit 70, and the like. The light flux passing through the lens unit 10 forms a subject image on the light receiving surface of the image pickup element of the image pickup unit 20. This subject image is photoelectrically converted by the image sensor, and the pixel signal of each pixel of the image sensor is sent to the image processing unit 30. The image processing unit 30 performs various image processing on the RAW data composed of the pixel signals of each pixel to generate the image data. The display unit 50 displays the image data generated by the image processing unit 30. The storage unit 60 stores the image data generated by the image processing unit 30.

The "image data" is data that constitutes an image (still image, moving image, live view image) captured by the imaging unit 20, and is data before image processing is performed by the image processing unit 30 (that is, RAW data). ) And the data after the image processing is performed. Further, the "RAW data" refers to image data before image processing is performed in the image processing unit 30. Further, "image data" may be referred to as "image signal".

The live view image is an image in which the image data generated by the image processing unit 30 is sequentially output to the display unit 50 and displayed on the display unit 50. The live view image is used for the user to confirm the image of the subject captured by the imaging unit 20. The live view image is also called a through image or a preview image.

The system control unit 70 controls the overall processing and operation of the digital camera 1. The system control unit 70 has a CPU (Central Processing Unit). The details of the processing and operation of the system control unit 70 will be described later (see FIGS. 6 and 7). The details of the configuration inside the camera body 2 will also be described later (see FIG. 6).

FIG. 2 is a cross-sectional view of the stacked image sensor. The stacked image sensor 100 is an image sensor provided in the image pickup unit 20 shown in FIG. The stacked image sensor 100 is described in Japanese Patent Application No. 2012-139026 previously filed by the applicant of the present application. The image sensor 100 includes an image pickup chip 113 that outputs a pixel signal corresponding to incident light, a signal processing chip 111 that processes the pixel signal, and a memory chip 112 that stores the pixel signal. The imaging chip 113, the signal processing chip 111, and the memory chip 112 are laminated and electrically connected to each other by a conductive bump 109 such as Cu.

As shown in the figure, the incident light is mainly incident in the Z-axis plus direction indicated by the white arrow. In the present embodiment, in the image pickup chip 113, the surface on the side where the incident light is incident is referred to as the back surface. Further, as shown in the coordinate axes, the left direction of the paper surface orthogonal to the Z axis is defined as the X axis plus direction, and the Z axis and the front direction of the paper surface orthogonal to the X axis are defined as the Y axis plus direction. In some subsequent figures, the coordinate axes are displayed so that the orientation of each figure can be understood with reference to the coordinate axes of FIG.

An example of the image pickup chip 113 is a back-illuminated MOS image sensor. The PD layer 106 is arranged on the back surface side of the wiring layer 108. The PD layer 106 has a plurality of photodiodes (Photodiode; hereinafter referred to as PD) 104 arranged two-dimensionally and accumulating charges according to incident light, and a transistor 105 provided corresponding to the PD 104. ..

A color filter 102 is provided on the incident side of the incident light in the PD layer 106 via the passivation film 103. The color filter 102 is a filter that allows visible light to pass through a specific wavelength region. The color filter 102 has a plurality of types that transmit different wavelength regions from each other, and has a specific arrangement corresponding to each of the PD 104. The arrangement of the color filters 102 will be described later. A set of a color filter 102, a PD 104, and a transistor 105 forms one pixel.

A microlens 101 is provided on the incident side of the incident light in the color filter 102 corresponding to each pixel. The microlens 101 collects incident light toward the corresponding PD 104.

The wiring layer 108 has a wiring 107 that transmits a pixel signal from the PD layer 106 to the signal processing chip 111. The wiring 107 may have multiple layers, and may be provided with a passive element and an active element. A plurality of bumps 109 are arranged on the surface of the wiring layer 108. These plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the facing surfaces of the signal processing chip 111. Then, when the imaging chip 113 and the signal processing chip 111 are pressurized or the like, the aligned bumps 109 are joined to each other and electrically connected.

Similarly, a plurality of bumps 109 are arranged on the surfaces of the signal processing chip 111 and the memory chip 112 facing each other. These bumps 109 are aligned with each other. Then, when the signal processing chip 111 and the memory chip 112 are pressurized or the like, the aligned bumps 109 are joined to each other and electrically connected.

The bonding between the bumps 109 is not limited to Cu bump bonding by solid phase diffusion, and micro bump bonding by solder melting may be adopted. Further, for example, about one bump 109 may be provided for one unit group described later. Therefore, the size of the bump 109 may be larger than the pitch of the PD 104. Further, in the peripheral region other than the pixel region (pixel region 113A shown in FIG. 3) in which the pixels are arranged, a bump larger than the bump 109 corresponding to the pixel region may be provided together.

The signal processing chip 111 has a TSV (Through-Silicon Via; through silicon via) 110 that connects circuits provided on the front surface and the back surface to each other. The TSV 110 is provided in the peripheral area. Further, the TSV 110 may be provided in the peripheral area of the image pickup chip 113 or in the memory chip 112.

FIG. 3 is a diagram illustrating a pixel array and a unit group of the imaging chip. FIG. 3 shows, in particular, a state in which the imaging chip 113 is observed from the back surface side. The region in which the pixels are arranged in the imaging chip 113 is referred to as a pixel region (imaging region) 113A. More than 20 million pixels are arranged in a matrix in the pixel area 113A. In the example shown in FIG. 3, 16 pixels of adjacent 4 pixels × 4 pixels form one unit group 131. The grid lines in FIG. 3 show the concept that adjacent pixels are grouped to form a unit group 131. The number of pixels forming the unit group 131 is not limited to this, and may be about 1000, for example, 32 pixels × 64 pixels, more or less.

As shown in the partially enlarged view of the pixel region 113A, the unit group 131 includes four so-called Bayer arrays including four pixels of green pixels Gb, Gr, blue pixels B, and red pixels R in the vertical and horizontal directions. The green pixel is a pixel having a green filter as the color filter 102, and receives light in the green wavelength band among the incident light. Similarly, the blue pixel is a pixel having a blue filter as the color filter 102 and receives light in the blue wavelength band. The red pixel is a pixel having a red filter as the color filter 102 and receives light in the red wavelength band.

FIG. 4 is a circuit diagram corresponding to a unit group of imaging chips. In FIG. 4, a rectangle surrounded by a dotted line typically represents a circuit corresponding to one pixel. At least a part of each transistor described below corresponds to the transistor 105 of FIG.

As described above, the unit group 131 is formed from 16 pixels. The 16 PD 104s corresponding to each pixel are connected to the transfer transistor 302, respectively. The gate of each transfer transistor 302 is connected to the TX wiring 307 to which the transfer pulse is supplied. In this embodiment, the TX wiring 307 is commonly connected to the 16 transfer transistors 302.

The drain of each transfer transistor 302 is connected to the source of each corresponding reset transistor 303, and the so-called floating diffusion FD (charge detection unit) between the drain of the transfer transistor 302 and the source of each reset transistor 303 is an amplification transistor 304. Connected to the gate. The drain of each reset transistor 303 is connected to the Vdd wiring 310 to which the power supply voltage is supplied. The gate of each reset transistor 303 is connected to the reset wiring 306 to which the reset pulse is supplied. In the present embodiment, the reset wiring 306 is commonly connected to the 16 reset transistors 303.

The drain of each amplification transistor 304 is connected to the Vdd wiring 310 to which the power supply voltage is supplied. Also, the source of each amplification transistor 304 is connected to the drain of each corresponding selection transistor 305. The gate of each selection transistor 305 is connected to a decoder wiring 308 to which a selection pulse is supplied. In this embodiment, the decoder wiring 308 is provided independently for each of the 16 selection transistors 305. Then, the source of each selection transistor 305 is connected to the common output wiring 309. The load current source 311 supplies current to the output wiring 309. That is, the output wiring 309 for the selection transistor 305 is formed by the source follower. The load current source 311 may be provided on the imaging chip 113 side or the signal processing chip 111 side.

Here, the flow from the start of charge accumulation to the pixel output after the end of accumulation will be described. A reset pulse is applied to the reset transistor 303 through the reset wiring 306. At the same time, a transfer pulse is applied to the transfer transistor 302 through the TX wiring 307. As a result, the potentials of the PD 104 and the floating diffusion FD are reset.

When the application of the transfer pulse is released, the PD 104 converts the received incident light into an electric charge and accumulates it. After that, when the transfer pulse is applied again in the state where the reset pulse is not applied, the charge accumulated in the PD 104 is transferred to the floating diffusion FD. As a result, the potential of the floating diffusion FD changes from the reset potential to the signal potential after charge accumulation. Then, when the selection pulse is applied to the selection transistor 305 through the decoder wiring 308, the fluctuation of the signal potential of the floating diffusion FD is transmitted to the output wiring 309 via the amplification transistor 304 and the selection transistor 305. By the operation of such a circuit, the pixel signal corresponding to the reset potential and the signal potential is output from the unit pixel to the output wiring 309.

As shown in FIG. 4, in the present embodiment, the reset wiring 306 and the TX wiring 307 are common to the 16 pixels forming the unit group 131. That is, the reset pulse and the transfer pulse are applied to all 16 pixels at the same time. Therefore, all the pixels forming the unit group 131 start the charge accumulation at the same timing and end the charge accumulation at the same timing. However, the pixel signal corresponding to the accumulated charge is selectively output to the output wiring 309 by sequentially applying the selection pulse to each selection transistor 305. Further, the reset wiring 306, the TX wiring 307, and the output wiring 309 are separately provided for each unit group 131.

By configuring the circuit with the unit group 131 as a reference in this way, the charge accumulation time can be controlled for each unit group 131. In other words, pixel signals with different charge accumulation times can be output between the unit groups 131. Furthermore, while one unit group 131 is made to perform one charge accumulation, the other unit group 131 is made to repeat the charge accumulation many times to output a pixel signal each time. It is also possible to output each frame for moving images at different frame rates among the unit groups 131.

FIG. 5 is a block diagram showing a functional configuration of the image sensor. The analog multiplexer 411 sequentially selects 16 PD 104s forming a unit group 131. Then, the multiplexer 411 outputs each pixel signal of the 16 PD 104s to the output wiring 309 provided corresponding to the unit group 131. The multiplexer 411 is formed on the imaging chip 113 together with the PD 104.

The pixel signal of the analog signal output via the multiplexer 411 is amplified by the amplifier 412 formed on the signal processing chip 111. Then, the pixel signal amplified by the amplifier 412 is generated by a signal processing circuit 413 formed on the signal processing chip 111, which performs correlated double sampling (CDS) and analog / digital (Analog / Digital) conversion. Along with the signal processing of correlated double sampling, A / D conversion (conversion from analog signal to digital signal) is performed. The noise of the pixel signal is reduced by performing the signal processing of the correlation double sampling of the pixel signal in the signal processing circuit 413. The A / D converted pixel signal is delivered to the demultiplexer 414 and stored in the pixel memory 415 corresponding to each pixel. The demultiplexer 414 and the pixel memory 415 are formed on the memory chip 112.

The arithmetic circuit 416 processes the pixel signal stored in the pixel memory 415 and hands it over to the image processing unit in the subsequent stage. The arithmetic circuit 416 may be provided on the signal processing chip 111 or the memory chip 112. Note that FIG. 5 shows the connections for one unit group 131, but in reality, these exist for each unit group 131 and operate in parallel. However, the arithmetic circuit 416 does not have to exist for each unit group 131. For example, one arithmetic circuit 416 may sequentially refer to the value of the pixel memory 415 corresponding to each unit group 131 for sequential processing.

As described above, the output wiring 309 is provided corresponding to each of the unit groups 131. The image pickup device 100 is laminated with an image pickup chip 113, a signal processing chip 111, and a memory chip 112. Therefore, by using the electrical connection between the chips using the bumps 109 for these output wirings 309, the wiring can be routed without enlarging each chip in the plane direction.

Next, a block set in the pixel region 113A (see FIG. 3) of the image sensor 100 will be described. In the present embodiment, the pixel region 113A of the image sensor 100 is divided into a plurality of blocks. The plurality of blocks are defined to include at least one unit group 131 per block. The pixels included in each block are controlled by different control parameters for each block. That is, pixel signals having different control parameters are acquired between the pixel group included in a certain block and the pixel group included in another block. The control parameters include, for example, charge accumulation time or number of times, frame rate, gain, thinning rate (pixel thinning rate), number of additional rows or columns to add pixel signals (pixel addition number), digitization bit. The number etc. can be given. Further, the control parameter may be a parameter in image processing after acquiring an image signal from a pixel.

Here, the charge accumulation time means the time from the start to the end of the charge accumulation of the PD 104. This charge accumulation time is also referred to as an exposure time or a shutter speed (shutter speed). The number of times the electric charge is accumulated means the number of times the PD 104 accumulates the electric charge per unit time. The frame rate is a value representing the number of frames processed (displayed or recorded) per unit time in a moving image. The unit of frame rate is fps (Frames Per Second). The higher the frame rate, the smoother the movement of the subject (that is, the object to be imaged) in the moving image.

Further, the gain means the gain rate (amplification rate) of the amplifier 412. By changing this gain, the ISO sensitivity can be changed. This ISO sensitivity is a standard for photographic film established by ISO, and indicates how weak a photographic film can record. However, in general, ISO sensitivity is also used when expressing the sensitivity of the image sensor 100. In this case, the ISO sensitivity is a value representing the ability of the image sensor 100 to capture light. Increasing the gain also improves the ISO sensitivity. For example, when the gain is doubled, the electric signal (pixel signal) is also doubled, and even if the amount of incident light is halved, the brightness becomes appropriate. However, when the gain is increased, the noise contained in the electric signal is also amplified, so that the noise increases.

The thinning rate refers to the ratio of the number of pixels in which the pixel signal is not read out to the total number of pixels in a predetermined area. For example, when the thinning rate of a predetermined area is 0, it means that the pixel signal is read out from all the pixels in the predetermined area. Further, when the thinning rate of the predetermined area is 0.5, it means that the pixel signal is read out from half the pixels in the predetermined area. Specifically, when the unit group 131 is a Bayer array, one unit of the Bayer array is placed in the vertical direction, that is, it is read as a pixel in which a pixel signal is alternately read out by two pixels (two rows each) in pixel units. Pixels that are not output are set. Note that the image resolution is reduced when the pixel signal is thinned out. However, since the image sensor 100 has 20 million or more pixels arranged, an image can be displayed with 10 million or more pixels even if thinning is performed at a thinning rate of 0.5, for example. Therefore, it is considered that the user (photographer) does not mind the decrease in resolution.

Further, the number of lines to be added means the number of pixels (number of lines) in the vertical direction to be added when the pixel signals of pixels adjacent to each other in the vertical direction are added. The number of columns to be added means the number of pixels (number of columns) in the horizontal direction to be added when the pixel signals of pixels adjacent to each other in the horizontal direction are added. Such addition processing is performed, for example, in the arithmetic circuit 416. By performing the process of adding the pixel signals of a predetermined number of pixels adjacent to each other in the vertical direction or the horizontal direction by the arithmetic circuit 416, the same effect as the process of reading out the pixel signals by thinning out at a predetermined thinning rate is obtained. In the above-mentioned addition process, the average value may be calculated by dividing the addition value by the number of rows or columns added by the arithmetic circuit 416.

The number of bits for digitization refers to the number of bits when the signal processing circuit 413 converts an analog signal into a digital signal in A / D conversion. As the number of bits of the digital signal increases, the brightness and color change are expressed in more detail.

In the present embodiment, the charge accumulation condition refers to a condition relating to charge accumulation in the image sensor 100. Specifically, the charge accumulation condition refers to the charge accumulation time or the number of times of accumulation, the frame rate, and the gain among the above-mentioned control parameters. Since the frame rate can change depending on the charge accumulation time and the number of times of accumulation, the frame rate is included in the charge accumulation condition. In addition, the amount of light with proper exposure changes according to the gain, and the charge accumulation time or the number of times of accumulation may change according to the amount of light with proper exposure. Therefore, the gain is included in the charge accumulation condition.

Further, in the present embodiment, the reading condition means a condition relating to reading of a pixel signal in the image sensor 100. Specifically, the read condition refers to the thinning rate and the number of additions (number of addition rows or addition columns) for adding pixel signals among the above-mentioned control parameters. The number of pixel signals output from the image sensor 100 changes according to the thinning rate and the number of additions.

Further, in the present embodiment, the imaging condition means a condition relating to imaging of a subject. Specifically, the imaging condition refers to a control parameter including the above-mentioned charge accumulation condition and read condition. The image pickup conditions include control parameters for controlling the image sensor 100 (for example, charge accumulation time or number of times, frame rate, gain), as well as control parameters for controlling reading of a signal from the image sensor 100 (for example,). For example, the thinning rate, the number of added rows or columns for adding pixel signals), control parameters for processing the signal from the image sensor 100 (for example, the number of digitization bits, the image processing unit 30 described later performs image processing. (Control parameters for executing) are also included.

FIG. 6 is a block diagram showing a configuration of a digital camera (electronic device) according to the first embodiment. As shown in FIG. 6, the digital camera 1 as an electronic device includes a camera body 2 and a lens unit 10. As described above, the lens unit 10 is an interchangeable lens that can be attached to and detached from the camera body 2. Therefore, the digital camera 1 does not have to include the lens unit 10. However, the lens unit 10 may be integrally configured with the digital camera 1. The lens unit 10 guides the luminous flux from the subject to the imaging unit 20 in a state of being connected to the camera body 2.

As described above, the lens unit 10 has a lens drive control device 15 (see FIG. 1). Further, the lens unit 10 has a plurality of lens groups as the photographing optical system 11, that is, a lens 11a, a zooming lens 11b, and a focusing lens 11c. When the lens unit 10 is connected to the camera body 2, the lens drive control device 15 transmits the lens information stored in the memory to the system control unit 70 of the camera body 2. Further, the lens drive control device 15 receives control information transmitted from the system control unit 70 when the lens unit 10 is connected to the camera body 2. The lens drive control device 15 controls the drive of the zooming lens 11b, the focusing lens 11c, and the aperture 11d based on the control information.

As shown in FIG. 6, the camera body 2 includes an image pickup unit 20, an image processing unit 30, a work memory 40, a display unit 50, an operation unit 55, a recording unit 60, and a system control unit 70.

The image pickup unit 20 includes an image pickup element 100 and a drive unit 21. The drive unit 21 is a control circuit that controls the drive of the image sensor 100 according to an instruction from the system control unit 70. Here, the drive unit 21 controls the timing (or timing cycle) of applying the reset pulse and the transfer pulse to the reset transistor 303 and the transfer transistor 302, respectively, to set the charge accumulation time or the number of times of accumulation, which is a control parameter. Control. Further, the drive unit 21 controls the frame rate by controlling the timing (or timing cycle) of applying the reset pulse, the transfer pulse, and the selection pulse to the reset transistor 303, the transfer transistor 302, and the selection transistor 305, respectively. To do. Further, the drive unit 21 controls the thinning rate by setting the pixels to which the reset pulse, the transfer pulse, and the selection pulse are applied.

Further, the drive unit 21 controls the ISO sensitivity of the image sensor 100 by controlling the gain (also referred to as a gain rate or amplification factor) of the amplifier 412. Further, the drive unit 21 sets the number of addition rows or the number of addition columns to which the pixel signals are added by sending an instruction to the arithmetic circuit 416. Further, the drive unit 21 sets the number of digitization bits by sending an instruction to the signal processing circuit 413. Further, the drive unit 21 sets an area in the pixel area (imaging area) 113A of the image sensor 100 in block units. In this way, the drive unit 21 has a function of an image sensor control unit that causes the image sensor 100 to take an image for each of a plurality of blocks under different imaging conditions and output a pixel signal. The system control unit 70 gives instructions such as the position, shape, and range of the block to the drive unit 21.

The image sensor 100 delivers the pixel signal from the image sensor 100 to the image processing unit 30. The image processing unit 30 uses the work memory 40 as a workspace to perform various image processing on the RAW data composed of the pixel signals of each pixel to generate image data in a predetermined file format (for example, JPEG format). .. The image processing unit 30 executes the following image processing. For example, the image processing unit 30 generates an RGB image signal by performing color signal processing (color tone correction) on the signal obtained in the Bayer arrangement. Further, the image processing unit 30 performs image processing such as white balance adjustment, sharpness adjustment, gamma correction, and gradation adjustment on the RGB image signal. Further, the image processing unit 30 performs a process of compressing in a predetermined compression format (JPEG format, MPEG format, etc.) as necessary. The image processing unit 30 outputs the generated image data to the recording unit 60. Further, the image processing unit 30 outputs the generated image data to the display unit 50.

In the present embodiment, in addition to the above-described processing, the image processing unit 30 performs a process (trimming process) of cutting out image data in a predetermined area instructed by the system control unit 70. The system control unit 70 outputs the image data of the predetermined area cut out by the image processing unit 30 to the display unit 50, and outputs the image based on the image data of the predetermined area to the display panel (display screen) 51 of the display unit 50. Controls the enlarged display. Cutting out a part of an image and enlarging it in this way is called electronic zoom or digital zoom.

Further, in the present embodiment, the image processing unit 30 performs a process of detecting a main subject from the image data in addition to the above-described process. Here, the "main subject" refers to a subject that is an object to be imaged and is a subject that is noticed by the user (photographer) or a subject that is presumed to be noticed by the user. The main subject is not limited to the case where only one is present in the image data, and may be present in a plurality of main subjects. Further, in the present embodiment, in addition to the above-described processing, the image processing unit 30 performs processing for detecting feature points of the main subject from the image data. Here, the "feature point" refers to a characteristic part of the main subject. The feature point is not limited to the case where only one is present in the image data, and may be present in the plurality of feature points.

The parameters referred to when the image processing unit 30 performs image processing are also included in the control parameters (imaging conditions). For example, parameters such as color signal processing (color tone correction), white balance adjustment, gradation adjustment, and compression rate are included in the control parameters. The signal read from the image sensor 100 changes according to the charge accumulation time and the like, and the parameters referred to when performing image processing also change according to the change in the signal. The image processing unit 30 sets different control parameters for each block, and executes image processing such as color signal processing based on these control parameters.

The work memory 40 temporarily stores image data and the like when image processing is performed by the image processing unit 30. The display unit 50 displays images (still images, moving images, live view images) captured by the imaging unit 20 and various types of information. The display unit 50 has a display panel (display screen) 51 such as a liquid crystal display panel. A touch panel 51 is formed on the display panel 51 of the display unit 50. The touch panel 51 outputs a signal indicating the position touched by the user to the system control unit 70 when the user performs an operation such as selecting a menu.

The operation unit 55 includes a release switch (a switch pressed when shooting a still image), a moving image switch (a switch pressed when shooting an operation), an optical zoom switch (a switch pressed when shooting an optical zoom), and an electronic zoom operated by the user. Switches (switches that are pressed during electronic zoom), various operation switches, etc. The operation unit 55 outputs a signal corresponding to the operation by the user to the system control unit 70. The recording unit 60 has a card slot into which a storage medium such as a memory card can be mounted. The recording unit 60 stores the image data and various data generated by the image processing unit 30 in the recording medium installed in the card slot. Further, the recording unit 60 has an internal memory. The recording unit 60 can also record the image data and various data generated by the image processing unit 30 in the internal memory.

The system control unit 70 controls the overall processing and operation of the digital camera 1. As described above, the system control unit 70 has a CPU (Central Processing Unit). In the present embodiment, the system control unit 70 divides the imaging surface (pixel region 113A) of the image sensor 100 (imaging chip 113) into a plurality of blocks, and the charge accumulation time (or the number of charge accumulations) and the frame rate differ between the blocks. , Get the image with gain. Therefore, the system control unit 70 instructs the drive unit 21 of the position, shape, range, and charge accumulation conditions for each block.

Further, the system control unit 70 causes the image to be acquired by the thinning rate different between the blocks, the number of additional rows or columns for adding the pixel signals, and the number of digitization bits. Therefore, the system control unit 70 instructs the drive unit 21 of the imaging conditions (thinning rate, the number of additional rows or columns for adding the pixel signals, and the number of digitization bits) for each block. Further, the image processing unit 30 executes image processing under different imaging conditions (control parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate) between blocks. Therefore, the system control unit 70 instructs the image processing unit 30 of imaging conditions (control parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate) for each block.

Further, the system control unit 70 causes the recording unit 60 to record the image data generated by the image processing unit 30. Further, the system control unit 70 causes the display unit 50 to display an image by outputting the image data generated by the image processing unit 30 to the display unit 50. Further, the system control unit 70 reads the image data recorded in the recording unit 60 and outputs the image data to the display unit 50 to display the image on the display unit 50. The image displayed on the display unit 50 includes a still image, a moving image, and a live view image.

Further, the system control unit 70 outputs a control signal instructing the movement of the zooming lens 11b to the lens drive control device 15 in response to the operation of the optical zoom switch by the user. The lens drive control device 15 drives the zooming lens 11b by controlling the drive control circuit based on the control information output from the system control unit 70. As a result, the display magnification of the image displayed on the display panel 51 is changed. Further, the system control unit 70 outputs an instruction signal indicating an area for cutting out an image to the image processing unit 30 in response to an operation of an electronic zoom switch or the like by the user. As a result, the image based on the image data of the region cut out by the image processing unit 30 is enlarged and displayed on the display panel 51 of the display unit 50 at a predetermined display magnification. Further, the system control unit 70 causes the lens drive control device 15 to execute the drive control of the focusing lens 11c and the drive control of the aperture 11d by outputting the control information to the lens drive control device 15.

FIG. 7 is a functional block diagram of the image processing unit and the system control unit shown in FIG. As shown in FIG. 7, the image processing unit 30 includes an image generation unit 31, a subject detection unit (first detection unit) 32, and a feature point detection unit (second detection unit) 33. The image generation unit 31 generates image data by performing various image processing on the RAW data composed of the pixel signals of each pixel output from the image pickup unit 20. The subject detection unit 32 detects a main subject (for example, a person's face) from the image data generated by the image generation unit 31. For example, the subject detection unit 32 detects a main subject by using a known face detection function as described in, for example, Japanese Patent Application Laid-Open No. 2010-16621 (US2010 / 0002940). The feature point detection unit 33 detects feature points (for example, human eyes) of a main subject from the image data generated by the image generation unit 31. For example, the feature point detection unit 33 detects feature points by using a known feature point detection function as described in, for example, Japanese Patent Application Laid-Open No. 2010-16621 (US2010 / 0002940).

Further, the system control unit 70 includes a first magnification change unit (magnification change unit) 71, a second magnification change unit (magnification change unit) 72, an imaging control unit 73, and a display control unit 74. The first magnification changing unit 71 is a processing unit that executes optical zoom in response to an operation of the optical zoom switch by the user. The second magnification changing unit 72 is a processing unit that executes electronic zoom in response to an operation of the electronic zoom switch by the user or the like.

The image pickup control unit 73 sets a plurality of areas in block units in the pixel area (imaging area) 113A of the image pickup device 100. Further, the imaging control unit 73 sets imaging conditions (including charge accumulation conditions and reading conditions) for a plurality of regions. Further, the image pickup control unit 73 causes the drive unit 21 to execute the drive control of the image pickup element 100 in response to the operation of the release switch or the moving image switch by the user. Further, the image pickup control unit 73 causes the drive unit 21 to execute the drive control of the image pickup element 100 even at the time of capturing the live view image (that is, after the start of the shooting operation after the power is turned on). The display control unit 74 controls the display unit 50 to output the image data generated by the image generation unit 31 and displays the image (still image, moving image, live view image) on the display panel 51.

Next, the operation of the optical zoom by the first magnification changing unit 71 will be described. FIG. 8 is a diagram showing a display example when optical zoom is performed. As shown in FIGS. 8A and 8B, a touch panel 52 is formed on the display panel 51 of the display unit 50. Further, as shown in FIGS. 8A and 8B, the optical axis (optical center) O of the photographing optical system 11 is located at the center of the display panel 51. Further, in the display example shown in FIG. 8A, the image C1 of a person is displayed substantially in the center of the display panel 51. The user can enlarge the image C1 of the person by operating the optical zoom switch in a specific direction, and reduce the image C1 of the person by operating the optical zoom switch in the direction opposite to the specific direction.

The first magnification changing unit 71 receives a signal indicating that the optical zoom switch has been operated in a specific direction by the user. The first magnification changing unit 71 sends a control signal instructing to move the zooming lens 11b in the direction corresponding to the specific direction (in the case of FIG. 8A, the direction in which the image C1 of the person is magnified). Output to the lens drive control device 15. The lens drive control device 15 moves the zooming lens 11b in a direction corresponding to a specific direction based on the control signal transmitted from the first magnification changing unit 71. By moving the zooming lens 11b, the subject image of a person on the image pickup surface of the image pickup device 100 is enlarged around the optical axis. In the display example shown in FIG. 8B, the image C1 of a person enlarged in response to the operation of the optical zoom switch by the user is displayed on the display panel 51.

Further, the operation of the electronic zoom by the second magnification changing unit 72 will be described. FIG. 9 is a diagram showing a display example when electronic zooming is performed. Also in FIGS. 9A and 9B, a touch panel 52 is formed on the display panel 51 of the display unit 50. Further, as shown in FIG. 9A, the optical axis (optical center) O of the photographing optical system 11 is located at the center of the display panel 51. Further, in the display example shown in FIG. 9A, the image C2 of the person is displayed at the lower left of the display panel 51. The user selects the area 510 enlarged and displayed by the electronic zoom, for example, by tracing the touch panel 52 with a finger. In the case shown in FIG. 9A, the area 510 magnified by the electronic zoom is the lower left area of the display panel 51. Hereinafter, the area enlarged and displayed by the electronic zoom is referred to as an enlargement target area. Then, the user can expand the enlargement target area 510 by operating the electronic zoom switch.

The second magnification changing unit 72 receives a signal indicating the enlargement target area 510 selected according to the operation of the touch panel 52 by the user. Then, when the image pickup control unit 73 receives the signal indicating that the electronic zoom switch has been operated by the user, the image processing unit 73 outputs a signal instructing the cutout (trimming) of the enlargement target area 510 to the image processing unit 30. The image generation unit 31 performs a process of cutting out the image data of the enlargement target area 510 instructed by the second magnification change unit 72. Further, the image generation unit 31 outputs the image data of the cut-out enlargement target area 510 to the display unit 50. As shown in FIG. 9B, the display unit 50 enlarges and displays an image (image C2 of a person) based on the image data of the enlargement target area 510 on the entire surface of the display panel 51. In the example shown in FIG. 9A, the user has selected the lower left area of the display panel 51 as the enlargement target area 510. However, the user can select any area of the display panel 51.

Next, the photographing operation according to the first embodiment will be described. FIG. 10 is a flowchart showing an imaging process (main process) according to the first embodiment. In the process shown in FIG. 10, after the power of the digital camera 1 is turned on, the user operates the operation unit 55 to start shooting the live view image, and the user starts shooting the moving image. It is executed by operating the operation unit 55 (moving image switch) for the purpose. That is, the process shown in FIG. 10 is a process executed at the time of shooting the live view image and the moving image.

In the process shown in FIG. 10, first, the image pickup control unit 73 starts a shooting operation (step S1). That is, the image pickup control unit 73 outputs an instruction signal to the drive unit 21 to cause the drive unit 21 to execute the drive control of the image pickup element 100 under predetermined image pickup conditions (including charge accumulation condition and read condition). Next, the system control unit 70 determines whether or not the optical zoom switch is being operated by the user (step S2). When the system control unit 70 determines that the optical zoom switch is being operated, the first magnification changing unit 71 and the imaging control unit 73 execute control corresponding to the optical zoom (step S3).

FIG. 11 is a flowchart showing the control (step S3) corresponding to the optical zoom shown in FIG. Further, FIG. 12 is a diagram showing a central region and a peripheral region in the imaging region. In the process shown in FIG. 11, the first magnification changing unit 71 executes the optical zoom based on the operation of the optical zoom switch by the user (step S10). That is, the first magnification changing unit 71 outputs a control signal according to the operation amount of the optical zoom switch to the lens drive control device 15. Next, the image pickup control unit 73 displays a region 200 (see FIG. 12 (A)) including the center (optical axis) O of the magnification fluctuation (enlargement or reduction) due to the optical zoom in the pixel region (imaging region) 113A. It is designated in the center of the screen of 51 (step S11). Here, the region including the center O of the magnification fluctuation due to the optical zoom is referred to as a central region. In the example shown in FIG. 12A, the central region 200 is a circular region centered on the optical axis O. Further, as shown in FIG. 12A, a region other than the central region 200 in the pixel region 113A is referred to as a peripheral region 210.

Next, the image pickup control unit 73 adjusts (changes) the size of the central region 200 according to the speed of the magnification fluctuation in the optical zoom (step S12). Here, the speed of magnification fluctuation means the speed at which the image is magnified (enlarged or reduced) by the optical zoom. The first magnification changing unit 71 detects the operation amount of the optical zoom switch per unit time based on the signal from the optical zoom switch. Then, the first magnification changing unit 71 determines the speed of the magnification fluctuation by the operation amount of the optical zoom switch. The operating amount of the optical zoom switch corresponds to the moving amount (driving amount) of the zooming lens 11b. The image pickup control unit 73 outputs an instruction signal for designating the central region 200 according to the speed of magnification fluctuation to the drive unit 21.

Next, the image pickup control unit 73 outputs an instruction signal to the drive unit 21 to read out the pixel signal at a different frame rate for each region (central region 200, peripheral region 210) in the pixel region 113A (step). S13).

When the user operates the optical zoom switch to enlarge or reduce the image (subject image) displayed on the display panel 51, the image flows in the radial direction faster in the peripheral portion than in the central portion of the display panel 51. .. Therefore, when the image is flowing fast in the peripheral portion, it is difficult for the user to recognize the image in the peripheral portion, and the image in the peripheral portion is rarely watched. On the other hand, when the user is enlarging or reducing the image with the optical zoom, the user is paying attention to the image at the center of the display panel 51. In addition, the faster the magnification fluctuation speed, the faster the image flows, and the user pays attention to the image in the central part of a narrower range.

Therefore, in step S12, the image pickup control unit 73 adjusts the size of the central region 200, which is assumed to be the focus of the user, according to the speed of the magnification fluctuation in the optical zoom. For example, the image pickup control unit 73 narrows the range of the central region 200 as the speed of magnification fluctuation is faster, and widens the range of the central region 200 as the speed of magnification fluctuation is slower. The image pickup control unit 73 outputs an instruction signal instructing the central region 200 according to the speed of magnification fluctuation.

Further, in step S13, the image pickup control unit 73 reads out the pixel signal at a high frame rate in the central region 200 that the user is paying attention to, and at a low frame rate in the peripheral region 210 that the user is not paying attention to. The pixel signal is read out. For example, at the time of capturing a live view image, the imaging control unit 73 outputs an instruction signal indicating 60 fps as the frame rate of the central region 200 to the drive unit 21, and indicates an instruction signal indicating 30 fps as the frame rate of the peripheral region 210. Is output to the drive unit 21. The drive unit 21 reads out the pixel signal at 60 fps in the central region 200 and reads out the pixel signal at 30 fps in the peripheral region 210 based on the instruction signal from the image pickup control unit 73. As a result, a live view image with smooth movement is displayed in the central portion (region corresponding to the central region 200) rather than the peripheral portion (region corresponding to the peripheral region 210) of the display panel 51.

In step S13, the image pickup control unit 73 reads out the pixel signal at a different thinning rate for each area, instead of executing the reading of the pixel signal at a different frame rate for each area (central area 200, peripheral area 210). You may let it run. For example, at the time of capturing a live view image, the imaging control unit 73 outputs an instruction signal indicating 0.5 as the thinning rate of the central region 200 to the driving unit 21. Further, the image pickup control unit 73 outputs an instruction signal indicating 0.8 as the thinning rate of the peripheral region 210 to the drive unit 21. The drive unit 21 reads out the pixel signal at a thinning rate of 0.5 in the central region 200, and reads out the pixel signal at a thinning rate of 0.8 in the peripheral area 210. As a result, a live view image having a higher resolution is displayed in the central portion than the peripheral portion of the display panel 51.

Further, in step S13, the imaging control unit 73 reads out the pixel signal at a different frame rate for each region (central region 200, peripheral region 210), but instead performs a different addition number (addition number of rows or addition) for each region. The pixel signal addition process may be executed according to the number of columns). For example, at the time of capturing a live view image, the imaging control unit 73 outputs an instruction signal indicating “2” as the addition number of the central region 200 to the driving unit 21. Further, the image pickup control unit 73 outputs an instruction signal indicating "5" as the addition number of the peripheral region 210 to the drive unit 21. The drive unit 21 causes the arithmetic circuit 416 to execute an addition process based on the addition number instructed by the instruction signal from the image pickup control unit 73. The arithmetic circuit 416 adds two rows or two columns of pixel signals stored in the pixel memory 415 corresponding to the central region 200. Further, the arithmetic circuit 416 adds the pixel signals of 5 rows or 5 columns stored in the pixel memory 415 corresponding to the peripheral area 210. As a result, a live view image having a higher resolution is displayed in the central portion than the peripheral portion of the display panel 51.

Further, in step S13, the image pickup control unit 73 executes reading of the pixel signal at a different frame rate for each region (central region 200, peripheral region 210), and also executes reading of the pixel signal at a different thinning rate for each region. You may let it. Further, in step S13, the image pickup control unit 73 reads out the pixel signal at a different frame rate for each region (central region 200, peripheral region 210), and adds the pixel signal with a different addition number for each region. You may let it run.

In FIG. 12A described above, the image pickup control unit 73 sets one central region 200 in the pixel region 113A. However, it is not limited to such a configuration. For example, as shown in FIG. 12B, the imaging control unit 73 may set two central regions 201 and 202 in the pixel region 113A. That is, the image pickup control unit 73 may set a ring-shaped central region 202 around the circular central region 201.

Also in this case, in step S12, the image pickup control unit 73 adjusts (changes) the sizes of the two central regions 201 and 202 according to the speed of the magnification fluctuation in the optical zoom. However, the image pickup control unit 73 may adjust the size of only one of the central regions 201 and 202.

Further, in step S13, the image pickup control unit 73 reads out the pixel signal at a different frame rate for each region (central regions 201 and 202, peripheral regions 210) in the pixel region 113A. For example, the image pickup control unit 73 reads out the pixel signal at the highest frame rate (for example, 60 fps) in the central region 201 that the user is paying attention to most. Further, the image pickup control unit 73 reads out the pixel signal in the central region 202 at a frame rate (for example, 30 fps) lower than that in the central region 201. Further, the image pickup control unit 73 reads out the pixel signal at a low frame rate (for example, 15 fps) in the peripheral region 210 that the user is not paying attention to. As described above, the imaging control unit 73 replaces changing the frame rate for each area, or changes the frame rate for each area, and the thinning rate or the number of additions (the number of lines to be added or the number of additions) for each area. The number of columns) may be changed. Further, the image pickup control unit 73 may set three or more central regions in the pixel region 113A.

Returning to the description of FIG. 10, when the system control unit 70 determines that the optical zoom switch has not been operated by the user (NO in step S2), it determines whether or not the electronic zoom switch has been operated. Determine (step S4). When the system control unit 70 determines that the electronic zoom switch has been operated, the second magnification changing unit 72 and the imaging control unit 73 execute control corresponding to the electronic zoom (step S5).

FIG. 13 is a flowchart showing the control for electronic zoom shown in FIG. 10 (step S5). Further, FIG. 14 is a diagram showing an enlargement target region in the imaging region. In the process shown in FIG. 13, the second magnification changing unit 72 designates the enlargement target area 220 (see FIG. 14) selected according to the operation of the touch panel 52 by the user in the pixel area (imaging area) 113A (see FIG. 14). Step S21). At this time, the second magnification changing unit 72 outputs a signal instructing the cutting out of the enlargement target area 220 to the image processing unit 30. If the aspect ratio is different between the enlargement target area 220 and the entire area of the display panel 51, the shape of the image will be distorted after the enlargement by the electronic zoom. Therefore, the second magnification changing unit 72 designates an enlargement target area 220 having the same aspect ratio as that of the display panel 51.

Next, the image pickup control unit 73 causes the second magnification changing unit 72 to read out the pixel signal for the enlargement target area 220 designated in step S21 at a thinning rate according to the enlargement ratio of the electronic zoom (step S22). ). Here, the enlargement ratio means the ratio of the area of the enlargement target area to the area of the entire screen of the display panel 51. The "thinning rate according to the enlargement ratio" is, for example, a thinning ratio such that the resolution of the image after enlargement by the electronic zoom is the same as or close to the resolution of the image before enlargement.

In general, live view images and moving images do not require high resolution imaging from the user. Therefore, during the acquisition of the live view image or the moving image, the pixel signal is read out at a predetermined thinning rate. For example, the pixel signal is read out at a thinning rate of 0.75 during the imaging of the live view image. That is, the pixel signal is read out in 1/4 of all the pixels. However, the electronic zoom cuts out a part of the image and enlarges it on the display unit 50. Therefore, when the image is enlarged by the electronic zoom, the resolution (that is, the pixel density) of the image is lowered, and the image quality is lowered. In this case, the deterioration of the image quality may be noticeable depending on the magnification of the electronic zoom. Therefore, in step S22, the image pickup control unit 73 reads out the pixel signal at a thinning rate according to the enlargement rate of the electronic zoom.

For example, assume that the image is magnified twice by electronic zoom. Further, it is assumed that the thinning rate before the electronic zoom is 0.75 and the thinning rate after the electronic zoom is 0.5. That is, before the electronic zoom, the pixel signal is read out with 1/4 pixel, and after the electronic zoom, the pixel signal is read out with 1/2 pixel. In this case, the resolution of the image does not change before and after the enlargement by the electronic zoom. This prevents a decrease in the resolution of the image after electronic zooming.

As a specific process in step S22, the image pickup control unit 73 calculates the enlargement ratio by the ratio of the area of the enlargement target area 220 selected by the user to the area of the entire screen of the display panel 51. Then, the image pickup control unit 73 calculates the thinning rate after the electronic zoom based on the calculated magnification and the thinning rate before the electronic zoom. Further, the image pickup control unit 73 outputs an instruction signal instructing the thinning rate after the electronic zoom to the drive unit 21. The drive unit 21 reads out the pixel signal from the enlargement target area 220 at the thinning rate instructed by the image pickup control unit 73.

Next, the image pickup control unit 73 is set so as not to read the pixel signal from the area other than the enlargement target area 220 in the pixel area 113A (step S23). This is because, in the present embodiment, after the electronic zoom, the image of the area other than the enlargement target area 220 is not displayed on the display panel 51, and it is not necessary to read out the pixel signal.

In step S22, the image pickup control unit 73 reads the pixel signal at the thinning rate according to the enlargement ratio, and instead of reading out the pixel signal at the thinning rate according to the enlargement ratio, the pixel signal is added by the number of additions (the number of addition rows or the number of addition columns) according to the enlargement ratio. May be executed. For example, when the addition number before enlargement by the electronic zoom is "4" and the enlargement ratio is double, the image pickup control unit 73 indicates an instruction signal to indicate "2" as the addition number after the enlargement by the electronic zoom. Is output to the drive unit 21. Even with such a configuration, it is possible to prevent the resolution of the image from changing before and after the enlargement by the electronic zoom.

Returning to the description of FIG. 10, when the system control unit 70 determines that the electronic zoom switch has not been operated by the user (NO in step S4), the image pickup control unit 73 executes normal read control. (Step S6). That is, the image pickup control unit 73 outputs an instruction signal indicating normal image pickup conditions (for example, image pickup conditions for proper exposure) at the time of capturing a live view image or when capturing a moving image to the drive unit 21. The drive unit 21 executes imaging under normal imaging conditions in all pixel regions 113A.

The image pickup unit 20 outputs the pixel signal of each pixel read in steps S3, S5, and S6 to the image processing unit 30. The image generation unit 31 generates image data in a predetermined file format by performing various image processing on the pixel data of each pixel. At this time, when the electronic zoom switch is operated, the image generation unit 31 performs a process of cutting out the image data of the enlargement target area 220 instructed by the second magnification change unit 72.

Next, the display control unit 74 outputs the image data generated by the image generation unit 31 to the display unit 50, and displays the image (moving image, live view image) on the display panel 51 (step S7). At this time, when the electronic zoom switch is being operated, the display control unit 74 displays the image of the enlargement target area 220 cut out by the image generation unit 31 on the entire surface of the display panel 51 (see FIG. 9). ..

Next, the system control unit 70 stores the image data generated by the image generation unit 31 in the storage unit 60 when the moving image is being shot (step S8). After that, the system control unit 70 determines whether or not the shooting is completed (step S9). The system control unit 70 operates the operation unit 55 (video switch) when the user operates the operation unit 55 to end the shooting of the live view image, or the user operates the operation unit 55 (video switch) to end the shooting of the moving image. When the above is performed, it is determined that the shooting is completed. The system control unit 70 repeatedly executes the processes shown in FIG. 10 (processes of steps S1 to S9) at predetermined intervals until it is determined that the photographing is completed.

As described above, in the first embodiment, the display unit 50 having the first display area in which the first image is displayed and the second display area in which the second image is displayed, and the first image are shown. An imaging unit 20 having a first imaging region for generating one image data and a second imaging region for generating a second image data indicating a second image, and a first image and a second image displayed on the display unit 50. Imaging control that changes the charge accumulation condition or read condition in the first imaging region and the second imaging region when the display magnification is changed by the magnification changing units 71 and 72 that change the display magnification and the magnification changing units 71 and 72. A unit 73 is provided.

Here, as an example of the "first display region", in the case of the optical zoom, it is an region on the display panel 51 corresponding to the central region 200 (or the central regions 201 and 202) on the image sensor 100. In the case of electronic zoom, it is an area on the display panel 51 corresponding to the enlargement target area 220 on the image sensor 100. Further, as an example of the "second display area", in the case of the optical zoom, it is an area on the display panel 51 corresponding to the peripheral area 210 on the image sensor 100. In the case of the electronic zoom, it is an area on the display panel 51 corresponding to an area other than the enlargement target area 220 on the image sensor 100. Further, as an example of the "first imaging region", in the case of the optical zoom, it is the central region 200 (or the central regions 201 and 202) on the image sensor 100. In the case of electronic zoom, it is the enlargement target area 220 on the image sensor 100. Further, as an example of the "second imaging region", in the case of the optical zoom, it is the peripheral region 210 on the image sensor 100. In the case of electronic zoom, it is an area other than the enlargement target area 220 on the image sensor 100.

According to such a configuration, power consumption can be suppressed when the display magnification of the image is changed by the optical zoom or the electronic zoom. That is, in the case of the optical zoom, the imaging control unit 73 reads out the signal in the second imaging region (peripheral region 210) at a frame rate lower than that in the first imaging region (central region 200). As a result, the power consumption can be reduced as compared with the case where the signal is read out uniformly at a high frame rate in the pixel region 113A. Further, in the case of the electronic zoom, the power consumption can be reduced by not reading the signal from the second imaging region (area other than the enlargement target region 220) by the imaging control unit 73.

The fact that the magnification changing unit 72 "changes the display magnification of the second image" includes not displaying the second image. Further, the imaging control unit 73 "changes the charge accumulation condition or the reading condition between the first imaging region and the second imaging region" means that the signal is displayed under a predetermined charge accumulation condition or the reading condition in the first imaging region. It also includes reading and not reading the signal in the second imaging region.

Further, in the first embodiment, the image pickup control unit 73 changes at least one of the frame rate, the pixel thinning rate, and the number of pixel additions as the charge accumulation condition or the read condition. According to such a configuration, when the imaging control unit 73 changes the frame rate between the first imaging region and the second imaging region, the smoothness of movement can be changed between the first image and the second image. it can. Further, when the image pickup control unit 73 changes the pixel thinning rate or the number of pixel additions between the first image pickup area and the second image pickup area, the resolutions of the first image and the second image can be made different.

Further, in the first embodiment, the imaging control unit 73 performs at least one of a decrease in the frame rate, an increase in the pixel thinning rate, and an increase in the number of pixel additions in the second imaging region with respect to the first imaging region. According to such a configuration, when the image pickup control unit 73 changes the frame rate between the first image pickup area and the second image pickup area, the movement of the first image is smoother than that of the second image on the display unit 50. be able to. Further, when the image pickup control unit 73 changes the pixel thinning rate or the number of pixel additions between the first image pickup area and the second image pickup area, the resolution of the first image can be made higher than that of the second image.

Further, in the first embodiment, there are a plurality of magnification changing units 71 and 72 of different types, and the imaging control unit 73 is made to image under different charge accumulation conditions or reading conditions depending on the magnification changing units 71 and 72. Control. According to such a configuration, the image pickup control unit 73 can perform imaging under the charge accumulation condition or the read condition for each region suitable for the optical zoom, and the charge accumulation condition or the read condition for each region suitable for the electronic zoom. It can also be imaged with.

Further, in the first embodiment, the magnification changing unit 73 includes an optical zoom that moves the photographing optical system 11 to change the display magnification of the image. According to such a configuration, the charge accumulation condition or the reading condition between the first imaging region and the second imaging region can be changed according to the change in the display magnification of the image by the optical zoom. Further, in the first embodiment, the first imaging region is a region including the optical axis of the photographing optical system. According to such a configuration, it is possible to acquire an image with smooth movement and an image with high resolution in the first imaging region including the optical axis.

Further, in the first embodiment, the magnification changing unit 73 includes an electronic zoom that cuts out and magnifies a part of the captured image. According to such a configuration, the charge accumulation condition or the reading condition between the first imaging region and the second imaging region can be changed according to the change in the display magnification of the image by the electronic zoom. Further, in the first embodiment, the image pickup control unit 73 controls so that when the magnification change unit 72 changes the display magnification of the first image by the electronic zoom, the second image pickup region is not imaged. According to such a configuration, power consumption can be reliably suppressed.

Further, in the first embodiment, the image pickup control unit 73 changes the first image pickup region according to the change of the display magnification by the magnification change unit 71. According to such a configuration, the first imaging region that the user pays attention to is changed according to the speed at which the image is enlarged or reduced by the magnification changing unit 71, and the second imaging region that is difficult for the user to see is changed. Will also be changed. Therefore, even if the charge accumulation condition or the read condition between the first imaging region and the second imaging region is changed, the user does not feel uncomfortable. In addition, the first image can be made even more noticeable.

Further, in the first embodiment, the image pickup control unit 73 changes the charge accumulation condition or the read condition according to the change of the display magnification by the magnification change units 71 and 72. According to such a configuration, it is possible to prevent the resolution of the image from being changed (or not changed significantly) in the process of changing the display magnification of the image or before and after the change. Therefore, it is possible to prevent the user from feeling uncomfortable during the process of changing the display magnification of the image and before and after the change.

<Second Embodiment>
In the first embodiment described above, the system control unit 70 designates the central region 200 according to the speed of magnification fluctuation regardless of the subject, and the enlargement target region 220 selected by the user. Was specified. On the other hand, in the second embodiment, an area including the main subject and an area including the feature points of the main subject are designated.

FIG. 15 is a flowchart showing an imaging process according to the second embodiment. In the process shown in FIG. 15, first, the image pickup control unit 73 starts a shooting operation (step S1). Next, the image pickup control unit 73 causes the subject detection unit 32 to estimate the main subject by outputting an instruction signal to the subject detection unit 32 (step S31). When the subject detection unit 32 receives the instruction signal from the image pickup control unit 73, the subject detection unit 32 detects (estimates) the face of a person who is the main subject by using a known face detection function based on the image data generated by the image generation unit 31. To do. Then, the subject detection unit 32 outputs a signal indicating the position and size of the detected person's face to the system control unit 70.

Further, the image pickup control unit 73 causes the feature point detection unit 33 to estimate the feature points of the main subject by outputting an instruction signal to the feature point detection unit 33 (step S32). When the feature point detection unit 33 receives the instruction signal from the image pickup control unit 73, the feature point detection unit 33 detects the eyes, which are the feature points of the main subject, using a known feature point detection function based on the image data generated by the image generation unit 31. (presume. Then, the feature point detection unit 33 outputs a signal indicating the position and size of the eyes of the detected person to the system control unit 70.

Next, the system control unit 70 determines whether or not the optical zoom switch has been operated by the user (step S2). When the system control unit 70 determines that the optical zoom switch has been operated, the first magnification changing unit 71 and the imaging control unit 73 execute control corresponding to the optical zoom (step S3A).

FIG. 16 is a flowchart showing the control (step S3A) corresponding to the optical zoom shown in FIG. Further, FIG. 17 is a diagram showing a main subject area and a feature point area set in the image pickup area at the time of optical zoom. The display example shown in FIG. 17A shows a state before the optical zoom is performed, and the display example shown in FIG. 17B shows a state after the optical zoom is performed.

In the process shown in FIG. 16, the first magnification changing unit 71 executes the optical zoom based on the operation of the optical zoom switch by the user (step S40). That is, the first magnification changing unit 71 outputs a control signal according to the operation amount of the optical zoom switch to the lens drive control device 15. Next, as shown in FIG. 17A, the image pickup control unit 73 includes a region including the image C3 of the face, which is the main subject (main subject region 230 shown in FIG. 17A), and feature points of the main subject. The region including the images f31 and f32 of the eyes (feature point regions 241,242 shown in FIG. 17A) is selected in block units (step S41).

Specifically, the image pickup control unit 73 selects the main subject area 230 including the face image C3 in block units based on the signal indicating the position and size of the face image C3 from the subject detection unit 32. Further, the image pickup control unit 73 includes feature point regions 241,242 including eye images f31 and f32 which are feature points, based on signals indicating the positions and sizes of eye images f31 and f32 from the feature point detection unit 33. Is selected in block units. The main subject and the feature point are not always located in the central portion including the optical axis O. However, in general, when the optical zoom is performed, the main subject and the feature point are often located in the central portion including the optical axis O.

As shown in FIG. 17B, the image C3 of the human face is enlarged by the optical zoom. As the size of the face image C3 increases, the main subject area 231 including the face image C3 also increases. The main subject area 230 shown in FIG. 17A is an area having 7 × 8 vertical and horizontal blocks. On the other hand, the main subject area 231 shown in FIG. 17B is an area having 13 × 14 vertical and horizontal blocks. In this way, since the size of the main subject area changes due to the optical zoom, the image pickup control unit 73 selects the main subject area at any time in step S41 according to the change in the magnification in the optical zoom.

Further, as shown in FIG. 17B, the eye images f31 and f32 are enlarged by the optical zoom. As the size of the eye images f31 and f32 increases, the main subject areas 243 and 244 including the eye images f31 and f32 also increase. The feature point regions 241,242 shown in FIG. 17A are regions in which vertical and horizontal blocks are 2 × 2, respectively. On the other hand, the feature point regions 243 and 244 shown in FIG. 17B are regions in which vertical and horizontal blocks are 3 × 3, respectively. In this way, since the size of the feature point region changes depending on the optical zoom, the image pickup control unit 73 selects the feature point region at any time in step S41 according to the change in the magnification in the optical zoom.

In FIG. 17, the main subject area is a rectangular area including the main subject, but as shown in FIG. 12, it may be a circular area centered on the optical axis O. However, it is assumed that the main subject is included in this circular area. Further, although the feature point region is a rectangular region including the feature points, it may be a circular region centered on the optical axis O as shown in FIG. However, it is assumed that the feature points are included in this circular area.

Next, the image pickup control unit 73 outputs an instruction signal to the drive unit 21 to read out the pixel signal at a thinning rate suitable for each area (main subject area, feature point area, and other areas). Step S42). Specifically, the image pickup control unit 73 has the lowest thinning rate with respect to the feature point region. Further, the image pickup control unit 73 makes the thinning rate higher than that of the feature point region with respect to the main subject region. Further, the image pickup control unit 73 makes the thinning rate higher than that of the main subject area with respect to the area other than the main subject area. Images such as backgrounds other than the main subject are presumed to be images that have not been noticed by the user. Therefore, it is considered that the user does not care even if the image is blurred when the image is enlarged or reduced by the optical zoom. Therefore, the image pickup control unit 73 increases the thinning rate of the area other than the main subject area and lowers the image resolution.

On the other hand, the image of the human eye, which is a feature point, is presumed to be the image that attracts the most attention from the user. For example, in portrait photography, the focal position is often aligned with the eyes of a person, and the user wants to check the image of the eyes. Therefore, the image pickup control unit 73 has the lowest thinning rate with respect to the feature point region and the highest image resolution. Further, it is presumed that the image of the face of a person who is the main subject is an image that is attracting attention from the user. However, since the contrast of a person's skin is low, for example, the resolution is not so necessary for shooting. Therefore, the image pickup control unit 73 lowers the thinning rate with respect to the main subject area as compared with the areas other than the main subject area, and raises the thinning rate as compared with the feature point area.

If the speed of magnification fluctuation is high, the image is moving at high speed, so high resolution is not necessary. Therefore, the image pickup control unit 73 may increase the thinning rate of each region as the speed of magnification fluctuation increases. In this case, the image pickup control unit 73 may change the thinning rate at a different rate depending on the speed of magnification fluctuation for each region.

Further, in step S42, the imaging control unit 73 executes the pixel signal addition processing with the number of added rows or the number of added columns suitable for each area, instead of executing the reading of the pixel signal with the thinning rate suitable for each area. You may let it. Further, in step S42, the image pickup control unit 73 may execute the pixel signal addition processing at the frame rate suitable for each region instead of executing the reading of the pixel signal at the thinning rate suitable for each region. Good.

Returning to the description of FIG. 15, when the system control unit 70 determines that the optical zoom switch has not been operated by the user (NO in step S2), it determines whether or not the electronic zoom switch has been operated. Determine (step S4). When the system control unit 70 determines that the electronic zoom switch has been operated, the second magnification changing unit 72 and the imaging control unit 73 execute control corresponding to the electronic zoom (step S5A).

FIG. 18 is a flowchart showing the control for electronic zoom shown in FIG. 15 (step S5A). Further, FIG. 19 is a diagram showing a main subject area and a feature point area set in the imaging area at the time of electronic zooming. In the process shown in FIG. 18, the second magnification changing unit 72 includes a region including the image C3 of the face, which is the main subject, in the pixel region (imaging region) 113A (main subject region 250 shown in FIG. 19A). Regions (feature point regions 261,262 shown in FIG. 19A) including eye images f31 and f32, which are feature points of the main subject, are selected in block units. Further, the second magnification changing unit 72 selects the enlargement target area 270 including the main subject area 250 and the feature point areas 261,262 (step S51). Then, the second magnification changing unit 72 outputs an instruction signal instructing to cut out the selected enlargement target area 270 to the image processing unit 30.

As shown in FIG. 19A, before the electronic zoom is performed, the enlargement target area 270 including the main subject area 250 and the feature point areas 261,262 is selected. As described above, in the second embodiment, the main subject area is not selected by the user, but is automatically selected according to the main subject. In the example shown in FIG. 19A, the main subject area 250 is an area having 7 × 8 vertical and horizontal blocks. Further, each of the feature point regions 261,262 is an region having 2 × 2 vertical and horizontal blocks. Further, the expansion target area 270 is an area having 10 × 14 vertical and horizontal blocks. As shown in FIG. 19B, the sizes of the main subject area 250, the feature point areas 261,262, and the enlargement target area do not change even when the display is enlarged by the electronic zoom. That is, the number of blocks constituting each area is not changed.

Next, the image pickup control unit 73 causes the enlargement target area 270 selected in step S51 to read out the pixel signal from each area at a thinning rate according to the enlargement rate of the electronic zoom (step S52). For example, the image pickup control unit 73 sets the thinning rate of the enlargement target area 270 to a thinning rate such that the resolution of the image does not change before and after the enlargement by the electronic zoom. Further, the image pickup control unit 73 sets the thinning rate of the main subject area 250 in the enlargement target area 270 to be lower than the thinning rate of the areas other than the main subject area 250 in the enlargement target area 270. Further, the image pickup control unit 73 sets the thinning rate of the feature point regions 261,262 to be higher than the thinning rate of the regions other than the feature point regions 261,262 in the main subject region 250.

In step S52, the image pickup control unit 73 calculates the enlargement ratio by the ratio of the area of the enlargement target area 270 to the area of the entire surface of the display panel 51. Then, the image pickup control unit 73 calculates the thinning rate after the electronic zoom based on the calculated magnification and the thinning rate before the electronic zoom. Further, the image pickup control unit 73 outputs an instruction signal indicating the thinning rate of each region after the electronic zoom to the drive unit 21. The drive unit 21 reads out the pixel signal from each area in the enlargement target area 270 at the thinning rate instructed by the image pickup control unit 73.

Next, the image pickup control unit 73 is set so as not to read the pixel signal from the area other than the enlargement target area 270 in the pixel area 113A (step S53). This is because after the electronic zoom, the image of the area other than the enlargement target area 270 in the pixel area 113A is not displayed on the display panel 51 (see FIG. 19B), so that it is not necessary to read out the pixel signal.

In step S52, the imaging control unit 73 reads the pixel signal at the thinning rate according to the enlargement ratio, and instead of reading out the pixel signal, the pixel signal is added (the number of rows to be added or the number of columns to be added) according to the enlargement ratio. May be executed.

Returning to the description of FIG. 15, when the system control unit 70 determines that the electronic zoom switch has not been operated by the user (NO in step S4), the image pickup control unit 73 executes normal read control. (Step S6). Since the processing of steps S6 to S9 shown in FIG. 15 is the same as the processing of steps S6 to S9 shown in FIG. 10, the same reference numerals are given and duplicate description will be omitted.

As described above, in the second embodiment, the first detection unit 32 that detects the subject based on the image captured by the image pickup unit 20 is provided, and the image pickup control unit 73 detects the subject by the first detection unit 32. The region including the subject (main subject region 230) is set as the first imaging region. According to such a configuration, it is possible to maintain a high resolution of the image of the subject that the user pays attention to.

Further, in the second embodiment, the second detection unit 33 that detects the feature points based on the image of the specific region is provided, and the imaging control unit 73 includes a third imaging region (for example, a feature) including the feature points in the specific region. The charge accumulation condition or the read condition is changed between the point region) and the region other than the third imaging region. According to such a configuration, the resolution of the image of the feature point that the user pays most attention to can be increased.

<Third Embodiment>
In the first and second embodiments described above, when the electronic zoom is performed, the image pickup control unit 73 is set not to read out the signal in a region other than the enlargement target region. Then, the display control unit 74 enlarged and displayed the image of the enlargement target area on the entire surface of the display panel 51. On the other hand, in the third embodiment, the image pickup control unit 73 reads out the signal even in an area other than the enlargement target area. Then, in the third embodiment, the display control unit 74 enlarges and displays the image of the enlargement target area on the entire surface of the display panel 51, and images the entire pixel area 113A in a predetermined area in the entire surface of the display panel 51. The image is also displayed.

FIG. 20 is a flowchart showing the control corresponding to the electronic zoom according to the third embodiment. The control for electronic zoom shown in FIG. 20 is executed in step S5A of FIG. After the electronic zoom, the image pickup control unit 73 performs an operation to display an image in the enlargement target area and simultaneously display an image captured in the entire pixel area 113A (hereinafter, referred to as an overall display operation). It is determined whether or not the image is present (step S61).

The second magnification changing unit 72 selects the main subject area, the feature point area, and the enlargement target area in the pixel area (imaging area) 113A in block units (step S62). The process of step S61 is the same as the process of step S51. Then, the image pickup control unit 73 reads out the pixel signal in the entire pixel region 113A at the thinning rate for each region according to the enlargement ratio of the electronic zoom (step S63). For example, the image pickup control unit 73 sets the thinning rate of the enlargement target area to a thinning rate such that the resolution of the image does not change before and after the enlargement by the electronic zoom. Further, the image pickup control unit 73 sets the thinning rate of the main subject area in the enlargement target area to be lower than the thinning rate of the area other than the main subject area in the enlargement target area. Further, the image pickup control unit 73 sets the thinning rate of the feature point region to be higher than the thinning rate of the region other than the feature point region in the main subject region. Further, the image pickup control unit 73 sets the thinning rate of the area other than the enlargement target area in the pixel area 113A to be higher than the thinning rate of the enlargement target area. As described above, in the third embodiment, the pixel signal is read out even in the area other than the expansion target area in the pixel area 113A. However, since the area other than the expansion target area is not paid attention to by the user, the thinning rate is set to be the highest.

When the image pickup control unit 73 determines in step S61 that the entire display operation has not been performed, the second magnification change unit 72 or the image pickup control unit 73 is the same as the process of steps S51 to S53 of FIG. 18 described above. The processing (steps S64 to S66 in FIG. 20) is executed.

FIG. 21 is a diagram showing a display example when the electronic zoom is performed in the third embodiment. In step S7 of FIG. 15, the display control unit 74 displays the image shown in FIG. 21 on the display panel 51. As shown in FIG. 21, the image C4 of the face, which is the main subject (specifically, the image of the enlargement target area 270 shown in FIG. 19A) is enlarged and displayed on the display panel 51. Further, a sub screen 51A in a small area is provided at the lower right of the display panel 51. An image (an image of the upper body of a person) captured in the entire pixel region 113A is displayed on the sub screen 51A. Further, the frame 600 surrounding the face image C4 displayed on the display panel 51 and the sub screen 51A is displayed so that the user can recognize the position of the face image C4.

As described above, in the third embodiment, the image pickup control unit 73 reads out the pixel signal in the entire pixel region 113A (see step S63). Therefore, the display control unit 74 can display the image captured in the entire pixel region 113A on the sub screen 51A. At this time, since the image displayed on the sub screen 51A is small, a high value is set as the thinning rate of the area other than the enlargement target area.

FIG. 22 is a diagram showing another display example when the electronic zoom is performed in the third embodiment. In the example shown in FIG. 22, the images f31 and f32 of the eyes, which are the feature points (specifically, the images of the feature point regions 261,262 shown in FIG. 19A) are enlarged and displayed on the display panel 51. ing. Further, a sub screen 51A in a small area is provided at the lower right of the display panel 51. An image of the enlargement target area 270 is displayed on the sub screen 51A. Further, the frame 600 surrounding the face image C4 displayed on the sub screen 51A is displayed so that the user can recognize the position of the face image C4.

As described above, in the third embodiment, the display control unit 74 may perform the enlarged display of the electronic zoom (display of FIG. 21 and display of FIG. 22) in two stages according to the operation of the electronic zoom switch by the user. It is possible.

As described above, in the third embodiment, the first image whose display magnification is enlarged by the electronic zoom by the magnification changing unit 72 and the wide area image (image displayed on the sub screen 51A) including the first image are Includes a display control unit 74 that simultaneously displays both of the above. According to such a configuration, the user can confirm the wide area image even after the electronic zoom. Further, in the third embodiment, since the wide area image is the whole image captured by the imaging unit, the user can check the whole image even after the electronic zoom.

<Fourth Embodiment>
In the fourth embodiment, the digital camera 1 in the first embodiment described above is separated into an imaging device 1A and an electronic device 1B.

FIG. 23 is a block diagram showing a configuration of an imaging device and an electronic device according to a fourth embodiment. In the configuration shown in FIG. 23, the imaging device 1A is a device that images a subject. The image pickup device 1A includes a lens unit 10, an image pickup unit 20, an image processing unit 30, a work memory 40, an operation unit 55, a recording unit 60, and a first system control unit 75. Of the image pickup apparatus 1A, the configurations of 10, the imaging unit 20, the image processing unit 30, the work memory 40, the operation unit 55, and the recording unit 60 are the same as those shown in FIG. Therefore, the same reference numerals are given to the same configurations, and duplicate description will be omitted.

Further, the electronic device 1B is a device that displays an image (still image, moving image, live view image). The electronic device 1B includes a display unit 50 and a second system control unit (control unit) 70B. The configuration of the display unit 50 in the electronic device 1B is the same as the configuration shown in FIG. Therefore, the same reference numerals are given to the same configurations, and duplicate description will be omitted.

The first system control unit 75 has a first communication unit 75A. Further, the second system control unit 76 has a second communication unit 76B. The first communication unit 75A and the second communication unit 76B transmit and receive signals by wire or wirelessly to each other.

The configuration shown in FIG. 7 (first magnification changing unit 71, second magnification changing unit 72, imaging control unit 73, and display control unit 74) is provided in either the first system control unit 75 or the second system control unit 76. May be done. All the configurations shown in FIG. 7 may be provided in the first system control unit 75 or the second system control unit 76, and a part of the configurations shown in FIG. 7 is provided in the first system control unit 75. A configuration other than a part of the configuration shown in 7 may be provided in the second system control unit 76.

The image pickup device 1A is composed of, for example, a digital camera, a smartphone, a mobile phone, a personal computer or the like having an image pickup function and a communication function, and the electronic device 1B is, for example, a smartphone, a mobile phone, or a portable personal computer having a communication function. It consists of mobile terminals such as computers and portable tablets.

The first system control unit 75 shown in FIG. 23 is realized by a CPU (not shown) executing a process based on a control program. Further, the second system control unit 76 shown in FIG. 23 is realized by the CPU (not shown) executing the process based on the control program.

In the configuration shown in FIG. 23, the image processing unit 30 and the first system control unit 75 may be integrally configured. In this case, the system control unit having one or more CPUs performs the processing based on the control program to take on the functions of the image processing unit 30 and the functions of the first system control unit 75.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various changes or improvements can be made to the above embodiments without departing from the spirit of the present invention. Moreover, one or more of the conditions described in the above-described embodiment may be omitted. Such modifications, improvements, or omitted forms are also included in the technical scope of the present invention. Further, it is also possible to appropriately combine and apply the configurations of the above-described embodiments and modifications.

In each of the above-described embodiments, the drive unit 21 is configured to drive and control each pixel in the pixel region 113A and thin out the reading of the pixel signal according to the thinning rate. However, not limited to such a configuration, the drive unit 21 does not drive control the pixels of a predetermined ratio (ratio corresponding to the thinning ratio) among the pixels of the pixel area 113A, but drives and controls the other pixels. It may be configured. This configuration also has the same effect as the case where the reading of the pixel signal is thinned out at a predetermined thinning rate. Further, the signal processing chip 111 or the like may be configured to discard the pixel signal read from each pixel of the pixel area 113A. This configuration also has the same effect as the case where the reading of the pixel signal is thinned out at a predetermined thinning rate.

Further, in the first embodiment described above, the first magnification changing unit 71 changes the central region 200 according to the speed of the magnification fluctuation, but even if the central region 200 of a predetermined size is not changed. Good. Further, the first magnification changing unit 71 may specify the central region according to the user's selection.

Further, in the first embodiment described above, the second magnification changing unit 72 specifies the enlargement target area 220 according to the user's selection, but as described in the second embodiment, the subject detecting unit 32 An area corresponding to the detected main subject or feature point may be automatically designated as an enlargement target area. Further, in the second embodiment described above, the second magnification changing unit 72 automatically designates the area corresponding to the main subject and the feature point as the enlargement target area, but as described in the first embodiment. , The expansion target area may be specified according to the user's selection.

Further, in each of the above-described embodiments, the digital camera 1 has been mentioned as an example of the electronic device. However, the electronic device is not limited to the digital camera 1, and may be composed of a device such as a smartphone, a mobile phone, or a personal computer having an imaging function, for example. Further, in the digital camera 1 according to the first embodiment shown in FIG. 6, the display unit 50 may be provided outside the electronic device. In this case, each of the system control unit 70 and the display unit 50 is provided with a communication unit for transmitting and receiving signals (image data, control signals, etc.) by wire or wirelessly. Further, the image processing unit 30 and the system control unit 70 may be integrally configured. In this case, the system control unit having one CPU takes on the functions of the image processing unit 30 and the system control unit 70 by executing the processing based on the control program.

Further, in each of the above-described embodiments, the arrangement of the color filters 102 is a Bayer arrangement, but an arrangement other than this arrangement may be used. Further, the number of pixels forming the unit group 131 may include at least one pixel. Further, the block may also include at least one pixel. Therefore, it is possible to perform imaging under different imaging conditions for each pixel.

Further, in each of the above-described embodiments, the drive unit 21 may have a part or all of the configurations mounted on the imaging chip 113, or a part or all of the configurations may be mounted on the signal processing chip 111. Further, a part of the configuration of the image processing unit 30 may be mounted on the image pickup chip 113 or the signal processing chip 111. Further, a part of the configuration of the system control unit 70 may be mounted on the image pickup chip 113 or the signal processing chip 111.

Further, in the above-described first embodiment, the charge accumulation condition or the read condition is changed for each region, but the imaging condition other than the charge accumulation condition or the read condition (for example, the control parameter in the image processing unit 30) is changed for each region. ) May be changed according to the imaging situation.

Further, in each of the above-described embodiments, the optical zoom and the electronic zoom may be performed at the same time, and even in that case, the read control for each region during the optical zoom and the region during the electronic zoom as described above may be performed. It is configured to perform read control for each.

Further, in each of the above-described embodiments, the thinning rate and the like are changed so that the image resolution (image quality) does not change before and after the electronic zoom. However, the resolution of the image after the electronic zoom may be higher than that before the electronic zoom, or may be lower than that before the electronic zoom. Further, in each of the above-described embodiments, in the case of the optical zoom, the pixel signal is read out also in the peripheral region, but the pixel signal may not be read out in the peripheral region.

1,1B ... Digital camera 1A ... Image sensor, 20 ... Imaging unit, 30, 30A ... Image processing unit, 31 ... Image generation unit, 32 ... Subject detection unit (first detection unit), 33 ... Feature point detection unit (No. 1) 2 detection unit), 50 ... display unit, 51 ... display panel (display screen), 52 ... touch panel (selection unit), 70 ... system control unit, 70A ... first system control unit, 70B ... second system control unit, 71 ... 1st magnification change unit (magnification change unit), 72 ... 2nd magnification change unit (magnification change unit), 73 ... image pickup control unit, 74 ... display control unit, 100 ... image sensor

Claims (7)

A display unit having a first display area on which the first image is displayed and a second display area on which the second image is displayed, and
The first imaging region from which the first pixel signal for generating the first image data indicating the first image is read and the second pixel signal for generating the second image data indicating the second image are read out. An imaging unit having a second imaging region and
A magnification changing unit that changes the display magnification of the first image and the second image displayed on the display unit by optical zoom, and a magnification changing unit.
When the display magnification is changed by the magnification changing unit, the first pixel signal and the second pixel signal are read out under different charge accumulation conditions in the first imaging region and the second imaging region. A control unit that controls the imaging unit and
Electronic equipment equipped with. In the electronic device according to claim 1,
The first imaging region has a first photoelectric conversion unit that converts light into electric charges.
The second imaging region has a second photoelectric conversion unit that converts light into electric charges.
The control unit reads out a signal generated based on the charge converted by the first photoelectric conversion unit as the first pixel signal, and generates the signal based on the charge converted by the second photoelectric conversion unit. An electronic device that reads out the signal to be generated as the second pixel signal. In the electronic device according to claim 2.
The first imaging region has a plurality of the first photoelectric conversion units.
The second imaging region is an electronic device having a plurality of the second photoelectric conversion units. In the electronic device according to claim 3,
The first photoelectric conversion unit is arranged side by side in the first imaging region in the first direction and the second direction intersecting the first direction.
The second photoelectric conversion unit is an electronic device arranged side by side in the first direction and the second direction in the second imaging region. In the electronic device according to any one of claims 1 to 4.
The control unit is an electronic device that sets charge accumulation conditions in the first imaging region and charge accumulation conditions in the second imaging region based on a display magnification changed by the magnification changing unit. In the electronic device according to any one of claims 1 to 5.
The magnification changing unit is an electronic device that changes the display magnification of the first image and the second image by an electronic zoom. In the electronic device according to any one of claims 1 to 6.
The charge storage condition is an electronic device including a charge storage time or number of times, a frame rate, and a gain.

Images