JP6973599B2 - Electronics - Google Patents

Description

The present invention relates to an electronic device.

An imaging device equipped with an image sensor capable of setting imaging conditions for each screen area is known (see Patent Document 1).

Japanese Unexamined Patent Publication No. 2006-49361

In the prior art, there is a problem that it is difficult to set the imaging conditions for each area.

The electronic device according to one aspect of the invention includes an image pickup device having a plurality of pixels, a display unit that displays a plurality of images based on signals generated by at least a plurality of first pixels among the plurality of pixels, and the above-mentioned. Using an image selected from the plurality of images displayed on the display unit, a control unit that controls the plurality of first pixels and the plurality of second pixels among the plurality of pixels with different control parameters. Be prepared.

It is a block diagram which illustrates the structure of the camera by one Embodiment. It is sectional drawing of the laminated type image sensor. It is a figure explaining the pixel arrangement and the unit area of an image pickup chip. It is a figure explaining the circuit in a unit area. It is a block diagram which shows the functional structure of the image pickup element corresponding to the circuit of FIG. It is a figure which shows typically the image of the subject which is imaged on the image sensor of a camera. It is a figure which illustrates the setting screen of the imaging condition. It is a figure which illustrates the setting screen of the imaging condition. It is a figure which illustrates the display screen which displays the candidate image side by side. It is a figure which illustrates the display screen which displays the candidate image side by side. It is a figure which illustrates the display screen which displays the candidate image side by side. It is a figure which illustrates the display screen which displays the candidate image side by side. It is a figure which illustrates the display screen which displays the candidate image side by side. It is a flowchart explaining the flow of the process of setting the image pickup condition for each area and image-taking. It is a figure which illustrates the display screen by the modification 1. FIG. It is a figure which illustrates the display screen by the modification 1. FIG. It is a figure which illustrates the display screen by the modification 2. FIG. It is a figure which illustrates the display screen by the modification 2. FIG. It is a block diagram which illustrates the structure of the image pickup system by the modification 5. It is a figure explaining the supply of a program to a mobile device.

<Camera description>
FIG. 1 is a block diagram illustrating a configuration of an electronic device (for example, a camera 1) according to an embodiment. In FIG. 1, the camera 1 includes an image pickup optical system 31, an image pickup unit 32, an image processing unit 33, a work memory 34, a control unit 35, a liquid crystal monitor 36, an operation member 37, and a non-volatile memory 38. It has a recording unit 39 and a light emitting device 40.

The image pickup optical system 31 guides the light flux from the field of view to the image pickup unit 32. The image pickup unit 32 includes the image pickup element 100 and the drive unit 32a, and photoelectrically converts the image of the subject imaged by the image pickup optical system 31. The image pickup unit 32 can take an image under the same conditions over the entire image pickup surface of the image pickup element 100, or can take an image under different conditions for each region of the image pickup surface of the image pickup element 100. The details of the image pickup unit 32 will be described later. The drive unit 32a generates a drive signal necessary for causing the image pickup device 100 to perform storage control. An imaging instruction such as a charge accumulation time for the imaging unit 32 is transmitted from the control unit 35 to the driving unit 32a.

The image processing unit 33 cooperates with the work memory 34 to perform image processing on the image data captured by the image pickup unit 32. The image processing includes, for example, contour enhancement processing, gamma correction, white balance adjustment, display luminance adjustment, saturation adjustment, and the like. As will be described later, the image processing unit 33 may apply the same parameters or the like to the entire area of the image to perform image processing, or may apply different parameters or the like to each area of the image to perform image processing. can.

The work memory 34 is a memory for temporarily recording image data before and after image processing. The control unit 35 is composed of, for example, a CPU, and controls the entire operation by the camera 1. For example, a predetermined exposure calculation is performed based on the image signal acquired by the image pickup unit 32, and the charge accumulation time (exposure time) of the image pickup element 100 required for proper exposure, the aperture value of the image pickup optical system 31, the ISO sensitivity, and the like are determined. The exposure condition is determined and instructed to the drive unit 32a. Further, the image processing conditions for adjusting the saturation, contrast, sharpness, etc. are determined according to the image pickup scene mode set in the camera 1 and the type of the detected subject element, and are instructed to the image processing unit 33. The detection of the subject element will be described later. Furthermore, when the light emitting device 40 is set to allow light emission, the light emitting amount of the light emitting device 40 required for proper exposure is determined and instructed to the light emitting device 40.

The control unit 35 includes an object detection unit 35a, an area division unit 35b, a setting unit 35c, an image pickup condition setting unit 35d, an image pickup control unit 35e, an AF calculation unit 35f, and a display control unit 35g. .. These are realized by software by the control unit 35 executing the program stored in the non-volatile memory 38, but these may be configured by ASIC or the like.

The object detection unit 35a performs a known object recognition process, and from the image acquired by the image pickup unit 32, the object detection unit 35a includes a person (human face), an animal such as a dog or a cat (animal face), a plant, a bicycle, or an automobile. Detects subject elements such as vehicles such as trains, buildings, stationary objects, landscapes such as mountains and clouds, and specific predetermined objects.

The area division unit 35b divides (divides) the image pickup screen by the image pickup unit 32 into a plurality of areas including the detected subject element as described above. The setting unit 35c ranks a plurality of areas divided by the area division unit 35b. The method of ordering will be described later.

The imaging condition setting unit 35d sets imaging conditions for a plurality of regions divided by the region dividing unit 35b. The imaging conditions include the above exposure conditions (charge accumulation time, gain, ISO sensitivity, frame rate, etc.) and the above image processing conditions (for example, white balance adjustment parameters, gamma correction curve, display brightness adjustment parameters, saturation adjustment parameters, etc.). ) And. As for the imaging conditions, the same imaging conditions can be set for all of the plurality of regions, or different imaging conditions can be set for the plurality of regions.

The image pickup control unit 35e controls the image pickup unit 32 (image sensor 100) and the image processing unit 33 by applying the image pickup conditions set for each region by the image pickup condition setting unit 35d. As a result, it is possible for the image pickup unit 32 to perform imaging under different exposure conditions for each of a plurality of regions divided by the area division unit 35b, and for the image processing unit 33 to perform image capture under different exposure conditions. It is possible to perform image processing under different image processing conditions for each of a plurality of regions divided by 35b.

The AF calculation unit 35f performs an automatic focus adjustment (autofocus: AF) calculation for focusing on a corresponding subject at a predetermined position (referred to as a focus point) on the image pickup screen. The AF calculation unit 35f sends a drive signal for moving the focus lens of the image pickup optical system 31 to the in-focus position based on the AF calculation result.
The AF method may be either a contrast detection method or a phase difference detection method.

The display control unit 35g controls the display of the image to be displayed on the liquid crystal monitor 36. The display control unit 35g may, for example, display one image on the display surface of the liquid crystal monitor 36, or display a plurality of images side by side on the display surface of the liquid crystal monitor 36. Further, the display control unit 35g scrolls and displays the image being displayed on the display surface of the liquid crystal monitor 36 in response to the user's operation.

The liquid crystal monitor 36 reproduces and displays an image processed by the image processing unit 33 and an image read by the recording unit 39. The liquid crystal monitor 36 also displays an operation menu screen, a setting screen for setting imaging conditions, and the like.

The operation member 37 is composed of various operation members such as a release button and a menu button. The operation member 37 sends an operation signal corresponding to each operation to the control unit 35. The operation member 37 also includes a touch operation member provided on the display surface of the liquid crystal monitor 36.

The non-volatile memory 38 records a program or the like executed by the control unit 35. The recording unit 39 records image data or the like on a recording medium composed of a memory card or the like (not shown) in response to an instruction from the control unit 35. Further, the recording unit 39 reads out the image data recorded on the recording medium in response to an instruction from the control unit 35.

The light emitting device 40 is a photographing auxiliary light source that illuminates the subject. For example, when the light emission permission is set, the light emitting device 40 emits light with the amount of light instructed by the control unit 35 in response to the light emission instruction from the control unit 35.

<Explanation of stacked image sensor>
The stacked image sensor 100 provided in the camera 1 described above will be described. The laminated image sensor 100 is described in International Publication WO13 / 164915, which was previously filed and published by the applicant of the present application. FIG. 2 is a cross-sectional view of the stacked image sensor 100. The image pickup element 100 includes a back-illuminated image pickup chip 113 that outputs a pixel signal corresponding to incident light, a signal processing chip 111 that processes the pixel signal, and a memory chip 112 that records the pixel signal. The image pickup chip 113, the signal processing chip 111, and the memory chip 112 are laminated and electrically connected to each other by a conductive bump 109 such as Cu.

As shown in the figure, the incident light is mainly incident in the Z-axis plus direction indicated by the white arrow. In the present embodiment, in the image pickup chip 113, the surface on the side where the incident light is incident is referred to as a back surface (imaging surface). Further, as shown in the coordinate axes, the left direction of the paper surface orthogonal to the Z axis is defined as the X-axis plus direction, and the Z-axis and the front direction of the paper surface orthogonal to the X-axis are defined as the Y-axis plus direction. In some of the subsequent figures, the coordinate axes are displayed so that the orientation of each figure can be understood with reference to the coordinate axes of FIG.

An example of the image pickup chip 113 is a back-illuminated MOS image sensor. The PD layer 106 is arranged on the back surface side of the wiring layer 108. The PD layer 106 has a plurality of PDs (photodiodes) 104 arranged two-dimensionally and accumulating charges according to incident light, and a transistor 105 provided corresponding to the PD 104.

A color filter 102 is provided on the incident side of the incident light in the PD layer 106 via the passivation film 103. The color filter 102 has a plurality of types that transmit different wavelength regions from each other, and has a specific arrangement corresponding to each of the PD 104. The arrangement of the color filters 102 will be described later. A set of a color filter 102, a PD 104, and a transistor 105 forms one pixel.

A microlens 101 is provided on the incident side of the incident light in the color filter 102 corresponding to each pixel. The microlens 101 collects incident light toward the corresponding PD 104.

The wiring layer 108 has a wiring 107 that transmits a pixel signal from the PD layer 106 to the signal processing chip 111. The wiring 107 may have multiple layers, and may be provided with passive elements and active elements.

A plurality of bumps 109 are arranged on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the facing surfaces of the signal processing chip 111, and the image pickup chip 113 and the signal processing chip 111 are aligned by being pressurized or the like. The bumps 109 are joined together and electrically connected.

Similarly, a plurality of bumps 109 are arranged on the surfaces of the signal processing chip 111 and the memory chip 112 facing each other. These bumps 109 are aligned with each other, and the signal processing chip 111 and the memory chip 112 are pressurized, so that the aligned bumps 109 are joined to each other and electrically connected.

The bonding between the bumps 109 is not limited to the Cu bump bonding by solid phase diffusion, but the micro bump bonding by solder melting may be adopted. Further, for example, about one bump 109 may be provided for one block described later. Therefore, the size of the bumps 109 may be larger than the pitch of the PD 104. Further, in the peripheral region other than the pixel region in which the pixels are arranged, a bump larger than the bump 109 corresponding to the pixel region may be provided together.

The signal processing chip 111 has TSVs (Through Silicon Vias) 110 that connect circuits provided on the front and back surfaces to each other. The TSV 110 is preferably provided in the peripheral region. Further, the TSV 110 may also be provided in the peripheral area of the image pickup chip 113 and the memory chip 112.

FIG. 3 is a diagram illustrating a pixel arrangement of the image pickup chip 113 and a unit region 131. In particular, the state in which the image pickup chip 113 is observed from the back surface (imaging surface) side is shown. For example, 20 million or more pixels are arranged in a matrix in the pixel area. In the example of FIG. 3, 16 pixels of adjacent 4 pixels × 4 pixels form one unit region 131. The grid lines in the figure show the concept that adjacent pixels are grouped to form a unit region 131. The number of pixels forming the unit region 131 is not limited to this, and may be about 1000, for example, 32 pixels × 64 pixels, or more or less.

As shown in the partially enlarged view of the pixel region, the unit region 131 of FIG. 3 includes four so-called Bayer arrays including four pixels of green pixels Gb, Gr, blue pixels B, and red pixels R in the vertical and horizontal directions. The green pixels Gb and Gr are pixels having a green filter as the color filter 102, and receive light in the green wavelength band among the incident light. Similarly, the blue pixel B is a pixel having a blue filter as a color filter 102 and receives light in the blue wavelength band, and the red pixel R is a pixel having a red filter as the color filter 102 and has a red wavelength band. Receives light.

In the present embodiment, a plurality of blocks are defined so as to include at least one unit region 131 per block, and each block can control the pixels included in each block with different control parameters. That is, it is possible to acquire an imaging signal having different imaging conditions between the pixel group included in a certain block and the pixel group included in another block. Examples of control parameters are frame rate, gain, thinning rate, number of rows or columns to be added to add pixel signals, charge accumulation time or number of times, digitization bit number (word length), and the like. The image pickup element 100 can freely thin out not only in the row direction (X-axis direction of the image pickup chip 113) but also in the column direction (Y-axis direction of the image pickup chip 113). Further, the control parameter may be a parameter in image processing after acquiring an image signal from a pixel.

FIG. 4 is a diagram illustrating a circuit in the unit region 131. In the example of FIG. 4, one unit region 131 is formed by 9 pixels of adjacent 3 pixels × 3 pixels. As described above, the number of pixels included in the unit region 131 is not limited to this, and may be less than this or more. The two-dimensional positions of the unit region 131 are indicated by reference numerals A to I.

The pixel reset transistor included in the unit region 131 is configured to be individually on / off for each pixel. In FIG. 4, a reset wiring 300 for turning on / off the reset transistor of the pixel A is provided, and a reset wiring 310 for turning on / off the reset transistor of the pixel B is provided separately from the reset wiring 300. Similarly, the reset wiring 320 for turning on / off the reset transistor of the pixel C is provided separately from the reset wirings 300 and 310. Dedicated reset wiring for turning on / off each reset transistor is also provided for each of the other pixels D to the pixel I.

The transfer transistor of the pixel included in the unit region 131 is also configured to be individually on / off for each pixel. In FIG. 4, a transfer wiring 302 for turning on / off the transfer transistor of pixel A, a transfer wiring 312 for turning on / off the transfer transistor of pixel B, and a transfer wiring 322 for turning on / off the transfer transistor of pixel C are separately provided. Dedicated transfer wiring for turning on / off each transfer transistor is also provided from the other pixel D to the pixel I.

Further, the pixel selection transistor included in the unit region 131 is also configured to be individually on / off for each pixel. In FIG. 4, the selection wiring 306 for turning on / off the selection transistor of pixel A, the selection wiring 316 for turning on / off the selection transistor of pixel B, and the selection wiring 326 for turning on / off the selection transistor of pixel C are separately provided. Dedicated selection wiring for turning on / off each selection transistor is also provided for each of the other pixels D to the pixel I.

The power supply wiring 304 is commonly connected by pixels A to pixels A included in the unit area 131. Similarly, the output wiring 308 is commonly connected by pixels I from pixels A included in the unit area 131. Further, the power supply wiring 304 is commonly connected between the plurality of unit areas, but the output wiring 308 is individually provided for each unit area 131. The load current source 309 supplies current to the output wiring 308. The load current source 309 may be provided on the image pickup chip 113 side or the signal processing chip 111 side.

By turning on and off the reset transistor and transfer transistor of the unit region 131 individually, charge accumulation including charge accumulation start time, accumulation end time, and transfer timing is controlled for pixels A to I included in the unit region 131. can do. Further, by individually turning on and off the selection transistor of the unit region 131, the pixel signal of the pixel I can be output from each pixel A via the common output wiring 308.

Here, a so-called rolling shutter method is known in which charge accumulation is controlled in a regular order with respect to rows and columns for pixels A to I included in the unit region 131. When a pixel is selected for each row by the rolling shutter method and then a column is specified, pixel signals are output in the order of "ABCDEFGHI" in the example of FIG.

By configuring the circuit with the unit region 131 as a reference in this way, the charge accumulation time can be controlled for each unit region 131. In other words, pixel signals with different frame rates can be output between the unit regions 131. Further, in the image pickup chip 113, the unit region 131 included in the other blocks is rested while the unit region 131 included in some blocks is charged (imaging), so that the unit region 131 included in the other blocks is rested in a predetermined block of the image pickup chip 113. Only the image can be taken and the pixel signal can be output. Further, it is also possible to switch the block for accumulating charges (imaging) between frames (the target block for accumulating control), sequentially perform imaging with different blocks of the imaging chip 113, and output a pixel signal.

FIG. 5 is a block diagram showing a functional configuration of the image pickup device 100 corresponding to the circuit illustrated in FIG. The analog multiplexer 411 sequentially selects nine PD 104s forming the unit region 131, and outputs each pixel signal to the output wiring 308 provided corresponding to the unit region 131. The multiplexer 411 is formed on the image pickup chip 113 together with the PD 104.

The pixel signal output via the multiplexer 411 is CDS and A / by a signal processing circuit 412 formed on the signal processing chip 111 for performing correlated double sampling (CDS) analog / digital (A / D) conversion. D conversion is performed. The A / D converted pixel signal is passed to the demultiplexer 413 and stored in the pixel memory 414 corresponding to each pixel. The demultiplexer 413 and the pixel memory 414 are formed on the memory chip 112.

The arithmetic circuit 415 processes the pixel signal stored in the pixel memory 414 and hands it over to the image processing unit in the subsequent stage. The arithmetic circuit 415 may be provided on the signal processing chip 111 or the memory chip 112. Note that FIG. 5 shows the connections for one unit area 131, but in reality, these exist for each unit area 131 and operate in parallel. However, the arithmetic circuit 415 does not have to exist for each unit region 131. For example, even if one arithmetic circuit 415 processes sequentially while referring to the values of the pixel memory 414 corresponding to each unit region 131 in order. good.

As described above, the output wiring 308 is provided corresponding to each of the unit regions 131. Since the image pickup element 100 has an image pickup chip 113, a signal processing chip 111, and a memory chip 112 laminated, each chip can be oriented in the plane direction by using an electrical connection between the chips using the bump 109 for the output wiring 308. Wiring can be routed without making it large.

<Setting of imaging conditions for each area>
In the present embodiment, the imaging conditions can be set for each of a plurality of blocks in the image pickup device 100 (imaging chip 113). The control unit 35 makes the area divided by the area dividing unit 35b correspond to the block, and causes the control unit 35 to perform imaging under the imaging conditions set for each area.

For example, the user sets imaging conditions for a desired area while checking the live view image displayed on the display surface of the liquid crystal monitor 36. A live view image is a monitor image that is repeatedly imaged at a predetermined frame rate (for example, 60 fps).

When the user sets (changes) the imaging condition for a certain area, the control unit 35 causes the image processing unit 33 to perform image processing on the area of the live view image. The image processing performed here is an image processing for confirming the effect of changing the imaging conditions. For example, when a change is made to increase the brightness, image processing is performed to increase the display brightness displayed on the liquid crystal monitor 36. Further, for example, when the light emitting device 40 is changed to emit light, image processing is performed on a predetermined subject element (for example, a person) to increase the display brightness displayed on the liquid crystal monitor 36. Furthermore, for example, when a change is made to increase the saturation, an image process for increasing the saturation is performed on a predetermined subject element (for example, a flower) to be displayed on the liquid crystal monitor 36.
In this way, the user can confirm the effect of changing the imaging conditions on the image-processed live view image before the main imaging. The setting of imaging conditions for each region will be described in detail below.
It should be noted that the method is not limited to the method of performing image processing and displaying it on the LCD monitor 36 in order to confirm the effect of changing the image pickup condition operated for a certain area, but the change of the image pickup condition operated for a certain area is the same. A method may be used in which the live view image of the set (changed) image pickup condition is displayed on the liquid crystal monitor 36 by reflecting on the image pickup element corresponding to the area.

FIG. 6 is a diagram schematically showing an image of a subject imaged on the image sensor 100 of the camera 1. The camera 1 photoelectrically converts a subject image to acquire a live view image before an imaging instruction is given.

The control unit 35 first sets the same imaging conditions for the entire area of the imaging chip 113 (that is, the entire area of the imaging surface). Of the imaging conditions, the exposure condition is determined based on the exposure condition calculated so as to obtain an appropriate exposure according to the photometric value of the subject brightness, or the exposure condition manually set by the user. Of the image pickup conditions, the image processing conditions are determined to be image processing conditions prepared in advance so as to obtain standard image quality. Hereinafter, the imaging conditions determined in this way are referred to as reference conditions. Further, an image acquired by applying a reference condition is called a reference image.

An example of determining the reference condition will be described. The control unit 35 is set to the "program auto" mode, the "shutter speed priority auto" mode, the "aperture priority auto" mode, and the "manual" mode according to the operation signal from the exposure mode changeover switch constituting the operation member 37. Select one of them. Then, the control unit 35 calculates the above exposure conditions (charge accumulation time, gain, ISO sensitivity, frame rate, etc.) by performing a predetermined exposure calculation according to the selected exposure mode.

Specifically, in the "program auto" mode, the control unit 35 determines, for example, the aperture value, shutter speed (charge accumulation time), gain, and the like so that the average brightness in the screen becomes appropriate. In the "shutter speed priority auto" mode, the control unit 35 uses the shutter speed (charge accumulation time) set by the user to, for example, set the aperture value, gain, etc. so that the average brightness in the screen becomes appropriate. decide. In the "aperture priority auto" mode, the control unit 35 determines the shutter speed (charge accumulation time), gain, etc. so that the average brightness in the screen becomes appropriate, for example, using the aperture value set by the user. do. In the "manual" mode, the control unit 35 calculates the deviation from the proper exposure based on the aperture value and the shutter speed (charge accumulation time) set by the user.

In FIG. 6, an image including a person 61a, a car 62a, a bag 63a, a mountain 64a, and clouds 65a and 66a is imaged on the image pickup surface of the image pickup chip 113. The person 61a is holding the bag 63a with both hands. The car 62a is stopped at the right rear of the person 61a.

<Area division>
The control unit 35 divides the image acquired as the live view image into a plurality of areas based on the live view image as follows. First, the object detection unit 35a detects the subject element from the live view image. A known subject recognition technique is used to detect the subject element. In the example of FIG. 6, the object detection unit 35a detects a person 61a, a car 62a, a bag 63a, a mountain 64a, a cloud 65a, and a cloud 66a as subject elements.

Next, the area division unit 35b divides the image acquired as the live view image into the area including the subject element based on the detection means. In this example, the region including the person 61a is referred to as the first region 61, the region including the automobile 62a is referred to as the second region 62, the region including the bag 63a is referred to as the third region 63, and the region including the mountain 64a is referred to as the fourth region 64. The region including the cloud 65a will be referred to as the fifth region 65, and the region including the cloud 66a will be referred to as the sixth region 66.

The setting unit 35c ranks a plurality of areas divided by the area division unit 35b. When the setting unit 35c detects the following areas (1) to (4) among the areas divided by the area division unit 35b, the imaging condition is prioritized over the other areas divided by the area division unit 35b. Set as an area to be changed.
(1) Area in which a specific subject is detected according to the imaging scene mode The specific subject differs depending on the imaging scene mode of the set camera 1. For example, when the imaging scene mode setting of the camera 1 is set to the portrait mode, the setting unit 35c sets the region including the automobile 62a as the second region 62 with respect to the first region 61 including the person 61a. The region including the bag 63a is the third region 63, the region including the mountain 64a is the fourth region 64, the region including the clouds 65a is the fifth region 65, and the region including the clouds 66a is the imaging condition more than the sixth region 66. Is set as an area to be changed preferentially. In the case of the portrait mode, it is based on the idea that there is a high possibility that the imaging conditions of the first region 61 including the person 61a are set (changed).

For example, when the image pickup scene mode setting of the camera 1 is set to the close-up mode and the flower is detected as a subject element by the object detection unit 35a, the setting unit 35c sets the area including the flower. , Set as an area where the imaging conditions are preferentially changed over other areas. In the close-up mode, it is based on the idea that there is a high possibility of setting (changing) the imaging conditions of the area including flowers.

For example, when the image pickup scene mode setting of the camera 1 is set to the landscape mode and the mountain is detected as a subject element by the object detection unit 35a, the setting unit 35c sets the area including the mountain to the area including the mountain. It is set as an area where the imaging conditions are preferentially changed over other areas. In the landscape mode, it is based on the idea that there is a high possibility of setting (changing) the imaging conditions of the area including mountains.

For example, when the image pickup scene mode setting of the camera 1 is set to the beach mode and the sea is detected as a subject element by the object detection unit 35a, the setting unit 35c refers to a region including the sea. It is set as an area where the imaging conditions are preferentially changed over other areas. In the case of beach mode, it is based on the idea that there is a high possibility of setting (changing) the imaging conditions of the area including the sea.

For example, when the image pickup scene mode setting of the camera 1 is set to the sunset mode and the red sky is detected as a subject element by the object detection unit 35a, the setting unit 35c sets the area including the red sky. Therefore, it is set as an area where the imaging conditions are preferentially changed over other areas. In the case of the sunset mode, it is based on the idea that there is a high possibility of setting (changing) the imaging conditions of the area including the red sky.

For example, when the image pickup scene mode setting of the camera 1 is set to the cooking mode, and the object detection unit 35a detects the food served on the plate as the subject element, the setting unit 35c uses the plate and the food. The area including the above is set as an area in which the imaging conditions are preferentially changed over the other areas. In the cooking mode, it is based on the idea that there is a high possibility of setting (changing) the imaging conditions of the area including the dishes served on the plate.

The image pickup scene mode of the camera 1 may be set by the user operating the operation member 37, or may be set by the control unit 37 based on the subject element detected from the screen of the live view image. You may do it.

(2) Area including an in-focus subject element When an AF operation for focusing on a subject corresponding to the above focus point is performed, the setting unit 35c sets the in-focus subject element (that is, the focus point). The area including the subject element corresponding to the above) is determined as an area in which the imaging condition is preferentially changed over the area including the subject element out of focus.
If there is no in-focus subject element, the area including the subject element with the smallest amount of focus shift (that is, the subject element closest to the in-focus position from the position of the subject element) is imaged. It may be decided as an area where the condition is preferentially changed.

(3) The area setting unit 35c including the subject element having the minimum brightness or the maximum brightness selects the subject element having the darkest brightness or the subject element having the brightest brightness among the plurality of subject elements detected by the object detection unit 35a. The region to be included is determined as a region in which the imaging conditions are preferentially changed over the other regions. It is based on the idea that there is a high possibility that imaging conditions will be set (changed) for areas that include too dark subject elements such as blackouts and too bright subject elements such as overexposure.

(4) The area setting unit 35c including the subject element having a high ratio of the specific color component is the above-mentioned green pixel Gb, Gr, or blue pixel B among the plurality of subject elements detected by the object detection unit 35a. Alternatively, a region including a subject element in which the ratio of the color component of the red pixel R is higher than the ratio of the other color components is determined as a region in which the imaging conditions are preferentially changed over the other regions. Areas containing subject elements such as sky blue, sea blue, mountain green, and sunset red are based on the idea that imaging conditions are likely to be set (changed).

(5) Area including the subject element closest to the camera 1 The setting unit 35c preferentially changes the imaging condition to the area including the subject element closest to the camera 1 over the distant subject element. decide. For example, the control unit 35 acquires distance information from the camera 1 to the divided subject element, and determines a region including the subject element close to the camera 1. Further, when the distance information cannot be obtained, the control unit 35 may determine the area including the subject element having the largest proportion of the screen as the subject element closest to the camera 1.

When the setting unit 35c detects the following areas (6) to (10) among the areas divided by the area dividing unit 35b, the order is lower than the above-mentioned areas (1) to (5). Therefore, it is determined as an area for changing the imaging conditions.

(6) Area including a subject element having the smallest amount of focus shift When an AF operation for focusing on a subject corresponding to the focus point is performed, a plurality of setting units 35c are detected by the object detection unit 35a. Of the subject elements in the above, the region containing the subject element having the smallest amount of focus shift among the regions other than the region including the subject element in focus is set as the region in which the imaging conditions are preferentially changed over the other regions. Set. In general, the smaller the amount of focus shift, the higher the sharpness of the outline of the subject element. Therefore, it can be said that the subject element having a high degree of contour sharpness has a smaller amount of focus shift. The setting unit 35c sets a region including a subject element having a high contour sharpness as a region in which the imaging condition is preferentially changed over a region having a low contour sharpness.

(Example 1)
For example, in FIG. 6, it is assumed that the person 61a is in focus because the AF operation is performed on the first region 61 by the AF calculation unit. The setting unit 35c is the subject element having the smallest amount of defocus among the automobile 62a, the bag 63a, the mountain 64a, the cloud 65a, and the cloud 66a (that is, the position of the person 61a in focus). The third region 63 including the bag 63a) located at a close position is set as an region for preferentially changing the imaging conditions next to the first region 61 including the person 61a. It is based on the idea that there is a high possibility that the imaging conditions of the third region 63 including the bag 63a will be set (changed) next to the first region 61 including the person 61a.

(Example 2)
For example, in FIG. 6, it is assumed that the AF calculation unit is in focus between the person 61a and the automobile 62a. The setting unit 35c moves the subject element (that is, the position closest to the in-focus position) having the smallest amount of defocus among the automobile 62a, the bag 63a, the mountain 64a, the cloud 65a, and the cloud 66a. The second region 62 including a certain automobile 62a) is set as a region for preferentially changing the imaging conditions next to the first region 61 including the person 61a. It is based on the idea that there is a high possibility that the imaging conditions of the second region 62 including the automobile 62a are set (changed) next to the first region 61 including the person 61a.

(7) The area setting unit 35c including the subject element having a small distance difference sets the order based on the difference in the distance from the camera 1 to the subject element. For example, in FIG. 6, it is assumed that the person 61a is in focus due to the AF operation performed on the first region 61. In this case, the setting unit 35c has a distance from the camera 1 of the automobile 62a, the bag 63a, the mountain 64a, the cloud 65a, and the cloud 66a close to the distance from the camera 1 to the person 61a (distance difference). The area containing the subject element (smaller) is set as the area for preferentially changing the imaging conditions.

(8) Region including a subject element having a large luminance difference For example, in FIG. 6, it is assumed that the person 61a is in focus due to the AF operation performed on the first region 61. The setting unit 35c covers a region including a subject element having the largest brightness difference between the automobile 62a, the bag 63a, the mountain 64a, the cloud 65a, the cloud 65a, and the cloud 66a. Next to the first region 61 including 61a, the region is set as a region for preferentially changing the imaging conditions. For example, when the brightness difference of the mountain 64a from the person 61a is the maximum, the setting unit 35c preferentially gives the fourth region 64 including the mountain 64a the imaging condition next to the first region 61 including the person 61a. Set as the area to be changed. The region including the subject element having a large luminance difference with the person 61a is based on the idea that the imaging conditions are likely to be set (changed).

(9) Area containing a large subject element When the difference in the amount of focus shift between the areas including the subject element is poor and the difference in brightness between the areas is also poor, the setting unit 35c may be used for another subject. An area containing a subject element larger than the element, in other words, an area occupying a larger area in the screen is set as an area for preferentially changing the imaging conditions.

(10) Area including a subject element near the center of the screen When the difference in the amount of focus shift between the areas including the subject element is poor and the difference in brightness between the areas is also poor, the setting unit 35c may be used. , The area including the subject element closer to the center of the screen is set as the area for preferentially changing the imaging conditions.

In the present embodiment, one of the above (6) to (10) may be selected in advance by the user operating the operation member 37. Further, a plurality of the above (6) to (10) may be combined.

By the above procedure, the setting unit 35c ranks the second region 63 to the sixth region 66. For example, the second region 62 including the automobile 62a, which is a large subject element, is ranked second, the third region 63 including the bag 63a is ranked third, the fourth region 64 including the mountain 64a is ranked fourth, and the cloud 65a is included. The fifth region 65 is the fifth place, and the sixth region 66 including the cloud 66a is the sixth place.

The setting unit 35c treats the regions that cannot be ranked even after the processes according to (6) to (10) as having the same rank.

<Setting of imaging conditions for the area>
When the screen is divided into a plurality of areas by the area dividing unit 35b, the control unit 35 causes the liquid crystal monitor 36 to display a setting screen as illustrated in FIG. 7. In FIG. 7, the live view image 60a is displayed, and the icon 71 and the icon 72 are displayed on the live view image 60a. The live view image 60a is a reference image acquired by applying the above-mentioned reference conditions.

The icon 71 is an operation icon for selecting “luminance” as an example of setting items for imaging conditions. The icon 72 is an operation icon for selecting “saturation” as an example of setting items for imaging conditions. The operation icon for selecting the setting item of the imaging condition may be added as appropriate in addition to those illustrated in FIG. 7. Further, the setting item of the imaging condition may be selected and determined by the photographer, or may be determined and determined by the control unit 35 of the camera 1 from the detected subject image.

In FIG. 7, of the first to sixth regions, the region in which the outline is emphasized and displayed (thick display, bright display, display in different colors, display by broken line, blinking display, etc.) is set (changed) for imaging conditions. ) Indicates the target area. In the example of FIG. 7, since the live view image 60a in which the outline of the first region 61 is emphasized is displayed, the first region 61 is the target for setting (changing) the imaging conditions. By emphasizing the outline of the first region 61, it is possible to clearly show the user the region for which the imaging condition is set (changed). In the setting screen of FIG. 7, the first target for setting (changing) the imaging condition is the area in which the order given by the setting unit 35c is the first.

For example, in the camera 1 capable of touch operation, when the icon 71 is tapped by the user, the control unit 35 sets (changes) the highlighted area (first area 61). Select brightness as a setting item for imaging conditions.
In the following description, the camera 1 will be described on the premise of a touch operation, but the setting items of the imaging conditions may be selected by operating the buttons and the like constituting the operation member 37.

<When changing the brightness>
In the case of the setting screen illustrated in FIG. 7, the control unit 35 that selects the brightness as the setting item of the imaging condition determines the luminance of the first region 61 that is the target of setting (changing) the imaging condition in the live view image. The imaging conditions corresponding to the first region 61 are determined so as to be higher (or lower) than the brightness of the reference image. For the regions (regions other than the first region 61) that are not the target of setting (changing) the imaging conditions, the set imaging conditions are maintained so as to maintain the brightness of the reference image.
Of the regions that are not subject to the setting (change) of the imaging conditions, in the vicinity of the boundary with the first region 61, the balance of the entire image is taken into consideration so as to blur the discontinuity of the imaging conditions in the vicinity of the boundary. The imaging conditions may be changed.

When increasing the brightness of the first region 61, the imaging condition setting unit 35d may, for example, make the charge accumulation time longer than the charge accumulation time of the reference condition among the imaging conditions of the first region 61, or increase the gain. The imaging condition for increasing the brightness is determined by increasing the gain as compared with the gain of the reference condition.

Further, when the brightness of the first region 61 is lowered, the imaging condition setting unit 35d may shorten the charge accumulation time of the imaging conditions of the first region 61, for example, as compared with the charge accumulation time of the reference condition, or gain. Is lower than the gain of the above reference condition, so that the imaging condition for lowering the brightness is determined.

When the imaging condition setting unit 35d determines the imaging conditions for increasing or decreasing the brightness, the imaging conditions of multiple stages (for example, 8 stages) so that the brightness is gradually different from the brightness obtained under the reference conditions. To decide. However, the upper limit when increasing the luminance is a value at which the high luminance region in the first region 61 of the live view image is not saturated (so-called whiteout does not occur). Further, the lower limit when the brightness is lowered is a value at which the low brightness region in the first region 61 of the live view image is not too dark (so-called black crushing is not performed).

The image pickup condition setting unit 35d determines the multi-step image pickup conditions for increasing or decreasing the brightness of the first region 61 as described above, and sets the multi-step image pickup conditions determined for the brightness in the image processing unit 33. The reason for setting the image processing unit 33 is to obtain a multi-stage candidate image, which will be described later, by performing image processing on the live view image.

<When changing the saturation>
In the case of the setting screen illustrated in FIG. 7, the control unit 35 that selects the saturation as the setting item of the imaging condition has the saturation of the first region 61 that is the target of setting (changing) the imaging condition in the live view image. The imaging condition corresponding to the first region 61 is determined so as to increase (or decrease) the saturation of the reference image. For the regions (regions other than the first region 61) that are not the target of setting (changing) the imaging conditions, the set imaging conditions are maintained so as to maintain the saturation of the reference image.
Of the regions that are not subject to the setting (change) of the imaging conditions, in the vicinity of the boundary with the first region 61, the balance of the entire image is taken into consideration so as to blur the discontinuity of the imaging conditions in the vicinity of the boundary. The imaging conditions may be changed.

When increasing the saturation of the first region 61, the imaging condition setting unit 35d receives a signal read from, for example, the green pixel Gb, Gr, or the blue pixel B, or the red pixel R among the imaging conditions of the first region 61. By increasing the value of to relative to the signal values of other color components, the imaging conditions for making a specific color vivid are determined.

Further, when the saturation of the first region 61 is reduced, the imaging condition setting unit 35d is read from, for example, the green pixel Gb, Gr, the blue pixel B, and the red pixel R among the imaging conditions of the first region 61. By relatively lowering the value of the signal of the color component that protrudes from the other color components of the signal, the imaging conditions for suppressing the vividness of a specific color are determined.

When determining the imaging conditions for increasing or decreasing the saturation, the imaging condition setting unit 35d captures images in multiple stages (for example, 8 stages) so that the saturation is gradually different from the saturation obtained under the reference conditions. Determine the conditions. However, the upper limit when increasing the saturation is a value at which the high saturation (protruding color component) signal in the first region 61 of the live view image is not saturated. Further, the lower limit when the saturation is lowered is a value that does not suppress the vividness until the color in the first region 61 of the live view image becomes achromatic.

The image pickup condition setting unit 35d determines the multi-step imaging conditions for increasing or decreasing the saturation of the first region 61 as described above, and sets the multi-step imaging conditions determined for the saturation in the image processing unit 33. do. The reason for setting the image processing unit 33 is to obtain a multi-stage candidate image, which will be described later, by performing image processing on the live view image.

For example, when the image pickup scene mode setting of the camera 1 is set to the portrait mode, the image pickup condition setting unit 35d is so as to reduce the saturation of the person with respect to the first region 61 including the person 61a. The imaging conditions corresponding to one region 61 are determined. Further, the image pickup condition setting unit 35d increases the saturation of the mountain with respect to the fourth area 64 in the area including the mountain 64a as the background (for example, the value of the signal read from the green pixels Gr and Gb is used as another value). The imaging conditions corresponding to the fourth region 64 are determined so as to be relatively higher than the value of the signal of the color component). In the case of portrait mode, it is based on the idea that there is a high possibility that the background will be vivid and the person 61a will be complemented.

For example, when the image pickup scene mode setting of the camera 1 is set to the cooking mode and the object detection unit 35a detects the food placed on the plate as the subject element, the image pickup condition setting unit 35d uses the plate. And the imaging conditions are determined so as to increase the saturation for the area including the dish.

For example, when the image pickup scene mode setting of the camera 1 is set to the autumn leaves mode and the red autumn leaves are detected as the subject element by the object detection unit 35a, the image pickup condition setting unit 35d is the area including the red autumn leaves. On the other hand, the imaging condition is determined so as to increase the saturation (for example, the value of the signal read from the red pixel R is relatively increased with respect to the value of the signal of another color component). The same applies when the image pickup scene mode setting of the camera 1 is the sunset mode.

For example, when the image pickup scene mode setting of the camera 1 is set to the beach mode and the sea is detected as a subject element by the object detection unit 35a, the image pickup condition setting unit 35d is an area including the sea and the sky. On the other hand, the imaging conditions are determined so as to increase the saturation (for example, the value of the signal read from the blue pixel B is relatively increased compared to the value of the signal of another color component).

<For other live view images>
FIG. 8 is a diagram illustrating a setting screen of the liquid crystal monitor 36 that displays a live view image 60b different from the live view image 60a. For example, when the image pickup scene mode setting of the camera 1 is set to the close-up mode and the object detection unit 35a detects a red flower as a subject element, the setting unit 35c relates to a region including the red flower. Place the order first. As a result, the contour of the first region 61b is highlighted and displayed as a target region for setting (changing) the imaging conditions. As described above, in the close-up mode, it is based on the idea that there is a high possibility that the imaging conditions of the region including flowers are set (changed) with priority over other regions.
Since the icon 71 and the icon 72 in FIG. 8 are the same as in the case of the setting screen illustrated in FIG. 7, the description thereof will be omitted.

In the case of the setting screen illustrated in FIG. 8, the control unit 35 that selects the saturation as the setting item of the imaging condition relates to the first region 61b that is the target of setting (changing) the imaging condition in the live view image. , The imaging condition is determined so as to increase the saturation (for example, the value of the signal read from the red pixel R is relatively increased with respect to the value of the signal of another color component). By increasing the value of the signal read from the red pixel R relatively higher than the value of the signal of another color component, the imaging condition for making the color of the red flower vivid is determined.

As illustrated above, the image pickup condition setting unit 35d is a signal read from the green pixels Gb, Gr, the blue pixel B, and the red pixel R for each color of the subject element in the area divided by the area division unit 35b. By changing the signal value of the color component that is more prominent than the other color components, the imaging conditions in which the saturation is changed are determined.

<Display of candidate images>
The image processing unit 33 is included in the first region 61 of the reference images acquired by applying the reference conditions when the multi-step imaging conditions for the above-mentioned luminance are set by the image pickup control unit 35e. A plurality of images (8 images in the case of the above 8 stages) in which the brightness of the image is different in multiple stages are generated. In the present embodiment, the plurality of images thus generated are referred to as candidate images. The plurality of candidate images generated by the image processing unit 33 are temporarily recorded in the work memory 34.

Further, in the image processing unit 33, when the multi-step imaging conditions for the saturation described above are set by the imaging condition setting unit 35d, the saturation of the image in the first region 61b of the reference image is changed. Multiple images (8 images in the case of the above 8 stages) that are different in multiple stages are generated. These plurality of images are also called candidate images. The plurality of candidate images generated by the image processing unit 33 are temporarily recorded in the work memory 34.

When the plurality of candidate images described above are recorded in the work memory 34, the display control unit 35g displays the candidate images side by side on the display surface of the liquid crystal monitor 36 as shown in FIG. 9 to 12 are diagrams illustrating the screen of the liquid crystal monitor 36, which displays candidate images side by side.

When the above-mentioned candidate images having different luminancees in multiple stages are recorded in the work memory 34, the display control unit 35g causes the display surface of the liquid crystal monitor 36 to display the candidate images having these luminances side by side. Further, when the candidate images having different saturations in multiple stages described above are recorded in the work memory 34, the display control unit 35g arranges the candidate images for these saturations on the display surface of the liquid crystal monitor 36. Display.

In FIG. 9, four candidate images (candidate images 60-1 to 60-4) out of eight candidate images are displayed. The brightness (or saturation) of the candidate image 60-2 is one step higher than that of the candidate image 60-1. The brightness (or saturation) of the candidate image 60-3 is one step higher than that of the candidate image 60-2. The brightness (or saturation) of the candidate image 60-4 is one step higher than that of the candidate image 60-3.

The left icon 73 is displayed at the left position of the candidate image 60-1, and the right icon 74 is displayed at the right position of the candidate image 60-4. For example, when the user taps the right icon 74 with the finger 80 on the display screen of FIG. 9, the display control unit 35g has four candidate images (candidate images) out of eight candidate images as illustrated in FIG. 60-2 to the candidate image 60-5) are displayed. According to FIG. 10, the candidate image 60-1 displayed in FIG. 9 is no longer displayed, and the candidate image 60-5 that was not displayed in FIG. 9 is newly displayed. The brightness (or saturation) of the candidate image 60-5 is one step higher than that of the candidate image 60-4.

Although not shown, for example, when the user taps the left icon 73 with the finger 80 on the display screen of FIG. 10, the display control unit 35g is 4 out of 8 candidate images as illustrated in FIG. Two candidate images (candidate images 60-1 to 60-4) are displayed. That is, the candidate image 60-5 displayed in FIG. 10 is no longer displayed, and the candidate image 60-1 not displayed in FIG. 10 is newly displayed.

FIG. 11 is a diagram illustrating a screen of a liquid crystal monitor 36 for displaying four candidate images (candidate images 60-3 to 60-6) out of eight candidate images. The brightness of the candidate image 60-6 is one step higher than that of the candidate image 60-5.

For example, when the user slides the display screen from top to bottom in the direction of the arrow with the finger 80 on the display screen of FIG. 11, the display control unit 35g has eight candidate images as illustrated in FIG. Four of the candidate images (candidate images 60-1 to 60-4) are displayed. According to FIG. 12, the candidate images 60-4 and 60-5 displayed in FIG. 11 are no longer displayed, and the candidate images 60-1 and 60-2 that were not displayed in FIG. 11 are newly replaced. Is displayed.

In this way, the user can confirm a plurality of candidate images. Further, the user can display and confirm other candidate images that are not displayed on the liquid crystal monitor 36 by tapping the left icon 73 or the right icon 74 or sliding with the finger 80.

FIG. 13 is a diagram illustrating a screen of a liquid crystal monitor 36 that displays candidate images having different saturations in multiple stages side by side. In FIG. 13, four candidate images (candidate images 160-1 to 160-4) out of eight candidate images are displayed. The saturation of the candidate image 160-2 is one step higher than that of the candidate image 160-1. The candidate image 160-3 has one step higher saturation than the candidate image 160-2. The candidate image 160-4 has one step higher saturation than the candidate image 160-3. Since the left icon 73 and the right icon 74 are the same as those illustrated in FIGS. 9 and 10, the description thereof will be omitted.

<Selection operation>
The user selects any candidate image from a plurality of candidate images. For example, when the user double-tap the position where any candidate image is displayed among the plurality of candidate images on the screens exemplified in FIGS. 9 to 12 with the finger 80, the control unit 35 is then instructed to take an image. When this is performed, the image pickup conditions to be applied to the first region 61 are determined as follows, and the process returns to the state of displaying the setting screen as illustrated in FIG. 7 on the liquid crystal monitor 36.

That is, the control unit 35 (imaging control unit 35e) sets the imaging conditions applied to the first region 61 of the candidate image that has been double-tapped to the imaging unit 32 or the image processing unit 33. The reason for setting the image pickup unit 32 is that the brightness can be changed by changing the exposure condition of the image pickup element 100 at the time of the next imaging. The reason for setting the image processing unit 33 is to change the saturation by changing the image processing conditions of the image processing unit 33 at the time of the next imaging.

Specifically, when one of the plurality of candidate images for brightness is double-tapped, the imaging condition setting unit 35d first sets the exposure condition for obtaining the brightness corresponding to the double-tapped candidate image. It is determined as an image pickup condition (exposure condition) to be applied to the region 61, and the image pickup control unit 35e sets the condition in the image pickup unit 32.

Further, when one of the plurality of candidate images for saturation is double-tapped on the screen illustrated in FIG. 13, the image pickup condition setting unit 35d obtains the saturation corresponding to the double-tapped candidate image. The image processing condition for this is determined as an image pickup condition (image processing condition) to be applied to the first region 161, and the image pickup control unit 35e sets the condition in the image processing unit 33.

The user can perform an operation of selecting an arbitrary candidate image from a plurality of candidate images related to luminance and then an operation of selecting an arbitrary candidate image from a plurality of candidate images related to saturation. Further, the user can perform an operation of selecting an arbitrary candidate image from a plurality of candidate images related to saturation and then an operation of selecting an arbitrary candidate image from a plurality of candidate images related to luminance.

Further, the user selects an arbitrary candidate image from a plurality of candidate images in terms of brightness or saturation even in an area other than the first area 61, that is, an area in which the order given by the setting unit 35c is second or lower. It can be performed. For example, when the user taps an area other than the first area 61 with a finger while the setting screen illustrated in FIG. 7 is displayed on the liquid crystal monitor 36, the control unit 35 replaces the first area 61 with the control unit 35. The tapped area is highlighted. The imaging condition setting unit 35d sets (changes) the imaging condition in the highlighted area. Subsequent setting operations are the same as in the case of the first region 61 described above.

In addition, without the user tapping an area other than the first area 61, the control unit 35 sets (changes) the imaging conditions in order of the areas in the second or lower rank given by the setting unit 35c. It may be switched as a target area.

In the present embodiment, an example of highlighting the outline of the region (first region 61 in the above example) that is the target of setting (changing) the imaging conditions has been described, but instead of highlighting the contour, the contrast of the entire target region is displayed. It may be raised and displayed, or the entire target area may be blinked. Further, the target area may be surrounded by a frame. The display of the frame surrounding the target area may be a double frame or a single frame, and the display mode such as the line type of the surrounding frame may be appropriately changed. Further, the control unit 35 may display an arrow or the like in the vicinity of the target area to indicate a region to be set as an imaging condition. The control unit 35 may display a dark image other than the target area for which the imaging condition is set (changed), or may display the image with a low contrast other than the target area.

After the imaging conditions for the regions described above are set, when the release button (not shown) constituting the operating member 37 or the display instructing the start of imaging is operated, the control unit 35 controls each of the divided regions. Imaging is performed under the imaging conditions set for the above, image-processed image data is generated, and the image data is recorded on a recording medium composed of a memory card (not shown) or the like.

<Explanation of flowchart>
FIG. 14 is a flowchart illustrating a flow of processing in which imaging conditions are set for each region and imaging is performed. When the main switch of the camera 1 is turned on, the control unit 35 activates a program that executes the process shown in FIG. In step S10, the control unit 35 causes the liquid crystal monitor 36 to start the live view display, and proceeds to step S20.

Specifically, the control unit 35 instructs the imaging unit 32 to start acquiring the live view image, and sequentially displays the acquired live view image on the liquid crystal monitor 36. As described above, at this time, the same imaging conditions are set for the entire area of the imaging chip 113, that is, the entire screen.
When the AF operation is set during the live view display, the control unit 35 (AF calculation unit 35f) controls the AF operation for focusing on the subject element corresponding to the predetermined focus point. If the AF operation is not set during the live view display, the control unit 35 (AF calculation unit 35f) performs the AF operation when the AF operation is later instructed.

In step S20, the control unit 35 (object detection unit 35a) detects the subject element from the live view image and proceeds to step S30. In step S30, the control unit 35 (area division unit 35b) divides the screen of the live view image into an area including the subject element, and proceeds to step S40.

In step S40, the control unit 35 determines the above-mentioned reference condition based on the setting state of the camera 1 or the state of the subject. That is, the same imaging conditions are set for the entire screen, and the process proceeds to step S50.

In step S50, the control unit 35 acquires a live view image (reference image) to which the reference condition is applied, and proceeds to step S60. In step S60, the control unit 35 selects an area to be set (changed) for the imaging condition from the areas divided by the area dividing unit 35b. For example, the control unit 35 selects in order from the region in which the order given by the setting unit 35c is the first.

In step S70, the control unit 35 determines whether or not an operation for displaying the candidate image has been performed. As described above, when the icon 71 or 72 of FIG. 7 (or FIG. 8) is tapped by the user, the control unit 35 positively determines step S70 and proceeds to step S80, and taps both the icon 71 and 72. If not, step S70 is negatively determined and the process proceeds to step S120.

In step S80, the control unit 35 determines the imaging conditions to be set (changed). That is, the control unit 35 determines the setting item corresponding to the icon 71 or 72 tapped by the user, and proceeds to step S90.

In step S90, the control unit 35 (imaging condition setting unit 35d) acquires a plurality of candidate images for the setting items determined in step S80 and proceeds to step S100. In step S100, the control unit 35 displays a plurality of candidate images on the display surface of the liquid crystal monitor 36 and proceeds to step S110.

In step S110, when the user performs the above selection operation (for example, double tapping), the control unit 35 corresponds to the candidate image double-tapped for the area to be set (changed) in the imaging condition. The image pickup condition is determined, the condition is set in the image pickup unit 32 or the image processing unit 33, and the process proceeds to step S120.
The display transition of the liquid crystal monitor 36 and the setting of the imaging conditions according to the user operation in step S110 are as described above.

In step S120, the control unit 35 determines whether or not there is an imaging instruction. When the release button (not shown) constituting the operation member 37 or the display instructing the image pickup is operated, the control unit 35 positively determines step S120 and proceeds to step S130. If the image pickup instruction is not given, the control unit 35 makes a negative determination in step S120 and returns to step S70.

In step S130, the control unit 35 performs the predetermined main imaging process. That is, the image pickup control unit 35e controls the image pickup unit 32 so as to take an image under the image pickup conditions set for each of the above regions, and the image processing unit 33 performs predetermined image processing on the image data obtained by the image pickup. Give. Then, the recording unit 39 records the image data on a recording medium (not shown).

In step S140, the control unit 35 determines whether or not the end operation has been performed. When the end operation is performed, the control unit 35 positively determines step S140 and ends the process according to FIG. If the end operation is not performed, the control unit 35 negatively determines step S140 and returns to step S20. When returning to step S20, the control unit 35 repeats the above-mentioned process.

According to the above-described embodiment, the following effects can be obtained.
(1) The camera 1 includes an image pickup unit 32 that captures a subject image by an image pickup element 100 configured to be able to set imaging conditions for a plurality of regions on the imaging surface, and parameters for changing the imaging conditions for each region. It includes a control unit 35 that determines based on the subject element detected from the captured live view image. As a result, the camera 1 automatically determines the parameters for changing the imaging conditions for each region, so that the user can easily set the imaging conditions for each region.

(2) The control unit 35 determines the parameters based on the subject element and the set imaging mode. As a result, appropriate parameters are automatically determined according to the type of subject element (for example, person, mountain, flower, etc.) and the imaging mode (for example, portrait mode, landscape mode, etc.), so that the user can use it. It becomes easier to set the imaging conditions for each area.

(3) The camera 1 generates a candidate image in which the imaging conditions are changed by using the parameters determined by the control unit 35 in at least one region among the plurality of regions with respect to the captured live view image. A processing unit 33 is provided. As a result, the user can confirm the effect of changing the imaging conditions for the area by the candidate image obtained by performing image processing on the live view image before the main imaging.

(4) The control unit 35 determines a plurality of parameters, and the image processing unit 33 stepwise determines the imaging conditions for the captured live view image using a plurality of different parameters determined by the control unit 35. Generate multiple modified candidate images. As a result, the user can compare the effect of changing the imaging conditions with a plurality of candidate images having different imaging conditions in stages.

(5) The imaging condition is brightness or saturation, and the image processing unit 33 generates a plurality of candidate images whose brightness or saturation is changed stepwise. This allows the user to compare the effect of changing the luminance (or saturation) with a plurality of candidate images having stepwise differences in luminance (or saturation).

(6) The image processing unit 33 generates the captured live view image as a reference image before changing the imaging conditions, and also generates a candidate image after changing the imaging conditions. This allows the user to compare the effect of changing the imaging conditions with respect to the area compared to before changing the imaging conditions.

(7) The image processing unit 33 generates a reference image, an image having a high brightness with respect to the reference image, and / or an image having a low brightness with respect to the reference image. This allows the user to compare the effect of changing the brightness with respect to the area compared to before changing the brightness.

(8) The image processing unit 33 generates a reference image, an image having high saturation with respect to the reference image, and / or an image having low saturation with respect to the reference image. This allows the user to compare the effect of changing the saturation on the area with that before changing the saturation.

(9) Since the liquid crystal monitor 36 for displaying the image generated by the image processing unit 33 is provided, the user can visually compare the effect when the imaging conditions are changed with respect to the area. Become.

(10) When the operation of selecting the candidate image displayed on the liquid crystal monitor 36 is performed, the candidate image is generated for the region of the imaging surface of the imaging unit 32 corresponding to the region where the candidate image is generated. A control unit 35 for setting imaging conditions corresponding to the above parameters is provided. This makes it easier for the user to set the imaging conditions for each area.

(11) The display control unit 35g is provided, in which a region for displaying an image generated by changing the imaging conditions by the image processing unit 33 among the plurality of regions is displayed on the liquid crystal monitor 36 in a manner different from that of the other regions. This makes it possible to clearly indicate to the user the area to be set (changed).

The following modifications are also within the scope of the present invention, and one or more of the modifications can be combined with the above-described embodiment.

(Modification 1)
In the above description, as illustrated in FIGS. 9 and 13, an example in which the display control unit 35g displays a plurality of candidate images side by side on the display surface of the liquid crystal monitor 36 has been described. Instead, a reference image is displayed on the display surface of the liquid crystal monitor 36, and only the image in the region (first region 61 in the above example) to be set (changed) in the imaging conditions of the reference image is displayed. , You may replace it with the image of the corresponding area in another candidate image and display it.

15 and 16 are diagrams illustrating a display screen according to the first modification. In the first modification, when the plurality of candidate images described above are stored in the work memory 34, the display control unit 35g displays a reference image (reference image) under the reference condition on the display surface of the liquid crystal monitor 36 as shown in FIG. 60-1) is displayed. At this time, the image in the first region 61 is the reference image 61-1.
Here, the point that the outline of the first region 61, which is the target of setting (changing) the imaging conditions, is highlighted is the same as in the case of the above embodiment.

In FIG. 15, the left icon 73 is displayed at the left side position of the first area 61, and the right icon 74 is displayed at the right side position of the first area 61. For example, when the user taps the right icon 74 with the finger 80 on the display screen of FIG. 15, the display control unit 35g selects only the image in the first region 61 as the candidate image 61-as illustrated in FIG. The composite image replaced with 2 is displayed. The background other than the first region 61 remains as the reference image 60-1.

The candidate image 61-2 is an image whose brightness (or saturation) is one step higher than that of the candidate image 61-1. In this way, the display control unit 35g reads out the necessary candidate images 60-2 from the eight candidate images temporarily stored in the work memory 34 in response to the tap operation of the right icon 74, and the read out candidates. Only the candidate image 61-2 in the first region 61 of the images 60-2 is extracted and used for the composite display illustrated in FIG.

Although not shown, for example, when the user taps the left icon 73 with the finger 80 on the display screen of FIG. 16, the display control unit 35g is an image in the first region 61 as illustrated in FIG. Is replaced with the reference image 61-1 to display a composite image. The background other than the first region 61 remains as the reference image 60-1.

According to the first modification, the user can confirm the candidate image for the first region 61 that is the target of setting (changing) the imaging condition. Further, the user can display and confirm another candidate image in the first area 61 by tapping the left icon 73 or the right icon 74.

In the first modification, when the user double-taps any position on the screen with the finger 80 while an arbitrary candidate image is displayed, the control unit 35 (imaging condition setting unit 35d) is at that time. The imaging conditions applied to the candidate images 61-n (n is a natural number of 1 to 8) displayed in the first region 61 are applied to the first region 61 when an imaging instruction is next given. It is determined as the image pickup condition to be performed, and the process returns to the state where the setting screen illustrated in FIG. 7 is displayed on the liquid crystal monitor 36.

It should be noted that the user can perform an operation of selecting an arbitrary candidate image from a plurality of candidate images related to saturation and then an operation of selecting an arbitrary candidate image among a plurality of candidate images related to saturation. The above-mentioned point that the user can select an arbitrary candidate image from a plurality of candidate images related to brightness after performing an operation of selecting an arbitrary candidate image from a plurality of candidate images related to saturation. It is the same as the said embodiment.

Further, the user can set (change) the imaging conditions regarding the brightness or the saturation even in the area other than the first area 61, that is, the area in which the order given by the setting unit 35c is the second or lower. , The same as the above-described embodiment.

(Modification 2)
17 and 18 are views illustrating the display screen according to the modified example 2. In the second modification, when the plurality of candidate images described above are stored in the work memory 34, the display control unit 35g displays a reference image (reference image) of the reference condition on the left side of the display surface of the liquid crystal monitor 36 as shown in FIG. The reference image 60-1) is displayed, and the candidate image 60-2 is displayed on the right side of the display surface.
Here, the point that the outline of the first region 61, which is the target of setting (changing) the imaging conditions, is highlighted is the same as in the case of the above embodiment.

In FIG. 17, the upper icon 75 is displayed at the upper position of the candidate image 60-2 on the right side, and the lower icon 76 is displayed at the lower position of the candidate image 60-2. For example, when the user taps the lower icon 76 with the finger 80 on the display screen of FIG. 17, the display control unit 35g replaces the candidate image 60-2 on the right side with the candidate image 60 as illustrated in FIG. Display -3. The display of the reference image 60-1 on the left side remains unchanged.

The candidate image 60-3 is an image whose brightness (or saturation) is one step higher than that of the candidate image 60-2. In this way, the display control unit 35g reads out the necessary candidate images 60-3 in response to the tap operation of the lower icon 76 from the eight candidate images temporarily stored in the work memory 34, and is shown in FIG. Used for the illustrated display.

Although not shown, for example, when the user taps the upper icon 75 with the finger 80 on the display screen of FIG. 18, the display control unit 35g will display the candidate image 60-3 on the right side as illustrated in FIG. The candidate image 60-2 is displayed instead of. The display of the reference image 60-1 on the left side remains unchanged.

According to the second modification, the user can confirm the candidate image together with the reference image. Further, the user can display and confirm other candidate images that are not displayed on the liquid crystal monitor 36 by tapping the upper icon 75 or the lower icon 76.

In the second modification, when the user double-taps any position on the screen with the finger 80 while an arbitrary candidate image is displayed, the control unit 35 (imaging condition setting unit 35d) is at that time. The imaging conditions applied to the candidate images 60-n (n is a natural number of 1 to 8) displayed on the right side of the display surface are applied to the first region 61 when an imaging instruction is next given. It is determined as the image pickup condition to be performed, and the process returns to the state where the setting screen illustrated in FIG. 7 is displayed on the liquid crystal monitor 36.

It should be noted that the user can perform an operation of selecting an arbitrary candidate image from a plurality of candidate images related to saturation and then an operation of selecting an arbitrary candidate image among a plurality of candidate images related to saturation. The above-mentioned point that the user can select an arbitrary candidate image from a plurality of candidate images related to brightness after performing an operation of selecting an arbitrary candidate image from a plurality of candidate images related to saturation. It is the same as the said embodiment.

Further, the user can set (change) the imaging conditions regarding the brightness or the saturation even in the area other than the first area 61, that is, the area in which the order given by the setting unit 35c is the second or lower. , The same as the above-described embodiment.

(Modification 3)
In the above-described embodiment, one of "program auto" mode, "shutter speed priority auto" mode, "aperture priority auto" mode, and "manual" mode is selected and selected as an example of determining the reference condition. Appropriate exposure is determined by performing a predetermined exposure calculation according to the exposure mode, and image processing conditions prepared in advance so as to obtain standard image quality are determined, and these exposure conditions and image processing conditions determined are determined. An example of using the reference condition was explained. Instead of this, the standard conditions may be determined as follows.

<Example of determining standard conditions for brightness>
In the third modification, the method of determining the appropriate exposure is changed depending on the image pickup scene mode setting of the camera 1 and the subject element detected by the object detection unit 35a, and this is set as a reference condition (exposure condition). For example, when the image pickup scene mode setting of the camera 1 is set to the portrait mode and a person is detected as a subject element by the object detection unit 35a, the control unit 35 transfers the area including the person to the area including the person. The exposure condition is determined so that the brightness of the person is appropriate, and it is used as the reference condition.

For example, when the image pickup scene mode setting of the camera 1 is set to the night view portrait mode and the object detection unit 35a detects a person as a subject element, the control unit 35 transfers the area including the person to the area including the person. In addition to brightening the brightness of the person, the exposure conditions are determined so as to be appropriate in consideration of the brightness of the night view for the area including the night view as the background, and used as the reference condition. Further, when the light emitting device 40 emits light, the appropriate exposure is determined so as to balance the brightness of both the person and the background, and the required light emitting amount of the light emitting device 40 is calculated.

For example, when the image pickup scene mode setting of the camera 1 is set to the landscape mode and the shooting distance is farther than a predetermined distance (almost infinity), the control unit 35 controls the color component (green pixel) of the subject element. Based on the ratio of Gb, Gr, blue pixel B, red pixel R) and the brightness of the subject element, the exposure condition is set so that the brightness of the mountain is appropriate for the area including the subject element (for example, the mountain). Determined and used as the standard condition.

<Example of determining standard conditions for saturation>
In the third modification, the image processing condition prepared separately from the standard image quality is set as the reference condition (image processing condition) according to the image pickup scene mode setting of the camera 1 and the subject element detected by the object detection unit 35a.

For example, the control unit 35 protrudes from the other color components of the signals read from the green pixels Gb, Gr, the blue pixel B, and the red pixel R in the area divided by the area dividing unit 35b. When it has a color component, a condition for obtaining a raw image (flat) without image creation is determined for that region, and the condition determined in this way is used as a reference condition.

For example, when the image pickup scene mode setting of the camera 1 is set to the close-up mode and the object detection unit 35a detects a red flower as a subject element, the setting unit 35c is set to the area including the red flower. On the other hand, the condition for lowering the exposure and lowering the saturation of the red component is determined, and the condition determined in this way is used as the reference condition.

(Modification example 4)
In the above description, the camera 1 has been described as an example, but it may be configured by a high-performance mobile phone 250 having a camera function such as a smartphone or a mobile device such as a tablet terminal.

(Modification 5)
In the above-described embodiment, the camera 1 in which the image pickup unit 32 and the control unit 35 are configured as a single electronic device has been described as an example. Instead, for example, an image pickup system 1B may be configured in which the image pickup unit 32 and the control unit 35 are separately provided and the control unit 35 controls the image pickup unit 32 via communication.
Hereinafter, an example in which the image pickup apparatus 1001 provided with the image pickup unit 32 is controlled by the control device 1002 provided with the control unit 35 will be described.

FIG. 19 is a block diagram illustrating the configuration of the imaging system 1B according to the modified example 5. In FIG. 19, the image pickup system 1B is composed of an image pickup device 1001 and a display device 1002. The image pickup apparatus 1001 includes a first communication unit 1003 in addition to the image pickup optical system 31 and the image pickup unit 32 described in the above embodiment. Further, the display device 1002 is in addition to the image processing unit 33, the work memory 34, the control unit 35, the liquid crystal monitor 36, the operation member 37, the non-volatile memory 38, the recording unit 39, and the light emitting device 40 described in the above embodiment. A second communication unit 1004 is provided.

The first communication unit 1003 and the second communication unit 1004 can perform bidirectional data communication by, for example, a well-known wireless communication technology or optical communication technology.
The image pickup device 1001 and the display device 1002 may be connected by wire with a wired cable, and the first communication unit 1003 and the second communication unit 1004 may be configured to perform bidirectional data communication.

In the image pickup system 1B, the control unit 35 controls the image pickup unit 32 by performing data communication via the second communication unit 1004 and the first communication unit 1003. For example, by transmitting and receiving predetermined control data between the image pickup device 1001 and the display device 1002, the display device 1002 divides the screen into a plurality of areas or divides the screen based on the image as described above. The regions are ordered, different imaging conditions are set for each divided region, and the imaging signal captured in each region is read out.

According to the fifth modification, the live view image acquired on the image pickup device 1001 side and transmitted to the display device 1002 is displayed on the liquid crystal monitor 36 of the display device 1002, so that the user can move to a position away from the image pickup device 1001. Remote operation can be performed from a certain display device 1002.
The display device 1002 can be configured by, for example, a high-performance mobile phone 250 such as a smartphone. Further, the image pickup apparatus 1001 can be configured by an electronic device including the above-mentioned stacked image pickup element.
The control unit 35 of the display device 1002 includes an object detection unit 35a, an area division unit 35b, a setting unit 35c, an image pickup condition setting unit 35d, an image pickup control unit 35e, an AF calculation unit 35f, and a display control unit 35g. Although the example of providing the above is described, the object detection unit 35a, the area division unit 35b, the setting unit 35c, the image pickup condition setting unit 35d, the image pickup control unit 35e, the AF calculation unit 35f, and a part of the display control unit 35g are imaged. It may be provided in the device 1001.

(Modification 6)
The supply of the program to the mobile device such as the camera 1, the high-performance mobile phone 250, or the tablet terminal described above is performed by infrared communication or short-range wireless communication from the personal computer 205 storing the program, as illustrated in FIG. 20, for example. Can be sent to mobile devices.

The program may be supplied to the personal computer 205 by setting a recording medium 204 such as a CD-ROM in which the program is stored in the personal computer 205, or by a method via a communication line 201 such as a network to the personal computer 205. It may be loaded. When passing through the communication line 201, the program is stored in the storage device 203 or the like of the server 202 connected to the communication line.

Further, the program can be directly transmitted to the mobile device via the access point (not shown) of the wireless LAN connected to the communication line 201. Further, a recording medium 204B such as a memory card in which the program is stored may be set in the mobile device. As described above, the program can be supplied as various forms of computer program products such as those provided via a recording medium or a communication line.

Although various embodiments and modifications have been described above, the present invention is not limited to these contents. Other aspects considered within the scope of the technical idea of the present invention are also included within the scope of the present invention.

1 ... Camera 1B ... Imaging system 32 ... Imaging unit 35 ... Control unit 35a ... Object detection unit 35b ... Area division unit 35c ... Setting unit 35d ... Imaging condition setting unit 35e ... Imaging control unit 35g ... Display control unit 36 ... LCD monitor 40 ... Light emitting device 100 ... Image pickup element 1001 ... Image pickup device 1002 ... Display device

Claims (16)

An image sensor with multiple pixels and
A display unit that displays a plurality of images based on signals generated by at least a plurality of first pixels among the plurality of pixels, and a display unit.
A control unit that controls a plurality of first pixels and a plurality of second pixels among the plurality of pixels with different control parameters by using an image selected from the plurality of images displayed on the display unit.
Electronic equipment equipped with. In the electronic device according to claim 1,
The pixel has a photoelectric conversion unit that converts light into electric charges.
The control unit has a different storage time between the charge storage time converted by the photoelectric conversion unit of the first pixel and the charge storage time converted by the photoelectric conversion unit of the second pixel. An electronic device that controls to be. In the electronic device according to claim 1 or 2.
The pixel has a photoelectric conversion unit that converts light into electric charges.
The control unit has a different number of times of accumulation of electric charges converted by the photoelectric conversion unit of the first pixel and the number of times of accumulation of electric charges converted by the photoelectric conversion unit of the second pixel. An electronic device that controls to be. In the electronic device according to any one of claims 1 to 3.
The control unit is an electronic device that controls the signal generated by the first pixel and the signal generated by the second pixel so that they are read out at different frame rates. In the electronic device according to any one of claims 1 to 4.
The image pickup device has a first signal processing circuit connected to the first pixel and a second signal processing circuit connected to the second pixel.
The control unit is an electronic device that controls the signal generated by the first pixel and the signal generated by the second pixel so that different signal processing is performed. In the electronic device according to claim 5,
The first signal processing circuit has a circuit that amplifies the signal generated by the first pixel.
The second signal processing circuit has a circuit that amplifies the signal generated by the second pixel.
The control unit is an electronic device that controls the signal generated by the first pixel and the signal generated by the second pixel so as to be amplified at different amplification factors. In the electronic device according to claim 5 or 6.
The first signal processing circuit has a circuit that converts a signal generated by the first pixel into a digital signal.
The second signal processing circuit has a circuit that converts a signal generated by the second pixel into a digital signal.
The control unit is an electronic device that controls the signal generated by the first pixel and the signal generated by the second pixel so as to be converted into a digital signal by a different number of bits. In the electronic device according to any one of claims 5 to 7.
The image pickup element is an electronic device having an image pickup chip in which the plurality of pixels are arranged, and a signal processing chip in which the first signal processing circuit and the second signal processing circuit are arranged. In the electronic device according to claim 8,
The image pickup chip is an electronic device laminated by the signal processing chip. In the electronic device according to any one of claims 1 to 9.
The plurality of images are electronic devices having different brightness. In the electronic device according to any one of claims 1 to 9.
The plurality of images are electronic devices having different saturations. In the electronic device according to any one of claims 1 to 11.
The plurality of images are electronic devices displayed side by side in the display unit. In the electronic device according to any one of claims 1 to 11.
The plurality of images are electronic devices that are sequentially displayed on the display unit. In the electronic device according to any one of claims 1 to 13.
Equipped with an operation unit that accepts user operation instructions
The control unit uses the image selected based on the operation instruction received by the operation unit from the plurality of images, and the plurality of first pixels and a plurality of second pixels among the plurality of pixels. And electronic devices that control with different control parameters. In the electronic device according to any one of claims 1 to 14.
The plurality of first pixels are arranged side by side in the row direction.
The plurality of second pixels are electronic devices arranged side by side in the row direction. In the electronic device according to any one of claims 1 to 15.
The plurality of first pixels are arranged side by side in the column direction.
The plurality of second pixels are electronic devices arranged side by side in the row direction.

Images