JP6673319B2 - Electronics - Google Patents

Description

  The present invention relates to an electronic device including an image sensor.

  2. Description of the Related Art An electronic device including an imaging element in which a back-illuminated imaging chip and a signal processing chip are stacked (hereinafter, referred to as a stacked imaging element) has been proposed (see Patent Document 1). The stacked image sensor is stacked such that a back-illuminated imaging chip and a signal processing chip are connected to each other via a microbump for each predetermined region.

JP 2006-49361 A

  However, there are not many proposals to divide an image into blocks each having one or more of the above regions and obtain a captured image for each block in an electronic device including a conventional stacked image sensor. The convenience of the electronic equipment provided was not sufficient.

An electronic device according to one embodiment of the present invention includes a plurality of first pixels including a first photoelectric conversion unit that converts light into electric charges, a plurality of second pixels including a second photoelectric conversion unit that converts light into electric charges, A first wiring connected to the plurality of first pixels and outputting a first control signal for controlling the first pixel; and a first wiring connected to the plurality of second pixels and controlling the second pixel. An image sensor having a second wiring different from the first wiring to which a second control signal is output; a first image based on a first signal from the first pixel; and a second image from the second pixel. A control unit configured to display a second image based on the signal on a display unit, wherein the first photoelectric conversion unit is configured to intersect a first direction and a second direction in the first region of the image sensor. And the second photoelectric conversion unit is disposed in a second region of the image sensor. Respectively disposed on the first direction and the second direction.

  According to the present invention, an electronic device with improved image visibility can be realized.

FIG. 3 is a cross-sectional view of a stacked image sensor. FIG. 3 is a diagram illustrating a pixel array and a unit area of an imaging chip. FIG. 3 is a circuit diagram corresponding to a unit area of the imaging chip. FIG. 2 is a block diagram illustrating a functional configuration of an imaging device. FIG. 1 is a block diagram illustrating a configuration of an imaging device according to an embodiment. FIG. 3 is an external view illustrating a first display unit and a second display unit provided in the imaging device. FIG. 7A is a diagram exemplifying a live view image before the attention area and the surrounding area are determined. FIG. 7B is a diagram illustrating a region of interest determined by the control unit in the human body detection process based on the live view image. 8A is a diagram illustrating an image acquired corresponding to a peripheral region, FIG. 8B is a diagram illustrating an image acquired corresponding to a region of interest, and FIG. FIG. 7 is a diagram illustrating a first live view image to be displayed. 6 is a flowchart illustrating a flow of a shooting operation performed by a control unit of the imaging device. FIGS. 10A and 10B are diagrams illustrating a scene where a plurality of main subjects exist. FIG. 3 is an external view illustrating a first display unit and a second display unit provided in the imaging device.

Hereinafter, embodiments for implementing the present invention will be described with reference to the drawings.
<Explanation of the stacked image sensor>
First, a stacked image sensor 100 mounted on an electronic device (for example, the imaging device 1) according to an embodiment of the present invention will be described. Note that this stacked-type image sensor 100 is described in Japanese Patent Application No. 2012-139026 filed earlier by the present applicant. FIG. 1 is a cross-sectional view of the stacked image sensor 100. The imaging device 100 includes a backside illumination type imaging chip 113 that outputs pixel signals corresponding to incident light, a signal processing chip 111 that processes pixel signals, and a memory chip 112 that stores pixel signals. The imaging chip 113, the signal processing chip 111, and the memory chip 112 are stacked, and are electrically connected to each other by a conductive bump 109 such as Cu.

  As shown in the figure, the incident light mainly enters in the Z-axis plus direction indicated by the white arrow. In the present embodiment, the surface of the imaging chip 113 on which incident light is incident is referred to as a back surface. As shown by the coordinate axes, the left direction perpendicular to the Z axis is defined as the plus direction of the X axis, and the near direction perpendicular to the Z axis and the X axis is defined as the plus direction of the Y axis. In the following figures, coordinate axes are displayed so that the directions of the respective figures can be understood with reference to the coordinate axes in FIG.

  One example of the imaging chip 113 is a back-illuminated MOS image sensor. The PD layer 106 is provided on the back side of the wiring layer 108. The PD layer 106 includes a plurality of PDs (photodiodes) 104 that are two-dimensionally arranged and accumulates charges corresponding to incident light, and a transistor 105 provided corresponding to the PD 104.

  The color filter 102 is provided on the incident side of the incident light in the PD layer 106 via the passivation film 103. The color filters 102 have a plurality of types that transmit different wavelength regions, and have a specific arrangement corresponding to each of the PDs 104. The arrangement of the color filters 102 will be described later. A set of the color filter 102, the PD 104, and the transistor 105 forms one pixel.

  On the incident side of the incident light in the color filter 102, a micro lens 101 is provided corresponding to each pixel. The micro lens 101 collects incident light toward the corresponding PD 104.

  The wiring layer 108 has a wiring 107 for transmitting the pixel signal from the PD layer 106 to the signal processing chip 111. The wiring 107 may be a multilayer, and may be provided with a passive element and an active element.

  A plurality of bumps 109 are arranged on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the opposite surface of the signal processing chip 111, and the imaging chip 113 and the signal processing chip 111 are aligned by pressing or the like. The bumps 109 are joined and electrically connected.

  Similarly, a plurality of bumps 109 are arranged on surfaces of the signal processing chip 111 and the memory chip 112 that face each other. These bumps 109 are aligned with each other, and the signal processing chip 111 and the memory chip 112 are pressurized or the like, whereby the aligned bumps 109 are joined and electrically connected.

  Note that the bonding between the bumps 109 is not limited to Cu bump bonding by solid-phase diffusion, and micro-bump bonding by solder melting may be employed. Further, for example, about one bump 109 may be provided for one unit region described later. Therefore, the size of the bump 109 may be larger than the pitch of the PD 104. In a peripheral region other than the pixel region where the pixels are arranged, a bump larger than the bump 109 corresponding to the pixel region may be additionally provided.

  The signal processing chip 111 has a TSV (through-silicon via) 110 that connects circuits provided on the front and back surfaces to each other. The TSV 110 is preferably provided in a peripheral area. Further, the TSV 110 may be provided in a peripheral area of the imaging chip 113 and the memory chip 112.

  FIG. 2 is a diagram illustrating the pixel array of the imaging chip 113 and the unit area 131. In particular, this shows a state in which the imaging chip 113 is observed from the back side. In the pixel area, for example, more than 20 million pixels are arranged in a matrix. In the present embodiment, for example, 16 pixels of 4 × 4 adjacent pixels form one unit region 131. The grid lines in the figure show the concept of forming the unit area 131 by grouping adjacent pixels. The number of pixels forming the unit region 131 is not limited to this, and may be about 1000, for example, 32 × 64 pixels, or more or less (eg, one pixel).

  As shown in the partial enlarged view of the pixel area, the unit area 131 includes four so-called Bayer arrangements composed of four pixels of green pixels Gb, Gr, blue pixels B, and red pixels R in up, down, left, and right directions. The green pixel is a pixel having a green filter as the color filter 102, and receives light in a green wavelength band among incident light. Similarly, a blue pixel is a pixel having a blue filter as the color filter 102 and receives light in the blue wavelength band, and a red pixel is a pixel having a red filter as the color filter 102 and emits light in the red wavelength band. Receive light.

  In the present embodiment, a plurality of blocks are defined so as to include at least one unit area 131 per block, and each block can control pixels included in each block with different control parameters. That is, image pickup signals having different image pickup conditions can be acquired between a pixel group included in a certain block and a pixel group included in another block. Examples of the control parameters include a frame rate, a gain, a thinning rate, the number of rows or columns to be added for adding pixel signals, the charge accumulation time or the number of accumulations, and the number of bits for digitization. Further, the control parameter may be a parameter in image processing after acquiring an image signal from a pixel.

  FIG. 3 is a circuit diagram corresponding to the unit area 131 of the imaging chip 113. In FIG. 3, a rectangle typically surrounded by a dotted line indicates a circuit corresponding to one pixel. Note that at least part of each transistor described below corresponds to the transistor 105 in FIG.

  As described above, the unit region 131 is formed from 16 pixels. The 16 PDs 104 corresponding to each pixel are connected to the transfer transistor 302, and each gate of each transfer transistor 302 is connected to the TX wiring 307 to which a transfer pulse is supplied. In the present embodiment, the TX wiring 307 is commonly connected to the 16 transfer transistors 302.

  The drain of each transfer transistor 302 is connected to the source of the corresponding reset transistor 303, and the so-called floating diffusion FD between the drain of the transfer transistor 302 and the source of the reset transistor 303 is connected to the gate of the amplification transistor 304. The drain of the reset transistor 303 is connected to a Vdd wiring 310 to which a power supply voltage is supplied, and the gate is connected to a reset wiring 306 to which a reset pulse is supplied. In the present embodiment, the reset wiring 306 is commonly connected to the 16 reset transistors 303.

  The drain of each amplifying transistor 304 is connected to a Vdd wiring 310 to which a power supply voltage is supplied. Further, the source of each amplification transistor 304 is connected to the drain of each corresponding selection transistor 305. Each gate of the selection transistor is connected to a decoder wiring 308 to which a selection pulse is supplied. In the present embodiment, the decoder wiring 308 is provided independently for each of the 16 selection transistors 305. The source of each selection transistor 305 is connected to a common output wiring 309. The load current source 311 supplies a current to the output wiring 309. That is, the output wiring 309 for the selection transistor 305 is formed by a source follower. Note that the load current source 311 may be provided on the imaging chip 113 side or on the signal processing chip 111 side.

  Here, a flow from the start of charge accumulation to the pixel output after the end of accumulation will be described. When a reset pulse is applied to the reset transistor 303 through the reset wiring 306 and a transfer pulse is simultaneously applied to the transfer transistor 302 through the TX wiring 307, the potentials of the PD 104 and the floating diffusion FD are reset.

  When the application of the transfer pulse is released, the PD 104 converts the received incident light into a charge and stores the charge. Thereafter, when the transfer pulse is applied again in a state where the reset pulse is not applied, the accumulated charge is transferred to the floating diffusion FD, and the potential of the floating diffusion FD changes from the reset potential to the signal potential after the charge accumulation. . Then, when a selection pulse is applied to the selection transistor 305 through the decoder wiring 308, a change in the signal potential of the floating diffusion FD is transmitted to the output wiring 309 via the amplification transistor 304 and the selection transistor 305. Accordingly, a pixel signal corresponding to the reset potential and the signal potential is output from the unit pixel to the output wiring 309.

  As shown in FIG. 3, in the present embodiment, the reset wiring 306 and the TX wiring 307 are common to the 16 pixels forming the unit area 131. That is, the reset pulse and the transfer pulse are respectively applied to all 16 pixels at the same time. Therefore, all the pixels forming the unit region 131 start charge accumulation at the same timing and end charge accumulation at the same timing. However, a pixel signal corresponding to the stored charge is selectively output from the output wiring 309 by sequentially applying a selection pulse to each selection transistor 305. Further, the reset wiring 306, the TX wiring 307, and the output wiring 309 are separately provided for each unit region 131.

  By configuring the circuit based on the unit regions 131 as described above, the charge accumulation time can be controlled for each unit region 131. In other words, pixel signals at different frame rates can be output between the unit areas 131. More specifically, while one unit area 131 performs one charge accumulation, the other unit area 131 repeats the charge accumulation many times to output a pixel signal each time. Each frame for a moving image can be output at a different frame rate between the unit areas 131.

  FIG. 4 is a block diagram illustrating a functional configuration of the image sensor 100. The analog multiplexer 411 sequentially selects the 16 PDs 104 forming the unit area 131 and outputs each pixel signal to the output wiring 309 provided corresponding to the unit area 131. The multiplexer 411 is formed on the imaging chip 113 together with the PD 104.

  The pixel signals output via the multiplexer 411 are converted into CDS and A / D signals by a signal processing circuit 412 formed on the signal processing chip 111 and performing correlated double sampling (CDS) / analog / digital (A / D) conversion. D conversion is performed. The A / D-converted pixel signal is delivered to a demultiplexer 413 and stored in a pixel memory 414 corresponding to each pixel. The demultiplexer 413 and the pixel memory 414 are formed on the memory chip 112.

  The arithmetic circuit 415 processes the pixel signal stored in the pixel memory 414 and delivers it to the subsequent image processing unit. The arithmetic circuit 415 may be provided in the signal processing chip 111 or may be provided in the memory chip 112. Note that FIG. 4 shows connections for one unit area 131, but these actually exist for each unit area 131 and operate in parallel. However, the arithmetic circuit 415 may not be provided for each unit area 131. For example, even if one arithmetic circuit 415 sequentially processes the values in the pixel memory 414 corresponding to each unit area 131, the processing may be sequentially performed. Good.

  As described above, the output wiring 309 is provided corresponding to each of the unit regions 131. Since the image pickup device 100 has the image pickup chip 113, the signal processing chip 111, and the memory chip 112 stacked, by using the electrical connection between the chips using the bump 109 for the output wiring 309, each chip can be placed in the plane direction. The wiring can be routed without increasing the size.

<Description of imaging device>
FIG. 5 is a block diagram illustrating the configuration of the imaging device 1 including the above-described imaging device 100. 5, the imaging device 1 includes an imaging optical system 10, an imaging unit 20, an image processing unit 30, a work memory 40, a display unit 50, a recording unit 60, and a control unit 70.

  The imaging optical system 10 includes a plurality of lenses, and guides a light beam from the object scene to the imaging unit 20. The imaging optical system 10 may be configured integrally with the imaging device 1 or may be configured to be replaceable with respect to the imaging device 1. The imaging optical system 10 may have a built-in focus lens or a built-in zoom lens.

  The imaging unit 20 includes the above-described imaging device 100 and a driving unit 21 that drives the imaging device 100. The drive of the image sensor 100 is controlled by a control signal output from the drive unit 21 so that the above-described independent accumulation control can be performed on a block-by-block basis. The control unit 70 instructs the drive unit 21 on the position, shape, range, and the like of the block.

  The image processing unit 30 performs image processing on the image data captured by the imaging unit 20 in cooperation with the work memory 40. In the present embodiment, the image processing unit 30 performs detection processing of a main subject included in an image in addition to normal image processing (color signal processing, gamma correction, and the like). The detection of the main subject by the image processing unit 30 can be performed using a known face detection function. In addition to the face detection, a human body included in an image may be detected as a main subject as described in, for example, Japanese Patent Application Laid-Open No. 2010-16621 (US2010 / 0002940).

  The work memory 40 temporarily stores image data before and after JPEG compression and before and after MPEG compression. The display unit 50 includes, for example, a liquid crystal display panel, and displays an image (still image or moving image) captured by the imaging unit 20 and various information, and displays an operation input screen. The imaging device 1 according to the present embodiment has a first display unit 51 and a second display unit 53 as the display unit 50, a first touch panel 52 on the first display unit 51, and a second touch panel 54 on the second display unit 53. , Respectively.

  FIG. 6 is an external view illustrating the first display unit 51 and the second display unit 53 provided in the imaging device 1. 6, the first display unit 51 is provided on the back surface of the imaging device 1, and the second display unit 53 is provided so as to be foldable via a hinge unit 55. The first touch panel 52 outputs a signal indicating the position touched by the user to the first display unit 51, and the second touch panel 54 outputs a signal indicating the position touched by the user to the second display unit 53.

  In the present embodiment, the display unit 50 is configured by the first display unit 51 and the second display unit 53. Instead, the display unit 50 is configured by one display device, and the display device is configured by two display devices. The display area may be divided and displayed, and the first display area may be displayed by the first display unit 51 and the second display area may be displayed by the second display unit 53. In addition, the display unit 50 is constituted by one display device, and the display unit 50 performs display such that a display to be displayed on the first display unit 51 or a display to be displayed on the second display unit 53 is performed. The content may be switched. For example, the display content is switched at the timing when the touch panel provided on the display unit 50 is operated.

  Returning to the description of FIG. 5, the recording unit 60 stores various data such as image data in a storage medium such as a memory card. The control unit 70 has a CPU and controls the entire operation of the imaging device 1. In the present embodiment, the control unit 70 divides the imaging surface of the imaging device 100 into a plurality of blocks, and obtains images at different frame rates among the blocks. To this end, the control unit 70 instructs the drive unit 21 on the position, shape, range, and control parameters for each block.

<Attention area and surrounding area>
In the present embodiment, the concept of an attention area and a peripheral area is introduced into a screen, and the screen is made to correspond to the plurality of blocks. The control unit 70 sets a region including a human body detected by the human body detection process as a region of interest based on, for example, a live view image. Then, an area other than the attention area is set as a peripheral area. Here, the live view image is also called a preview image before the main imaging is performed, and refers to a monitor image acquired by the imaging device 100 at a predetermined frame rate (for example, 90 fps).

  Instead of performing the human body detection process, a motion vector may be detected from among a plurality of frames forming the live view image, and a region including the moving subject detected based on the motion vector may be set as the attention region.

  Further, in a state where the live view image is displayed on the first display unit 51 of the display unit 50, the main subject corresponding to (displayed) the position where the user who views the live view image touches the first touch panel 52. May be set as the attention area.

  In a state where the live view image is displayed on the second display unit 53 of the display unit 50, the main subject corresponding to (displayed) the position at which the user viewing the live view image touches the second touch panel 54. May be set as the attention area.

  When the control unit 70 determines the attention area and the peripheral area in this manner, the control unit 70 instructs the drive unit 21 to perform accumulation control by dividing the image sensor 100 into blocks corresponding to the attention area and blocks corresponding to the peripheral area. send. Note that a common control parameter is applied to the entire live view image of the live view image before the attention area and the surrounding area are determined.

  FIG. 7A is a diagram exemplifying a live view image before the attention area and the surrounding area are determined. In FIG. 7A, the live view image is composed of, for example, a total of 48 unit areas 131 of 6 × 8. In FIG. 7A, the shape of the unit region 131 is vertically long, but the shape of the unit region 131 may be square or horizontally long.

  Here, when displaying the live view image before determining the attention area and the surrounding area on the display unit 50 (the first display unit 51 or the second display unit 53), the grid lines 71 indicating the unit area are displayed in the live view. It may be displayed over an image.

  FIG. 7B is a diagram illustrating a region of interest determined by the control unit 70 by the human body detection processing based on the live view image. In FIG. 7B, an area including the detected human body (consisting of a total of 20 unit areas 131 of 4 × 5 in a frame 72) is set as the attention area. An area other than the attention area (consisting of a total of 28 unit areas 131 outside the frame 72) is set as a peripheral area.

  When the control unit 70 determines the attention area, a frame 72 indicating the determined attention area is superimposed on the live view image being displayed on the display unit 50 (the first display unit 51 or the second display unit 53). It may be displayed.

  In the present embodiment, after determining the attention area and the peripheral area, different control parameters are applied to the block corresponding to the attention area of the image sensor 100 and the block corresponding to the peripheral area. Then, different images are displayed on the first display unit 51 and the second display unit 53 based on a plurality of images obtained by applying different control parameters.

  Specifically, the image corresponding to the attention area and the image corresponding to the peripheral area are combined, and the combined image is displayed on the first display unit 51 as the first live view image. Further, only the image corresponding to the attention area is displayed on the second display unit 53 as the second live view image.

  8A shows an image 90 obtained corresponding to the peripheral region, FIG. 8B shows an image 80 obtained corresponding to the region of interest, and FIG. It is a figure which illustrates one live view image 90A. In FIG. 8A, an image 90 corresponding to a peripheral area is acquired at a frame rate of, for example, 90 fps. In FIG. 8B, the image 80 corresponding to the region of interest is acquired at a frame rate of 180 fps, which is twice as large as the peripheral region, for example.

  According to FIGS. 8A and 8B, while acquiring four frames of the image 90 corresponding to the peripheral region, eight frames of the image 80 corresponding to the attention region can be acquired. As described above, by acquiring the attention area at a higher frame rate than the peripheral area, the visibility of the second live view image 80A for the attention area displayed on the second display unit 53 (FIG. 6) is improved. Can be.

  Regarding the first live view image 90A displayed on the first display unit 51 (FIG. 6), when the image 90 corresponding to the attention area is combined with the image 90 corresponding to the peripheral area, the peripheral area and the acquisition start timing Fits and synthesizes the same frame (frame surrounded by a thick line in FIG. 8B). As described above, the reproduction frame rate of the first live view image 90A is adjusted to the frame rate of the peripheral area where the frame rate at the time of acquisition is low, so that the user who observes the first live view image 90A may feel uncomfortable. There is no. In FIG. 6, the dashed line frame displayed on the screen of the first display unit 51 is described for convenience in order to make the range of the image 80 that has been fitted and synthesized easy to understand, and is not necessarily displayed on the screen of the first display unit 51. You may. Here, when displaying an icon or the like on the first live view image 90A being displayed on the first display unit 51, the icon is superimposed on a position corresponding to the peripheral region, avoiding the image 80 corresponding to the attention region. And

  In the example of FIG. 6, the first live view image 90A including the attention area and the peripheral area is displayed on the first display unit 51, and the second live view image 80A corresponding to only the attention area is displayed on the second display unit 53. Although the display is performed, the display content may be reversed between the first display unit 51 and the second display unit 53. For example, the display content is reversed when the first touch panel 52 or the second touch panel 54 is operated. When the second live view image 80A corresponding to the attention area is displayed, the second live view image 80A may be enlarged and displayed by electronic zoom processing. In FIG. 6, the second live view image 80A displayed on the second display unit 53 is a second live view image displayed on the second display unit 53, rather than the image 80 displayed on the first display unit 51. A state in which 80A is enlarged by an electrical enlargement process (so-called electronic zoom) so that 80A is displayed in a large size is illustrated. The electronic zoom process in this case is performed by the image processing unit 30.

<Explanation of flowchart>
FIG. 9 is a flowchart illustrating a flow of a shooting operation performed by the control unit 70 of the imaging device 1. When an ON-OFF switch (not shown) is turned on and power is supplied to each unit of the imaging apparatus 1, the control unit 70 repeatedly starts the processing in FIG. In step S101 of FIG. 9, the control unit 70 causes the imaging unit 20 to start imaging. The acquisition of the live view image started in step S101 is performed by applying a common control parameter in all regions of the image sensor 100. That is, imaging is performed at the same frame rate (for example, 90 fps) in the entire region of the image sensor 100.

  The control unit 70 causes the image processing unit 30 to perform image processing on the live view image data acquired by the imaging unit 20 and then causes the first display unit 51 to display the live view image data. In this embodiment, the display of the second display unit 53 is turned off at this stage.

  In step S102, the control unit 70 determines a region of interest based on the live view image as described above, and proceeds to step S103. The control unit 70 sets an area other than the attention area as a peripheral area. In step S103, the control unit 70 performs control on the imaging unit 20 and display control on the display unit 50 based on the attention area and the peripheral area determined in step S102.

  The control unit 70 sends an instruction to the drive unit 21 to change the frame rate of the block corresponding to the attention area of the image sensor 100 from 90 fps to 180 fps, which is twice as large. In addition, you may change from 90 fps to 270 fps which is 3 times. The control unit 70 turns on the display of the second display unit 53 in response to increasing the frame rate of the block corresponding to the attention area, and displays the second live view image 80A corresponding to only the attention area on the second display unit. 53 is displayed (FIG. 6). On the other hand, the first display unit 51 displays a first live view image 90A in which the image 80 corresponding to the attention area is inserted and synthesized in the image 90 corresponding to the peripheral area (FIG. 6).

  At this time, in order to improve the visibility of the attention area, the image processing unit 30 performs the above-described electronic zoom processing, and the second display unit 53 displays the second live view image 80A in which the attention area is enlarged. The enlargement ratio (electronic zoom magnification) in this case may be set so that the display of the attention area after the enlargement does not protrude from the display area of the second display unit 53. Accordingly, the user can visually recognize the second live view image 80A having a high frame rate of the attention area from the second display unit 53 while visually recognizing the entire composition on the first live view image 90A of the first display unit 51.

  In step S104, the control unit 70 determines whether or not an operation has been performed on the imaging device 1. When the operation is performed, the control unit 70 makes an affirmative determination in step S104 and proceeds to step S105. When the operation is not performed, the control unit 70 makes a negative determination in step S104 and repeats the determination processing.

  The operation of determining the presence or absence in step S104 includes an operation performed on the first touch panel 52 or the second touch panel 54, and an operation performed on an operation member (not shown). In the present embodiment, since the second display section 53 displays the second live view image 80A corresponding to the attention area, it is determined whether or not an operation has been performed on the second touch panel 54.

  The control unit 70 causes the second display unit 53 to display a front pin icon 56, a rear pin icon 57, an icon 58 for increasing the frame rate, and an icon 59 for decreasing the frame rate on the second display unit 53 as illustrated in FIG. A touch operation on the touch panel 54 is accepted. In step S105, the control unit 70 executes a process corresponding to the icon being displayed on the second display unit 53 and being displayed at the touch position on the second touch panel 54, and then proceeds to step S106. move on. For example, when the front focus icon 56 being displayed is touch-operated, the control unit 70 drives the focus lens of the imaging optical system 10 to the front focus side so that the main subject in the attention area is a so-called front focus.

  In step S106, the control unit 70 determines whether or not a release operation has been performed. When a release button (not shown) is pressed (full-press operation), the control unit 70 makes an affirmative determination in step S106 and proceeds to step S107, and when no press operation is performed, makes a negative determination in step S106. And returns to step S104. When returning to step S104, the above processing is repeated. In addition, a release icon may be displayed on the second display unit 53, and an affirmative determination may be made in step S106 when the release icon being displayed is touched.

  In step S107, the control unit 70 executes a shooting process, causes the recording unit 60 to store the acquired image data in a memory card or the like, and ends the process in FIG. In the photographing process, a common control parameter is applied to the entire region of the image sensor 100 to acquire (record) a still image of one frame, record the image, and acquire images of a plurality of frames after acquiring the still image. . Then, the slow reproduction moving image data is generated and recorded based on the images of a plurality of frames acquired during a predetermined time before and after the release operation. The slow reproduction moving image data refers to moving image data reproduced at a frame rate (for example, 60 fps) lower than the frame rate obtained by the image sensor 100.

  The control unit 70 generates slow reproduction moving image data as follows. That is, the first live view image 90A and the second live view image 80A described above are displayed from the first predetermined time (for example, 0.6 seconds before) to the time t1 before the release operation time (t1). A plurality of frame images temporarily stored in the work memory 40 (the frame rate is the frame rate set in the peripheral area. For example, when acquired at 90 fps, 0.6 seconds is 54 frames); And a plurality of frame images stored in the work memory 40 from the time t1 to the second predetermined time after the time t1 (for example, 0.4 seconds) (the frame rate after the release operation is the frame rate set in the attention area) For example, the slow reproduction moving image data is generated based on the value of 72 frames for 0.4 seconds obtained at 180 fps. . As a result, reproduction is performed based on a plurality of frame images (126 images in total) stored in the work memory 40 for one second (0.6 seconds before the time t1 to 0.4 seconds after the time t1) sandwiching the time t1. The slow reproduction moving image data for about 2 seconds is generated.

  When the slow playback moving image is generated in this manner, slow playback moving image data based on a frame image acquired at a slow frame rate is obtained before a release operation, and slow playback moving image data based on a frame image acquired at a fast frame rate after a release operation. Data is obtained. Note that the slow image moving image data is generated by the image processing unit 30 as MPEG data or JPEG data.

According to the embodiment described above, the following operation and effect can be obtained.
(1) The imaging apparatus 1 captures an image of a peripheral region at a first frame rate of 90 fps, and captures an image of a region of interest at a second frame rate of 180 fps faster than the first frame rate. Since the control unit 70 displays the image captured in the first display unit 51 on the first display unit 51 and the control unit 70 displays the image captured in the attention area on the second display unit 53, the visibility of the image is improved. Can be improved.

(2) In the imaging device 1 of the above (1), the control unit 70 further displays an icon related to an operation related to the image near the image displayed on the second display unit 53, so that the visibility is improved. Operability with respect to images can also be improved.

(3) In the imaging device 1 of (1) or (2), the control unit 70 further performs additional display on the image captured in the attention area displayed on the first display unit 51. Therefore, for example, the range of the attention area can be displayed in an easily understandable manner.

(4) In the imaging device 1 of (1) to (3), the control unit 70 performs an enlargement process on an image captured in the attention area and displays the image on the second display unit 53. Can be further improved.

(5) In the imaging device 1 of (1) to (4), the control unit 70 includes a combining unit that combines an image captured in the peripheral region and an image captured in the attention region, and the combining unit (First live view image 90A) is displayed on the first display unit 51. Thereby, the captured image can be displayed as one live view image.

(6) In the imaging device 1 of the above (5), the synthesizing unit (control unit 70) selects the first frame rate in the peripheral area from the time-series images 80 imaged at the second frame rate 180 fps in the attention area. Since an image corresponding to 90 fps is selected and the selected image is sequentially combined with the image 90 captured in the peripheral area, it is possible to suppress a sense of discomfort given to a user who observes the combined first live view image 90A. Can be.

(7) In the imaging apparatus 1 of (1) to (5), the control unit 70 that determines a part of the image captured as the peripheral area as the attention area is further provided. An area including a person's face in the performed image or an area where a touch operation is performed on a screen displaying an image captured in a peripheral area may be determined as the attention area.

(8) In the imaging device 1 of (7), since the control unit 70 determines the attention area based on the face detection or the human body detection for the image captured in the peripheral area, the control unit 70 automatically focuses attention even without touch operation. The area can be determined.

(Modification 1)
In the above-described embodiment, the case where there is one main subject has been described as an example. Modification Example 1 describes a case where the number of main subjects is plural. FIG. 10 is a diagram illustrating a scene where a plurality of main subjects exist. The control unit 70 in the case of photographing the scene illustrated in FIG. 10A sets, for example, a boy behind the slide as the first main subject and a girl sliding on the slide as the second main subject. In this case, an area including the first main subject is defined as a first attention area, and an area including the second main subject is defined as a second attention area.

  The control unit 70 in the case of photographing the scene illustrated in FIG. 10B sets, for example, a girl on a swing as a first main subject and a boy on a swing as a second main subject. Also in this case, an area including the first main subject is defined as a first attention area, and an area including the second main subject is defined as a second attention area.

  In the first modification, when there are a first main subject and a second main subject that move at different speeds in a screen, the control unit 70 includes a first main subject that includes the first main subject based on the live view image. An attention area is determined, and a second attention area including the second main subject is determined. In this case, the peripheral region is a region other than the first region of interest and the second region of interest.

  The control unit 70 sends an instruction to the driving unit 21 to change the frame rate of the block corresponding to the first region of interest of the image sensor 100 from, for example, 90 fps to 150 fps, and to change the frame rate of the block corresponding to the second region of interest. For example, it is changed from 90 fps to 240 fps. The control unit 70 further divides the display into two display areas on the second display unit 53 as shown in FIG. 11, and displays a live view image 81A corresponding to the first attention area on the right side of the screen, for example. Then, the live view image 82A corresponding to the second attention area is displayed on the left side of the screen.

  In the first modification, an arrangement of a plurality of live view images 81A and 81B corresponding to a plurality of attention areas is displayed as a second live view image. In FIG. 11, a dashed line drawn at the center of the screen of the second display unit 53 is drawn for convenience in order to make the boundaries between the plurality of live view images 81A and 81B easier to understand, and is not necessarily the screen of the second display unit 53. May not be displayed.

  On the other hand, the first display unit 51 displays a first live view image 90A in which an image 81 corresponding to the first attention area and an image 82 corresponding to the second attention area are respectively fitted and synthesized in the image corresponding to the peripheral area. Display. Note that, in FIG. 11, the dashed line frame displayed on the screen of the first display unit 51 is described for convenience in order to make the range of the plurality of images 81 and 82 easy to understand, and is not necessarily displayed on the screen of the first display unit 51. It does not have to be displayed.

  In FIG. 11, the frame rate of the front pin icon 56A, the rear pin icon 57A, and the frame rate of the block corresponding to the first attention area are increased near the live view image 81A corresponding to the first attention area of the second display unit 53. An icon 58A and an icon 59A for lowering the frame rate of the block corresponding to the first region of interest are displayed, and a touch operation on the second touch panel 54 is accepted. The user can perform a focus operation and a frame rate adjustment operation by focusing on the live view image 81A.

  Further, in FIG. 11, near the live view image 82A corresponding to the second attention area of the second display unit 53, the front pin icon 56B, the rear pin icon 57B, and the frame rate of the block corresponding to the second attention area are displayed. An icon 58B for raising and an icon 59B for lowering the frame rate of the block corresponding to the second region of interest are displayed, and a touch operation on the second touch panel 54 is accepted. The user can perform a focus operation and a frame rate adjustment operation by focusing on the live view image 82A.

(Modification 2)
In the above-described embodiment and Modification Example 1, a camera having an optical finder is illustrated as the imaging device 1, but the imaging device 1 may be configured with a camera having no optical finder.

(Modification 3)
In the above-described embodiment and the first modification, the example in which the second display unit 53 is rotatably provided via the hinge has been described, but the second display unit 53 is attached to and detached from the housing of the imaging device 1. You may be comprised freely. In this case, a wireless communication unit is provided on each of the housing side of the imaging device 1 and the housing side of the second display unit 53, and information to be displayed on the second display unit 53 by wireless communication via these wireless communication units is captured by the imaging device. The data is transmitted from the first side to the second display unit 53 side. The wireless communication unit may be incorporated inside the housing or may be mounted outside the housing. As the wireless communication, for example, infrared light communication, Bluetooth (registered trademark), wireless LAN communication, or the like is used.

(Modification 4)
In the above-described embodiment and Modification 1, an example has been described in which the second display unit 53 is configured as a dedicated display unit. Instead, the liquid crystal display of the high-performance mobile phone may be operated as the second display, or the liquid crystal display of the tablet terminal may be operated as the second display. In this case, a wireless LAN communication unit is provided on the housing side of the imaging device 1, and the wireless LAN communication between the imaging device 1 and the advanced mobile phone (or tablet terminal) allows the liquid crystal of the advanced mobile phone (or tablet terminal) to be displayed. Information to be displayed on the display unit is transmitted from the imaging device 1 to the advanced mobile phone (or tablet terminal). The wireless LAN communication unit may be incorporated inside the housing of the imaging device 1 or may be mounted outside the housing.

(Modification 5)
In the above-described embodiment and Modification 1, a camera is illustrated as the imaging device 1, but the imaging device 1 may be configured by a high-performance mobile phone (or tablet terminal). In this case, a camera unit mounted on a high-performance mobile phone (or tablet terminal) is configured by using the above-described stacked imaging device 100.

  In the fifth modification, in addition to the first advanced mobile phone (or the first tablet terminal) constituting the imaging device 1, a second advanced mobile phone (or the second tablet terminal) operated as the second display unit is used. . In this case, the second advanced mobile phone (or the second tablet) is established by wireless LAN communication between the first advanced mobile phone (or the first tablet terminal) and the second advanced mobile phone (or the second tablet terminal). The information to be displayed on the liquid crystal display unit of the terminal is transmitted from the first advanced mobile phone (or the first tablet terminal) to the second advanced mobile phone (or the second tablet terminal).

  The above description is merely an example, and is not limited to the configuration of the above embodiment. The configurations of the above embodiment and each modified example may be appropriately combined.

DESCRIPTION OF SYMBOLS 1 ... Imaging device 10 ... Imaging optical system 20 ... Imaging part 30 ... Image processing part 40 ... Work memory 50 ... Display part 51 ... First display part 52 ... First touch panel 53 ... Second display part 54 ... Second touch panel 60 ... Recording unit 70 Control unit 80A Second live view image 90A First live view image 100 Image pickup device 109 Bump 111 Signal processing chip 112 Memory chip 113 Image pickup chip 131 Unit area

Claims (28)

An imaging element having a plurality of imaging regions having a photoelectric conversion unit that converts light into electric charges is disposed in a first direction and a second direction that intersects the first direction. are arranged in the region, a plurality of first pixel including a first photoelectric conversion unit that converts light into electric charge are arranged in the second imaging area among the plurality of imaging regions, the second photoelectric converting the light into an electric charge conversion A plurality of second pixels including a portion, a first wiring connected to the plurality of first pixels and outputting a first control signal for controlling the first pixels, and a connection to the plurality of second pixels An image sensor having a second wiring different from the first wiring to which a second control signal for controlling the second pixel is output;
A setting unit configured to set the first imaging region to an imaging condition different from an imaging condition set to the second imaging region, and an image captured in the first imaging region by an imaging condition set by the setting unit. A first image based on a first signal from the first pixel is displayed on a first display monitor of a display unit, and the image of the subject captured in the second imaging area is the second image . A display control unit for displaying a second image based on a second signal from the pixel on a second display monitor of the display unit,
The first photoelectric conversion unit, a plurality of arranged respectively to the first direction and the front Stories second direction before Symbol first imaging area,
The second photoelectric conversion unit, the electronic equipment to be more disposed respectively in front SL to the first direction in the second imaging region and the second direction. The first pixel has a first circuit unit connected to the first wiring,
The electronic device according to claim 1 , wherein the second pixel has a second circuit unit connected to the second wiring. The first circuit unit includes a first transfer unit connected to the first wiring and transferring the charge converted by the first photoelectric conversion unit,
3. The electronic device according to claim 2 , wherein the second circuit unit includes a second transfer unit connected to the second wiring and transferring the charge converted by the second photoelectric conversion unit. 4. Said second control signal, the electronic device according to claim 2 or claim 3 wherein the first control signal is output to the second wiring at a timing different from the timing to be output to the first wiring. The image sensor is connected to the plurality of first pixels, a third wiring for outputting a third control signal for controlling the first pixel, and connected to the plurality of second pixels, A fourth wiring different from the third wiring to which a fourth control signal for controlling a pixel is output,
The first circuit unit includes a first reset unit connected to the third wiring and resetting a potential of a first floating diffusion to which a charge from the first photoelectric conversion unit is transferred,
The second circuit unit includes a second reset unit connected to the fourth wiring and resetting a potential of a second floating diffusion to which a charge from the second photoelectric conversion unit is transferred by the fourth control signal. The electronic device according to claim 3 or 4 . The electronic device according to claim 5 , wherein the fourth control signal is output to the fourth wiring at a timing different from a timing at which the third control signal is output to the third wiring. The first circuit unit includes a first reset unit connected to the first wiring and resetting a potential of a first floating diffusion to which charges from the first photoelectric conversion unit are transferred,
Said second circuit section, connected to said second wiring, electrons according to claim 2, charge from the second photoelectric conversion section has a second reset unit resets the potential of the second floating diffusion to be transferred machine. The electronic device according to claim 7 , wherein the second control signal is output to the second wiring at a timing different from a timing at which the first control signal is output to the first wiring. The electronic device according to claim 1, wherein the control unit displays the first image on the second display monitor. The electronic device according to claim 9 , wherein the control unit displays the first image on the first display monitor in an enlarged manner than the first image on the second display monitor. Wherein the control unit, the first display monitor to display the image at a first frame rate, claim from claim 1 to display an image with a different second frame rate from the first frame rate said second display monitor 10 electronic device according to any one of. The image sensor has a signal processing unit that performs signal processing on the first signal and the second signal,
The control unit, the first image based on the first signal subjected to signal processing by the signal processing unit, and the second image based on the second signal subjected to signal processing by the signal processing unit The electronic device according to any one of claims 1 to 11 , further comprising: displaying on the display unit. The signal processing unit includes an amplification unit that amplifies the first signal and the second signal,
The control unit causes the display unit to display the first image based on the first signal amplified by the amplification unit and the second image based on the second signal amplified by the amplification unit. The electronic device according to claim 12 . The signal processing unit has a conversion unit used to convert the first signal and the second signal into a digital signal,
The control unit is configured to control the first image based on the first signal converted to a digital signal using the conversion unit, and the second image based on the second signal converted to a digital signal using the conversion unit. the electronic device according to claim 12 or claim 13 displaying the image, to the display unit. The image sensor includes a first semiconductor chip on which the first photoelectric conversion unit and the second photoelectric conversion unit are arranged, and a second semiconductor chip different from the first semiconductor chip on which the conversion unit is arranged. The electronic device according to claim 14 , comprising: The electronic device according to claim 15 , wherein the first semiconductor chip is stacked on the second semiconductor chip. The image sensor has a storage unit that stores the first signal converted to a digital signal using the conversion unit and the second signal converted to a digital signal using the conversion unit,
The control unit causes the display unit to display the first image based on the first signal stored in the storage unit and the second image based on the second signal stored in the storage unit. The electronic device according to claim 14 . A first semiconductor chip on which the first photoelectric conversion unit and the second photoelectric conversion unit are arranged; a second semiconductor chip different from the first semiconductor chip on which the conversion unit is arranged; The electronic device according to claim 17 , further comprising: a third semiconductor chip different from the first semiconductor chip and the second semiconductor chip on which a storage unit is arranged. The electronic device according to claim 18 , wherein the first semiconductor chip is stacked on the third semiconductor chip. The signal processing unit has a first signal processing circuit that performs signal processing on the first signal, and a second signal processing circuit that performs signal processing on the second signal,
The controller is configured to control the first image based on the first signal processed by the first signal processing circuit and the second image based on the second signal processed by the second signal processing circuit. The electronic device according to claim 12 , wherein a second image and a second image are displayed on the display unit. The first signal processing circuit has a first amplifier circuit that amplifies the first signal,
The second signal processing circuit has a second amplifier circuit that amplifies the second signal,
The control unit displays the first image based on the first signal amplified by the first amplifier circuit and the second image based on the second signal amplified by the second amplifier circuit. The electronic device according to claim 20 , which is displayed on a unit. The first signal processing circuit has a first conversion circuit used to convert the first signal into a digital signal,
The second signal processing circuit has a second conversion circuit used to convert the second signal into a digital signal,
The control unit is configured to control the first image based on the first signal converted to a digital signal using the first conversion circuit, and the second signal converted to a digital signal using the previous second conversion circuit. 22. The electronic device according to claim 20 , wherein the second image and the second image are displayed on the display unit. The imaging device includes a first semiconductor chip on which the first photoelectric conversion unit and the second photoelectric conversion unit are arranged, and a first semiconductor chip on which the first conversion circuit and the second conversion circuit are arranged. 23. The electronic device according to claim 22 , comprising a different second semiconductor chip. The electronic device according to claim 23 , wherein the first semiconductor chip is stacked on the second semiconductor chip. A first storage circuit that stores the first signal converted to a digital signal using the first conversion circuit; and a second signal that is converted to a digital signal using the second conversion circuit. And a second storage circuit for storing
The control unit displays the first image based on the first signal stored in the first storage circuit and the second image based on the second signal stored in the first storage circuit. 23. The electronic device according to claim 22 , which is displayed on a unit. The imaging device includes a first semiconductor chip on which the first photoelectric conversion unit and the second photoelectric conversion unit are arranged, and a first semiconductor chip on which the first conversion circuit and the second conversion circuit are arranged. 26. The semiconductor device according to claim 25 , further comprising a different second semiconductor chip, and a third semiconductor chip different from the first semiconductor chip and the second semiconductor chip on which the first storage circuit and the second storage circuit are arranged. Electronics. The electronic device according to claim 26 , wherein the first semiconductor chip is stacked on the third semiconductor chip. The image sensor includes a first output line connected to the first pixel and outputting the first signal, and a second output line connected to the second pixel and outputting the second signal. The electronic device according to any one of claims 1 to 27 , comprising:

Images