JP2015111761A - Electronic apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an electronic apparatus good in usability.SOLUTION: An electronic apparatus 1 includes: an imaging device 100 having a first region including a plurality of blocks arranged two-dimensionally and discretely and a second region different from the first region and including a plurality of blocks arranged two-dimensionally and discretely; and a storage control part 70, with respect to one frame time of a frame rate of the first and second regions, storing the first region in a first storage time longer than the frame time and storing the second region in a second storage time longer than the frame time.

Description

  The present invention relates to an electronic device.

  An imaging apparatus that divides pixel signals into a plurality of groups in units of rows and performs rolling shutter control for each group is known (see Patent Document 1).

JP 2011-244253 A

  In the prior art, for example, it is difficult to divide pixel signals included in the same row into a plurality of groups, which is not convenient.

  An electronic apparatus according to the present invention includes a first region having a plurality of blocks discretely arranged in two dimensions, and a second region having a plurality of blocks discretely arranged in two dimensions, unlike the first region. The first area is accumulated with a first accumulation time longer than the frame time with respect to one frame time of the frame rates of the first and second areas, and the second area is And an accumulation control unit that accumulates in a second accumulation time longer than the frame time.

According to the present invention, a user-friendly electronic device can be obtained.

It is sectional drawing of a multilayer type image pick-up element. It is a figure explaining the pixel arrangement | sequence and block of an imaging chip. It is a circuit diagram corresponding to the unit of an imaging chip. It is a block diagram which shows the functional structure of an image pick-up element. It is a block diagram which illustrates the composition of an imaging device. It is a figure explaining arrangement | positioning of the group in an imaging chip. It is a figure which illustrates four images. It is a figure which illustrates the order of the frame acquired at the time of imaging | photography of a moving image, and the order of the frame displayed at the time of 8 times slow reproduction of a moving image. It is a figure which illustrates the order of the frame acquired at the time of imaging | photography of a moving image, and the order of the frame displayed at the time of another 8 times slow reproduction. It is a figure which illustrates the order of the frame acquired at the time of imaging | photography of a moving image, and the order of the frame displayed at the time of 2 times slow reproduction of a moving image. It is a figure which illustrates the order of the frame acquired at the time of imaging | photography of a moving image, and the order of the frame displayed at the time of normal reproduction of a moving image. It is a figure which illustrates the order of the frame acquired at the time of imaging | photography of a moving image, and the order of the frame displayed at the time of the other 2 times slow reproduction. It is a flowchart explaining the flow of the imaging | photography operation | movement which the control part of an imaging device performs. It is a figure explaining arrangement | positioning of the group in the imaging chip of the modification 1. FIG. It is a figure explaining arrangement of a group in an imaging chip of modification 2. 10 is a circuit diagram corresponding to a unit of an imaging chip according to Modification 3. FIG. FIG. 11 is a block diagram illustrating a functional configuration of an image sensor according to Modification 3.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.
<Description of Laminated Image Sensor>
First, a multilayer imaging device 100 mounted on an electronic apparatus (for example, the imaging device 1) according to an embodiment of the present invention will be described. The multilayer image sensor 100 is described in Japanese Patent Application No. 2012-139026 filed earlier by the applicant of the present application. FIG. 1 is a cross-sectional view of the multilayer image sensor 100. The imaging device 100 includes a backside illumination type imaging chip 113 that outputs a pixel signal corresponding to incident light, a signal processing chip 111 that processes the pixel signal, and a memory chip 112 that stores the pixel signal. The imaging chip 113, the signal processing chip 111, and the memory chip 112 are stacked, and are electrically connected to each other by a conductive bump 109 such as Cu.

  As shown in the figure, incident light is incident mainly in the positive direction of the Z-axis indicated by a white arrow. In the present embodiment, in the imaging chip 113, the surface on the side where incident light is incident is referred to as a back surface. Further, as shown in the coordinate axes, the left direction of the paper orthogonal to the Z axis is the X axis plus direction, and the front side of the paper orthogonal to the Z axis and the X axis is the Y axis plus direction. In the following several figures, the coordinate axes are displayed so that the orientation of each figure can be understood with reference to the coordinate axes of FIG.

  An example of the imaging chip 113 is a back-illuminated MOS image sensor. The PD layer 106 is disposed on the back side of the wiring layer 108. The PD layer 106 includes a plurality of PDs (photodiodes) 104 that are two-dimensionally arranged and store charges corresponding to incident light, and transistors 105 that are provided corresponding to the PDs 104.

A color filter 102 is provided on the incident side of incident light in the PD layer 106 via a passivation film 103. The color filter 102 has a plurality of types that transmit different wavelength regions, and has a specific arrangement corresponding to each of the PDs 104. The arrangement of the color filter 102 will be described later. A set of the color filter 102, the PD 104, and the transistor 105 forms one pixel.

  On the incident light incident side of the color filter 102, a microlens 101 is provided corresponding to each pixel. The microlens 101 condenses incident light toward the corresponding PD 104.

  The wiring layer 108 includes a wiring 107 that transmits the pixel signal from the PD layer 106 to the signal processing chip 111. The wiring 107 may be multilayer, and a passive element and an active element may be provided.

  A plurality of bumps 109 are disposed on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the opposing surfaces of the signal processing chip 111, and the imaging chip 113 and the signal processing chip 111 are pressed and aligned. The bumps 109 are joined and electrically connected.

  Similarly, a plurality of bumps 109 are disposed on the mutually facing surfaces of the signal processing chip 111 and the memory chip 112. The bumps 109 are aligned with each other, and the signal processing chip 111 and the memory chip 112 are pressurized, so that the aligned bumps 109 are joined and electrically connected.

  The bonding between the bumps 109 is not limited to Cu bump bonding by solid phase diffusion, and micro bump bonding by solder melting may be employed. Further, for example, about one bump 109 may be provided for one block described later. Therefore, the size of the bump 109 may be larger than the pitch of the PD 104. Further, a bump larger than the bump 109 corresponding to the pixel region may be provided in a peripheral region other than the pixel region where the pixels are arranged.

  The signal processing chip 111 has a TSV (silicon through electrode) 110 that connects circuits provided on the front and back surfaces to each other. The TSV 110 is preferably provided in the peripheral area. The TSV 110 may also be provided in the peripheral area of the imaging chip 113 and the memory chip 112.

  FIG. 2 is a diagram for explaining the pixel arrangement of the imaging chip 113. In particular, a state where the imaging chip 113 is observed from the back side is shown. In the pixel area, for example, 8 million or more pixels are arranged in a matrix. In the present embodiment, for example, one block 131 is formed by 4 pixels of 2 pixels × 2 pixels adjacent to each other. Then, one unit U is formed by four blocks of adjacent 2 blocks × 2 blocks. The grid lines in the figure indicate the concept of forming blocks 131 and units U by grouping adjacent pixels. The number of pixels forming the block 131 and the number of blocks 131 forming the unit U are not limited to the above example, and may be more or less.

  As shown in the partially enlarged view of the pixel region, the block 131 includes four so-called Bayer arrays including four pixels, that is, green pixels Gb, Gr, blue pixels B, and red pixels R, vertically and horizontally. The green pixel is a pixel having a green filter as the color filter 102, and receives light in the green wavelength band of incident light. Similarly, a blue pixel is a pixel having a blue filter as the color filter 102 and receives light in the blue wavelength band, and a red pixel is a pixel having a red filter as the color filter 102 and receiving light in the red wavelength band. Receive light.

  In the present embodiment, a plurality of blocks 131 are defined so that one block 131 includes at least one Gb, Gr, B, and R pixels. Each block 131 can control the four pixels in the block 131 with control parameters determined for each block 131. That is, it is possible to acquire imaging signals having different imaging conditions between a pixel group included in one block 131 and a pixel group included in another block 131. Examples of the control parameters are a frame rate, a gain, a thinning rate, the number of addition rows or addition columns to which pixel signals are added, the charge accumulation time or accumulation count, the number of digitization bits, and the like. Furthermore, the control parameter may be a parameter in image processing after obtaining an image signal from a pixel.

  FIG. 3 is a circuit diagram corresponding to one unit U in the imaging chip 113. In FIG. 3, a rectangle enclosed by a dotted line typically represents a circuit corresponding to one pixel. In addition, a rectangle surrounded by a one-dot chain line corresponds to one block 131. Note that at least some of the transistors described below correspond to the transistor 105 in FIG.

  As described above, the unit U is formed of four blocks 131. The reset transistors 303 of the pixels included in the unit U are turned on / off in units of blocks 131. Further, the transfer transistors 302 of the pixels included in the unit U are also turned on / off in units of the block 131. In the example shown in FIG. 3, the reset wiring 300-1 for turning on / off the four reset transistors 303 corresponding to the upper left block 131-1 is provided, and the four transfer transistors corresponding to the block 131-1 are provided. A TX wiring 307-1 for supplying a transfer pulse to 302 is also provided.

  Similarly, a reset wiring 300-3 for turning on / off the four reset transistors 303 corresponding to the lower left block 131-3 is provided separately from the reset wiring 300-1. Further, a TX wiring 307-3 for supplying transfer pulses to the four transfer transistors 302 corresponding to the block 131-3 is provided separately from the TX wiring 307-1.

  Similarly, in the upper right block 131-2 and the lower right block 131-4, the reset wiring 300-2 and the TX wiring 307-2 and the reset wiring 300-4 and the TX wiring 307-4 are provided in the respective blocks 131, respectively. It has been.

  The 16 PDs 104 corresponding to the respective pixels are connected to the corresponding transfer transistors 302, respectively. A transfer pulse is supplied to the gate of each transfer transistor 302 via the TX wiring for each block 131. The drain of each transfer transistor 302 is connected to the source of the corresponding reset transistor 303, and a so-called floating diffusion FD between the drain of the transfer transistor 302 and the source of the reset transistor 303 is connected to the gate of the corresponding amplification transistor 304. The

  The drains of the reset transistors 303 are commonly connected to a Vdd wiring 310 to which a power supply voltage is supplied. A reset pulse is supplied to the gate of each reset transistor 303 via the reset wiring for each block 131.

  The drains of the amplification transistors 304 are commonly connected to a Vdd wiring 310 to which a power supply voltage is supplied. The source of each amplification transistor 304 is connected to the drain of the corresponding selection transistor 305. The gate of each selection transistor 305 is connected to a decoder wiring 308 to which a selection pulse is supplied. In the present embodiment, the decoder wiring 308 is provided independently for each of the 16 selection transistors 305. The source of each selection transistor 305 is connected to a common output wiring 309. The load current source 311 supplies current to the output wiring 309. That is, the output wiring 309 for the selection transistor 305 is formed by a source follower. Note that the load current source 311 may be provided on the imaging chip 113 side or may be provided on the signal processing chip 111 side.

  Here, the flow from the start of charge accumulation to pixel output after the end of accumulation will be described. When a reset pulse is applied to the reset transistor 303 through the reset wiring for each block 131 and simultaneously, a transfer pulse is applied to the transfer transistor 302 through the TX wiring for each block 131, the PD 104 and the floating diffusion FD for each block 131. Is reset.

  When the application of the transfer pulse is canceled, each PD 104 converts received incident light into electric charge and accumulates it. Thereafter, when the transfer pulse is applied again without the reset pulse being applied, the accumulated charge is transferred to the floating diffusion FD, and the potential of the floating diffusion FD changes from the reset potential to the signal potential after the charge accumulation. . When a selection pulse is applied to the selection transistor 305 through the decoder wiring 308, a change in the signal potential of the floating diffusion FD is transmitted to the output wiring 309 through the amplification transistor 304 and the selection transistor 305. Thereby, a pixel signal corresponding to the reset potential and the signal potential is output from the unit pixel to the output wiring 309.

  As described above, in the present embodiment, the reset wiring and the TX wiring are common to the four pixels that form the block 131. That is, the reset pulse and the transfer pulse are simultaneously applied to the four pixels in the block 131, respectively. Accordingly, all the pixels forming a certain block 131 start charge accumulation at the same timing and end charge accumulation at the same timing. However, the pixel signal corresponding to the accumulated charge is selectively output from the output wiring 309 by sequentially applying the selection pulse to each selection transistor 305.

  As described above, in this embodiment, the charge accumulation start timing can be controlled for each block 131. In other words, different blocks 131 can be imaged at different timings.

  FIG. 4 is a block diagram illustrating a functional configuration of the image sensor 100. The analog multiplexer 411 sequentially selects the 16 PDs 104 forming the unit U, and outputs each pixel signal to the output wiring 309 provided corresponding to the unit U. The multiplexer 411 is formed on the imaging chip 113 together with the PD 104.

  The pixel signal output via the multiplexer 411 is supplied to the signal processing chip 111 by a signal processing circuit 412 that performs correlated double sampling (CDS) / analog / digital (A / D) conversion. D conversion is performed. The A / D converted pixel signal is transferred to the demultiplexer 413 and stored in the pixel memory 414 corresponding to each pixel. The demultiplexer 413 and the pixel memory 414 are formed in the memory chip 112.

  The arithmetic circuit 415 processes the pixel signal stored in the pixel memory 414 and passes it to the subsequent image processing unit. The arithmetic circuit 415 may be provided in the signal processing chip 111 or may be provided in the memory chip 112. Note that FIG. 4 shows connections for one unit U, but actually these exist for each unit U and operate in parallel. However, the arithmetic circuit 415 may not exist for each unit U. For example, one arithmetic circuit 415 may perform sequential processing while sequentially referring to the values of the pixel memory 414 corresponding to each unit U.

  As described above, the output wiring 309 is provided corresponding to each unit U. Since the image pickup device 100 has the image pickup chip 113, the signal processing chip 111, and the memory chip 112 laminated, by using electrical connection between the chips using the bump 109 for the output wiring 309, each chip is arranged in the surface direction. Wiring can be routed without increasing the size.

<Description of imaging device>
FIG. 5 is a block diagram illustrating the configuration of the image pickup apparatus 1 having the image pickup device 100 described above. In FIG. 5, the imaging apparatus 1 includes an imaging optical system 10, an imaging unit 20, an image processing unit 30, a work memory 40, a display unit 50, a recording unit 60, and a control unit 70.

  The imaging optical system 10 includes a plurality of lenses, and guides a light beam from the object scene to the imaging unit 20. The imaging optical system 10 may be configured integrally with the imaging device 1 or may be configured to be replaceable with respect to the imaging device 1. Further, the imaging optical system 10 may include a focus lens or a zoom lens.

  The imaging unit 20 includes the above-described imaging device 100 and a driving unit 21 that drives the imaging device 100. The imaging element 100 can be controlled in accordance with the control signal output from the driving unit 21, thereby enabling the above-described imaging timing control for each block 131. The control unit 70 instructs the drive unit 21 to perform imaging control.

  The image processing unit 30 performs image processing on the image data imaged by the imaging unit 20 in cooperation with the work memory 40. In the present embodiment, the work memory 40 temporarily stores image data before and after JPEG compression and before and after MPEG compression, and is used as a buffer memory for images captured by the imaging unit 20. The display unit 50 includes, for example, a liquid crystal display panel 51, and displays images (still images and moving images) and various information captured by the imaging unit 20, and displays an operation input screen. The display unit 50 has a configuration in which a touch panel 52 is laminated on the display surface of the liquid crystal display panel 51. The touch panel 52 outputs a signal indicating a position where the user touches the liquid crystal display panel 51.

  The recording unit 60 stores various data such as image data acquired in response to an imaging instruction (for example, operation of the touch panel 52) in a storage medium such as a memory card. The control unit 70 has a CPU and controls the overall operation of the imaging apparatus 1. The control unit 70 drives the control parameter so that each block 131 of the imaging device 100 (imaging chip 113) acquires an image at a predetermined frame rate and gain, and controls reading of the acquired image data. 21 is instructed. In addition, the control unit 70 causes the reproduction processing unit 71 to perform a reproduction process for displaying an image on the display unit 50 based on the pixel signal read from the image sensor 100.

<Video shooting>
Moving image shooting performed by the imaging apparatus 1 will be described. When a touch operation instructing shooting of a moving image is performed, the control unit 70 performs accumulation control of the image sensor 100 (image pickup chip 113) so as to acquire an image at a frame rate of 240 fps, for example. FIG. 8 is a diagram illustrating the order of frames acquired at the time of moving image shooting and the order of frames to be displayed at the time of moving image playback. When shooting a movie, the timing is shifted by a predetermined time (in this example, 1/240 seconds) in the order of group 1, group 2, group 3, and group 4, which will be described later, and imaged every 1/60 seconds for each of the four groups. repeat.

  In the accumulation control for the imaging chip 113, one unit U is divided into four groups (group 1 to group 4) in units of blocks 131, and the accumulation control for each group is performed in cooperation between the units U. FIG. 6 is a diagram for explaining the arrangement of groups 1 to 4 in the imaging chip 113.

  The number of units U in the case of having about 8 million pixels of about 4000 pixels in the horizontal direction × about 2000 pixels in the vertical direction is about 1000 in the horizontal direction × about 500 in the vertical direction, for a total of about 500,000 units. By shifting the charge accumulation start timing of the four blocks 131 in one unit U by 1/240 seconds and dividing the image into four stages, four sets of pixel signals having different acquisition timings per unit U can be obtained. . In FIG. 6, reference numerals 1, 2, 3, and 4 indicate the blocks 131 in the order of early acquisition timing. According to FIG. 6, for the four blocks 131 included in the unit U, the image acquisition order (reference numerals 1, 2, 3, 4) in the unit U is the same among all the units U.

  In the present embodiment, the group 131 is configured by collecting the blocks 131 represented by reference numeral 1 from all the units U of the imaging chip 113. A group 131 is formed by collecting the blocks 131 represented by reference numeral 2 from all the units U of the imaging chip 113. Further, a group 131 is formed by collecting blocks 131 represented by reference numeral 3 from all the units U of the imaging chip 113. Similarly, the group 131 is formed by collecting blocks 131 represented by reference numeral 4 from all the units U of the imaging chip 113.

  By performing the accumulation control for each group in cooperation between all the units U of the imaging chip 113, the imaging timings (charge accumulation timings) of all the blocks 131 constituting the group 1 are included in different units U. Are controlled in the same way. Similarly, the imaging timings (charge accumulation timings) of all the blocks 131 constituting the group 2, group 3, and group 4 are also controlled to be the same in each group.

<Video playback display>
The control unit 70 displays a moving image on the display unit 50 based on the pixel signal read from the image sensor 100. The playback display of the moving image can be selected from normal playback for playback and display at the same frame rate as that at the time of imaging and slow playback for playback and display at a frame rate slower than the frame rate at the time of imaging. The control unit 70 instructs the reproduction processing unit 71 to perform normal reproduction display or slow reproduction display in accordance with an operation performed via the touch panel 52.

<8x slow playback>
The reproduction processing unit 71 instructed to perform the 8 × slow reproduction performs the slow reproduction process at a frame rate (30 fps) that is 1/8 of the frame rate at the time of shooting (for example, 240 fps) based on the pixel signal read from the image sensor 100. Do. As shown in FIG. 7, the reproduction processing unit 71 extracts pixel signals belonging to the group 1 (that is, the block 131 indicated by reference numeral 1 in FIG. 6) to obtain one image 81. Further, the reproduction processing unit 71 extracts pixel signals belonging to the group 2 (that is, the block 131 indicated by reference numeral 2 in FIG. 6) to obtain one image 82. Further, the reproduction processing unit 71 extracts pixel signals belonging to the group 3 (that is, the block 131 indicated by reference numeral 3 in FIG. 6) to obtain one image 83. Furthermore, the reproduction processing unit 71 extracts pixel signals belonging to the group 4 (that is, the block 131 indicated by reference numeral 4 in FIG. 6) to obtain one image 84. FIG. 7 is a diagram illustrating the four images 81 to 84 extracted in this way. The aspect ratios of the images 81 to 84 are substantially equal to the aspect ratio of the image captured in the entire imaging region of the imaging device 100 (imaging chip 113).

  The pixel sizes of the four images 81 to 84 are about 2000 million in total, each having about 2000 in the horizontal direction and about 1000 in the vertical direction (so-called full HD equivalent). As illustrated in FIG. 8, the reproduction processing unit 71 sequentially performs the group 1 image, the group 3 image, the group 4 image, the next group 1 image, and the next group in order from the group 1 image with the earlier acquisition timing. The second image is reproduced and displayed at 30 fps.

<8x slow playback (2)>
When the accumulation time is longer than the frame rate, the reproduction processing unit 71 instructed to perform the 8 × slow reproduction based on the pixel signal read from the image sensor 100 フ レ ー ム frames of 1/8 of the frame rate at the time of shooting (for example, 240 fps) Slow playback processing is performed at a rate (30 fps). In this example, the accumulation time per frame is 1/60 second, which is longer than the frame rate at the time of shooting.

  As illustrated in FIG. 7, the reproduction processing unit 71 obtains four images 81 to 84 corresponding to the groups 1 to 4. Then, as illustrated in FIG. 9, the reproduction processing unit 71 sequentially starts with the group 1 image, the group 3 image, the group 4 image, the next group 1 image, The group 2 images are reproduced and displayed at 30 fps.

  Since the accumulation time is longer than the frame rate, when shooting a fast-moving subject, the subject image may be blurred in each frame, but natural display is possible by performing slow playback.

<2x slow playback>
The playback processing unit 71 instructed to perform double slow playback performs slow playback processing at a frame rate (30 fps) that is ½ of the frame rate at the time of shooting (for example, 60 fps) based on the pixel signal read from the image sensor 100. Do. The reproduction processing unit 71 obtains four images 81 to 84 corresponding to the groups 1 to 4 as in the case of 8 × slow reproduction (FIG. 7).

  The reproduction processing unit 71 adds four pixels 81 to 84 corresponding to the groups 1 to 4 and the acquisition timings are continuous to obtain one image. In pixel addition, corresponding pixel signals of the same color are added in the corresponding unit U. The pixel size of the four images 81 to 84 is about 2000 million in the horizontal direction and about 1000 in the vertical direction (equivalent to so-called full HD). The total pixel size is about 2000 million in the horizontal direction and about 1000 in the vertical direction (so-called full HD equivalent).

  As shown in FIG. 10, since four images 81 to 84 having different imaging timings by 1/240 seconds are subjected to pixel addition, the frame rate of the image after the addition is equivalent to 60 fps. Here, the blocks 131 constituting the images 81 to 84 of the group 1 to the group 4 have different relative positions in the unit U, so that the images of the groups 1 to 4 may have a spatial dead zone. . However, the spatial dead zone can be eliminated by the pixel addition.

  As illustrated in FIG. 10, the reproduction processing unit 71 reproduces and displays the added image at 30 fps each time the four images 81 to 84 corresponding to the groups 1 to 4 are added (60 fps). It should be noted that the accumulation time per frame may be longer than the frame rate at the time of shooting, as in the above-mentioned 8-times slow playback (part 2).

<Normal playback>
The reproduction processing unit 71 instructed to perform normal reproduction performs reproduction processing at the same frame rate (30 fps) as the frame rate at the time of shooting (for example, 30 fps) based on the pixel signal read from the image sensor 100. The reproduction processing unit 71 obtains four images 81 to 84 corresponding to the groups 1 to 4 as in the case of 8 × slow reproduction (FIG. 7).

  The reproduction processing unit 71 sets four images 81 to 84 corresponding to the groups 1 to 4 as one cycle, and adds eight pixels over two consecutive cycles to obtain one image. In pixel addition, corresponding pixel signals of the same color are added in the corresponding unit U. Since the pixel size of 8 images for 2 cycles is about 2 million in total having about 2000 in the horizontal direction and about 1000 in the vertical direction (equivalent to so-called full HD), The pixel size of the image is about 2,000 in the horizontal direction and about 1,000 in the vertical direction, which is about 2 million in total (equivalent to so-called full HD).

  As shown in FIG. 11, since eight images having different imaging timings by 1/240 seconds are subjected to pixel addition, the frame rate of the image after the addition is equivalent to 30 fps. Since the blocks 131 constituting the images 81 to 84 of the groups 1 to 4 have different relative positions in the unit U, the images of the groups 1 to 4 may have spatial dead zones. Spatial dead zones can be eliminated by pixel addition.

  As illustrated in FIG. 11, the reproduction processing unit 71 reproduces and displays the added image at 30 fps every time two images 81 to 84 corresponding to group 1 to group 4 are added for two cycles (30 fps). To do. As described above, the accumulation time per frame may be longer than the frame rate at the time of shooting.

<Double slow playback (2)>
The other reproduction processing unit 71 instructed to perform the double slow reproduction reproduces at a frame rate (30 fps) that is ½ of the frame rate at the time of shooting (for example, 60 fps) based on the pixel signal read from the image sensor 100. Perform playback processing. For example, the reproduction processing unit 71 obtains only the image 81 corresponding to the group 1.

  As illustrated in FIG. 12, the reproduction processing unit 71 reproduces and displays the group 1 image at 30 fps each time four images 81 to 84 corresponding to the group 1 to group 4 are acquired (60 fps). Here, the image used for the double slow reproduction (part 2) is not limited to the image 81 corresponding to the group 1. For example, only the image 82 corresponding to the group 2 can be used, or only the image 84 corresponding to the group 4 can be used. As described above, the accumulation time per frame may be longer than the frame rate at the time of shooting.

<4x slow playback>
The playback processing unit 71 instructed to perform 4 × slow playback performs slow playback processing at a frame rate (equivalent to 15 fps) that is ¼ of the frame rate at the time of shooting (for example, 60 fps) based on the pixel signal read from the image sensor 100. I do. For example, the reproduction processing unit 71 obtains an image 81 and an image 83 corresponding to the group 1 and the group 3.

  Each time the reproduction processing unit 71 acquires four images 81 to 84 corresponding to the groups 1 to 4 (60 fps), the image of the group 1 and the image of the group 3 are each 1/30 second. Display playback. Here, the images used for the 4 × slow playback are not limited to the images 81 and 83 corresponding to the groups 1 and 3, and the images 82 and 84 corresponding to the groups 2 and 4 can also be used. . As described above, the accumulation time per frame may be longer than the frame rate at the time of shooting.

<Description of flowchart>
FIG. 13 is a flowchart for describing the flow of a shooting operation that is executed by the control unit 70 of the imaging apparatus 1 in the moving image mode. The control unit 70 repeatedly performs the process of FIG. 13 when the imaging apparatus 1 is set to the moving image mode. In step S <b> 101 of FIG. 13, the control unit 70 starts imaging by the imaging unit 20. As illustrated in FIG. 8, the acquisition of the image starting in step S101 is performed by dividing the charge accumulation start timing by 1/240 seconds for each of the groups 1 to 4 and dividing them into four stages.

  In step S <b> 102, the control unit 70 displays a monitor image on the display unit 50 based on the image data acquired by the imaging unit 20. The control unit 70 performs normal reproduction as illustrated in FIG. 9, for example. Specifically, every time four images 81 to 84 corresponding to group 1 to group 4 are added for two cycles (30 fps), the added image is reproduced and displayed at 30 fps.

  In step S103, the control unit 70 performs control on the imaging unit 20 and display control on the display unit 50 based on the set imaging conditions (such as shutter speed) and display conditions (normal reproduction or slow reproduction). Do.

  In step S104, the control unit 70 determines whether a change operation has been performed. When an operation is performed via the touch panel 52, the control unit 70 makes a positive determination in step S104 and returns to step S103. In this case, the control unit 70 instructs the imaging unit 20 to change the imaging condition or instructs the reproduction processing unit 71 to change the reproduction speed in accordance with the performed operation. When the operation is not performed via the touch panel 52, the control unit 70 makes a negative determination in step S104 and proceeds to step S105.

  In step S105, the control unit 70 determines whether a recording start operation has been performed. When an operation is performed via the touch panel 52, the control unit 70 makes a positive determination in step S105, starts the recording process, and proceeds to step S106. In the recording process, for example, the data of all acquired images for the groups 1 to 4 are stored in a memory card or the like by the recording unit 60. When the recording start operation is not performed, the control unit 70 makes a negative determination in step S105 and returns to step S103.

  In step S106, the control unit 70 determines whether or not a recording end operation has been performed. When an operation is performed via the touch panel 52, the control unit 70 makes a positive determination in step S106, ends the recording process, and ends the process of FIG. When the recording end operation is not performed, the control unit 70 makes a negative determination in step S106 and waits for the end operation to be performed.

According to the embodiment described above, the following operational effects can be obtained.
(1) Unlike the first area (reference numeral 1) and the first area (reference numeral 1), the imaging device 1 is arranged in two dimensions discretely, which includes a plurality of blocks 131 discretely arranged in two dimensions. With respect to one frame time (1/240 seconds) of the frame rate of the image sensor 100 having the second area (for example, reference numeral 2) having the plurality of blocks 131 and the first and second areas, The first region is accumulated with a first accumulation time (1/60 seconds) longer than the frame time (1/240 seconds), and the second region is accumulated with a second accumulation time (1 / 60 seconds). Thereby, for example, a plurality of images having different accumulation timings can be obtained for the same subject, and the usability is improved. This is particularly suitable for dark subjects.

(2) Since the control unit 70 makes the first accumulation time and the second accumulation time substantially the same, a plurality of images having a common accumulation time can be obtained.

(3) The control unit 70 starts the accumulation of all the blocks 131 in the first area (reference numeral 1) substantially simultaneously and starts the accumulation of all the blocks in the second area (for example, numeral 2) almost simultaneously. The accumulation start timings of the blocks 131 can be aligned in the area.

(4) It has the control part 70 which reads the accumulation | storage charge of the image pick-up element 100, and the control part 70 is imaged by the whole area | region of the image pick-up element 100 about the aspect ratio of the image imaged in 1st area | region (code | symbol 1). Therefore, the aspect ratio between the images can be made uniform.

(5) It has the control part 70 which reads the stored charge of the image pick-up element 100, and the control part 70 has the some block 131 of a 1st area | region (code | symbol 1) and the some block 131 of a 2nd area | region (code | symbol 2). Are arranged on the basis of regularity, so that the image pickup device 100 can be easily manufactured as compared with the case where they are arranged irregularly.

(6) Since the control unit 70 (reproduction processing unit 71) that processes image data captured by controlling the accumulation of the first and second areas by the control unit 70 is provided, the blocks 131 that are discretely arranged are provided. It is possible to appropriately process the imaging data obtained by.

(7) Since the control unit 70 (reproduction processing unit 71) sets the frame rate at the time of reproduction display of the imaging data to be slower than the frame rate at the time of imaging data capture, the moving image is appropriately reproduced by slow reproduction. it can.

(8) The control unit 70 (reproduction processing unit 71) sets the imaging data captured in the first area (reference numeral 1) and the imaging data captured in the second area (for example, reference numeral 2) to the position of each block 131. Addition is performed in correspondence to generate additional imaging data, and the frame rate when reproducing and displaying the additional imaging data is set slower than the frame rate when imaging data is captured. it can. In addition, even if the imaging data captured in the first area (reference numeral 1) or the imaging data captured in the second area (reference numeral 2) includes a spatial dead band, The effect of eliminating the spatial dead zone can also be expected.

(9) The control unit 70 (reproduction processing unit 71) includes a plurality of imaging data before and after shooting in the first area (reference numeral 1), and a plurality of imaging data before and after shooting in the second area (for example, numeral 2). Are sequentially added corresponding to the position of each block 131 to generate added image data, and the added image data is sequentially reproduced and displayed, so that the moving image can be appropriately reproduced and displayed normally. In addition, even if the imaging data captured in the first area (reference numeral 1) or the imaging data captured in the second area (reference numeral 2) includes a spatial dead band, The effect of eliminating the spatial dead zone can also be expected.

(10) The control unit 70 (reproduction processing unit 71) sequentially reproduces either the image data captured in the first area (reference numeral 1) or the image data captured in the second area (for example, numeral 2). Since the frame rate for displaying and sequentially reproducing and displaying is set to be slower than the frame rate at the time of capturing the imaging data, the moving image can be appropriately reproduced and displayed in slow motion.

(11) The control unit 70 (reproduction processing unit 71) sequentially captures the imaging data of the first area (symbol 1) and the imaging data of the second area (symbol 2) at a frame rate corresponding to a time longer than the accumulation time. Since the process for reproducing and displaying is performed, the moving image can be appropriately reproduced by slow reproduction.

(12) In the imaging device 100, in addition to the first area (reference numeral 1) and the second area (reference numeral 2), a plurality of blocks 131 different from the blocks 131 included in the first area and the second area are two-dimensional. At least one other region (for example, reference numeral 3) that is discretely arranged in each of the regions, and the control unit 70 is configured so that each region corresponds to one frame time (1/240 seconds) of the frame rate of each region. Are stored with an accumulation time (1/60 seconds) longer than the frame time (1/240 seconds). Thereby, for example, at least three images having different accumulation timings are obtained for the same subject, and the usability is improved. This is particularly suitable for dark subjects.

(13) It has the control part 70 which reads the accumulation | storage charge of the image pick-up element 100, and since the control part 70 arrange | positions the several blocks 131 of each area | region based on regularity, it images compared with the case where it arranges irregularly. The device 100 can be easily manufactured.

(14) Since the image pickup device 100 employs a structure in which a back-illuminated image pickup chip and a signal processing chip are stacked, each chip can be wired without increasing in the surface direction.

(Modification 1)
In the above-described embodiment, an example in which the order of image acquisition (reference numerals 1, 2, 3, and 4) in the unit U is the same among all the units U for the four blocks 131 included in the unit U is described. did. Instead, for the four blocks 131 included in the unit U, the order of image acquisition in the unit U may be different at random from the other units U.

  FIG. 14 is a diagram illustrating the arrangement of groups 1 to 4 in the imaging chip of the first modification. In FIG. 14, the relative positions of reference numerals 1, 2, 3, and 4 in one unit U are randomly changed in each unit U. That is, the order of image acquisition (symbols 1, 2, 3, 4) of the four blocks 131 in a certain unit U differs between the adjacent units U. By eliminating the regularity of the order of image acquisition in the unit U, it helps to reduce moire that tends to occur when, for example, a subject having a regular repeating pattern is imaged.

(Modification 2)
For the four blocks 131 included in the unit U, the order of image acquisition (reference numerals 1, 2, 3, 4) in the unit U may be given a predetermined regularity with other units U. . FIG. 15 is a diagram illustrating the arrangement of groups 1 to 4 in the imaging chip of the second modification. In FIG. 15, the order of image acquisition in the unit U (blocks 1, 2, 3, 4) in the unit U are adjacent to each other so that the blocks 131 having the same image acquisition order (reference numerals 1, 2, 3, 4) are adjacent to each other. The reference numerals 1, 2, 3, 4) are given regularity.

  According to FIG. 15, the codes 1, 2, 3, and 4 in a certain unit U are adjacent to each other between the adjacent units U. As a result, four blocks 131 having the same code are gathered together. In FIG. 15, the hatched area is an example of an area where four blocks 131 having the same sign are gathered. In this manner, by arranging the blocks 131 having the same imaging timing close to each other, the imaging device 100 can be easily manufactured as compared to the case where the blocks 131 having different imaging timings are gathered.

(Modification 3)
In the embodiment described above, the example in which the sources of the total 16 select transistors 305 provided in the four blocks 131 included in the unit U are connected to the common output wiring 309 has been described. Instead of this, the source of the four selection transistors 305 provided for each block 131 may be connected to the output wiring provided for each block 131.

  FIG. 16 is a circuit diagram corresponding to one unit U in the imaging chip 113 of the third modification. Compared with the case of FIG. 3, FIG. 16 is different in that output wirings 309-1 to 309-4 are provided in units of blocks. That is, the sources of the four selection transistors 305 corresponding to the upper left block 131-1 are connected to the output wiring 309-1 for the upper left block.

  The sources of the four selection transistors 305 corresponding to the upper right block 131-2 are connected to the output wiring 309-2 for the upper right block. The sources of the four selection transistors 305 corresponding to the lower left block 131-3 are connected to the output wiring 309-3 for the lower left block. Further, the sources of the four selection transistors 305 corresponding to the lower right block 131-4 are connected to the output wiring 309-4 for the lower right block. The load current sources 311-1 to 311-4 supply current to the corresponding output wirings 309-1 to 309-4, respectively.

  FIG. 17 is a block diagram illustrating a functional configuration of the image sensor 100 according to the third modification. Compared to the case of FIG. 4, FIG. 17 is different in that four sets of a multiplexer 411, a signal processing circuit 412, and a demultiplexer 413 are provided for one unit U. That is, one multiplexer 411 sequentially selects the four PDs 104 in the corresponding block 131 and outputs each pixel signal to the corresponding output wiring 309.

  The pixel signal output via the multiplexer 411 is subjected to CDS and A / D conversion by the signal processing circuit 412. The A / D converted pixel signal is transferred to the demultiplexer 413 and stored in the pixel memory 414 corresponding to each pixel. Thus, the 16 pixel signals of one unit U are distributed to four sets of paths and processed in parallel. According to the third modification, since one signal processing circuit 412 only needs to perform A / D conversion processing on four pixel signals, there are 16 signal processing circuits 412 as shown in FIGS. Compared with the case where A / D conversion processing is performed on this pixel signal, the driving time is reduced to ¼. Shortening the driving time is effective in reducing the amount of heat generated by the circuit.

(Modification 4)
In the embodiment described above, the camera is exemplified as the electronic device. However, the electronic device may be configured by a high-function mobile phone or a tablet terminal. In this case, a camera unit mounted on a high-function mobile phone (or tablet terminal) is configured using the multilayer image sensor 100.

  The above description is merely an example, and is not limited to the configuration of the above embodiment. You may combine suitably the structure of the said embodiment and each modification.

DESCRIPTION OF SYMBOLS 1 ... Imaging device 10 ... Imaging optical system 20 ... Imaging part 30 ... Image processing part 40 ... Work memory 50 ... Display part 60 ... Recording part 70 ... Control part 71 ... Reproduction processing part 81-84 ... Image 100 ... Imaging element 111 ... Signal processing chip 112 ... Memory chip 113 ... Imaging chip 131 ... Block U ... Unit

Claims (16)

Imaging having a first region having a plurality of blocks discretely arranged in two dimensions and a second region having a plurality of blocks discretely arranged in two dimensions unlike the first region Elements,
With respect to one frame time of the frame rate of the first and second regions, the first region is stored with a first storage time longer than the frame time, and the second region is stored with a first time longer than the frame time. And an accumulation control unit that accumulates according to two accumulation times. The electronic device according to claim 1,
The electronic apparatus according to claim 1, wherein the accumulation control unit makes the first accumulation time and the second accumulation time substantially the same. The electronic device according to claim 1 or 2,
The electronic apparatus is characterized in that the accumulation control unit starts accumulation of all blocks in the first area substantially simultaneously and starts accumulation of all blocks in the second area almost simultaneously. The electronic device according to any one of claims 1 to 3,
A readout unit for reading out the accumulated charge of the image sensor;
The electronic device according to claim 1, wherein the reading unit sets an aspect ratio of an image captured in the first area to be substantially the same as an aspect ratio of an image captured in the entire area of the image sensor. The electronic device according to any one of claims 1 to 4,
A readout unit for reading out the accumulated charge of the image sensor;
The electronic device, wherein the reading unit arranges the plurality of blocks in the first area and the plurality of blocks in the second area based on regularity. The electronic device according to any one of claims 1 to 5,
An electronic apparatus comprising: a processing unit that processes image data captured by controlling the accumulation of the first and second areas by the accumulation control unit. The electronic device according to claim 6,
The electronic device, wherein the processing unit makes a frame rate at the time of reproduction display of the imaging data slower than a frame rate at the time of imaging of the imaging data. The electronic device according to claim 6,
The processing unit generates the added imaging data by adding the imaging data captured in the first area and the imaging data captured in the second area in correspondence with the position of each block, and generates the additional imaging data. An electronic apparatus characterized in that a frame rate at the time of reproducing and displaying data is made slower than a frame rate at the time of photographing the image data. The electronic device according to claim 6,
The processing unit sequentially adds a plurality of imaging data before and after shooting of the first area and a plurality of imaging data before and after shooting of the second area corresponding to the position of each block, and performs addition imaging An electronic apparatus that generates data and sequentially reproduces and displays the added imaging data. The electronic device according to claim 6,
The processing unit sequentially reproduces and displays either the image data captured in the first area or the image data captured in the second area, and sets the frame rate for the sequential reproduction display of the imaging data. An electronic device characterized by being slower than the frame rate at the time of shooting. The electronic device according to claim 6,
The electronic device performs processing for sequentially reproducing and displaying the imaging data of the first area and the imaging data of the second area at a frame rate corresponding to a time longer than the accumulation time. . The electronic device according to any one of claims 1 to 11,
In the imaging device, in addition to the first area and the second area, a plurality of blocks different from the blocks included in the first area and the second area are two-dimensionally discretely arranged. Further having at least one other region,
The electronic apparatus is characterized in that the accumulation control unit accumulates each area with an accumulation time longer than the frame time with respect to one frame time of the frame rate of each area. The electronic device according to claim 12,
A readout unit for reading out the accumulated charge of the image sensor;
The electronic device according to claim 1, wherein the reading unit arranges the plurality of blocks in each region based on regularity. The electronic device according to claim 12,
A readout unit for reading out the accumulated charge of the image sensor;
The electronic device is characterized in that the reading unit is arranged so that the plurality of blocks are adjacent to each other in the respective regions of the image sensor. The electronic device according to claim 12,
A readout unit for reading out the accumulated charge of the image sensor;
The electronic device is characterized in that the reading unit is arranged so that the plurality of blocks are different between adjacent regions in each region of the imaging element. The electronic device according to any one of claims 1 to 15,
The electronic device has a structure in which a back-illuminated imaging chip and a signal processing chip are stacked.

Images