JP2017034567A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To simultaneously record a full screen image and an enlarged image of a subject that is moving around with a camera constituted of two sensors that receive light of the same optical image and record data without discomfort feeling even when being simultaneously reproduced by adjusting reading timing at a time of recording.SOLUTION: The imaging apparatus starts accumulation of a first sensor after determining a specific area for enlarging of a first imaging element, and starts image signal reading from the first sensor before completion of reading the next frame of a frame of a second sensor, which is a base for determining with the part region specifying means.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus, and more particularly to an imaging apparatus equipped with two imaging elements.

  Conventionally, many image pickup devices equipped with a CCD or CMOS image pickup device are used. In particular, since a CMOS image sensor can use a general-purpose semiconductor manufacturing apparatus, the manufacturing cost of the image sensor can be reduced. A product or technology in which a plurality of CMOS image sensors are mounted on an image pickup apparatus has been widely created as one of the advantages.

  In Patent Document 1, in an imaging apparatus equipped with a plurality of image sensors, one image sensor reads a part of interest at high speed and high resolution, and the other image sensor reads the entire screen. A technique for synthesizing and recording an image is disclosed.

  Japanese Patent Application Laid-Open No. 2004-228561 has a technique for changing the accumulation period of each of a plurality of image sensors, selecting and outputting a signal from one of the image sensors according to the signal amount at the pixel position, and outputting the image as a single image. It has been published.

JP 2008-228232 A JP 2000-78463 A

  However, in the prior art disclosed in the above-mentioned patent document, in an imaging device equipped with two imaging elements, one image sensor reads the entire screen, and the other imaging element uses a high-resolution part of interest. Reading and recording each is not listed.

  Therefore, an object of the present invention is to read an entire screen with the first image sensor and analyze and record the image on the entire screen in an image pickup apparatus equipped with two image sensors. In parallel, the position of the second image sensor to be read at high resolution is determined from the analysis result of the full screen (a part of the image is read at high resolution and is displayed in an enlarged manner to be output to a display device having the same size as the full screen. This reading method is hereinafter referred to as enlarged reading). Further, driving is performed so that the position extracted by the second image sensor becomes a position desired by the user. In addition, there is provided an imaging apparatus capable of driving both imaging elements during recording so that the user does not feel uncomfortable when the first full-screen video and the enlarged video are reproduced simultaneously.

In order to achieve the above object, an imaging apparatus according to the present invention includes:
A first imaging element in which pixels including photoelectric conversion elements are two-dimensionally arranged, a second imaging element in which pixels including photoelectric conversion elements are two-dimensionally arranged, and first and second imaging elements, respectively A light splitting means arranged so that an image is incident on a common imaging optical system, and a charge accumulation control means for controlling charge accumulation in each of the pixel portions of the first and second imaging elements. In the imaging apparatus having a reading area specifying means for specifying the second reading area based on the image read by the imaging element, the first sensor starts to accumulate after the specific area of the first imaging element is determined, The image signal reading from the first sensor is started before the reading of the next frame of the frame of the second sensor, which is a basis determined by the partial region specifying means, is started.

  According to the imaging apparatus according to the present invention, it is possible to improve the followability of an image to be enlarged and read, and to record an enlarged moving image desired by a user. Furthermore, it is possible to prevent the user from feeling uncomfortable when the full-screen moving image and the enlarged reading moving image are simultaneously reproduced.

FIG. 3 is a drive timing chart of the image sensor in Embodiment 1 of the present invention. The block diagram which shows the structure of the imaging device in the Example of this invention Simplified configuration diagram of an image sensor 1401 in an embodiment of the present invention Configuration diagram of image sensor 203 in the embodiment of the present invention Image sensor drive timing chart when recording a moving image in an embodiment of the present invention FIG. 6 is a diagram for explaining the number of pixels of the image sensor in Embodiment 1 of the present invention. 1 is a simplified configuration diagram of an image sensor 1400 according to a first embodiment of the present invention. The figure which shows the example of an image of a full screen and expansion when the expansion area is specified in Example 1 of this invention. The figure which shows the example of an image when an expansion area moves in Example 1 of this invention. The flowchart figure in Example 1 of this invention. Timing chart of driving of image pickup device when embodiment 2 of the present invention is not carried out Timing chart of driving of image pickup device in Embodiment 2 of the present invention The flowchart figure in Example 2 of this invention. Timing chart of driving of image pickup device in Embodiment 3 of the present invention The flowchart figure in Example 3 of this invention. Explanatory drawing when setting the AF point in the fourth embodiment of the present invention as an enlarged region

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 2 is a block diagram of the imaging apparatus according to the embodiment of the present invention. Reference numeral 100 denotes an image processing apparatus, 200 denotes a recording medium such as a memory card or a hard disk, and 300 denotes a lens unit. Next, details of the block will be described.

  Reference numeral 100 denotes an image processing apparatus, reference numeral 12 denotes a shutter that controls the accumulation amount of the imaging element 1400, and reference numerals 1400 and 1401 denote imaging elements that convert an optical image into an electrical signal. In this embodiment, the configurations of the image pickup elements 1400 and 1401 are the same. The configuration may be different as long as the imaging element can obtain the same optical image even in different configurations. The resolution may be the same or different. Reference numeral 13 denotes a transmission mirror that splits light incident from the lens unit 300.

  Reference numerals 1600 and 1601 denote analog front ends (hereinafter referred to as AFEs) each including an A / D converter that converts analog signals output from the image sensors 1400 and 1401 into digital signals. Reference numerals 1800 and 1801 denote timing generators (hereinafter referred to as TG) for supplying a clock signal and a control signal to the image sensor and the A / D converter, respectively.

  The AFEs 1600 and 1601 and the TGs 1800 and 1801 can be incorporated in the image sensors 1400 and 1401. Reference numeral 50 denotes a system control circuit (hereinafter referred to as CPU) that controls the entire imaging apparatus 100 including image processing. Reference numeral 60 denotes a mode dial switch, which can be set by switching between each function shooting mode such as an automatic shooting mode, a program shooting mode, a shutter speed priority shooting mode, an aperture priority shooting mode, and a bulb shooting mode.

  Reference numeral 61 denotes a shutter switch. The shutter switch 64 is in two stages. Pressing lightly to the first level is called half-pressing, and pressing deeply to the second level is called full-pressing. When half-pressed, the automatic pin is aligned and the shutter speed and aperture value are set by the automatic exposure mechanism before shooting. When fully pressed, the shutter 12 operates and the photographing operation is performed. Reference numeral 62 denotes a moving image recording / start / stop switch. When an instruction to start recording is given, the moving image recording operation is continuously performed.

  Reference numeral 63 denotes a moving image full-screen / enlargement and simultaneous recording start / stop switch. When recording start is instructed, the moving image recording operation for both full-screen / enlarged screen described below is performed continuously. 64 ISO sensitivity setting switch, which can set the sensitivity of the imaging apparatus 100 to the amount of light. Reference numeral 65 denotes a dial type switch to be set, which sets a period during which the shutter 12 is opened and stored in the image sensor 1400. In addition, in the moving image, the time from the reset start of each line to the start of reading is set.

  Reference numeral 66 denotes a power switch that switches the power of the imaging apparatus 100 on and off. In addition, the power-on and power-off settings of various accessory devices such as the lens unit 300, the external strobe, and the recording medium 200 connected to the imaging device 100 can be switched. Reference numeral 70 denotes a volatile memory (hereinafter referred to as RAM) that temporarily records image data converted into digital data by the AFEs 1600 and 1601 and image data processed by image processing units 72 and 74 described later. It also has a function as a work memory of the CPU 50.

  Reference numeral 71 denotes a nonvolatile memory (ROM) that stores a program used when the CPU 50 operates. An image processing unit 72 performs processing such as still image correction / compression. An image processing unit 73 performs processing such as moving image correction / compression. An image processing unit 74 performs processing such as correction / compression of a moving image enlarged image.

  In this case, it has one CPU 50 and is configured to process both still images or enlarged movies and movies. However, it has two CPUs, while still processing still images or enlarged movies on the other side. You may process screen animation. Reference numeral 75 denotes an enlarged reading portion specifying circuit for detecting a target portion from a full-screen moving image, which will be described later, and specifying an area when creating an enlarged moving image. A power control unit 80 includes a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, and the like. Further, the presence / absence of a battery, the type of battery, the remaining battery level are detected, the DC-DC converter is controlled based on the detection result and the instruction of the system control circuit 50, and the necessary voltage is recorded for a necessary period. Supply to each part including

  Reference numerals 82 and 84 denote connectors, and 86 denotes a power supply unit including a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a Li battery, an AC adapter, or the like. Reference numeral 90 denotes an interface with a recording medium such as a memory card or hard disk, and reference numeral 92 denotes a connector for connecting to a recording medium such as a memory card or hard disk. Reference numeral 200 denotes a recording medium such as a memory card or a hard disk. The recording medium 200 includes a recording unit 202 composed of a semiconductor memory, a magnetic disk, and the like, and an interface 206 with the imaging device 100.

  Reference numeral 300 denotes a lens unit, 310 denotes a photographing lens, 312 denotes an aperture, and 316 and 106 denote lens mounts for connecting the lenses. Reference numeral 320 denotes a lens control unit, and reference numeral 322 denotes a connector that electrically connects the lens unit 300 to the imaging apparatus 100. The lens control unit 320 receives a signal from the imaging apparatus 100 via the connectors 322 and 122. The position of the imaging lens 310 on the optical axis is changed by the signal to control the focus. Similarly, the lens control unit 320 receives a signal from the image processing apparatus 100 and controls the size of the diaphragm 312. Reference numeral 120 denotes an interface for communicating with the lens unit 300 using an electrical signal.

  FIG. 3 is a simplified configuration diagram of the image sensor 1401.

  The image sensor 1401 pixel 203 is two-dimensionally arranged. Arranged in a matrix, the vertical arrangement is called a “column” and the horizontal arrangement is called a “row”. The vertical scanning circuit 202 outputs a signal necessary for row selection for reading out the selected row and readout of electric charge in each row to the circuit of each pixel.

  The image signal of each pixel in the row selected by the row selection circuit 206 provided in the vertical scanning circuit 202 is output to the vertical output line 310. It also has a function of resetting each pixel in the row selected by the row selection circuit 206 when reading a moving image. The signal output to the vertical output line 310 is output to the horizontal scanning circuit 201 via the column gain 204 and the column circuit 205 connected to each vertical output line. The horizontal scanning circuit 201 sequentially outputs the signal output for one row in the horizontal direction.

  FIG. 4 is a diagram showing details of the pixel 203.

  The charge generated and stored in the photodiode 304 is stored in the floating diffusion (hereinafter referred to as FD) 307 by controlling the transfer control signal 301 and operating the transfer switch 305. The source follower amplifier 308 amplifies a voltage signal based on the electric charge accumulated in the FD 307 and outputs it as a pixel signal. A row selection control signal 303 controls a row selection switch 309 to connect the output of the source follower amplifier to the vertical output line 310.

  When resetting unnecessary charges accumulated in the photodiode 304 or the FD 307, the reset switch 306 is controlled by the reset control signal 302, the transfer control signal 301 is controlled to control the transfer switch 305, and the reset is executed. The transfer control signal 301, the reset control signal 302, and the row selection control signal 303 are connected to the vertical scanning circuit 202, and each row has a control signal. The transfer control signal 301, the reset control signal 302, and the row selection control signal 303 are output by the CPU 50 controlling the TGs 1800 and 1801.

  3 and 4 show simplified configuration diagrams of the image sensor of the image sensor 1401, the configuration of the image sensor 1402 is basically the same. A difference part is demonstrated in Example 1. FIG. FIG. 5 is a diagram showing, in chronological order, resetting and reading of each row when moving image exposure and reading operations are performed by the image sensor 1401 in the imaging apparatus 100. At the timing of t403, the CPU 50 selects the first row by the row selection circuit 206 in the vertical scanning circuit 202 via the TG 1801, and resets each pixel 203 in the selected row by the reset operation. After the reset, the operation of shifting the selected row to the next row and resetting the row is repeated.

  The dotted line 400 indicates the reset execution timing for each row. The process waits from the reset first row selection timing t403 to the timing t404 which is the accumulation time designated by the shutter second setting switch 65. At t404, the CPU 50 selects the first row for reading by the row selection circuit 206 in the vertical scanning circuit 202 again via the TG 1801. The charge accumulated in the photodiode 304 by the readout operation is read out as an electrical signal from the selected corner pixel 203 in the first row. After the reading, the operation of shifting the selected row to the next row and reading the row is repeated. A solid line 402 indicates the read execution timing for each row. The accumulation timing of each row is 401.

  There is one selected row circuit 206 in the vertical scanning circuit 202, which has a shift register configuration. When the reading inside the image sensor 1401 is set, the odd-numbered first row selection is driven as the resetting row selection, and the even-numbered first row selection is driven as the reading row selection. Each selected row is sent at the same timing by a line feed signal output by TG 1801. In the above description, the number of selected row circuits in the vertical scanning circuit 202 is one, but each of them may be scanned for resetting and reading.

[Example 1]
In the first embodiment, a method for detecting a person's face in an image obtained by reading the entire screen and designating an enlarged area in an imaging apparatus having two imaging elements having different resolutions will be described. In the first embodiment, a method for starting the accumulation of enlargement at the timing when the designation of the enlargement area is determined will be described.

  FIG. 6 shows an outline of the number of pixels of the image sensors 1400 and 1401. The pixel size 600 of the image sensor 1400 and the size 601 of the image sensor 1401 are different and 600 is smaller. In the same image sensor size, each pixel has a different pixel size, and each image sensor has a different number of pixels. The image sensor 1400 has the number of pixels for still images, and the image sensor 1401 has the number of pixels necessary for the entire moving image screen. For example, in order to express horizontal 1920 pixels and vertical 1080 pixels, which are the display pixels of FULL HD, as the number of pixels necessary for the moving image, the imaging device 1401 has effective pixels of horizontal 1920 pixels and vertical 1300 pixels. The image sensor 1400 has, for example, effective pixel horizontal 5850 rows and vertical 3900 pixels for a still image. As described above, in a future embodiment, a case will be described in which the three pixel sizes of the image sensor 1400 are aligned with the pixel size of the image sensor 1401 in the V direction.

  FIG. 7 is a simplified configuration diagram of an image sensor 1400 for still image and moving image enlargement reading.

  6 is different from the simplified configuration diagram of the image sensor 1401 in FIG. 3 in a vertical skip circuit 602 connected to the vertical scanning circuit 202 and a horizontal skip circuit 601 connected to the horizontal scanning circuit 201. The vertical skip circuit 602 can designate the start and end rows of the selection circuit 206, and the vertical scanning circuit 202 reads only the number of rows designated by the vertical skip circuit 602. The horizontal skip circuit 601 can also specify the start column and the end column, and only the data from the start column to the end column among the signals read out from one row is transferred.

  In FIG. 7, the vertical skip circuit 602 specifies the start row 603 and the end row 604, and the horizontal skip circuit 601 specifies the start column 605 and the end column 606. Only the area 607 of the total number of pixels of the image sensor 1401 can be enlarged and read out.

  In this embodiment, a method is described in which enlarged reading is read out in the horizontal 1920 and vertical 1080 pixels according to the moving image FULL HD. For example, in one image sensor having the same number of pixels for moving image FULL HD, one of them is read by FULL HD pixels. However, the technique of the present invention may be realized by reading the other with a different number of pixels such as another number of pixels (for example, horizontal 1366, vertical 768).

  Next, an enlarged area designation method will be described. In this embodiment, one frame is read from the image sensor 1401 to the full-screen moving image, and a portion to be read of the read image is analyzed to designate an enlarged region of the image sensor 1401. In this embodiment, a person's face will be described as one of the parts to be read. When the start / stop switch 63 for full-screen / enlargement / simultaneous recording is pressed, the CPU 50 drives the TG 1801 to read the full-screen image from 1400, digitized by the A / D converter 1601, and temporarily stored in the RAM 71. To do. The stored read image is analyzed by the enlarged reading unit specifying circuit 75.

  The part to be identified this time is a human face. The face detection method is not described in detail, but there is a method for detecting whether a human face part (eyebrow, eye, nose, mouth, ear, etc.) is within a predetermined distance in the image, for example. . For example, in the image of FIG. 8 (a), a person is shown, the face part is detected, the face part comes to the center, and further can be read out with FULL HD as shown in FIG. The enlarged area 700 is specified. The CPU 50 performs image processing on the full-screen frame temporarily stored in the RAM 71 by the moving image processing unit 73 and stores it in the memory card 200.

  The CPU 50 sends an enlarged specific area, which is an analysis result performed by the enlarged reading unit specifying circuit 75, to the image sensor 1400, and sets the start / end rows of the vertical skip circuit 602 and the start / end columns of the horizontal skip circuit 601. The CPU 50 drives the image sensor 1400 via the TG 1800 to read the specified portion. The read data is digitized by the A / D converter 1600 and temporarily stored in the RAM 71. The enlarged data becomes an image as shown in FIG. The CPU 50 processes the enlarged image frame temporarily stored in the RAM 71 by the enlarged moving image processing unit 74 and stores it in the memory card 200.

  In this example, both images are written to the same memory card, but they may be stored in different memory cards. Both the full-screen moving image and the enlarged moving image are stored as separate files. However, each image information indicates that the enlarged moving image is recorded at the same time and analyzes the full-screen moving image and the full-screen moving image to specify the enlargement area. Thus, playback can be performed in synchronization with playback.

  FIG. 9 shows how the enlarged area changes when the face image moves from the center of the screen to the right side.

  As a result of the analysis by the enlarged reading unit specifying circuit 75, the enlarged area is moved from 800 to 801, and the CPU 50 sends the result to the image sensor 1400, the start / end rows of the vertical skip circuit 602, and the start / end of the horizontal skip circuit 601. Set the column. Based on the determined area, an image is read out in the same manner as the enlarged image reading procedure.

  FIG. 1 shows the relationship between the image accumulation timings of the image sensor 1400 and the image sensor 1401 and the timing of an enlarged region specifying signal indicating the timing at which the enlarged reading unit specifying circuit 75 detects the face image and specifies the enlarged region.

  At t900, the CPU 50 starts resetting the moving image full screen of the image sensor 1401 via the TG 1801. After that, the reset timing of the image sensor 1401 is sent sequentially by the CPU 50 via the TG 1801 to the rows selected by the selected row 206, and the reset timing is as indicated by a dotted line 910.

  Next, at t <b> 901, the CPU 50 starts reading the moving image full screen of the image sensor 1401 via the TG 1801 based on the shutter time switch 65. Thereafter, the CPU 50 sequentially sends the number of rows selected by the selected row 206 via the TG 1801, and the reading is like a solid line 912. The accumulation timing of the moving picture full screen is as indicated by a portion 911.

  At t902, the reading of the full screen frame 912 is completed, the enlarged reading part specifying circuit 75 analyzes the moving picture full screen, and the enlarged area specifying signal becomes HIGH at timing t903 for specifying the enlarged area. When the specific area is known at t903, the CPU 50 immediately designates the enlarged area of the image sensor 1400, and the enlarged image reset timing is started in accordance with the designated area. Here, the vertical start line is v930 and the end line is v932.

  Thereafter, the reset timing of the image sensor 1400 is sent by the CPU 50 via the TG 1800, and the rows in which the selection row 206 sequentially selects the range v930 to v932 designated by the vertical skip circuit 602 is sent, and the reset timing is as indicated by the dotted line 921.

  At t904, in order to start accumulation of the next frame of the moving image full screen, the CPU 50 starts resetting the moving image full screen of the image sensor 1401 via the TG 1801. At t905, the readout of moving image enlargement is started based on the shutter time switch 65. Thereafter, the CPU 50 sends the rows in which the selection row 206 sequentially selects the range v930 to v932 specified by the vertical skip circuit 602 via the TG 1800, and the reading timing is as indicated by a solid line 923. The accumulation timing of moving image enlargement is as shown by the portion 921. Although not shown in the figure, skipping is also executed during horizontal transfer during reading.

  At t906, reading of the moving image full screen is started based on the shutter time switch 65 for the next frame of the moving image full screen. Thereafter, the CPU 50 sequentially sends the rows selected by the selected row 206 via the TG 1801, and the readout is as indicated by a solid line 915. The timing for accumulating the next frame of the entire moving image is as shown by the portion 914. Reading of the full screen frame 915 is completed at t907, the enlarged reading part specifying circuit 75 analyzes the moving picture full screen, and the enlarged area specifying signal becomes HIGH at timing t908 for specifying the enlarged area.

  When the specific area is known at t908, the CPU 50 immediately designates the enlargement area of the image sensor 1400, and the enlarged image reset timing is started in accordance with the designated area. Here, a different part from the previous frame is read such that the vertical start line is v931 and the end line is v933.

  It is desirable to read out the entire screen at a predetermined time interval (for example, after 16.6 msec in the case of 60 fps) during the interval t901 to t905 according to the frame rate. Further, it is preferable that the enlarged reading period 923 and the next frame (not shown) are also set to the above-mentioned time interval, but since it depends to some extent on the timing at which the enlarged area specifying signal is output, the frame rate at the time of recording However, there is no problem if the reading is in time at 16.6 msec.

  If the enlargement area specifying signal is not in time for timing t907 when the next frame reading 915 is finished after the full screen reading 912 end timing t902, the same position as the previous frame may be read out.

  FIG. 10 shows a flowchart of the first embodiment.

  When the simultaneous recording switch for moving image full screen / enlargement is pressed and a simultaneous recording request is generated, the CPU 50 drives the image sensor 1401 via the TG 1801 to read the image for moving image full screen (s1000). Face detection is executed based on the full screen image (s1001). When the face is detected, the CPU 50 calculates the moving image enlargement read position so that the face position is in the center (s1002). The CPU 50 drives the image sensor 1400 via the TG 1800 based on the calculation result, and reads an enlarged image.

  In parallel, the CPU 50 drives the image sensor 1401 via the TG 1801 to read the video for the moving image full screen (s1003). When face detection cannot be performed, the position determined in the initial stage or the same position as the area when the face can be detected in the previous frame is set as a moving image enlargement readout position. The CPU 50 drives the image sensor 1400 via the TG 1800 to read the enlarged position. In parallel, the CPU 50 drives the image sensor 1401 via the TG 1801 to read the video for the moving image full screen (s1004). After each video is read out, it is determined whether or not to continue recording (s1005). When the video is continued, face detection is performed based on the entire moving image screen read in s1003 or s1004. If the recording is not continued, the simultaneous recording ends.

  In the explanation so far, simultaneous recording of full screen and enlargement has been explained on the premise of recording moving images for almost the same time, but it can be changed in the setting so that only the part where the face is detected for a certain time or more is recorded. May be.

  In this way, the full-screen moving image is read from the moving image pickup device, and face detection is performed based on the moving image. When the face portion is specified, an enlargement readout region is determined so that the portion is the center, and the portion is read out from the still image sensor. As a result, it is possible to read and record a moving image enlarged image of a face portion desired by the user simultaneously with the entire moving image screen even for a subject with a large movement.

[Example 2]
In the first embodiment, an example in which there is a portion where the moving image full screen read from the image sensor 1401 overlaps the moving image expansion exposure period is shown. However, for example, as shown in FIG. 11, when the accumulation periods 1111, 1114 of the moving picture full screen and the accumulation period 1121 of the moving picture enlargement are short and the moving picture frame rate is slow (the frame interval is long), The accumulation timing may not overlap.

  Specifically, the reset operation is started from timing t1100 when the face image is detected from the reading frame 1112 of the moving image full screen. The moving image enlargement accumulation period is from that point to the moving image enlargement reading operation end timing t1101 of the image sensor 1400. The reading of the next frame of the moving image full screen reading 1112 and the accumulation start of the reading 1115 start from t1102. From that point, the moving image full-screen accumulation period starts until the moving image enlargement reading operation end timing t1103 of the image sensor 1400, and the accumulation periods do not overlap at all.

  In the second embodiment, an example in which the accumulation time is changed so that the accumulation time of the next frame of the face detection frame of the movie full screen and the movie enlargement reading frame reflecting the face detection result of the movie full screen is the same timing. Show. FIG. 12 shows the relationship between the image accumulation timings of the image sensor 1400 and the image sensor 1401 according to the second embodiment and the timing of the enlarged region specifying signal indicating the timing when the enlarged reading unit specifying circuit 75 detects the face image and specifies the enlarged region. Show.

  At t1200, the CPU 50 starts resetting the moving image full screen of the image sensor 1401 via the TG 1801. After that, the reset timing of the image sensor 1401 is sent sequentially by the CPU 50 via the TG 1801 through the rows selected by the selected row 206, and the reset timing is as indicated by a dotted line 1220.

  The CPU 50 starts reading the image sensor 1401 via the TG 1801 after an accumulation period such that the exposure becomes appropriate from t1200 at t1201. In the first embodiment, the accumulation time is set to a fixed time. In the second embodiment, an example in which the accumulation time can be automatically moved for exposure control will be described.

  Reading of the full screen frame 1221 is completed at t1202, and the enlarged reading part specifying circuit 75 analyzes the moving picture full screen, and the enlarged area specifying signal becomes HIGH at timing t1203 for specifying the enlarged area. Here, the vertical start line is v1240 and the end line is v1241.

  Each sensor is moved so that the accumulation timing of the portion corresponding to the center v1242 in the V direction of the enlarged region overlaps after the time t1203 until the start of full screen reading t1204. v1242 is the upper part of the screen on the full screen. For this reason, it takes more time to enlarge the moving image from the reading time from v1240 to v1241 of moving image enlargement and from the start row of the moving image full screen to the row v1250 of the moving image full screen corresponding to the moving image enlargement position v1242. Therefore, it starts from the reset of the video enlargement.

  At t1203, the CPU 50 sends the rows in which the selection row 206 sequentially selects the range v1240 to v1241 designated by the vertical skip circuit 602 via the TG 1800, and the reset timing becomes like a dotted line 1230. Thereafter, the resetting of the moving image full screen is started so that the reset timing of v1242 for moving image enlargement and the reset timing of the row v1250 corresponding to the moving image full screen at the moving image enlargement position v1242 become the same.

  At t1204, the CPU 50 starts resetting the entire moving image screen of the image sensor 1401 via the TG1801. Thereafter, the reset timing of the image sensor 1401 is sent by the CPU 50 sequentially through the TG 1801 through the rows selected by the selected row 206, and the reset timing is as indicated by a dotted line 1223. At t1205, the reset timing of the same vertical portion on the screen, v1250 for the entire moving image, and v1242 for the moving image enlargement are the same.

  For example, it is assumed that the start line v1240 is the 121st line, the end is the 1200th line, and the center part v1242 is the 660th line in the moving image enlargement. If the previous screen has three times the line width, the 660th line of the moving image corner is the 220th line corresponding to the full-screen moving image v1250n, and can be read at that speed. For this reason, the reset start timing t1204 for reading the entire screen is started after the end of 440 lines after the start of the moving image enlargement reset timing t1203. In this example, the horizontal scanning period is the same.

  At t1206, the CPU 50 starts reading the enlarged area of the image sensor 1400 after a predetermined time from the reading start timing t1203. Start by reading out the enlarged movie. The CPU 50 sends a row for sequentially selecting the range v1240 to v1241 designated by the vertical skip circuit 602 via the selection row 206 via the TG 1800, and the read timing becomes like a solid line of 1232.

  The accumulation period is the same as the full screen read start timing t1207 from the reset start timing t1204 of the moving image full screen. It is assumed that the full screen read start timing t1207 starts at the timing matching the frame rate from the read start t1201 of the previous frame. In this embodiment, the full screen read cycle period from t1201 to t1207 is set to about 33.3 msec with 30 fps.

  t1207 is a timing obtained by adding a period from t1203 to t1206, which is an accumulation period of moving image enlargement, from the reset start timing t1204 of the moving image full screen, and after 33.3 msec from t1205. The CPU 50 sequentially sends the rows selected by the selection row 206 via the TG 1801, and the readout timing of the moving image full screen is as indicated by a solid line 1225.

  Just as the reset start timing of the moving picture full screen and the moving picture enlargement matches at t1205, the reading timing of the moving picture full screen and the moving picture enlargement matches at t1208. The moving image enlargement v1242 and the row v1250 at the same position on the entire moving image have the same accumulation timing. At time t1209, when the reading of the full time movie screen is completed, the operation moves to face detection again.

  In the above example, the accumulation period of the entire moving image is greatly different between the accumulation period 1221 of the first frame and the accumulation period 1224 of the next frame. However, when an appropriate exposure is obtained in the accumulation period 1221, it is necessary to match the exposure in the accumulation period 1224. In the second embodiment, the gain is decreased by the gain portion 204 of the image sensor 1401 or the moving image processing 73 of the CPU 50. For example, it is assumed that the period from t1200 to t1201 in the frame accumulation period 1221 is 5 msec, and the period from the accumulation period t1204 to t1207 in the next frame 1224 is 20 msec. Since the accumulation period has quadrupled, the gain portion 204 of the image sensor 1401 is set to ¼, or the gain is set to ¼ by the moving image processing 73 of the CPU 50. In this way, control is performed so that proper exposure is not changed.

  FIG. 13 shows a flowchart of the second embodiment.

  When the simultaneous recording switch for moving image full screen / enlargement is pressed and a simultaneous recording request is generated, the CPU 50 drives the image sensor 1401 via the TG 1801 to read the image for moving image full screen (s1300). Face detection is executed based on the full screen image (s1301). When the face is detected, the CPU 50 calculates the moving image enlargement readout position so that the face position is in the center (s1302). Thereafter, it is confirmed whether or not the accumulation period is a period that can be controlled so as to be the same for the entire moving image and the moving image enlargement (s1303). Thereafter, the CPU 50 enlarges the moving image and reads out the entire moving image from the imaging devices 1400 and 1401 via the TGs 1800 and 1801 (s1306).

  If it is longer than the predetermined accumulation time in s1302, the CPU 50 executes the enlargement of the moving image and the reading of the entire moving image from the image sensors 1400 and 1401 via the TGs 1800 and 1801 while keeping the accumulation time as it is (s1306). If the face cannot be detected in s1301, the position determined in the initial stage or the same position as the area when the face can be detected in the previous frame is set as the moving image enlargement readout position. The CPU 50 drives the image sensor 1400 via the TG 1800 to read the enlarged position. In parallel, the CPU 50 drives the image sensor 1401 via the TG 1801 to read the video for the moving image full screen (s1305). After each video is read, it is determined whether or not to continue recording (s1307). If so, face detection is performed based on the entire moving image screen read in s1304 or s1305. If the recording is not continued, the simultaneous recording ends.

  In the second embodiment, the case where the accumulation timings coincide with each other has been described. However, for example, the accumulation periods may be driven so as to overlap each other for a predetermined period. For example, it is only necessary to overlap for 10 msec or more, and the accumulation period of 20 msec in the second embodiment can be expanded, and the entire period of the moving image can be set to 15 msec and the overlapping period can be set to 10 msec.

    As in the second embodiment, if the accumulation timing at the center position of the moving image enlargement position determined by face detection is controlled to be the same as the accumulation timing of the entire moving image screen, the timing is shifted when simultaneously reproduced. It is possible to obtain a reproduced image with less influence and no sense of incongruity.

[Example 3]
In the third embodiment, as shown in FIG. 11, the moving image enlargement detection timing is changed at t1100 so that the face detection matches the accumulation timing of the whole moving image. In the third embodiment, the moving image enlargement is performed without changing the accumulation period. An example is shown in which the storage timing of the video is delayed so that the video expansion and the video full-screen storage timing match.

  At t <b> 1400, the CPU 50 starts resetting the moving image full screen of the image sensor 1401 via the TG 1801. After that, the reset timing of the image sensor 1401 is such that the CPU 50 sequentially sends the rows selected by the selected row 206 via the TG 1801 and the reset timing is 1220.

  The CPU 50 starts reading the image sensor 1401 via the TG 1801 after an accumulation period in which the exposure becomes appropriate from t1400 at t1401. In the third embodiment, an example in which it can be automatically moved for exposure control as in the second embodiment will be described. Reading of the full screen frame 1411 is completed at t1402, and the enlarged reading part specifying circuit 75 analyzes the moving picture full screen, and the enlarged area specifying signal becomes HIGH at timing t1403 for specifying the enlarged area.

  It is assumed that the moving image enlargement read start position is v1430, the end position is v1432, and the center position is v1432. In the third embodiment, it is assumed that v1432 matches the central portion v1440 of the moving image full screen. Therefore, at t1403, the reset timing for moving image enlargement is also matched with the reset start timing t1404 for the moving image full screen.

  At t1404, the CPU 50 starts resetting the moving image full screen of the image sensor 1400 via the TG 1801 and resetting the moving image full screen of the image sensor 1401. Thereafter, the CPU 50 sends the rows selected by the sequential selection row 206 of the reset image sensor 1401 via the TG 1801, and the reset timing of the moving image full screen is as indicated by a dotted line 1413. Similarly, the CPU 50 sequentially sends a row in which the selection row 206 selects the range v1430 to v1431 specified by the vertical skip circuit 602 of the image sensor 1400 via the TG 1800, and the reset timing for moving image enlargement is as indicated by the dotted line 1420. .

  Thereafter, readout of the moving image full screen and moving image enlargement is started in the same accumulation period as that of the moving image full screen frame 1411. At t <b> 1405, the CPU 50 starts reading the full screen of the moving image of the image sensor 1400 via the TG 1801. The CPU 50 sends the rows selected by the sequential selection row 206 of the reset image sensor 1401 via the TG 1801, and the moving image full screen readout timing is as indicated by a solid line 1415. Similarly, the CPU 50 sequentially sends a row in which the selection row 206 selects the range v1430 to v1431 designated by the vertical skip circuit 602 of the image sensor 1400 via the TG 1800, and the reset timing for moving image enlargement is as indicated by the solid line 1422 .

  FIG. 15 shows a flowchart of the third embodiment.

  When the simultaneous recording switch for moving image full screen / enlargement is pressed and a simultaneous recording request is generated, the CPU 50 drives the image sensor 1401 via the TG 1801 to read the image for moving image full screen (s1500). Face detection is executed based on the full screen image (s1501). When the face is detected, the CPU 50 calculates the moving image enlargement readout position so that the face position is in the center (s1502). The CPU 50 drives the image sensor 1400 via the TG 1800 based on the calculation result, and reads an enlarged image. The CPU 50 drives the image sensor 1401 via the TG 1801 so that the accumulation timing of the central row of the moving image and the row corresponding to the moving image full screen coincide with each other, and reads the video for the moving image full screen (s1503).

  When face detection cannot be performed, the position determined in the initial stage or the same position as the area when the face can be detected in the previous frame is set as a moving image enlargement readout position. The CPU 50 drives the image sensor 1400 via the TG 1800 to read the enlarged position. The CPU 50 drives the image sensor 1401 via the TG 1801 so that the accumulation timing of the central row of the moving image and the row corresponding to the moving image full screen coincide with each other, and reads the video for the moving image full screen (s1504). After each video is read out, it is determined whether or not to continue recording (s1505), and when it is continued, face detection is performed based on the entire moving picture read out in s1503 or s1504. If the recording is not continued, the simultaneous recording ends.

  In the third embodiment, the moving image full screen and the moving image enlargement are recorded without changing the accumulation time and the gain, and when they are reproduced at the same time, the influence of the timing shift is suppressed, and a reproduced image with no sense of incongruity is obtained. For example, this is particularly effective when the face detection part does not move for a while (when the enlarged area does not move much).

  In the first to third embodiments, an example of the accumulation timing of moving image enlargement for the entire moving image screen is shown. However, if the timing of the reset start and readout start, which is video enlargement accumulation, falls within the period from immediately after determining the previous video enlargement region to the end of readout of the next frame of the full-screen frame, the above embodiment Other read operations are possible.

[Example 4]
In the fourth embodiment, an example in which an enlarged area is determined after selection of an automatic AF point will be described. For example, when the imaging surface AF can be performed by the full-screen imaging device 1400 during shooting of a moving image as shown in FIG. 16A, the object of the moon 1602 is recognized and the area 1600 is set as the AF point. With the AF area as the center, the enlarged area is 1601, and a moving image enlarged image can be read from the image sensor 1401.

  As in the fourth embodiment, by determining the moving image enlargement area based on various information read from the moving image full-screen sensor, it is possible to provide an imaging device that meets various needs of the user. By driving as in the first to third embodiments, it is possible to provide a user with a function of comfortable moving picture full screen and enlarged simultaneous recording.

100 imaging device, 1400 first imaging element, 1401 second imaging element,
13 Transmission mirror, 1800 TG for first image sensor,
1801 TG for second image sensor, 75 Enlarged reading part specifying circuit

Claims (11)

A first imaging element in which pixels including a photoelectric conversion element are two-dimensionally arranged;
A second imaging element in which pixels including a photoelectric conversion element are two-dimensionally arranged;
A light splitting means arranged so that an image is incident on a common imaging optical system to each of the first and second imaging elements;
Read area specifying means for specifying the second read area based on the image read by the second image sensor, having charge accumulation control means for controlling charge accumulation in the pixel portions of the first and second image sensors. In an imaging apparatus having
The accumulation of the first sensor is started after the specific area of the first image sensor is determined, and before the reading of the next frame of the frame of the second sensor, which is the base determined by the partial area specifying means, An image pickup apparatus which starts reading image signals from the sensors. The imaging apparatus according to claim 1, wherein the first imaging element has a higher resolution than the second imaging element. The imaging apparatus according to claim 1, wherein the start of accumulation of the first image sensor, the start of reading, and the accumulation start of the second image sensor are determined according to imaging conditions. The imaging apparatus according to claim 3, wherein the imaging condition is an accumulation period of the first and second imaging elements. The imaging apparatus according to claim 4, wherein accumulation of the first sensor is started immediately after determination of a reading area of the first imaging element. The first sensor starts accumulation, readout starts, and first so that the accumulation period corresponding to the readout area of the first sensor in the next frame of the second imaging element overlaps the accumulation period of the first imaging element by a predetermined value or more. The imaging apparatus according to claim 5, wherein the start of accumulation of the two imaging elements is controlled. The imaging apparatus according to claim 6, wherein the predetermined value is such that an accumulation period of the first imaging element and an accumulation period of the second imaging element in a predetermined row of the partial region specifying unit coincide with each other. The first sensor starts accumulation, readout starts, and first so that the accumulation period corresponding to the readout area of the first sensor in the next frame of the second imaging element overlaps the accumulation period of the first imaging element by a predetermined value or more. The image pickup apparatus according to claim 4, wherein the start of accumulation of the two image pickup elements is controlled. 8. The image pickup apparatus according to claim 7, wherein the method of the region specifying means specifies the feature amount of the image read by the second image sensor. The imaging apparatus according to claim 9, wherein the feature amount is a facial feature. 2. The imaging apparatus according to claim 1, wherein the method of the area specifying means is an automatic AF area point.