JP6521731B2 - Imaging device, control method therefor, and control program - Google Patents

Description

  The present invention relates to an imaging device, a control method thereof and a control program, and more particularly to an imaging device capable of capturing an object of interest in an image as a moving image with high resolution.

  In recent years, in an imaging apparatus such as a digital camera or a digital video camera, the number of pixels is reduced by performing a process of reading out a part of all pixels of an imaging element and a process of adding and reading pixel outputs. The processing as described above is generally performed when shooting a moving image. Further, the frame rate can be improved and the power consumption can be suppressed by reducing the readout time of the pixel output and the time required for the readout.

  In the imaging apparatus that performs the above-described read processing, when capturing a still image, capturing is performed using all pixels to obtain a high-resolution still image. Then, when shooting a moving image, shooting is performed with an appropriate number of pixels in consideration of the balance between the frame rate and the resolution.

  By the way, when shooting a moving image, there is a demand for continuing shooting without moving away from the angle of view of a subject of interest such as a moving person. In particular, there is a demand as described above in the case of zooming in and capturing a target object that moves quickly and is difficult to predict as a child.

  In order to meet such a demand, for example, by focusing at a wide angle of view of the imaging device, the subject of interest can be contained within the angle of view. However, when the angle of view is made wide, the subject of interest inevitably becomes smaller, and for example, it becomes difficult to record a situation that the user wants to catch, such as a child's expression.

  Therefore, a subject of interest may be enlarged and recorded as a zoom-up image among images obtained as a result of photographing at a wide angle. However, when the enlargement process is performed, it is difficult to obtain sufficient resolution. Furthermore, if the number of pixels to be read out is increased in order to secure a sufficient resolution, the balance between the frame rate and the resolution may be lost, which may be an obstacle to the frame rate and the power consumption.

  Thus, it is extremely difficult to achieve both resolution and frame rate balance and reduction of power consumption.

  On the other hand, in order to support shooting of a moving subject, two imaging units are provided, and one imaging unit captures an object while the other imaging unit captures an object at a wider angle than one imaging unit. There is an imaging device (see Patent Document 1). In this imaging device, the image obtained by the other imaging unit is displayed, and the image obtained by one imaging unit is recorded. At this time, the image obtained by the other imaging unit is displayed in such a manner that the imaging range of one imaging unit can be recognized.

JP, 2010-273046, A

  As described above, in the imaging device described in Patent Document 1, a subject (focused subject) in which the user moves in order to display an image obtained by the other imaging unit in a mode in which the imaging range of one imaging unit can be recognized. It is easy to take a picture so as not to deviate from the angle of view. However, in the imaging device described in Patent Document 1, when the size of the subject of interest changes, the user needs to perform a zoom operation or the like. Therefore, the burden on the user can not be reduced when the size of the subject of interest changes.

  Furthermore, in the imaging device described in Patent Document 1, it is necessary to control two imaging units independently, and the processing load of a control unit such as a CPU increases.

  Therefore, it is an object of the present invention to provide an imaging device capable of easily obtaining an image of a user's desired angle of view with a sufficient resolution by controlling the number of readout pixels appropriately and reducing power consumption, and control method thereof To provide the program.

To achieve the above object, an imaging apparatus according to the present invention is an imaging apparatus having an imaging means comprising an imaging device for outputting the images by imaging an object, in the imaging device, the first column signal line Reading out a signal from a first pixel group connected to the first pixel group as a first readout operation to obtain a first image, and providing a first image physically independent of the first column signal line The signal readout from the second pixel group different from the first pixel group is performed, which is connected to the two column signal lines and at least a part of the pixel group is included in the imaging angle of view of the first pixel group determining means for determining an area as the attention object region including reading means for obtaining a second image performed in parallel with the first read operation as second read operation, a target object using a pre-Symbol first image When, in accordance with prior Kichu th subject area By controlling the serial reading means, wherein the second image obtained by the second reading operation possess a control means for an image including the target object, the second said from the image of the first image It is characterized in that the number of pixels of the second image is larger than that of the wide angle and the number of pixels of the subject region of interest in the first image .

According to the present invention, it is possible to obtain an image of easy user desired angle of view by reducing the power consumption while appropriately controlling the number of pixels out read with sufficient resolution.

It is a block diagram showing the composition about an example of the imaging device by a 1st embodiment of the present invention. It is a figure for demonstrating an example of the thinning-out drive performed by the 1st imaging part shown in FIG. It is a figure for demonstrating an example of the thinning-out drive performed by the 2nd imaging part shown in FIG. It is a figure for demonstrating the other example of the thinning-out drive performed by the 2nd imaging part shown in FIG. FIG. 6 is a timing chart for explaining imaging timing in a moving image mode in the camera shown in FIG. 1. It is a flowchart for demonstrating the imaging | photography process performed with the camera shown in FIG. It is a figure which shows an example of the image pick-up element drive selection table used with the camera shown in FIG. It is a block diagram showing the composition about an example of the camera by a 2nd embodiment of the present invention. It is a perspective view which shows the structure of the image pick-up element shown in FIG. It is a block diagram for demonstrating an example of a structure of the image pick-up element shown in FIG. It is a figure for demonstrating an example of pixel selection at the time of driving the image pick-up element shown in FIG. 10 by 1st read-out operation. It is a figure for demonstrating an example of pixel selection at the time of driving the image pick-up element shown in FIG. 10 by 2nd read-out operation. FIG. 9 is a timing chart for explaining imaging timing in a moving image mode in the camera shown in FIG. 8; It is a flowchart for demonstrating the imaging | photography process performed with the camera shown in FIG. It is a figure which shows an example of the image pick-up element drive selection table used with the camera shown in FIG.

  Hereinafter, an example of an imaging device according to an embodiment of the present invention will be described with reference to the drawings.

First Embodiment
FIG. 1 is a block diagram showing a configuration of an example of an imaging device according to a first embodiment of the present invention.

  The illustrated imaging apparatus is, for example, an electronic still camera or a digital video camera (hereinafter simply referred to as a camera) with a moving image shooting function. The camera has a first imaging unit and a second imaging unit. The first imaging unit includes an imaging lens unit (hereinafter simply referred to as a lens) 110, a lens control unit 111, an imaging element 112, and an imaging element drive circuit 113. Similarly, the second imaging unit includes a lens 120, a lens control unit 121, an imaging element 122, and an imaging element drive circuit 123. The camera further includes a signal processing unit 130, a compression / decompression unit 140, a control unit 150, a light emitting unit 160, an operation unit 170, an image display unit 180, and an image recording unit 190.

  Although not shown, each of the lenses 110 and 120 includes a plurality of lenses and an optical mechanism. Then, the lenses 110 and 120 form an object image (optical image) on the imaging elements 112 and 122, respectively.

  The optical mechanism unit includes, for example, an AF mechanism, a zoom drive mechanism, a mechanical shutter mechanism, and an aperture mechanism. The lens control units 111 and 121 drive and control the optical mechanism units provided in the lens 110 and the lens 120 under the control of the control unit 150.

  In the following description, the lenses 110 and 120 are described as independent lenses, but the two lenses 110 and 120 may be identical if the effects of the present embodiment can be obtained.

  Each of the imaging elements 112 and 122 has a pixel portion and an A / D converter (not shown), and is, for example, a so-called XY readout type CMOS image sensor. Then, under the control of the control unit 150, the imaging element drive circuits 113 and 123 perform exposure processing, signal readout processing, reset processing and the like of the imaging elements 112 and 122. As a result, the imaging elements 112 and 122 respectively output first and second imaging signals (also referred to as first and second image signals).

  The signal processing unit 130 performs signal processing such as white balance adjustment processing, color correction processing, and AE (Auto Exposure) processing on the first and second image signals under the control of the control unit 150. Output 1 and 2nd image data. The signal processing unit 130 is provided with a subject of interest determination unit 1301. The object-of-interest discrimination unit 1301 includes a moving body detection unit 1302, a person detection unit 1303, and a face detection unit 1304.

  The moving subject detection unit 1302 performs correlation calculation between frames in each of the first and second image signals to detect a moving subject (moving subject), as described later. In the illustrated example, the moving object detection unit 1302 outputs an angle of view with an aspect ratio of 16: 9 including the detected moving object to the control unit 150 as a moving object detection region.

  The human detection unit 1303 performs pattern matching on each of the first and second image signals to detect a human. In the illustrated example, the human detection unit 1303 detects and outputs an angle of view having an aspect ratio of 16: 9 including the human to the control unit 150 as a human detection area.

  The face detection unit 1304 performs pattern matching on each of the first and second image signals to detect the face of a person. In the illustrated example, the face detection unit 1304 outputs an angle of view with an aspect ratio of 16: 9 including the detected face as a face detection area to the control unit 150.

  Under control of the control unit 150, the compression / decompression unit 140 applies predetermined still image data such as JPEG (Joint Photographic Coding Experts Group) method to the first and second image data output from the signal processing unit 130. The compression encoding process is performed according to the format to generate first and second encoded image data. Furthermore, the compression / decompression unit 140 performs decompression decoding processing on the first and second encoded image data sent from the control unit 150 to decode the image data (that is, first and second image data). I assume.

  Note that the compression / decompression unit 140 may perform compression encoding / decompression decoding processing on moving image data, which is the first and second image data, according to the MPEG (Moving Picture Experts Group) method or the like.

  The control unit 150 is a microcontroller including, for example, a central processing unit (CPU), a read only memory (ROM), and a random access memory (RAM). The CPU controls the entire camera by executing a program stored in the ROM.

  When the control unit 150 determines that the exposure value of the subject is lower than the predetermined exposure value by the AE processing by the signal processing unit 130, the control unit 150 controls the light emitting unit 160 to emit light and illuminate the subject. As the light emitting unit 160, for example, a strobe device or an LED light emitting device using a xenon tube is used.

  The operation unit 170 includes, for example, various operation keys such as a shutter release button, a lever, a dial, and a touch panel. The operation unit 170 provides the control unit 150 with an operation signal according to the user's input operation.

  The image display unit 180 includes, for example, a display device such as a liquid crystal display (LCD) and an interface circuit of the display device. Then, the image display unit 180 displays an image corresponding to the image data sent from the control unit 150 on the display device.

  The image recording unit 190 is, for example, a recording medium such as a portable semiconductor memory, an optical disk, an HDD (Hard Disk Drive), or a magnetic tape. The image recording unit 190 stores, as an image file, encoded image data that has been compression-encoded by the compression / decompression unit 140. Further, the image recording unit 190 reads the image file specified by the control unit 150 and outputs the image file to the control unit 150. Then, the control unit 150 subjects the read encoded image data to decompression decoding processing by the compression / decompression unit 140 to obtain decoded image data.

  Here, the basic operation of the camera shown in FIG. 1 will be described.

  For example, when capturing a still image, CDS (Correlated Double Sampling) processing and AGC (analog CD) processing are sequentially performed on the analog signals output from the pixels in each of the imaging elements 112 and 122 before imaging (that is, before main imaging). Automatic Gain Control) processing is performed. Thereafter, in each of the imaging elements 112 and 122, an A / D converter A / D converts an analog signal and outputs an image signal (digital image signal). Then, this image signal is sent to the signal processing unit 130.

  In the signal processing unit 130, the attention subject determination unit 1301 performs a subject determination process on a first image signal which is an output of the first imaging unit. Then, the attention object determination unit 1301 outputs attention object region information indicating the attention object region (that is, the moving object detection region, the person detection region, and the face detection region) to the control unit 150.

  The control unit 150 controls the imaging element drive circuit 123 by determining the imaging conditions in the second imaging unit based on the target subject area information as described later. As a result, in the second imaging unit, the subject region of interest is cut out of the imaging angle of view of the imaging element 122 and imaging is performed.

  The signal processing unit 130 performs, for example, image quality correction processing on the first and second image signals to generate first and second camera through image signals, respectively. Then, the signal processing unit 130 sends the first and second camera through image signals to the control image display unit 180 under the control of the control unit 150. As a result, a camera through image (live view image) corresponding to the first and second camera through image signals is displayed on the image display unit 180. The user can perform angle-of-view alignment on the subject of interest while viewing these camera through images.

  In the above-described state, that is, in a state where the camera through image is displayed on the image display unit 180, it is assumed that the shutter release button provided on the operation unit 190 is pressed. Thus, the control unit 150 controls the imaging device drive circuit 113 and the imaging device drive circuit 123 to send the first and second image signals of one frame to the signal processing unit 130 from the imaging device 112 and the imaging device 122, respectively. take in. The signal processing unit 130 performs image quality correction processing on the first and second image signals for one frame, and sends the processed digital image signals (that is, the first and second image data) to the compression and decompression unit 140. .

  The compression and decompression unit 140 compresses and encodes the first and second image data. The compression and decompression unit 140 sends the first and second encoded image data to the image recording unit 190 under the control of the control unit 150. As a result, the first and second encoded image data are recorded in the image recording unit 190 as image files (first and second image files) related to a still image.

  When reproducing the image file recorded in the image recording unit 190, the control unit 150 reads from the image recording unit 190 the image file selected according to the operation input from the operation unit 170. Then, the control unit 150 sends the read image file to the compression / decompression unit 140. The compression / decompression unit 140 decompresses and decodes the image file to obtain decoded image data. The control unit 150 sends the decoded image data to the image display unit 180, and reproduces and displays a still image corresponding to the decoded image data on the image display unit 180.

  When recording moving image data, the first and second image signals output from the imaging elements 112 and 122 are taken into the signal processing unit 130 under the control of the control unit 150. The signal processing unit 130 sequentially processes the first and second image signals to generate first and second image data (that is, moving image data here). These moving image data are compressed and encoded by the compression / decompression unit 140, and the encoded moving image data is sequentially transferred to the image recording unit 190 and recorded as a moving image file.

  When reproducing the moving image file recorded in the image recording unit 190, the control unit 150 reads the selected moving image file from the image recording unit 190 in accordance with the operation input from the operation unit 170. Then, the control unit 150 sends the read moving image file to the compression / decompression unit 140. The compression / decompression unit 140 decompresses and decodes the moving image file to obtain decoded moving image data. The control unit 150 sends the decoded moving image data to the image display unit 180, and causes the image display unit 180 to reproduce and display a moving image according to the moving image data.

  FIG. 2 is a diagram for explaining an example of the thinning drive performed by the first imaging unit shown in FIG.

  In the following description, it is assumed that the number of pixels of the imaging elements 112 and 122 and the angle of view of the subject photographed by the first and second imaging units are the same. They may be different from one another.

  In FIG. 2, for convenience of explanation, each of the imaging elements 112 and 122 is assumed to have 32 rows × 18 columns of pixel units. In practice, the number of pixels of the imaging device (that is, the pixel portion) is 7680 rows × 4320 columns. Also, in general, the pixels are arranged in a Bayer arrangement in the pixel portion, but here, for convenience of explanation, the color filter is omitted.

  First, thinning readout in the first imaging unit will be described. In the first imaging unit, readout of the imaging element 112 is performed so as to acquire the entire angle of view of the imaging element 112. As shown in FIG. 2, in the first imaging unit, it is assumed that readout is performed by thinning out the pixels to 1/6 in each of the horizontal and vertical directions. By this readout, 1280 × 720 pixels are read out, and the first image data is recorded and displayed as moving image data of HD. This makes it possible to obtain an image relating to the entire angle of view.

  FIG. 3 is a diagram for explaining an example of the thinning drive performed by the second imaging unit shown in FIG.

  When thinning out reading is performed on the second imaging unit, in the second imaging unit, as described later, the attention area designated by the control unit 150 is cut out by the imaging device drive circuit 123 and readout is performed. . The attention object discrimination unit 1301 discriminates the attention area from the first image signal.

  As shown in FIG. 3, here, the area around the person is taken as the attention area. In the second imaging unit, the readout is performed by thinning out the pixels to 1/2 in each of the horizontal direction and the vertical direction in consideration of a thinning rate that can obtain an appropriate resolution according to the size of the attention area Do. By this reading, 1920 × 1080 pixels are read out, and the second image data is recorded and displayed as full HD moving image data. By this, it is possible to obtain an image of a region around a person with high resolution.

  FIG. 4 is a view for explaining another example of the thinning drive performed by the second imaging unit shown in FIG.

  In the example shown in FIG. 4, the region of interest is narrower than the example shown in FIG. The image related to the attention area has higher image quality as the resolution is higher. Therefore, in FIG. 4, the read-out is performed with the thinning rate lower than that in FIG.

  In the example shown in FIG. 4, the area around the face of the person is taken as the attention area. Then, in consideration of the thinning rate at which an appropriate resolution corresponding to the size of the attention is obtained, in the second imaging unit, reading is performed without performing thinning in both the horizontal and vertical directions. By this reading, 1920 × 1080 pixels are read out, and the second image data is recorded and displayed as full HD moving image data. By this, it is possible to obtain an image of a region around the face of a person with high resolution.

  As described above, in the first image signal read by the first imaging unit, the number of pixels in the region of interest does not reach the number of pixels of the second image signal read by the second imaging unit. . That is, in the first image signal read out by the first imaging unit, the second image signal read out by the second imaging unit is shown as compared to the image obtained by performing enlargement processing on the region of interest. Images are much higher quality.

  FIG. 5 is a timing chart for explaining imaging timing in the moving image mode in the camera shown in FIG.

  As illustrated, the imaging timing is defined by the vertical synchronization signal (VD) sent from the imaging device driving circuits 113 and 123 under the control of the control unit 150. In the illustrated example, for convenience of explanation, the frame rate in the first imaging unit is twice that of the second imaging unit.

  Here, the first image signal which is the output of the first imaging unit is used as live view display, and the imaging element 112 is constantly driven. It is assumed that a start command of moving image recording is issued by the operation unit 170 before the frame of the period T0 to T1. In the frame in the period T0 to T1, the first image signal obtained by the first imaging unit is displayed on the image display unit 180 as a live view image. At this time, the region of interest (region of interest) of the object of interest is discriminated in the first image signal by the object of interest discrimination unit 1301 and given to the control unit 150 as object of interest region information.

  The control unit 150 sends driving conditions (i.e., imaging conditions) such as cutout position information and a thinning rate used at the time of shooting by the second imaging unit to the imaging element driving circuit 123 according to the subject region information of interest.

  In the frame of the period T1 to T3, the second image pickup unit reads the second image signal based on the cutting position information and the driving condition such as the thinning rate given to the image pickup device driving circuit 123 at time T1. . Then, after the second image signal is processed by the signal processing unit 130, the second image signal is recorded as image data (moving image data) obtained by photographing the subject of interest at high resolution.

  Similarly, in the periods T1 to T2 and the periods T2 to T3, the first image signal which is the output of the first imaging unit is used as a live view image. At this time, as described above, the processing for discriminating the attention area is performed by the attention object discrimination unit 1301, and the control unit 150 sends the drive condition to the imaging element drive circuit 123.

  As described above, in the period T0 to T5, the same operation is performed for each frame until the moving image recording is stopped for both the first and second imaging units.

  In this manner, the second image signal (here, moving image) obtained by the second imaging unit by performing the reading process and the determination process of the attention area is a high resolution image limited to the attention subject area. The high resolution image is to be recorded.

  FIG. 6 is a flowchart for explaining the photographing processing performed by the camera shown in FIG. Note that the illustrated flowchart is performed under the control of the control unit 150.

  When the power of the camera is turned on and the camera is set to the photographing mode by the operation unit 170, the control unit 150 controls the imaging device driving circuit 113 to start live view imaging by the first imaging unit.

  Subsequently, the control unit 150 causes the first imaging unit to capture a live view image (first image signal) for one frame (Fr1) in the period T0 to T1 shown in FIG. 5 (step S401). In the process of step S401, as described with reference to FIGS. 2 to 5, the angle of view is set so as to capture the entire angle of view captured by the image sensor 112, and thinning is performed to maintain a sufficient frame rate. Processing is performed.

  The first image signal obtained in the process of step S401 is referred to as Fr1-0. The first image signal (live view image) read in the process of step S401 is subjected to image processing and screen display as a live view or a recorded moving image, but the detailed description is omitted here.

  Next, the control unit 150 checks whether or not a user setting presence / absence flag U = 0 indicating whether the subject area of interest is manually set by the user operation of the operation unit 170 (step S402). The user setting presence / absence flag U is a flag used when the user uniquely determines the subject area of interest by manual operation. For example, in the case where the operation unit 170 is a touch panel display, the user can specify a target subject area by touching the target area with a finger or the like.

  Here, the target subject area to be set is an area with an aspect ratio of 16: 9. Then, when the user manually sets the target subject area, the user setting flag U = 1. In addition, when the user has not set the subject region of interest manually, the user setting flag U = 0.

  If the user setting presence / absence flag U is 1 (NO in step S402), the control unit 150 saves the object of interest area set by the user as the area A in the built-in memory (step S403). Then, the control unit 150 proceeds to the process of step S414 described later.

  When user setting presence / absence flag U is 0 (YES in step S402), control unit 150 detects a moving object detection condition of the subject region of interest according to the operation by operation unit 170 or not. It is confirmed whether or not the setting presence / absence flag M = 0 (step S404).

  The moving object detection setting presence / absence flag M is a flag for setting a moving subject in an image as a region of interest. The moving object detection setting presence / absence flag M is used, for example, when it is difficult for the user to frame the moving object to an optimum size. When the user performs moving object detection setting, the moving object detection setting presence / absence flag M = 1. In addition, when the user does not perform moving object detection setting, the moving object detection setting presence / absence flag M = 0.

  If the moving object detection setting presence / absence flag M is 1 (NO in step S404), the control unit 150 inputs the first image signal Fr1 obtained by the first imaging unit to the moving object detection unit 1302. Then, under the control of the control unit 150, the moving body detection unit 1302 extracts a contour from the first image signal Fr1 to obtain contour information (current contour information). Then, the moving body detection unit 1302 performs a moving body detection process of detecting a moving body area by comparing the past contour information stored before the first image signal Fr1 is input with the current contour information (step S405). . Then, the moving body detection unit 1302 stores the current contour information in the built-in memory as past contour information. In addition, a moving body area | region is made into an area | region of aspect ratio 16: 9 including the periphery.

  Subsequently, the control unit 150 sets the moving object region detected by the moving object detection unit 1302 as a target object region. Then, the control unit 150 stores the subject area of interest as an area A in the built-in memory (step S406). Thereafter, the control unit 150 proceeds to the process of step S414 described later.

  If moving object detection setting presence / absence flag M is 0 (YES in step S404), control unit 150 causes the operation unit 170 to operate the operation unit 170 to indicate whether the detection condition of the target subject area is the person detection setting or not. It is confirmed whether or not the detection setting presence / absence flag P = 0 (step S407).

  The person detection setting presence / absence flag P is a flag for setting a person as an attention area in an image. The person detection setting presence / absence flag P is used, for example, when it is difficult for the user to frame a person to an optimum size. Then, when the user performs the person detection setting, the person detection setting presence / absence flag P = 1. Further, when the user does not perform the person detection setting, the person detection setting presence / absence flag P = 0.

  If the person detection setting presence / absence flag P is 1 (NO in step S407), the control unit 150 inputs the first image signal Fr1 obtained by the first imaging unit to the person detection unit 1303. Then, under the control of the control unit 150, the person detection unit 1303 performs person detection processing for detecting a person area in the image indicated by the first image signal Fr1 using pattern matching in comparison with the shape of the person (step S408). The person area is an area with an aspect ratio of 16: 9 including the periphery thereof.

  Subsequently, the control unit 150 sets the human area detected by the human detection unit 1303 as a target subject area. Then, the control unit 150 stores the subject area of interest as an area A in the built-in memory (step S409). Thereafter, the control unit 150 proceeds to the process of step S414 described later.

  If the person detection setting presence / absence flag P is 0 (YES in step S407), the control unit 150 causes the operation unit 170 to operate the operation unit 170 to indicate whether the detection condition of the target subject area is the face detection setting. It is confirmed whether or not the detection setting presence / absence flag F = 0 (step S410).

  The face detection setting presence / absence flag F is a flag for setting a face as an attention area in an image. The face detection setting presence / absence flag F is used, for example, when it is difficult for the user to frame the face of a person to an optimal size. Then, when the user performs face detection setting, the face detection setting presence / absence flag F = 1. Further, when the user does not perform the face detection setting, the face detection setting presence / absence flag F = 0.

  If the face detection setting presence / absence flag F is 1 (NO in step S410), the control unit 150 inputs the first image signal Fr1 obtained by the first imaging unit to the face detection unit 1304. Then, under the control of the control unit 150, the face detection unit 1304 performs a face detection process of detecting a face area by comparing the shape of the face of a person using pattern matching in the image indicated by the first image signal Fr1. (Step S411). The face area is an area with an aspect ratio of 16: 9 including the periphery thereof.

  Subsequently, the control unit 150 sets the face area detected by the face detection unit 1304 as a target subject area. Then, the control unit 150 stores the subject area of interest as an area A in the built-in memory (step S412). Thereafter, the control unit 150 proceeds to the process of step S414 described later.

  If the face detection setting presence / absence flag F is 0 (YES in step S410), the control unit 150 sets the entire image as a target subject region because the detection condition for the target subject region is not set. Then, the control unit 150 stores the subject area of interest as an area A in the built-in memory (step S413).

  Next, the control unit 150 calculates the ratio r of the area A (that is, the target subject area) to the entire image (step S414). Then, the control unit 150 selects the driving method of the imaging element 122 according to the imaging element drive selection table Tbl1 described later according to the ratio r (step S415).

  FIG. 7 is a diagram showing an example of an imaging element drive selection table used in the camera shown in FIG. The imaging element drive selection table is stored in the ROM provided in the control unit 150.

  The image pickup device drive selection table Tbl1 determines the thinning rate or the pixel addition number in the horizontal and vertical directions at the time of photographing by the second image pickup unit according to the ratio r of the object area A of interest to the image. In the illustrated example, when the ratio r is 56.25% or more, the thinning rates in the horizontal and vertical directions are set to 1⁄4. When the ratio r is 25% or more and less than 56.25%, the thinning rates in the horizontal and vertical directions are reduced to 1/3.

  Furthermore, when the ratio r is 6.25% or more and less than 25%, the horizontal and vertical thinning rates are set to 1/2 thinning. Then, when the ratio r is less than 6.25%, thinning in the horizontal and vertical directions is regarded as none.

  Here, although the imaging device drive selection table Tbl1 is a table that defines the thinning rate according to the ratio r, when reading out the object area of interest with the optimal number of pixels or the optimal frame rate, the imaging device drive selection is performed. The table Tbl1 is a table which defines the pixel addition number according to the ratio r.

  Referring back to FIG. 6, control unit 150 sends the image sensor driving method (that is, the thinning rate here) selected in the process of step S 415 to the image sensor driving circuit 123 and the position in the image of subject area A of interest. . That is, the control unit 150 instructs the second imaging unit to perform imaging conditions (step S416). At this time, when the ratio r is 56.25% or more (that is, when the subject region of interest is a predetermined size or more), the control unit 150 sets the angle of view in advance to the lens control unit 121. A drive instruction is issued to make the lens more wide-angle as much as possible. That is, the control unit 150 shifts the angle of view to the wide-angle side by a predetermined amount. This avoids in advance the risk that the subject of interest will deviate from the image in the subsequent frames.

  Next, under the control of the control unit 150, the second imaging unit performs imaging of one frame in the periods T1 to T3 illustrated in FIG. 5 (step S417). The second image signal obtained by this imaging is an image obtained by reading out the target object area at a high resolution.

  Here, the obtained second image signal is called Fr2-1. The second image signal is subjected to image processing and screen display as a moving image, but the detailed description is omitted here.

  Thereafter, the control unit 150 processes the next frame. That is, after performing the process of step S417, the control unit 150 returns to the process of step S401, performs imaging by the first imaging unit during the period T1 to T2 shown in FIG. One image signal) is obtained for one frame. The first image signal obtained here is called Fr1-1.

  Thereafter, imaging is performed by the first imaging unit also during the period T2 to T3 illustrated in FIG. 5 to obtain a live view image for one frame. The first image signal obtained here is called Fr1-2. The first image signals Fr1-1 and Fr1-2 are processed as described in steps S401 to S416. Then, the object-of-interest area information obtained in the first image signals Fr1-1 and Fr1-2 is reflected in the imaging by the second imaging unit in step S417.

  As described above, in the first embodiment of the present invention, the target subject area is determined in the first image signal obtained by the first imaging unit. Then, imaging by the second imaging unit is controlled in accordance with the target subject area. Thus, an image having a desired angle of view can be cut out at a sufficient resolution to perform imaging. Then, unnecessary pixel readout and power consumption can be suppressed.

  When cutting out the target subject area, only the target subject area may be cut out, and further, an area within a predetermined range including the target subject area may be cut out.

Second Embodiment
Subsequently, an example of a camera according to a second embodiment of the present invention will be described.

  FIG. 8 is a block diagram showing the configuration of an example of a camera according to the second embodiment of the present invention. In FIG. 8, the same components as in the camera shown in FIG.

  The illustrated camera includes an imaging unit, and the imaging unit includes a lens 610, a lens control unit 611, an imaging element 612, and an imaging element driving circuit 613. Although not shown, the lens 610 includes a plurality of lenses and an optical mechanism. Then, the lens 610 forms an object image (optical image) on the image sensor 612.

  The optical mechanism unit includes, for example, an AF mechanism, a zoom drive mechanism, a mechanical shutter mechanism, and an aperture mechanism. The lens control unit 611 controls driving of an optical mechanism provided in the lens 610 under the control of the control unit 650.

  The imaging element 612 has a pixel portion and an A / D converter (not shown), and is, for example, a so-called XY readout type CMOS image sensor. Then, under the control of the control unit 650, the imaging device drive circuit 613 performs exposure processing of the imaging device 612, signal readout processing, reset processing, and the like. As a result, the imaging element 612 outputs the first and second image signals as described later.

  The control unit 150 is a microcontroller including, for example, a CPU, a ROM, and a RAM. The CPU controls the entire camera by executing a program stored in the ROM. Further, when the control unit 650 determines that the exposure value of the subject is lower than the predetermined exposure value by the AE processing by the signal processing unit 130, the control unit 650 controls the light emitting unit 160 to emit light to illuminate the subject.

  Here, the basic operation of the camera shown in FIG. 8 will be described.

  For example, when capturing a still image, the imaging device 612 sequentially performs CDS processing and AGC processing on analog signals output from pixels before capturing (that is, before capturing a main image). Thereafter, in the imaging device 612, an A / D converter A / D converts an analog signal and outputs an image signal (digital image signal). Then, this image signal is sent to the signal processing unit 130.

  In the signal processing unit 130, the attention subject determination unit 1301 performs a subject determination process on the first image signal output from the imaging element 612 in the first readout operation described later. Then, the attention object determination unit 1301 outputs attention object region information indicating the attention object region (that is, the moving object detection region, the person detection region, and the face detection region) to the control unit 650.

  The control unit 650 controls the imaging element drive circuit 613 by determining the imaging conditions in the second readout operation based on the target subject area information as described later. As a result, in the second readout operation, pixel readout for cutting out the target object area out of the imaging angle of view of the imaging element 612 is performed, and is output as a second image signal.

  The signal processing unit 130 performs, for example, image quality correction processing on the first and second image signals obtained respectively in the first and second readout processing, and outputs the first and second camera through image signals. Generate Then, the signal processing unit 130 sends the first and second camera through image signals to the control image display unit 180 under the control of the control unit 650. As a result, a camera through image (live view image) corresponding to the first and second camera through image signals is displayed on the image display unit 180. The user can perform angle-of-view alignment on the subject of interest while viewing these camera through images.

  In the above-described state, that is, in a state where the camera through image is displayed on the image display unit 180, it is assumed that the shutter release button provided on the operation unit 190 is pressed. Thus, the control unit 650 controls the imaging device driving circuit 613 to process the first and second image signals of one frame obtained by the first and second readout operations from the imaging device 612 as the signal processing unit 130. Capture to The signal processing unit 130 performs image quality correction processing on the first and second image signals for one frame, and sends the processed digital image signals (that is, the first and second image data) to the compression and decompression unit 140. .

  When recording moving image data, the first and second image signals output from the imaging element 612 are taken into the signal processing unit 130 under the control of the control unit 650. The signal processing unit 130 sequentially processes the first and second image signals to generate first and second image data (that is, moving image data here). These moving image data are compressed and encoded by the compression / decompression unit 140, and the encoded moving image data is sequentially transferred to the image recording unit 190 and recorded as a moving image file.

  FIG. 9 is a perspective view showing the structure of the imaging device shown in FIG.

  The imaging element 612 includes a first chip (pixel portion) 70 and a second chip 71, and the first chip (first element) is formed on a second chip (also referred to as a second element portion) 71. Section 70 is stacked. The first chip 70 has a plurality of pixels arranged in a two-dimensional matrix, and the first chip 70 is disposed on the light incident side (that is, located on the light receiving side of the optical image) ). On the second chip 71, pixel drive circuits such as a column scan circuit and a row scan circuit, which will be described later, are formed.

  As described above, when pixels are formed in the first chip 70 and a pixel drive circuit is formed in the second chip 71, the manufacturing process of the peripheral circuit of the imaging element 612 and the pixel portion can be divided. As a result, speeding up of the reading operation can be achieved by thinning and densifying the wiring in the peripheral circuit, and downsizing and high functionality of the imaging device can be achieved.

  FIG. 10 is a block diagram for explaining an example of the configuration of the imaging device shown in FIG. In the illustrated example, two signal processing units 130 are shown for convenience of explanation.

  In the first chip 70, a plurality of pixels 701 are arranged in a two-dimensional matrix. The pixel 701 is connected to the transfer signal line 703, the reset signal line 704, and the row selection signal line 705 in the horizontal direction (row direction), and is connected to the column signal lines 702-a and 702-b in the vertical direction (column direction). ing. The connection destinations of the column signal lines 702-a and 702-b differ depending on the read row unit.

  As illustrated, each of the pixels 701 includes a photodiode PD which is a photoelectric conversion element, a transfer transistor M1, a reset transistor M2, an amplification transistor M3, a selection transistor M4, and a floating diffusion FD.

  In the illustrated example, each of the transistors is an n-channel MOSFET (MOS Field-Effect Transistor).

  The transfer signal line 703, the reset signal line 704, and the row selection signal line 705 are connected to the gates of the transfer transistor M1, the reset transistor M2, and the selection transistor M4, respectively. The signal lines 703 to 705 extend in the horizontal direction, and the pixels 701 in the same row are simultaneously driven. By this, it is possible to control the operation of the line sequential operation type rolling shutter or the all row simultaneous operation type global shutter. Furthermore, the column signal line 702-a or 702-b is connected to the source of the selection transistor M4 in units of rows.

  The photodiode PD accumulates the charge generated by photoelectric conversion. The P side of the photodiode PD is grounded, and the N side is connected to the source of the transfer transistor M1. When the transfer transistor M1 is turned on, the charge of the photodiode PD is transferred to the FD, and since the parasitic capacitance exists in the FD, the charge transferred to the FD is accumulated.

  The power supply voltage Vdd is applied to the drain of the amplification transistor M3, and its gate is connected to the FD. The amplification transistor M3 amplifies the charge (that is, voltage) of the FD and converts it into a voltage signal. The selection transistor M4 is for selecting the pixels 701 for reading out signals in units of rows, and the drain thereof is connected to the source of the amplification transistor M3. The source of the selection transistor M4 is connected to the column signal line 702.

  When the selection transistor M4 is turned on, a voltage signal corresponding to the voltage of the FD is output to the column signal line 702. The power supply voltage Vdd is applied to the drain of the reset transistor M2, and the source is connected to the FD. When the reset transistor M2 is turned on, the voltage of the FD is reset to the power supply voltage Vdd.

  The second chip 71 is provided with a column ADC block 711, and the column ADC 711 is connected to the column signal line 702-a or 702-b. Furthermore, the second chip 71 is provided with a row scanning circuit 712, column scanning circuits 713-a and 713-b, a timing control circuit 714, and horizontal signal lines (output means) 715-a and 715-b. .

  The timing control circuit 714 controls operation timings of the row scanning circuit 712, the column scanning circuits 713-a and 713-b, and the column ADC block 711 under the control of the control unit 650. The row scanning circuit 712 scans each row, and the column scanning circuits 713a and 713b scan each column.

  The horizontal signal lines 715-a and 715-b transfer the output signal (image signal) of the column ADC block 711 according to the timing controlled by the column scanning circuits 713- a and 713- b, respectively.

  As described later, the image signal transferred to the horizontal signal line 715-a is processed by the signal processing unit 650 as a first image signal (image signal for live view) obtained by the first read operation. It is given to 130. On the other hand, the image signal transferred to the horizontal signal line 715-b is given to the signal processing unit 130 as a second image signal (image signal of the object of interest) obtained by the second readout operation.

  FIG. 11 is a diagram for explaining an example of pixel selection when the image sensor shown in FIG. 10 is driven by the first readout operation.

  FIG. 12 is a diagram for explaining an example of pixel selection when the image sensor shown in FIG. 10 is driven by the second read operation.

  Here, for convenience of explanation, it is assumed that the image sensor 612 has 32 rows × 18 columns of pixels. In practice, the number of pixels of the imaging element 612 is 7680 rows × 4320 columns. Also, in general, pixels are arranged in a Bayer pattern, but here, for convenience of explanation, color filters are omitted.

  In FIG. 11, the pixels read out by the first read operation are all or part of the pixels 701 connected to the column signal line 702-a. The pixels read out by the second read operation are all or part of the pixels 701 connected to the column signal line 702-b.

  The column signal lines 702-a and 702-b are physically independent, so that the first and second read operations can be performed simultaneously. Therefore, the imaging element 612 can simultaneously output the first image signal obtained in the first read operation and the second image signal obtained in the second read operation.

  In the illustrated example, the specific pixel group (first pixel group) read out by the first read operation covers the entire angle of view, but the thinning ratio is large and the total number of pixels is small. By this, while maintaining the resolution required for the object-of-interest discrimination processing, the frame rate is snow-covered.

  On the other hand, for the pixel group (second pixel group) read out by the second read operation, the thinning rate is small and the total number of pixels is also compared with the first pixel group, since the subject of interest is obtained as high resolution as possible. There are many. Note that, since the area of the second pixel group is variable according to the result of the object-of-interest determination process, all pixels except the first pixel group may be set as the second pixel group.

  As described above, by dividing the live view imaging (first readout operation) and the imaging of the object of interest (second readout) for each selected row, images of frame rates different in data size with different charge accumulation times are obtained. You can get a signal.

  Next, voltage signals (analog signals) output to each of the column signal lines 702-a and 702-b are converted into digital signals (image signals) in a column ADC block 711 shown in FIG.

  The image signal which is the output of the column ADC block 711 is read out from the column ADC block 711 to the horizontal signal line 715-a or 715-b by the column scanning circuit 713-a or 713-b. The image signals read out to the horizontal signal lines 715-a and 715-b are sent to the signal processing unit 130.

  Here, thinning out reading in the first reading operation will be described with reference to FIG.

  In the first readout operation, readout of the imaging element 612 is performed so as to acquire the entire angle of view of the imaging element 612. In the first reading operation, it is assumed that reading is performed by thinning the pixel to 1/6 in each of the horizontal and vertical directions. By this read operation, 1280 × 720 pixels are read out, and the first image data is recorded and displayed as moving image data of HD. This makes it possible to obtain an image relating to the entire angle of view.

  Next, thinning out reading in the second reading operation will be described with reference to FIG.

  In the second read operation, as described later, a focused area designated by the control unit 650 is cut out by the imaging device drive circuit 623 and read is performed. Note that the attention object discrimination unit 1301 discriminates the attention area from the first image signal.

  In FIG. 12, the area around the person is taken as the attention area. In the second readout operation, readout is performed by thinning out the pixels to 1/2 in each of the horizontal and vertical directions in consideration of a decimation rate that can obtain an appropriate resolution according to the size of the attention area Do. By this read operation, 1920 × 1080 pixels are read out, and the second image data is recorded and displayed as full HD moving image data. By this, it is possible to obtain an image of a region around a person with high resolution.

  As described above, in the first image signal read in the first read operation, the number of pixels in the region of interest does not reach the number of pixels in the second image signal read in the second read operation. . That is, in the first image signal read out in the first read operation, the second image signal read out in the second read out operation is shown as compared to the image obtained by performing enlargement processing on the region of interest. Images are much higher quality.

  FIG. 13 is a timing chart for explaining imaging timing in the moving image mode in the camera shown in FIG.

  As illustrated, the imaging timing is defined by the vertical synchronization signal (VD) sent from the imaging device driving circuit 613 under the control of the control unit 650. In the illustrated example, for convenience of explanation, the frame rate in the first read operation is twice the frame rate in the second read operation.

  Here, the first image signal obtained in the first read operation is used as live view display, and the imaging element 612 is always driven. It is assumed that a start command of moving image recording is issued by the operation unit 170 before the frame of the period T0 to T1. In the frame in the period T0 to T1, the first image signal obtained in the first read operation is displayed on the image display unit 180 as a live view image. At this time, the region of interest (region of interest) of the object of interest is discriminated in the first image signal by the object of interest discrimination unit 1301 and given to the control unit 650 as object of interest region information.

  The control unit 650 sends driving conditions (i.e., imaging conditions) such as cutout position information and a thinning ratio used in the second reading operation to the imaging element driving circuit 623 according to the subject region information of interest.

  In the frame of period T1 to T3, in the second reading operation, the second image signal is read based on the cutting position information and the driving condition such as the thinning rate given to the imaging device driving circuit 623 at time T1. . Then, after the second image signal is processed by the signal processing unit 130, the second image signal is recorded as image data (moving image data) obtained by photographing the subject of interest at high resolution.

  Similarly, in periods T1 to T2 and periods T2 to T3, the first image signal obtained in the first readout operation is used as a live view image. At this time, as described above, the processing for discriminating the attention area is performed by the attention object discrimination unit 1301, and the control unit 650 sends the drive condition to the imaging element drive circuit 623.

  As described above, in the periods T0 to T5, both the first and second read operations are performed frame by frame until the moving image recording is stopped.

  As described above, the second image signal (here, a moving image) obtained in the second reading operation by performing the reading process and the determination process of the attention area has a high resolution limited to the attention subject area. It becomes an image, and the high resolution image is recorded.

  FIG. 14 is a flowchart for explaining the photographing process performed by the camera shown in FIG.

  Note that the illustrated flowchart is performed under the control of the control unit 650. Further, in the illustrated flowchart, the processes of steps S1102 to S1115 are the same as the processes of steps S402 to S415 shown in FIG.

  When the power of the camera is turned on and the camera is set to the photographing mode by the operation unit 170, the control unit 650 controls the imaging device driving circuit 613 to start the first read operation.

  Subsequently, the control unit 650 captures one frame (Fr1) of the live view image (first image signal) by the first read operation in the period T0 to T1 shown in FIG. 13 (step S1101). In the process of step S 1101, as described in connection with FIGS. 11 to 13, the angle of view is set so as to capture the entire angle of view captured by the image sensor 612, and thinning is performed to maintain a sufficient frame rate. Processing is performed.

  The first image signal obtained in the process of step S1101 is referred to as Fr1-0. The first image signal (live view image) read in the process of step S1101 is subjected to image processing and screen display as a live view or a recording moving image, but the detailed description is omitted here.

  Thereafter, as described with reference to FIG. 6, the control unit 650 performs the processing of steps S1102 to S1115 using the subject-of-interest determination unit 1301. Then, in step S115, the control unit 650 selects the drive method of the imaging element 122 according to the imaging element drive selection table Tbl2 described later according to the ratio r.

  FIG. 15 is a diagram showing an example of an imaging element drive selection table used in the camera shown in FIG. The imaging element drive selection table is stored in the ROM provided in the control unit 650.

  The imaging element drive selection table Tbl2 determines the thinning rate or the pixel addition number in the horizontal and vertical directions at the time of operation in the second reading according to the ratio r of the object area A of interest to the image. In the illustrated example, when the ratio r is 56.25% or more, the thinning rates in the horizontal and vertical directions are set to 1⁄4. When the ratio r is 25% or more and less than 56.25%, the thinning rates in the horizontal and vertical directions are reduced to 1/3.

  Furthermore, when the ratio r is less than 25%, the thinning rates in the horizontal and vertical directions are reduced to 1⁄2. Here, although the imaging device drive selection table Tbl2 is a table that defines the thinning ratio according to the ratio r, when reading out the object area of interest with the optimal number of pixels or the optimal frame rate, the imaging device drive selection is performed. The table Tbl2 is a table which defines the pixel addition number in accordance with the ratio r.

  Referring again to FIG. 14, the control unit 650 sends the image sensor driving method (that is, the thinning rate here) selected in the process of step S 1115 to the image sensor driving circuit 613 and the position in the image of the object area A of interest. . That is, the control unit 650 instructs the imaging conditions used in the second readout operation (step S1116). At this time, when the ratio r is 56.25% or more, the control unit 650 instructs the lens control unit 611 to make the angle of view a wide angle side by a predetermined amount. This avoids in advance the risk that the subject of interest will deviate from the image in the subsequent frames.

  Next, the control unit 650 controls the imaging element driving circuit 613 to perform imaging of one frame in the second readout operation in the periods T1 to T3 illustrated in FIG. 13 (step S1117). The second image signal obtained by this imaging is an image obtained by reading out the target object area at a high resolution.

  Here, the obtained second image signal is called Fr2-1. The second image signal is subjected to image processing and screen display as a moving image, but the detailed description is omitted here.

  Thereafter, the control unit 650 performs processing of the next frame. That is, after performing the process of step S1117, the control unit 650 returns to the process of step S1101 and performs imaging by the first readout operation during the period T1 to T2 shown in FIG. One image signal) is obtained for one frame. The first image signal obtained here is called Fr1-1.

  Furthermore, during the periods T2 to T3 shown in FIG. 13, imaging is performed by the first read operation to obtain one frame of a live view image. The first image signal obtained here is called Fr1-2. The first image signals Fr1-1 and Fr1-2 are processed in steps S1101 to S1116. Then, the object-of-interest area information obtained in the first image signals Fr1-1 and Fr1-2 is reflected in the second readout operation in step S117.

  As described above, in the second embodiment of the present invention, the target subject area is determined in the first image signal obtained in the first read operation. Then, the second read operation is controlled in accordance with the subject area of interest. Thus, an image having a desired angle of view can be cut out at a sufficient resolution to perform imaging. Then, unnecessary pixel readout and power consumption can be suppressed.

  When cutting out the target subject area, only the target subject area may be cut out, and further, an area within a predetermined range including the target subject area may be cut out.

  As is apparent from the above description, in the example shown in FIGS. 1 and 8, the object of interest discrimination unit 1301 and the control unit 150 or 650 function as discrimination means, and the control unit 150 or 650, the lens control unit 121 or 611. , And the imaging device drive circuit 123 or 613 function as control means. Further, the control unit 650 and the imaging device driving circuit 613 function as a reading unit.

  As mentioned above, although this invention was demonstrated based on embodiment, this invention is not limited to these embodiment, The various form of the range which does not deviate from the summary of this invention is also included in this invention .

  In addition, the control method may be executed by the imaging apparatus as the control method of the functions of the above-described embodiment. In addition, a program having the functions of the above-described embodiments may be used as a control program to cause a computer included in the imaging apparatus to execute the control program. The control program is recorded, for example, on a computer readable recording medium.

Other Embodiments
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. Can also be realized. It can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

111 and 121 lens control unit 112 and 122 imaging device 113 and 123 imaging device drive circuit 130 signal processing unit 140 compression and decompression unit 150 control unit 170 operation unit 180 image display unit 190 image recording unit 1301 object of interest discrimination unit

Claims (9)

An imaging apparatus having an imaging means comprising an imaging device for outputting the images by imaging an object,
In the image pickup element, readout of a signal from a first pixel group connected to a first column signal line is performed as a first readout operation to obtain a first image, and Is connected to a second column signal line provided physically independently, and at least a part of the pixel group is included in the imaging angle of view of the first pixel group, different from the first pixel group Reading means for obtaining a second image by reading signals from the two pixel groups in parallel with the first reading operation as a second reading operation;
And discriminating means for discriminating an area including the target object using a pre-Symbol first image as a target object region,
By controlling the reading means in accordance with the prior Kichu th subject area, and control means for the second image obtained by the second reading operation and the image containing the target object,
I have a,
The first image is wider than the second image, and the number of pixels of the second image is larger than the number of pixels of the region of interest in the first image. Imaging device. The imaging device according to claim 1, wherein a frame rate of the first read operation is higher than a frame rate of the second read operation. It said discrimination means, an imaging apparatus according to claim 1 or 2, characterized in that to determine the area of the moving subject Te first image odor as the attention object region. It said discrimination means, an imaging apparatus according to claim 1 or 2, characterized in that to determine the area of a person Te first image odor as the attention object region. It said discrimination means, an imaging apparatus according to claim 1 or 2, characterized in that to determine the face area of the first image odor Te person as the target object area. It said discriminating means, the imaging device according to user-specified area in the first image to claim 1 or 2, characterized in that to determine as the target object area. The control method according to any one of claims 1 to 6 , wherein a thinning rate is determined when obtaining the second image in accordance with the size of a target subject region in the first image. Imaging device. A method for controlling an imaging apparatus having an imaging means comprising an imaging device for outputting the images by imaging an object,
In the image pickup element, readout of a signal from a first pixel group connected to a first column signal line is performed as a first readout operation to obtain a first image, and Is connected to a second column signal line provided physically independently, and at least a part of the pixel group is included in the imaging angle of view of the first pixel group, different from the first pixel group Reading out a signal from the second pixel group as a second reading operation in parallel with the first reading operation to obtain a second image;
A determining step of determining a region including the target object using a pre-Symbol first image as a target object region,
And controlling the reading step in accordance with the prior Kichu th subject area, and a control step of the second image obtained by the second reading operation and the image containing the target object,
I have a,
The first image is wider than the second image, and the number of pixels of the second image is larger than the number of pixels of the region of interest in the first image. Control method. A control program used in an image pickup apparatus having an image pickup means comprising a imaging device for outputting the images by imaging an object,
In a computer provided in the imaging device,
In the image pickup element, readout of a signal from a first pixel group connected to a first column signal line is performed as a first readout operation to obtain a first image, and Is connected to a second column signal line provided physically independently, and at least a part of the pixel group is included in the imaging angle of view of the first pixel group, different from the first pixel group Reading out a signal from the second pixel group as a second reading operation in parallel with the first reading operation to obtain a second image;
A determining step of determining a region including the target object using a pre-Symbol first image as a target object region,
And controlling the reading step in accordance with the prior Kichu th subject area, and a control step of the second image obtained by the second reading operation and the image containing the target object,
Was executed,
The first image is wider than the second image, and the number of pixels of the second image is larger than the number of pixels of the region of interest in the first image. Control program.