JP5747638B2 - Imaging apparatus, imaging method, and program - Google Patents

Description

The present invention relates to an imaging apparatus , an imaging method, and a program capable of improving the speed of image processing on an image obtained by imaging .

In recent years, there has been an increasing demand for digital cameras capable of executing an imaging operation under set shooting conditions or performing image processing under set image processing conditions by setting shooting conditions and image processing conditions.
As described above, there is known a digital camera that performs various kinds of image processing on the data of the captured image for the purpose of improving the quality for appreciation of the captured image obtained by imaging or the like (see, for example, Patent Document 1). .

JP 2006-345509 A

  However, in order to execute image processing on an image obtained by imaging, it is necessary to wait for the processing until all image data is read from the imaging unit. For this reason, there is a problem that it takes time until image processing is performed on the captured image after the image is captured.

  The present invention has been made in view of such a situation, and an object thereof is to improve the speed of image processing on an image obtained by imaging.

In order to achieve the above object, an imaging device according to one embodiment of the present invention is configured to read pixel data from each pixel included in an imaging element in a predetermined reading order, so that the first pixel configured from pixel data of each pixel. First image acquisition means for acquiring captured image data, and second image acquisition for acquiring data of the second captured image before the first captured image data is acquired by the first image acquisition means means and to the data of the first of the captured image composed of pixel data of each pixel obtained by the image obtaining means, a specific image processing means for performing specific image processing, specific image definitive in the specific image processing means as a characteristic region the region in which take longer to process, the corresponding to the feature region of the second image, and the imaging range determining means for determining an imaging range of the imaging device, determine the imaging range determining means Based on the results, characterized in that it comprises a determining means for determining the reading order by the first image acquisition unit.

  ADVANTAGE OF THE INVENTION According to this invention, the speed of the image processing with respect to the image obtained by imaging can be improved.

It is a block diagram which shows the structure of the hardware of the imaging device of one Embodiment of this invention. It is a functional block diagram which shows the functional structure for performing an imaging process among the functional structures of the imaging device of FIG. It is a schematic diagram which shows the image pick-up element of the image pick-up device of FIG. 3 is a flowchart for explaining a flow of imaging processing executed by the imaging apparatus of FIG. 1 having the functional configuration of FIG. 2.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing a hardware configuration of an imaging apparatus 1 as an embodiment of the present invention.
The imaging device 1 is configured as a digital camera, for example.

  The imaging apparatus 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, an image processing unit 14, a bus 15, an input / output interface 16, and an imaging unit. 17, an input unit 18, an output unit 19, a storage unit 20, a communication unit 21, and a drive 22.

  The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 20 to the RAM 13.

  The RAM 13 appropriately stores data necessary for the CPU 11 to execute various processes.

The image processing unit 14 is configured by a DSP (Digital Signal Processor), a VRAM (Video Random Access Memory), and the like, and performs various image processing on image data in cooperation with the CPU 11.
For example, the image processing unit 14 performs image processing such as processing for lifting a dark portion or processing for performing makeup on a specific range such as a face on captured image data continuously output from the imaging unit 17. (Hereinafter referred to as “specific image processing”), and the data of the captured image after the specific image processing is output.

  The CPU 11, ROM 12, RAM 13, and image processing unit 14 are connected to each other via a bus 15. An input / output interface 16 is also connected to the bus 15. An imaging unit 17, an input unit 18, an output unit 19, a storage unit 20, a communication unit 21, and a drive 22 are connected to the input / output interface 16.

  Although not shown, the imaging unit 17 includes an optical lens unit including a focus lens and a zoom lens, and an imaging element.

The optical lens unit includes various lenses such as a focus lens and a zoom lens in order to photograph a subject.
The focus lens is a lens that forms a subject image on the light receiving surface of the image sensor. The zoom lens is a lens that freely changes the focal length within a certain range.
The optical lens unit is also provided with a peripheral circuit for adjusting setting parameters such as focus, exposure, and white balance as necessary.

As will be described with reference to FIG. 3 to be described later, the imaging element is configured by a photoelectric conversion element, an AFE (Analog Front End), or the like having a plurality of pixels.
The photoelectric conversion element is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element or the like. A subject image is incident on the photoelectric conversion element from the optical lens unit. Therefore, the photoelectric conversion element photoelectrically converts (captures) the subject image, accumulates the image signal for a predetermined time, and sequentially supplies the accumulated image signal as an analog signal to the AFE.
The AFE performs various signal processing such as A / D (Analog / Digital) conversion processing on the analog image signal. A digital signal is generated by various signal processing and output as an output signal of the imaging unit 17.
In the present specification, such an output signal of the imaging unit 17 is hereinafter referred to as “captured image data”. The captured image data is appropriately supplied to the CPU 11.

The input unit 18 includes various buttons such as a power button and a shutter button, and inputs various types of information according to user instruction operations.
The output unit 19 includes a display, a speaker, and the like, and outputs images and sounds.
The storage unit 20 is configured by a hard disk, a DRAM (Dynamic Random Access Memory), or the like, and stores various image data.
The communication unit 21 controls communication with other devices (not shown) via a network including the Internet.

  A removable medium 31 composed of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 22. The program read from the removable medium 31 by the drive 22 is installed in the storage unit 20 as necessary. The removable medium 31 can also store various data such as image data stored in the storage unit 20 in the same manner as the storage unit 20.

  FIG. 2 is a functional block diagram showing a functional configuration for executing the imaging process among the functional configurations of the imaging apparatus 1 as described above.

Here, “imaging processing” refers to the following series of processing executed by the imaging apparatus 1.
That is, when the specific image processing is performed on the data of the captured image for recording, the imaging device 1 places a load on the specific image processing among the recorded captured images (long processing time). Is required) based on data of a captured image (live view image described later) acquired before the captured image is acquired.
The imaging device 1 determines the reading order of the imaging unit 17 according to the determination result. In the present embodiment, the unit for reading out each pixel data from the image sensor of the imaging unit 17 is one line, but the reading order is not an order from top to bottom as in the conventional case, but an arbitrary order. Can do. For example, in the case of a captured image data in which a specific image processing load (long processing time) exists in the lower region, when the line from the top to the bottom is read as in the past, Until the line including the range is read, the image processing unit 14 needs to wait for processing. In order to eliminate the waiting time and shorten the entire processing time, the imaging apparatus 1 sets a reading order so that the pixel data of the line including the range is preferentially read, and the pixels of each line in the reading order. Data is read sequentially.
The imaging apparatus 1 performs specific image processing on each pixel data of each line read based on such a reading order. Such a series of processing is the imaging processing here.

When execution of the imaging process is controlled, the imaging control unit 41 functions in the CPU 11.
When the imaging process is started, the imaging control unit 41 performs a live view imaging process and a live view display process.
That is, the imaging control unit 41 continues the imaging operation by the imaging unit 17. Then, while the image capturing operation by the image capturing unit 17 is continued, the image capturing control unit 41 temporarily stores the captured image data sequentially output from the image capturing unit 17 in the memory (the storage unit 20 in the present embodiment). Remember me. Such a series of control processes is the “live view imaging process” referred to herein.
In addition, the imaging control unit 41 sequentially reads data of each captured image temporarily recorded in the memory (the storage unit 20 in the present embodiment) during the live view imaging process, and the captured image is described later. Control to sequentially display on the output unit 19 via the live view image acquisition unit 51 is executed. Such a series of control processes is the “live view display process” here, and the captured image displayed on the output unit 19 by the live view display process is the “live view image” here.

The user can determine the composition while viewing the live view image, and press the shutter button of the input unit 18 to the lower limit as an instruction operation for recording the captured image. In this manner, the operation of pressing the shutter button down to the lower limit is hereinafter referred to as “full press operation” or simply “full press”.
When the full press operation is performed, the imaging control unit 41 performs control until the captured image data output from the imaging unit 17 is subjected to specific image processing by the image processing unit 14 and is recorded in the removable media 31. To do.
As one of such controls, the imaging control unit 41 determines a reading order suitable for specific image processing, and executes control for sequentially reading pixel data of each line in the reading order.

  When the execution of the imaging process is controlled, in the image processing unit 14, the live view image acquisition unit 51, the pixel number determination unit 52, the imaging range determination unit 53, the recording image acquisition unit 54, and the identification The image processing unit 55 functions.

  The live view image acquisition unit 51 acquires the captured image data output from the imaging unit 17 as live view image data before the captured image data is acquired by the recording image acquisition unit 54 described later. The live view image is displayed on the output unit 19 as described above. The live view image acquisition unit 51 corresponds to the second image acquisition unit of the present invention.

Here, the user presses the shutter button of the input unit 18 halfway (predetermined position not reaching the lower limit) in order to cause the imaging apparatus 1 to execute AF (Auto Focus) processing or the like before performing the full pressing operation. The operation can be performed. The operation of pressing down to the middle of the shutter button (a predetermined position that does not reach the lower limit) is hereinafter referred to as “half-press operation” or simply “half-press”.
In the present embodiment, when a half-press operation is performed, the image processing unit 14 uses a live view image data to determine a reading order suitable for specific image processing. In this case, the pixel number determination unit 52 and the imaging range determination unit 53 function, and the data of the live view image is supplied to the pixel number determination unit 52 and the imaging range determination unit 53.

  The pixel number determination unit 52 determines whether or not the number of imaging pixels used for the specific image processing among the pixels constituting the imaging element (hereinafter referred to as “imaging pixels”) is a predetermined number or more. The pixel number discrimination unit 52 notifies the discrimination result to the imaging range discrimination unit 53 and the imaging control unit 41. The use of the determination result will be described later.

When it is assumed that the specific image processing is performed on the data of the live view image, the imaging range determination unit 53 corresponds to an area in the imaging element (hereinafter referred to as an area that requires processing time for the specific image processing). , Referred to as “imaging range”).
For example, it is assumed that the time required for specific image processing (processing time per unit of one pixel or the like) changes according to the luminance. In this case, the imaging range determination unit 53 can determine the imaging range corresponding to the region that takes a long processing time for the specific image processing by analyzing the luminance value (pixel value) of each pixel constituting the live view image. it can.

  The recording image acquisition unit 54 acquires the captured image data output from the imaging unit 17 as data for recording. The recording image acquisition unit 54 supplies the acquired captured image data to the specific image processing unit 55. The recording image acquisition unit 54 corresponds to the first image acquisition unit of the present invention.

  The imaging control unit 41 determines the line order for reading out pixel data from the image pickup element of the recording image acquisition unit 54 and the imaging unit 17, that is, the reading order, according to the determination result of the imaging range determination unit 53.

Here, with reference to FIG. 3, a specific method of determining the reading order applied in the present embodiment will be described.
FIG. 3 is a schematic diagram illustrating the imaging element of the imaging apparatus 1 and is a diagram for explaining the reading order of pixel data.
FIG. 3 shows the configuration of the pixel P of the image sensor of the imaging unit 17. That is, the imaging device is configured by arranging a plurality of lines Lk (k is an integer value of 1 to the total number of lines, and 1 to 13 are shown in FIG. 3) arranged in the vertical direction. Is done. One line Lk is configured by arranging a plurality of pixels P side by side in the horizontal direction. In the present embodiment, since the image sensor is a CMOS, reading of pixel data is performed in units of lines Lk.

As described above, the user performs an instruction operation to record a captured image by fixing his / her body while viewing the live view image and performing a full press operation. For this reason, the live view image immediately before the full-press operation and the captured image acquired for recording after the full-press operation are substantially matched images. Accordingly, among the captured images acquired for recording after the full-press operation, the image area corresponding to the imaging range determined by the imaging range determination unit 53 is also used for actual specific image processing by the specific image processing unit 55 described later. It can be grasped that it takes time. Therefore, the imaging control unit 41 determines the reading order so that the line Lk belonging to the imaging range is read preferentially.
Specifically, for example, when specific image processing is performed on live view image data, a region composed of a plurality of pixels belonging to any of the lines L11 to L13 takes a long time for the specific image processing. Shall. In such a case, the imaging control unit 41 determines the reading order so that the lines L11 to L13 are read preferentially.

In addition, when the number of imaging pixels to be subjected to the specific image processing is determined by the pixel number determination unit 52 to be equal to or less than the predetermined number, the processing time of the entire specific image processing is shortened by the smaller number of imaging pixels. The determination process of the imaging range determination unit 53 is not executed. Therefore, the determination result of the imaging range determination unit 53 is not notified to the imaging control unit 41.
In such a case, the imaging control unit 41 determines the reading order so that the lines are sequentially read from top to bottom as in the conventional case.

When the full-press operation is performed, the recording image acquisition unit 54 reads out each pixel data from the imaging element of the imaging unit 17 in units of lines according to the readout order determined by the imaging control unit 41, thereby recording imaging. Get image data.
The captured image data for storage is supplied to the specific image processing unit 55 according to the reading order in units of lines.

The specific image processing unit 55 performs specific image processing on the captured image data acquired for recording by the recording image acquisition unit 54.
Specifically, the unit of the specific image processing is each pixel constituting the captured image, and the processing order is in accordance with the reading order of the captured image in line units.
Here, for example, when there is an area that is burdensome for specific image processing (and therefore an area that requires a long time) below the captured image for recording, conventionally, the line reading order is simply the order from top to bottom. (Hereinafter referred to as “normal readout order”), it takes time to read out pixel data of a line belonging to the region, and a longer processing time is required for the entire specific image processing. It was.
On the other hand, in the present embodiment, the reading order is set so that lines belonging to a region that is burdensome for specific image processing (and hence a region that requires a long processing time) are read first. Therefore, the time until the pixel data of the line belonging to the area is read is shortened, and the processing time in the entire specific image processing can be shortened accordingly.
The specific image processing unit 55 records the captured image data subjected to the specific image processing on the removable medium 31 for recording.
As is clear from FIG. 1, data recording on the removable medium 31 is actually performed via the drive 22. However, in the description of FIG. 2 and FIG. 4 to be described later, for the sake of convenience of description, the description about the drive 22 is omitted.

Next, imaging processing executed by the imaging device 1 having the functional configuration of FIG. 2 described above will be described with reference to FIG.
FIG. 4 is a flowchart for explaining the flow of imaging processing executed by the imaging apparatus 1 of FIG. 1 having the functional configuration of FIG.

  The imaging process is started when a predetermined operation of the input unit 18 by the user is performed, and the following process is executed.

In step S11, the imaging control unit 41 of the CPU 11 starts a live view image display process. Thereby, the captured image data output from the imaging unit 17 is acquired by the live view image acquisition unit 51 and supplied to the output unit 19, and as a result, the live view image is displayed on the display of the output unit 19. .
In the present embodiment, it is assumed that the live view image continues to be displayed on the display of the output unit 19 until the imaging process is completed.

In step S12, the imaging control unit 41 determines whether or not the shutter button of the input unit 18 is half-pressed.
If it is not half-pressed, it is determined as NO in step S12, and the process returns to step S12 again. That is, until the half-press operation is performed, the determination process in step S12 is repeatedly executed, and the live view image is continuously displayed.
When half-pressed, it is determined as YES in step S12, and the process proceeds to step S13.

  In step S <b> 13, the number-of-pixels determination unit 52 has a predetermined number or more of imaging pixels used for specific image processing (the total number of pixels targeted for specific image processing among the pixels constituting the image sensor of the imaging unit 17). It is determined whether or not.

When the number of imaging pixels used for the specific image processing is less than the predetermined value, it is determined as NO in Step S13, and the process proceeds to Step S14.
In step S14, the imaging control unit 41 determines the normal reading order.

On the other hand, when the number of imaging pixels used for the specific image processing is equal to or larger than the predetermined number, it is determined as YES in Step S13, and the process proceeds to Step S15.
In step S <b> 15, the imaging range determination unit 53 determines the imaging range of the imaging element that takes time for specific image processing based on the live view image data acquired by the live view image acquisition unit 51 at that time.
In step S16, the imaging control unit 41 determines the reading order according to the determination result in step S15.

In this way, when the reading order is determined in the process of step S14 or step S16, the process proceeds to step S17.
In step S <b> 17, the imaging control unit 41 determines whether or not the shutter button of the input unit 18 has been fully pressed.
If it is not fully pressed, it is determined as NO in step S17, and the process returns to step S13 again. That is, a series of processing from step S13 to step S16 is repeatedly executed until the full press is performed.
If it is fully pressed, it is determined as YES in step S17, and the process proceeds to step S18.

  In step S <b> 18, the recording image acquisition unit 54 reads the pixel data of the line in the reading order determined in the immediately preceding step S <b> 14 or step S <b> 16.

  In step S19, the specific image processing unit 55 performs specific image processing on the pixel data of the line read out in step S18.

In step S20, the specific image processing unit 55 determines whether specific image processing has been performed on all lines.
In the case where the specific image processing is not performed on all the lines constituting the recording captured image, that is, when there is still a line on which the specific image processing is not performed, NO is determined in step S20. And the process returns to step S18 again.
That is, the loop processing from step S18 to step S20 is repeatedly executed for each line constituting the recording captured image.
When the specific image processing is performed on all the lines, it is determined as YES in Step S20, and the processing proceeds to Step S21.

In step S <b> 21, the specific image processing unit 55 records the captured image data after the specific image processing in the removable medium 31.
Thereby, the imaging process is completed.

As described above, the imaging apparatus 1 of the present embodiment includes the recording image acquisition unit 54, the live view image acquisition unit 51, the imaging range determination unit 53, and the imaging control unit 41.
The recording image acquisition unit 54 reads out pixel data from each pixel constituting the imaging element in a predetermined reading order, thereby obtaining data of a first captured image (captured image) configured from the pixel data of each pixel. get.
The live view image acquisition unit 51 acquires the data of the second captured image (live view image) before the recording image acquisition unit 54 acquires the data of the first captured image (captured image).
Based on the data of the second captured image (live view image) acquired by the live view image acquisition unit 51, the imaging range determination unit 53 determines the imaging range of the image sensor corresponding to the feature region of the second captured image. Determine.
The imaging control unit 41 determines the reading order based on the discrimination result of the imaging range discrimination unit 53, and controls the imaging operation by the imaging element.
In this case, based on the data of the second captured image (live view image) acquired before the first captured image (captured image) is acquired, the imaging range of the image sensor corresponding to the feature region is determined in advance. Can do. For this reason, it is possible to predict in advance an area that is burdensome for image processing or the like.
Then, it is possible to preferentially read from the image sensor corresponding to the area predicted to be loaded for image processing or the like, particularly when reading from the image sensor and image processing are performed in parallel. If it takes more time, or if image processing is performed in parallel for each pixel data that has been called in a predetermined order, the speed of image processing for an image obtained by imaging can be improved.

Furthermore, the imaging apparatus 1 according to the present embodiment further includes a specific image processing unit 55 that performs specific image processing on captured image data including pixel data of each pixel acquired by the recording image acquisition unit 54. Then, the imaging range determination unit 53 can determine an imaging range corresponding to the feature region, using a region where the specific image processing in the specific image processing unit 55 takes time as a feature region.
In this case, based on the data of the second captured image (live view image) acquired before the first captured image (captured image) is acquired, an image corresponding to the feature region is an area that takes time for specific image processing. The imaging range of the element can be determined in advance. For this reason, it is possible to predict an area where the specific image processing takes time before the recording image is acquired. Then, it is possible to preferentially read from the image sensor corresponding to the region predicted to take a long time for the specific image processing, particularly when reading from the image sensor and image processing are performed in parallel. If it takes more time, or if image processing is performed in parallel for each pixel data that has been called in a predetermined order, the speed of specific image processing on an image obtained by imaging can be improved.

Furthermore, the imaging range determination unit 53 of the imaging apparatus 1 according to the present embodiment determines the time taken for the specific image processing in the specific image processing unit 55 based on the luminance of each pixel constituting the second captured image (live view image). be able to.
In this case, the time required for the specific image processing in the specific image processing unit 55 can be predicted more easily.
If it takes a long time to capture image data after the mechanical shutter, circuit noise may be mixed in the image capturing unit 17. In this case, since there is little data of the original pixel in the dark portion of the captured image, the noise component is easily noticeable when noise is mixed into the corresponding pixel.
Even in such a case, the influence of noise due to the leakage current received from the circuit can be reduced by preferentially reading data from the imaging element corresponding to the dark portion, that is, the low luminance region.

  Furthermore, the imaging range determination unit 53 of the imaging apparatus 1 of the present embodiment can determine the time taken for the specific image processing in the specific image processing unit 55 based on the type of special effect performed in the specific image processing unit 55. In this case, since the time required for the specific image processing in the specific image processing unit 55 often depends on the type of the special effect, the time required for the specific image processing in the specific image processing unit 55 can be predicted more accurately. it can.

Furthermore, the imaging apparatus 1 according to the present embodiment includes a pixel number determination unit that determines whether or not the number of imaging pixels used in the specific image processing in the specific image processing unit 55 is greater than or equal to a predetermined number among the imaging pixels constituting the imaging element. 52 is further provided. The imaging range determination unit 53 determines the imaging range when the pixel number determination unit 52 determines that the number of imaging pixels is equal to or greater than a predetermined number. On the other hand, the imaging range determination unit 53 prohibits the determination of the imaging range when the pixel number determination unit 52 determines that the number of imaging pixels is less than the predetermined number.
Then, when the imaging range determination unit 53 prohibits the imaging range determination, the imaging control unit 41 can determine the reading order based on a criterion different from the determination result of the imaging range determination unit 53. Accordingly, when the number of imaging pixels determined by the pixel number determination unit 52 is less than a predetermined number, that is, when the number of imaging pixels is small, it can be predicted that the specific image processing will not take time. By prohibiting the above, unnecessary processing can be omitted and the processing speed can be improved as a whole. On the other hand, when the number of image pickup pixels determined by the pixel number determination unit 52 is a predetermined number or more, that is, when the number of image pickup pixels is large, it can be predicted that the specific image processing will take time. Therefore, in this case, by determining the imaging range, it is possible to improve the processing speed by preferentially reading from the area where the load is applied.

  In addition, this invention is not limited to the above-mentioned embodiment, The deformation | transformation in the range which can achieve the objective of this invention, improvement, etc. are included in this invention.

  In the above-described embodiment, when it is assumed that the specific image processing is performed on the live view image data, the imaging range determination unit 53 recognizes the region that takes a long processing time for the specific image processing as the specific region. However, the range in the image sensor corresponding to the specific area is determined, but the present invention is not limited to this. For example, the imaging range determination unit 53 recognizes a region recognized as characteristic by an arbitrary algorithm in the data of the live view image as a feature region, and determines a range in the imaging element corresponding to the specific region. Also good. As a region recognized as characteristic, a dark portion of live view image data, a region requiring special image processing, or the like is determined.

  Further, in the above-described embodiment, the imaging control unit 41 determines the line order for reading out pixel data from the imaging element of the recording image acquisition unit 54 and the imaging unit 17 according to the determination result of the imaging range determination unit 53. Is not limited to this. For example, the imaging control unit 41 may determine the order of pixel units as the order in which pixel data is read from the image sensor for recording 54 and the imaging unit 17 in accordance with the determination result of the imaging range determination unit 53. . In this case, the recording image acquisition unit 54 reads each pixel data from the imaging element of the imaging unit 17 in line units according to the readout order determined by the imaging control unit 41, but is not limited thereto. For example, the recording image acquisition unit 54 may read each pixel data from the imaging element of the imaging unit 17 in units of pixels according to the reading order determined by the imaging control unit 41. Furthermore, the recording image acquisition unit 54 does not have to read out in units of adjacent pixels or lines, and may read out in units of pixels or lines arranged apart from each other.

  In the above-described embodiment, the time required for specific image processing (processing time per unit of one pixel or the like) is changed according to the luminance, but is not limited thereto. For example, the time required for the specific image processing may be changed depending on the type of special effect performed in the specific image processing unit 55. In this case, the imaging range determination unit 53 obtains and analyzes in advance the types of special effects performed in the specific image processing unit 55 through the imaging control unit 41 and the like, thereby capturing an image corresponding to an area that takes processing time for the specific image processing. The range can be determined.

In addition, the imaging apparatus of the present embodiment is applied to the imaging apparatus 1, but is not limited to this, and can be applied to any apparatus as long as it can perform image processing.
For example, the present invention can be applied to electronic devices in general. Specifically, for example, the present invention can be applied to a smartphone, a notebook personal computer, a television receiver, a video camera, a portable navigation device, a mobile phone, a portable game machine, and the like.

The series of processes described above can be executed by hardware or can be executed by software.
In other words, the functional configuration of FIG. 2 is merely an example and is not particularly limited. That is, it is sufficient that the imaging apparatus 1 has a function capable of executing the above-described series of processing as a whole, and what functional blocks are used to realize this function is not particularly limited to the example of FIG.
In addition, one functional block may be constituted by hardware alone, software alone, or a combination thereof.

When a series of processing is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer incorporated in dedicated hardware. The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

  The recording medium including such a program is not only constituted by the removable medium 31 of FIG. 1 distributed separately from the apparatus main body in order to provide the program to the user, but also in a state of being incorporated in the apparatus main body in advance. It is comprised with the recording medium etc. which are provided in. The removable medium 31 is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disk is composed of, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), or the like. The magneto-optical disk is configured by an MD (Mini-Disk) or the like. In addition, the recording medium provided to the user in a state of being preliminarily incorporated in the apparatus main body includes, for example, the ROM 12 in FIG. 1 in which a program is recorded, the hard disk included in the storage unit 20 in FIG.

In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series along the order, but is not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.
Further, in the present specification, the term “system” means an overall apparatus configured by a plurality of devices, a plurality of means, and the like.

  As mentioned above, although several embodiment of this invention was described, these embodiment is only an illustration and does not limit the technical scope of this invention. The present invention can take other various embodiments, and various modifications such as omission and replacement can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in this specification and the like, and are included in the invention described in the claims and the equivalents thereof.

The invention described in the scope of claims at the beginning of the filing of the present application will be appended.
[Appendix 1]
First image acquisition means for acquiring data of a first captured image composed of pixel data of each pixel by reading out pixel data from each pixel constituting the image sensor in a predetermined reading order; Second image acquisition means for acquiring data of the second captured image before the data of the first captured image is acquired by the first image acquisition means; and the second image acquired by the second image acquisition means. An imaging range determining means for determining an imaging range of the imaging element corresponding to a feature area of the second captured image based on data of the captured image;
An imaging control unit that determines the readout order based on a determination result of the imaging range determination unit and controls an imaging operation by the imaging element;
An imaging apparatus comprising:
[Appendix 2]
Specific image processing means for performing specific image processing on captured image data composed of pixel data of each pixel acquired by the first image acquisition means;
The imaging range determination unit determines the imaging range corresponding to the feature region, using the region where the specific image processing in the specific image processing unit takes time as the feature region,
The imaging apparatus according to Supplementary Note 1, wherein
[Appendix 3]
The imaging range determining means determines the time required for the specific image processing in the specific image processing means based on the luminance of each pixel constituting the second captured image.
The imaging apparatus according to Supplementary Note 2, wherein
[Appendix 4]
The imaging range determining unit determines a time required for the specific image processing in the specific image processing unit according to a type of a special effect performed in the specific image processing unit;
The imaging apparatus according to Supplementary Note 2, wherein
[Appendix 5]
A pixel number determining means for determining whether or not the number of image pickup pixels used in the specific image processing in the specific image processing means is a predetermined number or more among the image pickup pixels constituting the image pickup device;
The imaging range discriminating unit discriminates the imaging range when the pixel number discriminating unit discriminates that the imaging pixel number is a predetermined number or more, and the pixel number discriminating unit determines the imaging pixel number. When it is determined that the number is less than the number, the determination of the imaging range is prohibited,
The imaging control unit determines the reading order based on a criterion different from the determination result of the imaging range determination unit when the determination of the imaging range is prohibited by the imaging range determination unit;
The imaging apparatus according to any one of supplementary notes 2 to 4, characterized in that:
[Appendix 6]
The image sensor is a CMOS, and the readout order is in line units,
The imaging control unit determines the reading order so that a horizontal line belonging to the imaging range determined by the imaging range determination unit is preferentially read.
The imaging apparatus according to any one of supplementary notes 1 to 5, wherein:
[Appendix 7]
A computer that controls an imaging device that acquires captured image data;
First image acquisition means for acquiring data of a first captured image composed of pixel data of each pixel by reading out pixel data from each pixel constituting the image sensor in a predetermined readout order;
Second image acquisition means for acquiring data of a second captured image before the data of the first captured image is acquired by the first image acquisition means;
An imaging range discriminating unit that discriminates an imaging range of the imaging element corresponding to a feature region of the second captured image based on the data of the second captured image acquired by the second image acquisition unit;
An imaging control unit that determines the readout order based on a determination result of the imaging range determination unit and controls an imaging operation by the imaging element;
A program characterized by functioning as

  DESCRIPTION OF SYMBOLS 1 ... Imaging device, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Image processing part, 15 ... Bus, 16 ... Input / output interface, 17 ... Image pick-up unit 18 ... Input unit 19 ... Output unit 20 ... Storage unit 21 ... Communication unit 22 ... Drive 31 ... Removable media 41 ... Imaging Control unit 51... Live view image acquisition unit 52... Pixel number determination unit 53. Imaging range determination unit 54... Recording image acquisition unit 55.

Claims (8)

First image acquisition means for acquiring data of a first captured image composed of pixel data of each pixel by reading out pixel data from each pixel constituting the image sensor in a predetermined readout order;
Second image acquisition means for acquiring data of a second captured image before the data of the first captured image is acquired by the first image acquisition means;
Specific image processing means for performing specific image processing on captured image data composed of pixel data of the respective pixels acquired by the first image acquisition means ;
The regions take longer to specific image processing definitive in the specific image processing means as a characteristic region, corresponding to the feature region of the second image, and the imaging range determining means for determining an imaging range of the imaging device,
A determination unit that determines the readout order by the first image acquisition unit based on a determination result of the imaging range determination unit;
An imaging apparatus comprising: The second captured image is a live view image that is displayed on a display unit so that a user of the imaging apparatus can confirm the imaging content captured by the imaging element.
The first captured image is a captured image acquired from the image sensor in response to a shooting instruction from the user.
The imaging apparatus according to claim 1. The imaging range determining means determines the time required for the specific image processing in the specific image processing means based on the luminance of each pixel constituting the second captured image.
The imaging apparatus according to claim 1 or 2 , wherein The imaging range determining unit determines a time required for the specific image processing in the specific image processing unit according to a type of a special effect performed in the specific image processing unit;
The imaging apparatus according to claim 1 or 2 , wherein A pixel number determining means for determining whether or not the number of image pickup pixels used in the specific image processing in the specific image processing means is a predetermined number or more among the image pickup pixels constituting the image pickup device;
The imaging range discriminating unit discriminates the imaging range when the pixel number discriminating unit discriminates that the imaging pixel number is a predetermined number or more, and the pixel number discriminating unit determines the imaging pixel number. When it is determined that the number is less than the number, the determination of the imaging range is prohibited,
The determination unit determines the reading order based on a criterion different from the determination result of the imaging range determination unit when the determination of the imaging range is prohibited by the imaging range determination unit;
The imaging apparatus according to any one of claims 1 to 4 , wherein the imaging apparatus is characterized in that: The image sensor is a CMOS, and the readout order is in line units,
The determining unit determines the reading order so that a horizontal line belonging to the imaging range determined by the imaging range determining unit is preferentially read;
The imaging apparatus according to any one of claims 1 to 5 , wherein An imaging method in an imaging apparatus for acquiring captured image data,
A first image acquisition step of acquiring data of a first captured image composed of pixel data of each pixel by reading out pixel data from each pixel constituting the image sensor in a predetermined reading order;
A second image acquisition step of acquiring data of a second captured image before the data of the first captured image is acquired by the first image acquisition step;
A specific image processing step of performing specific image processing on captured image data including pixel data of each pixel acquired in the first image acquisition step ;
The regions take longer to specific image processing definitive in the specific image processing steps as a characteristic region, corresponding to the feature region of the second image, and the imaging range determination step of determining the imaging range of the imaging device,
A determination step for determining the readout order by the first image acquisition step based on the determination result of the imaging range determination step;
An imaging method comprising: To a computer that controls an imaging device that acquires captured image data,
A first image acquisition function for acquiring data of a first captured image composed of pixel data of each pixel by reading out pixel data from each pixel constituting the image sensor in a predetermined readout order;
A second image acquisition function for acquiring data of a second captured image before the data of the first captured image is acquired by the first image acquisition function;
A specific image processing function for performing specific image processing on captured image data composed of pixel data of each pixel acquired by the first image acquisition function ;
Wherein the area that it takes a long time to locate the image processing at a specific image processing function as a characteristic region, corresponding to the feature region of the second image, the imaging range determination function of determining the imaging range of the imaging device,
A determination function for determining the readout order by the first image acquisition function based on the determination result of the imaging range determination function;
A program characterized by realizing.

Images