JP2009171191A - Image pickup device and image pickup method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image pickup device that accurately detects a tracking target on the basis of a color signal of an image to be imaged while preventing the tracking target from being lost, and to provide an image pickup method. <P>SOLUTION: In the image pickup device, before setting a tracking region of image data (a STEP 11), brightness correction of the image data is executed (a STEP 10). In the brightness correction, brightness is matched with that of the previously-stored color signal of the tracking target. Consequently, it is possible to set the tracking region on the basis of uniform image data when setting the tracking region. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image pickup apparatus represented by a digital video camera and an image pickup method thereof, and more particularly to an image pickup apparatus having a tracking function and an image pickup method thereof.

  In recent years, digital imaging devices such as digital still cameras and digital video cameras have become widespread. Some of such imaging devices have a function of automatically controlling the focus, exposure, and the like as a function to assist the user.

  In addition, some imaging apparatuses include a tracking function that detects an object determined by a user as a tracking target. As a tracking method, for example, there is a method in which the imaging apparatus recognizes and tracks a sound emitted from a tracking target or a radio wave emitted from a transmitter attached to the tracking target. However, if tracking is performed using these methods, the tracking target is limited.

Therefore, Patent Document 1 proposes a method of detecting and tracking a tracking target based on a color signal of image data to be captured. If tracking is performed by such a method, any object that can be visually recognized can be set as a tracking target.
JP-A-5-284411

  However, in the method of tracking based on the color signal of the image data to be captured, there is a problem that it becomes difficult to detect the tracking target if the image data greatly varies depending on the surrounding environment. In particular, when the brightness fluctuates, the entire image becomes brighter or darker, making it difficult to accurately detect and determine the color to be tracked from the image data. For this reason, the imaging apparatus cannot easily detect the tracking target and easily loses its sight, and in such a case, tracking becomes impossible.

  Therefore, an object of the present invention is to provide an imaging apparatus and an imaging method that accurately detect a tracking target based on a color signal of image data to be captured and make it difficult to lose track of the tracking target.

  In order to achieve the above object, an imaging apparatus according to the present invention includes an imaging unit that creates image data from an input optical image, and a tracking that indicates the position of a tracking target in the image data based on the input image data. A tracking area setting unit that sets an area; and a brightness of image data input to the tracking area setting unit, a storage unit that stores a tracking target color signal that is a signal indicating the tracking target A brightness correction unit that corrects the brightness, and the brightness correction unit corrects the brightness in a predetermined area of the image data input to the tracking area setting unit to be close to the brightness of the tracking target color signal. And the tracking region setting unit sets the tracking region based on the color signal included in the image data corrected by the brightness correction unit and the tracking target color signal. Imaging device according to symptoms.

  In the imaging apparatus having the above-described configuration, the overall photometric value is calculated based on the result of measuring the overall brightness of the image data, and the brightness of the first photometric area that is the area including the tracking area of the image data. A photometric unit that calculates a target photometric value based on the measurement result, the overall photometric value, and the target photometric value are each multiplied by a predetermined addition ratio to obtain a weighted addition value, An exposure control unit that controls the imaging unit based on the weighted addition value and controls the brightness of image data output from the imaging unit may be further provided.

  With this configuration, the brightness of the image data can be controlled not only by the brightness correction unit but also by the exposure control unit. Further, since the exposure correction unit controls the imaging unit that creates image data from the optical image, it is possible to perform fundamental correction of the image data. Therefore, it is possible to easily obtain optimal image data for performing the tracking process.

  In the imaging apparatus having the above configuration, when the target photometric value is out of the first range, an addition ratio by which the target photometric value is multiplied by the weighted addition value is set so that the target photometric value is in the first range. If it is a value within the range, it may be set to a value larger than that.

  With this configuration, when the target photometric value deviates from the first range, which is the normal range, it is possible to perform exposure control with an emphasis on the photometric value in the region near the tracking target. . Therefore, it is possible to make it easier to catch the tracking target.

  In the imaging apparatus having the above-described configuration, when the target photometric value continues to be a value within the first range, an addition ratio to be multiplied by the target photometric value in the weighted addition value is a normal exposure correction ratio value. It does not matter as being set to. Further, the normal exposure correction ratio value may be the lowest value that can be set or may be zero.

  With this configuration, if the target photometry value is continuously within the first range indicating the normal range, the addition ratio to be multiplied by the target photometry value is set to the normal exposure ratio value. Can do. Furthermore, if the lowest value that can be set, especially 0 is set, exposure correction is performed based only on the total photometric value. Therefore, when good tracking can be performed, the entire image data has good brightness.

  In the imaging device having the above-described configuration, when the target photometric value becomes a value within the first range from a value outside the first range, and then continuously becomes a value within the first range, The addition ratio to be multiplied by the target metering value in the weighted addition value may be set and maintained at the normal exposure correction ratio value after being set to a value other than the normal correction ratio value at least once. .

  By configuring in this way, it is possible to prevent the addition ratio to be multiplied by the target photometric value from being set to a small value when it once deviates from the first range which is the normal range and then returns to the first range. can do. That is, by performing exposure control with the addition ratio set to a small value, it is possible to prevent the target photometric value from deviating from the first range again. As a result, the exposure is controlled not to fluctuate greatly, and the brightness variation of the entire image data is reduced.

  In the imaging apparatus having the above-described configuration, when the target photometric value is outside the second range, the first photometric area is set larger than when the target photometric value is within the second range. It does not matter as well. The second range may be the same range as the first range.

  By configuring in this way, the first photometric area is enlarged when it deviates from the second range indicating the range in which the target photometric value is normal. That is, even if the tracking area cannot be set because it is difficult to track in the image data of the current frame, the probability that the tracking target exists in the first photometry area can be increased when the next photometry is performed. . Therefore, the next tracking process can be performed satisfactorily. Further, the first range and the second range indicate ranges where the target photometric values are normal, and are of the same quality. For this reason, the tracking process can be facilitated by collecting the same range.

  In the imaging apparatus having the above-described configuration, the target photometric value is a value obtained by measuring the brightness of the first photometric area, and a value obtained by measuring the brightness of a peripheral area that is a peripheral area of the first photometric area. It may be calculated on the basis of a value obtained by multiplying each of these by a predetermined addition ratio. The addition ratio multiplied by the first photometric area may be larger than the addition ratio multiplied by the peripheral area.

  By configuring in this way, it is possible to increase the probability that the photometry is performed based on the region including the tracking target. In addition, by setting the addition ratios of the first photometry area and the peripheral area, photometry according to the case can be performed, for example, when the tracking target is lost or good. In addition, by increasing the addition ratio multiplied by the first photometry area, it is possible to perform weighted addition with an emphasis on the first photometry area where the probability that a tracking target exists is greater.

  The imaging apparatus having the above-described configuration further includes a recording unit that records input image data, and the image data input to the recording unit is image data that has not been corrected by the brightness correction unit. It doesn't matter.

  With this configuration, the recorded image data is not subjected to brightness correction for tracking processing. Therefore, for example, it is possible to suppress recording of unbalanced image data in which only the periphery of the tracking target is clear and the periphery is unclear.

  The imaging method according to the present invention includes a first step of storing a tracking target color signal that is a signal indicating a tracking target, a second step of generating image data, and a predetermined step of image data generated in the second step. A third step for controlling the brightness of the region to approach the tracking target color signal stored in the first step, a color signal included in the image data corrected in the third step, and the first step And a fourth step of setting a tracking region indicating the position of the tracking target in the image data based on the tracking target color signal stored in one step.

  By adopting the configuration of the present invention, it is possible to perform correction so that the brightness of the image data subjected to the tracking process approaches the brightness of the tracking target color signal. Therefore, since the tracking area can be set based on the homogeneous image data, it is possible to suppress erroneous tracking target position by misidentifying the color to be detected. Therefore, accurate tracking can be performed.

  An embodiment of an imaging apparatus according to the present invention will be described with reference to the drawings. As an imaging apparatus, an imaging apparatus capable of recording audio, moving images, and still images, such as a digital still camera and a digital video camera, will be described as an example. Hereinafter, both moving image data and still image data will be described as image data.

(Basic configuration of imaging device)
First, the basic configuration of the imaging apparatus will be described with reference to FIG. FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus according to an embodiment of the present invention.

  As shown in FIG. 1, an imaging apparatus 1 includes an image sensor 2 including a solid-state imaging device such as a CCD (Charge Coupled Devices) or a CMOS (Complementary Metal Oxide Semiconductor) sensor that converts incident light into an electrical signal, and a subject. And the lens unit 3 that adjusts the amount of light and the like. The lens unit 3 and the image sensor 2 constitute an imaging unit, and image data is created by the imaging unit.

  Further, the imaging apparatus 1 converts the analog signal image data output from the image sensor 2 into a digital signal and converts the input sound into an electric signal and an AFE (Analog Front End) 4 that adjusts the gain. The image data of the digital signal of R (red) G (green) B (blue) output from the microphone 5 and the AFE 4 is converted into a signal using Y (luminance signal) U, V (color difference signal) and image data. The image processing unit 6 that performs various image processing, the audio processing unit 7 that converts analog audio data output from the microphone 5 into a digital signal, and the image data output from the image processing unit 6 are JPEG ( (Joint Photographic Experts Group) The image data output from the image processing unit 6 and the audio data from the audio processing unit 7 are subjected to compression encoding processing for still images such as a compression method. A compression processing unit 8 that performs compression encoding processing for moving images such as MPEG (Moving Picture Experts Group) compression method, an external memory 20 that records data encoded by the compression processing unit 8, and image data A driver unit 9 that records / reads data in / from the external memory 20 and a decompression processing unit 10 that decompresses and decodes the compression-encoded data read from the external memory 20 in the driver unit 9 are provided.

  In addition, the imaging apparatus 1 includes an image output circuit unit 11 that converts the image data decoded by the expansion processing unit 10 into an analog signal for display on a display device (not shown) such as a display, and the expansion processing unit 10 decodes the image data. And an audio output circuit unit 12 for converting the audio data into an analog signal for reproduction by a reproduction device (not shown) such as a speaker.

  In addition, the imaging device 1 includes a CPU (Central Processing Unit) 13 that controls the overall operation of the imaging device 1, a tracking processing unit 14 that detects a tracking target from input image data and performs a tracking process, and each process A memory 15 for storing each program for performing the program and temporarily storing data when the program is executed, and an operation unit to which an instruction from the user such as a button for starting imaging or a button for selecting an image correction condition is input 16, a timing generator (TG) unit 17 that outputs a timing control signal for matching the operation timing of each unit, a bus line 18 for exchanging data between the CPU 13 and each unit, a memory 15, and each unit And a bus line 19 for exchanging data with each other.

  The lens unit 3 includes various lenses (not shown) such as a zoom lens and a focus lens, and a diaphragm (not shown) that adjusts the amount of light input to the image sensor 2.

  The external memory 20 may be anything as long as it can record image data and audio data. For example, a semiconductor memory such as an SD (Secure Digital) card, an optical disk such as a DVD, a magnetic disk such as a hard disk, or the like can be used as the external memory 20. Further, the external memory 20 may be detachable from the imaging device 1.

(Basic operation of the imaging device)
Next, the basic operation of the imaging apparatus 1 will be described with reference to FIG. First, the imaging apparatus 1 acquires image data that is an electrical signal by photoelectrically converting light incident from the lens unit 3 in the image sensor 2. Then, the image sensor unit 2 sequentially outputs image data to the AFE 4 at a predetermined frame period (for example, 1/60 second) in synchronization with the timing control signal input from the TG unit 17.

  Then, the image data converted from the analog signal to the digital signal by the AFE 4 is input to the image processing unit 6. In the image processing unit 6, the image data is converted into a signal using YUV, and various image processing such as gradation correction and contour enhancement is performed. The memory 15 operates as a frame memory, and temporarily holds image data when the image processing unit 6 performs processing.

  At this time, based on the image data input to the image processing unit 6, the lens unit 3 adjusts the position of various lenses to adjust the focus, or adjusts the aperture and adjusts the exposure. It is done. This adjustment of focus and exposure is automatically performed based on a predetermined program so as to be in an optimum state, or manually performed based on a user instruction.

  Here, when the user instructs to perform the tracking process, the tracking processing unit 14 performs the tracking process based on the image data input from the image processing unit 6. Details of the tracking process will be described later.

  In the case of recording a moving image, not only image data but also audio data is recorded. Audio data that is converted into an electrical signal and output by the microphone 5 is input to the audio processing unit 7 and digitized and subjected to processing such as noise removal. The image data output from the image processing unit 6 and the audio data output from the audio processing unit 7 are both input to the compression processing unit 8 and compressed by the compression processing unit 8 using a predetermined compression method. At this time, the image data and the sound data are temporally associated with each other, and are configured so that the image and the sound are not shifted during reproduction. The compressed image data and audio data are recorded in the external memory 20 via the driver unit 9.

  On the other hand, when only a still image or audio is recorded, the image data or audio data is compressed by the compression processing unit 8 by a predetermined compression method and recorded in the external memory 20. Note that the processing performed in the image processing unit 6 may be different depending on whether a moving image is recorded or a still image is recorded.

  The compressed image data and audio data recorded in the external memory 20 are read to the expansion processing unit 10 based on a user instruction. The decompression processing unit 10 decompresses the compressed image data and audio data, and outputs the image data to the image output circuit unit 11 and the audio data to the audio output circuit unit 12, respectively. Then, in the image output circuit unit 11 and the audio output circuit unit 12, it is converted into a format that can be reproduced by a display device or a speaker and output.

  The display device and the speaker may be integrated with the imaging device 1 or may be separated and connected to a terminal provided in the imaging device 1 using a cable or the like. Absent.

  Further, in a so-called preview mode in which the user confirms an image displayed on a display device or the like without recording the image data, the image output circuit without compressing the image data output from the image processing unit 6 It may be output to the unit 11. In addition, when recording the image data of the moving image, the image data is output to the display device or the like via the image output circuit unit 11 in parallel with the compression processing unit 8 compressing it and recording it in the external memory 20. It does not matter.

(Configuration of tracking processing unit)
Next, the configuration of the tracking processing unit 14 will be described with reference to FIG. FIG. 2 is a block diagram illustrating a configuration of the tracking processing unit. As shown in FIG. 2, the tracking processing unit 14 includes a photometric unit 14 a that performs photometry of input image data, an exposure control unit 14 b that outputs an exposure control signal according to a photometric result of the photometric unit 14 a, and a photometric unit A brightness correction unit 14c that corrects the brightness of the image data based on the tracking target color signal and the image data after the photometry by 14a, and a tracking area based on the image data that has been subjected to the brightness correction by the brightness correction unit 14c. And a tracking region setting unit 14d that outputs a tracking result signal. Detailed operations and signals of each part will be described in the following description of the tracking processing operation.

(Tracking operation)
The tracking processing operation of the imaging apparatus 1 according to the embodiment of the present invention will be described with reference to FIGS. Here, the overall operation of the tracking process is shown in FIG. FIG. 3 is a flowchart illustrating the tracking process of the imaging apparatus according to the embodiment of the present invention.

  As shown in FIG. 3, when the tracking process is started, a tracking target is first selected (STEP 1). As a specific selection method, for example, there is a method of displaying a plurality of colors such as a color map and a color palette on a display device and allowing the user to select a color to be tracked.

  As another method of STEP 1, as shown in the schematic diagram of image data to be tracked in FIG. 4A, a captured image (still image or moving image) and an arrow cursor 21 are displayed on a display device. There is a method in which the user uses the arrow cursor 21 to select a tracking target. Moreover, it is good also as a structure which makes a user select a tracking object directly by using the display part of a display apparatus as a touch panel. With this method, the tracking target is selected simply by selecting the object to be tracked, and therefore the operation becomes very easy.

  As another setting method of STEP 1, as shown in FIG. 4B, a captured image (still image or moving image) is displayed on the display device, and several tracking target candidate colors detected by the tracking processing unit 14 are used. There is a method of displaying and allowing the user to select with the selection cursor 22a. When this method is applied, a region may be divided for each group of pixels having close signals, and each region may be set as a tracking target candidate. Further, as shown in FIG. 4B, the boundary 22b for each detected tracking target candidate may be highlighted. Furthermore, the detected moving body may be used as a tracking target candidate by using a moving body detection method as described later. As described above, when the user determines the tracking target and its color, it is possible to confirm whether or not the tracking target color and area are accurately recognized by the tracking processing unit 14.

  The tracking target selection method in STEP 1 described above is an example, and any method may be used as long as the tracking target can be selected.

  When the tracking target is selected in STEP 1, the tracking area and the initial search area are set (STEP 2). In this operation, for example, the tracking processing unit 14 detects a region close to the tracking target color from the input image data, sets a tracking region, and sets an initial search region based on the tracking region. It is done by methods. Further, for example, the initial search area is set to be a surrounding area including the tracking area.

  As another method of STEP2, there is a method of setting a tracking area and an initial search area based on the area selected by the user in STEP1.

  The method for setting the tracking area and the initial search area in STEP 2 described above is an example, and any other method can be used as long as the area including the tracking target is detected from the image data as the tracking area and the initial search area can be set. Such a method may be used. The operations in STEP1 and STEP2 may be performed by the tracking area setting unit 14d of the tracking processing unit 14.

  FIG. 4C is a schematic diagram of image data when the tracking area and the initial search area are set. FIG. 4C shows a case where the tracking target is the color of a person's jacket (color represented by a wavy line in the figure), and the tracking area 23 and the initial search area 24a are partial areas of the jacket. It has become. The tracking area 23 and the initial search area 24a are schematically shown as circles, but may be square or other shapes. Further, the tracking area 23 may have a shape including all the same color portions as the tracking target color.

  When the tracking area is set in STEP 2, the signal of the pixel included in the tracking area, that is, the tracking target color signal is stored in, for example, the memory 15 (STEP 3). Note that the tracking target color selected and determined in STEP 1 may be used as the tracking target color signal as it is.

  The tracking target color signal includes a signal related to pixel luminance and a signal related to color. The tracking target color signal may be in any format such as a signal expressed in YUV or a signal expressed in RGB.

  The tracking target color signal stored in STEP 3 is used as a reference value when performing the subsequent operations. Therefore, it is preferable that the image data used when setting the tracking area 23 and the initial search area 24a have good exposure and focus and the like.

  When the tracking target color signal is stored in STEP 3, it is confirmed whether or not an instruction to end the tracking process is issued (STEP 4). If an instruction to end the tracking process has been issued, the tracking process ends (STEP 4, YES). On the other hand, if the instruction to end the tracking process is not issued, the tracking process is continued (STEP 4, NO).

  When the tracking process is continued (STEP 4, NO), image data of the next frame is captured (STEP 5). Then, the tracking processing unit 14 performs the subsequent processing on the image data of the frame captured at this time. First, the photometry unit 14a of the tracking processing unit 14 performs photometry of the image data of this frame (STEP 6).

  The photometry in STEP 6 is performed by obtaining the photometric value of the image data of the frame captured in STEP 5. The photometric values obtained at this time are, for example, luminance signals, AE evaluation values (values calculated from pixel signals such as luminance signals and RGB signals), and RAW data output from the AFE 4 (one pixel has one color) Data having only the above-mentioned data, a signal obtained by digitizing an analog signal output from the image sensor 2, and the like)). The data used for obtaining the photometric value may be the above-described data, a combination of these, or other data.

  In addition, as shown in the schematic diagram of the photometry area in FIG. 4D, this photometry is performed in the photometry area 26 including the peripheral area 25 around the search area 24. When photometry is performed for the first time, the initial search area 24 a set in STEP 2 becomes the search area 24.

Then, a weighted addition is performed on the average value of the photometric values in the search area 24 and the average value of the photometric values in the surrounding area to obtain the target photometric value. Specifically, it is calculated as in the following formula (I).
(Average value of search area photometric value) × α + (Average value of photometric value of surrounding area) × β (I)
Thus, the target photometric value is obtained. Here, α + β = 1 may be set.

  When photometry is performed in STEP 6 and the target photometry value is obtained, the target photometry value is next determined (STEP 7). The determination at this time is a determination as to whether or not the target photometric value is within an allowable range. This determination will be described with reference to FIG. FIG. 5 is a schematic diagram of image data of a frame in which photometry, target photometric value determination, and search area setting are performed.

  FIG. 5A shows a case where the target photometric value is small and not included in the allowable range. At this time, since the entire image becomes dark, it becomes difficult for the tracking region setting unit 14d of the tracking processing unit 14 to detect a portion that is similar to the tracking target color signal from the image data. Therefore, when the tracking area position determination (STEP 11) described later is performed, the color of the image data is not accurately recognized, and the probability of determining the tracking area as an incorrect position is increased.

  Conversely, FIG. 5D shows a case where the target photometric value is large and not included in the allowable range. At this time, since the entire image becomes bright, it becomes difficult for the tracking region setting unit 14d of the tracking processing unit 14 to detect the color that is the tracking target from the image data. For this reason, as in the case of darkening, the color of the image data is not accurately recognized in tracking area position determination (STEP 11), which will be described later, so that the probability of determining the tracking area as an incorrect position is high.

  Therefore, when the target photometric value falls outside the allowable range as shown in FIGS. 5A and 5D, the search area 24 is set to be expanded (STEP 8). For example, it is assumed that the search area 24 is expanded by an amount deviating from the allowable range.

  At this time, the search area 24 is enlarged and set for the image data of the next frame that is input to the tracking processing unit 14 next to the current frame. By enlarging the search area 24 in this way, even if the tracking target is moving in the image data of the next frame, the probability that the photometry (STEP 6) of the portion including the tracking target is performed is increased. Note that the photometric area 26 may be enlarged in accordance with the enlargement of the search area 24 for the next frame, and the values of α and β in the above-described formula (I) may be changed. For example, the photometric area 26 may be enlarged and α may be increased.

  Further, as shown in FIGS. 5B and 5C, even if the target photometric value is low or high, it can be adjusted by brightness correction (STEP 10) described later if within the allowable range. Since there is a high probability that tracking area position determination (STEP 11), which will be described later, is normally performed by this adjustment, the search area 24 for the next frame is not enlarged here.

  When the search area 24 for the next frame is set as described above, exposure control is performed by the exposure control unit 14b of the tracking processing unit 14 (STEP 9). This exposure control will be described separately in two cases. First, the case where the target photometric value is within the allowable range as shown in FIGS. 5B and 5C will be described with reference to FIG. FIG. 6 is a schematic diagram of image data of a frame for which exposure control is performed, and a graph showing the luminance distribution and photometric value of the image data.

  FIG. 6A is a schematic diagram of image data in which the target photometric value falls within the allowable range, and shows the photometric area 26 together. Note that the photometric area 26 shown in this figure is in the current frame and indicates the area where photometry was performed in STEP 6. That is, the photometric area 26 in FIG.

  FIG. 6B is a histogram showing the luminance distribution of the image data shown in FIG. 6A, and a range 26a in the figure indicates a range in which the photometric values in the photometric area 26 in FIG. 6A are distributed. ing. FIG. 6C shows a photometric value A of the entire image data, a photometric value C of the photometric area 26, and a photometric value B obtained by weighted addition of the photometric value A of the entire image data and the photometric value C of the photometric area 26. It is the graph which showed one photometric value. The photometric value A for the entire image data is obtained by calculating the photometric value for the entire image data regardless of the photometric area 26. Note that the photometric value A of the entire image data may be calculated from a predetermined area such as the central portion of the image data, or may be calculated from an area including all of the image data.

A photometric value B obtained by weighted addition of the photometric value A of the entire image data and the photometric value C of the photometric area 26 is calculated as in the following formula (II).
B = γ × A + δ × C (II)
Exposure control is performed so that the photometric value B obtained by the formula (II) becomes the target photometric value Y _target . Then, as shown in FIG. 6, when the photometric value C of the photometric area 26 is within the allowable range, γ = 1 and δ = 0 are set. Then, B = A, and exposure control based only on the photometric value A of the entire image data is performed. That is, exposure control is performed in the same manner as normal exposure control.

FIGS. 6D and 6E show graphs of luminance distribution and photometric values of the image data of the next frame. FIGS. 6D and 6E correspond to FIGS. 6B and 6C shown for the current frame, respectively. By controlling as described above, in the next frame, the photometric value A and the weighted photometric value B of the entire image data are closer to the target photometric value Y _target .

  Next, the case where the photometric value in the photometric area 26 is outside the allowable range as shown in FIGS. 5A and 5D will be described with reference to FIG. Here, a case where the photometric value in the photometric area 26 is largely outside the allowable range as shown in FIG. 5D will be described as an example. FIG. 7 is a schematic diagram of image data of a frame for which exposure control is performed, and a graph showing the luminance distribution and photometric values of the image data, which are the same as those in FIG. Similarly to FIG. 6, the photometric area 26 shown in this figure is in the current frame, and indicates the area where photometry was performed in STEP6. That is, the photometric area 26 in FIG.

  In this example, in the histogram of the luminance distribution shown in FIG. 7B, the range 26a is a portion where the photometric value is large, and the photometric value C of the photometric area 26 in FIG. 7C exceeds the allowable range. Thus, when the photometric value C in the photometric area 26 is outside the allowable range, γ in the above-described formula (II) is set to be smaller than 1 and δ is set to be larger than 0. In this example, γ = 0.8 and δ = 0.2. Thus, by changing γ and δ, the ratio of the photometric value C of the photometric area 26 to the photometric value B subjected to weighted addition increases.

  Then, as shown in FIGS. 7D and 7E, when exposure control is performed based on the photometric value B weighted and added in the next frame, the influence of the photometric value C in the photometric area 26 is reflected. The photometric value C in the photometric area 26 is likely to be within the allowable range. In addition, since the photometric value A of the entire image data is greatly reflected, it is possible to prevent the brightness of the entire image data from becoming inappropriate. Therefore, it is possible to perform exposure control of image data with emphasis on the photometric value around the tracking target.

  In addition, γ and δ are controlled to change gradually. A specific example of this control method will be described with reference to FIG. FIG. 8 is a graph showing photometric values of image data of a frame for which exposure control is performed. Further, the example shown in FIG. 7 will be further described, and FIG. 8A and FIG. 7E show the same graph. Further, FIGS. 8A, 8B, 8C, 8D, and 8E show graphs of photometric values in the same frame of image data.

FIG. 8A shows a case where the photometric value B weighted and added at γ = 0.8 and δ = 0.2 is substantially equal to the target photometric value Y _target . At this time, since the photometric value C of the search area 26 is within the allowable range, it is assumed that adjustment is performed step by step so that γ = 0.9 and δ = 0.1.

Then, as shown in the graph of FIG. 8B, the weighted photometric value B becomes smaller than the target photometric value Y _target . Therefore, exposure control is performed so that the weighted photometric value B becomes the target photometric value Y _target . At this stage, since the next frame has not yet been captured, the photometric value A of the entire image data and the photometric value C of the photometric area 26 have not changed.

In the next frame, the photometric value B obtained by weighted addition as shown in FIG. 8C is close to the target photometric value Y _target . At this time, when the photometric value C of the search area 26 is within the allowable range as shown in FIG. 8C, exposure control is performed by setting γ = 1 and δ = 0 as shown in FIG. 8D. Or may be maintained at γ = 0.9 and δ = 0.1. On the other hand, if the photometric value C in the photometric area 26 falls outside the allowable range again, γ = 0.8 and δ = 0.2 may be returned. Note that any method may be used as long as exposure control is performed while suppressing fluctuations in photometric values.

  While the above control is continued, the imaging environment changes or the exposure is set to an appropriate value. Finally, as shown in FIG. 8E, when γ = 1 and δ = 0, the photometric value C in the photometric area 26 falls within the allowable range. In such a state, unless the photometric value C of the photometric area 26 falls outside the allowable range, exposure control is performed with γ = 1 and δ = 0 fixed in the subsequent exposure control.

  The exposure control in STEP 9 described above is performed in the exposure control unit 14b of the tracking processing unit 14, and is performed by the exposure control unit 14b outputting an exposure control signal. For example, this is performed by the CPU 13 controlling the aperture of the lens unit 3 and the accumulation time of the image sensor 2 based on the exposure control signal.

  If exposure control is performed in STEP 9, brightness correction is performed next (STEP 10). This brightness correction is to correct the brightness of the image data of the current frame in order to accurately perform tracking area position determination (STEP 11) described later. Thereby, for example, a signal indicating brightness, such as a luminance signal of a pixel included in the image data and a value of each color of the RGB signal, is corrected.

  Here, the brightness correction in STEP 10 is not performed on the recording image data sent from the image processing unit 6 to the compression processing unit 8 or the reproduction image data sent to the display device during preview or the like. . On the other hand, the image data used for the tracking process is subjected to brightness correction in the brightness correction unit 14c. However, since the exposure control in STEP 9 is performed by controlling the lens unit 3, the image sensor 2, and the like, the optical image input to the image sensor 2 and the analog image data itself input to the AFE 4 change. To do. For this reason, image data for recording and reproduction also changes. Note that the image processing unit 6 may separately perform image processing such as gamma correction on the image data for recording or reproduction.

  The brightness correction in STEP 10 will be specifically described. This brightness correction is performed by the brightness correction unit 14c of the tracking processing unit 14. Here, it is performed in order to make the brightness of the signal of the pixel included in the tracking area approach the brightness of the tracking target color signal stored in STEP3.

  For example, this is performed by comparing the pixel signal of the current frame in the set tracking area with the tracking target color signal stored in STEP 3. Based on the comparison result, the brightness correction unit 14c corrects the brightness of the image data of the current frame. As a result, the brightness of the image data of the current frame is corrected and approaches the brightness of the tracking target color signal.

  The part to be corrected at this time may be the entire image data, or only the tracking area and its surrounding area. Further, the corrected portion can be a search area or a peripheral area including the search area. Since the search area is enlarged when tracking becomes difficult as described above, including the search area can increase the probability that brightness correction is performed on the area where the tracking target exists.

  Next, the tracking area setting unit 14d of the tracking processing unit 14 determines the position of the tracking area based on the image data that has been subjected to brightness correction in STEP 10 (STEP 11). As a method for determining the position of the tracking region, for example, there is a method using a moving object detection method such as a background difference method, an interframe difference method, or an estimation method based on an optical flow.

  In the imaging apparatus 1 according to the embodiment of the present invention, any method may be used as long as the position of the tracking region is determined based on image data that has been subjected to brightness correction. Here, as a specific example, a method for determining the position of the tracking region using these moving object detection methods will be described below.

  First, the case of using the background difference method will be described with reference to FIG. FIG. 9 is a schematic diagram of image data showing the background subtraction method. The background subtraction method is a method for detecting a region including a moving object by subtracting background image data from image data of a target frame. For this reason, when applied to an imaging apparatus that is fixedly used such as a fixed point camera, background image data is stored in the memory 15 or the like by capturing an image as a background in advance. On the other hand, when applied to an imaging apparatus that is not used in a fixed manner, background image data is created and stored in the memory 15 or the like by taking the average or variance of several frames of image data.

  For example, by subtracting the background image data as shown in FIG. 9B from the image data of the target frame for which the tracking area as shown in FIG. 9A is to be determined, it is shown in FIG. 9C. Such difference image data is obtained. Then, in this difference image data, an area 27a including a moving object is extracted.

  The background subtraction method is very vulnerable to environmental changes such as brightness. Specifically, when the environment changes, almost all signals of the image data change, so that almost all parts are regarded as moving objects. However, in the imaging device 1 according to the embodiment of the present invention, since brightness correction is performed in STEP 10, the environmental change of the image data is reduced. Therefore, even if the background subtraction method is used, it is difficult to be affected by environmental changes, and the region 27a including the moving object can be accurately extracted.

  Next, the case of using the interframe difference method will be described with reference to FIG. FIG. 10 is a schematic diagram of image data showing the interframe difference method. The inter-frame difference method is such that (n−1) frame image data as shown in FIG. 10A and n frame image data as shown in FIG. 10B are adjacent in time series. The moving object is detected by taking the difference between the image data of the frames to be performed. Then, difference image data as shown in FIG. 10C is obtained, and a region 27b including a moving object is extracted from the image data.

  This inter-frame difference method is also very weak to environmental changes such as brightness as in the background difference method described above, and if the environment changes, it is considered that almost all parts of the image data have moved. However, in the imaging device 1 according to the embodiment of the present invention, since the brightness correction is performed in STEP 10, the environmental change is reduced in the image data. Therefore, even if the inter-frame difference method is used, it is difficult to be affected by the environmental change, and the region 27b including the moving object can be accurately extracted.

  By using the moving object detection method as described above, the areas 27a and 27b including the moving object in the image data can be detected. Then, predetermined feature values (for example, tracking target color data and hue information obtained from pixels included in the tracking area of the previous frame, and so on), and hue information of pixels in the areas 27a and 27b including the moving object And determining the level of the correlation and determining the position of the tracking region. For example, a correlation value may be obtained based on a difference in hue information, and a region where the correlation value is a predetermined value or less may be determined as a region having a high correlation. Then, the position of the area determined to have a high correlation by such a method may be determined as the position of the tracking area.

  When using the background subtraction method and the interframe subtraction method, a region having a high correlation with a predetermined feature amount is extracted from background image data or image data of each frame, and the position of the tracking region is determined by taking these differences. It does not matter as a method.

  In addition, an estimation method using an optical flow, in particular, a case where a block matching method is used will be described with reference to FIG. FIG. 11 is a schematic diagram of image data shown for the block matching method. FIG. 11A shows the image data of the (n−1) th frame, and the tracking block 101 in the image data of the (n−1) th frame is set to the outer jacket portion to be tracked. Shows the case. FIG. 11B shows the image data of the nth frame, and the candidate block 111 and the search block 112 are set in the image data of the nth frame.

  In this method, a block correlation value between the tracking block 101 and the candidate block 111 is calculated. At this time, the candidate block 111 is moved one pixel at a time in the horizontal direction or the vertical direction in the search block 112, and a block correlation value is calculated for each movement. The block correlation value is calculated based on a difference in luminance signal between the tracking block 101 and the candidate block 111.

  The block correlation value decreases as the correlation between the pixel signal in the tracking block 101 and the pixel signal in the candidate block 111 increases. Therefore, the position of the tracking block 101 in the image data of the nth frame is obtained by obtaining the position of the candidate block 111 having the smallest block correlation value. At this time, the motion vector of the tracking block 101 between the image data of the (n−1) th frame and the image data of the nth frame is obtained.

  Then, the tracking block 101 is set by repeatedly obtaining a motion vector between adjacent frames in time series and sequentially moving the tracking block 101 in accordance with the motion vector. The tracking block 101 obtained in this way becomes a region including a moving object, and the position of the tracking region is determined by correlating the pixel signal of this region with a predetermined feature amount.

  The search block 112 may be the search area described above, or the search block 112 may be set based on the search area. Also, the block correlation value between the tracking block 101 and the candidate block 111 may be obtained from the difference in hue information.

  The position of the tracking area is determined in STEP 11 by the method shown in the above specific example. However, in some cases, the imaging environment has changed too much, and the correlation of hue information in the tracking area may be low and reliability may be low. Therefore, the reliability of the tracking area determined next is determined (STEP 12).

  If the reliability of the tracking area is low because a high correlation was not obtained in STEP 11, it is determined that the tracking area cannot be set (STEP 12, NO). In this case, the tracking area whose position is determined in STEP 11 is not set. That is, the tracking area is the same location as the previously set location and is maintained. The tracking area setting unit 14d outputs a tracking result signal indicating that the position of the tracking area is not changed from the previous time. Then, the process returns to STEP 4 to check whether or not an instruction to end the tracking process is input, and the same process as the process described above is performed.

  On the other hand, when a high correlation is obtained in STEP 11, it is determined that the tracking area can be set (STEP 12, YES). In this case, the position of the tracking area is set at the position determined in STEP 11 (STEP 13). Further, the search area is set to be around the tracking area (STEP 14). At this time, a tracking result signal indicating that the positions of the tracking area and the search area are to be changed is output from the tracking area setting unit 14d.

  Then, the process returns to STEP 4 to check whether or not an instruction to end the tracking process is input, and the same process as the process described above is performed.

  A process of repeating STEP4 to STEP14, that is, a tracking process that is continuously performed on a plurality of frames will be described with reference to FIG. FIG. 12 is a sequence diagram illustrating an example of the tracking process of the imaging apparatus according to the embodiment of the present invention. FIG. 12 shows a state when STEP 12 or STEP 14 in FIG. 3 ends and the process returns to STEP 4.

  FIGS. 12A to 12F show the image data of the m-th to (m + 5) frames, respectively. In FIG. 12, for the sake of simplicity, specific examples of the image data as shown in FIGS. 4 to 7 are omitted.

  First, in FIGS. 12A and 12B showing the image data of the m-th and (m + 1) th frames, the target photometric value calculated in STEP 6 of FIG. 3 described above is determined to be within the allowable range in STEP 7. Therefore, the search area 24 is not enlarged and set in STEP 8, but is a normal range.

  Further, since the photometric value is within the allowable range in STEP 7, in the exposure control in STEP 9, γ = 1 and δ = 0 are set in the above-described formula (II). That is, exposure control is performed based on the photometric value of the entire image data, and exposure control similar to normal exposure control is performed. Further, the brightness correction in STEP 10 is performed, and the brightness of the image data is brought close to the tracking target color signal before the tracking area is determined in STEP 11. 12A and 12B, since the tracking area 23 set in STEP 11 has a high correlation, the tracking area 23 and the search area 24 are set in STEP 13 and STEP 14, respectively.

  On the other hand, in FIG. 12C showing the image data of the (m + 2) frame, since the imaging environment has changed, it is determined that the target photometric value determination in STEP 7 is outside the allowable range. Therefore, the search area 24 is enlarged and set in STEP8. Further, in the exposure control in STEP 9, γ in the formula (II) is smaller than 1 and δ is larger than 0, and exposure control is performed with emphasis on the photometric value in the photometric area.

  Further, in FIG. 12C, although the brightness correction is performed in STEP 10, it is determined that the correlation is not high enough to set the tracking region 23 in STEP 12. In this case, the tracking area 23 is the same area as the area set in FIG.

  In the processing of the (m + 2) -th frame image data shown in FIG. 12C, the search area 24 is enlarged, and exposure control is performed with emphasis on the photometric area including the search area 24, so that FIG. In the image data of the (m + 3) th frame shown in (), the target photometric value is approaching the allowable range. However, the symmetrical photometric value is not within the allowable range even in the image data of this frame. Further, it is determined that even if the brightness correction of STEP 10 is performed, a correlation high enough to set the tracking area 23 in STEP 12 cannot be obtained. In this case, as in FIG. 12C, the tracking area 23 is maintained in the same area as the area set in FIG.

  In the processing of the image data of the (m + 2) th and (m + 3) th frames shown in FIGS. 12C and 12D, by continuously performing the exposure control with emphasis on the search area 24 and the photometry area, In the processing of the image data of the (m + 4) th frame shown in FIG. 12E, the target photometric value is recovered until it is slightly outside the allowable range. Therefore, the search area 24 is slightly reduced. At this time, since it has been determined in STEP 12 that the tracking area 23 can be set, the tracking area 23 is newly set.

  Then, in the processing of the image data of the (m + 5) th frame shown in FIG. 12 (f), the target photometric value is within the allowable range, and the tracking area 23 is also set. Here, in the exposure control in STEP 9, γ in formula (II) approaches 1 and δ approaches 0. As a result, the addition ratio of the photometric values in the photometric area subsequently decreases. When the target photometric value is maintained within the allowable range thereafter, exposure control is finally performed based only on the photometric value of the entire image data.

  By performing the tracking process as described above, the tracking region 23 can be determined while keeping the brightness of the tracking target constant. Therefore, it is possible to reduce the tracking region setting unit 14d of the tracking processing unit 14 from erroneously recognizing the color of the image data. Therefore, it is possible to continue to perform reliable tracking processing that is less susceptible to changes in the imaging environment.

  In addition, since the search area is expanded when the target photometric value exceeds the allowable range, the tracking area is not set in the image data of the current frame, and the tracking is maintained in the image data of the next frame even if the tracking area is maintained at the previous position. It is possible to suppress the region from being outside the search region. Therefore, the probability that the tracking area is set in the next frame can be increased.

  Further, when performing exposure control in STEP 9, exposure control is performed based on weighted addition with the photometric value of the entire image data, without performing exposure control based only on the photometric value component of the photometric area. Therefore, it is possible to suppress the deterioration of the brightness of the entire input image data. In particular, the exposure control affects the image data for recording and reproduction. Therefore, when exposure control is performed in this way, it is possible to suppress degradation of image data for recording or reproduction by performing tracking processing.

  Further, when the photometric value of the photometric area falls within the allowable range, the addition ratio of the photometric value of the photometric area in the target photometric value (δ in the above formula (II)) is gradually approached to 0 without immediately setting it to 0. It is configured. Therefore, it is possible to suppress a sudden change in the brightness of the entire image data. Therefore, it is possible to reduce fluctuations in the brightness of the entire image data caused by abrupt exposure correction, so that tracking processing is performed well and deterioration of image data for recording and reproduction is suppressed. be able to.

  Further, by performing the tracking process as described above, the position of the tracking target in the image data can be detected. Therefore, based on the tracking result signal output from the tracking area setting unit 14d, for example, it is possible to perform control for focusing on the tracking target, imaging control for arranging the tracking target at the center of the image data, and the like.

  As a control method for focusing on the tracking target, for example, there is a method in which the high frequency component of the luminance of the pixel in the tracking area is increased. In addition, as a method of arranging the tracking target at the center of the image data, a mechanical method for controlling the direction of the imaging device or the optical system, or a tracking target and its peripheral region are extracted from the obtained image data and edited. There are such image processing methods. The method of using these tracking result signals is merely an example, and the tracking result signal may be used in any way.

  Further, before setting the tracking target color signal and the initial search area in STEP 1 to STEP 3, it is determined whether or not the image data that is the setting source is good, and these settings are determined based on the good image data. It may be performed. For example, it may be determined whether exposure, focus, etc. are appropriate, or photometric values such as STEP 6 and STEP 7 may be measured and determined.

  By making such a determination, it is possible to further suppress the tracking region setting unit 14d from erroneously recognizing or lowering the correlation in STEP11. In addition, for the above-described determination of exposure, focus, etc., any value such as an average value of luminance signals of pixels in a predetermined area of image data or a high frequency component may be used.

  Further, while the tracking process is being performed, the tracking target color signal may be displayed on the display device or the like to the user. By displaying in this way, it is possible to make the user confirm whether or not the tracking target color signal is accurately stored. Further, the tracking target may be displayed with a frame. By displaying in this way, it is possible to make the user confirm whether tracking is being performed accurately.

  Further, the color space used for obtaining the correlation value in STEP 11 is expressed in a format such as HSB (Hue, Saturation, Brightness), HSV (Hue, Saturation, Value), or HSL (Hue, Saturation, Lightness). The H component may be indicated. Further, the H component may be represented by an angle. For example, 0 ° may be defined as red, 120 ° may be defined as green, and 240 ° may be defined as blue, and the respective intermediate colors may be included in the angles therebetween. Then, the angle difference may be calculated when obtaining the correlation value.

  By setting the hue information in this way, the color signal detected from the tracking target color signal and the color signal detected from the image data whose brightness is close to the tracking target color signal in STEP 10 are used. Thus, the correlation value is obtained. For this reason, there is no signal relating to luminance that tends to fluctuate, and the correlation value can be calculated accurately.

  Further, as an example of the brightness correction in STEP 10, the brightness of the current frame is corrected by comparing the pixel signal of the image data of the current frame included in the previous tracking area with the tracking target color signal. However, in the present invention, it is also possible to correct the brightness using other methods. For example, the motion vector is detected when the previous tracking area is obtained, and thereby the tracking area in the current frame is predicted, and the pixel signal included in the predicted tracking area is compared with the tracking target color signal. Brightness correction may be performed. For example, the motion vector may be obtained by comparing the tracking area set in STEP 13 with the tracking area before setting, or is calculated when the tracking area is obtained as in the block matching method described above. You may use a motion vector.

  In this way, by performing the brightness correction in STEP 10, it is possible to increase the probability that the brightness of the tracking target in the current frame is directly corrected. For example, this method can be used even when the amount of movement of the tracking target is large and the overlapping of the tracking target is small between the image data of the previous frame and the image data of the current frame.

  In the example shown in FIG. 12E, the search area 24 is reduced because the target photometric value is sufficiently close to the allowable range. However, the search area 24 is set until the tracking area 23 of STEP 13 is set. It is also possible to set as much as possible. That is, even if the target photometric value is sufficiently close to or within the allowable range, if the tracking area 23 is not set, the search area 24 may be set to the maximum size.

  Further, by setting the tracking area in STEP 13, the search area may be set to be reduced in STEP 14. By configuring as described above, it is possible to suppress the tracking target from being outside the search region, and it is possible to further reduce the portion of the photometry region that does not include the tracking target when performing the photometry in STEP 6. .

  Further, the allowable range for determining whether or not to expand the search area in STEP 7 may be different from the allowable range used for setting the weighted addition ratio when performing exposure control in STEP 9. It does not matter as a range. In the above example, the same range is used. Then, since it is assumed that tracking becomes difficult when the allowable range is exceeded, weighted addition is performed by enlarging the search area and increasing the ratio of the photometric area.

  The tracking process described above may be performed on the image data of all the frames output from the image sensor 2 or may be performed every several frames. In either case, the frame number (n, m, etc.) described above indicates the frame number input to the tracking processing unit 14.

  The tracking process described above can be used when capturing a moving image or a still image. When a moving image is captured, it can be used both at the time of preview without recording image data and at the time of recording. On the other hand, when a still image is captured, it can be used mainly before a still image recording instruction is issued, for example, during preview. In addition, when image data of a still image is created based on image data of a plurality of frames, it can be used during still image capturing.

  Moreover, about the imaging device 1 in the embodiment of the present invention, the image processing unit 6, the audio processing unit 7, the compression processing unit 8, the expansion processing unit 10, the tracking processing unit 14 (photometry unit 14 a, exposure control unit 14 b, brightness correction). Each of the operations of the unit 14c, the tracking area setting unit 14d) and the like may be performed by a control device such as a microcomputer. Further, all or part of the functions realized by such a control device is described as a program, and the program is executed on a program execution device (for example, a computer) to realize all or part of the functions. It doesn't matter if you do.

  In addition to the above-described case, the imaging apparatus 1 in FIG. 1 and the tracking processing unit 14 in FIG. 2 can be realized by hardware or a combination of hardware and software. Further, when the imaging apparatus 1 and the tracking processing unit 14 are configured using software, a block diagram of a part realized by software represents a functional block diagram of the part.

  The embodiment of the imaging device according to the present invention has been described above, but the scope of the present invention is not limited to this, and various modifications can be made without departing from the spirit of the invention.

  The present invention relates to an imaging apparatus represented by a digital video camera and an imaging method thereof. In particular, the present invention is preferably applied to an imaging apparatus having a function of performing tracking using a color signal of image data and an imaging method thereof.

These are block diagrams which show the structure of the imaging device in embodiment of this invention. These are block diagrams which show the structure of the tracking process part of the imaging device in embodiment of this invention. These are the flowcharts shown about the tracking process of the imaging device in embodiment of this invention. These are schematic diagrams of image data to be subjected to tracking processing. FIG. 5 is a schematic diagram of image data of a frame in which photometry, target photometric value determination, and search area setting are performed. These are the schematic diagram of the image data of the flame | frame in which exposure control is performed, and the graph which shows the luminance distribution and photometric value of image data. These are the schematic diagram of the image data of the flame | frame in which exposure control is performed, and the graph which shows the luminance distribution and photometric value of image data. These are graphs showing photometric values of image data of frames for which exposure control is performed. These are the schematic diagrams of the image data shown about the background subtraction method. These are the schematic diagrams of the image data shown about the interframe difference method. These are the schematic diagrams of the image data shown about a block matching method. These are sequence diagrams which show an example of the tracking process of the imaging device in the embodiment of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 2 Image sensor 3 Lens part 4 AFE
5 Microphone 6 Image Processing Unit 7 Audio Processing Unit 8 Compression Processing Unit 9 Driver Unit 10 Decompression Processing Unit 11 Image Output Circuit Unit 12 Audio Output Circuit Unit 13 CPU
DESCRIPTION OF SYMBOLS 14 Tracking process part 14a Photometry part 14b Exposure control part 14c Brightness correction part 14d Tracking area setting part 15 Memory 16 Operation part 17 TG part 18 Bus 19 Bus 20 External memory 21 Cursor 22a Selection cursor 22b Boundary 23 Tracking area 24 Search area 24a Initial search area 25 Peripheral area 26, 26a Photometric area 27a, 27b Area including moving object

Claims (9)

An imaging unit that creates image data from the input optical image;
In an imaging apparatus comprising: a tracking area setting unit that sets a tracking area indicating a position of a tracking target in image data based on input image data;
A storage unit for storing a tracking target color signal which is a signal indicating the tracking target;
A brightness correction unit for correcting the brightness of the image data input to the tracking region setting unit,
The brightness correction unit corrects the brightness in a predetermined area of the image data input to the tracking area setting unit to be close to the brightness of the tracking target color signal, and
The imaging apparatus, wherein the tracking area setting unit sets the tracking area based on a color signal included in the image data corrected by the brightness correction unit and the tracking target color signal. The total photometric value is calculated based on the result of measuring the overall brightness of the image data, and the target photometry is based on the result of measuring the brightness of the first photometric area that is the area including the tracking area of the image data. A metering unit for calculating a value;
A weighted addition value is obtained by multiplying each of the overall photometry value and the target photometry value by a predetermined addition ratio, and the imaging unit is controlled based on the weighted addition value, thereby performing the imaging An exposure control unit that controls the brightness of image data output from the unit;
The imaging apparatus according to claim 1, further comprising:   When the target photometric value is outside the first range, the addition ratio multiplied by the target photometric value in the weighted addition value is greater than when the target photometric value is within the first range. The imaging device according to claim 2, wherein the imaging device is set to a value of the height.   When the target photometry value continues to be a value within the first range, an addition ratio to be multiplied by the target photometry value in the weighted addition value is set to a normal exposure correction ratio value. The imaging device according to claim 3.   When the target photometric value changes from a value outside the first range to a value within the first range, and subsequently becomes a value within the first range, the target photometric value is changed to the target photometric value in the weighted addition value. The imaging apparatus according to claim 4, wherein the addition ratio to be multiplied is set and maintained at the normal exposure correction ratio value after being set to a value other than the normal correction ratio value at least once.   The said 1st photometry area | region is set larger than the case where the said object photometry value exists in the said 2nd range when the said object photometry value becomes a value outside a 2nd range. The imaging device according to claim 5.   The target metering value includes a predetermined addition ratio for each of a value obtained by measuring the brightness of the first photometric area and a value obtained by measuring the brightness of a peripheral area that is a peripheral area of the first photometric area. The imaging device according to claim 2, wherein the imaging device is calculated based on a value obtained by multiplication and addition. A recording unit for recording input image data;
The image pickup apparatus according to claim 1, wherein the image data input to the recording unit is image data that has not been corrected by the brightness correction unit. A first step of storing a tracking target color signal that is a signal indicating the tracking target;
A second step of creating image data;
A third step of controlling the brightness of a predetermined area of the image data created in the second step to approach the tracking target color signal stored in the first step;
Based on the color signal included in the image data corrected in the third step and the tracking target color signal stored in the first step, a tracking area indicating the position of the tracking target in the image data is set. The fourth step;
An imaging method comprising: