JP4814380B2 - Imaging apparatus, integrated circuit, and imaging method - Google Patents

Description

  The present invention relates to a technique for encoding an image taken at a high speed in an environment with flicker.

  In moving picture coding, in the MPEG (Moving Picture Experts Group) method, for example, the absolute value difference of luminance in a rectangular area of a specific size called a macroblock is used as an evaluation function for motion search. For example, when the size of the macroblock is l × m, the absolute value difference SAD is expressed by the following formula (1).

However, || is a symbol representing an absolute value. Further, f _n (x, y) is the luminance value of the coordinate (x, y) in the image frame (encoding target image frame) of frame number n, and f _{n−n ′} (x, y) is the frame number nn ′. Is a luminance value of coordinates (x, y) in the image frame (reference frame).
In the motion search, a combination of n ′, x ′, and y ′ that minimizes the absolute value difference SAD is searched, a reference frame is determined, and a motion vector is detected. There are a method in which n ′ can be arbitrarily specified as in MPEG-4 AVC (H.264) and a method in which n ′ is fixed to 1 as in MPEG-4 Simple Profile.

If the luminance value of the light source at the time of imaging is different between the image frame of frame number n and the image frame of frame number nn ′, the absolute value difference obtained by Expression (1) is a coincidence regardless of the movement of the original object. It becomes the minimum value near the luminance value, and the compression rate decreases.
As a fluorescent lamp flicker removal technique, a technique for correcting an image by performing secondary or higher-order spectrum analysis from an image taken by an imaging means has been proposed (see, for example, Patent Document 1), and an outline thereof is shown in FIG. I will explain.

  In the imaging apparatus 100, the luminance detection unit 102 measures the luminance in units of lines for an image captured by the imaging unit 101 configured by a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The flicker spectrum detection unit 103 analyzes the frequency component of the luminance value by using Fourier transform, extracts the component of the fluorescent lamp flicker, and estimates the waveform of the fluorescent lamp flicker. Subsequently, the luminance correction coefficient generation unit 104 generates a luminance correction coefficient so as to cancel the fluorescent lamp flicker component based on the estimated fluorescent lamp flicker waveform, and the gain adjustment unit 105 captures an image based on the luminance correction coefficient. Adjust the gain of the image and correct the brightness. Although the imaging apparatus 100 is not intended for high-speed shooting, more accurate luminance correction is performed by predicting a flicker spectrum up to a high frequency component in consideration of a shift in imaging timing for each pixel of the CMOS image sensor. enable.

Also, a moving image coding technique that takes into account the change in luminance of the light source between image frames is proposed to perform motion search after correcting the luminance of the reference frame to be the same as the luminance value of the image frame to be encoded. (See, for example, Patent Document 2), and its outline will be described with reference to FIG.
In the imaging apparatus 200, the image captured by the imaging unit 201 is recorded in the input image frame recording unit 202. The gain detection unit 208 detects the luminance difference between the image frame recorded in the input image frame recording unit 202 and the reference frame recorded in the reference image frame recording unit 207, and the gain adjustment unit 209 is based on the luminance difference. Reference frame brightness correction is performed. The motion search unit 203 performs a motion search between the image frame and the luminance corrected reference frame, and the encoding unit 204 encodes a signal obtained as a result of the motion search and stores the encoded data in the recording unit 205. . In addition, the decoding unit 206 decodes the encoded data, and records the reference frame obtained by the decoding in the reference image frame recording unit 207. The imaging apparatus 200 can maintain a high compression rate even when the luminance value of the image frame changes due to the influence of fluorescent flicker or the like.

  In addition, as what shows the technical level regarding parallelization of moving image encoding, there exists what operates an encoding part in parallel (for example, refer patent document 3), The outline | summary is demonstrated using FIG. In the imaging apparatus 300, an image captured by the imaging unit 301 is distributed to the encoding units 303, 304, and 305 by the distribution unit 302 in units of frames, and is encoded by the encoding units 303, 304, and 305. The encoded data output from the encoding units 303, 304, and 305 are integrated by the synthesis unit 306 and accumulated in the recording unit 307. This technique improves the processing speed of encoding, and is useful for applications that require high encoding calculation capability such as high-speed shooting.

JP 2004-222228 A (page 33, FIG. 4) JP 2001-231045 (page 10, FIG. 1) Japanese Patent Laying-Open No. 2007-202026 (page 14, FIG. 1)

  However, in high-speed shooting, the shooting frame rate may be higher than the frequency of the fluorescent light flicker, and in this case, the image is shot without convolution of the luminance change of the fluorescent light flicker. For this reason, when gain adjustment is performed as in the fluorescent light flicker removal technique, a change in the S / N ratio due to gain adjustment greatly occurs between image frames, and a different S / N ratio for each image frame is a moving image. Encoding efficiency is degraded.

In addition, in the above-described moving image coding technique considering the luminance change of the light source between the image frames, the luminance correction is performed between the image frames. If it is not possible to determine whether it is derived, there is no fluorescent lamp flicker, and if the luminance difference due to the original change of the image is large, the compression rate is lowered.
Therefore, the present invention provides an imaging device, an integrated circuit, and an imaging method capable of efficiently encoding even a video shot of fluorescent lamp flicker and maintaining a conventional compression rate even in an environment without fluorescent lamp flicker. The purpose is to do.

  In order to achieve the above object, an imaging apparatus according to the present invention includes an imaging unit, a detection unit that detects a time waveform of flicker of a light source, and an image frame imaged by the imaging unit based on a detection result by the detection unit. Estimating means for estimating the luminance of the light source at the time of imaging, determining means for determining an encoding parameter used for encoding the image frame based on the estimated luminance value of the light source estimated by the estimating means, and the determining means Encoding means for encoding the image frame on the basis of the encoding parameter determined by.

  The integrated circuit of the present invention estimates the luminance of the light source at the time of imaging of the image frame captured by the imaging means based on the detection means for detecting the time waveform of the flicker of the light source and the detection result by the detection means. An estimation unit; a determination unit that determines an encoding parameter to be used for encoding the image frame based on a luminance estimation value of the light source estimated by the estimation unit; and an encoding parameter determined by the determination unit. Coding means for coding the image frame.

  Furthermore, the imaging method of the present invention includes an imaging procedure, a detection procedure for detecting a time waveform of flicker of a light source, and a light source at the time of imaging of an image frame imaged in the imaging procedure based on a detection result in the detection procedure. An estimation procedure for estimating the luminance of the image frame; a determination procedure for determining an encoding parameter used for encoding the image frame based on the luminance estimation value of the light source estimated in the estimation procedure; and a code determined in the determination procedure An encoding procedure for encoding the image frame based on an encoding parameter.

  According to each of the imaging device, the integrated circuit, and the imaging method, the time waveform of the flicker of the light source is detected to estimate the luminance of the light source at the time of imaging the image frame, and the encoding parameter is determined based on the estimated luminance value of the light source. The determination is performed, and the image frame is encoded based on the determined encoding parameter. For this reason, it is possible to efficiently encode an image obtained by photographing a fluorescent lamp flicker, and to maintain a conventional compression rate even in an environment without the fluorescent lamp flicker.

In the above imaging apparatus, the encoding parameter is a reference frame used for motion search, and the encoding unit is configured to detect a motion vector between the image frame and the reference frame determined as the encoding parameter by the determining unit. Detection may be performed to encode the image frame.
According to this, since the reference frame is determined from the estimated brightness value of the light source, an appropriate reference frame can be used according to the change in the brightness of the light source.

In the imaging apparatus, the determination unit refers to a candidate reference frame having a luminance estimated value closest to a luminance estimated value of a light source at the time of imaging of the image frame from among a plurality of candidate reference frames that are candidate reference frames. The frame may be determined.
According to this, since a candidate reference frame having a luminance estimated value close to the luminance estimated value of the light source at the time of capturing an image frame is used as the reference frame, the image quality can be improved.

In the imaging apparatus, the determination unit may determine a candidate reference frame having the largest estimated luminance value of the light source as a reference frame from among a plurality of candidate reference frames.
According to this, encoding efficiency can be improved by using a candidate reference frame having a high S / N ratio as a reference frame.
In the above imaging apparatus, the encoding parameter is a coefficient for correcting a luminance value for each of the image frame and the reference frame of the evaluation function used for motion search, and the encoding means is encoded by the determining means. The image frame may be encoded by performing a motion search using the coefficients for the image frame and the reference frame determined as parameters.

According to this, since the coefficient of the evaluation function is determined based on the estimated luminance value of the light source, the motion vector can be accurately detected even if the luminance of the light source changes.
In the imaging apparatus, the encoding unit includes a plurality of sub-encoding units that perform encoding in parallel and the plurality of sub-encodings of the image frame based on the encoding parameter determined by the determination unit. Distribution means for outputting to any one of the means, and the determination means determines, as an encoding parameter, a sub-encoding means for encoding the image frame from among the plurality of sub-encoding means. You may do it.

  According to this, encoding can be efficiently performed in parallel even in a situation where there is a change in the luminance of the light source.

1 is a configuration diagram of an imaging apparatus according to a first embodiment. FIG. The block diagram of the encoding part of FIG. 3 is a flowchart showing a procedure of image processing by the imaging apparatus of FIG. The block diagram of the imaging device of 2nd Embodiment. The block diagram of the encoding part of FIG. 5 is a flowchart showing a procedure of image processing by the imaging apparatus of FIG. The block diagram of the imaging device of 3rd Embodiment. The block diagram of the encoding part of FIG. 8 is a flowchart showing a procedure of image processing by the imaging apparatus of FIG. The block diagram of the imaging device which has the conventional flicker removal function. The block diagram of the imaging device which has the function of the moving image encoding which considered the luminance change of the light source between the conventional image frames. The block diagram of the imaging device which performs the conventional moving image encoding in parallel.

  Embodiments of the present invention will be described below with reference to the drawings. However, the “encoding parameter” in the present document means items that can be set or selected when performing the encoding process. Note that image correction processing such as gain adjustment substantially changes the image to be encoded, and the image correction parameter that substantially changes the image to be encoded is the “encoding parameter” in this document. Not included.

<< First Embodiment >>
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. However, in the present embodiment, the encoding parameters are reference frames used for motion search, and coefficients G _n and G _{n−n ′} in an evaluation function of equation (2) described later used for motion search. In each of the first to third embodiments, it is assumed that the area where the imaging apparatus is used is two areas, that is, an area having a power frequency of 50 Hz and an area having a power frequency of 60 Hz.

<Configuration of imaging device>
The configuration of the imaging apparatus according to the present embodiment will be described with reference to FIG. FIG. 1 is a configuration diagram of an imaging apparatus according to the present embodiment.
The imaging apparatus 1 includes a CCD (Charge Coupled Device) camera module 2, an image processing unit 3, a DRAM (Dynamic Random Access Memory) 4, and a memory card 5. The imaging device 1 has a high-speed shooting function and the like, and performs shooting at a set frame rate. Note that the frame rate set in the imaging device 1 may be higher than the fluorescent lamp flicker frequency.

The CCD camera module 2 alternately outputs 8-bit luminance value (Y) and hue value (Cb, Cr) data relating to the captured image to an image input unit 14 (to be described later) of the image processing unit 3.
The image processing unit 3 is for encoding an image input from the CCD camera module 2, and details thereof will be described later.

  In the input frame storage area 6 of the DRAM 4, the luminance value (Y) and hue value (Cb, Cr) data of each image frame constituting the image captured by the CCD camera module 2 is stored in the image input unit of the image processing unit 3. 14 is stored. In the reference frame storage areas 7 a to 7 d of the DRAM 4, image frames obtained when the image frame is encoded by the image processing unit 3 are stored as reference frame candidates. The image frames stored in the reference frame storage areas 7a to 7d are referred to as “candidate reference frames”. However, in this embodiment, as will be described later, four candidate reference frames are stored in the reference frame storage areas 7a to 7d frame by frame.

The memory card 5 stores encoded data obtained by encoding an image by the image processing unit 3.
[Configuration of Image Processing Unit 3]
As shown in FIG. 1, the image processing unit 3 includes a processor 11, a DRAM control unit 12, a memory card control unit 13, an image input unit 14, a frame luminance detection unit 15, a flicker spectrum detection unit 16, A luminance estimation unit 17, an encoding unit 18, and an internal bus 19 are provided. The processor 11, DRAM control unit 12, memory card control unit 13, image input unit 14, and encoding unit 18 are each connected to an internal bus 19.

For example, the processor 11 controls the activation of each unit in the image processing unit 3 via the internal bus 19. The DRAM control unit 12 writes data to the DRAM 4 and reads data from the DRAM 4. The memory card control unit 13 writes data to the memory card 5 and reads data from the memory card 5.
The image input unit 14 outputs the luminance value (Y) data input from the CCD camera module 2 to the frame luminance detection unit 15, and the luminance value (Y) data and hue value input from the CCD camera module 2. The data (Cb, Cr) is output to the DRAM controller 12 via the internal bus 19. An image frame composed of the luminance value (Y) data and the hue value (Cb, Cr) data is stored in the input frame storage area 6 by the DRAM controller 12.

The frame luminance detection unit 15 has a 32-bit counter. The frame luminance detection unit 15 integrates the luminance values of pixels in one frame using a counter in units of frames, and detects a flicker spectrum using a value obtained as a result of the integration (hereinafter referred to as “frame luminance value”). To the unit 16.
The flicker spectrum detection unit 16 has 512 32-bit buffers for storing the frame luminance values input from the frame luminance detection unit 15, and can store frame luminance values for 512 frames. ing.

  When a new frame luminance value is input from the frame luminance detection unit 15, the flicker spectrum detection unit 16 replaces the oldest frame luminance value stored in the buffer with the input new frame luminance value. The flicker spectrum detection unit 16 samples 512 frame luminance values stored in 512 buffers in time series, and performs a fast Fourier transform (hereinafter referred to as “FFT”) to obtain 512 pieces. A 32-bit spectral value is obtained.

Subsequently, the flicker spectrum detection unit 16 performs the following processing to determine whether there is a fluorescent light flicker component derived from the power supply frequency 50 Hz or whether there is a fluorescent light flicker component derived from the power supply frequency 60 Hz. .
The flicker spectrum detector 16 compares the spectrum value of 100 Hz with the spectrum value of 120 Hz.

When the spectral value of 100 Hz is larger than the spectral value of 120 Hz, the flicker spectrum detection unit 16 compares the spectral value of 100 Hz with the threshold value, and when the spectral value of 100 Hz exceeds the threshold value, the fluorescent lamp flicker derived from the power supply frequency 50 Hz. If it does not exceed, it is determined that there is no fluorescent light flicker component.
On the other hand, when the 100 Hz spectrum value is not larger than the 120 Hz spectrum value, the flicker spectrum detection unit 16 compares the 120 Hz spectrum value with the threshold value, and when the 120 Hz spectrum value exceeds the threshold value, the flicker spectrum detection unit 16 is derived from the power supply frequency 60 Hz. It is determined that there is a fluorescent light flicker component, and if it does not exceed, it is determined that there is no fluorescent light flicker component.

The flicker spectrum detection unit 16 outputs the obtained spectrum value and flicker information indicating the determination result to the luminance estimation unit 17.
However, the above threshold value is determined using, for example, imaging in advance in an environment where there is fluorescent flicker and an environment where there is no fluorescent flicker, considering that the characteristics differ depending on the sensor, for example. It is preferable.

The luminance estimation unit 17 performs the following processing according to the content of the flicker information input from the flicker spectrum detection unit 16 to estimate the luminance of the light source when the image frame is captured.
When the flicker information indicates that there is a fluorescent flicker component derived from the power supply frequency of 50 Hz, the luminance estimation unit 17 selects 100 Hz, 200 Hz, multiples of 100 Hz from the spectrum values input from the flicker spectrum detection unit 16. The spectrum value of each frequency of 300 Hz,. Then, the luminance estimation unit 17 performs inverse fast Fourier transform (hereinafter referred to as “IFFT”) using the extracted spectral values of each frequency to obtain luminance values for 512 frames, and obtains the obtained luminance values. It outputs to the motion search part 31 mentioned later of the encoding part 18 as a luminance estimated value of a light source.

  Further, when the flicker information indicates that there is a fluorescent flicker component derived from the power supply frequency 60 Hz, the luminance estimation unit 17 selects 120 Hz, which is a multiple of 120 Hz, from the spectrum value input from the flicker spectrum detection unit 16. A spectral value of each frequency of 240 Hz, 360 Hz,. Then, the luminance estimation unit 17 performs IFFT using the extracted spectral values of each frequency to obtain luminance values for 512 frames, and uses the obtained luminance values as luminance estimation values of the light source to perform motion search of the encoding unit 18. To the unit 31.

By performing the above processing, it is possible to exclude frequency components not related to the fluorescent lamp flicker and to estimate only the luminance change derived from the fluorescent lamp flicker for 512 frames.
Further, when the flicker information indicates that there is no fluorescent light flicker component, the luminance estimation unit 17 outputs a predetermined fixed value to the motion search unit 31 of the encoding unit 18 as the luminance estimation value of the light source.

Since fluorescent lamp flicker has periodicity, it is expected that a continuous image frame will show the same pattern. Therefore, FFT calculation and IFFT calculation may be performed every image frame, or FFT calculation and IFFT calculation are performed. You may thin out at regular intervals.
The encoding unit 18 encodes an image frame imaged by the CCD camera module 2 and stored in the input frame storage area 6 using the luminance estimation value of the light source input from the luminance estimation unit 17. is there. In the present embodiment, it is assumed that the encoding unit 18 performs moving image encoding according to MPEG-4 AVC.

(Configuration of encoding unit 18)
The encoding unit 18 in FIG. 1 will be described with reference to FIG. FIG. 2 is a block diagram of the encoding unit 18 of FIG. In FIG. 2, for simplification of the drawing, the internal bus 19 and the DRAM control unit 12 between the encoding unit 18 and the DRAM 4 are omitted, and the internal unit between the encoding unit 18 and the memory card 5 is omitted. The bus 19 and the memory card control unit 13 are omitted.

The encoding unit 18 includes a motion search unit 31, a difference unit 32, a DCT unit 33, a quantization unit 34, a variable length encoding unit 35, an inverse quantization unit 36, and an IDCT unit 37.
However, in moving picture coding in MPEG-4 AVC, it is possible to use another picture frame as a reference frame. Therefore, in the present embodiment, candidate reference frames for the past four frames are always accumulated in the reference frame storage areas 7a to 7d in a ring buffer shape so that they can be used as reference frames. In the case of the ring buffer structure, if the frame number is counted from the start of encoding, the memory location can be uniquely identified from the frame number.

In the following, for the sake of convenience of explanation, the encoding unit 18 has completed encoding up to the image frame of frame number n−1, and a case where the image frame of frame number n is encoded will be described as an example.
The image frame of frame number n stored in the input frame storage area 6 is read by the DRAM control unit 12 and input to the motion search unit 31 and the difference unit 32 via the internal bus 19.

Motion estimation unit 31 calculates the reciprocal of the luminance estimation value Y _n of the luminance at the time of capturing the image frame of the frame number n inputted from the luminance estimation unit 17, a frame the reciprocal 1 / Y _n of the luminance estimation value Y _n Accumulated as the luminance correction coefficient G _n of the image frame of number n. However, the motion search unit 31 encodes the image frame with the frame number n-i (i = 1, 2, 3, 4), and the estimated luminance value Y when the image frame with the frame number n-i is captured. By calculating the reciprocal of _ni , the luminance correction coefficient G _ni (= 1 / Y _ni ) of the image frame of frame number n−i is calculated, and the calculated luminance correction coefficient G _ni is accumulated. The brightness correction coefficient G _ni is the brightness correction coefficient of the candidate reference frame with the frame number _ni (i = 1, 2, 3, 4).

Subsequently, the motion search unit 31 refers to the candidate reference frame of the luminance correction coefficient closest to the luminance correction coefficient G _n from among the candidate reference frames for the past four frames stored in the reference frame storage areas 7a to 7d. Decide on a frame. Then, the motion search unit 31 acquires candidate reference frames determined as reference frames from the reference frame storage areas 7 a to 7 d via the internal bus 19 and the DRAM control unit 12. Hereinafter, for convenience of explanation, it is assumed that the candidate reference frame determined as the reference frame is the candidate reference frame with the frame number nn ′, and the luminance correction coefficient of the candidate reference frame with the frame number _{nn ′} is described as G _{n−n ′.} To do.

  Subsequently, the motion search unit 31 uses the candidate reference frame with the frame number nn ′ determined as the reference frame for each macroblock whose size of the rectangular area is l × m, and uses the following formula (2 The motion vector is detected by searching for x ′ and y ′ that minimize the absolute value difference SAD. The motion search unit 31 performs motion compensation using the detected motion vector, and outputs a prediction signal obtained by the motion compensation to the difference unit 32.

However, || is a symbol representing an absolute value. Further, f _n (x, y) is a luminance value at the coordinates (x, y) of the image frame of frame number n to be encoded, and f _{n−n ′} (x, y) is a frame number determined as a reference frame. It is a luminance value at the coordinates (x, y) of the candidate reference frame of nn ′.
The difference unit 32 generates a residual signal by subtracting the prediction signal input from the motion search unit 31 from the image of the frame number n. The DCT unit 33 generates a DCT coefficient by performing a discrete cosine transform (hereinafter referred to as “DCT”) on the residual signal input from the difference unit 32. The quantization unit 34 calculates a quantized value by quantizing the DCT coefficient input from the DCT unit 33. The variable length encoding unit 35 generates encoded data related to the image frame of frame number n by variable length encoding the quantized value input from the quantizing unit 34, and the generated encoded data is transmitted to the internal bus 19. And stored in the memory card 5 via the memory card control unit 13.

  The inverse quantization unit 36 calculates the inverse quantization value by inverse quantization of the quantization value input from the quantization unit 34, and the IDCT unit 37 receives the inverse quantization input from the inverse quantization unit 36. The value is subjected to inverse discrete cosine transform (hereinafter referred to as “IDCT”). By these processes, the image frame of frame number n is decoded. The IDCT unit 37 stores the candidate reference frame with the oldest frame number (here, the candidate reference frame with the frame number n-4) with the decoded image frame with the frame number n as the candidate reference frame with the frame number n. Stored in the reference frame storage area via the internal bus 19 and the DRAM controller 12. As a result, when the image frame with frame number n + 1 is encoded, four reference frame candidates with frame numbers n-3 to n are stored in the reference frame storage areas 7a to 7d.

<Operation of Imaging Device 1>
Hereinafter, the operation of the imaging apparatus 1 of FIG. 1 will be described with reference to FIG. FIG. 3 is a flowchart showing a procedure of image processing by the imaging apparatus 1 in FIG. 1, and shows a flow of processing for one image frame.
An image is picked up by the CCD camera module 2, and data of the luminance value (Y) of the picked-up image is input to the frame luminance detecting unit 15 by the image input unit 14, and the luminance value ( Y) and hue value (Cb, Cr) data are stored in the input frame storage area 6 by the image input unit 14 via the internal bus 19 and the DRAM control unit 12 (step S1).

  The frame luminance detection unit 15 calculates the frame luminance value by integrating the luminance values of the pixels in one frame in units of frames (step S2). The flicker spectrum detection unit 16 obtains 512 spectrum values using the frame luminance values for 512 frames calculated by the frame luminance detection unit 15 (step S3). The flicker spectrum detector 16 has a fluorescent lamp flicker component derived from the power supply frequency 50 Hz, a fluorescent lamp flicker component derived from the power supply frequency 60 Hz, or a fluorescent lamp flicker component based on the calculated spectrum value. It is determined whether there is any (step S4).

  When it is determined that there is a fluorescent lamp flicker component derived from the power supply frequency 50 Hz (S4: power supply 50 Hz), the luminance estimation unit 17 has 100 Hz, 200 Hz, 300 Hz,... Obtained by the flicker spectrum detection unit 16. Using the spectrum value, an estimated luminance value of the light source at the time of imaging 512 image frames is calculated (step S5). Further, when it is determined that there is a fluorescent lamp flicker component derived from the power supply frequency 60 Hz (S4: power supply 60 Hz), the luminance estimation unit 17 determines 120 Hz, 240 Hz, 360 Hz,... Obtained by the flicker spectrum detection unit 16. The estimated luminance value of the light source at the time of imaging each image frame for 512 frames is calculated using the spectrum value (step S6). If it is determined that there is no fluorescent light flicker component (S4: none), the luminance estimation unit 17 determines the fixed value as the luminance estimation value of the light source when the image frame is captured (step S7).

  An image frame to be encoded is input from the input frame storage area 6 to the motion search unit 31 and the difference unit 32 via the DRAM control unit 12 and the internal bus 19. The motion search unit 31 obtains and accumulates the luminance correction coefficient of the image frame as described above from the luminance estimation value of the image frame to be encoded calculated or determined by the luminance estimation unit 17. The motion search unit 31 then selects a luminance correction coefficient candidate closest to the luminance correction coefficient of the image frame to be encoded from among the four candidate reference frames stored in the reference frame storage areas 7a to 7d. The reference image frame is determined as a reference frame (step S8). It should be noted that the luminance correction coefficient of each image frame is calculated and accumulated at the time of encoding each image frame for the past four frames, and each accumulated luminance correction coefficient is referred to each candidate for the past four frames. Used as a frame luminance correction coefficient.

Subsequently, the motion search unit 31 acquires candidate reference frames determined as reference frames from the reference frame storage areas 7a to 7d via the internal bus 19 and the DRAM control unit 12, and uses the acquired candidate reference frames to The motion vector is detected by searching for x ′ and y ′ in which the absolute value difference SAD in Equation (2) is minimized (step S9).
The motion search unit 31 performs motion compensation based on the detected motion vector to generate a prediction signal. Then, the difference unit 32 generates a residual signal by subtracting the prediction signal from the image of the image frame to be encoded, and the DCT unit 33, the quantization unit 34, and the variable length encoding unit 35 encode the residual signal. Data is generated, and the variable length encoding unit 35 stores the generated encoded data in the memory card 5 via the internal bus 19 and the memory card control unit 13. The inverse quantization unit 36 and the IDCT unit 37 decode the image frame to be encoded from the quantized value output from the quantization unit 34, and the IDCT unit 37 stores the candidate reference frame with the oldest frame number. The decoded image frame is stored in the reference frame storage area as a candidate reference frame via the internal bus 19 and the DRAM control unit 12 (step S10).

According to the embodiment described above, since the fluorescent light flicker component is analyzed from the frame luminance values of the image frames for 512 frames and the luminance of the light source is estimated, high estimation accuracy of the luminance of the light source can be obtained. .
In motion search, a candidate reference frame having a luminance estimated value closest to the luminance estimated value of the light source at the time of capturing the image frame to be encoded is determined as a reference frame, and further, determined as an image frame to be encoded and a reference frame. The motion vector is detected while correcting the luminance difference with the candidate reference frame. Thus, efficient coding can be performed while reducing erroneous search for motion vectors due to changes in the luminance of the light source during shooting.

Since the image frame is encoded without correcting the luminance of the image frame itself, in the case of high-speed shooting, the encoded data can include information on the phenomenon of the fluorescent lamp flicker itself, and the phenomenon of the fluorescent lamp flicker itself It can also be applied to applications for which imaging is performed.
Unlike the prior art described with reference to FIG. 11, it is not necessary to include a gain value in encoded data, so that compatibility with other encoders can be achieved.

<< Second Embodiment >>
The second embodiment of the present invention will be described below with reference to the drawings. However, in the present embodiment, the encoding parameter is a reference frame used for motion search.
The encoding unit 18 according to the first embodiment determines a candidate reference frame with a luminance correction coefficient closest to the luminance correction coefficient of the image frame to be encoded among the reference reference frames for the past four frames as a reference frame, A motion vector is detected using the uncorrected luminance value and luminance correction coefficient. On the other hand, the encoding unit 18a of the present embodiment determines a candidate reference frame having the smallest luminance correction coefficient among the candidate reference frames for the past four frames as a reference frame, and uses the corrected luminance value. The motion vector used is detected.

<Configuration of imaging device>
The configuration of the imaging apparatus according to the present embodiment will be described with reference to FIG. FIG. 4 is a configuration diagram of the imaging apparatus according to the present embodiment. However, in the present embodiment, components that are substantially the same as those in the first embodiment are denoted by the same reference numerals, and the description thereof can be applied.

The imaging device 1a includes a CCD camera module 2, an image processing unit 3a, a DRAM 4, and a memory card 5.
The image processing unit 3a is for encoding an image input from the CCD camera module 2, and the image processing unit 3a will be described below.
[Configuration of Image Processing Unit 3a]
As shown in FIG. 4, the image processing unit 3a includes a processor 11, a DRAM control unit 12, a memory card control unit 13, an image input unit 14, a frame luminance detection unit 15, a flicker spectrum detection unit 16, The brightness estimation unit 17, the gain adjustment unit 51, the encoding unit 18 a, and the internal bus 19 are provided. The processor 11, DRAM control unit 12, memory card control unit 13, gain adjustment unit 51, and encoding unit 18 a are each connected to the internal bus 19. The image input unit 14 outputs the image luminance value (Y) data to the frame luminance detection unit 15, and the image luminance value (Y) and hue value (Cr, Cb) data to the gain adjustment unit 51. Output. In addition, the luminance estimation unit 17 outputs the luminance estimation value of the light source at the time of capturing an image frame to the gain adjustment unit 51 and a motion search unit 31a described later of the encoding unit 18a.

For convenience of description of the gain adjustment unit 51, the image frame with the frame number n will be described as an object. However, the gain adjustment unit 51 performs the same processing for an image frame with a frame number other than the frame number n. The gain adjusting unit 51 calculates the reciprocal of the luminance estimated value Y _n of the image frame with the frame number n input from the luminance estimating unit 17, and inputs the reciprocal 1 / Y _n of the calculated luminance estimated value from the image input unit 14. The luminance value (Y) is corrected by multiplying the luminance value (Y) constituting the image frame of the frame number n. Then, the gain adjustment unit 51 outputs the luminance value (Y) and the hue value (Cb, Cr) subjected to the luminance correction constituting the image frame with the frame number n to the DRAM control unit 12 via the internal bus 19. . The luminance value (Y) and hue value (Cb, Cr) subjected to luminance correction constituting the image frame of frame number n are stored in the input frame storage area 6 by the DRAM control unit 12. Thus, unlike the first embodiment, the image frame stored in the input frame storage area 6 in the present embodiment is an image frame whose luminance value (Y) has been corrected for luminance.

The encoding unit 18a encodes an image frame imaged by the CCD camera module 2 and stored in the input frame storage area 6, using the luminance estimation value of the light source input from the luminance estimation unit 17. is there. In the present embodiment, it is assumed that the encoding unit 18a performs moving image encoding according to MPEG-4 AVC.
(Configuration of Encoder 18a)
The encoder 18a of FIG. 4 will be described with reference to FIG. FIG. 5 is a block diagram of the encoding unit 18a of FIG. In FIG. 5, for simplification of the drawing, the internal bus 19 and the DRAM control unit 12 between the encoding unit 18a and the DRAM 4 are omitted, and the internal unit between the encoding unit 18a and the memory card 5 is omitted. The bus 19 and the memory card control unit 13 are omitted.

  The encoding unit 18a includes a motion search unit 31a, a difference unit 32, a DCT unit 33, a quantization unit 34, a variable length encoding unit 35, an inverse quantization unit 36, and an IDCT unit 37. However, also in the present embodiment, as in the first embodiment, candidate reference frames for the past four frames are always accumulated in the reference frame storage areas 7a to 7d in a ring buffer form, and these are stored in the reference frames. The configuration can be used.

In the following, for the sake of convenience of explanation, the encoding unit 18a has described the case where the encoding has been completed up to the image frame having the frame number n−1 and the image frame having the frame number n is encoded.
Motion search unit 31a calculates the reciprocal of the luminance estimation value Y _n at the time of capturing the image frame of the frame number n inputted from the luminance estimation unit 17, the reciprocal 1 / Y _n the frame number n of the luminance estimation value Y _n Is stored as the luminance correction coefficient G _n of the image frame. However, the motion search unit 31a, when encoding the image frame with the frame number ni (i = 1, 2, 3, 4), estimates the luminance value Y when the image frame with the frame number ni is captured. By calculating the reciprocal of _ni , the luminance correction coefficient G _ni (= 1 / Y _ni ) of the image frame of frame number n−i is calculated, and the calculated luminance correction coefficient G _ni is accumulated. The brightness correction coefficient G _ni is the brightness correction coefficient of the candidate reference frame with the frame number _ni (i = 1, 2, 3, 4).

  Subsequently, the motion search unit 31a determines the candidate reference frame having the smallest luminance correction coefficient as the reference frame from among the candidate reference frames for the past four frames stored in the reference frame storage areas 7a to 7d. Then, the motion search unit 31a acquires the candidate reference frames determined as reference frames from the reference frame storage areas 7a to 7d via the internal bus 19 and the DRAM control unit 12. Hereinafter, for convenience of explanation, it is assumed that the candidate reference frame determined as the reference frame is a candidate reference frame of frame number n-n ′.

  Subsequently, the motion search unit 31a uses the candidate reference frame with the frame number nn ′ determined as the reference frame for each macroblock whose size of the rectangular area is l × m, and uses the following formula (3 The motion vector is detected by searching for x ′ and y ′ that minimize the absolute value difference SAD. Then, the motion search unit 31a performs motion compensation using the detected motion vector, and outputs a prediction signal obtained by the motion compensation to the difference unit 32.

However, || is a symbol representing an absolute value. Further, f _n (x, y) is a luminance value at the coordinates (x, y) of the image frame of frame number n to be encoded, and f _{n−n ′} (x, y) is a frame number determined as a reference frame. It is a luminance value at the coordinates (x, y) of the candidate reference frame of nn ′.
Note that the same evaluation function as that in the conventional example is used based on the fact that the luminance adjustment is performed by the gain adjustment unit 51.

<Operation of Imaging Device 1a>
Hereinafter, the operation of the imaging apparatus 1a of FIG. 4 will be described with reference to FIG. FIG. 6 is a flowchart showing a procedure of image processing by the imaging apparatus 1a of FIG. 4, and shows a flow of processing for one image frame.
An image is picked up by the CCD camera module 2, and data of the luminance value (Y) of the picked-up image is input to the frame luminance detecting unit 15 by the image input unit 14, and the luminance value ( Y) and hue value (Cb, Cr) data are input to the gain adjusting unit 51 (step S31).

  The frame luminance detection unit 15 performs substantially the same processing as step S2 (step S32), and the flicker spectrum detection unit 16 performs substantially the same processing as step S3 (step S33). Subsequently, the luminance estimation unit 17 performs substantially the same processing as steps S4 to S7 (steps S34 to S37). The gain adjustment unit 51 calculates the reciprocal of the luminance estimation value calculated or determined by the luminance estimation unit 17 and calculates the luminance value (Y) of the image frame having the same frame number input from the image input unit 14. The luminance value (Y) is corrected by multiplying the reciprocal of the estimated value. The gain adjusting unit 51 uses the luminance value (Y) and hue value (Cb, Cr) subjected to luminance correction as image frame data of the frame number as an input frame via the internal bus 19 and the DRAM control unit 12. Store in the storage area 6 (step S38).

  An image frame to be encoded is input from the input frame storage area 6 to the motion search unit 31 a and the difference unit 32 via the DRAM control unit 12 and the internal bus 19. The motion search unit 31a obtains and accumulates the luminance correction coefficient of the image frame as described above from the luminance estimation value of the encoding target image frame calculated or determined by the luminance estimation unit 17. Then, the motion search unit 31a determines the candidate reference frame having the smallest luminance correction coefficient as the reference frame from the candidate reference frames for the past four frames (step S39). It should be noted that the luminance correction coefficient of each image frame is calculated and accumulated at the time of encoding each image frame for the past four frames, and each accumulated luminance correction coefficient is referred to each candidate for the past four frames. Used as a frame luminance correction coefficient.

Subsequently, the motion search unit 31a acquires candidate reference frames determined as reference frames from the reference frame storage areas 7a to 7d via the internal bus 19 and the DRAM control unit 12, and uses the acquired candidate reference frames to The motion vector is detected by searching for x ′ and y ′ in which the absolute value difference SAD in Equation (3) is minimized (step S40).
The motion search unit 31a performs motion compensation based on the detected motion vector to generate a prediction signal. Then, the difference unit 32, the DCT unit 33, the quantization unit 34, and the variable length coding unit 35 perform an encoding process, and the inverse quantization unit 36 and the IDCT unit 37 perform a decoding process (step S41).

According to the above-described embodiment, the candidate reference frame used as the reference frame has the highest luminance of the light source at the time of shooting, that is, the high quality with the highest S / N ratio. Encoding can be performed.
<< Third Embodiment >>
The third embodiment of the present invention will be described below with reference to the drawings. However, in the present embodiment, the encoding parameter is an encoding unit that encodes an image frame to be encoded.

The imaging devices 1 and 1a according to the first and second embodiments perform encoding using one encoding unit. In contrast, the imaging device 1b according to the third embodiment performs encoding in parallel using three encoding units.
<Configuration of imaging device>
The configuration of the imaging apparatus according to the present embodiment will be described with reference to FIG. FIG. 7 is a configuration diagram of the imaging apparatus of the present embodiment. However, in the present embodiment, components that are substantially the same as those in the first embodiment are denoted by the same reference numerals, and the description thereof can be applied.

The imaging device 1b includes a CCD camera module 2, an image processing unit 3b, a DRAM 4b, and a memory card 5.
The DRAM 4b has reference frame storage areas 72a, 73a, and 74a corresponding to the encoding units 72, 73, and 74 in the image processing unit 3b. The reference frame storage areas 72a, 73a, and 74a each have one frame. Reference frames are stored.

The image processing unit 3b is for encoding an image input from the CCD camera module 2, and the image processing unit 3b will be described below.
[Configuration of Image Processing Unit 3b]
As shown in FIG. 7, the image processing unit 3b includes a processor 11, a DRAM control unit 12, a memory card control unit 13, an image input unit 14, a frame luminance detection unit 15, a flicker spectrum detection unit 16, A luminance estimation unit 17, a distribution unit 71, encoding units 72, 73 and 74, a synthesis unit 75, and an internal bus 19 are provided. The processor 11, the DRAM control unit 12, the memory card control unit 13, the image input unit 14, the distribution unit 71, the encoding units 72, 73 and 74, and the synthesis unit 75 are each connected to the internal bus 19. Note that the luminance estimation unit 17 outputs the luminance estimation value of the light source to the distribution unit 71.

For each of the encoding units 72 to 74, the distribution unit 71 encodes the luminance estimation value of the light source at the time of imaging of the image frame currently being encoded most recently if the encoding is not currently performed. The luminance estimation value of the light source at the time of capturing the image frame is stored.
The distribution unit 71 receives the luminance estimation value of the light source when the image frame to be encoded is captured from the luminance estimation unit 17. The distribution unit 71, among the encoding units 72 to 74 that have not performed the encoding process, is the light source at the time of imaging of the image frame that is most recently encoded to the luminance estimation value of the light source at the time of imaging of the image frame to be encoded. Is determined as the encoding unit that encodes the image frame to be encoded. Then, the distribution unit 71 acquires the encoding target image frame from the input frame storage area 6 via the internal bus 19 and the DRAM control unit 12, and acquires the acquired encoding target image frame and the encoding target image frame. The estimated luminance value of the light source at the time of imaging is output to the determined encoding unit. At the same time, the distribution unit 71 updates the estimated luminance value of the light source corresponding to the determined encoding unit to the estimated luminance value of the light source at the time of imaging the image frame to be encoded.

The encoding units 72, 73, and 74 encode the image frames captured by the CCD camera module 2 and input from the distribution unit 71, and output encoded data to the combining unit 75. Details thereof will be described later. In the present embodiment, the encoding units 72, 73, and 74 perform moving image encoding according to MPEG-4 AVC.
The synthesizer 75 concatenates the encoded data input from the encoders 72, 73, and 74 into one encoded data in the order of frame numbers, and controls the concatenated encoded data via the internal bus 19 for memory card control. To the unit 13. The linked encoded data is stored in the memory card 5 by the memory card control unit 13.

(Configuration of encoding units 72, 73, 74)
The encoding unit 72 in FIG. 7 will be described with reference to FIG. FIG. 8 is a block diagram of the encoding unit 72 of FIG. In FIG. 8, the internal bus 19 and the DRAM control unit 12 between the encoding unit 72 and the DRAM 4 are omitted for simplification of the drawing. However, in this embodiment, an encoding unit that performs substantially the same configuration and the same operation as the encoding unit 72 is used as the encoding units 73 and 74. In this case, the description of the encoding unit 72 is applied. Therefore, the description of the encoding units 73 and 74 is omitted in this embodiment.

The encoding unit 72 includes a motion search unit 31 b, a difference unit 32, a DCT unit 33, a quantization unit 34, a variable length encoding unit 35, an inverse quantization unit 36, and an IDCT unit 37.
In the following, it is assumed that the frame number of the image frame that the encoding unit 72 will perform encoding from now on is n, and the frame number of the image frame that the encoding unit 72 has encoded most recently is nn ′.
Motion search unit 31b calculates the reciprocal of the luminance estimation value Y _n of the light source at the time of capturing the image frame of the frame number n inputted from the distribution unit 71, the frame number the reciprocal 1 / Y _n of the luminance estimation value Y _n Accumulated as luminance correction coefficient G _n of n image frames. Note that the motion search unit 31b uses the estimated luminance value Y _{n-n ′} of the light source at the time of imaging the image frame with the frame number nn ′ when the encoding unit 72 encodes the image frame with the frame number _{nn ′} . By calculating the reciprocal, the luminance correction coefficient G _{n-n ′} (= 1 / Y _{n-n ′} ) of the frame number nn _′ is calculated, and the calculated luminance correction coefficient G _{n-n ′} is accumulated. The luminance correction coefficient G _{n-n ′} is the luminance correction coefficient of the reference frame with the frame number nn ′.

  Then, the motion search unit 31b acquires the reference frame having the frame number n-n ′ from the reference frame storage area 72a via the DRAM control unit 12 and the internal bus 19. Subsequently, the motion search unit 31b uses the reference frame of the acquired frame number nn ′ for each macroblock whose rectangular area size is l × m, and calculates the absolute value difference SAD of the above equation (2). The motion vector is detected by searching for x ′ and y ′ that minimizes. Then, the motion search unit 31a performs motion compensation using the detected motion vector, and outputs a prediction signal obtained by the motion compensation to the difference unit 32.

In the present embodiment, the variable length encoding unit 35 outputs the encoded data to the synthesizing unit 35, and the IDCT unit 37 uses the decoded image frame of frame number n as a reference frame in the reference frame storage area 72a. Store.
<Operation of Imaging Device 1b>
Hereinafter, the operation of the imaging apparatus 1b of FIG. 7 will be described with reference to FIG. FIG. 9 is a flowchart showing a procedure of image processing by the imaging apparatus 1b of FIG. 7, and shows a flow of processing for one image frame.

The CCD camera module 2 and the image input unit 14 perform substantially the same processing as step S1 (step S51).
The frame luminance detection unit 15 performs substantially the same process as step S2 (step S52), and the flicker spectrum detection unit 16 performs substantially the same process as step S3 (step S53). Subsequently, the luminance estimation unit 17 performs substantially the same processing as steps S4 to S7 (steps S54 to S57).

  The distribution unit 71 determines the encoding unit that encodes the image frame from the encoding units 72 to 74 as described above (step S58). Then, the distribution unit 71 acquires the encoding target image frame from the input frame storage area 6 via the internal bus 19 and the DRAM control unit 12, and acquires the acquired encoding target image frame and the encoding target image frame. The estimated luminance value of the light source at the time of imaging is output to the determined encoding unit (step S59). At this time, the distribution unit 71 updates the estimated luminance value of the light source corresponding to the determined encoding unit to the estimated luminance value of the light source at the time of capturing the image frame to be encoded.

  In the encoding unit determined in step S58, the motion search unit 31b obtains the luminance correction coefficient of the image frame as described above from the luminance estimation value of the image frame to be encoded input from the distribution unit 71, and accumulate. Then, the motion search unit 31a uses the reference frame stored in the reference frame storage area to search for x ′ and y ′ that minimize the absolute value difference SAD in the above equation (2), thereby moving the motion vector. Is detected (step S60). Note that the luminance correction coefficient of the reference frame is calculated and stored at the time of encoding the image frame that is the source of the reference frame.

  The motion search unit 31b performs motion compensation based on the detected motion vector to generate a prediction signal. Then, the difference unit 32, the DCT unit 33, the quantization unit 34, and the variable length coding unit 35 perform coding processing, and the variable length coding unit 35 outputs the encoded data to the synthesis unit 75, and the synthesis unit 75. The connected encoded data is concatenated and stored in the memory card 5 via the internal bus 19 and the memory card control unit 13. In addition, the inverse quantization unit 36 and the IDCT unit 37 perform decoding processing, and the IDCT unit 37 uses the decoded image frame as a reference frame and a reference frame corresponding to its own encoding unit via the internal bus 19 and the DRAM control unit 12. Store in the storage area (step S61).

  According to the above-described embodiment, the encoding unit that has encoded the image frame having the luminance estimated value closest to the luminance estimated value of the light source at the time of imaging the image frame to be encoded is the image frame to be encoded. And the motion vector is detected while correcting the luminance difference between the image frame to be encoded and the reference frame. As a result, the coding processing speed is improved by parallelizing the coding units, and efficient coding is performed while reducing erroneous motion vectors due to changes in the luminance of the light source during shooting. be able to.

<Supplement>
The present invention is not limited to the above embodiment, and may be as follows, for example.
(1) In each of the first and third embodiments described above, the motion vector detectors 31 and 31b use the evaluation function of Expression (2) for detecting a motion vector. However, the evaluation function used for detecting the motion vector is not limited to the above equation (2), and for example, the evaluation functions of the following equations (4) and (5) may be used.

  In the second embodiment described above, the motion vector detection unit 31a uses the evaluation function of Expression (3) for detecting a motion vector. However, the evaluation function used for detecting the motion vector is not limited to the above equation (3), and for example, the evaluation functions of the following equations (6) and (7) may be used.

  (2) In each of the first to third embodiments described above, the flicker spectrum detector 16 uses a spectral value of 100 Hz and a spectral value of 120 Hz in order to determine the presence or absence of a fluorescent lamp flicker component. However, it is not limited to this. For example, the flicker spectrum detection unit 16 determines the presence or absence of a fluorescent lamp flicker derived from the power supply frequency of 50 Hz (200 Hz, 300 Hz, ...) The spectral value of the component may be used. Further, the flicker spectrum detector 16 determines the presence / absence of a fluorescent lamp flicker derived from the power supply frequency of 60 Hz (240 Hz, 360 Hz, a frequency (240 Hz), a multiple of that frequency (120 Hz). ...) The spectral value of the component may be used.

(3) In each of the first and second embodiments described above, the candidate reference frames are image frames for the past four frames. However, the present invention is not limited to this. N = 2, 3, 5, 6,...) Image frames.
In the third embodiment, the number of encoding units is three. However, the number of encoding units is not limited to this. For example, the number of encoding units is N (N = 2, 4, 5, ...).

  (4) In each of the first and second embodiments described above, the luminance correction coefficient is used to determine the reference frame. However, the present invention is not limited to this. For example, the reference frame is determined using the frame luminance value. May be determined. In this case, in the first embodiment, the motion search unit 31 refers to the candidate reference of the frame luminance value closest to the frame luminance value at the time of imaging the image frame to be encoded among the candidate reference frames for the past four frames. The frame may be determined as a reference frame. Further, in the second embodiment, the motion search unit 31a may determine a candidate reference frame having the largest frame luminance value as a reference frame among candidate reference frames for the past four frames.

In the third embodiment, the distribution unit 71 determines the distribution destination of the image frame to be encoded using the luminance estimation value of the light source at the time of capturing the image frame. However, the present invention is not limited to this. For example, the distribution unit 71 may use the reciprocal of the estimated luminance value of the light source at the time of imaging the image frame to determine the distribution destination of the image frame to be encoded.
(5) The encoding units 18 and 18a described in the first and second embodiments may be used as the encoding units 72, 73, and 74 in the third embodiment. You may make it use the encoding part of a general structure. In addition, the encoding units 72, 73, and 74 may use an evaluation function that does not include a luminance correction coefficient (for example, equations (3), (6), and (7)) as the evaluation function.

(6) In the first to third embodiments described above, the encoding units 18, 18a, 72, 73, and 74 perform moving image encoding according to MPEG-4 AVC. Any method may be used as long as it is a moving image encoding that uses a past image frame as a reference frame.
(7) In each of the first to third embodiments described above, the luminance of the light source at the time of shooting is estimated by the frame luminance detection unit 15, the flicker spectrum detection unit 16, and the luminance estimation unit 17, but this is not limitative. For example, an illuminance sensor may be provided in the imaging apparatus and the luminance of the light source may be estimated by the illuminance sensor.

(8) In each of the first to third embodiments described above, FFT is used to detect the flicker spectrum value. However, the present invention is not limited to this, and means capable of detecting the flicker spectrum value. If it is.
(9) In the first embodiment, both the determination of the reference frame and the detection of the motion vector are performed using the luminance correction coefficient corresponding to the frame luminance value. However, the present invention is not limited to this. A luminance correction coefficient corresponding to the frame luminance value may be used for only one of the determination of the reference frame and the detection of the motion vector. However, in the case where the luminance correction coefficient is not used, the conventional normal encoding technique can be applied by regarding the luminance correction coefficient as a fixed value of 1. When the luminance correction coefficient is used for only one of them, the coding efficiency is reduced but the cost can be reduced as compared with the case where the luminance correction coefficient is used for both.

(10) In each of the first to third embodiments, the luminance value (Y) and the hue value (Cb, Cr) are used. However, the present invention is not limited to this. For example, RGB data, Bayer data, etc. As long as it includes data correlated in luminance, the techniques of the first to third embodiments can be applied as they are or by performing predetermined conversion.
(11) In the first to third embodiments, the description has been made on the assumption that the CCD camera module 2 calculates the luminance value (Y). However, the present invention is not limited to this. As with a typical imaging device, the image input unit 14 may calculate the luminance value (Y). In this case, although it becomes difficult for the image input unit 14 to support various image sensors, the cost can be reduced.

(12) In each of the first to third embodiments, the CCD camera module 2 is used. However, the present invention is not limited to this. For example, a CMOS image sensor or the like can be used instead of the CCD camera module 2. Other imaging devices may be used.
(13) In the imaging apparatuses 1, 1a, 1b described in the first to third embodiments, a mode for executing the encoding process described in each embodiment and other encoding processes (for example, A mechanism for switching the mode for executing the normal encoding process) and a mode selection button for the user to select the mode may be provided.

In addition, the imaging apparatus 1, 1 a, 1 b executes a mode for executing the encoding process described in each embodiment and other encoding processes (for example, a normal encoding process) according to the imaging frame rate. You may make it provide the mechanism for switching to the mode to perform.
(14) The configurations of the first to third embodiments described above may be realized as an LSI (Large Scale Integration) that is typically an integrated circuit. These may be individually made into one chip, or may be made into one chip so as to include all or part of the configurations of the respective embodiments.

Although referred to as LSI here, it may be called IC (Integrated Circuit), system LSI, super LSI, or ultra LSI depending on the degree of integration.
Further, the method of circuit integration is not limited to LSI's, and implementation using dedicated circuitry or general purpose processors is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  Further, if integrated circuit technology comes out to replace LSI's as a result of the advancement of semiconductor technology or a derivative other technology, it is naturally also possible to carry out function block integration using this technology. There is a possibility of adaptation of biotechnology.

  INDUSTRIAL APPLICABILITY The present invention has a fluorescence flicker spectrum detection function and can be used for an imaging apparatus that performs moving image coding, and is particularly useful for an imaging apparatus such as a digital video camera or a digital camera having a high-speed shooting function. It is. The present invention can also be applied to applications such as LSIs for these imaging devices.

DESCRIPTION OF SYMBOLS 1 Imaging device 2 CCD camera module 3 Image processing unit 4 DRAM
5 memory card 6 input frame storage area 7a to 7d reference frame storage area 11 processor 12 DRAM control unit 13 memory card control unit 14 image input unit 15 frame luminance detection unit 16 flicker spectrum detection unit 17 luminance estimation unit 18 encoding unit 19 inside Bus 31 Motion search unit 32 Difference unit 33 DCT unit 34 Quantization unit 35 Variable length encoding unit 36 Inverse quantization unit 37 IDCT unit

Claims (6)

Imaging means;
Detecting means for detecting a time waveform of flicker of the light source;
Estimating means for estimating the luminance of the light source at the time of imaging the image frame imaged by the imaging means based on the detection result by the detecting means;
Determining means for determining a reference frame from among a plurality of candidate reference frames based on the estimated luminance value of the light source estimated by the estimating means;
Encoding means for encoding an image frame to be encoded based on the reference frame determined by the determining means;
An imaging apparatus comprising: The determining means uses, as a reference frame, a candidate reference frame having a luminance estimated value closest to a luminance estimated value of a light source at the time of imaging the image frame to be encoded among a plurality of candidate reference frames that are candidate reference frames. The imaging device according to claim 1. The imaging apparatus according to claim 1, wherein the determining unit determines a candidate reference frame having the largest estimated luminance value of the light source as a reference frame from among a plurality of candidate reference frames. Imaging means;
Detecting means for detecting a time waveform of flicker of the light source;
Estimating means for estimating the luminance of the light source at the time of imaging the image frame imaged by the imaging means based on the detection result by the detecting means;
A plurality of sub-encoding means for performing encoding in parallel;
A determining unit that determines a sub-encoding unit that encodes an image frame to be encoded from among the plurality of sub-encoding units, based on the estimated luminance value of the light source estimated by the estimating unit;
Distributing means for outputting the image frame to be encoded to the sub-encoding means determined by the determining means;
An imaging apparatus comprising: Detecting means for detecting a time waveform of flicker of the light source;
Estimating means for estimating the luminance of the light source at the time of imaging of the image frame imaged by the imaging means based on the detection result by the detecting means;
Determining means for determining a reference frame from among a plurality of candidate reference frames based on the estimated luminance value of the light source estimated by the estimating means;
Encoding means for encoding an image frame to be encoded based on the reference frame determined by the determining means;
An integrated circuit comprising: Imaging procedure;
A detection procedure for detecting the time waveform of the flicker of the light source;
Based on the detection result in the detection procedure, an estimation procedure for estimating the luminance of the light source at the time of imaging the image frame imaged in the imaging procedure;
A determination procedure for determining a reference frame from among a plurality of candidate reference frames based on the estimated luminance value of the light source estimated in the estimation procedure;
An encoding procedure for encoding an image frame to be encoded based on the reference frame determined in the determination procedure;
An imaging method comprising:

Images