JP2006319784A - Image processing apparatus and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus and imaging apparatus capable of obtaining a good image without increasing jiggling or blurring even in such a condition as an object moves at a high speed without sufficient illuminating light quantity for the object. <P>SOLUTION: A motion vector operator 224 calculates a motion vector for associating an image block of a partial area of an image frame configuring an image signal at a first frame rate with a plurality of image frames based on a motion in the image. A synthesizer 225 synthesizes the plurality of image frames by adding the image block associated by the motion vector with the plurality of image frames to generate one piece of image frame, and outputs the image frame as the image signal at a second frame rate lower than the first frame rate. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an imaging apparatus that reduce blurring and blurring of an image caused by movement of a subject under low illumination conditions.

  In conventional imaging devices, when the illumination light to the subject becomes low illuminance, the image signal output of the imaging device is usually increased by adjusting the gain, slowing down the shutter speed, or opening the aperture. Thus, the intensity of the image signal output, that is, the brightness of the image is secured. In particular, under low illuminance conditions, the ratio of noise is high compared to the intensity of the image signal, and the sensitivity is often insufficient. In the method of ensuring the signal intensity by adjusting the gain, if the gain is adjusted under such an imaging condition, the influence of noise increases.

  Therefore, when photographing under such a low illuminance condition, a method of adjusting the light amount is often mainly used. However, if an attempt is made to increase the amount of charge accumulated in the image sensor by reducing the shutter speed, the image is blurred and blurred when the subject is moving, and the image quality is degraded. Although a method of opening the light amount diaphragm is also conceivable, the image quality is affected in other aspects, such as a wider range of blurring due to the shallow depth of field.

A technique for imaging a subject under a condition with low illuminance is disclosed in Patent Document 1, for example. Patent Document 1 describes a method of acquiring a predetermined output level by controlling an intermittent operation period (electrical storage time) and a gain. In this method, the period of intermittent operation (electrical storage time) is controlled so as to avoid image degradation due to excessive gain of the amplifier. Therefore, since the charge accumulation time becomes long under low illuminance conditions, if there is a moving object in the subject, blurring or blurring occurs in the image.
Japanese Patent Laid-Open No. 7-231403

  As described above, when a sufficient signal intensity, that is, the brightness of an image is secured under a condition where the illuminance of outside light is reduced, the image may be blurred or blurred when the subject is moving. There was a point.

  The present invention has been made in view of the above-described problems, and is good without increasing blur and blur even under conditions where the amount of illumination light of the subject is insufficient and the subject is moving at high speed. An object of the present invention is to provide an image processing apparatus and an imaging apparatus capable of obtaining a clear image.

  The present invention has been made to solve the above problems, and the invention according to claim 1 is directed to an image block that is a partial area of an image frame constituting an image signal having a first frame rate. A motion vector calculating means for calculating a motion vector associated between the plurality of image frames based on the motion of the image frame, and adding the image blocks associated with the motion vector between the plurality of image frames. An image processing apparatus comprising: a frame synthesizing unit that synthesizes image frames to generate one image frame and outputs the image frame as an image signal having a second frame rate lower than the first frame rate. is there.

  According to a second aspect of the present invention, in the image processing apparatus according to the first aspect, the motion vector calculation means includes the image block of the reference image frame that is a reference for synthesis and the image block of the other image frame. A difference between pixel values of the reference image frame and the frame composition means, when the absolute value of the difference is equal to or greater than a predetermined value, the image block of the reference image frame and the same image block of the reference image frame If the absolute value of the difference is less than a predetermined value, adding the image block of the reference image frame and the image block of another image frame that are related by the motion vector Features.

  According to a third aspect of the present invention, in the image processing device according to the first or second aspect, a signal for detecting a luminance level of the image signal of the first frame rate or the image signal of the second frame rate. Level detection means, frame synthesis determination means for determining whether or not to synthesize the image frame based on the detected luminance level, and a plurality of the image frames constituting the image signal of the first frame rate A frame storage means for storing the image frame, and a block setting means for setting the image block in the image frame stored in the frame storage means when the frame composition determination means determines that the image frame is to be combined. It further has these.

  According to a fourth aspect of the present invention, there is provided image pickup means for picking up an image and outputting an image signal having a first frame rate, and an image block being a partial region of an image frame constituting the image signal having the first frame rate. Adding a motion vector calculating means for calculating a motion vector associated between the plurality of image frames based on a motion in the image, and adding the image blocks associated by the motion vector between the plurality of image frames. A frame synthesizing unit that synthesizes the plurality of image frames to generate one image frame and outputs the image frame as an image signal having a second frame rate lower than the first frame rate; Device.

  According to a fifth aspect of the present invention, in the imaging apparatus according to the fourth aspect, the motion vector calculation means calculates the relationship between the image block of the reference image frame used as a reference for synthesis and the image block of the other image frame. A difference between pixel values is calculated, and when the absolute value of the difference is equal to or greater than a predetermined value, the frame composition unit calculates the image block of the reference image frame and the same image block of the reference image frame. When the absolute value of the difference is less than a predetermined value, the image block of the reference image frame associated with the motion vector and the image block of another image frame are added. And

  According to a sixth aspect of the present invention, in the imaging device according to the fourth or fifth aspect, a signal level for detecting a luminance level of the image signal having the first frame rate or the image signal having the second frame rate. Detecting means; frame composition determining means for determining whether or not to synthesize the image frame based on the detected luminance level; and a plurality of the image frames constituting the image signal of the first frame rate. Frame storing means for storing, and block setting means for setting the image block in the image frame stored in the frame storing means when the frame combining determining means determines that the image frames are to be combined. Furthermore, it is characterized by having.

  According to the present invention, the plurality of image frames constituting the image signal having the first frame rate higher than the second frame rate are synthesized by adding the blocks associated by the motion vector. Even under the condition that the amount of illumination light is not sufficient and the subject is moving at high speed, it is possible to obtain an effect that a good image can be obtained without increasing blur and blur.

  The best mode for carrying out the present invention will be described below with reference to the drawings. FIG. 1 shows the configuration of an imaging apparatus according to an embodiment of the present invention. In the figure, the imaging unit 1 has a configuration for performing imaging, and the image processing unit 2 has a configuration for processing an image signal generated by imaging. The imaging unit 1 and the image processing unit 2 may be configured as an integrated device, or may be configured as separate devices. In the imaging unit 1, the imaging lens 11 condenses light from the subject. The light condensed by the imaging lens 11 is imaged in the solid-state imaging device 12 (imaging means). The solid-state imaging device 12 includes an imaging device that performs imaging, converts light collected by the imaging lens 11 into an electrical signal, and outputs an image signal based on the subject image. The solid-state imaging device 12 realizes generation of a high-frame-rate moving image signal by reading pixel signals from the imaging element in parallel.

  The image signal output from the solid-state imaging device 12 is input to the AGC unit 13. The AGC unit 13 amplifies the image signal and outputs it to the A / D conversion unit 14. The A / D converter 14 samples and quantizes the image signal and converts it into digital data. The drive circuit 15 generates various pulses for driving the solid-state imaging device 12 and controls the operation of the solid-state imaging device 12.

  In the image processing unit 2, the video memory 21 stores image data (image frame) for one frame output from the A / D conversion unit 14. The frame composition unit 22 extracts necessary image frames from the storage video memory 24 in accordance with an instruction from the control unit 26 and composes a plurality of image frames. That is, when the illuminance is low, the frame synthesizing unit 22 adds a plurality of image frames as necessary to generate one image frame, and an image signal having a second frame rate lower than the first frame rate. Is output to the camera signal processing unit 23. In other cases, the frame synthesizing unit 22 thins out the image frames generated at the first frame rate, and outputs them to the camera signal processing unit 23 as image frames constituting the image signal at the second frame rate. .

  The camera signal processing unit 23 performs color adjustment and other signal processing on the input image signal and outputs it. The video memory 24 (frame storage means) stores a plurality of image frames constituting the first frame rate image signal. The level detector 25 (signal level detection means) detects the image signal level (such as the average value of the luminance levels of all pixels) of the image frame output from the video memory 21 to the frame synthesis unit 22. The level detector 25 detects the image signal level by detecting the image signal level (luminance level) of the image frame constituting the image signal of the second frame rate output from the camera signal processing unit 23. May be. The control unit 26 controls each configuration described above.

  Next, the internal configuration of the solid-state imaging device 12 will be described. FIG. 2 shows the internal configuration of the image sensor provided in the solid-state imaging device 12. The imaging area of the imaging device is divided into a plurality of areas (divided areas), and a charge signal is taken out from each divided area and output in parallel as a plurality of output signals. The imaging area is divided into four imaging areas I, II, III, and IV. The charges in the imaging region I are read out by the vertical transfer CCD 125 and the horizontal transfer CCD 121 and output as an output a. The charges in the imaging region II are read out by the vertical transfer CCD 126 and the horizontal transfer CCD 122 and output as an output b. The charges in the imaging region III are read out by the vertical transfer CCD 127 and the horizontal transfer CCD 123 and output as an output c. The charges in the imaging region IV are read out by the vertical transfer CCD 128 and the horizontal transfer CCD 124 and output as an output d. Thus, a high frame rate is realized by dividing the imaging region and reading out pixel signals in parallel. Note that an image pickup device including a CMOS sensor may be used in place of the image pickup device including a CCD sensor.

  Here, outputs a to d are obtained, but signals are read from all the pixel regions by all (four) signal outputs. The solid-state imaging device 12 includes a signal processing circuit that buffers the four pixel signals in units of one frame, combines them, and outputs them as an image signal having a first frame rate. When the image signal of the first frame rate is output, the subject is moving because the shutter speed is faster than when the image signal of the second frame rate lower than the first frame rate is output. Even in this case, occurrence of blurring and blurring can be reduced.

  Next, the configuration of the frame synthesizing unit 22 and the method of synthesizing image frames will be described. FIG. 3 shows the configuration of the frame synthesis unit 22. The frame synthesis unit 22 includes a frame synthesis determination unit 221 (frame synthesis determination unit), a video memory operation unit 222, a blocking unit 223 (block setting unit), a motion vector calculation unit 224 (motion vector calculation unit), and a synthesis unit 225 ( Frame synthesis means). Operations of these components are controlled by the control unit 26. The video memory 24 stores image frames to be processed by the video memory operation unit 222 to the synthesis unit 225. The video memory 24 stores image frames for a plurality of past frames as accumulated frames. When an image frame is output from the video memory 21, the level detector 25 detects the image signal level of the image frame, and the image frame is input to the frame composition determination unit 221. The control unit 26 notifies the frame synthesis determination unit 221 of the image signal level detected by the level detector 25.

  The frame synthesis determination unit 221 determines whether or not to perform image frame synthesis using the notified image signal level. The determination is performed, for example, by obtaining an average luminance level of all the pixels of the image frame and comparing it with a prescribed level, or comparing an area occupied by pixels having a luminance level equal to or less than a certain value with a prescribed value. If the average luminance level is equal to or higher than the specified level, or the area is equal to or higher than the specified value, the frame composition determining unit 221 determines that the illuminance is sufficient and determines not to combine the image frames. On the other hand, when the average luminance level is less than the prescribed value or the area is less than the prescribed value, the frame composition determination unit 221 determines that the illuminance is insufficient, and a desired exposure amount is secured. In order to obtain an image frame corresponding to the image frame in this case, it is determined that the image frames are to be combined.

  The frame composition determination unit 221 notifies the determination result to the control unit 26 and outputs the image frame output from the video memory 21 to the video memory operation unit 222. The control unit 26 controls the operations of the video memory operation unit 222 to the combining unit 225 based on the notified determination result. Hereinafter, the operation of each component when it is determined by the frame composition determination unit 221 that image frames are to be combined will be described. The video memory operation unit 222 stores the image frame output from the frame synthesis determination unit 221 in the video memory 24, reads out the reference image frame serving as a reference frame in the synthesis from the video memory 24, and outputs the reference image frame to the blocking unit 223. To do.

  Blocking section 223 to combining section 225 perform processing for combining image frames. As shown in FIG. 4, by synthesizing a predetermined number of image frames, an image signal having a second frame rate is generated from an image signal having a first frame rate. As an example, a case where one image frame is generated by combining three image frames will be described. When image frames are combined at the time of processing the frame a3 in FIG. 4, one frame is generated from the frames a1 to a3. At this time, the frames a1 and a2 are already stored in the video memory 24, and they are read from the video memory 24 and synthesized with the frame a3. Similarly, one frame is generated from each of the frames a4 to a6 and the frames a7 to a9.

  Hereinafter, a procedure for generating one frame from the frames a1 to a3 will be described. Assuming that the reference frame is the frame a2, the video memory operation unit 222 reads out the frame a2 and outputs it to the blocking unit 223. The blocking unit 223 divides the frame a2 and generates a plurality of blocks (image blocks) that are partial areas of the frame a2. Thereby, a plurality of blocks are set in the frame a2. The blocking unit 223 divides the frame a2 into, for example, M × N blocks, and sequentially outputs the divided blocks to the motion vector calculation unit 224. The state of block division is shown in FIG.

  The motion vector calculation unit 224 calculates a motion vector that associates image blocks between a plurality of image frames based on the motion in the image. The motion vector calculation unit 224 reads the frame a1 (reference frame) from the video memory 24, and the block (referred to as block A in FIG. 5A) output from the blocking unit 223 is the same coordinate (see FIG. 5). The block of the frame a2 and the block of the frame a1 are shifted while being shifted in the range where H and W dots are expanded vertically (see the area C in FIG. 5B and FIG. 6) centering on the area B) in FIG. The absolute value difference between the pixel values is obtained. The absolute value difference between pixel values is obtained by the following equation. Note that f (x, y) is a pixel value at coordinates (x, y) in the frame a1, and g (x, y) is a pixel value at coordinates (x, y) in the frame a2. In the area B, the x coordinate range is x0 to x0 + a, and the y coordinate range is y0 to y0 + b.

  The motion vector calculation unit 224 changes I and J in a range of −H ≦ I ≦ H and −W ≦ J ≦ W, and obtains a position where the value of Φ (I, J) is minimized. A block of the frame a1 having the same size as the block A around the position is associated with the block A as a movement destination block. When there is no moving object in the block, the same coordinate (I = 0, J = 0) where block A completely overlaps area B has the minimum value of Φ (I, J), and the motion vector is 0. . High-speed processing is possible in areas with little movement. When the object is moving, I and J are non-zero, so that the correspondence is performed using the amount of deviation in the I and J directions as a motion vector. The motion vector calculation unit 224 outputs the obtained motion vector and block information associated with the motion vector to the synthesis unit 225. The setting of the block of the reference frame associated with the block of the base frame is performed by the motion vector calculation unit 224, but may be performed by the blocking unit 223.

  Subsequently, the synthesizing unit 225 synthesizes the frames a1 and a2 by adding the pixel values of the blocks associated by the motion vector. In this synthesis, if the sum of all the pixels in the block of the absolute value difference of the pixel values calculated by the motion vector calculation unit 224 is less than the specified value, the synthesis unit 225 Then, the pixel value is added for each pixel having the same relative position. If the sum of all pixels in the block of absolute value differences of pixel values is equal to or greater than the specified value, the combining unit 225 does not combine the frames a1 and a2 but uses the pixel values of the frame a2. The pixel value of the original frame a2 is added. This is because the moving body is assumed to be deformed or rotated.

  Similarly, the frames a2 and a3 are combined. The synthesizing unit 225 holds the addition result of the pixel values in the synthesis processing of the frames a1 and a2 and the addition result of the pixel values in the synthesis processing of the frames a2 and a3, and further adds the results of both for each pixel. One image frame is generated and output. As a result, one image frame is generated from the frames a1 to a3. Similarly, one image frame is generated from each of the frames a4 to a6 and the frames a7 to a9, so that one third of the first frame rate is obtained. An image frame is output at a frame rate of 1.

  Next, an operation when the frame composition determination unit 221 determines not to perform image frame composition will be described. The video memory operation unit 222 stores the image frame output from the frame composition determination unit 221 in the video memory 24. The control unit 26 monitors the image frames sequentially stored in the video memory 24, and instructs the video memory operation unit 222 to output an image frame when the output timing of the image frame corresponding to the second frame rate comes. . Receiving the instruction, the video memory operation unit 222 outputs the image frame finally stored in the video memory 24 to the camera signal processing unit 23. When image frames are not combined by the above operation, that is, when the illuminance is sufficient, the image frames generated at the first frame rate are thinned out, and the image frames are generated at the second frame rate. Is output.

  Next, an example of a detailed procedure for synthesizing image frames in the present embodiment will be described with reference to FIGS. The level detector 25 calculates the average value of the image signal levels of all the pixels of the image frame output from the video memory 21 to the frame synthesis unit 22, and notifies the control unit 26 (step S701 in FIG. 7). The control unit 26 notifies the frame synthesis determination unit 221 of the average value of the image signal level. The frame composition determination unit 221 determines whether or not the notified average value of the image signal level is greater than or equal to a specified value (step S702).

  If the average value of the image signal levels is equal to or greater than the specified value, the frame synthesis determination unit 221 turns off the synthesis FLAG used to determine whether or not to perform image frame synthesis (step S703). This composite FLAG is set in an internal register. If the average value of the image signal level is less than the specified value, the frame synthesis determination unit 221 turns on the synthesis FLAG and sets the take-out CNT used when taking out the image frame stored in the video memory 24 to 0. (Step S704). This take-out CNT is also set in the internal register.

  Subsequently, the frame composition determination unit 221 determines whether to perform image frame composition based on the composition FLAG (step S705). When it is determined by the frame composition determination unit 221 that image frames are to be combined, the control unit 26 determines whether or not image frames necessary for composition are stored in the video memory 24 (step S706). ). When the image frames necessary for the composition are not stored in the video memory 24, the control unit 26 instructs the video memory operation unit 222 to store the image frames. Receiving the instruction, the video memory operation unit 222 stores the image frame output from the frame composition determination unit 221 in the video memory 24 (step S713).

  On the other hand, when the image frames required for the synthesis are stored in the video memory 24, the control unit 26 performs the synthesis of the image frames in cooperation with the video memory operation unit 222 to the synthesis unit 225 (step). S707). In step S707, the reference image frame and another one image frame are combined. After synthesizing the image frames, the control unit 26 determines whether or not the value of the extracted CNT is less than the predetermined value N (step S708). If the value of the extracted CNT is less than the predetermined value N, the control unit 26 increases the value of the extracted CNT by 1 (step S709). Subsequently, the process returns to step S707. In step S708, if the value of the extracted CNT is greater than or equal to N, the synthesis unit 225 outputs the synthesized image frame to the camera signal processing unit 23 (step S712).

  If it is determined in step S705 that the composition is not performed, the video memory operation unit 222 stores the image frame output from the frame composition determination unit 221 in the video memory 24 (step S710). Subsequently, the control unit 26 determines whether or not the frame number of the current image frame to be processed is a predetermined number based on the second frame rate (step S711). When the frame number of the image frame is a predetermined number, the video memory operation unit 222 outputs the current image frame to be processed to the camera signal processing unit 23 in accordance with an instruction from the control unit 26 (step S712). ). Thus, when the illuminance is sufficient and the image frame is not synthesized, the image frame is output to the subsequent processing circuit at a timing based on the second frame rate. If it is determined in step S711 that the frame number of the image frame is not a predetermined number, the series of processing ends, and processing for the next image frame is started.

  The procedure of the image frame synthesis process in step S707 is as follows (see FIG. 8). As an example, a case where P image frames are synthesized will be described. This P is the same as the predetermined value N to be compared with the extracted CNT in step S708. If the frame number of a reference image frame that is a reference for composition is L, the image frames are video memory so that the frame numbers are in the order of 1, 2,..., L−1, L + 1,. 24 and is sequentially synthesized with the image frame having the frame number L. FIG. 8 shows the procedure of the composition process of the reference image frame and another one image frame. The process shown in FIG. 8 is repeated P−1 times, and finally one image frame is synthesized.

  First, the control unit 26 calculates the frame number L of the reference image frame by L = P / 2 (truncated), and instructs the video memory operation unit 222 to read the image frame having the calculated frame number. Receiving the instruction, the video memory operation unit 222 reads the reference image frame from the video memory 24 and outputs it to the blocking unit 223 (step S801). The blocking unit 223 divides the reference image frame into M × N blocks (image blocks) (step S802).

  Subsequently, the blocking unit 223 determines whether or not the processing of all the blocks of the reference image frame has been completed (step S803). If all blocks have been processed, the process returns to step S708 in FIG. If an unprocessed block remains, the blocking unit 223 outputs one of the divided blocks to the motion vector calculation unit 224 (step S804).

  Subsequently, the motion vector calculation unit 224 selects an image frame to be processed (referred to as a reference frame) from among image frames having frame numbers 1, 2,..., L−1, L + 1,. Read from the video memory 24, set a block to be processed on the reference frame, and calculate an absolute value difference of pixel values between the two blocks of the block and the block of the standard image frame divided by the blocking unit 223. The calculation is performed while sequentially changing the blocks on the reference frame (step S805). The motion vector calculation unit 224 obtains the position of the block where the calculated absolute value difference is minimum on the reference frame, calculates the motion vector, and outputs the motion vector information and the information of the two blocks to the synthesis unit 225 ( Step S806).

  The synthesizer 225 determines whether or not the absolute value difference between the pixel values of the two blocks associated by the motion vector is less than a predetermined value (step S807). If the absolute value difference is less than the predetermined value, the synthesizer 225 adds the pixel values for each pixel having the same relative position between the blocks associated by the motion vector, and holds the addition result (step) S808). If the absolute value difference is greater than or equal to the predetermined value, the synthesis unit 225 adds the pixel value for each pixel having the same relative position between the block of the reference image frame and the same block, and the addition result is obtained. Hold. The result of addition in steps S807 and S808 is the addition result of the base image frame and other reference frames. After steps S807 and S808, the process returns to step S803.

  As described above, according to the present embodiment, since imaging is performed at a higher shutter speed, occurrence of image blurring and blurring can be reduced. In addition, since one image frame is generated from a plurality of image frames captured at a high shutter speed, a necessary exposure amount can be ensured even when the illumination light is dark. Furthermore, when a plurality of image frames are combined, blocks associated by motion vectors are added together, so that even when the subject is moving at high speed, there is little blurring or blurring of the image. Therefore, a good image can be obtained without increasing blur and blur even under conditions where the amount of illumination of the subject is not sufficient and the subject is moving at high speed.

  In addition, since the occurrence positions of noise generated in each image frame are dispersed according to a predetermined probability distribution, the influence of noise in the image can be reduced by combining a plurality of image frames. Further, when the sum of the absolute value differences of the pixel values of the two blocks associated by the motion vector is equal to or greater than a predetermined value, the relative position is equal between the block of the reference image frame and the same block. By adding pixel values for each pixel, it is possible to reduce the influence of subject deformation and rotation.

  As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to these embodiments, and includes design changes and the like without departing from the gist of the present invention. .

It is a block diagram which shows the structure of the imaging device by one Embodiment of this invention. It is a block diagram which shows the structure of the image pick-up element with which the imaging device by one Embodiment of this invention is provided. It is a block diagram which shows the structure of the flame | frame synthetic | combination part with which the imaging device by one Embodiment of this invention is provided. It is a conceptual diagram which shows the mode of the synthesis | combination of the image frame in one Embodiment of this invention. FIG. 5 is a reference diagram for explaining a motion vector calculation method according to an embodiment of the present invention. FIG. 5 is a reference diagram for explaining a motion vector calculation method according to an embodiment of the present invention. It is a flowchart which shows an example of the synthetic | combination procedure of the image frame in one Embodiment of this invention. It is a flowchart which shows an example of the synthetic | combination procedure of the image frame in one Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Imaging part, 2 ... Image processing part, 11 ... Imaging lens, 12 ... Solid-state imaging device, 13 ... AGC part, 14 ... A / D conversion part, 15 ... Drive circuit, 21 ... video memory, 22 ... frame synthesis unit, 23 ... camera signal processing unit, 24 ... video memory, 25 ... level detector, 26 ... control unit, 121, 122, 123, 124 ... Horizontal transfer CCD, 125, 126, 127, 128 ... Vertical transfer CCD, 221 ... Frame composition determination unit, 222 ... Video memory operation unit, 223 ... Blocking unit, 224 ... motion vector calculation unit, 225 ... synthesis unit

Claims (6)

Motion vector calculating means for calculating a motion vector for associating an image block, which is a partial region of an image frame constituting the image signal of the first frame rate, between the plurality of image frames based on the motion in the image;
By adding the image blocks associated by the motion vector between the plurality of image frames, the plurality of image frames are combined to generate one image frame, which is lower than the first frame rate Frame synthesizing means for outputting as an image signal of the second frame rate;
An image processing apparatus comprising: The motion vector calculation means calculates a difference between pixel values of the image block of the reference image frame as a reference for synthesis and the image block of the other image frame,
When the absolute value of the difference is greater than or equal to a predetermined value, the frame composition unit adds the image block of the reference image frame and the same image block of the reference image frame, and calculates the absolute value of the difference The image block of the reference image frame associated with the motion vector is added to the image block of another image frame when the value is less than a predetermined value. Image processing device. Signal level detecting means for detecting a luminance level of the image signal of the first frame rate or the image signal of the second frame rate;
Frame composition determination means for determining whether to synthesize the image frame based on the detected luminance level;
Frame storage means for storing a plurality of the image frames constituting the image signal of the first frame rate;
A block setting unit that sets the image block in the image frame stored in the frame storage unit when the frame synthesis determination unit determines to combine the image frame;
The image processing apparatus according to claim 1, further comprising: Imaging means for performing imaging and outputting an image signal of a first frame rate;
Motion vector calculating means for calculating a motion vector for associating an image block, which is a partial region of an image frame constituting the image signal of the first frame rate, between the plurality of image frames based on a motion in the image;
By adding the image blocks associated by the motion vector between the plurality of image frames, the plurality of image frames are combined to generate one image frame, which is lower than the first frame rate Frame synthesizing means for outputting as an image signal of the second frame rate;
An imaging device comprising: The motion vector calculation means calculates a difference between pixel values of the image block of the reference image frame as a reference for synthesis and the image block of the other image frame,
When the absolute value of the difference is greater than or equal to a predetermined value, the frame composition unit adds the image block of the reference image frame and the same image block of the reference image frame, and calculates the absolute value of the difference 5. The method according to claim 4, wherein when the value is less than a predetermined value, the image block of the reference image frame associated with the motion vector is added to the image block of another image frame. Imaging device. Signal level detecting means for detecting a luminance level of the image signal of the first frame rate or the image signal of the second frame rate;
Frame composition determination means for determining whether to synthesize the image frame based on the detected luminance level;
Frame storage means for storing a plurality of the image frames constituting the image signal of the first frame rate;
A block setting unit that sets the image block in the image frame stored in the frame storage unit when the frame synthesis determination unit determines to combine the image frame;
The imaging device according to claim 4, further comprising:

Images