JP2010183207A - Image processing method, and image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce variation in luminance among a plurality of output images even when captured by any imaging means. <P>SOLUTION: For a plurality of acquired images of which photographing regions are continuously captured in a time series by the imaging means 2, an image processing apparatus 1 sets average luminance of a portion in which face detection is performed to be representative luminance for each acquired image. A processing means 15 for periodical variation sets a processing term that becomes an integer multiple of a temporal variation period of the representative luminance, and extracts one prescribed acquired image or a plurality of acquired images at timing in synchronization with the period of the processing term in the plurality of acquired images included in the processing term to generate output images in all processing terms. The processing means 15 for periodical variation successively outputs the generated output image to an image display means 3 at the variation period of the representative luminance. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing method and an image processing apparatus that acquire a plurality of images obtained by continuously capturing an imaging region in time series and perform image processing on the plurality of acquired images.

  2. Description of the Related Art Conventionally, there is known an image processing apparatus that acquires a plurality of images obtained by continuously capturing a shooting region in time series and performs image processing on the plurality of acquired images.

  Such an image processing apparatus can be used in an intercom system capable of displaying an image, particularly an apartment intercom system. When the image processing apparatus is used in an intercom system for condominiums, imaging means connected to the image processing apparatus is attached to a lobby of the apartment. A discharge lamp such as a fluorescent lamp is generally used as a light source for a lobby to which an image pickup means is attached. Since the discharge lamp is lit at a cycle that is ½ that of the AC power supply, flicker occurs when an image is taken in the illumination environment of the discharge lamp. For example, in the case of the interlace method, as shown in FIG. 15, the discharge lamp fluctuation period is 10 milliseconds (= 1 / (50 Hz × 2)), and the rate of each field image is about 60 fields / second (field interval). 16.67 milliseconds), the brightness of the field image will fluctuate with a period of 50 milliseconds.

  Conventionally, when flicker occurs as described above, the imaging means reduces the flicker by setting the exposure time to an integral multiple of the fluctuation cycle of the discharge lamp and performing imaging. In Patent Document 1, the imaging unit removes flicker by changing the exposure time by changing the shutter speed according to the flicker cycle.

Japanese Patent Laid-Open No. 11-205658

  However, as described above, overexposure and underexposure may occur in the method in which the imaging means sets the exposure time to an integral multiple of the fluctuation cycle of the discharge lamp and performs imaging to reduce flicker. In addition, when the imaging unit does not have the above-described function, flicker cannot be reduced, and thus luminance fluctuation between output images cannot be reduced for a plurality of output images.

  The present invention has been made in view of the above points, and an object of the present invention is to perform image processing capable of reducing luminance fluctuation between output images with respect to a plurality of output images regardless of any imaging means. It is to provide a method and an image processing apparatus.

  The image processing method according to the first aspect of the present invention acquires a plurality of images obtained by continuously capturing a shooting region in time series, performs image processing on the plurality of acquired images, and outputs an image after image processing as an output image. As an image processing method for outputting to the image display means, for each of the plurality of acquired images, the luminance of a predetermined portion is set as the representative luminance for each acquired image, and is an integral multiple of the time variation period of the representative luminance. The predetermined period is set as the processing period, and in all the processing periods, the predetermined one or more acquired images are extracted at a timing synchronized with the period of the processing period among the plurality of acquired images included in the processing period. An output image is generated, and the output image is sequentially output to the image display means at the variation cycle of the representative luminance.

  According to a second aspect of the present invention, there is provided an image processing method according to the first aspect, wherein, in all the processing periods, the plurality of acquired images included in the processing period have the same coordinates between two or more acquired images. The output image is generated by performing a process including at least one of a synthesizing process for adding pixel values and a thinning process for thinning a part of acquired images.

  According to a third aspect of the present invention, there is provided the image processing method according to the second aspect, wherein the plurality of acquired images are an interlaced odd field image and an even field image, and the combining process includes one or more odd field images. And a process of weighting and adding pixel values of the same coordinates between one or more even field images, and in the synthesis process, the sum of the weights of the pixel values of all odd field images and the pixel values of all even field images The total weight is equal.

  An image processing method according to a fourth aspect of the present invention is the image processing method according to the second aspect, wherein the plurality of acquired images are an interlaced odd field image and an even field image, and the combining process includes two or more odd field images. An odd field composition process for adding pixel values of the same coordinate between the two and an even field composition process of adding pixel values of the same coordinate between two or more even field images.

  According to a fifth aspect of the present invention, there is provided an image processing method according to the fourth aspect of the present invention, wherein two or more odd field image processing sections for performing the odd field composition processing and two or more even field images for performing the even field composition processing are provided. The odd-number field synthesis process and the even-number field synthesis process are performed so as not to overlap each other.

  According to a sixth aspect of the present invention, there is provided an image processing method according to the fourth aspect of the present invention, wherein two or more odd field image processing sections for performing the odd field composition processing and two or more even field images for performing the even field composition processing are provided. The odd-number field synthesis process and the even-number field synthesis process are performed so that the overlap with the processing section is maximized.

  The invention of the image processing apparatus according to claim 7 acquires a plurality of images obtained by continuously capturing a shooting region in time series, performs image processing on the plurality of acquired images, and outputs an image after image processing as an output image As an image processing device that outputs to the image display means, for each of the plurality of acquired images, the luminance of a predetermined portion is set as the representative luminance and is an integral multiple of the time variation period of the representative luminance. The predetermined period is set as the processing period, and in all the processing periods, the predetermined one or more acquired images are extracted at a timing synchronized with the period of the processing period among the plurality of acquired images included in the processing period. The image processing apparatus includes processing means for generating an output image and sequentially outputting the output image to the image display means at a variation cycle of the representative luminance.

  The invention of an image processing apparatus according to claim 8 is the invention according to claim 7, wherein, in all the processing periods, the processing means includes two or more acquired images for a plurality of acquired images included in the processing period. The output image is generated by performing a process including at least one of a synthesizing process for adding pixel values of the same coordinates and a thinning process for thinning out a part of acquired images.

  The image processing apparatus according to a ninth aspect is the invention according to the eighth aspect, wherein the plurality of acquired images are an interlaced odd-numbered field image and an even-numbered field image. An odd field composition process for adding pixel values of the same coordinates between the odd field images and an even field composition process of adding pixel values of the same coordinates between two or more even field images. To do.

  According to the first aspect of the present invention, flicker can be reduced by generating an output image having the same luminance for each period that is an integral multiple of the time variation period of the representative luminance. Even so, it is possible to reduce the luminance fluctuation between the output images with respect to a plurality of output images.

  According to the second aspect of the present invention, flicker can be reduced by generating an output image having the same luminance for each period that is an integral multiple of the time variation period of the representative luminance. Even so, it is possible to reduce the luminance fluctuation between the output images with respect to a plurality of output images.

  According to the invention of claim 3, when the interlaced output image is sequentially displayed on the image display means, the fluctuation of the display image can be reduced.

  According to the invention of claim 4, in the interlace method, it is possible to keep the resolution while maintaining the independence of each field.

  According to the invention of claim 5, since the processing interval of the odd field image and the processing interval of the even field image are separated, the processing timing can be shifted between the odd field image and the even field image. When a set of frame images composed of an output image obtained by synthesizing field images and an output image obtained by synthesizing two or more even-numbered field images is displayed as a moving image on the image display means, the movement of the subject can be smoothed.

  According to the sixth aspect of the present invention, since the processing interval of the odd field image and the processing interval of the even field image overlap, the processing timing can be made closer to the odd field image and the even field image. When one frame image composed of an output image obtained by synthesizing field images and an output image obtained by synthesizing two or more even field images is displayed as a still image on the image display means, blurring as a still image can be reduced. it can.

  According to the invention of claim 7, flicker can be reduced by generating an output image having the same luminance for each period that is an integral multiple of the time variation period of the representative luminance. Even so, it is possible to reduce the luminance fluctuation between the output images with respect to a plurality of output images.

  According to the invention of claim 8, flicker can be reduced by generating an output image having the same luminance for each period that is an integral multiple of the time variation period of the representative luminance. Even so, it is possible to reduce the luminance fluctuation between the output images with respect to a plurality of output images.

  According to the invention of claim 9, in the interlace method, it is possible to keep the resolution while maintaining the independence of each field.

1 is a block diagram illustrating a configuration of a first embodiment. It is a figure which shows the usage example same as the above. It is a block diagram which shows the structure of the process means for a periodic fluctuation which concerns on the same as the above. It is a figure for demonstrating operation | movement of the process means for periodic fluctuations based on the same as the above. It is a block diagram which shows the structure of the processing means for irregular fluctuations based on the same as the above. (A) is an example of setting the measurement points of the motion vector of the same subject area, (b) is a measurement result of each motion vector measurement point of the same subject area, (c) is the movement of the face area. FIG. 4D is a setting example diagram of vector measurement points, and FIG. 4D is a measurement result diagram of each motion vector measurement point in the face area same as above. It is an equivalent circuit diagram of the infinite length impulse response IIR filter corresponding to the processing operation of the irregular variation processing means according to the above. It is explanatory drawing of the addition ratio of the preservation | save image of a synthesized image, and an acquisition image in the same as the above. It is principle explanatory drawing of the motion correction in the same as the above. It is a flowchart which shows operation | movement of the image processing apparatus which concerns on the same as the above. FIG. 10 is a diagram for explaining an operation of a periodic variation processing unit according to the second embodiment. It is a figure for demonstrating operation | movement of the process means for periodic fluctuations concerning Embodiment 3. FIG. It is a figure for demonstrating operation | movement of the process means for periodic fluctuations concerning Embodiment 4. FIG. FIG. 10 is a diagram for explaining an operation of a periodic variation processing unit according to the fifth embodiment. It is a figure which shows the relationship between a light source and a captured image.

(Embodiment 1)
First, the configuration of the image processing apparatus according to the first embodiment will be described with reference to FIG. The image processing apparatus 1 shown in FIG. 1 acquires a moving image obtained by continuously capturing a shooting area in time series from the image capturing unit 2, performs image processing on a plurality of acquired images constituting the moving image, and performs image processing. Is an image processing apparatus that outputs the image as an output image to the image display means 3. The image processing apparatus 1 includes a moving object detection unit 10 and a face detection unit 11 for setting a target subject area, a pixel value correction unit 12 that corrects a pixel value of each pixel of an acquired image, and a temporary acquired image. The image storage means 13 to preserve | save, the determination means 14 which performs determination using the below-mentioned representative brightness | luminance and the brightness | luminance of an evaluation object pixel, and the process means 19 which performs an image process with respect to each acquisition pixel are provided.

  The image pickup means 2 includes an image pickup device (not shown) such as a CCD (Charged Coupled Device), for example, and is attached to a ceiling C of a building (ceiling of a condominium lobby) C as shown in FIG. Then, a predetermined imaging region including the subject Mn is continuously imaged in time series. A discharge lamp (not shown) such as a fluorescent lamp is attached as a lighting fixture to the ceiling C to which the imaging means 2 is attached.

  Each acquired image captured by the imaging unit 2 is an interlaced odd field image or even field image. Therefore, as shown in FIG. 15, the rate of each field image is about 60 fields / second (field interval 16.67 milliseconds). The fluctuation cycle of the discharge lamp is 10 milliseconds (= 1 / (50 Hz × 2)). Therefore, the exposure amounts of the first field (first odd field) and the fourth field (second even field) shown in FIG. 15 are the same, and the second field (first even field) and The exposure amount for the fifth field (third odd field) is the same.

  The image display means 3 is, for example, an interphone master set installed in a house, has a display screen 30 such as a liquid crystal panel, and sequentially displays output images from the image processing apparatus 1 on the display screen 30.

  The moving object detection means 10 shown in FIG. 1 creates a difference image between, for example, an acquired image and a stored image to be synthesized, extracts a change area, binarizes the change area, and sets the change areas of the proximity distances to one. A circumscribed rectangle to be combined into one is generated to be an integrated region, and for each integrated region, density pattern matching is performed between the acquired image and the saved image, and an integrated region having a low similarity is extracted as a moving object candidate (FIG. 6A). (Refer to FIG. 5), which will be described later as subject area information.

  For example, the face detection unit 11 compares the color histogram generated from the image data of the acquired image with the template histogram of the face part created in advance to determine the face area using a method of determining the face part area. This face area information is given to the image motion calculation means 163.

  The pixel value correction unit 12 calculates the average luminance of the region used for face detection from the pixel value of each pixel of the region used for face detection for each of the plurality of acquired images from the imaging unit 2. Calculated as representative luminance, and for each acquired image, corrects the pixel value of each pixel (red, green, and blue pixel values in the case of a color image) so that the representative luminance is included in the preset range. To do. Specifically, the pixel value correcting unit 12 converts the pixel value of each pixel according to how much the representative luminance is out of the above range, using a conversion table stored in advance in the storage unit 120. The acquired image output from the pixel value correction unit 12 is temporarily stored in the image storage unit 13. Note that the representative luminance may be the average luminance of the region where motion detection is performed, instead of the average luminance of the region used for face detection. When the representative luminance is included in the above range from the beginning, the acquired image is temporarily stored in the image storage unit 13 without being corrected by the pixel value correcting unit 12.

  The determination unit 14 includes a luminance change rate determination unit 140 that determines whether or not the luminance change rate indicating the representative luminance after correction with respect to the representative luminance before correction by the pixel value correction unit 12 deviates from the specified range, and the representative luminance Periodic variation determination means 141 that determines whether or not temporal variation is periodic, and temporal variation in luminance of one or more pixels (evaluation target pixels) included in the region used for face detection are irregular. And irregular fluctuation determining means 142 for determining whether or not.

  The irregular fluctuation determination unit 142 obtains data obtained by collecting the luminance values of the evaluation target pixels in the time axis direction, and when the standard deviation of the data exceeds a preset value, the temporal fluctuation of the luminance of the evaluation target pixels Is determined to be irregular. On the other hand, when the standard deviation of the data is less than or equal to the set value, the irregular variation determination unit 142 determines that the temporal variation in luminance of the evaluation target pixel is not irregular. Note that the evaluation target pixel may not be one or more pixels included in the region used for face detection, but may be one or more pixels included in the region where motion detection is performed.

  The processing unit 19 includes a periodic variation processing unit 15 that is executed when the temporal variation of the representative luminance is periodic, and an irregular variation that is performed when the temporal variation of the luminance of the evaluation target pixel is irregular. Processing means 16.

  The periodic variation processing unit 15 is executed when the periodic variation determination unit 141 determines that the temporal variation of the representative luminance is periodic. As shown in FIG. An image composition / thinning unit 150 that performs composition processing and thinning processing, and an image storage unit 151 that temporarily stores an acquired image and an output image are provided.

  Next, the operation of the periodic variation processing means 15 will be described. First, the period of the representative luminance is 50 milliseconds (see FIG. 15). Therefore, as shown in FIG. 4, three field images (acquired images) are included in one period of time variation of the representative luminance.

  The image composition / thinning means 150 sets the first odd field image among the three field images included in one period of the temporal variation of the representative luminance as a single image. The image composition / thinning unit 150 generates a single image for each period of time variation of the representative luminance. The periodic fluctuation processing means 15 that generates a single image for each period of time variation of the representative luminance sequentially outputs the single image as an output image to the image display means 3 (see FIG. 1) with the fluctuation period of the representative luminance. . On the other hand, for the remaining field images (second field image and third field image), the image composition / thinning process 150 performs a thinning process. That is, the image composition / thinning means 150 outputs the field image as an output image by skipping two, and outputs the odd field image and the even field image alternately to the image display means 3 at 20 fps. In this way, the luminance between output images can be made substantially equal, and flicker does not occur.

  When the luminance change rate determining unit 140 determines that the luminance change rate is out of the specified range, the irregular variation processing unit 16 has an irregular time variation in the luminance of the evaluation target pixel. As shown in FIG. 5, an image storage unit 160 that stores an output image after image synthesis, which is described later, and a storage image stored in the image storage unit 160 and the latest image are stored. Reference is made to an image composition unit 161 that performs image composition with the acquired image, an image motion conversion unit 162 that converts a motion vector in the acquired image from the posture information and condition information of the imaging unit 2, and a target subject area on the acquired image. And a motion vector measuring point 163 for setting motion vector measurement points and calculating a motion vector between the acquired image and the saved image. The attitude information of the imaging unit 2 is acquired from the attitude control unit 17 (see FIG. 1) that controls the attitude of the imaging unit 2, and the condition information of the imaging unit 2 is an imaging condition control unit that controls the imaging conditions of the imaging unit 2. 18 (see FIG. 1).

  The irregular fluctuation processing means 16 smoothes the acquired image with the acquired image, updates the smoothed image as a new saved image, and sets it as an output image. Details will be described below.

  The image motion calculation unit 163 sets a plurality of motion vector measurement points for the moving object (target subject region) detected by the moving object detection unit 10 (see FIG. 1). Here, when the subject Mn (see FIG. 2) is an article, even if the subject Mn moves within the frame of the captured image, there is no deformation, but when the subject Mn is a person, partial movement or deformation of the limbs or the like. In order to avoid these effects and measure the overall movement of a person, it is necessary to measure the movement of the face and torso. Therefore, the image motion calculation means 163 has the motion vector measurement points (FIG. 6A) so as to be dense in the upper half of the region set as a moving body (the circumscribed rectangular region indicated by the broken line in FIG. 6A). In FIG. 6B, the motion vector at each motion vector measurement point is obtained as indicated by “→” in the block matching method.

  Further, the image motion calculation means 163 calculates a representative value from these calculated motion vectors. The representative value is obtained by, for example, setting a motion vector equal to more than a majority of the obtained motion vectors as the representative value motion vector M1. In this embodiment, since a plurality of motion vectors are handled as described later, the representative value is not limited to a unique value, and a plurality of motion vector measurement values may be output to the image synthesis unit 161.

  The image motion calculating unit 163 applies shadows such as eyes, ears, and nose to the face area (inside the rectangle indicated by the broken line in FIG. 6C) detected by the face detecting unit 11 (see FIG. 1). Correspondingly, motion vector measurement points (indicated by □ in FIG. 6C) are set as illustrated, and the motion vectors at the respective motion vector measurement points are indicated by “→” in FIG. 6D by the block matching method. Measure as follows. In this case, since the mouth periphery is deformed when words are generated, it is necessary to set the motion vector measurement points around the mouth densely, but the mouth periphery is not important when determining the displacement of the entire face. It is not necessary to set motion vector measurement points around the mouth. At each motion vector measurement point, an accurate position can be obtained by recognizing facial parts such as eyes, ears, and nose by a method of extracting a facial region. Depending on the required set position accuracy, the relative position of each face part with respect to the face contour may be obtained in advance from the statistical information of the face shape and geometrically applied to the rectangle serving as the face contour.

  FIG. 6D shows a motion vector distribution M2 (x, y) (hereinafter referred to as “motion vector M2”) obtained at each motion vector measurement point set in the target subject region (face region). Assuming that the face is a rigid body with no deformation except for the deformed portion, the movement of the face between the acquired image and the saved image can be approximated by a primary transformation. However, as shown in the drawing, processing such as data mapping is required to cope with a state in which the movement is not uniform around the mouth.

  The image pickup means 2 is controlled by a posture control means 17 (see FIG. 1) by a drive means (not shown) such as a motor, whereby the horizontal (pan) and vertical (tilt) postures are controlled. However, the image motion conversion means 162 shown in FIG. 5 takes in the posture information of the imaging means 2 from the posture control means 17 and measures the relative shake angle θ at the time of acquisition of each image used for image composition. Of course, when the posture control of the imaging means 2 is not performed, zero displacement is adopted as the posture information. The image motion conversion unit 162 converts the shake angle θ obtained as described above into a motion vector using condition information including the focal length l of the optical system of the imaging unit 2 obtained from the imaging condition control unit 18 (see FIG. 1). Convert. The amount of movement Δx of the image on the imaging plane α due to the difference in shooting time between the acquired image and the stored image using the shake angle θ and the focal length l is expressed as Δx = [(l2 + x2) / l] tan θ. The motion vector distribution M3 (x, y) (hereinafter referred to as “motion vector M3”) is set by adding the distortion information from the posture control means 17 as necessary.

  The image synthesizing unit 161 obtains a weighted average of m: n pixel values of the same coordinates on the acquired image and the saved image, outputs an image based on the averaged pixel value as a synthesized image, and outputs the synthesized image The image is stored in the image storage unit 160 and used as a new saved image when processed with a new image acquired from the imaging unit 2. This series of processing can be shown as an infinite length impulse response IIR filter shown in FIG. 7, and this filter acts as a smoothing filter to reduce noise.

  By the way, the addition ratio of the individual images acquired in time series in the latest synthesized image is as shown in FIG. 8, and the moving information is blurred because the past information remains as if it is tailed. Note that the bar at the left end in FIG. 8 shows the newly acquired image, and shows the bar of the image acquired in the form of going back in the past from the right to the right, and the bar length of this image is the latest composite. An addition ratio in an image, that is, a stored image is shown.

  Therefore, the image synthesizing unit 161 shown in FIG. 5 performs motion correction using the motion vector M1, the motion vector M2 from the image motion calculating unit 163, and the motion vector M3 from the image motion converting unit 162 as motion vector candidates. .

  In the stored image and the acquired image, the subject Mn captured over time has a movement. Here, as shown in FIG. 9 (d), a processing unit consisting of one or more pixels (□ shown in FIG. 9 (d)) is set on the acquired image, the subject Mn is a person, and the torso of the moving person In the processing unit corresponding to, since the similarity with the acquired image is high in the saved image in which the motion is corrected by the motion vector M1 obtained by moving object detection from the acquired image (FIG. 9D), the motion vector M1 is applied. (FIG. 9 (c)). On the other hand, in the processing unit corresponding to the face of a moving person, the saved image that has been motion-corrected by the motion vector M2 obtained by face detection has a high similarity to the acquired image, so the motion vector M2 is applied (FIG. 9 (b)). Furthermore, in the processing unit corresponding to the background, the saved image that has been motion-corrected by the motion vector M3 obtained by the displacement of the imaging means 2 has a high similarity to the acquired image, so motion correction by the motion vector M3 is applied. The

  Subsequently, the image synthesizing unit 161 performs a process of evaluating the similarity between the corrected image after motion correction and the acquired image as described above before synthesizing the image.

  In this case, the absolute value of the difference between the pixel values between the corrected image and the acquired image is integrated within the processing unit or in the vicinity of the processing unit to obtain a block matching error for evaluation, or a spatial difference image Similarly, a block matching error is obtained and evaluated. In this case, in the block matching error, the larger the value, the lower the similarity.

  Through the process of evaluating the similarity in this way, the image synthesis is performed by determining the addition ratio of the pixel values of the stored image and the acquired image at the time of synthesis based on the evaluation result. Here, the ratio of the added value is determined by, for example, using an evaluation value of similarity to a stored image that has been subjected to motion correction using the motion vector M1 and an evaluation value of similarity to a stored image that has been subjected to motion correction using the motion vector M2. In comparison, when the similarity on the motion vector M1 side is high, the acquired image and the saved image corrected by the motion vector M1 are weighted average at a ratio of 1: 3. When the similarity on the side of the motion vector M2 is high, the weighted average is taken at a ratio of 1: 3 between the acquired image and the stored image corrected for motion with the motion vector M2. If any evaluation value does not exceed a preset threshold value, the pixel value of the stored image is not used and the pixel value of the acquired image is applied as it is.

  Here, each unit configuring the image processing apparatus 1 may be configured by a function realized by executing a program in a microcomputer, or may be configured independently by a hardware configuration. . In addition, the image processing apparatus 1 is separate from the imaging unit 2 as shown in FIG. 2, but may be configured integrally with the imaging unit 2. Furthermore, the image display unit 3 may be integrated into the imaging device.

  Next, the operation of the image processing apparatus 1 according to the present embodiment will be described with reference to FIG. First, the image processing apparatus 1 acquires a new image from the imaging unit 2 (S1 in FIG. 10). The moving object detection means 10 detects a moving object from the acquired image, and the face detection means 11 detects a face portion from the acquired image (S2). For each acquired image, the area used for face detection is set as the luminance evaluation area (S3), and the average luminance of the luminance evaluation area is measured as the representative luminance (S4).

  Subsequently, the periodic variation determination unit 141 analyzes the temporal variation of the representative luminance (S5), and determines whether the temporal variation of the representative luminance is periodic (S6). If the temporal variation of the representative luminance is periodic, the periodic variation processing means 15 operates (intra-variation period synthesis processing on) (S7). That is, the periodic variation processing unit 15 sets the first odd field image among the three field images included in one period of the temporal variation of the representative luminance as a single image (output image), and the remaining field images. (2nd field image, 3rd field image) is thinned out.

  Subsequently, for each acquired image, the pixel value correction unit 12 corrects the pixel value of each pixel using the conversion table stored in the storage unit 120 so that the representative luminance is included in a preset range. (S8).

  Thereafter, the luminance change rate determination means 140 determines whether or not the luminance change rate deviates from the specified range (whether or not the luminance change amount is a predetermined value or less) (S9). When the luminance change rate deviates from the specified range (when the luminance change amount is a predetermined value or less), the luminance of the acquired image pixel is measured (S10), and the pixel luminance statistical processing is performed (S11).

  Thereafter, the irregular variation determination unit 142 determines whether or not the temporal variation in luminance of the evaluation target pixel is irregular (S12). That is, it is determined whether or not the standard deviation of the luminance value is equal to or less than a predetermined value. When the standard deviation of the luminance value is larger than the predetermined value, the irregular variation processing means 16 operates (S13). That is, the irregular fluctuation processing means 16 smoothes the acquired image with the acquired image (the previous output image integration process is on).

  As described above, according to the present embodiment, when the time variation of the luminance of the evaluation target pixel is irregular, the random noise can be reduced by smoothing the acquired image in the time axis direction, and the time of the representative luminance can be reduced. When the variation is periodic, flicker can be reduced by generating a single image with the same luminance for each period of time variation of the representative luminance, and as a result, between output images for a plurality of output images. The luminance fluctuation can be reduced.

  Further, according to the present embodiment, the output image can be displayed at the same speed as the acquired image by sequentially outputting the output image in accordance with the time variation period of the representative luminance.

  Furthermore, according to the present embodiment, each process can be performed using the luminance of the important part in the acquired image.

(Embodiment 2)
As shown in FIG. 11, the image processing apparatus 1 according to the second embodiment is different from the first embodiment in that the periodic variation processing unit 15 generates an output image by combining two field images (acquired images). This is different from the image processing apparatus 1. Note that the description of the same components as those in the first embodiment is omitted.

  The image synthesizing / decimating unit 150 (see FIG. 3) of the periodic variation processing unit 15 according to the present embodiment performs two consecutive field images (first field image, 2) for each period of time variation of the representative luminance. The second field image) is integrated into an output image. Specifically, the image composition / thinning unit 150 adds a pixel value between the second field image and the first field image stored in the image storage unit 151 to perform a bit shift operation (1/2 times). I do. Thereafter, the image composition / thinning means 150 outputs the composite image as an output image to the image display means 3 at 20 fps. On the other hand, for the third field image, the image composition / thinning means 150 performs a thinning process. The periodic variation processing unit 15 of the present embodiment is the same as the periodic variation processing unit 15 of the first embodiment except for the points described above.

  As described above, according to the present embodiment, the luminance between the output images can be made substantially equal as in the first embodiment, and further, the vertical movement of the output image can be prevented from occurring.

  Further, since the rate of each field image is precisely 59.94 fields / second, it is actually 10 seconds (= 0.05 seconds × 0.01 seconds / {(3 / 59.94 fields / second). )-(3/60)}) in some cases, the luminance fluctuation may occur. However, according to the present embodiment, the luminance fluctuation in the 10-second period can also be reduced.

(Embodiment 3)
As shown in FIG. 12, the image processing apparatus 1 according to the third embodiment is different from the first embodiment in that the periodic variation processing unit 15 generates an output image by combining three field images (acquired images). This is different from the image processing apparatus 1. Note that the description of the same components as those in the first embodiment is omitted.

  The image synthesizing / decimating unit 150 (see FIG. 3) of the periodic variation processing unit 15 according to the present embodiment performs three consecutive field images (all fields in one cycle) for each period of time variation of the representative luminance. Image) is integrated into an output image. Specifically, the image composition / thinning means 150 adds the pixel value and multiplies the constant by the third field image and the first field image and the second field image stored in the image storage means 151. Perform bit shift operations. At this time, the first field image (odd field image in the illustrated example): the second field image (even field image in the illustrated example): the weight of the third field image (odd field image in the illustrated example) is 1: 2. : 1. That is, the sum of the weights of the odd-numbered field images is made equal to the sum of the weights of the even-numbered field images. Thereafter, the image composition / thinning means 150 outputs the composite image as an output image to the image display means 3 at 20 fps. The periodic variation processing unit 15 of the present embodiment is the same as the periodic variation processing unit 15 of the first embodiment except for the points described above.

  As described above, according to the present embodiment, the luminance between output images is substantially equal, and flicker does not occur. Further, when the interlace output image is sequentially displayed on the image display means 3, it is possible to reduce the vertical movement of the output image, that is, the shaking of the display image.

  In the present embodiment, the weighting of the first field image: the second field image: the third field image is 1: 2: 1. However, as a modification of the present embodiment, the first field image: The weight of the second field image: third field image may be 1: 1: 1. That is, all the field images may have the same weight. In such a case, it is possible to maximize the effect of suppressing the luminance fluctuation in the 10-second cycle.

(Embodiment 4)
As shown in FIG. 13, the image processing apparatus 1 according to the fourth embodiment is an image according to the first embodiment in that the periodic variation processing unit 15 synthesizes odd field images and synthesizes even field images. It is different from the processing apparatus 1. Note that the description of the same components as those in the first embodiment is omitted.

  The image composition / thinning unit 150 (see FIG. 3) of the periodic variation processing unit 15 according to the present embodiment sets a processing period that is twice the time variation period of the representative luminance. The first odd field image and the second odd field image (third field image) are combined with a weight of 1: 1 (the pixel value multiplied by 1/2: 1/2) to generate an output image. On the other hand, the first even field image (second field image) and the second even field image (fourth field image) are 1: 1 (1/2: 1/2 as a value to be multiplied by the pixel value). ) To generate an output image. For the third odd field image (fifth field image) and the third even field image (sixth field image), the image composition / thinning means 150 performs a thinning process. In the present embodiment, as shown in FIG. 13, the processing sections of the odd-numbered field images and the processing sections of the even-numbered field images do not overlap and are alternated. That is, in this embodiment, the center (centroid) of the processing section of the odd field image is separated from the center (centroid) of the processing section of the even field image. The output image by the odd field image and the output image by the even field image are alternately output to the image display means 3.

  According to the present embodiment, in the interlace method, it is possible to keep the resolution while maintaining the independence of each field.

  Further, according to the present embodiment, since the processing interval of the odd field image and the processing interval of the even field image are separated, the processing timing can be shifted between the odd field image and the even field image. When a set of frame images composed of an output image obtained by synthesizing an image and an output image obtained by synthesizing two even field images is displayed on the image display means 3 as a moving image, the movement of the subject Mn can be made smooth.

(Embodiment 5)
As illustrated in FIG. 14, the image processing apparatus 1 according to the fifth embodiment is different from the fourth embodiment in that a processing section for combining odd field images and a processing section for combining even field images overlap. This is different from the image processing apparatus 1. Note that the description of the same components as those in the fourth embodiment is omitted.

  The image composition / thinning unit 150 (see FIG. 3) of the periodic variation processing unit 15 according to the present embodiment sets a processing period that is twice the time variation period of the representative luminance. The 1: 1 odd field image, the second odd field image (third field image), and the third odd field image (fifth field image) are 1: 1: 2 (the value multiplied by the pixel value is 1 / 4: 1/4: 1/2) is combined to generate an output image, while a first even field image (second field image) and a second even field image (fourth field image). And the third even field image (sixth field image) are combined at a weighting ratio of 2: 1: 1 (the pixel value multiplied by 1/2: 1/4: 1/4) to produce an output image. Generate. In the present embodiment, as shown in FIG. 14, the processing interval of the odd field image and the processing interval of the even field image overlap. That is, in this embodiment, the center (centroid) of the processing section of the odd field image and the center (centroid) of the processing section of the even field image are substantially the same. The output image by the odd field image and the output image by the even field image are alternately output to the image display means 3.

  According to the present embodiment, since the processing interval of the odd field image and the processing interval of the even field image overlap, the processing timing can be made closer to the odd field image and the even field image. When one frame image consisting of the synthesized output image and the output image obtained by synthesizing the three even field images is displayed on the image display means 3 as a still image, blurring as a still image can be reduced.

  In the first to fifth embodiments, the periodic fluctuation processing means 15 and the irregular fluctuation processing means 16 are used together. However, as a modification of the above embodiment, the periodic fluctuation processing means 15 is used alone. The image processing apparatus may be provided.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 10 Moving body detection means 11 Face detection means 12 Pixel value correction means 141 Periodic fluctuation determination means 142 Irregular fluctuation determination means 15 Periodic fluctuation processing means 16 Irregular fluctuation processing means 2 Imaging means 3 Image display means

Claims (9)

  An image processing method for acquiring a plurality of images obtained by continuously capturing a shooting region in time series, performing image processing on the plurality of acquired images, and outputting the image processed image to an image display means as an output image. Then, for each of the plurality of acquired images, the luminance of a predetermined portion is set as the representative luminance, and a predetermined period that is an integral multiple of the time variation period of the representative luminance is set as the processing period, In the period, a predetermined one or a plurality of acquired images are extracted from the plurality of acquired images included in the processing period at a timing synchronized with the period of the processing period to generate the output image, and the output image is represented by the representative An image processing method comprising: sequentially outputting to the image display means at a luminance fluctuation period.   Among all the processing periods, a combination process of adding pixel values of the same coordinates between two or more acquired images and a thinning process of thinning out some acquired images for a plurality of acquired images included in the processing period The image processing method according to claim 1, wherein the output image is generated by performing processing including at least one.   The plurality of acquired images are an interlaced odd field image and an even field image, and pixel values of the same coordinates are weighted and added between one or more odd field images and one or more even field images as the combining process. 3. The image processing method according to claim 2, wherein the sum of the weights of the pixel values of all odd field images is equal to the sum of the weights of the pixel values of all even field images. .   The plurality of acquired images are an interlaced odd field image and an even field image, and as the combining process, an odd field combining process of adding pixel values of the same coordinates between two or more odd field images; 3. The image processing method according to claim 2, wherein an even field composition process of adding pixel values of the same coordinates between the even field images is performed.   The odd field composition process and the even field are processed so that a processing section of two or more odd field images that perform the odd field composition process and a processing section of two or more even field images that perform the even field composition process do not overlap. The image processing method according to claim 4, wherein synthesis processing is performed.   The odd field combining process and the odd field combining process so that the overlap between the processing section of two or more odd field images for performing the odd field combining process and the processing section of two or more even field images for performing the even field combining process is maximized. 5. The image processing method according to claim 4, wherein the even field synthesis process is performed. An image processing apparatus that acquires a plurality of images in which a shooting region is continuously captured in time series, performs image processing on the plurality of acquired images, and outputs the processed image as an output image to an image display unit. And
For each acquired image for the plurality of acquired images, the luminance of a predetermined portion is set as the representative luminance,
A predetermined period that is an integral multiple of the time variation period of the representative luminance is set as a processing period, and is determined at a timing synchronized with the period of the processing period among a plurality of acquired images included in the processing period in all the processing periods. An image processing apparatus comprising: processing means for extracting one or a plurality of acquired images to generate the output image, and sequentially outputting the output image to the image display means at a variation cycle of the representative luminance.   In all the processing periods, the processing means performs a combining process of adding pixel values of the same coordinates between two or more acquired images and a part of the acquired images for a plurality of acquired images included in the processing period. The image processing apparatus according to claim 7, wherein the output image is generated by performing processing including at least one of thinning processing. The plurality of acquired images are interlaced odd field images and even field images,
The processing means adds the pixel values of the same coordinates between two or more even field images and the odd field composition processing of adding the pixel values of the same coordinates between two or more odd field images as the composition processing. The image processing apparatus according to claim 8, wherein an even field synthesis process is performed.

Images