JP5860298B2 - Image processing apparatus and program - Google Patents

Description

  The present invention relates to an image processing apparatus and program for extending the dynamic range of video.

  As a technique for extending the dynamic range of video, there is a technique for capturing a plurality of images with different exposure amounts and combining them. At this time, there is a method of dividing a light path by a spectroscopic optical system, providing a photoelectric conversion means in each light path, and synthesizing images obtained by a plurality of photoelectric conversion means to obtain a high dynamic range image (Patent Document 1).

  In addition, there is a technique for achieving a high dynamic range by aligning and overlapping a plurality of video frames (Patent Document 2).

JP 2006-165826 A JP-T-2006-525747

  However, the conventional dynamic range expansion method using a spectroscopic optical system requires a special optical system, and cannot expand the dynamic range from an existing optical system or an already captured image.

  In addition, the dynamic range expansion method using a plurality of frames, which is a conventional technique that does not require a special optical system, has regularity in the luminance change of the entire image without considering the regularity. We tried to expand the dynamic range only from the luminance distribution function. Therefore, this technique is affected by sudden or local luminance changes.

  Therefore, the present invention has been made in view of the above problems, and an image processing apparatus and program capable of extending the dynamic range and improving the image quality by reducing the influence of sudden or local luminance change. The purpose is to provide.

  An image processing apparatus according to an aspect of the present invention provides an analysis unit that performs filter processing for extracting regularity with respect to temporal change in luminance using luminance between frames in an input video, and luminance of each frame Correcting means for correcting the value filtered by the analyzing means so that the luminance value distribution in each frame becomes a target distribution, and superimposing a plurality of frames whose luminance is corrected by the correcting means Means.

  The analysis unit may perform curve fitting on the average and / or variance of the luminance of each frame, and the correction unit may correct the average and / or variance of the luminance value of each frame.

  In addition, the superimposing means determines the validity of the superimposed frame, outputs the superimposed frame when it is determined to be valid, and the frame that is not superimposed when it is determined to be invalid. May be output.

  Further, the superimposing means determines that it is valid if the number of luminance levels of the superimposed frame exceeds a threshold value, and determines that it is invalid if the number of luminance levels of the superimposed frame is less than or equal to the threshold value. May be.

  Further, the program according to another aspect of the present invention includes an analysis step of performing a filtering process for extracting regularity with respect to temporal change of the luminance using luminance between frames in the input video, and luminance of each frame. A correction step for correcting the filtered value so that the luminance value distribution in each frame becomes a target distribution, and a superimposing step for superimposing the plurality of frames whose luminance has been corrected. To run.

  According to the present invention, it is possible to expand the dynamic range and improve the image quality by reducing the influence of sudden or local luminance change.

1 is a block diagram illustrating an example of a configuration of an image processing device according to Embodiment 1. FIG. The figure for demonstrating an analysis process. The figure for demonstrating a correction process. The figure for demonstrating the range conversion and image composition of a brightness | luminance. The figure for demonstrating the determination method 1 of effectiveness determination. The figure for demonstrating the determination method 2 of effectiveness determination. 6 is a flowchart illustrating an example of dynamic range expansion processing according to the first exemplary embodiment. FIG. 6 is a block diagram illustrating an example of a configuration of an image processing apparatus according to a second embodiment. 9 is a flowchart illustrating an example of dynamic range expansion processing according to the second embodiment. FIG. 9 is a block diagram illustrating an example of a configuration of an image processing apparatus according to a third embodiment.

  The present invention will be described below with reference to the accompanying drawings.

[Example 1]
<Configuration>
FIG. 1 is a block diagram illustrating an example of the configuration of the image processing apparatus 1 according to the first embodiment. An image processing apparatus 1 shown in FIG. 1 is an apparatus that expands the dynamic range of an input video by batch processing. Although the dynamic range can be expanded by sequential processing, an image processing apparatus in this case will be described in a second embodiment.

  Hereinafter, the video is also referred to as a moving image. Further, a frame (image) at the time t of the moving image X is expressed as X (t), and the pixel value at the image coordinates (x, y) of the frame at the time t of the moving image X is expressed as X (t; x, y).

  The image processing apparatus 1 includes an analysis unit 101, a frame memory 104, a luminance correction unit 105, an image superimposing unit 106, and a quantization unit 107. Note that the image processing apparatus 1 may be configured not to include the quantization unit 107.

  The analysis unit 101 performs filter processing for extracting regularity with respect to temporal change in luminance using luminance values between frames in the input video. For example, the analysis unit 101 analyzes a temporal change pattern of a luminance feature amount (for example, average luminance value, variance) possessed by each frame of the input video as a whole. The input video may be any video such as an original video, a quantized video, and a video input from an analog sensor.

  Specifically, the analysis unit 101 performs an operation of extracting regularity of a luminance feature amount or applying a temporal change of the luminance feature amount to a dynamics model. Therefore, the analysis unit 101 includes, for example, a luminance feature extraction unit 102 and a filter unit 103.

  The luminance feature extraction unit 102 obtains a representative value or distribution of luminance of each frame of the input video. A representative value or distribution of luminance at time t is expressed as luminance feature F (t).

  For example, the luminance feature extraction unit 102 obtains the sum (or average luminance value) in each frame of the input video. When the input video A is a monochrome image, the luminance feature extraction unit 102 obtains a luminance feature by, for example, the equation (1).

When the input video A is a color or the like and is composed of a plurality of color components and wavelength (band) components, the luminance feature extraction means 102, for example, for a pixel value A (t; x, y) as a vector value The luminance feature is obtained by the equation (2) or the equation (3).

Alternatively, a conversion matrix (row vector) M from a color vector to a luminance value

M: Conversion matrix from color vector to luminance value (row vector)
Here, it is assumed that the pixel value A (t; x, y) is a three-dimensional column vector, the first component represents the red component, the second component represents the green component, and the third component represents the blue component. . In this case, as the transformation matrix M, for example, the following ITU-R BT. A matrix (row vector) for luminance signal calculation defined in the 601 standard is used.

In addition, the following ITU-R BT. A matrix (row vector) for calculating a luminance signal defined in the 709 standard may be used.

Further, for example, the luminance feature extraction unit 102 may adopt a luminance histogram in the frame A (t) as the luminance feature F (t). The luminance feature extraction unit 102 outputs the extracted luminance feature F (t) to the filter unit 103.

  The filter unit 103 performs filter processing on the time series of the luminance feature F (t) acquired from the luminance feature extraction unit 102, for example, performs smoothing or curve fitting, and obtains the result G (t) of the filter processing. obtain.

  For example, the filter unit 103 applies a digital filter to the luminance feature F (t) by the following equation (4).

Here, v and _wτ are filter coefficients, and for example, predetermined constants are used. Further, a and b are time ranges where the filter is applied, and a ≦ b. The range of b−a includes, for example, 10 frames.

  The filter unit 103 may apply a Kalman filter, for example. Furthermore, the filter means 103 may apply curve fitting.

  When performing the curve fitting, the filter unit 103 sets the function to be fitted as f (p, t). It is assumed that the shape of this function can be adjusted by a parameter p (p is a scalar or a vector). As the parameter p, for example, an average value of luminance and a variance of luminance in each frame are used as a scalar, and an average of luminance and a luminance variance are used as vectors.

The filter means 103 obtains the optimum parameter p _opt in the vicinity of the time t (interval from t + a to t + b) by, for example, Expression (5).

For this optimization, for example, a nonlinear least square method such as the Levenberg-Marquardt method can be used. Subsequently, the filter unit 103 determines the filter result G (t) according to Expression (6).

As the function f (p, t) for curve fitting, for example, an M-order function (M is an integer of 0 or more; for example, M = 3) is used.

The parameter p can be defined as an M + 1 dimensional vector as follows:

The filter means 103 may determine the function f (p, t) for curve fitting based on a sine function as shown in the equation (8) when the luminance fluctuation is periodic.

In this case, the parameter p is a four-dimensional vector as follows.

The filter unit 103 outputs the filter result G (t) to the luminance correction unit 105.

The frame memory 104 stores frames of the input video in the past or current time point N (N is an integer of 2 or more). N points of time stored in the frame memory 104 are set as t ₁ , t ₂ ,..., T _N.

The luminance correction unit 105 analyzes the time points corresponding to the frames A (t ₁ ), A (t ₂ ),..., A (t _N ) stored in the frame memory 104 at the past or present time point _N. Based on the filter results G (t ₁ ), G (t ₂ ),..., G (t _N ) of the output from 101, the luminance value distribution is corrected to become a target distribution. Here, the correction results are set as correction frames H (t ₁ ), H (t ₂ ),..., H (t _N ).

For example, the luminance correction unit 105 performs a correction operation on the frame A (t _n ) at the time point t _n (n is an integer equal to or greater than 1 and equal to or smaller than N) using Equation (9).

The luminance correction unit 105 multiplies A (t _n ) by a gain such that the average of luminance values is the average of G (t ₁ ), G (t ₂ ),..., G (t _N ), and corrects the correction frame H ( t _n ). The luminance correction unit 105 outputs the corrected frame to the image superimposing unit 106.

The image superimposing means 106 superimposes the correction frames H (t ₁ ), H (t ₂ ),..., H (t _N ) to obtain the frame C (t _O ) of the composite video C at the time point t _O. Incidentally, the time _{t O} is the N _point, t _1, t 2, ..., any point having a predetermined index of _{t N} or _{t 1,} t 2,, ..., a minimum value or more and the maximum value of _{t N} A certain time point (for example, an average value of the minimum value and the maximum value) is set as follows.

For example, the image superimposing means 106 calculates the average value of the correction frames H (t ₁ ), H (t ₂ ),..., H (t _N ) at the N time points by using the following formula (10). _O ).

In addition, when the image superimposing means 106 calculates the composite video frame C (t _O ) using the average value, the luminance value before luminance correction is “the maximum luminance value (eg, 255 for 8-bit video), and When the luminance value after correction is not the maximum value at the synthesis target times t ₁ , t ₂ ,... _TN ”and the luminance value before luminance correction is“ the minimum luminance value (for example, 0) and One or both of the cases where the luminance value is not the minimum value at times t ₁ , t ₂ ,... T _{N to} be combined may be excluded.

For example, the image superimposing unit 106 obtains a composite video frame C (t ₀ ) by the following equation (11).

In this way, when the luminance value that is saturated in the input video or the luminance value that is at the 0 level becomes the maximum luminance or the minimum luminance as a result of luminance correction (when both ends of the range after luminance conversion) ), It can be excluded. The image superimposing unit 106 outputs the superimposed video to the quantizing unit 107. Note that the quantization means 107 is not always necessary.

  Note that the image superimposing unit 106 may determine the validity of the superimposed frame. For example, the image superimposing unit 106 determines that it is valid if the number of luminance levels of the superimposed frame exceeds a threshold, and determines that it is invalid if the number of luminance levels is equal to or less than the threshold. This determination of effectiveness will be described later.

The quantizing means 107 quantizes each pixel value of the composite video frame C and outputs the result as an output video frame (t _O ). Preferably, the output video is quantized with a number of quantization steps larger than the number of steps of quantization of the input video.

  By having the above configuration, the dynamic range can be expanded and the image quality can be improved by reducing the influence of sudden or local luminance change.

<Analysis processing and correction processing>
Next, analysis processing by the analysis unit 101 and correction processing by the luminance correction unit 105 will be described in detail.

  FIG. 2 is a diagram for explaining the analysis process. FIG. 2A shows an example of each frame of the input video. fm11 to 18 represent each frame of the input video. This image has periodic brightness. Such light and darkness is called flicker, and may occur when video is shot under an electric lamp that is lit with alternating current (particularly a fluorescent lamp). In this embodiment, the dynamic range can be suitably extended for an image shot in an environment where flicker occurs.

  In the example shown in FIG. 2A, the frames fm12, fm13, fm16, and fm18 include the noises ns21 to 24, respectively. This noise is, for example, sudden or local noise or a subject imaged instantaneously.

  FIG. 2B shows an example of a luminance histogram of the frames fm11-18. hg31 to 38 represent histograms. pk41 to 44 represent peaks due to the noise ns21 to 24.

  FIG. 2C shows the filtering process of the luminance feature of each image frame fm11 to 18 of the input video. The luminance feature in this example is, for example, an average value of luminance. Av51 to 58 represent the average value of the luminance of each frame. cv61 represents a curve when curve fitting is performed on the luminance features av51 to av58.

  For example, if the video shown in FIG. 2A is corrected for luminance by simply normalizing the average luminance value of the entire image frame, noise such as ns21 to 24 has an adverse effect on the calculation of the average luminance. The brightness of the subject in the area other than noise fluctuates.

  Therefore, the image processing apparatus 1 according to the first embodiment performs, for example, curve fitting as shown by a curve cv61 in FIG. 2C by the analysis unit 101 (filter unit 103). In the luminance correction unit 105, instead of using the luminance features (for example, average value of luminance) av 51 to 58 extracted by the luminance feature extraction unit 102 for luminance correction, the luminance correction unit 105 performs luminance correction on the value on the curve cv 61 obtained by the filter unit 103. Used for. Thereby, the bad influence of noise can be reduced.

  Therefore, the luminance correction means 105 is stored in the frame memory 104 so that the value becomes the target value (target distribution) while referring to the value at each time point of the curve cv61 shown in FIG. Convert the range of the input video frame at each time point.

  Here, for example, the luminance correction unit 105 performs the luminance correction so that the value at each time point of the curve cv61 becomes the average value of the curve cv61.

  FIG. 3 is a diagram for explaining the correction process. FIG. 3A is similar to FIG. 2A and represents the frames fm11 to 18 of the input video.

  FIG. 3B shows the corrected luminance feature. In the example shown in FIG. 3B, the histograms hg71 to 78 almost coincide with the histogram shapes other than the noise components. Note that the luminance feature to be corrected may correct not only the average luminance value but also the luminance dispersion.

  FIG. 3C shows the luminance correction result. Reference numerals fm81 to 88 denote frames as a result of luminance correction. As shown in FIG. 3C, each frame can have a stable luminance value.

<Brightness range conversion and image composition>
FIG. 4 is a diagram for explaining luminance range conversion and image composition. Image composition is used in the same meaning as image superposition. Black dots 91 and 92 shown in FIG. 4 indicate the luminance on a certain line on the same subject captured at another time point.

  It is assumed that any luminance is quantized in the luminance direction (vertical axis direction). The image including the luminance 92 is captured approximately 4/3 times brighter than the image including the luminance 91. Here, it is assumed that the gain at the time of correcting the luminance 91 is 1.00 and the gain at the time of correcting the luminance 92 is 0.75.

  The correction results of the luminance 91 and the luminance 92 are shown as a luminance 93 and a luminance 94. After the luminance correction by the luminance correction unit 105, the luminance value is held by finer quantization than the original quantization step. In the case of the luminance 94, the interval between the values that the luminance value can take is 0.75 times the interval between the luminance values of the input video.

  The image superimposing means 106 calculates an average value of the two luminances 93 and 94 after the luminance correction, and generates one frame of the synthesized video C including the luminance 95 as a result. The composite video C can express more abundant luminance values than, for example, the case of an image alone including the luminance 91.

  Since more abundant luminance values can be expressed, the quantization means 107 may preferably perform quantization with a larger number of steps than the input video. As a result, the dynamic range of the video can be expanded. Note that the quantization by the quantization means 107 is not necessarily a necessary process.

<Effectiveness determination of superimposed image>
Next, the effectiveness determination for the superimposed image will be described. The image superimposing means 106 may output a superimposed image if the image superimposition is valid, and may output an image that is not superimposed if the image superimposition is invalid (not valid). Further, the image superimposing means 106 may output the validity determination result together with the superimposed image.

(Judgment method 1)
First, the determination method 1 for determining the effectiveness will be described. In the determination method 1, the number of luminance levels in one frame of the composite video C is used.

  FIG. 5 is a diagram for explaining a determination method 1 for determining validity. FIG. 5A shows a case where the number of luminance levels in one frame of the composite video C is eight. FIG. 5B shows a case where the number of luminance levels in one frame of the composite video C is 11.

  In the example shown in FIG. 5, the frame shown in FIG. 5B has richer gradation expression than the frame shown in FIG. That is, it can be said that the frame shown in FIG. 5B has a wider dynamic range than the frame shown in FIG.

  Here, when determining the validity of one frame of the composite video C, the image superimposing unit 106 determines “effective” if the number of appearing luminance levels exceeds the threshold 1. The image superimposing unit 106 determines “invalid” if the number of appearing luminance levels is equal to or less than the threshold value 1. As the threshold value 1, for example, the number of luminance levels that appear in the input video or a preset number may be used. This determination method 1 can be said to be a practical and appropriate method for determining the effectiveness.

(Judgment method 2)
Next, the determination method 2 for determining the effectiveness will be described. The determination method 2 is a method of obtaining a quantization level after luminance conversion (luminance correction) of one frame of the synthesized video C and obtaining a luminance level that can be taken by an average value of the frames to be synthesized.

  FIG. 6 is a diagram for explaining a determination method 2 for determining validity. FIG. 6A shows an example 1 of possible values of the luminance value of one frame (composite frame) of the composite video. FIG. 6B shows Example 2 of possible values of the luminance value of the composite frame. FIG. 6C shows an example 3 of possible values of the luminance value of the composite frame. Black circles shown in FIG. 6 represent possible values of the luminance value of the composite frame.

  The image superimposing unit 106 determines that the validity is “invalid” when the possible value of the luminance value of the composite frame is equal to or less than the threshold value 2. The image superimposing means 106 determines that the validity is “valid” when the possible value of the luminance value of the combined frame is greater than the threshold value 2. The threshold 2 is, for example, a value that can be taken by the luminance value of the input video or a preset value.

  For example, one frame of the superimposed synthesized video shown in FIG. 6C is determined to be “invalid”, and one frame of the superimposed synthesized video in FIG. 6A or 6B is It is determined to be “valid”.

  As described above, the image superimposing unit 106 determines whether the superimposed image is valid, thereby switching the ON / OFF of whether or not to output the superimposed image, and outputs the validity determination result. Can be.

<Operation>
Next, the operation of the image processing apparatus 1 in the first embodiment will be described. FIG. 7 is a flowchart illustrating an example of dynamic range expansion processing according to the first embodiment. The process shown in FIG. 7 is a batch process.

In step S101, the image processing apparatus 1 initializes a variable n and sets n to 0. In step S102, the image processing apparatus 1 determines whether or not the following expression (12) is satisfied.
n% N == N−1 (12)
N: Number of frames to be superimposed n% N: Remainder of dividing n by N If equation (12) is satisfied (step S102—YES), the process proceeds to step S108, and if equation (12) is not satisfied (step S102—NO), step The process proceeds to S103. When the expression (12) is satisfied, it means that the number N of frames to be superimposed is stored in the frame memory 104.

  In step S103, the frame memory 104 stores the input video in the storage area M (n% N) of the n% Nth frame.

  In step S104, the luminance feature extraction unit 102 extracts the luminance feature of the input video n. The luminance feature is an average value and / or variance of the luminance of the frame.

  In step S105, the filter unit 103 performs a filtering process on the extracted luminance feature. The filter process is, for example, the curve fitting described above. Note that the processes in steps S104 and S105 are processes for analyzing the regularity of the luminance of the input video.

  In step S <b> 106, the image processing apparatus 1 adds 1 to n and performs processing for the next frame.

  In step S107, the image processing apparatus 1 determines whether or not the processing of all the videos has been completed. If all the video processing has been completed (step S107—YES), the processing is terminated. If all video processing has not been completed (step S107—NO), the processing returns to step S102.

  In step S108, the brightness correction unit 105 initializes parameters, sets k to 0, and sets C to 0. C represents the luminance value of all pixels.

  In step S109, the brightness correction unit 105 corrects the brightness of each pixel of the video M (k) of the k-th frame in the frame memory 104 based on the filter processing result (formula (9)). To do.

  In step S110, the image superimposing unit 106 adds and accumulates the luminance of H to the luminance of the corresponding pixel of C, and adds 1 to k. In step S111, the image superimposing unit 106 determines whether k <N. If k <N (step S111-YES), the process returns to step S109, and if k <N (step S111-NO), the process proceeds to step S112.

  In step S112, the image superimposing means 106 sets the image obtained by dividing C by N and averaging it as a new C.

  In step S113, the quantization unit 107 quantizes C and outputs the quantized result as an output video. Note that the quantization in step S113 is not necessarily a necessary process, and the image superimposing unit 106 may output an image on which the image is superimposed as an output video.

  Further, the image superimposing unit 106 may determine the validity of the superimposed image, and may control the output based on the determination result or output the determination result.

  As described above, according to the first embodiment, it is possible to improve the image quality by extending the dynamic range by reducing the influence of sudden or local luminance change.

[Example 2]
Next, the image processing apparatus 2 according to the second embodiment will be described. In the second embodiment, the dynamic range expansion process is performed sequentially.

<Configuration>
FIG. 8 is a block diagram illustrating an example of the configuration of the image processing apparatus 2 according to the second embodiment. In the example illustrated in FIG. 8, the image processing apparatus 2 includes an analysis unit 201, a luminance correction unit 205, a frame memory 208, an image superimposing unit 206, and a quantization unit 207. Note that the quantization unit 207 is not necessarily required, and thus the image processing apparatus 2 may not include the quantization unit 207.

  Since the analysis unit 201 and the quantization unit 207 perform basically the same processing as in the first embodiment, the description thereof is omitted.

  The luminance correction unit 205 performs luminance correction on each frame of the input video based on the filter processing result. The brightness correction process and the like are the same as those in the first embodiment. The brightness correction unit 205 stores each frame whose brightness has been corrected in the frame memory 208.

  The frame memory 208 stores the frame corrected by the luminance correction unit 205. The image superimposing unit 206 sequentially superimposes the N frames stored in the frame memory 208 to generate a composite image and outputs it to the quantization unit 207.

  Thus, it is possible to perform image brightness correction processing, superimposition processing, and the like by sequential processing.

<Operation>
Next, the operation of the image processing apparatus 2 in the second embodiment will be described. FIG. 9 is a flowchart illustrating an example of dynamic range expansion processing according to the second embodiment. The process shown in FIG. 9 is a sequential process.

  Steps S201 to S203 are the same as steps S101, S104, and S105 shown in FIG.

  In step S204, the luminance correction unit 205 corrects the luminance of each frame of the input video based on the filter processing result, and the correction result is stored in the storage area M (n% N) of the n% Nth frame of the frame memory 208. To remember.

  In step S205, the image superimposing unit 206 determines whether n ≧ N−1. If n ≧ N−1 (step S205—YES), the process proceeds to step S206, and if n ≧ N−1 (step S205—NO), the process proceeds to step S211.

  This is a process of determining whether the number of frames to be superimposed on the frame memory 208 is stored.

  In step S206, the image superimposing unit 206 sets k = 0 and sets C to 0. C represents the luminance value of all pixels.

  In step S207, the image superimposing unit 206 adds the luminance of the video M (k) of the k-th frame in the frame memory 208 to the luminance of the corresponding pixel of C and accumulates, so that k = k + 1.

  In step S208, the image superimposing unit 206 determines whether k <N. If k <N (step S208—YES), the process returns to step S207, and if k <N (step S208—NO), the process proceeds to step S209.

  The processing of steps S209-S210 is the same as the processing of steps S112-S113 shown in FIG.

  The processing of steps S211 to S212 is the same as the processing of steps S106 to S107 shown in FIG.

  Note that the quantization in step S210 is not necessarily a necessary process, and the image superimposing unit 106 may output an image on which the image is superimposed as an output video.

  Further, the image superimposing unit 206 may determine the validity of the superimposed image, and may control the output based on the determination result or output the determination result.

  In the sequential process shown in FIG. 9, the process of sequentially superimposing the latest N frames stored in the frame memory 208 after the first N frames has been described. Further, the image superimposing unit 206 may superimpose an image with the number of frames stored in the frame memory 208 up to the first N frames.

  As described above, according to the second embodiment, the batch processing of the first embodiment can be performed by sequential processing. Therefore, the dynamic range can be expanded by reducing the influence of sudden or local luminance change, and the image quality can be improved.

[Example 3]
FIG. 10 is a block diagram illustrating an example of the configuration of the image processing apparatus 3 according to the third embodiment. An image processing apparatus 3 illustrated in FIG. 10 is an example of an apparatus in which the image processing described in the first and second embodiments is implemented by software.

  As illustrated in FIG. 10, the image processing apparatus 3 includes a control unit 301, a main storage unit 302, an auxiliary storage unit 303, a drive device 304, a network I / F unit 306, an input unit 307, and a display unit 308. These components are connected to each other via a bus so as to be able to transmit and receive data.

  The control unit 301 is a CPU that controls each device, calculates data, and processes in a computer. The control unit 301 is an arithmetic device that executes an image encoding processing program stored in the main storage unit 302 or the auxiliary storage unit 303. The control unit 301 receives data from the input unit 307 and the storage device, calculates and processes the data, and outputs the data to the display unit 308 and the storage device.

  The control unit 301 can realize the processing described in each embodiment by executing the above-described image processing program.

  The main storage unit 302 is a ROM (Read Only Memory), a RAM (Random Access Memory), or the like. The main storage unit 302 is a storage device that stores or temporarily stores programs and data such as OS (Operating System) and application software that are basic software executed by the control unit 301.

  The auxiliary storage unit 303 is an HDD (Hard Disk Drive) or the like, and is a storage device that stores data related to application software and the like.

  The drive device 304 reads the program from the recording medium 305, for example, a flexible disk, and installs it in the storage unit.

  A predetermined program is stored in the recording medium 305, and the program stored in the recording medium 305 is installed in the image processing apparatus 3 via the drive device 304. The installed predetermined program can be executed by the image processing apparatus 3.

  The network I / F unit 306 is a peripheral having a communication function connected via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network) constructed by a data transmission path such as a wired and / or wireless line. This is an interface between the device and the image processing apparatus 3.

  The input unit 307 includes a keyboard having cursor keys, numeric input, various function keys, and the like, a mouse and a slice pad for selecting keys on the display screen of the display unit 308, and the like. The input unit 307 is a user interface for a user to give an operation instruction to the control unit 301 or input data.

  The display unit 308 is configured by a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), or the like, and performs display according to display data input from the control unit 301.

  The frame memory described in the first and second embodiments can be realized by the main storage unit 302 or the auxiliary storage unit 303, for example. Further, configurations other than the frame memory described in the first and second embodiments can be realized by the control unit 301 and the main storage unit 302 as a work memory, for example.

  The program executed by the image processing apparatus 3 has a module configuration including each unit described in the first and second embodiments. As actual hardware, when the control unit 301 reads out and executes a program from the auxiliary storage unit 303, one or more of the above-described units are loaded onto the main storage unit 302, and one or more of the respective units are loaded. Are generated on the main storage unit 302.

  As described above, the image processing (dynamic range expansion processing) described in the first and second embodiments may be realized as a program for causing a computer to execute the image processing. The processing described in each embodiment can be realized by installing this program from a server or the like and causing the computer to execute the program.

  It is also possible to record the program on the recording medium 305 and cause the computer or portable terminal to read the recording medium 305 on which the program is recorded to realize the above-described image processing. The recording medium 305 is a recording medium that records information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, or a magneto-optical disk, and information is electrically stored such as a ROM or flash memory. Various types of recording media such as a semiconductor memory for recording can be used.

  Note that various integrated circuits and electronic circuits can be adopted as the means described in the first and second embodiments. Further, a part of each means described in the first embodiment can be replaced with another integrated circuit or electronic circuit. For example, examples of the integrated circuit include ASIC (Application Specific Integrated Circuit) and FPGA (Field Programmable Gate Array). Examples of the electronic circuit include a central processing unit (CPU) and a micro processing unit (MPU).

  Each embodiment has been described in detail above. However, the present invention is not limited to the specific embodiment, and various modifications and changes other than the above-described modification are possible within the scope described in the claims. .

1, 2, 3 Image processing apparatus 101, 201 Analysis means 102, 202 Luminance feature extraction means 103, 203 Filter means 104, 208 Frame memory 105, 206 Brightness correction means 106, 206 Image superimposition means 107, 207 Quantization means

Claims (5)

Using the luminance between frames in the input video, and analysis means that to analyze the regularity against the time change of the luminance,
Correction means for correcting the luminance value distribution of each frame based on the regularity so that the luminance value distribution in each frame becomes a target distribution;
Superimposing means for superimposing a plurality of frames whose luminance is corrected by the correcting means;
Equipped with a,
The image processing apparatus according to claim 1, wherein the correcting unit corrects one or more frames so that an interval between possible values of luminance in the frame is reduced . The analysis means includes
Perform curve fitting for the average and / or variance of the brightness of each frame,
The correction means includes
The image processing apparatus according to claim 1, wherein the average and / or variance of the luminance value of each frame is corrected. The superimposing means includes
The validity of the superimposed frame is determined, the superimposed frame is output when it is determined to be valid, and the non-superimposed frame is output when it is determined to be invalid. 2. The image processing apparatus according to 2. The superimposing means includes
The image according to claim 3, wherein the image is determined to be valid if the number of luminance levels of the superimposed frame exceeds a threshold value, and is determined to be invalid if the number of luminance levels of the superimposed frame is equal to or less than the threshold value. Processing equipment. An analysis step of analyzing regularity with respect to temporal change of the luminance using luminance between frames in the input video;
Regarding the luminance of each frame , based on the regularity , the luminance value distribution in each frame becomes a target distribution , and the interval of possible values of luminance in the frame is reduced for one or more frames. and a correction step of correcting, as,
A superimposing step of superimposing a plurality of frames whose luminance has been corrected;
A program that causes a computer to execute.