JP2008252734A - Image processing apparatus, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of displaying a perspective video image. <P>SOLUTION: An enhance correction unit 11 applies, to an input signal T0, enhance processing corresponding to a moving amount MV and outputs a resultant signal as an output signal T3. An LPF section 22 of an enhance suppression correction unit 12 applies LPF processing to the input signal T0 and outputs a resultant signal as an output signal B1. In that case, a tap amount variable section 21 varies a tap length of the LPF section 22 in accordance with the moving amount MV. When it is judged by a moving amount gap judgement unit 13 that large and small moving amounts MV are mixed, a switch unit 14 outputs the output signal T3 of the enhance correction unit 11 for a portion of the large moving amount MV in a frame image and outputs the output signal B1 of the enhance suppression correction unit 12 for a portion of the small moving amount therein, respectively, as an output signal T4. The present invention may be applicable to television systems. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, method, and program, and more particularly, to an image processing apparatus, method, and program that can execute image processing that can contribute to creation of a perspective video.

A video signal of a moving image captured by a camera has image degradation caused by motion blur (hereinafter referred to as imaging blur) that occurs at the time of imaging, that is, image degradation that appears as a so-called blurred image. There are many. Therefore, there exists an image processing apparatus that performs image processing for suppressing imaging blur on such a video signal. For example, the image processing apparatus described in Patent Document 1 is applicable. Also, an image processing apparatus that can solve the problems of Patent Document 1, including a specification attached to the application of Japanese Patent Application No. 2005-319017 or Japanese Patent Application No. 2004-234051 already invented by the present inventors. The image processing apparatus described in the above also falls under this category.
Japanese Unexamined Patent Publication No. 7-59054

  However, when a video signal after image processing by such an image processing apparatus is provided to a display or the like, there is a problem that a video with poor perspective may be displayed on the screen of the display or the like. .

  The present invention has been made in view of such circumstances, and makes it possible to execute image processing that can contribute to the creation of a perspective video.

  An image processing apparatus according to an aspect of the present invention is an image processing apparatus that performs processing on each of a plurality of access units that configure a moving image, and for each pixel that configures the access unit to be processed, First correction means for correcting the access unit to be processed is provided by performing low pass filter processing with a tap length corresponding to the magnitude of the movement amount.

  A second correcting unit that corrects the access unit to be processed by performing an enhancement process to enhance each pixel constituting the access unit to be processed according to the amount of movement; When the access unit to be processed includes a portion having a first movement amount and a portion having a second movement amount larger than the first movement amount, the access unit to be processed is When outputting, the correction result of the second correction unit is output for the portion having the first movement amount, and the correction result of the first correction unit is output for the portion having the second movement amount. And switch means for performing.

  An image processing method and program according to one aspect of the present invention are a method and program corresponding to the above-described image processing apparatus according to one aspect of the present invention.

  In the image processing apparatus, method, and program according to one aspect of the present invention, the following processing is performed on each of a plurality of access units constituting a moving image. That is, the access unit to be processed is corrected by applying low pass filter processing to each pixel constituting the access unit to be processed with a tap length corresponding to the amount of movement. .

  As described above, according to the present invention, it is possible to realize image processing for a moving image. In particular, it is possible to realize image processing that can contribute to display a perspective moving image (video).

  DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below. Correspondences between constituent features described in claims and specific examples in the specification or the drawings are exemplified as follows. This description is intended to confirm that specific examples supporting the invention described in the claims are described in the specification or the drawings. Accordingly, even if there are specific examples that are described in the specification or drawings but are not described here as corresponding to the configuration requirements, the specific examples are not included in the configuration requirements. It does not mean that it does not correspond to. On the contrary, even if a specific example is described here as corresponding to a configuration requirement, this means that the specific example does not correspond to a configuration requirement other than the configuration requirement. not.

  Further, this description does not mean that all the inventions corresponding to the specific examples described in the specification or the drawings are described in the claims. In other words, this description is an invention corresponding to a specific example described in the specification or drawings, and there is an invention that is not described in the claims of this application, that is, a divisional application may be made in the future. The existence of an invention added by amendment is not denied.

An image processing apparatus according to one aspect of the present invention (for example, the image processing apparatus in FIG. 1)
In an image processing apparatus that performs processing on each of a plurality of access units constituting a moving image,
First correction means for correcting the access unit to be processed by applying low pass filter processing to each pixel constituting the access unit to be processed with a tap length corresponding to the amount of movement (For example, the enhancement suppression correction unit 12 in FIG. 1 is a tap amount variable unit 21 that outputs a signal TP indicating the tap length according to the movement amount MV, and an LPF unit 22 that varies the tap length based on the signal TP. Enhancement suppression correction unit 12)
Is provided.

Second correction means for correcting the access unit to be processed (for example, by performing enhancement processing for enhancing each pixel constituting the access unit to be processed according to the amount of movement thereof, for example, 1 is an enhancement correction unit 11 shown in FIG. 1, specifically, for example, an enhancement correction unit 11 having the configuration shown in FIG.
When the access unit to be processed includes a portion having a first movement amount and a portion having a second movement amount larger than the first movement amount, the access unit to be processed is When outputting, the correction result of the second correction unit is output for the portion having the first movement amount, and the correction result of the second correction unit is output for the portion having the first movement amount. The image processing apparatus according to claim 1, further comprising: switch means (for example, a switch unit 14 in FIG. 1) for performing the operation.

An image processing method according to one aspect of the present invention (for example, the method based on image processing in FIG. 8) includes:
An image processing method of an image processing apparatus (for example, an image processing apparatus having the enhancement suppression correction unit 12 of FIG. 1) that performs processing on each of a plurality of access units constituting a moving image,
Each pixel constituting the access unit to be processed is subjected to low pass filter processing with a tap length corresponding to the amount of movement, thereby correcting the access unit to be processed (for example, FIG. 8). Step S2)
Including steps.

  A program according to one aspect of the present invention is a program corresponding to the above-described image processing method according to one aspect of the present invention. Although details will be described later, this program is recorded in a recording medium such as a removable medium 111 in FIG. 9 or a hard disk included in the storage unit 108 and executed by a computer having the configuration in FIG.

  In addition, as one aspect of the present invention, a recording medium on which the program according to one aspect of the present invention described above is recorded is also included.

  The image processing apparatus according to one aspect of the present invention described above can be used as, for example, the entire television system or one component thereof. The television system refers to a system composed of one or more AV (Audio and Visual) devices including a television broadcast receiver.

  Next, embodiments of the present invention will be described with reference to the drawings.

  In the following description, it is assumed that the following assumptions are satisfied.

  That is, in various embodiments to be described later, various image processes for moving images are performed in units of access units. An access unit refers to a unit of a moving image such as a frame or a field, and specifically refers to, for example, the entire still image (frame or the like) or a part (field or the like) constituting the moving image. However, hereinafter, for the sake of simplicity, it is assumed that a unit of various image processing for a moving image is a frame unit.

  In addition, as a method for expressing a moving image (video) in which image processing is performed, the name of signal or data is used.

  Furthermore, signals input to each functional block (including an arithmetic unit such as an adding unit) constituting an image processing apparatus (see FIG. 1) described later are collectively referred to as an input signal as appropriate. That is, the input signal is appropriately referred to as an input signal collectively regardless of input units such as a moving image, each frame constituting the moving image, and each pixel (each pixel value) constituting each frame. Similarly, signals output from each functional block are collectively referred to as output signals as appropriate regardless of their output units. In other words, when it is necessary to distinguish between the input unit and the output unit, the description will be made using that unit (mainly the frame or the pixel value), and in other cases, the description will be made simply using the input signal or the output signal.

  FIG. 1 shows an embodiment of an image processing apparatus to which the present invention is applied.

  The image processing apparatus in the example of FIG. 1 includes an enhancement correction unit 11, an enhancement suppression correction unit 12, a movement amount gap determination unit 13, and a switch unit 14.

  In the image processing apparatus of the example of FIG. 1, each frame constituting a moving image becomes a processing target frame sequentially, and the pixel value of each pixel constituting the processing target frame is set as an input signal T0, and the enhancement correction unit 11 and the enhancement suppression. Provided to each of the correction units 12.

  Further, although details will be described later, it is assumed that the movement amount MV is given from the outside assuming that it is known. That is, for the input signal T0, a device for calculating or detecting the movement amount MV exists outside, and the movement amount MV is given from such a device. Of course, the functions of such an apparatus can be delegated to the image processing apparatus in the example of FIG.

  In the following description, in order to facilitate understanding of the present invention, the input signal T0 may mean an aggregate of pixel values (frame image data) of each pixel constituting the processing target frame. The pixel value of a pixel to be noted as a processing target among these pixels (hereinafter referred to as a noticed pixel) may be meant. However, unless otherwise specified, it is assumed that the input signal T0 means an aggregate (frame image data) of pixel values of each pixel constituting the processing target frame.

  Here, when the input signal T0 is frame image data about a moving image captured by an imaging device such as a camera, the input signal T0 includes motion blur about a subject (such as an object) generated during imaging by the camera. That is, an imaging blur may be included.

  Therefore, the enhancement correction unit 11 corrects the input signal T0 (each pixel value of the processing target frame) so as to suppress the imaging blur, and provides the resultant signal to the switch unit 14 as the output signal T3.

  Specifically, as shown in FIG. 2, for example, the enhancement correction unit 11 includes an HPF (High Pass Filter) unit 31, a gain variable unit 32, a multiplication unit 33, an addition unit 34, and a tap amount variable unit 35. Can do.

  The input signal T0 is input to the HPF unit 31 and the adding unit 34. The HPF unit 31 performs HPF processing on the input signal T0, and provides a signal obtained as a result to the multiplication unit 33 as an output signal T1. The multiplier 33 multiplies the output signal T1 of the HPF unit 31 by the gain G1 provided from the gain variable unit 32, and provides the resulting signal G1 × T1 to the adder 34 as the output signal T2. The adder 34 adds the output signal T2 of the multiplier 33 to the input signal T0, and provides the resultant signal T0 + T2 as the output signal T3 to the switch unit 14 (FIG. 1).

  The gain G1 is varied for each pixel by the gain varying unit 32 in accordance with the movement amount MV. Specifically, for example, the gain variable unit 32 holds a function f1 (α) as shown in FIG. Therefore, the gain variable unit 32 substitutes the movement amount MV at the target pixel as the input value α into the function f1 (α), and provides the output value f1 (MV) to the multiplication unit 33 as the gain G1 for the target pixel. .

  Further, the tap length of the HPF unit 31 is variably adjusted by the tap amount variable unit 35 so as to increase as the movement amount MV increases for each pixel. That is, the tap amount variable unit 35 holds a function f2 (α) as shown in FIG. 4, for example. Therefore, the tap amount variable unit 35 substitutes the movement amount MV at the target pixel as the input value α into the function f2 (α), and uses the output value f2 (MV) as the signal TP1 indicating the tap length for the target pixel as HPF. Provided to part 31. The tap length of the HPF unit 31 is variably adjusted according to the magnitude of the output signal TP1 of the tap amount variable unit 35.

  Hereinafter, the operation of the enhancement correction unit 11 will be described.

  The characteristic of imaging blur is expressed as LPF (Low Pass Filter). In other words, the image signal after imaging blur is a signal equivalent to a signal obtained by applying a low-pass filter to the image signal (ideal image signal) before imaging blur. Therefore, the frequency characteristic of the image signal after the imaging blur is deteriorated as compared with the image signal before the imaging blur. That is, in the image signal after the imaging blur, the gain generally decreases as the frequency becomes higher than the image signal before the imaging blur.

  Therefore, each pixel constituting the input signal T0 including the imaging blur, that is, each pixel constituting the processing target frame is set as the target pixel, and the frequency characteristic that has been dropped due to the imaging blur with respect to the pixel value of the target pixel. By performing a process of adding an amount corresponding to the improved amount as an enhancement amount (correction amount), it is possible to suppress imaging blur. Hereinafter, such processing is referred to as enhancement processing.

  A functional block that generates this enhancement amount as a signal is the HPF unit 31. That is, the level of the output signal T1 of the HPF unit 31 basically becomes the enhancement amount.

  The reason why “basically” is described here is as follows.

  That is, the characteristics of imaging blur are not always the same, and generally vary according to the amount of movement of a subject (such as an object). That is, the characteristics of imaging blur can be generally expressed in a form depending on the amount of movement of a subject (such as an object).

  Note that the amount of movement of the subject is the amount of movement of the subject (image) within the frame when the subject is captured by the camera when the subject moves in real space and the camera is fixed. Of course included. Further, the amount of movement of the subject here means that the subject is fixed when the subject is fixed in real space and the camera moves due to camera shake or when the camera moves following the subject in real space. It also includes the relative amount of movement of the subject (image) within the frame when taken by the camera.

  Therefore, the characteristics of the imaging blur can be expressed in a form depending on the movement amount in each pixel constituting the image of the subject.

  The movement amount of the target pixel refers to a spatial distance between the target pixel in the processing target frame and a corresponding pixel (corresponding point) in the previous frame in time. For example, the spatial distance between the target pixel in the processing target frame and the corresponding pixel (corresponding point) in the immediately preceding frame (temporarily before) is MV (MV is 0 or more). In the case of an arbitrary integer value) pixel, the movement amount in the target pixel is MV [pixel / frame].

  Hereinafter, for the sake of simplicity, it is assumed that the movement amount MV is a predetermined direction in the frame image, for example, the horizontal direction. Of course, the movement amount MV is not necessarily in a predetermined direction, but the following discussion and processing can be basically applied regardless of which direction the movement amount MV is.

  Here, if a predetermined one of the pixels constituting the subject image is set as the target pixel, the characteristic of the imaging blur at the target pixel is the magnitude of the movement amount MV [pixel / frame] at the target pixel. It can be expressed in a form dependent on

  More specifically, for example, when the movement amount MV at the target pixel is 2, 3, 4 [pixel / frame], the frequency characteristics of the imaging blur at the target pixel are the curves H2 to H4 in FIG. Can be expressed as

  That is, FIG. 5 shows each of the frequency characteristics of the imaging blur at the target pixel when the movement amount MV at the target pixel is 2, 3, 4 [pixel / frame]. In FIG. 5, the horizontal axis represents frequency, and the vertical axis represents gain. However, each value on the horizontal axis represents a relative value when the Nyquist frequency is 1.

  As can be seen from the characteristics of the imaging blur in FIG. 5, the imaging blur amount increases as the movement amount MV increases. On the other hand, as the movement amount MV decreases, the imaging blur amount decreases.

  That is, in the output signal T1 of the HPF unit 31, the imaging blur characteristic depending on the movement amount MV is not yet taken into consideration. In other words, the level of the output signal T1 of the HPF unit 31 cannot be said to be an amount corresponding to the frequency characteristic that has fallen due to the imaging blur.

  Therefore, in order to take into account such imaging blur characteristics, the multiplication unit 33 multiplies the output signal T1 of the HPF unit 31 by a gain G1 that varies according to the amount of movement MV, and G1 × T1 obtained as a result is obtained. It is output as the output signal T2. That is, the level of the output signal T2 of the multiplication unit 33 is an amount corresponding to the improvement of the frequency characteristic that has been dropped due to the imaging blur, and is provided to the addition unit 34 as an enhancement amount (correction amount).

  Therefore, in the adder 34, the level of the output signal T2 of the multiplier 33 is added to the level of the input signal T0 as an enhancement amount (correction amount), thereby making it possible to suppress imaging blur. . That is, the output signal T3 (= T0 + T2) of the adding unit 34 is provided to the switch unit 14 in FIG. 1 as a signal corrected to suppress imaging blur.

  However, when the output signal T3 of the enhancement correction unit 11 is given to a display or the like as the final output signal of the image processing apparatus, the above-described problem may occur in the [Problems to be solved by the invention] column. is there. Hereinafter, the cause of the problem will be described.

  As is apparent from the configuration of FIG. 2, the enhancement correction unit 11 performs enhancement processing on the entire input signal T0. That is, the enhancement processing is performed on all the pixels constituting the processing target frame.

  Here, for example, a frame including a foreground image moving with a small movement amount MVA and a background image moving with a large movement amount MVb is considered as a processing target frame. Specifically, for example, when an imager captures an object moving in the real world while chasing with a camera lens, the signal of each frame constituting the moving image obtained by the imaging becomes the input signal T0 It is. In other words, since the camera is moving following the subject, the relative speed of the subject with respect to the camera is slow (0 if it is able to follow completely). On the other hand, a background that is stationary in the real world is moving at the speed of the subject (following speed to the subject) when viewed from a camera that moves following the subject. That is, the relative speed of the background with respect to the camera is faster than the relative speed of the subject with respect to the camera. Therefore, when the image is captured by such a camera, the pixel group corresponding to the subject moving in the real world moves with a small movement amount MVA in the processing target frame corresponding to the input signal T0. In addition, a pixel group corresponding to a stationary background in the real world is a background image that moves with a large movement amount MVb.

  In this case, as is apparent from the property of the function f1 (α) in FIG. 3, the gain G1b for the background image moving with the large movement amount MVb is the gain G1a for the foreground image moving with the small movement amount MPa. Bigger than. Therefore, the enhancement amount for the background image moving with the large movement amount MVb (the level of the output signal T2 of the multiplication unit 33) is larger than the enhancement amount for the foreground image moving with the small movement amount MVA. That is, the background image moving with the large movement amount MVb is enhanced than the foreground image moving with the small movement amount MVA, and as a result, the processing target frame displayed on the display is In this case, the background image appears closer to the foreground image, and the sense of depth disappears. The above contents are the cause of the problem described above.

  Therefore, in order to solve such a problem, the image processing apparatus in the example of FIG. 1 is provided with an enhancement suppression correction unit 12, a movement amount gap determination unit 13, and a switch unit 14 in addition to the enhancement correction unit 11. -ing

  The enhancement suppression correction unit 12 includes a tap amount variable unit 21 and an LPF unit 22.

  The LPF unit 22 performs LPF processing on the input signal T0 (each pixel value of the processing target frame), and provides the resulting signal to the switch unit 14 as an output signal B1.

  The tap length of the LPF unit 22 is variably adjusted by the tap amount variable unit 21 so as to increase as the movement amount MV increases for each pixel. That is, the tap amount variable unit 21 holds a function g (β) as shown in FIG. 6, for example. Therefore, the tap amount variable unit 21 substitutes the movement amount MV at the target pixel as the input value β into the function g (β), and outputs the output value g (MV) as a signal TP indicating the tap length for the target pixel as an LPF. Part 22 is provided. The tap length of the LPF unit 22 is variably adjusted according to the magnitude of the output signal TP of the tap amount variable unit 21.

  The movement amount gap determination unit 13 uses, for example, the movement amounts MV in each pixel constituting the processing target frame (the frame corresponding to the input signal T0), for example, the movement amounts in the four corners and the center of the entire image of the processing target frame. Based on the comparison, a difference between a small movement amount MV and a large movement amount MV is calculated, and a difference signal indicating the calculation result is output to the switch unit 14 as an output signal D1. provide. Hereinafter, the process of the movement amount gap determination unit 13 is referred to as movement amount MV gap determination in the processing target frame.

  The switch unit 14 holds a function h (γ) as shown in FIG. 7, for example. Therefore, the switch unit 14 substitutes the output signal D1 of the movement amount gap determination unit 13 as the input value γ into the function h (γ), and calculates the output value h (D1) as the MIX ratio A. Next, the switch unit 14 uses the MIX ratio A, the output signal T3 of the enhancement correction unit 11 and the output signal B1 of the enhancement suppression correction unit 12 to calculate an equation of A × T3 + (1.0 A) × B1. The frame image data composed of the pixel values output by the calculation is output as the output signal T4.

  That is, when it is determined from the output signal D1 of the movement amount gap determination unit 13 that a small movement amount MV and a large movement amount MV are mixed in the processing target frame (frame corresponding to the input signal T0), Among the processing target frames, the output signal T3 of the enhancement correction unit 11 is adopted for the portion where the movement amount MV is small, while the output signal B1 of the enhancement suppression correction unit 12 is adopted for the portion where the movement amount MV is large. As a result, the output signal T4 of the processing target frame is generated.

  More specifically, when the processing target frame is composed of a background image that moves with a large movement amount MVb and a foreground image that moves with a small movement amount MVA, a pixel group constituting the foreground image Since the pixel value corresponding to the output signal T3 of the enhancement correction unit 11 is adopted, the foreground image is enhanced. On the other hand, the pixel group constituting the background image adopts the pixel value corresponding to the output signal B1 of the enhancement suppression correction unit 12, so that the background image is not enhanced and conversely, the LPF is blurred. Will be. As a result, the processing target frame displayed on the display has a background image far behind the foreground image for the user's eyes, thereby creating a depth. That is, the problem described above can be solved.

  In other words, it can be understood that the image processing apparatus in FIG. 1 is executing processing according to the flowchart in FIG. 8 (hereinafter referred to as image processing).

  In the following description of the processing of each step in FIG. 8, the frame image data corresponding to the input signal T0 in the frame image data is referred to as input image data. On the other hand, the frame image data corresponding to the output signal T4 is referred to as output image data. Each pixel value constituting the frame image data is referred to as pixel data.

  The processing by the enhancement correction unit 11 is referred to as enhancement processing or blur removal processing corresponding to the movement amount MV, and the frame image data corresponding to the output signal T3 of the enhancement correction unit 11 is referred to as blur removal data. On the other hand, the processing by the enhancement suppression correction unit 12 is referred to as enhancement suppression processing or blur processing, and the frame image data corresponding to the output signal B1 of the enhancement suppression correction unit 12 is referred to as blur processing data.

  In step S1, the enhancement correction unit 11 performs enhancement processing (blur removal processing) corresponding to the movement amount MV on the input image data (frame image data), and outputs it as blur removal data.

  In step S <b> 2, the enhancement suppression correction unit 12 performs an enhancement suppression process (blurring processing) according to the movement amount MV on the input image data (frame image data), and outputs it as blurring processing data.

  In step S3, the movement amount gap determination unit 13 performs a movement amount MV gap determination in the input image data (frame image data), and outputs the determination result to the switch unit 14 as an output signal D1.

  Note that the processing order of steps S1 to S3 is not particularly limited. This is because the processes in steps S1 to S3 are basically executed independently.

  In step S <b> 4, the switch unit 14 determines whether a small movement amount MV and a large movement amount MV are mixed based on the output signal D <b> 1 of the movement amount gap determination unit 13.

  If it is determined in step S4 that the small movement amount MV and the large movement amount MV are mixed, the switch unit 14 determines each pixel data (output pixel) constituting the output image data (frame image data) in step S5. As the data), blur removal data is used for pixel data with a small movement amount MV, and blur processing data is used for pixel data with a large movement amount MV. In step S7, the switch unit 14 outputs output image data.

  On the other hand, when it is determined in step S4 that the small movement amount MV and the large movement amount MV are not mixed, the switch unit 14 removes the blur as output image data (frame image data) in step S6. Adopt data. In step S7, the switch unit 14 outputs output image data.

  In this way, when the output image data is output in the process of step S7, the image processing is ended.

  As described above, generally, a video obtained by imaging becomes blurred as the movement of an object image within each frame constituting the video becomes faster. Therefore, the enhancement correction unit 11 in the example of FIG. 1 enhances the object image in accordance with the movement amount MV in order to eliminate this blurring, that is, enhancement processing (blur removal processing) according to the movement amount MV. Can be executed. However, for example, in a frame image, when the foreground image (first object image) is moving at a slow speed while the background image (second object image) is moving at a high speed, enhancement correction is performed. When the unit 11 performs enhancement processing (blur removal processing) corresponding to the movement amount MV on the entire frame image data, the background image is enhanced more than the foreground image. Therefore, if the output signal T3 of the enhancement correction unit 11 is directly input to a display or the like, a video image that looks like a background image close to the foreground image and impairs the sense of depth is displayed on the display screen. The above-mentioned problem will occur.

  Therefore, in order to solve such a problem, an enhancement suppression correction unit capable of executing LPF processing using a tap length corresponding to the magnitude of the movement amount MV, that is, enhancement suppression processing (blurring processing processing) corresponding to the movement amount MV. 12 is provided in the image processing apparatus of the example of FIG.

  As a result, due to the relationship of the magnitude of the movement amount MV in the frame image, an enhancement process (blur removal process) corresponding to the movement amount MV is applied to the small movement amount MV portion of the frame image. On the other hand, on the part of the large movement amount MV, the enhancement suppression process (blurring processing) according to the movement amount MV, that is, the LPF process using the tap length according to the movement amount MV is performed to blur. Can be. As a result, an image with a sense of depth can be displayed on a screen such as a display.

  Furthermore, although not shown, the present invention is also applicable to frame interpolation. In other words, it is difficult to accurately perform frame interpolation on an object with a large movement amount MV in frame interpolation, but when a small movement amount MV and a large movement amount MV are mixed in a frame image, the movement Complete motion compensation is performed where the amount MV is small (pixel group), and the average between two frames is taken as the amount of movement MV increases (for such a pixel group). As a result, bankruptcies can be suppressed.

  By the way, the above-described series of processes (or a part of them) can be executed by hardware, but can also be executed by software.

  In this case, the entire image processing apparatus of FIG. 1 or a part thereof can be configured by a computer as shown in FIG. 9, for example.

  In FIG. 9, a CPU (Central Processing Unit) 101 executes various processes according to a program recorded in a ROM (Read Only Memory) 102 or a program loaded from a storage unit 108 to a RAM (Random Access Memory) 103. To do. The RAM 103 also appropriately stores data necessary for the CPU 101 to execute various processes.

  The CPU 101, ROM 102, and RAM 103 are connected to each other via a bus 104. An input / output interface 105 is also connected to the bus 104.

  The input / output interface 105 includes an input unit 106 including a keyboard and a mouse, an output unit 107 including a display, a storage unit 108 including a hard disk, and a communication unit 109 including a modem and a terminal adapter. It is connected. The communication unit 109 performs communication processing with other devices via a network including the Internet.

  A drive 110 is connected to the input / output interface 105 as necessary, and a removable medium 111 made up of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached, and a computer program read from them is loaded. These are installed in the storage unit 108 as necessary.

  When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, a general-purpose personal computer is installed from a network or a recording medium.

  As shown in FIG. 9, the recording medium including such a program is distributed to provide a program to the user separately from the apparatus main body, and a magnetic disk (including a floppy disk) on which the program is recorded. , Removable media (package media) consisting of optical disks (including compact disk-read only memory (CD-ROM), DVD (digital versatile disk)), magneto-optical disks (including MD (mini-disk)), or semiconductor memory ) 111 as well as a ROM 102 on which a program is recorded and a hard disk included in the storage unit 108 provided to the user in a state of being incorporated in the apparatus main body in advance.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the order, but is not necessarily performed in chronological order, either in parallel or individually. The process to be executed is also included.

  In addition, as described above, in this specification, the system represents the entire apparatus including a plurality of processing apparatuses and processing units.

  In the above-described example, for the sake of simplicity, the movement amount MV is a movement in one direction, for example, a horizontal direction in the frame, but it can be a movement in other directions as a matter of course. In such a case, that is, even when the movement direction of the movement amount MV is other than the horizontal direction, the image processing apparatus can perform basically the same processing as the above-described various processes by handling the movement amount MV as a vector.

It is a block diagram which shows the functional structural example of the image processing apparatus to which this invention is applied. It is a block diagram which shows the detailed functional structural example of the enhancement correction | amendment part of FIG. It is a figure which shows the example of a characteristic of the function which the gain variable part of the enhancement correction | amendment part of FIG. 2 hold | maintains. It is a figure which shows the example of a characteristic of the function which the tap amount variable part of the enhancement correction | amendment part of FIG. 2 hold | maintains. It is a figure which shows each characteristic example of the imaging blur in three movement amounts. It is a figure which shows the example of a characteristic of the function which the tap amount variable part of FIG. 1 hold | maintains. It is a figure which shows the example of a characteristic of the function which the switch part of FIG. 1 hold | maintains. 3 is a flowchart illustrating an example of image processing executed by the image processing apparatus in FIG. 1. It is a block diagram which shows an example of the hardware constitutions of all or one part of the image processing apparatus with which this invention is applied.

Explanation of symbols

  11 enhancement correction unit, 12 enhancement suppression correction unit, 13 movement amount gap determination unit, 14 switch unit, 21 tap amount variable unit, 22 LPF unit, 31 HPF unit, 32 gain variable unit, 33 multiplication unit, 34 addition unit, 35 Tap amount variable unit, 101 CPU, 102 ROM, 103 RAM, 108 storage unit, 111 removable media

Claims (4)

In an image processing apparatus that performs processing on each of a plurality of access units constituting a moving image,
First correction means for correcting the access unit to be processed by subjecting each pixel constituting the access unit to be processed to low pass filter processing with a tap length corresponding to the amount of movement. An image processing apparatus comprising: Second correction means for correcting the access unit to be processed by performing enhancement processing for enhancing each pixel constituting the access unit to be processed according to the amount of movement;
When the access unit to be processed includes a portion having a first movement amount and a portion having a second movement amount larger than the first movement amount, the access unit to be processed is When outputting, the correction result of the second correction unit is output for the portion having the first movement amount, and the correction result of the first correction unit is output for the portion having the second movement amount. The image processing apparatus according to claim 1, further comprising: a switching unit that In an image processing method of an image processing apparatus that performs processing on each of a plurality of access units constituting a moving image,
Image processing including a step of correcting the access unit to be processed by applying low pass filter processing to each pixel constituting the access unit to be processed with a tap length corresponding to the amount of movement Method. A program for causing a computer to control processing performed on each of a plurality of access units constituting a moving image,
A program including a step of correcting the access unit to be processed by performing low-pass filter processing on each pixel constituting the access unit to be processed with a tap length corresponding to the amount of movement.

Images