JP2005160015A - Image processing unit, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make it possible to easily detect a motion vector taking harmony with the ambient. <P>SOLUTION: A histogram part 35 determines a motion vector candidate with the highest frequency out of motion vector candidates in an attention pixel and respective surrounding pixels supplied from a template matching part 34 as a motion vector in the attention pixel, and supplies it to a motion vector correction part 36. The motion vector correction part 36 evaluates reliability of the motion vector supplied from the histogram part 35 based on a luminance slope of surrounding pixels of the attention pixel detected by the luminance slope detection part 33 and a control signal which is supplied from the template matching part 34, and orders to execute correction or not. If the unit evaluates that the reliability of the motion vector is low, the unit corrects the motion vector. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus, method, and program, and more particularly, to an image processing apparatus, method, and program that can easily detect a motion vector that is in harmony with the surroundings.

  Conventionally, a CRT (Cathode Ray Tub) has been widely used as a moving image display device, but in recent years, a liquid crystal display device (see, for example, Patent Document 1) has also become widespread.

  When the display of a predetermined one of a plurality of frames constituting a moving image is instructed, the CRT moves the built-in electron gun along each of a plurality of horizontal lines (scanning lines) constituting the CRT screen. By sequentially scanning, the instructed frame (hereinafter, the frame instructed to display is referred to as a display target frame) is displayed on the screen.

  Accordingly, when attention is paid to each of the plurality of pixels constituting the display target frame, each of the plurality of pixels is displayed in an impulse shape in the time direction (the moment when the electron gun scans and reaches the corresponding position) Will be displayed only). Hereinafter, display devices that employ a display method similar to that of the CRT are collectively referred to as an impulse display device.

  On the other hand, the liquid crystal display device is instructed to display a predetermined one (display target frame) of a plurality of frames constituting a moving image until a display of the next frame is instructed. The display target frame is displayed on the screen by holding all the liquid crystal displays constituting the screen.

  Specifically, for example, it is assumed that one pixel is associated with one liquid crystal. In this case, as an instruction to display a frame, each pixel value of each pixel constituting the display target frame is instructed to the liquid crystal display device. Then, the liquid crystal display device applies a voltage of a level corresponding to the instructed pixel value to each liquid crystal (each liquid crystal corresponding to each pixel) constituting the screen. Thereby, each liquid crystal outputs light according to the applied voltage. That is, the level of light output from a predetermined liquid crystal corresponds to the level of voltage applied to the liquid crystal.

  Thereafter, at least until the display of the next frame is instructed, the same level of voltage is continuously applied to each of the liquid crystals, so that each of the liquid crystals continues to output a certain level of light. That is, each liquid crystal continues to display pixels having the designated pixel value.

  When the display of the next frame is instructed and the pixel value of a predetermined pixel needs to be changed, the voltage corresponding to the changed pixel value is applied to the liquid crystal corresponding to the pixel. Therefore (because the level of the applied voltage changes), the output level (light level) of the liquid crystal also changes.

Thus, since the liquid crystal display device employs a display method different from the impulse display device such as a CRT, the installation space is small and the power consumption is small compared to the impulse display device. And it has the feature that distortion hardly occurs.
JP 2002-219811 A

  However, the liquid crystal display device has a first problem in that when moving images are displayed, motion blur is generated more frequently than the impulse display device.

  Conventionally, the occurrence of the first problem, that is, the occurrence of motion blur in a liquid crystal display device, was considered to be caused by the slow response speed of the liquid crystal. In other words, in the liquid crystal display device, the output level of each liquid crystal is the instructed target level (for example, when one pixel is associated with one liquid crystal, the level corresponding to the instructed pixel value). Since it takes time to reach, it was thought that motion blur would occur.

  Therefore, in order to solve the first problem, that is, to suppress the occurrence of motion blur in the liquid crystal display device, Patent Document 1 discloses the following technique. That is, the technique disclosed in Patent Document 1 is a technique (hereinafter referred to as an overdrive method) in which a voltage higher than a target level is applied to each liquid crystal (each pixel). In other words, the overdrive method is a method of setting a target level higher than the conventional level, that is, a method of correcting the target level.

  However, even when such an overdrive method is applied, the occurrence of motion blur cannot be suppressed. In other words, there is no effective technique for suppressing the motion blur of moving images in the liquid crystal display device, and the first problem cannot be solved.

  Therefore, in view of such circumstances, the applicant of the present application suppresses the factor that cannot solve the first problem even when the conventional overdrive method is applied, that is, the occurrence of motion blur in the liquid crystal display device. The factor which cannot be analyzed was analyzed, and the image processing apparatus which can solve the first problem based on the analysis result was invented. This invention has been filed earlier by the applicant of the present application (Japanese Patent Application No. 2003-270965).

  That is, as described above, in the liquid crystal display device, one of the causes of motion blur is the slow response speed of the liquid crystal (pixels), and the overdrive method is a method that takes this cause into consideration.

  However, the occurrence of motion blur in the liquid crystal display device is not only due to the slow response of the liquid crystal, but also due to a characteristic called human vision (a user viewing the liquid crystal display device) called “following vision”. One. Nevertheless, the Applicant has analyzed that the conventional overdrive method does not consider tracking vision at all, and thus it is impossible to suppress the occurrence of motion blur. Note that the tracking vision is a characteristic also called a retina afterimage, and refers to a characteristic that the human eye unconsciously follows the object when the object is moving.

  In other words, since the human eye has the characteristic of following vision, all the pixel values of the liquid crystal (pixel) that displays the moving object (corresponding to the liquid crystal corresponding thereto) as in the conventional overdrive method. The present applicant has analyzed that the motion blur cannot be eliminated even if the applied voltage level is corrected, that is, only the time response of the output level of the liquid crystal is improved.

  Therefore, as described above, the applicant of the present application has invented an image processing apparatus that executes image processing in consideration of not only the slow response of the liquid crystal but also the follow-up vision.

  In other words, the image processing apparatus invented by the applicant of the present application uses an edge portion of an object in which a pixel of interest (hereinafter, referred to as a pixel of interest) as a processing target among pixels constituting a display target frame is a moving object. , By correcting the pixel value of the target pixel according to the motion vector (direction and size) of the target pixel, it is possible to suppress motion blur due to follow-up vision.

  However, at this time, if a vector that is different from the motion vector of the surrounding pixels of the pixel of interest is used as the motion vector of the pixel of interest (if a motion vector that is in harmony with the surroundings is not used), the final The second problem is that the pixel value (correction value) of the pixel of interest also becomes a different value from the pixel values (correction values) of the surrounding pixels in the corrected image (display target frame) obtained in the above. Will occur. In other words, the pixel of interest in the finally obtained corrected image becomes a pixel that is different from the surrounding pixels, and a second problem occurs that the image quality is degraded.

  Although the liquid crystal display device has been described as a problem, the first problem and the second problem are problems not only for the liquid crystal display device but also for the entire display device having the following characteristics. That is, the feature of the display device having the first problem and the second problem is that a plurality of display elements that require a predetermined time from when the target level is designated until the output level reaches the target level. And each of the plurality of display elements is associated with at least a part of a predetermined one of the pixels constituting the frame or the field.

  In the display device having such a feature, for a predetermined time (for example, a time until the display of the next frame or field is instructed) after the display of the predetermined frame or field is instructed, In many cases, a display method for holding at least a part of the display elements constituting the screen is employed. Hereinafter, display devices that employ such a display method, such as a liquid crystal display device, are collectively referred to as a hold-type display device. The display form of each display element (liquid crystal in the case of a liquid crystal display device) constituting the screen of the hold type display device is referred to as a hold display. That is, it can be said that the first problem and the second problem are problems that the hold type display device has.

  The present invention has been made in view of such a situation, and makes it possible to easily detect a motion vector in harmony with the surroundings.

  The first image processing apparatus according to the present invention sets a predetermined pixel among the pixels constituting the first access unit as a target pixel, the first access unit, and the second just before the first access unit. The frequency of the motion vector candidates in each of the target pixel generated by the candidate generation unit and its surrounding pixels is the highest. Processing results of a motion vector determination unit that determines a high motion vector candidate as a motion vector in the target pixel, a luminance change calculation unit that calculates the degree of change in luminance around the target pixel, a luminance change calculation unit, and a candidate generation unit Based on the above, the reliability of the motion vector determined by the motion vector determination means is evaluated, and if the reliability of the motion vector is low If you value, characterized in that it comprises a correcting means for correcting the motion vector.

  The correction means can correct the motion vector by evaluating that the reliability of the motion vector is low when the degree of change in brightness calculated by the brightness change calculation means is less than a threshold value.

  The candidate generating means detects the second pixel from the first access unit as the corresponding pixel of the first pixel arranged at the position in the second access unit corresponding to the arrangement position of the target pixel, and A vector starting from the second pixel and ending at the second pixel can be generated as a motion vector candidate at the target pixel.

  If the candidate generating means further determines that there are a plurality of corresponding pixel candidates in the first access unit, or if it is determined that the reliability of the second pixel being a corresponding pixel is low, First information indicating a vector correction command is provided to the correction unit. When the first information is provided from the candidate generation unit, the correction unit evaluates that the reliability of the motion vector is low and determines the motion vector. It can be corrected.

  The candidate generation means further provides the correction means with second information indicating that, when the target pixel itself is included in the plurality of corresponding pixel candidates, the correction means receives the second information from the candidate generation means. If this information is provided, it can be evaluated that the reliability of the motion vector is low, and the motion vector can be corrected to a zero vector.

  According to the first image processing method of the present invention, a predetermined pixel among the pixels constituting the first access unit is set as a target pixel, and the first access unit and the second just before the first access unit are set. A candidate generation step for generating a motion vector candidate at the target pixel by comparing the access unit with the access unit, and the frequency of the motion vector candidates at each of the target pixel generated by the processing of the candidate generation step and its surrounding pixels A motion vector determination step for determining a motion vector candidate having the highest value as a motion vector at the target pixel, a luminance change calculation step for calculating the degree of change in luminance around the target pixel, a luminance change calculation step, and a candidate generation step. The reliability of the motion vector determined by the processing of the motion vector determination step based on the processing result Evaluates, when assessed to be low reliability of the motion vector, characterized in that it comprises a step of correcting a motion vector.

  A first program according to the present invention is a program to be executed by a computer, wherein a predetermined pixel among pixels constituting the first access unit is set as a target pixel, and image processing in units of the target pixel is performed by the computer. A candidate generation step of generating a motion vector candidate at the pixel of interest by comparing the first access unit and the second access unit immediately before the first access unit, A motion vector determination step for determining a motion vector candidate having the highest frequency among motion vector candidates in each of the target pixel and the surrounding pixels generated by the processing of the generation step as a motion vector in the target pixel; Brightness change calculation step to calculate the degree of brightness change of Based on the processing results of the step and the candidate generation step, the reliability of the motion vector determined by the processing of the motion vector determination step is evaluated, and if the reliability of the motion vector is evaluated to be low, the correction for correcting the motion vector And a step.

  In the first image processing apparatus and method and the first program of the present invention, an image in which a predetermined pixel among the pixels constituting the first access unit is set as a target pixel, and the target pixel is a unit. Processing is executed. That is, the first access unit and the second access unit immediately before the first access unit are compared, and a motion vector candidate at the target pixel is generated based on the comparison result. Such motion vector candidates are generated not only for the pixel of interest but also for each of the surrounding pixels. Next, the motion vector candidate having the highest frequency among the motion vector candidates in the pixel of interest generated in this manner and the surrounding pixels is determined as the motion vector in the pixel of interest. Then, when the reliability of the motion vector is evaluated based on the degree of change in luminance around the pixel of interest and the processing result when the motion vector candidate is generated, and the reliability of the motion vector is evaluated as low, The motion vector is corrected.

  The second image processing apparatus of the present invention sets a predetermined pixel of the pixels constituting the first access unit as a target pixel, and the first access unit and the second just before the first access unit. The frequency of the motion vector candidates in each of the target pixel generated by the candidate generation unit and its surrounding pixels is the highest. Using a motion vector determination unit that determines a high motion vector candidate as a motion vector at the target pixel, a correction unit that corrects the motion vector determined by the motion vector determination unit, and a motion vector corrected by the correction unit, Process executing means for executing a predetermined process, and the correcting means is based on characteristics of the predetermined process of the process executing means. By using the first correction method, and corrects the motion vector.

  Further provided is a luminance change calculating means for calculating the degree of luminance change around the target pixel, and the correcting means is further determined by the motion vector determining means based on the processing results of the luminance change calculating means and the candidate generating means. When the reliability of the motion vector is evaluated and it is determined that the reliability of the motion vector is low, the motion vector corrected by the first correction method can be further corrected by the second correction method.

  According to a second information processing method of the second information processing apparatus of the present invention, a predetermined pixel among pixels constituting the first access unit is set as a target pixel, and the first access unit, By comparing with the second access unit immediately before the access unit, a candidate generation step for generating a motion vector candidate at the target pixel, and at each of the target pixel generated by the processing of the candidate generation step and its surrounding pixels A motion vector determination step for determining a motion vector candidate having the highest frequency among the motion vector candidates as a motion vector in the target pixel, a correction step for correcting the motion vector determined by the processing of the motion vector determination step, and a correction step Predetermined processing using the motion vector corrected by the processing in the second information processing apparatus A correction control step that corrects the motion vector using a first correction method based on characteristics of a predetermined process executed by the information processing apparatus under the control of the process control step. Including at least the step of:

  The second program of the present invention uses a predetermined one of a plurality of access units constituting a moving image as a processing target, and uses a motion vector in each of the pixels constituting the processing target access unit. A program that is executed by a computer that controls a processing execution apparatus that executes processing, wherein a predetermined pixel among pixels constituting the first access unit is set as a target pixel, and the first access unit, A candidate generation step for generating a motion vector candidate at the pixel of interest by comparing with the second access unit immediately before the access unit, and each of the pixel of interest generated by the processing of the candidate generation step and its surrounding pixels The motion vector candidate with the highest frequency among the motion vector candidates in A motion vector determination step for determining as a vector, a correction step for correcting the motion vector determined by the processing of the motion vector determination step, and a predetermined processing using the motion vector corrected by the processing of the correction step A correction control step using a first correction method based on characteristics of a predetermined process executed by the process execution device under the control of the process control step. It includes at least a correcting step.

  In the second information processing apparatus and method and the second program of the present invention, a predetermined pixel among the pixels constituting the first access unit is set as a target pixel, and the first access unit, The second access unit immediately before one access unit is compared, a motion vector candidate at the target pixel is generated based on the comparison result, and the generated motion vector at each of the generated target pixel and its surrounding pixels The motion vector candidate having the highest frequency among the candidates is determined as a motion vector in the target pixel, the determined motion vector is corrected, and a predetermined process is performed using the corrected motion vector. The motion vector used in the predetermined process executed at this time is a motion vector corrected using the first correction method based on the feature of the predetermined process.

  As described above, according to the present invention, a pixel of interest used in image processing that suppresses motion blur (particularly motion blur caused by human follow-up) generated in a hold-type display device such as a liquid crystal display device. Motion vectors can be detected. In particular, it is possible to easily detect a motion vector in harmony with the surroundings.

  Embodiments of the present invention will be described below. Correspondences between constituent elements described in the claims and specific examples in the embodiments of the present invention are exemplified as follows. This description is to confirm that specific examples supporting the invention described in the claims are described in the embodiments of the invention. Therefore, even if there are specific examples that are described in the embodiment of the invention but are not described here as corresponding to the configuration requirements, the specific examples are not included in the configuration. It does not mean that it does not correspond to a requirement. On the contrary, even if a specific example is described here as corresponding to a configuration requirement, this means that the specific example does not correspond to a configuration requirement other than the configuration requirement. not.

  Further, this description does not mean that all the inventions corresponding to the specific examples described in the embodiments of the invention are described in the claims. In other words, this description is an invention corresponding to the specific example described in the embodiment of the invention, and the existence of an invention not described in the claims of this application, that is, in the future, a divisional application will be made. Nor does it deny the existence of an invention added by amendment.

  According to the present invention, a first image processing apparatus is provided. This first image processing apparatus (for example, the motion detection unit 14 in the image processing apparatus 1 in FIG. 1 and the detailed configuration example is shown in FIG. 2) is included in the pixels constituting the first access unit. A candidate for generating a motion vector candidate at the target pixel by setting a predetermined pixel as the target pixel and comparing the first access unit with the second access unit immediately before the first access unit. The motion vector candidates (for example, each pixel in the region 95 in FIG. 11) in the generation unit (for example, the template matching unit 34 in FIG. 2) and the pixel of interest generated by the candidate generation unit and the surrounding pixels, respectively. Among the motion vector candidates (for example, “+4” having the highest frequency in the histogram of FIG. 12) The motion vector determining means (for example, the histogram unit 35 in FIG. 2) that determines the motion vector in the eye pixel, and the degree of change in luminance around the target pixel (for example, a luminance gradient described later, specifically, For example, the processing result of the luminance change calculating means (for example, the luminance gradient detecting unit 33 in FIG. 2) for calculating a value slope expressed by equation (2) described later, the luminance change calculating means, and the candidate generating means On the basis of this, when the reliability of the motion vector determined by the motion vector determination means is evaluated and the reliability of the motion vector is evaluated to be low (for example, a third inequality (9) described later holds) When at least one of the condition and the fourth condition that the control signal flag supplied from the template matching unit 34 in FIG. 2 is “1” is satisfied) It said correcting means for correcting the motion vector (e.g., a motion vector correction unit 36 of FIG. 2, the motion vector correction unit 36C of the motion vector correction unit 36A or 19 in FIG. 13), characterized in that it comprises a.

  The correction unit of the first image processing apparatus may be configured such that a third condition that an inequality (9) described later is satisfied when the degree of change in luminance calculated by the luminance change calculation unit is less than a threshold value. ), The motion vector can be corrected by evaluating that the reliability of the motion vector is low.

  The candidate generation means of the first image processing apparatus is configured to select a position (for example, a position i in FIG. 5) in the second access unit corresponding to the arrangement position of the target pixel (for example, the target pixel 51 in FIG. 5). ) As the corresponding pixel of the first pixel (for example, the pixel 71 in FIG. 5), the second pixel (for example, the search range i-6 to i + 6 in FIG. 5) from the first access unit. Any one of the pixels) and a vector starting from the target pixel and ending at the second pixel (for example, as a second pixel, the center pixel of the pixel group 73-(+ 6) in FIG. That is, when the pixel at the arrangement position i + 6 is detected, “+6”) can be generated as the motion vector candidate in the target pixel.

  The candidate generating means of the first image processing apparatus further determines that there are a plurality of corresponding pixel candidates in the first access unit (for example, as shown in FIG. 9, the search position This is a case where the pixel at i-1 and the pixel at the search position i + 5 are candidates for the corresponding pixel. Specifically, for example, all of inequality (5) to inequality (7) described later hold. When the condition of 1 is satisfied, the parameters of inequality (5) to inequality (7) (see FIG. 10), or when the reliability of the detected second pixel is determined to be low (for example, When a second condition such that inequality (8) described later is satisfied is satisfied (min1 of inequality (8) is shown in FIG. 10), first information indicating the motion vector correction command (as described later) And “1” as the control signal flag) When the first information is provided from the candidate generation unit (the control signal flag supplied from the template matching unit 34 in FIG. 2 is “1”, which will be described later). If the fourth condition is satisfied), the motion vector can be corrected by evaluating that the reliability of the motion vector is low.

  The candidate generation means of the first image processing apparatus further includes the second smallest pixel when the pixel of interest itself is included in a plurality of candidate corresponding pixels (for example, as shown in FIG. 17). SAD, that is, when the search position pos2 corresponding to the minimum value min2 is the arrangement position i of the pixel of interest), the correction means provides the correction means with the second information indicating that, and the correction means When the second information is provided from the means (when a fifth condition to be described later is satisfied), it is evaluated that the reliability of the motion vector is low, and the motion vector is corrected to a zero vector. it can.

  According to the present invention, a first image processing method is provided. In the first image processing method (for example, the image processing method of the motion detection unit 14 in FIG. 2), a predetermined pixel among the pixels constituting the first access unit is set as a target pixel, and the first A candidate generation step (for example, the process of step S22 in FIG. 21) that generates a motion vector candidate at the target pixel by comparing the access unit with the second access unit immediately before the first access unit. , A motion vector that determines the motion vector candidate having the highest frequency among the motion vector candidates in each of the target pixel and its surrounding pixels generated by the candidate generation step as a motion vector in the target pixel. The determination step (for example, the process of step S24 in FIG. 21) and the degree of change in luminance around the target pixel are determined. Based on the luminance change calculation step to be calculated (for example, the processing of step S21 in FIG. 21) and the processing results of the luminance change calculation step and the candidate generation step (for example, the processing results of steps S21 and S23 in FIG. 21). Then, when the reliability of the motion vector determined by the processing of the motion vector determination step is evaluated and it is evaluated that the reliability of the motion vector is low, a correction step (for example, FIG. 21) corrects the motion vector. Steps S26 to S28).

  According to the present invention, a first program is provided. The first program sets a predetermined pixel among pixels constituting the first access unit as a target pixel, and executes image processing in units of the target pixel on a computer (for example, the CPU 201 in FIG. 23). A candidate generating step (eg, generating a motion vector candidate at the pixel of interest by comparing the first access unit and the second access unit immediately before the first access unit). , The process of step S22 in FIG. 21), and the motion vector candidate having the highest frequency among the motion vector candidates in each of the target pixel and its surrounding pixels generated by the process of the candidate generation step, A motion vector determination step (for example, a step in FIG. 21) determined as a motion vector at the target pixel. Processing in step S24), luminance change calculation step for calculating the degree of change in luminance around the target pixel (for example, step S21 in FIG. 21), processing in the luminance change calculation step, and candidate generation step Based on the result (for example, the processing results of steps S21 and S23 in FIG. 21), the reliability of the motion vector determined by the processing of the motion vector determination step is evaluated, and the reliability of the motion vector is low When the evaluation is made, the method includes a correction step for correcting the motion vector (for example, processing in steps S26 to S28 in FIG. 21).

  According to the present invention, a second image processing apparatus is provided. The second image processing apparatus (for example, the motion detection unit 14 and the image processing unit 12 in the image processing apparatus 1 of FIG. 1, and a detailed configuration example of the motion detection unit 14 is shown in FIG. 2) A predetermined pixel among the pixels constituting the access unit is set as a target pixel, and the first access unit is compared with the second access unit immediately before the first access unit, Of the motion vector candidates in each of the target pixel generated by the candidate generating unit (for example, the template matching unit 34 in FIG. 2) and the surrounding pixels generated by the candidate generating unit (for example, the template matching unit 34 in FIG. 2). Motion vector determination means for determining the motion vector candidate having the highest frequency as a motion vector in the target pixel (for example, the histogram of FIG. 35) and correction means for correcting the motion vector determined by the motion vector determination means (for example, the motion vector correction section 36 in FIG. 2 and the motion vector correction section 36A in FIG. 13 or FIG. 18). A motion vector correction unit 36B) and a processing execution unit (for example, the correction unit 22 of the image processing 12 in FIG. 1) that executes a predetermined process using the motion vector corrected by the correction unit; The correction unit corrects the motion vector by using a first correction method (for example, the method shown in FIG. 15 or FIG. 16) based on the characteristics of the predetermined process of the process execution unit (for example, The post-processing correction unit 101 in FIG. 13 or FIG. 18 executes the processing).

  In the second image processing apparatus, a luminance change calculating means (for example, a luminance gradient detecting unit 33 in FIG. 2) for calculating the degree of change in luminance around the target pixel is further provided, and the correcting means further includes the Based on the processing results of the luminance change calculation means and the candidate generation means, the reliability of the motion vector determined by the motion vector determination means is evaluated (for example, the reliability evaluation unit 103 in FIG. When the reliability of the motion vector is evaluated to be low, the motion vector corrected by the first correction method is converted into a second correction method (for example, a method represented by Expression (10) described later). ) Can be further corrected (for example, the reliability correction unit 104 in FIG. 13 executes the process).

  According to the present invention, a second information processing method is provided. This second information processing apparatus is an information processing method of an information processing apparatus (for example, the image processing apparatus 1 in FIG. 1), and a predetermined pixel among pixels constituting the first access unit is used as a target pixel. A candidate generation step of generating a motion vector candidate at the pixel of interest by setting and comparing the first access unit and the second access unit immediately before the first access unit (for example, FIG. 21). In step S22), and the motion vector candidate having the highest frequency among the motion vector candidates in each of the target pixel generated by the processing of the candidate generation step and the surrounding pixels is determined in the target pixel. A motion vector determination step (for example, the process of step S24 in FIG. 21) to be determined as a motion vector, and the motion vector A correction step for correcting the motion vector determined by the processing of the determination step (for example, the processing of steps S25 to S28 in FIG. 21), and a predetermined process using the motion vector corrected by the processing of the correction step. Including a process control step for controlling the execution of the information processing apparatus (the process of step S7 executed after the motion vector calculation process of step S3 of FIG. 20 corresponding to FIG. 21), At least a step of correcting the motion vector by using a correction method based on the characteristics of the predetermined processing executed by the information processing apparatus under the control of the processing control step (for example, the processing of step S25 in FIG. 21). It is characterized by that.

  According to the present invention, a second program is provided. The second program uses a predetermined one of a plurality of access units constituting a moving image as a processing target, and performs a predetermined process using a motion vector in each pixel constituting the access unit to be processed. Is a program to be executed by a computer (for example, the CPU 201 in FIG. 23) that controls the processing execution apparatus (for example, the image processing apparatus 1 in FIG. 1), and is a predetermined one of the pixels constituting the first access unit. Candidate generation for generating a motion vector candidate at the target pixel by comparing the first access unit with the second access unit immediately before the first access unit. Generated by the step (for example, the process of step S22 in FIG. 21) and the process of the candidate generation step. A motion vector determination step for determining the motion vector candidate having the highest frequency among the motion vector candidates in each of the eye pixel and the surrounding pixels as a motion vector in the target pixel (for example, the process in step S24 in FIG. 21). ), A correction step for correcting the motion vector determined by the processing of the motion vector determination step (for example, processing of steps S25 to S28 in FIG. 21), and the motion vector corrected by the processing of the correction step. A process control step for controlling the execution of the predetermined process by the process execution device (the process of step S7 executed after the motion vector calculation process of step S3 of FIG. 20 corresponding to FIG. 21); And the correction step is performed under the control of the processing control step. Using the correction method based on the feature of the predetermined processing execution unit executes the step of correcting the motion vector (for example, step S25 in FIG. 21), characterized in that it comprises at least a.

  Hereinafter, an image processing apparatus to which the present invention is applied will be described with reference to the drawings.

  FIG. 1 shows a configuration example of an image processing apparatus to which the present invention is applied.

  In FIG. 1, an image processing apparatus 1 controls the display of a moving image with respect to a hold type display apparatus 2 configured as a liquid crystal display apparatus or the like. In other words, the image processing apparatus 1 sequentially instructs display of each of a plurality of frames constituting the moving image. Thereby, as described above, the hold-type display device 2 displays each of a plurality of pixels constituting the first frame for a predetermined time after the display of the first frame is instructed. (Not shown) and display of at least some of the display elements is held. That is, at least a part of each display element is hold-displayed for a predetermined time.

  The processing unit (moving image unit) of the image processing apparatus 1 is described here as a frame as described above for the sake of brevity of description, but it is of course possible to use a field as a unit. That is, in this specification, a unit such as a frame or a field is also referred to as an access unit, but the image processing apparatus 1 can perform image processing using this access unit as a unit. However, in the following description, it is assumed that the image processing apparatus 1 performs image processing in units of frames, among the access units.

  Further, hereinafter, the hold-type display device 2 is configured to display all pixels constituting the first frame from when the display of the first frame is instructed until the display of the next second frame is instructed. Are displayed on corresponding display elements (not shown), and display of all the display elements is held (all display elements hold display).

  Specifically, the image processing apparatus 1 sequentially receives image data of a plurality of frames constituting a moving image. That is, the image processing apparatus 1 receives image data of a display target frame (for example, pixel values of all the pixels constituting the display target frame). Specifically, the image data of the display target frame is input to each of the image processing unit 11, the image processing unit 12, the reference image storage unit 13, and the motion detection unit 14.

  The image processing unit 11 performs predetermined image processing in units of pixels on the image data of the display target frame and outputs the processed image data to the switching unit 15. That is, the image processing unit 11 performs predetermined image processing on each of the plurality of pixels constituting the display target frame, thereby correcting the pixel values of those pixels, and supplying the corrected pixel values to the switching unit 15. Output sequentially in a predetermined order.

  Note that the image processing executed by the image processing unit 11 is not particularly limited. For example, in the example of FIG. 1, the image processing unit 11 includes an image of a reference image (a frame immediately before a display target frame, hereinafter simply referred to as a previous frame) output from a reference image storage unit 13 described later. Two pieces of information are input, such as data and a detection result (a motion vector of each pixel constituting the display target frame) described later. In this case, the image processing unit 11 may perform image processing using all of these two pieces of information, may perform image processing using any one piece of information, or both pieces of information. Image processing that is not used may be performed. Specifically, for example, the image processing unit 11 can correct each pixel value of each pixel constituting the display target frame according to a predetermined rule table (not shown).

  Further, the image processing unit 11 is not an essential component for the image processing apparatus 1 and may be omitted. In this case, the image data of the display target frame is the image processing unit 12, the reference image storage unit 13, the motion detection unit 14, and the input end of the switching unit 15 (the image processing apparatus of FIG. 11 is input to each of the input terminals connected to 11.

  The image processing unit 12 includes pixels corresponding to a moving object (for example, the magnitude of the motion vector detected by the motion detection unit 14 is greater than or equal to a threshold value) in the image data (pixel value of each pixel) of the display target frame. Pixel value) is corrected (including 0 correction), and the corrected pixel value is output to the switching unit 15.

  Note that the image processing unit 12 can set an object moving in an arbitrary spatial direction in the display target frame as a processing target, and can correct a pixel value of a pixel corresponding to the object. However, for the sake of simplicity, when the pixel at the upper left corner of the display target frame is used as a reference, it moves in the right (lateral) direction X (hereinafter referred to as the spatial direction X) or the opposite direction with respect to the reference pixel. The object is a processing target of the image processing unit 12. Therefore, as will be described later, the direction of the motion vector of the target pixel detected by the motion detection unit 14 can also be an arbitrary spatial direction in the frame. Is done.

  Specifically, the image processing unit 12 includes a step edge detection unit 21 and a correction unit 22.

  The step edge detection unit 21 detects pixels corresponding to the edge portion of the moving object from the image data of the display target frame based on the detection result (motion vector) of the motion detection unit 14, and the detection result Is supplied to the correction unit 22.

  Specifically, for example, the step edge detection unit 21 captures the step edge as an object, and decomposes the image data of the display target frame into an aggregate of image data of a plurality of step edges formed in the spatial direction X. In addition, pixels corresponding to edge portions of the plurality of step edges are detected, and the detection results are supplied to the correction unit 22.

  Note that the step edge is a second that is different from the first pixel value, with pixels having the first pixel value continuously arranged in a predetermined direction (here, the spatial direction X), with the predetermined pixel as a boundary. A pixel group in which pixels having the pixel value of are formed in a line continuously in that direction.

  That is, the step edge detection unit 21 determines a pixel value of the target pixel and a predetermined one of the adjacent pixels (in this case, a predetermined one of the spatial direction X and the opposite direction). The difference from the pixel value is calculated. For example, when the calculation result (difference value) is not 0, the step edge detection unit 21 detects the target pixel as a pixel corresponding to the edge portion of the step edge.

  However, since the main purpose is to suppress motion blur here, only a step edge with motion is sufficient as an object for which an edge portion is detected.

  Therefore, for example, when the magnitude of the motion vector of the target pixel supplied from the motion detection unit 14 is greater than or equal to the threshold value, the step edge detection unit 21 moves the step edge having the target pixel as one constituent element. And the process is executed. That is, the step edge detection unit 21 calculates a difference value (hereinafter, simply referred to as a difference value) between a pixel value of a target pixel and a pixel value of a pixel adjacent thereto, and calculates the difference value and the pixel value of the target pixel. The detection result is supplied to the correction unit 22.

  On the other hand, for example, when the magnitude of the motion vector of the target pixel supplied from the motion detection unit 14 is less than the threshold, the step edge detection unit 21 uses the target pixel as one constituent element. Judge that it is not moving and prohibit the execution of the process.

  For example, when the magnitude of the motion vector of the target pixel supplied from the motion detection unit 14 is greater than or equal to a threshold value (ie, the step edge having the target pixel as one component is the spatial direction X or vice versa) When moving in the direction), the pixel value of the target pixel supplied from the step edge detection unit 21 is corrected (including 0 correction). At this time, the correction unit 22 uses the motion vector (the movement direction and the movement amount of the step edge) of the target pixel supplied from the motion detection unit 14 and the difference value (the height of the step edge) supplied from the step edge detection unit 21. ) And the pixel value of the target pixel is corrected.

  That is, for example, when the supplied difference value is not 0 and the magnitude of the supplied motion vector is equal to or larger than the threshold value, the correcting unit 22 sets the target pixel at the edge portion of the step edge that moves. The pixel is determined to be a corresponding pixel, and the pixel value of the pixel of interest is corrected based on the supplied difference value and motion vector.

  On the other hand, for example, when the supplied difference value is 0 and the magnitude of the supplied motion vector is equal to or greater than a threshold value, the correcting unit 22 determines that the target pixel is a step edge having motion. It is determined that the pixel is one component but not the pixel corresponding to the edge portion (the pixel corresponding to the other portion), and the pixel value of the target pixel is corrected to 0 (not corrected).

  For example, when the magnitude of the supplied motion vector is less than the threshold value, the correction unit 22 determines that the step edge having the target pixel as one constituent element is not moving, as in the step edge detection unit 21. Then, the processing itself is prohibited (correction processing including zero correction is prohibited).

  The correction method of the pixel value of the correction unit 22 is a method of correcting the pixel value of the target pixel corresponding to the edge portion of the step edge with motion based on the motion vector of the target pixel detected by the motion detection unit 14. If it is, it will not be specifically limited. Specifically, for example, the following correction method can be applied.

  That is, the correction unit 22 calculates a right side correction value R by calculating the right side of the following equation (1), and adds the correction value R to the pixel value of the target pixel to obtain the pixel value of the target pixel. to correct.

・・・（１）

... (1)

  In Equation (1), Er represents the difference value supplied from the step edge detection unit 21, and V represents the magnitude of the motion vector supplied from the motion detection unit 14. Equation (1) is a first-order lag element in which the time response of all display elements of the hold-type display device 2 (for example, all liquid crystals when the hold-type display device 2 is a liquid crystal display device) is constant in time constant. In the equation (1), this time constant is indicated by τ. T indicates the display time of the display target frame (the time from when the display of the display target frame is instructed until the display of the next frame is instructed). Hereinafter, such a time T is referred to as a frame time T. In the liquid crystal display device, the frame time T is generally set to 16.6 ms.

  The reference image storage unit 13 stores the image data of the display target frame as the image data of the reference image for the next (immediately following) frame.

  That is, the motion detection unit 14 (and the image processing unit 11 described above), when image data of a new frame is input as image data of a display target frame, immediately preceding frame (previous frame stored in the reference image storage unit 13). Image data of a frame that was a display target frame) is acquired as image data of a reference image for the display target frame. Then, the motion detection unit 14 compares the image data of the display target frame with the image data of the reference image to detect the motion vector of the target pixel of the display target frame, and the image processing unit 11, the image processing unit 12 ( Step edge detection unit 21 and correction unit 22) and switching unit 15 are supplied.

  Note that the motion detection unit 14 can actually detect a motion vector having an arbitrary direction on a two-dimensional plane parallel to the spatial direction X and the spatial direction Y. That is, the direction of the motion vector can actually be an arbitrary direction on the two-dimensional plane. However, as described above, since the step edge moving in the spatial direction X or in the opposite direction is the object of processing, as described above, the direction of the motion vector is also in the spatial direction X or the opposite direction. The

  Specifically, for example, when the step edge moves by N (N is an arbitrary positive integer value) pixels in the spatial direction X during one frame, the motion detection unit 14 moves the motion vector ( “+ N” is detected as the motion vector of the target pixel, which is one component of the step edge. On the other hand, when the step edge moves by N pixels in the opposite direction of the spatial direction X during one frame, the motion detection unit 14 detects “−N” as the motion vector of the step edge. Thus, here, the direction of the motion vector is represented by “+” in the case of the spatial direction X, and is represented by “−” in the case of the reverse direction of the spatial direction X.

  The switching unit 15 switches the input according to the detection result (motion vector) of the motion detection unit 14.

  That is, when the magnitude of the motion vector of the target pixel supplied from the motion detection unit 14 is less than the threshold (when the target pixel is not a pixel corresponding to a step edge with motion), the switching unit 15 Is switched to the end on the image processing unit 11 side, and the data (pixel value) of the pixel of interest supplied from the image processing unit 11 is supplied to the display control unit 16.

  On the other hand, when the magnitude of the motion vector of the target pixel supplied from the motion detection unit 14 is greater than or equal to the threshold value (when the target pixel is a pixel corresponding to a step edge with motion), the switching unit 15 The input is switched to the end of the image processing unit 12 on the correction unit 22 side, and the data (pixel value) of the target pixel supplied from the correction unit 22 is supplied to the display control unit 16.

  The display control unit 16 associates each pixel data (pixel value) included in the display target frame sequentially supplied from the switching unit 15 with a signal of a predetermined format (in the hold type display device 2). Are converted to signals indicating the target levels of the display elements) and then output to the hold type display device 2. That is, the display control unit 16 instructs the hold-type display device 2 to display the display target frame by performing such processing.

  As described above, in the image processing apparatus 1 according to the present embodiment, the pixel value of the target pixel is corrected based on the motion vector detected by the motion detection unit 14. What should be noted here is that this motion vector is in harmony with other motion vectors in pixels around the target pixel. In other words, the motion detection unit 14 according to the present embodiment detects, for example, a motion vector in harmony with the surroundings, that is, in order to solve the above-described second conventional problem, for example, FIG. It is the point which has the structure of. Therefore, details of the motion detection unit 14 of the present embodiment will be described below with reference to FIG.

  As shown in FIG. 2, the motion detection unit 14 of the present embodiment includes a low-pass filter (hereinafter referred to as LPF) 31, an LPF 32, a luminance gradient detection unit 33, a template matching unit 34, a histogram unit 35, a motion vector. It is comprised from the correction | amendment part 36 and LPF37.

  An input image (image data), that is, image data of a display target frame is supplied to each of the luminance gradient detection unit 33 and the template matching unit 34 via the LPF 31.

  The luminance gradient detection unit 33 detects the luminance gradient at the arrangement position of the target pixel in the display target frame.

  The luminance gradient indicates the following value. That is, for example, there is a function f (x) in which a coordinate value in a predetermined direction (for example, the coordinate x in the spatial direction X) is input as a parameter and the luminance (pixel value) of the input coordinate x is output. In this case, the absolute value of the first-order differential value of the function f (x) at the arrangement position i (i is the coordinate value in the spatial direction X) of the target pixel, that is, | f ′ (i) | is the luminance gradient. .

  However, the generation of such a function f (x) and the calculation of the primary differential value using the function f (x) are heavy processing for the luminance gradient detector 33. Therefore, here, for example, a value slope represented by the following equation (2) is defined as a luminance gradient. That is, the luminance gradient detector 33 calculates the luminance gradient slope by calculating the right side of the equation (2). Thereby, the luminance gradient detection unit 33 can easily (at a light load) calculate the luminance gradient slope at the arrangement position of the target pixel (for example, the pixel 51 shown in FIG. 3) of the display target frame.

・・・（２）

... (2)

In Expression (2), Y _i indicates the luminance (pixel value) of the target pixel 51 at the arrangement position i, as shown in FIG. Y _i-1 indicates the luminance (pixel value) of the pixel 52 adjacent to the left of the target pixel 51 (the pixel 52 at the arrangement position i-1 adjacent to the target pixel 51 in the direction opposite to the spatial direction X). Y _{i + 1} indicates the luminance (pixel value) of the pixel 53 on the right side of the target pixel 51 (the pixel 53 at the arrangement position i + 1 adjacent to the target pixel 51 in the spatial direction X). Max (A, B,..., N) indicates a function in which each of numerical values A to N is input as a parameter, and the maximum value among the input numerical values A to N is output. . Note that the number of parameters (number A to number N) input to the function max () is not particularly limited.

That is, the brightness gradient detecting unit 33, the luminance Y _i and the difference absolute value A of the luminance Y _i-1 of the pixel 52 of the left side, the luminance Y _i and the pixel on the right side of the target pixel 51 of the pixel of interest 51 The difference absolute value B with the luminance Y _{i + 1 of} 53 is calculated, and the larger one of the difference absolute value A and the difference absolute value B (maximum value) is calculated at the position where the target pixel 51 is arranged. The luminance gradient slope is supplied to the motion vector correction unit 36.

  Incidentally, the image data of the display target frame is also supplied to the template matching unit 34 via the LPF 31 as described above. At this time, the template matching unit 34 further receives the image data of the immediately previous frame (the frame that was the display target frame immediately before) stored in the reference image storage unit 13 as the image data of the reference image for the display target frame. Is supplied through.

  Then, the template matching unit 34 cuts out a predetermined area including at least a position corresponding to the arrangement position of the target pixel (hereinafter, such area is referred to as a window) from the previous frame, and the window is a display target frame. Which region is to be matched is determined, the motion vector candidate pvec of the pixel of interest is determined based on the matching result, and is supplied to the histogram unit 35.

  Further, the template matching unit 34 generates a control signal flag used by the motion vector correction unit 36 and supplies the control signal flag to the motion vector correction unit 36. Details of the control signal flag will be described later.

  Details of the template matching unit 34 will be described below with reference to FIGS.

  FIG. 4 shows a detailed configuration example of the template matching unit 34 of the present embodiment.

  As shown in FIG. 4, the template matching unit 34 of the present embodiment includes a SAD (Sum of Absolute Difference) calculation unit 61, a SAD minimum value detection unit 62, and a SAD minimum value evaluation unit 63.

  For example, as shown in FIG. 5, the SAD calculation unit 61 arranges the position i (i is the coordinate value in the spatial direction X of the target pixel 51 of the display target frame from the immediately preceding frame, as described above. ), A window 72 composed of a predetermined number (5 in the example of FIG. 5) of pixels arranged continuously in the spatial direction X, with the pixel 71 arranged at the position i corresponding to) as the center.

  In addition, the SAD computing unit 61 cuts out the following area (referred to as a comparison area in order to distinguish such an area from a window) from the display target frame. That is, n (n is an integer value within a preset range from the arrangement position i of the target pixel in the spatial direction X or in the opposite direction, and in the example of FIG. 5 -6 to (+6) A predetermined number (same as the window 72) continuously arranged in the spatial direction X around the pixel arranged at the position i + n (hereinafter referred to as the search position i + n) separated by any integer value) The comparison region 73-n including 5 pixels in the example of FIG. 5 is cut out. Note that only the comparison area 73-(-6) corresponding to the search position i-6 and the comparison area 73-(+ 6) corresponding to the search position i + 6 are illustrated, but here (example of FIG. 5). Actually, each of the 13 comparison regions 73-n including the comparison region 73-(-6) and the comparison region 73-(+ 6) is sequentially cut out in a predetermined order.

  Then, the SAD computing unit 61 uses a predetermined evaluation function to correlate the window 72 and each of the 13 comparison regions 73-n (comparison region 73-(-6) to comparison region 73-(+ 6)). And each of the calculation results is supplied to the SAD minimum value detector 62.

  The evaluation function used by the SAD calculation unit 61 is not particularly limited. For example, the SAD calculation unit 61 can use a normalized correlation function, an SSD (Sum of Squared Difference), or the like. However, here, the SAD calculation unit 61 will be described as using SAD.

  That is, the SAD computing unit 61 computes the right side of the following equation (3), whereby the correlation value SAD (j) (j is i + n) between the window 72 and each comparison region 73-n. That is, here, j is determined for each of the search ranges i−6 to i + 6 in FIG. 5, and supplied to the SAD minimum value detector 62.

・・・（３）

... (3)

In Expression (3), C _k ^j is, as shown in FIG. 5, left in the figure among the five pixels constituting the comparison region 73-n at the search position j (j = i + n). The luminance (pixel value) of the k-th pixel is shown. As shown in FIG. 5, P _k ⁱ represents the luminance (pixel value) of the kth pixel from the left in the figure among the five pixels that constitute the window 72 centered on the pixel 71 at the arrangement position i. Show.

  Specifically, for example, as shown in FIG. 6, the change in luminance (pixel value) in the spatial direction X of the display target frame is a curve 81, and the luminance (pixel value) in the spatial direction X of the immediately preceding frame is shown. ) Is a curve 82. In FIG. 6, the vertical axis represents luminance (pixel value), and the horizontal axis represents pixel position (arrangement position). The pixel position i indicates the arrangement position of the target pixel 51 as described above.

  FIG. 7 shows the calculation result of the SAD calculation unit 61 in this case, that is, each correlation value SAD (j (= i + n)). In FIG. 7, the vertical axis indicates the correlation value SAD (j), and the horizontal axis indicates the search position (arrangement position). That is, FIG. 7 shows a window 72 composed of five pixels having the pixel value (luminance) shown by the curve 82 in FIG. 6 and five pixels having the pixel value (luminance) shown by the curve 81 in FIG. SAD (i + n (= j)) at the search position i + n with each of the comparison regions 73-n configured by

  Returning to FIG. 4, the smaller the correlation value SAD (j) is, the higher the correlation is. Therefore, the SAD minimum value detection unit 62 receives each correlation value SAD (j (= i + n)) supplied from the SAD calculation unit 61. ) To detect the minimum value min and supply the search position pos (pos = i + 3 in the example of FIG. 7) corresponding to the minimum value min to each of the SAD minimum value evaluation unit 63 and the histogram unit 35. To do.

  Note that the minimum value min is the minimum point (z, SAD (z)) (z is any one of i-6 to i + 6 in this case) connecting each correlation value SAD (j). The correlation value SAD (z) at the minimum point (i + 3, SAD (i + 3)) in the example of FIG. About 40). Therefore, hereinafter, the minimum value min is also referred to as a minimum value min.

  The above-described processes performed by the SAD calculation unit 61 and the SAD minimum value detection unit 62 are summarized as follows. That is, the SAD computing unit 61 detects the pixel 71 arranged at the position i corresponding to the arrangement position of the target pixel 51 from the immediately preceding frame, and cuts out the window 72 including at least the pixel 71. Next, the SAD calculation unit 61 sequentially shifts the window 72 on a horizontal line (a line parallel to the spatial direction X) including the target pixel 51 in the display target frame. At this time, the SAD computing unit 61 calculates the degree of coincidence (correlation) between the window 72 and the overlapping area (comparison area) 73-n at each transition position (search position) i + n of the window 72. Calculation is performed using a predetermined evaluation function (here, SAD). Then, the SAD minimum value detection unit 62 determines the central pixel of the comparison region 73-n that has the highest degree of matching (correlation) (that is, has the minimum value min) as the central pixel 71 (target pixel) of the window 72. 51 is set as the corresponding pixel of the pixel 71) corresponding to 51. That is, the pixel arranged at the search position pos corresponding to the minimum value min among the pixels constituting the display target frame is set as the corresponding pixel of the center pixel 71 of the window 72.

  Note that the method executed by the SAD calculation unit 61 and the SAD minimum value detection unit 62 is referred to as a window matching method, an area-based matching method, a template matching method, or the like. The corresponding pixel is also referred to as a corresponding point.

  However, to be precise, the histogram unit 35 has a vector n (amount of change from the reference position i of the window 72 and the amount of change) starting from the arrangement position i of the target pixel 51 and ending at the search position pos (= i + n). A value n) indicating the direction is supplied as a motion vector candidate pvec. That is, here, as described above, the absolute value of the value n indicates the magnitude of the vector (motion vector candidate pvec), and the sign of the value n indicates the direction of the vector (motion vector candidate pvec). . Specifically, when the value n is positive, the direction of the motion vector candidate pvec is the spatial direction X, that is, the right direction in FIG. On the other hand, when the value n is negative, the direction of the motion vector candidate pvec is opposite to the spatial direction X, that is, the left direction in FIG. Specifically, for example, in the example of FIG. 7, “+3” is supplied to the histogram unit 35 as the motion vector candidate pvec of the target pixel 51.

  More precisely, as will be described later, the histogram unit 35 generates a histogram of the motion vector candidate pvec of each of the target pixel 51 and a plurality of surrounding pixels, and the motion vector vec of the target pixel 51 based on this histogram. To decide. Therefore, the SAD calculation unit 61 and the SAD minimum value detection unit 62 are not only the motion vector candidate pvec of the pixel of interest 51 but also an area configured by a predetermined number of pixels lined up continuously in the X direction with the pixel of interest 51 as the center. (For example, an area 95 in FIG. 11 described later) is set, and motion vector candidate pvec of each pixel constituting this area is obtained (the method is exactly the same as the above-described method), and each of the histogram units 35 is obtained. Supply.

  By the way, for example, as shown in FIG. 8, the change in luminance (pixel value) in the spatial direction X of the display target frame is a curve 91, and the change in luminance (pixel value) in the spatial direction X of the immediately preceding frame. Is a curve 92. In FIG. 8, the vertical axis represents luminance (pixel value), and the horizontal axis represents pixel position (arrangement position). The pixel position i indicates the arrangement position of the target pixel 51 as described above.

  The calculation result of the SAD calculation unit 61 in this case, that is, each correlation value SAD (j (= i + n)) is shown in FIG. In FIG. 9, the vertical axis represents the correlation value SAD (j), and the horizontal axis represents the search position (arrangement position). That is, FIG. 9 shows a window 72 composed of five pixels having the pixel value (luminance) shown by the curve 92 in FIG. 8 and five pixels having the pixel value (luminance) shown by the curve 91 in FIG. SAD (i + n) at the search position i + n with each of the comparison regions 73-n constituted by

  In FIG. 9, the minimum points of the curve connecting the correlation values SAD (j) are the point (i-1, SAD (i-1)) and the point (i + 5, SAD (i + 5)). It is. That is, in the example of FIG. 9, there are two local minimum points. When there are a plurality of minimum points in this way, the SAD computing unit 61, among the correlation values SAD (j) corresponding to each of the plurality of minimum points, that is, among all SAD (j). The minimum value of is applied as the minimum value min. Specifically, for example, in the example of FIG. 9, the correlation value SAD (i + 5) (= about 27) at the minimum point (i + 5, SAD (i + 5)) is applied as the minimum value min. Therefore, the center pixel of the comparison region 73-5 (not shown), that is, the pixel at the arrangement position i + 5 in the display target frame is set as a corresponding pixel (corresponding point) of the center pixel 71 of the window 72. That is, “+5” is supplied to the histogram unit 35 as the motion vector candidate pvec of the target pixel 51.

  However, as shown in FIG. 9, this minimum value min, that is, the correlation value SAD (i + 5) (= about 27) at the minimum point (i + 5, SAD (i + 5)) and the minimum point ( i−1, SAD (i−1)) has a small difference from the correlation value SAD (i−1) (= about 30), and the corresponding pixel (corresponding point) of the center pixel 71 of the window 72 is the arrangement position i +. It cannot be said that it is 5 pixels. That is, the corresponding pixel (corresponding point) of the center pixel 71 of the window 72 may be a pixel at the arrangement position i-1.

  Thus, when there are two or more candidates for the corresponding pixel (corresponding point) of the center pixel 71 of the window 72 (as shown in FIG. 9, there are two or more minimum points in the graphed SAD (j)). When the difference between each correlation value SAD (j) corresponding to each local minimum point is small), the reliability of the set corresponding pixel (the pixel corresponding to the lowest local minimum point), that is, the histogram portion It can be said that the reliability of the motion vector candidate pvec of the target pixel 51 supplied to 35 is low.

  Also, even if there is only one candidate for the corresponding pixel (corresponding point) of the center pixel 71 of the window 72 (as shown in FIG. 7, only one local minimum point exists in the graphed SAD (j). Or, although not shown, there are two or more minimum points in the graphed SAD (j), but the minimum correlation value SAD (j) (= minimum value min) and other correlation values SAD If the correlation value SAD (j) (= minimum value min) of the set corresponding pixel is not sufficiently small even when the difference from (j) is large), the reliability of the corresponding pixel, that is, It can be said that the reliability of the motion vector candidate pvec of the target pixel 51 supplied to the histogram unit 35 is low.

  Therefore, the evaluation of such a minimum value min, that is, the reliability of the pixel corresponding to the minimum value min being the corresponding pixel (the reliability of the motion vector candidate pvec of the target pixel 51 supplied to the histogram unit 35). In order to perform the evaluation, the template matching unit 34 of the present embodiment has a configuration shown in FIG. That is, as shown in FIG. 4, the template matching unit 34 of the present embodiment is provided with a SAD minimum value evaluation unit 63 in addition to the SAD calculation unit 61 and the SAD minimum value detection unit 62 described above. Yes.

  The SAD minimum value evaluation unit 63 evaluates the reliability (the reliability of the motion vector candidate pvec of the target pixel 51 supplied to the histogram unit 35) that the pixel corresponding to the minimum value min is a corresponding pixel, When it is determined that the evaluation is low, for example, “1” is supplied to the motion vector correction unit 36 as the control signal flag described above. At this time, as will be described later, the motion vector correction unit 36 corrects the motion vector vec of the target pixel 51.

  On the other hand, when the SAD minimum value evaluation unit 63 determines that the above-described evaluation is high, the SAD minimum value evaluation unit 63 supplies, for example, “0” to the motion vector correction unit 36 as the control signal flag. At this time, as described later, the motion vector correction unit 36 prohibits (does not correct) the correction of the motion vector vec of the target pixel 51.

  Specifically, for example, in the present embodiment, a value necessary for the SAD minimum value evaluation unit 63 to perform the above-described evaluation is calculated or detected by the SAD minimum value detection unit 62, and the SAD minimum value evaluation unit 36.

  That is, in the present embodiment, the value detected by the SAD minimum value detection unit 62 is the minimum value min described above, that is, the minimum value min of the correlation value SAD (j) (however, it is distinguished from the value min2 described below). In addition to the minimum value min1, the search position pos1 corresponding to the minimum value min1 and the next smallest correlation value SAD (j) (ie, the minimum value min1) are shown in FIG. This is the second smallest value among all the correlation values SAD (j), and is hereinafter referred to as a minimum value min2, and the search position pos2 corresponding to the minimum value min2.

In the present embodiment, the value calculated by the SAD minimum value detector 62 is the correlation value SAD (j) in the search range (in the example of FIG. 5, the range of search positions i-6 to i + 6). The difference value eval1 between the average value SAD _ave and the minimum value min1 (depth eval1 from the average value SAD _ave to the minimum value min1) and the difference value eval2 between the average value SAD _ave and the minimum value min2 (average value SAD _ave To the minimum value min2 (eval2).

In this case (when the search range is search positions i−6 to i + 6), the average value SAD _ave is calculated by the following equation (4).

・・・（４）

... (4)

  Returning to FIG. 4, each such value (minimum value min1, its search position pos1, and its depth eval1, and its minimum value min2, its search position pos2, and its depth eval2) is detected as a SAD minimum value. When supplied from the unit 62, the SAD minimum value evaluation unit 63 determines whether or not the following first condition or second condition is satisfied.

  That is, the first condition is that all of the following inequalities (5) to (7) are satisfied.

eval1> eval_thresh (5)
eval2> eval_thresh (6)
| Pos1-pos2 |> pos_thresh (7)

  In inequality (5) and inequality (6), eval_thresh indicates a predetermined threshold and is a preset value. Similarly, in inequality (7), pos_thresh represents a predetermined threshold and is a preset value.

  As can be easily understood from the inequality (5) to the inequality (7), as shown in FIG. 9 as described above (two or more minimum points exist, and each correlation value S (j) corresponding to them exists. ), That is, when there are two or more candidates for the corresponding pixel (corresponding point) of the center pixel 71 of the window 72 and no clear corresponding pixel is found (in the example of FIG. 9, the search position i The pixels arranged at +5 and the search position i−1 are candidates for corresponding pixels), and the first condition is satisfied.

  On the other hand, the second condition is that the following inequality (8) holds.

  min1> min_thresh (8)

  In inequality (8), min_thresh indicates a predetermined threshold value, which is a preset value.

  As can be easily understood from the inequality (8), in the end, when the minimum value min1 is not sufficiently small, that is, the correspondence between the window 72 and the comparison region 73-n corresponding to the minimum value min1 is firmly established. If not, the second condition will be met.

  Therefore, when it is determined that at least one of the first condition and the second condition is satisfied, the SAD minimum value evaluation unit 63 determines the reliability (histogram) that the pixel corresponding to the minimum value min1 is the corresponding pixel. The reliability of the motion vector candidate pvec of the target pixel 51 supplied to the unit 35 is evaluated to be low, and “1” is output to the motion vector correction unit 36 as the control signal flag.

  On the other hand, if it is determined that neither the first condition nor the second condition is satisfied, the SAD minimum value evaluation unit 63 determines that the pixel corresponding to the minimum value min1 is a corresponding pixel. It is evaluated that the degree (the reliability of the motion vector candidate pvec of the target pixel 51 supplied to the histogram unit 35) is high, and “0” is output to the motion vector correction unit 36 as the control signal flag.

  Returning to FIG. 2, the histogram unit 35 is supplied with the output result of the template matching unit 34 for an area composed of a plurality of pixels including the target pixel 51 in the display target frame. That is, the motion vector candidates pvec of the plurality of pixels constituting the area are supplied to the histogram unit 35. Therefore, the histogram unit 35 generates a histogram of the supplied plurality of motion vector candidates pvec, detects the most frequent motion vector candidate pvec with reference to the histogram, and uses the detected motion vector candidate pvec as the target pixel 51. To the motion vector correction unit 36 as a motion vector vec.

  The target region for generating the histogram is not particularly limited as long as it is a region including the target pixel 51. That is, the number of pixels constituting the area and the position of the target pixel 51 in the area are not particularly limited. However, here, for example, an area composed of 17 pixels arranged continuously in the spatial direction X with the target pixel 51 as the center, that is, continuous to the left of the target pixel 51 (opposite to the spatial direction X). A region composed of eight pixels lined up in parallel, a target pixel 51, and eight pixels lined up continuously to the right of the target pixel 51 (spatial direction X) is a target region for generating a histogram. .

  Specifically, for example, it is assumed that the vector candidate pvec of each pixel constituting the region 95 as shown in FIG. In the area 95 of FIG. 11, each of the 17 small areas indicates each pixel, and the central small area among the 17 small areas indicates the target pixel 51. Yes. In addition, the value described in each pixel (each small region) indicates an example of a motion vector candidate pvec of the corresponding pixel. However, “+” is omitted.

  In this case, the histogram unit 35 generates a histogram as shown in FIG. In the example of FIG. 12, since the motion vector candidate pvec with the highest frequency (frequency 7) is “+4”, the histogram unit 35 determines “+4” as the motion vector vec of the pixel 51 of interest. 2 motion vector correction unit 36.

  The motion vector correction unit 36 processes the subsequent processing (for example, the processing content of the image processing unit 12 in FIG. 1 described above) or the processing results (for example, the luminance gradient detection unit 33 and the template matching unit 34). Here, the motion vector vec of the target pixel 51 supplied from the histogram unit 35 based on the above-described luminance gradient (slope, control signal flag), or a combination thereof is used as another motion vector in pixels around the target pixel 51. Correct so that it is in harmony.

  The motion vector vec of the target pixel 51 corrected by the motion vector correction unit 36 is output to the outside via the LPF 37.

  Details of the motion vector correction unit 36 will be described below with reference to FIGS.

  FIG. 13 shows an example of a detailed configuration of the motion vector correction unit 36 of the present embodiment.

  As shown in FIG. 18 and FIG. 19 described later, the motion vector correction unit 36 can take various forms. Therefore, when it is necessary to individually distinguish each of the forms shown in FIGS. 13, 18, and 19 among the various forms that the motion vector correction unit 36 can take, hereinafter, the motion vector correction unit 36A, the motion These are referred to as a vector correction unit 36B and a motion vector correction unit 36C, respectively. On the other hand, when it is not necessary to distinguish each of the motion vector correction unit 36A, the motion vector correction unit 36B, and the motion vector correction unit 36C, they are collectively referred to as a motion vector correction unit 36.

  As illustrated in FIG. 13, the motion vector correction unit 36 </ b> A includes a post-processing correction unit 101, a switching unit 102, a reliability evaluation unit 103, and a reliability correction unit 104.

  The post-processing correction unit 101 corrects the motion vector vec of the target pixel 51 based on the processing content of the post-processing, for example, the feature of the processing content of the image processing unit 12 of FIG. 1 described above.

  Hereinafter, the motion vector vec before correction, the motion vector vec corrected by the post-processing correction unit 101, and the motion vector vec further corrected by the reliability correction unit 104 described later are individually distinguished. Are referred to as motion vector vec, motion vector vec ′, and motion vector vec ″, respectively. On the other hand, when it is not necessary to distinguish each of the motion vector vec, the motion vector vec ′, and the motion vector vec ″, they are collectively referred to as a motion vector vec.

  That is, the corrected motion vector vec ′ is output from the post-processing correction unit 101 and supplied to the switching unit 102.

  The correction method of the post-processing correction unit 101 is not particularly limited, and can be varied according to the characteristics of the processing content of the post-stage.

  Specifically, for example, the following method can be applied as a correction method suitable for the processing of the correction unit 22 in FIG.

  That is, for example, as described above, the correction unit 22 corrects the pixel value of the target pixel 51 supplied from the step edge detection unit 21 in FIG. 1 (including 0 correction). At this time, the correction unit 22 uses the motion vector vec of the target pixel 51 supplied from the motion detection unit 14 (actually, the corrected motion vector vec ′ or motion vector vec ″). The pixel value is corrected.

  Note that when viewed from the subsequent stage of the motion detection unit 14, for example, the image processing unit 12 or the like, it is not necessary to individually distinguish each of the motion vector vec, the motion vector vec ′, and the motion vector vec ″. Therefore, hereinafter, in the description when viewed from the subsequent stage of the motion detection unit 14, that is, the description of the processing content at the subsequent stage of the motion detection unit 14, the description will be simply referred to as the motion vector vec.

  For example, when the correction unit 22 determines the correction amount (value) of the pixel value of the target pixel 51, instead of using the correction value R that is the calculation result of the above equation (1), FIG. It is assumed that the correction amount is determined according to the relationship as shown.

  That is, FIG. 14 is a diagram illustrating an example of a method for determining the correction amount of the pixel value of the target pixel 51 in the correction unit 22, and is an example different from the above-described example (an example using Expression (1)). FIG.

  In the example of the correction method in FIG. 14 (processing contents of the correction unit 22), the pixel until the magnitude (absolute value) of the motion vector vec reaches 6 (“−6” or “+6”) which is the search range. While the correction amount of the value increases linearly, the correction amount of the pixel value thereafter becomes 0 after the search range 6 as a boundary (that is, the correction is prohibited). ).

  Therefore, if a motion vector vec having a size of about 6 is output from the motion detection unit 14 as a motion vector vec for the pixel of interest 51 and its neighboring pixels, in the range of the image consisting of the pixel of interest 51 and its neighborhood, Pixels whose pixel values are corrected with the maximum correction amount and pixels whose pixel values are not corrected at all are mixed.

  Specifically, for example, “+6” is output from the motion detection unit 14 as the motion vector vec of the target pixel 51, while “+7” is the motion vector vec of other pixels in the vicinity thereof. Output from.

  In this case, only the pixel value of the target pixel 51 is corrected with the maximum correction amount, and the pixel values of other pixels near the target pixel 51 are not corrected at all. In the finally obtained corrected image (display target frame), The pixel value (correction value) of the pixel 51 is different from the pixel value (value for which correction is prohibited) of the surrounding pixels. That is, the conventional second problem described above occurs.

  In other words, “+6”, which is the motion vector vec of the pixel 51 of interest, becomes a different vector from “+7”, which is the motion vector vec of other pixels in the vicinity thereof. That is, “+6”, which is the motion vector vec of the pixel of interest 51, becomes a motion vector that is not in harmony with the surroundings.

  Therefore, in order to reduce the number of outputs from the motion detection unit 14 of such motion vectors vec that are not in harmony with the surroundings, that is, the motion vector vec generated by the correction unit 22 exceeds the search range. In order not to generate the discontinuity of the correction amount of the pixel value as described above, the post-processing correction unit 101 in FIG. 13 changes the motion vector vec to the motion vector vec ′ according to the relationship shown in FIG. It can be corrected.

  That is, FIG. 15 shows an example of a method for correcting the motion vector vec of the target pixel 51 in the post-processing correction unit 101.

  In the example of the correction method in FIG. 15, the motion vector vec before correction, that is, the magnitude (absolute value) of the motion vector vec supplied from the histogram unit 35 is 3 (“−3” or “+3”). Up to, the correction is made in the relationship of “vec ′ = vec” (that is, the correction is prohibited), and after that, until “6” (“−6” or “+6”) Correction is made in the relationship of '= -vec + 6 or -6', and after 6 the correction is made in the relationship of 'vec' = 0 '.

  Therefore, if the motion vector vec ′ corrected according to the relationship of FIG. 15 is output from the detection unit 14, the magnitude (absolute value) of the motion vector vec ′ is 3 at the maximum, and the magnitude is The motion vector vec of about 6 is not output, and as a result, it is possible to prevent the occurrence of discontinuity in the correction amount of the pixel value as described above. That is, the conventional second problem can be solved.

  When the post-processing correction unit 101 in FIG. 13 performs correction in accordance with the relationship in FIG. 15, a table as shown in FIG. 15 may be held and the motion vector vec may be corrected with reference to the table. . Alternatively, the post-processing correction unit 101 can use the functions as described above, ie, “vec ′ = vec: −3 ≦ vec ≦ 3”, “vec ′ = − vec-6: −6 ≦ vec <−3”, Holds functions such as “vec '=-vec + 6: 3 <vec ≦ -6”, “vec' = 0: vec <-6, 6 <vec”, and substitutes the motion vector vec for the corresponding function. And the output value of the function may be output as a corrected motion vector vec ′.

  In addition, the example shown in FIG. 15 is repeated, but the correction unit in FIG. 1 is a subsequent process among various correction methods of the motion vector vec of the pixel of interest 51 in the subsequent process correction unit 101. This is an example of a correction method suitable for the case where the process 22 has the characteristics shown in FIG. 14 described above.

  That is, the important point here is that the correction method of the post-processing correction unit 101 is simply a correction method suitable for the characteristics of the subsequent processing contents. This is a variable.

  For example, the processing content of the subsequent stage (not shown) is a motion vector whose magnitude (absolute value) is medium (for example, 3) among the motion vectors vec of the target pixel 51 output from the motion detection unit 14. It is assumed that the processing is focused on vec and has a characteristic that the influence of the medium motion vector vec needs to be increased.

  In such a case, the post-processing correction unit 101 can correct the motion vector vec to the motion vector vec ′, for example, according to the relationship shown in FIG.

  That is, FIG. 16 shows another example (an example different from FIG. 15) of the correction method of the motion vector vec of the pixel of interest 51 in the post-processing correction unit 101.

  In the example of the correction method in FIG. 16, the magnitude (absolute value) of the motion vector vec before correction, that is, the motion vector vec supplied from the histogram unit 35 is 3 (“−3” or “+3”). Is corrected according to the relationship of “vec '= 2vec”, and after that point, until “6” (“−6” or “+6”), “vec ′ = − 2vec + 12 or −” It is corrected in the relationship of “12”, and after 6 it is corrected in the relationship of “vec ′ = 0”.

  When the post-processing correction unit 101 performs correction according to the relationship of FIG. 16, similarly to the example of FIG. 15, a table as shown in FIG. 16 is held, and the motion vector vec is referred to by referring to the table. It may be corrected. Alternatively, the post-processing correction unit 101 can use the functions described above, that is, “vec ′ = 2vec: −3 ≦ vec ≦ 3”, “vec ′ = − 2vec-12: −6 ≦ vec <−3”, Holds functions such as “vec '=-2vec + 12: 3 <vec ≦ -6”, “vec' = 0: vec <-6, 6 <vec”, and substitutes the motion vector vec for the corresponding function. And the output value of the function may be output as a corrected motion vector vec ′.

  Further, when there are a plurality of subsequent processes or there is a possibility that there are a plurality of processes, the post-processing correction unit 101 can also correct the motion vector vec by appropriately switching the correction method. That is, when there are Q processes in the subsequent stage (Q is an arbitrary integer value equal to or greater than 1), the post-process correction unit 101 converts one motion vector vec of the pixel of interest 51 into each of Q types of correction methods. The Q motion vectors vec ′ obtained as a result of the correction can be output individually.

  Returning to FIG. 13, the motion vector vec of the target pixel 51 output from the histogram unit 35 is corrected by the post-processing correction unit 101 in this way to become a motion vector vec ′, and is supplied to the switching unit 102. .

  Based on the control of the reliability evaluation unit 103, the switching unit 102 switches the output destination to one of the external LPF 37 side and the reliability correction unit 104 side.

  The reliability evaluation unit 103 determines the motion vector vec of the pixel of interest 51 supplied from the histogram unit 35 based on the luminance gradient slope supplied from the luminance gradient detection unit 33 and the control signal flag supplied from the template matching unit 34. Evaluate confidence.

  When the reliability evaluation unit 103 evaluates that the reliability of the motion vector vec of the pixel of interest 51 is low, the reliability evaluation unit 103 switches the output destination of the switching unit 102 to the reliability correction unit 104 side.

  Accordingly, when the reliability evaluation unit 103 evaluates that the reliability of the motion vector vec of the target pixel 51 is low, the motion vector vec ′ corrected and output by the post-processing correction unit 101 is output as the reliability correction unit. 104. As will be described later, the motion vector vec ′ corrected by the post-processing correction unit 101 is further corrected by the reliability correction unit 104 to become a motion vector vec ″, and this motion vector vec ″ is converted to the LPF 37. To the outside of the motion detection unit 14 (the image processing unit 11, the image processing unit 12, and the switching unit 15 in FIG. 1).

In contrast, when the reliability evaluation unit 103 evaluates that the reliability of the motion vector vec of the target pixel 51 is high, the reliability evaluation unit 103 switches the output destination of the switching unit 102 to the external LPF 37 side.

  Therefore, when the reliability evaluation unit 103 evaluates that the reliability of the motion vector vec of the target pixel 51 is high, the motion vector vec ′ corrected and output by the post-processing correction unit 101 is output as the reliability correction unit. Instead of being supplied to 104, it is supplied as it is to the outside of the motion detector 14 via the LPF 37.

  Specifically, for example, the reliability evaluation unit 103 determines that at least one of the third condition that the following inequality (9) is satisfied and the fourth condition that the control signal flag is “1” is When it is satisfied, it is evaluated that the reliability of the motion vector vec of the target pixel 51 supplied from the histogram unit 35 is low, and the output destination of the switching unit 102 is switched to the reliability correction unit 104 side. .

  On the other hand, when neither the third condition nor the fourth condition is satisfied, the motion vector correction unit 36 has high reliability of the motion vector vec of the target pixel 51 supplied from the histogram unit 35. And the output destination of the switching unit 102 is switched to the external LPF 37 side.

  slope <slope_thresh (9)

  In the inequality (9), slope_thresh indicates a predetermined threshold and is a preset value.

  As can be easily understood from the inequality (9), in the end, when the luminance gradient in the target pixel 51 is low, that is, the target pixel 51 corresponds to a characteristic portion such as the edge portion of the step edge described above. If not, the third condition will be met.

  Further, as described above, the control signal flag can also be said to be a signal indicating the evaluation result of the reliability of the motion vector vec of the target image 51 in the template matching unit 34. Therefore, when the control signal flag is “1”, that is, When the template matching unit 34 has already evaluated that the reliability of the motion vector vec of the target image 51 is low, the fourth condition is satisfied.

  The reliability correction unit 104, when the output destination of the switching unit 102 is switched to its own side, the motion vector vec 'output from the post-processing correction unit 101, that is, input from the histogram unit 35. The motion vector vec ′ output as a result of correcting the motion vector vec ′ by the post-processing correction unit 101 is further corrected by a predetermined correction method, and the resulting motion vector vec ″ is obtained via the LPF 37. The data is output to the outside of the motion detection unit 14 (the image processing unit 11, the image processing unit 12, and the switching unit 15 in FIG. 1).

  The correction method of the reliability correction unit 104 is not particularly limited, and various methods can be applied.

  Specifically, for example, the motion vector vec of the target pixel 51 is used here by the image processing unit 12 described above. That is, the image processing unit 12 corrects the pixel value of the target pixel 51 according to the direction and size of the motion vector vec of the target pixel 51 (corrects in the direction of image enhancement). In such a case, when the motion vector vec of the pixel of interest 51 with low reliability is used, there arises a problem that the pixel value of the pixel of interest 51 is overcorrected.

  In order to solve this problem, for example, the reliability correction unit 104 performs the following on the motion vector vec ′ obtained as a result of correcting the motion vector vec of the target pixel 51 by the post-processing processing correction unit 101. The correction as shown on the right side of Expression (10) is performed, and the correction result, that is, the value vec '' on the left side of Expression (10) is used as the final (corrected) motion vector of the pixel 51 of interest. It can be output to the outside of the motion detector 14 via the LPF 37.

  vec '' = α × vec '(10)

  In Expression (10), α indicates a correction coefficient. The correction coefficient α is a value that can be arbitrarily set in the range of 0 to 1.

  By the way, the reliability evaluation unit 103 can also evaluate the reliability of the motion vector vec of the pixel of interest 51 supplied from the histogram unit 35 based on conditions other than the third condition and the fourth condition described above. It is.

  Specifically, for example, the next smallest correlation value SAD (j) after the minimum value min1, that is, the search position pos2 corresponding to the minimum value min2 is the arrangement position i of the pixel 51 of interest as shown in FIG. In some cases, the corresponding pixel (corresponding point) of the center pixel 71 of the window 72 is not the pixel at the search position pos1 corresponding to the minimum value min1 but the target pixel 51 at the arrangement position i (search position pos2) in many cases.

  Therefore, for example, in addition to the fifth condition that the search position pos2 corresponding to the minimum value min2 is the arrangement position i of the target pixel 51, and when the fifth condition is satisfied, the reliability evaluation of FIG. The unit 103 can evaluate that the reliability of the motion vector vec of the target pixel 51 supplied from the histogram unit 35 is low, and can switch the output destination of the switching unit 102 to the reliability correction unit 104 side. As a result, the reliability correction unit 104 further corrects the motion vector vec ′ obtained as a result of correcting the motion vector vec by the post-processing correction unit 101.

  When the fifth condition is satisfied, although not shown, the reliability evaluation unit 103 notifies the reliability correction unit 104 of the fact, so that the reliability correction unit 104 is centered on the window 72. The corresponding pixel (corresponding point) of the pixel 71 is regarded as the target pixel 51 at the arrangement position i (search position pos2), that is, it is considered that there is no movement at the arrangement position i of the target pixel 51, and the correction processing unit 101 for subsequent processing The corrected motion vector vec ′ of the target pixel 51 is replaced with “0”. Alternatively, the reliability correction unit 104 sets the correction coefficient α in the above-described equation (10) to 0 and corrects the motion vector vec ′ of the target pixel 51 corrected by the post-processing correction unit 101. Also good. That is, when the fifth condition is satisfied, the reliability correction unit 104 corrects the motion vector vec ′ of the pixel of interest 51 corrected by the post-processing correction unit 101 to a zero vector.

  However, in this case, the template matching unit 34 (SAD minimum value evaluation unit 63 in FIG. 4) needs to supply the search position pos2 to the reliability evaluation unit 103 in addition to the control signal flag.

  Alternatively, the SAD minimum value evaluation unit 63 may forcibly output “1” as the control signal flag when the fifth condition is satisfied.

  The configuration example of the motion vector correction unit 36A of the embodiment shown in FIG. 13 among the various embodiments that the motion vector correction unit 36 of FIG. 2 can take has been described above.

  The motion vector correction unit 36 in FIG. 2 can also be configured, for example, as a motion vector correction unit 36B shown in FIG. 18 or a motion vector correction unit 36C shown in FIG.

  The motion vector correction unit 36B of FIG. 18 includes a post-processing processing correction unit 101. The talk processing correction unit 101 has basically the same function and configuration as the corresponding one of the motion vector correction unit 36A of FIG. Accordingly, the motion vector correction unit 36B in FIG. 18 does not perform the correction processing corresponding to the reliability correction unit 104 in FIG. 13 on the motion vector vec input from the histogram unit 35, and performs the subsequent-stage processing correction unit. Only the correction processing by 101 is performed.

  On the other hand, the configuration of the motion vector correction unit 36C in FIG. 19 is configured such that the post-processing correction unit 101 is omitted from the configuration of the motion vector correction unit 36A in FIG. Accordingly, the motion vector correction unit 36C in FIG. 19 does not perform the correction process corresponding to the subsequent-stage processing correction unit 101 in FIG. 13 on the motion vector vec input from the histogram unit 35, and the reliability correction unit 36C. Only the correction process according to 104 is performed (however, when the reliability evaluation unit 103 determines that the reliability of the motion vector vec is low).

  The configuration example of the motion detection unit 14 in FIG. 2 has been described with reference to FIGS. 2 to 19. The contents of the description are summarized as follows.

  That is, in the motion detection unit 14 according to the present embodiment, a predetermined pixel of the pixels constituting the display target frame is set as a target pixel, a motion vector at the target pixel is generated, and the reliability of the motion vector is determined. In addition, it can be corrected based on the characteristics of the processing contents of the subsequent stage and output to the outside (image processing unit 11, image processing unit 12, and switching unit 15 in FIG. 1).

  Specifically, the template matching unit 34 generates a motion vector candidate pvec at the target pixel by comparing the display target frame (image data) with the immediately preceding frame (image data), and provides the motion vector candidate pvec to the histogram unit 35.

  In other words, the template matching unit 34 sets the first pixel (for example, as shown in FIG. 5, the pixel 51 is set as the target pixel) arranged at the position in the immediately preceding frame corresponding to the arrangement position of the target pixel. The second pixel (for example, any one of the search ranges i-6 to i + 6 in FIG. 5) is detected from the display target frame as the corresponding pixel of the pixel 71) at the arrangement position i. A vector starting from the target pixel and ending at the second pixel (for example, as shown in FIG. 5, the pixel 51 is set as the target pixel, and the center pixel of the pixel group 72-(+ 6) is set as the second pixel. That is, when a pixel at the arrangement position i + 6 is detected, “+6”) is generated as a motion vector candidate pvec at the target pixel and provided to the histogram unit 35.

  Further, at this time, the template matching unit 34 determines that there are a plurality of corresponding pixel candidates in the display target frame (for example, as shown in FIG. 9, the pixel at the search position i-1 and the search position This is a case where the pixel of i + 5 is a candidate for the corresponding pixel, and specifically, for example, when the first condition that all of the above inequality (5) to inequality (7) are satisfied is satisfied) Or, when it is determined that the reliability of the second pixel being a corresponding pixel is low (for example, when the second condition that the above inequality (8) is satisfied) is satisfied, a motion vector correction command Is provided to the motion vector correction unit 36 (that is, “1” as the control signal flag here).

  The histogram unit 35 is a motion vector candidate pvec (for example, each numerical value in each pixel (frame) in the region 95 in FIG. 11) in each of the target pixel and the surrounding pixels generated by the template matching unit 34. The motion vector candidate having the highest frequency (for example, “+4” having the highest frequency in the histogram of FIG. 12) is determined as the motion vector vec for the target pixel, and is output to the motion vector correction unit 36.

  The luminance gradient detection unit 33 calculates the degree of change in luminance around the target pixel (for example, the luminance gradient described above, specifically, for example, the value slope represented by the above-described equation (2)), This is provided to the motion vector correction unit 36.

  The motion vector correction unit 36 corrects the motion vector vec supplied from the histogram unit 35 based on the feature of the processing content of the subsequent stage (for example, the correction unit 22 of FIG. 1), and the feature vector vec ′ obtained as a result thereof Can be directly output to the outside (the image processing unit 11, the image processing unit 12, and the switching unit 15 in FIG. 1), and can be output to the outside after further performing the following correction processing. You can also.

  That is, the motion vector correction unit 36 is based on the processing result of the luminance gradient detection unit 33 (here, the luminance gradient slope) and the processing result of the template matching unit 34 (here, the control signal flag) from the histogram unit 35. When the reliability of the supplied motion vector vec is evaluated and it is determined that the reliability of the motion vector vec is low, specifically, for example, the third condition that the inequality (9) described above is satisfied, and FIG. When at least one of the fourth condition that the control signal flag supplied from the second template matching unit 34 is “1” is satisfied, the motion vector vec corrected according to the processing content of the subsequent stage 'Can be further corrected, and the resulting motion vector vec' 'can be output to the outside (the image processing unit 11, the image processing unit 12, and the switching unit 15 in FIG. 1).

  Alternatively, the motion vector correction unit 36 is supplied from the histogram unit 35 based on the processing result of the luminance gradient detection unit 33 and the processing result of the template matching unit 34 without performing the correction process according to the processing content of the subsequent stage. When the reliability of the motion vector vec is evaluated and it is determined that the reliability of the motion vector vec is low, the motion vector vec supplied from the histogram unit 35 is corrected and externally (the image processing unit 11 in FIG. It is also possible to output to the processing unit 12 and the switching unit 15).

  In this way, the motion detection unit 14 of the present embodiment can output a motion vector vec (actually, a corrected motion vector vec ′ or motion vector vec ″) that is in harmony with the surroundings. become. As a result, for example, the image processing unit 12 or the like in FIG. 1 accurately suppresses motion blur (particularly motion blur caused by follow-up vision) that occurs in the hold-type display device 2 so that there is no sense of discomfort with the surroundings. It becomes possible.

  Next, image processing of the image processing apparatus 1 (FIG. 1) of the present embodiment will be described with reference to the flowchart of FIG.

  First, in step S1, the image processing apparatus 1 inputs image data of a display target frame. Specifically, the image data of the display target frame is input to each of the image processing unit 11, the image processing unit 12, the reference image storage unit 13, and the motion detection unit 14.

  In step S <b> 2, the image processing apparatus 1 (the image processing unit 11, the image processing unit 12, the motion detection unit 14, and the like) sets a target pixel from among a plurality of pixels that constitute the display target frame.

  In step S <b> 3, the motion detection unit 14 compares the image data of the display target frame with the image data of the reference image (previous frame) stored in the reference image storage unit 23, so that the motion vector vec of the target pixel is obtained. And the motion vector vec is corrected if necessary, and supplied to each of the image processing unit 11, the image processing unit 12, and the switching unit 15.

  Such processing of the motion detection unit 14 (processing of step S3) is hereinafter referred to as “motion vector calculation processing”. The details of the “motion vector calculation process” will be described later with reference to the flowchart of FIG.

  In step S <b> 4, the image processing apparatus 1 (the image processing unit 11, the image processing unit 12, the switching unit 15, etc.) determines whether or not the magnitude of the motion vector vec of the target pixel is equal to or greater than a threshold value.

  Actually, in the processing of steps S4 to S7, as a result of the “motion vector calculation processing” that is the processing of step S3 of the motion detection unit 14, the motion vector vec ′ or the motion vector vec ′ of the pixel of interest corrected. 'Will be used, but in the description of the processing of steps S4 to S7, since it is not necessary to distinguish them individually, as described above, they are simply referred to as motion vectors vec.

  When it is determined in step S4 that the magnitude of the motion vector vec is less than the threshold (when it is determined that the magnitude of the motion vector vec is not greater than or equal to the threshold), that is, when there is no motion in the target pixel, switching is performed. The unit 15 switches the input to the end on the image processing unit 11 side. Then, in step S <b> 5, the image processing unit 11 performs predetermined image processing on the target pixel to correct the pixel value of the target pixel, and the corrected pixel value is displayed via the switching unit 15. To supply.

  On the other hand, if it is determined in step S4 that the magnitude of the motion vector vec is greater than or equal to the threshold value, that is, if there is motion in the pixel of interest, the switching unit 15 converts the input into the image processing unit 12 (correction). Switch to the end on the part 22) side.

  At this time, in step S <b> 6, the step edge detection unit 21 determines the pixel value of the target pixel and a predetermined direction (in this case, the spatial direction X or the opposite direction, and the motion vector vec supplied from the motion detection unit 24. The difference value with the pixel value of the pixel adjacent to the pixel (determined according to the direction (plus or minus)) is calculated, and the calculated difference value and the pixel value of the target pixel are supplied to the correction unit 22.

  In step S <b> 7, the correction unit 22 receives the attention supplied from the step edge detection unit 21 based on the motion vector of the target pixel supplied from the motion detection unit 14 and the difference value supplied from the step edge detection unit 21. The pixel value of the pixel is corrected, and the corrected pixel value is supplied to the display control unit 16 via the switching unit 15.

  In step S8, the display control unit 16 sets the pixel value of the target pixel supplied from the image processing unit 11 or the image processing unit 12 via the switching unit 15 (if necessary, the pixel value is stored in the hold type display device). 2) and output to the hold type display device 2. That is, the display control unit 16 outputs the pixel value of the target pixel to the hold type display device 2 as a target level for the display element corresponding to the target pixel among the display elements of the hold type display device 2.

  In step S9, the image processing apparatus 1 determines whether or not the pixel values of all the pixels have been output.

  If it is determined in step S9 that the pixel values of all the pixels have not yet been output, the process returns to step S2, and the subsequent processes are repeated. That is, of the plurality of pixels constituting the display target frame, pixels that have not yet been processed are sequentially set as the target pixel, the pixel value of the target pixel is corrected (including 0 correction), and the hold-type display device 2 Is output.

  When the above process is repeated and the pixel values of all the pixels constituting the display target frame are supplied to the hold type display device 2, it is determined in step S9 that the pixel values of all the pixels have been output, and the process Advances to step S10.

  At this time, the hold-type display device 2 applies a voltage of a level corresponding to the supplied pixel value (target level) to each of the display elements (liquid crystal or the like) constituting the screen, and the next frame. Until the display is instructed (until the pixel values of all the pixels constituting the next frame are supplied), the application of the voltage of that level is continued. That is, each display element holds and displays the corresponding pixel.

  In step S10, the image processing apparatus 1 determines whether or not all the frames constituting the moving image have been processed.

  If it is determined in step S10 that all the frames have not been processed yet, the process returns to step S1, the image data of the next frame is input as the image data of the display target frame, and the subsequent processing is repeated. It is.

  Then, when the pixel values of all the pixels constituting the last frame among the plurality of frames constituting the moving image are corrected (including 0 correction) and output to the hold type display device 2, in step S10, It is determined that all the frames have been processed, and the image processing of the image processing apparatus 1 is ended.

  In the example of FIG. 20, the image processing apparatus 1 individually outputs each pixel value (corrected pixel value) of each pixel constituting the display target frame to the hold type display apparatus 2. However, after correcting the pixel values of all the pixels constituting the display target frame, they may be output collectively (as an image signal of the display target frame).

  Next, the “motion vector calculation process (the process of step S3 of FIG. 20)” of the motion detection unit 14 of FIG. 2 will be described with reference to the flowchart of FIG.

  First, in step S <b> 21, the luminance gradient detection unit 33 detects a luminance gradient slope around the target pixel and supplies it to the motion vector correction unit 36.

  In step S <b> 22, the template matching unit 34 calculates each motion vector candidate pvec of each pixel (for example, each pixel configuring the region 95 in FIG. 11) that configures the pixel group including the target pixel, and the histogram unit 35 Supply.

  In step S <b> 23, the template matching unit 34 generates a control signal (flag) flag indicating whether the motion vector vec of the target pixel can be corrected, and supplies the control signal (flag) flag to the motion vector correction unit 36.

  In step S24, the histogram unit 35 generates a histogram (for example, the histogram of FIG. 12) of each motion vector candidate pvec of each pixel constituting the pixel group including the target pixel, and selects the most frequent motion vector candidate pvec. The motion vector vec of the pixel of interest is determined and supplied to the motion vector correction unit 36.

  In step S25, the post-stage correction unit 101 of the motion vector correction unit 36A in FIG. 13 corrects the motion vector vec of the target pixel using the first correction method, and the resulting motion vector vec ′ is sent to the switching unit 102. Supply.

  Note that when the motion vector correction unit 36 of FIG. 2 is configured as the motion vector correction unit 36C of FIG. 19, the process of step S25 is omitted.

  On the other hand, when the motion vector correction unit 36 of FIG. 2 is configured as the motion vector correction unit 36B of FIG. 18, the process of step S25 is executed, but the processes of steps S26 to S28 described later are omitted. become.

  As described above, the contents of the “motion vector calculation process” are slightly different depending on the form of the motion vector correction unit 36 in FIG. 2. In other words, FIG. 21 is a flowchart for explaining an example of “motion vector calculation processing” in the case where the motion vector correction unit 36 in FIG. 2 is configured as the motion vector correction unit 36A in FIG.

  By the way, when the process of step S25 described above is completed, the process proceeds to step S26.

  In step S26, the reliability evaluation unit 103 in FIG. 13 includes the control signal (flag) flag generated by the template matching unit 34 in step S23 and the luminance detected by the luminance gradient detection unit 33 in step S21. The reliability of the motion vector vec is evaluated based on the slope.

  In step S27, the reliability evaluation unit 103 determines whether the reliability of the motion vector vec is low based on the processing result of step S26.

  In step S27, when the reliability evaluation unit 103 determines that the reliability of the motion vector vec is high (not low), the output destination of the switching unit 102 is switched to the external LPF 37 side, and the process proceeds to step S29.

  In step S29, the switching unit 102 outputs the motion vector vec ′ corrected by the first-stage correction unit 101 in the processing in step S25 to the outside of the motion detection unit 14 via the LPF 37 (the image in FIG. 1). Output to the processing device 11, the image processing device 12, and the switching unit 15).

  On the other hand, in step S27, if the reliability evaluation unit 103 determines that the reliability of the motion vector vec is low, the output destination of the switching unit 102 is switched to the reliability correction unit 104 side, and the process is performed in step S28. Proceed to

  Then, the motion vector vec ′ corrected by using the first correction method in the process of step S25 by the post-processing correction unit 101 is supplied to the reliability correction unit 104. The degree correction unit 104 further corrects the motion vector vec ′ (corrected by the post-processing correction unit 101) of the target pixel by the second correction method, and in step S29, the corrected motion vector vec ′. 'Is output to the outside of the motion detector 14 via the LPF 37.

In this way, the motion vector vec ′ corrected for the pixel of interest in the process of step S29.
Alternatively, when the motion vector vec ″ is output, the “motion vector calculation process” ends.

  By the way, the image processing apparatus to which the present invention is applied is not limited to the configuration of FIG. 1 described above, and can take various embodiments.

  For example, an image processing apparatus to which the present invention is applied can be configured as shown in FIG. That is, FIG. 22 shows an example of the configuration of an image processing apparatus according to another embodiment of the present invention, and portions corresponding to those of the image processing apparatus 1 of FIG.

  As shown in FIG. 22, the image processing apparatus 151 of this embodiment includes image processing units 11 to 16 having basically the same configuration and function as the image processing apparatus 1 of FIG. The basic connection form is the same as that of FIG.

  However, in the image processing apparatus 1 of FIG. 1, the detection result (motion vector vec) of the motion detection unit 14 is supplied to the step edge detection unit 21. However, in the image processing apparatus 151 of FIG. The detection result of the unit 14 is not supplied to the step edge detection unit 21. Conversely, the detection result of the step edge detection unit 21 is supplied to each of the motion detection unit 14 and the image processing unit 11.

  Since the image processing apparatus 151 has such a configuration, the following operation is performed.

  In other words, in the image processing apparatus 151, first, the step edge detection unit 21 detects a pixel corresponding to the step edge from a plurality of pixels constituting a predetermined frame, and the detection result is displayed as a result of the correction unit 22. In addition, the data is supplied to each of the motion detection unit 14 and the image processing unit 11.

  Thereby, the motion detection unit 14 can execute the process only for the pixels detected by the step edge detection unit 21 (pixels corresponding to the step edges). That is, it can be said that the motion detector 14 detects whether or not the step edge detected by the step edge detector 31 is moving.

  Further, the image processing unit 11 prohibits the processing of the pixels detected by the motion detection unit 14 among the pixels detected by the step edge detection unit 21 (pixels corresponding to the step edges). To do. In other words, the image processing unit 11 prohibits the process for pixels corresponding to a moving step edge, and executes the process only for other pixels.

  As described above, in the image processing device 151 of FIG. 22, image processing for one pixel is executed by only one of the image processing unit 11 and the image processing unit 12. In other words, image processing is performed only once on one frame. Furthermore, the process of the motion detection unit 14 is also executed only for the pixels corresponding to the step edges. Accordingly, it is possible to suppress the overall processing amount of the image processing apparatus 151.

  The series of processes described above can be executed by hardware, but can also be executed by software.

  In this case, the image processing apparatus 1 in FIG. 1 and the image processing apparatus 151 in FIG. 22 can be configured by, for example, a personal computer as shown in FIG.

  23, a CPU (Central Processing Unit) 201 executes various processes according to a program recorded in a ROM (Read Only Memory) 202 or a program loaded from a storage unit 208 to a RAM (Random Access Memory) 203. To do. The RAM 203 also appropriately stores data necessary for the CPU 201 to execute various processes.

  The CPU 201, the ROM 202, and the RAM 203 are connected to each other via the bus 204. An input / output interface 205 is also connected to the bus 204.

  The input / output interface 205 includes an input unit 206 including a keyboard and a mouse, an output unit 207 including a display, a storage unit 208 including a hard disk, and a communication unit 209 including a modem and a terminal adapter. It is connected. The communication unit 209 performs communication processing with another information processing apparatus (not shown) via a network including the Internet.

  In this case, the output unit 207 itself may be a hold-type display device, or an external hold-type display device 2 ( 1) may be connected.

  A drive 210 is connected to the input / output interface 205 as necessary, and a removable recording medium 211 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted, and a computer program read from them is read. Are installed in the storage unit 208 as necessary.

  When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, a general-purpose personal computer is installed from a network or a recording medium.

  As shown in FIG. 23, the recording medium including such a program is distributed to provide a program to the user separately from the apparatus main body, and a magnetic disk (including a floppy disk) on which the program is recorded. Removable recording media (package) consisting of optical disks (including compact disk-read only memory (CD-ROM), DVD (digital versatile disk)), magneto-optical disk (including MD (mini-disk)), or semiconductor memory Medium) 211, and a ROM 202 in which a program is recorded and a hard disk included in the storage unit 208 provided to the user in a state of being incorporated in the apparatus main body in advance.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the order, but is not necessarily performed in chronological order, either in parallel or individually. The process to be executed is also included.

  In the above description, each display element (liquid crystal in the case of a liquid crystal display device) constituting the screen of the hold type display device 2 has a predetermined one of a plurality of pixels constituting the frame. Although associated with each other, a plurality of display elements may be associated with one pixel. That is, one pixel may be displayed by a plurality of display elements.

  Further, in the above description, the image processing apparatus generates a motion vector parallel to the spatial direction X. As described above, the image processing apparatus performs a process basically similar to the series of processes described above, thereby performing spatial processing. A motion vector parallel to the direction Y can be generated, or a motion vector in an arbitrary direction on a two-dimensional plane parallel to the spatial direction X and the spatial direction Y can be generated.

It is a block diagram which shows the structural example of the image processing apparatus of this Embodiment. It is a block diagram which shows the detailed structural example of the motion detection part of the image processing apparatus of FIG. It is a figure which shows the example of a process target of the brightness | luminance gradient detection part of FIG. It is a block diagram which shows the detailed structural example of the template matching part of the motion detection part of FIG. It is a figure which shows the example of a process target of the template matching part of FIG. It is a figure which shows the example of each pixel value of each pixel around the attention pixel of a display object frame, and each pixel value of each corresponding pixel of the frame immediately before that. It is a figure which shows the example of the calculation result with respect to FIG. 6 of the SAD calculating part of FIG. It is a figure explaining the value which the template matching part of FIG. 4 uses for a process. It is a figure which shows the other pixel example (different from FIG. 6) of each pixel value of each pixel around the pixel of interest of a display target frame, and each pixel value of each corresponding pixel of the immediately preceding frame. It is a figure which shows the other example (example different from FIG. 7) of the calculation result with respect to FIG. 6 of the SAD calculating part of FIG. It is a figure which shows the example of a process target of the histogram part of FIG. It is a figure which shows the example of the histogram with respect to FIG. FIG. 3 is a block diagram illustrating a detailed configuration example of a motion vector correction unit of the motion detection unit in FIG. 2. It is a figure which shows the example of the process content of a back | latter stage which the correction | amendment part for a back | latter stage process of the motion detection part of FIG. 13 should consider in the correction process. FIG. 15 is a diagram illustrating an example of a correction method for the post-processing correction unit of the motion detection unit of FIG. 13 to perform correction suitable for the characteristics of the processing content of the post-stage of the example of FIG. It is a figure which shows the other example of the correction method of the correction | amendment part for a back | latter stage process of the motion detection part of FIG. It is a figure which shows the result of SAD explaining other embodiment of the template matching part of FIG. It is a block diagram which shows the other example of a detailed structure of the motion vector correction | amendment part of the motion detection part of FIG. FIG. 10 is a block diagram showing still another example of a detailed configuration of a motion vector correction unit of the motion detection unit in FIG. 2. 2 is a flowchart illustrating image processing of the image processing apparatus in FIG. 1. FIG. 21 is a flowchart for describing motion vector calculation processing in FIG. 20. FIG. It is a block diagram which shows the other structural example (configuration example different from FIG. 1) of the image processing apparatus of this Embodiment. It is a block diagram which shows the further another structural example (configuration example different from FIG.1 and FIG.22) of the image processing apparatus of this Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Image processing apparatus, 2 Hold-type display apparatus, 11 Image processing part, 12 Image processing part, 13 Reference image memory | storage part, 14 Motion detection part, 15 Switching part, 16 Display control part, 21 Step edge detection part, 22 Correction | amendment part , 31 Low pass filter (LPF), 32 LPF, 33 Luminance gradient part, 34 Template matching part, 35 Histogram part, 36, 36A, 36B, 36C Motion vector correction part, 37 LPF, 51 pixel of interest, 61 SAD operation part, 62 SAD minimum detection unit, 63 SAD minimum value evaluation unit, 101 post-processing correction unit, 102 switching unit, 103 reliability evaluation unit, 104 reliability correction unit 201 CPU, 202 ROM, 203 RAM, 208 storage unit, 211 removable recoding media

Claims (11)

By setting a predetermined pixel of the pixels constituting the first access unit as a target pixel, and comparing the first access unit with the second access unit immediately before the first access unit. , Candidate generation means for generating a motion vector candidate at the pixel of interest;
Motion vector determining means for determining the motion vector candidate having the highest frequency among the motion vector candidates in each of the target pixel generated by the candidate generating means and surrounding pixels as a motion vector in the target pixel; ,
Luminance change calculating means for calculating the degree of change in luminance around the pixel of interest;
When evaluating the reliability of the motion vector determined by the motion vector determination unit based on the processing results of the luminance change calculation unit and the candidate generation unit, and when evaluating that the reliability of the motion vector is low, An image processing apparatus comprising: correction means for correcting the motion vector. The correction means evaluates that the reliability of the motion vector is low when the degree of change in the brightness calculated by the brightness change calculation means is less than a threshold, and corrects the motion vector. The image processing apparatus according to claim 1. The candidate generation means detects a second pixel from the first access unit as a corresponding pixel of the first pixel arranged at a position in the second access unit corresponding to the arrangement position of the target pixel. The image processing apparatus according to claim 1, wherein a vector starting from the target pixel and starting from the second pixel is generated as the motion vector candidate for the target pixel. If the candidate generation unit further determines that there are a plurality of candidates for the corresponding pixel in the first access unit, or if the reliability that the second pixel is the corresponding pixel is low If determined, providing the correction means with the first information indicating the motion vector correction command,
The said correction | amendment means evaluates that the reliability of the said motion vector is low, and correct | amends the said motion vector, when the said 1st information is provided from the said candidate production | generation means. Image processing apparatus. The candidate generation means further provides the correction means with second information indicating that the pixel of interest itself is included in a plurality of candidates for the corresponding pixel,
The correction unit, when the second information is provided from the candidate generation unit, evaluates that the reliability of the motion vector is low and corrects the motion vector to a zero vector. 5. The image processing apparatus according to 4. By setting a predetermined pixel of the pixels constituting the first access unit as a target pixel, and comparing the first access unit with the second access unit immediately before the first access unit. , A candidate generation step of generating a motion vector candidate at the target pixel;
Motion vector determination for determining the motion vector candidate having the highest frequency among the motion vector candidates in each of the target pixel and its surrounding pixels generated by the processing of the candidate generation step as a motion vector in the target pixel. Steps,
A luminance change calculating step for calculating the degree of luminance change around the target pixel;
Based on the processing results of the luminance change calculation step and the candidate generation step, the reliability of the motion vector determined by the processing of the motion vector determination step was evaluated, and the reliability of the motion vector was evaluated to be low A correction step of correcting the motion vector. A program that sets a predetermined pixel of pixels constituting the first access unit as a target pixel and causes a computer to execute image processing using the target pixel as a unit,
A candidate generation step of generating a motion vector candidate at the pixel of interest by comparing the first access unit with a second access unit immediately before the first access unit;
Motion vector determination for determining the motion vector candidate having the highest frequency among the motion vector candidates in each of the target pixel and its surrounding pixels generated by the processing of the candidate generation step as a motion vector in the target pixel. Steps,
A luminance change calculating step for calculating the degree of luminance change around the target pixel;
Based on the processing results of the luminance change calculation step and the candidate generation step, the reliability of the motion vector determined by the processing of the motion vector determination step was evaluated, and the reliability of the motion vector was evaluated to be low A correction step of correcting the motion vector. By setting a predetermined pixel of the pixels constituting the first access unit as a target pixel, and comparing the first access unit with the second access unit immediately before the first access unit. , Candidate generation means for generating a motion vector candidate at the pixel of interest;
Motion vector determining means for determining the motion vector candidate having the highest frequency among the motion vector candidates in each of the target pixel generated by the candidate generating means and surrounding pixels as a motion vector in the target pixel; ,
Correction means for correcting the motion vector determined by the motion vector determination means;
Processing execution means for executing predetermined processing using the motion vector corrected by the correction means;
The image processing apparatus, wherein the correction unit corrects the motion vector by using a first correction method based on a characteristic of the predetermined process of the process execution unit. A luminance change calculating means for calculating the degree of change in luminance around the target pixel;
The correction means further evaluates the reliability of the motion vector determined by the motion vector determination means based on the processing results of the brightness change calculation means and the candidate generation means, and the reliability of the motion vector The image processing apparatus according to claim 8, wherein when the evaluation is low, the motion vector corrected by the first correction method is further corrected by a second correction method. In the information processing method of the information processing apparatus,
By setting a predetermined pixel of the pixels constituting the first access unit as a target pixel, and comparing the first access unit with the second access unit immediately before the first access unit. , A candidate generation step of generating a motion vector candidate at the target pixel;
Motion vector determination for determining the motion vector candidate having the highest frequency among the motion vector candidates in each of the target pixel and its surrounding pixels generated by the processing of the candidate generation step as a motion vector in the target pixel. Steps,
A correction step of correcting the motion vector determined by the processing of the motion vector determination step;
A process control step for controlling the information processing apparatus to execute a predetermined process using the motion vector corrected by the process of the correction step,
The correction step includes at least a step of correcting the motion vector using a correction method based on the feature of the predetermined process executed by the information processing apparatus under the control of the process control step. Processing method. A processing execution device that executes a predetermined process using a motion vector in each of the pixels constituting the access unit to be processed, with a predetermined one of a plurality of access units constituting a moving image as a processing target. A program to be executed by a controlling computer,
By setting a predetermined pixel of the pixels constituting the first access unit as a target pixel, and comparing the first access unit with the second access unit immediately before the first access unit. , A candidate generation step of generating a motion vector candidate at the target pixel;
Motion vector determination for determining the motion vector candidate having the highest frequency among the motion vector candidates in each of the target pixel and its surrounding pixels generated by the processing of the candidate generation step as a motion vector in the target pixel. Steps,
A correction step of correcting the motion vector determined by the processing of the motion vector determination step;
A process control step for controlling the process execution device to execute the predetermined process using the motion vector corrected by the process of the correction step,
The correction step includes at least a step of correcting the motion vector using a correction method based on the feature of the predetermined process executed by the process execution device under the control of the process control step. .

Images