JP6012230B2 - Edge direction determination device, control method of edge direction determination device, and image processing device - Google Patents

Description

  The present invention relates to an edge direction determination device, a control method for an edge direction determination device, and an image processing device.

  As a method for converting an interlaced image signal into a progressive image signal by interpolation, there is a method called motion adaptive IP (interlace-progressive) conversion. This detects image motion (motion information) at the interpolation pixel generation position, and adapts whether to generate intra-field interpolation pixels or inter-field interpolation pixels according to the motion information. It is a method of switching automatically.

  In the motion-adaptive IP conversion, an interpolation pixel is generated using a pixel adjacent to the position in the same field in the same field for a position determined to be “with motion”. Therefore, a jagged edge is generated at an oblique edge portion (an oblique edge portion), and the image quality (image quality) is significantly deteriorated.

As a technique for suppressing the occurrence of jaggy, a method has been proposed in which the direction of an edge is determined from pixel information of an input interlaced image signal, and an interpolation pixel is generated using a pixel corresponding to the direction of the edge.
Such a method is disclosed in Patent Document 1, for example.
Specifically, Patent Document 1 discloses determining the edge direction by pattern matching between a predetermined oblique reference pattern and an input image signal. It is also disclosed that the direction of an edge having a shallower angle can be determined by increasing the size (block size) of the reference pattern.

JP 2008-182725 A

  However, in the conventional technique described above, the edge direction cannot be accurately determined, and an erroneous determination of the edge direction may occur. For example, when the block size is large, an image other than the oblique edge portion may greatly affect pattern matching in the oblique edge portion having a deep angle. Therefore, in the configuration in which the edge direction is determined by pattern matching with a large block size, the edge direction having a deep angle cannot be accurately determined. Conversely, in the configuration in which the edge direction is determined by pattern matching with a small block size, the direction of the edge having a shallow angle cannot be accurately determined. If there is an erroneous determination of the edge direction, the image quality deteriorates when image processing based on the determination result of the edge direction is performed. For example, an image having visual disturbance such as the above-described jaggy is generated.

  An object of the present invention is to provide a technique capable of accurately determining the direction of an edge.

The first aspect of the present invention is:
An edge direction determination device that determines the direction of an edge at a point of interest in an image,
For each of a plurality of directions that are edge direction candidates, setting means for setting n (n is an integer of 2 or more) blocks of the same size on a straight line in that direction passing through the point of interest;
Determination means for calculating the image similarity between the n blocks set by the setting means for each of the plurality of directions and determining the direction having the highest similarity as the edge direction at the point of interest; ,
Have
For each of the directions that are candidates for edge directions, the setting means
For at least one of the n blocks set for that direction, the gradient of the change in pixel value with respect to the change in the position in the predetermined direction at the position on the straight line, which is the set position of that block, is calculated. ,
The size of the n blocks to be set is controlled so that the length in the predetermined direction becomes longer when the gradient is small than when the gradient is large .

The second aspect of the present invention is:
An image processing apparatus that converts an interlaced image signal into a progressive image signal by interpolation,
The edge direction determination device;
Interpolation pixel generation means for generating an interpolation pixel by intra-field interpolation based on the edge direction determined by the edge direction determination device;
It is characterized by having.

The third aspect of the present invention is:
A control method of an edge direction determination device that determines the direction of an edge at a point of interest in an image,
A setting step of setting n (n is an integer of 2 or more) blocks of the same size on a straight line passing through the point of interest for each of a plurality of directions that are edge direction candidates;
A determination step of calculating an image similarity between the n blocks set in the setting step for each of the plurality of directions, and determining a direction having the highest similarity as an edge direction at the point of interest; ,
Have
In the setting step, for each direction that is a candidate for an edge direction,
For at least one of the n blocks set for that direction, the gradient of the change in pixel value with respect to the change in position in the predetermined direction at the position on the straight line, which is the set position of that block, is calculated. ,
The size of the n blocks to be set is controlled so that the length in the predetermined direction becomes longer when the gradient is small than when the gradient is large .

  According to the present invention, the edge direction can be determined with high accuracy.

1 is a diagram illustrating an example of a configuration of an image processing apparatus according to a first embodiment. The figure which shows an example of a structure of the interpolation pixel production | generation part in a field which concerns on Example 1. FIG. FIG. 6 is a diagram illustrating an example of a pixel selection method according to angle information according to the first embodiment. The figure which shows an example of a structure of the diagonal detection part which concerns on Example 1. FIG. 7 is a flowchart illustrating an example of a process flow of the oblique detection unit according to the first embodiment. The figure which shows an example of the center position of the block which concerns on Example 1. FIG. The figure which shows an example of the correspondence of the angle which concerns on Example 1, and block size The figure which shows an example of the process of the block matching part which concerns on Example 1. FIG. The figure which shows an example of the process of the block matching part which concerns on Example 1. FIG. The figure which shows an example of the correspondence of the angle which concerns on Example 1, and block size The figure which shows the specific example of SAD value calculation which concerns on Example 1. FIG. The figure which shows an example of a structure of the diagonal detection part which concerns on Example 2. FIG. 10 is a flowchart illustrating an example of a processing flow of an oblique detection unit according to the second embodiment. The figure which shows an example of the center position of the block which concerns on Example 2. FIG. The figure which shows the specific example of SAD value calculation which concerns on Example 2. FIG. The figure which shows an example of a structure of the diagonal detection part which concerns on Example 3. FIG. The figure which shows the specific example of operation | movement of the gradient calculation part which concerns on Example 3. FIG. 10 is a flowchart illustrating an example of the operation of the block size correction unit according to the third embodiment. The figure which shows an example of the mode of the block matching which concerns on Example 4.

<Example 1>
Hereinafter, an edge direction determination apparatus, an edge direction determination apparatus control method, and an image processing apparatus according to Embodiment 1 of the present invention will be described with reference to the drawings. In this embodiment, an example in which an image processing apparatus having an edge direction determination apparatus is an image processing apparatus that performs IP conversion (specifically, motion adaptive IP conversion) will be described. The device (and its control method) can be applied to any device that determines the direction of an edge at a point of interest in an image. IP conversion is a process of converting an interlaced image signal into a progressive image signal by interpolation. Motion-adaptive IP conversion detects the motion (motion information) of an image at the interpolation pixel generation position, and generates an intra-field interpolation pixel or an inter-field interpolation pixel as an interpolation pixel according to the motion information. This is IP conversion that adaptively switches whether to do.

FIG. 1 is a block diagram illustrating the configuration of the image processing apparatus according to the present embodiment.
In FIG. 1, field memories 100 and 101 store interlaced image signals input to the apparatus on a field-by-field basis, and output the interlaced image signals delayed by one field. That is, when the image signal FD0 of the Nth field (N field) is input to the image processing apparatus, the image signal FD1 of the (N-1) th field (N-1 field) is output from the field memory 100. . From the field memory 101, an image signal FD2 of the (N-2) th field (N-2 field) is output. In this embodiment, interpolation pixels are generated for the image signal FD1 of the N-1 field.

  The motion detector 102 detects the motion of the image at the interpolation pixel generation position. In the present embodiment, the motion detection unit 102 detects the motion of the image (motion information MV) at the interpolation pixel generation position using the image signals of the two fields of the N field signal FD0 and the N-2 field signal FD2. To do. Specifically, the motion detection unit 102 determines the presence or absence of motion for each pixel by comparing the difference value of the pixel values between the two fields with a threshold value. If the difference value is greater than or equal to the threshold value, it is determined that there is an image motion (MV = 1 (motion)). If the difference value is less than the threshold value, there is no image motion (MV = 0 (still)). Determined. Then, this determination result (MV value for each pixel) is output to the interpolated pixel generation unit 105 as motion information. Note that the motion detection method is not limited to this. For example, the motion may be detected by block matching.

  The oblique detection unit 103 determines an edge direction at an interpolation pixel generation position (attention point) (edge direction determination device). Specifically, the oblique detection unit 103 determines whether or not the interpolation pixel generation position is an edge portion. When the oblique detection unit 103 determines that the generation position of the interpolation pixel is an edge portion, the oblique detection unit 103 outputs information representing the edge direction to the intra-field interpolation pixel generation unit 104. In the present embodiment, angle information ANGLE indicating the inclination angle of the direction with respect to the horizontal direction is output as information indicating the direction of the edge. In this embodiment, it is assumed that the angle information ANGLE can take a value of −15 to +15. The oblique detection unit 103 corresponds to the edge direction determination device according to the present embodiment. Details of the operation of the oblique detection unit 103 will be described later.

  The intra-field interpolation pixel generation unit 104 generates and outputs an interpolation pixel (intra-field interpolation pixel) by intra-field interpolation based on the angle information ANGLE. Specifically, the intra-field interpolation pixel generation unit 104 calculates and outputs the pixel value ID1 of the intra-field interpolation pixel. The intra-field interpolation pixel is an interpolation pixel suitable for generation at a position where the image moves, and is an interpolation pixel generated using the pixel value of the N-1 field signal FD1 that is a target of generation of the interpolation pixel. is there.

The intra-field interpolation pixel generation unit 104 has a configuration as shown in FIG. 2, for example.
The N−1 field signal FD1 input to the intra-field interpolation pixel generation unit 104 is input to the line memory 200 and the selection unit A201.
The line memory 200 delays the input N-1 field signal FD1 by 1H (one horizontal scanning period), and outputs the delayed signal to the selection unit B202 as a 1H delay signal LD1.
For each interpolation pixel generation position, the selection unit A201 selects a pixel corresponding to the angle information ANGLE from the N-1 field signal FD1 (the image signal of the line immediately below the interpolation pixel generation position), and the pixel The pixel value S1 is output.
For each interpolation pixel generation position, the selection unit B202 selects a pixel corresponding to the angle information ANGLE from the 1H delay signal LD1 (the image signal of the line one line above the interpolation pixel generation position), and the pixel of the pixel The value S2 is output.

A pixel selection method in the selection unit A201 and the selection unit B202 will be described with reference to FIG.
In FIG. 3A, a pixel u (x) is a pixel on a line immediately above the interpolation pixel generation position H, and is a pixel represented by the 1H delay signal LD1 in FIG. The pixel d (x) is a pixel on the line immediately below the interpolation pixel generation position H, and is a pixel represented by the N-1 field signal FD1 in FIG. x is a horizontal shift amount of the positions of the pixels u (x) and d (x) with respect to the generation position of the interpolation pixel (the horizontal direction of the pixels u (x) and d (x) based on the generation position of the interpolation pixel). Is a value indicating the position). When the value of x is a negative value, the horizontal positions of the pixels u (x) and d (x) are shifted by | x | pixels to the left with respect to the interpolation pixel generation position. When the value of x is a positive value, the horizontal positions of the pixels u (x) and d (x) are shifted by | x | pixels to the right from the interpolation pixel generation position.

In the present embodiment, the oblique detection unit 103 outputs the value of x described above as the angle information ANGLE. Further, the oblique detection unit 103 outputs angle information ANGLE = 0 when the interpolation pixel generation position is not an edge portion.
When the angle information ANGLE = + A (A is a natural number), u (+ A) is selected by the selection unit A201, and d (−A) is selected by the selection unit B202. FIG. 3A shows an example in which the angle information ANGLE for the interpolation pixel generation position H is +3. In this case, u (+3) is selected by the selection unit A201, and d (-3) is selected by the selection unit B202. When the angle information ANGLE = 0, as shown in FIG. 3B, u (0) is selected by the selection unit A201, and d (0) is selected by the selection unit B202.

The adder 203 calculates a pixel value S3 by adding the pixel value S1 output from the selection unit A201 and the pixel value S2 output from the selection unit B202, and outputs the pixel value S3 to the multiplier 204.
The multiplier 204 halves the pixel value S3 and outputs the result (a value obtained by halving the pixel value S3) as the pixel value ID1 (intra-field interpolation value) of the intra-field interpolation pixel.
Thereby, in the N-1 field signal FD1, the average value of the pixel values of two pixels adjacent in the direction of the edge can be obtained as the pixel value of the interpolation pixel of the edge portion. Moreover, the average value of the pixel values of two pixels adjacent in the vertical direction can be obtained as the pixel value of the interpolation pixel at a position other than the edge portion.

  The interpolation pixel generation unit 105 switches the interpolation method according to the motion information MV for each generation position of the interpolation pixel, and generates an interpolation pixel (determines the pixel value ID2 of the interpolation pixel). That is, the interpolation pixel generation unit 105 generates an interpolation pixel by motion adaptive IP conversion. Specifically, when there is no image motion at the generation position of the interpolation pixel (when MV = 0), the interpolation pixel generation unit 105 outputs the pixel value of the N-2 field signal FD2 to the generation position. Is used as the pixel value ID2 to generate an interpolation pixel (inter-field interpolation pixel). When there is a motion of the image (when MV = 1), the interpolation pixel generation unit 105 uses the intra-field interpolation value ID1 as the pixel value ID2 for the generation position, and performs the interpolation pixel (intra-field interpolation pixel). ) Is generated. Then, the interpolation pixel generation unit 105 outputs the pixel value ID2 of each interpolation pixel to the double speed conversion unit 106. The inter-field interpolation pixel is an interpolation pixel suitable for generation at a position where there is no motion of the image, and is the same as the pixel value of the N-2 field signal (the pixel value at the same position as the generation position of the interpolation pixel). An interpolation pixel having a pixel value.

The double speed conversion unit 106 synthesizes the pixel value ID2 of each interpolation pixel and the N-1 field signal FD1 and outputs the result as a progressive image signal (an image signal for one frame of a progressive image). Specifically, the double speed conversion unit 106 alternately reads out the N-1 field signal FD1 and the pixel value ID2 of each interpolation pixel for each line at a speed twice the speed at which the interlaced image signal is input. Thereby, the data of the original line (line consisting of pixels constituting the interlaced image of the N-1 field) of the N-1 field signal FD1 and the data of the line (interpolation line) consisting of the interpolation pixel having the pixel value ID2 are obtained. It is alternately read for each line. Then, each time the data is read, the double speed conversion unit 106 outputs the read data as a progressive image signal (data of one line of the progressive image).

  Next, the configuration of the oblique detection unit 103 will be described. FIG. 4 is a block diagram illustrating an internal configuration of the oblique detection unit 103 according to the present embodiment. As shown in FIG. 4, the oblique detection unit 103 according to the present embodiment includes a line memory 300, a matching region calculation unit D301, a matching region calculation unit U302, a control unit 303, a cutout position determination unit 304, a matching angle determination unit 305, A block size determination unit 306, a block matching unit 307, a matching angle calculation unit 308, and the like are provided.

  The line memory 300 delays the input N-1 field signal FD1 by 1H (one horizontal scanning period), and outputs the delayed signal as a 1H delay signal D0.

For each of a plurality of directions (candidate directions) that are edge direction candidates, the matching area calculation unit D301 and the matching area calculation unit U302 are n (n is an integer of 2 or more) on a straight line in the direction passing through the point of interest. ) Block. The n blocks are n areas of the same size. The block is a rectangular region having a long side along a predetermined direction (horizontal direction in the present embodiment).
Specifically, the matching area calculation unit D301 receives the 1H delay signal D0 (the pixel value of the line immediately below the interpolation pixel generation position). The matching area calculation unit D301 sets, as one of the n blocks, an area composed of a plurality of pixels on a line immediately below the interpolation pixel generation position. Then, the matching area calculation unit D301 outputs the pixel value of the 1H delay signal D0 in the set block to the block matching unit 307. That is, the pixel value of the pixel d (x) is output as the pixel value in the block.
The matching area calculation unit U302 receives the N-1 field delay signal FD1 (the pixel value of the line immediately above the interpolation pixel generation position). The matching area calculation unit U302 sets, as one of the n blocks, an area composed of a plurality of pixels on a line that is one level above the interpolation pixel generation position. Then, the matching area calculation unit U302 outputs the pixel value of the N−1 field delay signal FD1 in the set block to the block matching unit 307. That is, the pixel value of the pixel u (x) is output as the pixel value in the block.

The block (the x range of the pixel u (x) and the x range of the pixel d (x)) is determined from P output from the cutout position determining unit 304 and the BS output from the block size determining unit 306. The P is a value representing the candidate direction. In this embodiment, P is the horizontal position x of the pixel u (x) on the straight line in the candidate direction passing through the point of interest (interpolation pixel generation position). The horizontal position x of the pixel d (x) on the straight line in the candidate direction passing through the point of interest is −P. BS is the size (number of pixels) in a predetermined direction (horizontal direction in this embodiment) of the block. The x range (horizontal range) of the pixel u (x) and the x range of the pixel d (x) are calculated by the following calculation formula. In the following expression, int (B / 2) means that the value is an integer. In this embodiment, int (B / 2) is a value obtained by rounding down the decimal point of B / 2.
u range: (P-int (BS / 2)) to (P + int (BS / 2))
Range of d: (-P-int (BS / 2)) to (-P + int (BS / 2))

The block matching unit 307 calculates the similarity of images between the set n (two in this embodiment) blocks for each of a plurality of candidate directions.
The matching angle calculation unit 308 determines the direction with the highest similarity among the plurality of candidate directions as the edge direction at the point of interest (interpolation pixel generation position).

Next, the processing flow of the oblique detection unit 103 according to the present embodiment will be described with reference to the flowchart of FIG.
First, in step S <b> 102, the control unit 303 initializes i and outputs an initial value of i to the cut-out position determination unit 304. i is a value representing the candidate direction. In this embodiment, i is the horizontal position x of the pixel u (x) on the straight line in the candidate direction passing through the point of interest (interpolation pixel generation position), and a value of -15 to 15 is set. In this embodiment, i = −15 is set as an initial value.

  Next, in S103, the cutout position determination unit 304 determines the cutout position P and outputs the cutout position P to the matching region calculation unit D301, the matching region calculation unit U302, the matching angle determination unit 305, and the block matching unit 307. Specifically, the cutout position determination unit 304 outputs i input from the control unit 303 as the cutout position P. As shown in FIG. 6, the matching area calculation unit D301 and the matching area calculation unit U302 determine the center position of the block to be set from the determined P value. Specifically, in the matching area calculation unit U302, the position of the pixel u (P) is determined as the center position. In the matching area calculation unit D301, the position of the pixel d (−P) is determined as the center position. The pixel u (P) is point-symmetric with the pixel d (−P) with the generation position H of the interpolation pixel as the center. FIG. 6 is an example of P = -3.

  In step S104, the matching angle determination unit 305 calculates the matching angle MA based on the cutout position P determined in step S103. The matching angle determination unit 305 outputs the calculated matching angle MA to the block size determination unit 306. As shown in FIG. 6, the matching angle MA is an angle of the candidate direction represented by the cutout position P with respect to a predetermined direction (horizontal direction in the present embodiment). In other words, the matching angle MA is an angle formed between a straight line connecting the pixel u (P) and the pixel d (−P) and the horizontal direction. In this embodiment, a table representing the matching angle MA for each cutout position P is prepared in advance, and the matching angle MA is determined from the cutout position P determined in S103 using the table.

Next, in S105, the block size determination unit 306 calculates the block size BS based on the matching angle MA calculated in S104. Specifically, the length of the predetermined direction (horizontal direction in the present embodiment) is longer in the block set for the candidate direction having the smaller matching angle MA than in the block set for the candidate direction having the larger matching angle MA. Next, the block size BS is calculated. In this embodiment, a table representing the block size BS for each matching angle (or for each range of the matching angle MA) is prepared in advance, and the block size BS is calculated from the matching angle MA calculated in S104 using the table. It is determined. An example of the table is shown in FIG.
The block size determination unit 306 outputs the calculated block size BS to the matching region calculation unit D301, the matching region calculation unit U302, and the block matching unit 307. Thereafter, in the matching area calculation unit D301 and the matching area calculation unit U302, the block having the determined block size BS is set, and the pixel value in the block is output to the block matching unit 307.
Thus, in the present embodiment, the size of n blocks to be set is controlled based on the matching angle MA.

In step S106, the block matching unit 307 uses the cutout position P calculated in step S103, the block size BS calculated in step S105, and the pixel values output from the matching region calculation unit D301 and the matching region calculation unit U302. Perform block matching. Block matching is a process of comparing the pixel value in the block set by the matching area calculation unit D301 with the pixel value in the block set by the matching area calculation unit U302. A SAD value (sum of absolute differences) is calculated by block matching.

  FIG. 8 shows an example of the set block. FIG. 8 shows a block set when the cutout position P is −3 and the block size BS is 5. In the example of FIG. 8, the areas of pixels u (−1) to u (−5) on the actual line U and the areas of pixels d (+1) to d (+5) on the actual line D are set as blocks, respectively. ing. The actual line U is a line immediately above the interpolation pixel generation position H, and the actual line D is a line immediately below the interpolation pixel generation position. Further, the interpolation line in FIG. 8 is a line in which an interpolation pixel is generated. The pixels d (+1) to d (+5) are point-symmetric with respect to the pixels u (−1) to u (−5) around the interpolation pixel generation position H.

When the block shown in FIG. 8 is set, the pixels u (−1) to u (−5) of the actual line U and the values of the pixels d (+1) to d (+5) of the actual line D are used. A SAD value is calculated. The SAD value is calculated using the following equation.

For example, in the example of FIG. 8, the SAD value is calculated by the following calculation. In the following expression, u (x) and d (x) represent pixel values of the pixel.
SAD (−3) = | u (−5) −d (+1) |
+ | U (-4) -d (+2) |
+ | U (-3) -d (+3) |
+ | U (-2) -d (+4) |
+ | U (-1) -d (+5) |

  Following S106, in S107, the block matching unit 307 normalizes the SAD value. As described above, in this embodiment, the block size is not constant (changes depending on the candidate direction). A plurality of SADs obtained with a plurality of block sizes cannot be handled on the same basis. Therefore, in this step, the SAD value calculated in S106 is normalized so that the value corresponds to a certain block size. In this embodiment, the SAD value calculated in S106 is normalized so that the value corresponds to the smallest block size. For example, when the minimum value of the block size is 3 and the SAD value is calculated with the block size being 15, the normalized SAD value is obtained by multiplying the calculated SAD value by 3/15. It is done. Note that the SAD value may be normalized so that the SAD value becomes a value corresponding to the smallest block size, or the SAD so that the SAD value becomes a value corresponding to the largest block size. The value may be normalized.

  Next, in S108, the control unit 303 determines whether i = + 15. This determination makes it possible to determine whether or not SAD values have been calculated for all of the plurality of candidate directions. That is, in the case of i = + 15, the control unit 303 determines that the SAD values have been calculated for all of the plurality of candidate directions, and this flow ends. When i is not +15, the control unit 303 determines that calculation of SAD values for all of the plurality of candidate directions is not completed, and increments i by 1 in S109. Then, the process returns to S103, and the processes of S103 to S107 are repeated until i = + 15.

The block matching unit 307 outputs a plurality of SAD values obtained by the above processing flow to the matching angle calculation unit 308. The plurality of SAD values are 31 SAD values (normalized values) corresponding to P = −15 to +15. The normalized SAD value corresponds to the image similarity between blocks (represents dissimilarity). Then, the matching angle calculation unit 308 determines the edge direction from the 31 SAD values (details will be described later).
The processing described above is performed for each interpolation pixel generation position.
Although FIG. 5 shows an example in which the value of i is counted and the SAD value is calculated for each candidate direction, a plurality of SAD values corresponding to a plurality of candidate directions may be calculated in parallel.

The matching angle calculation unit 308 determines the edge direction at the generation position of the interpolation pixel from the 31 SAD values (normalized values) calculated by the block matching unit 307, and uses the determination result as angle information ANGLE. Output. In the present embodiment, among the plurality (31) of candidate directions, the direction with the highest image similarity between blocks is determined as the edge direction at the point of interest (interpolation pixel generation position). The SAD value decreases as the image similarity between blocks increases. Therefore, in the present embodiment, the matching angle calculation unit 308 determines that the value of P corresponding to the minimum value of the 31 SAD values is a value that represents the direction of the edge at the interpolation pixel generation position. . The minimum SAD value (SADmin) is calculated by the following equation. In the following formula, P is −15 to 15, and MIN is a function for calculating the minimum value.
SADmin = MIN (SAD (P))
The matching angle calculation unit 308 outputs a value of P satisfying SADmin = SAD (P) as angle information ANGLE. When SADmin is equal to or greater than a predetermined threshold value, it may be determined that the interpolation pixel generation position is not an edge portion. If it is determined that the interpolation pixel generation position is not an edge portion, 0 is output as the angle information ANGLE.

  For example, when a signal representing an image having a pixel configuration as shown in FIG. 9A (an image of an oblique edge portion) is input, 31 SAD values calculated for the interpolation pixel generation position H The distribution of is as shown in FIG. In the example of FIG. 9B, the SAD value corresponding to the position of u (+3) on the actual line U (the position of P = + 3) is the minimum value SADmin. Therefore, angle information ANGLE = + 3 is output from the matching angle calculation unit 308.

Next, a specific example of SAD value calculation when the matching angle MA is 30 °, 10 °, and 5 ° will be described.
An example of the relationship between the matching angle MA and the block size BS is shown in FIG. When the relationship between the matching angle MA and the block size BS is as shown in FIG. 10, BS = 5 when MA = 30 °, BS = 9 when MA = 10 °, and when MA = 5 °. BS = 15.

An example in the case of MA = 30 ° will be described in detail with reference to FIG.
In the example of FIG. 11A, for each of the real line U and the real line D, the pixel on the straight line that passes through the point of interest (interpolation pixel generation position) located on the interpolation line and satisfies MA = 30 ° is centered. As a result, an area for five pixels in the horizontal direction is set as a block. A block set for the actual line U is referred to as a block 1A, and a block set for the actual line D is referred to as a block 2A.
Specifically, P = −3 is set in the cutout position determination unit 304, MA = 30 ° in the matching angle determination unit 305, and BS = 5 in the block size determination unit 306 from the relationship of FIG. Therefore, for the actual line U, an area corresponding to 5 pixels in the horizontal direction centering on a pixel (pixel on the actual line U) that is 3 pixels to the left of the attention point is set as a block 1A. For the real line D, an area corresponding to 5 pixels in the horizontal direction centered on a pixel (pixel on the real line D) that is 3 pixels to the right of the point of interest is set as a block 2A. The block 1A is point-symmetric with the block 2A with the point of interest as the center.
Then, in the block matching unit 307, the pixel value of the block 1A and the pixel value of the block 2A are compared, and the SAD value is calculated.

An example in the case of MA = 10 ° will be described in detail with reference to FIG.
In the example of FIG. 11B, for each of the real line U and the real line D, the pixel on the straight line that passes through the point of interest (interpolation pixel generation position) located on the interpolation line and satisfies MA = 10 ° is centered. As a result, an area for nine pixels in the horizontal direction is set as a block. A block set for the actual line U is referred to as a block 1B, and a block set for the actual line D is referred to as a block 2B.
Specifically, P = −6 is set in the cutout position determination unit 304, MA = 10 ° in the matching angle determination unit 305, and BS = 9 in the block size determination unit 306 from the relationship of FIG. Therefore, for the actual line U, an area corresponding to 9 pixels in the horizontal direction centered on a pixel (pixel on the actual line U) that is 6 pixels to the left of the point of interest is set as the block 1B. For the real line D, an area corresponding to 9 pixels in the horizontal direction centered on a pixel (pixel on the real line D) that is 6 pixels to the right of the point of interest is set as a block 2B. The block 1B is point-symmetric with the block 2B around the point of interest.
Then, in the block matching unit 307, the pixel value of the block 1B and the pixel value of the block 2B are compared to calculate the SAD value.
Here, assuming that the SAD value is normalized to a value corresponding to BS = 5, the calculated SAD value (SAD value before normalization; 9-pixel SAD) is normalized by the following equation. In the following formula, 5 pixel SAD is a normalized SAD value.
5 pixels SAD = 9 pixels SAD × (5/9)

An example in the case of MA = 5 ° will be described in detail with reference to FIG.
In the example of FIG. 11C, for each of the real line U and the real line D, the pixel on the straight line that passes through the point of interest (interpolation pixel generation position) located on the interpolation line and satisfies MA = 5 ° is the center. As a result, a region for 15 pixels in the horizontal direction is set as a block. A block set for the actual line U is referred to as a block 1C, and a block set for the actual line D is referred to as a block 2C.
Specifically, P = −12 is set in the cut-out position determination unit 304, MA = 5 ° is set in the matching angle determination unit 305, and BS = 15 is set in the block size determination unit 306 from the relationship of FIG. Therefore, for the real line U, an area for 15 pixels in the horizontal direction centering on a pixel 12 pixels away from the target point to the left (pixels on the real line U) is set as a block 1C. For the real line D, an area for 15 pixels in the horizontal direction centered on a pixel 12 pixels away from the target point on the right side (a pixel on the real line D) is set as a block 2C. The block 1C is point-symmetric with the block 2C around the point of interest.
Then, in the block matching unit 307, the pixel value of the block 1C and the pixel value of the block 2C are compared, and the SAD value is calculated.
Here, assuming that the SAD value is normalized to a value corresponding to BS = 5, the calculated SAD value (SAD value before normalization; 15 pixel SAD) is normalized by the following expression. In the following formula, 5 pixel SAD is a normalized SAD value.
5 pixels SAD = 15 pixels SAD × (5/15)

As described above, according to the present embodiment, the block size is changed depending on the angle of the candidate direction, which is a candidate for the edge direction, with respect to the predetermined direction (the horizontal direction in the present embodiment). Specifically, the block size is set so that the length in the predetermined direction is longer in the block set in the candidate direction having a small angle with respect to the predetermined direction than in the block set in the candidate direction having a large angle with respect to the predetermined direction. Be controlled. That is, in this embodiment, when calculating the degree of possibility that the edge direction is a shallow direction of the angle, the size of the block is increased, and the degree of possibility that the edge direction is a direction having a deep angle is calculated. In doing so, the block size is reduced. Thereby, the direction of the edge can be accurately determined. Note that the degree of possibility is the similarity of images between blocks.
Specifically, when the length of the block in a predetermined direction is long, the degree of possibility that the edge direction is a direction with a shallow angle can be calculated with high accuracy, but the edge direction is a direction with a deep angle. The degree of possibility cannot be calculated with high accuracy. On the other hand, when the length of the predetermined direction of the block is short, the degree of possibility that the edge direction is the deep direction of the angle can be accurately calculated, but the edge direction may be the shallow direction of the angle. The degree cannot be calculated with high accuracy. In this embodiment, by controlling the block size as described above, the degree of possibility that the edge direction is a direction with a shallow angle and the degree of possibility that the edge direction is a direction with a deep angle. Both can be calculated with high accuracy. Thereby, the direction of the edge can be determined with high accuracy, and hence an accurate interpolation pixel can be generated, and the occurrence of jaggy can be suppressed.

In this embodiment, an example in which the attention point is the interpolation pixel generation position is shown, but the attention point is not limited thereto. For example, the attention point may be a pixel position on an actual line.
In the present embodiment, an example in which an SAD value (normalized SAD value) is used as a value corresponding to the similarity of images has been described, but the value corresponding to the similarity is not limited to the SAD value. For example, the image pattern may be analyzed for each block, and the pattern difference between the blocks may be calculated as a value (dissimilarity) corresponding to the similarity. Further, the reciprocal of the SAD value may be calculated as the similarity.
In this embodiment, the blocks are set on the lines adjacent above and below the point of interest, but the blocks may be set on a line away from the point of interest. In this embodiment, the vertical size of the block is one pixel, but the vertical size may be a size corresponding to a plurality of pixels. The block may be an area including a point of interest.

<Example 2>
Hereinafter, an edge direction determination apparatus, an edge direction determination apparatus control method, and an image processing apparatus according to Embodiment 2 of the present invention will be described with reference to the drawings. In the first embodiment, an example in which two blocks are set for each candidate direction has been described. In this embodiment, an example in which three blocks are set for each candidate direction will be described. Note that descriptions of functions and configurations similar to those of the first embodiment are omitted as much as possible.
Since the general configuration of the image processing apparatus according to the present embodiment is the same as that of the first embodiment (FIG. 1), description thereof is omitted.

  In the present embodiment, the internal configuration of the oblique detection unit is different from that of the first embodiment. FIG. 12 is a block diagram illustrating an internal configuration of the oblique detection unit according to the present embodiment.

The line memory 400 delays the input N-1 field signal FD1 by 1H (one horizontal scanning period), and outputs the delayed signal as a 1H delay signal D0.
The line memory 401 delays the input 1H delay signal D0 by 1H, and outputs the delayed signal as a 2H delay signal D1. The 2H delay signal D1 is a signal obtained by delaying the N-1 field signal FD1 by 2H.

The matching area calculation unit 2D402, the matching area calculation unit D403, and the matching area calculation unit U404 set, for each of a plurality of candidate directions, three blocks on a straight line in that direction passing through the point of interest.
Specifically, a 2H delay signal D1 (a pixel value of a line three lines below the interpolation pixel generation position) is input to the matching region calculation unit 2D402. The matching area calculation unit 2D402 sets, as one of the above three blocks, an area composed of a plurality of pixels on a line three lower than the interpolation pixel generation position. Then, the matching area calculation unit 2D402 outputs the pixel value of the 2H delay signal D1 in the set block to the block matching unit 409. In this embodiment, the pixel on the line three lines below the interpolation pixel generation position is denoted as d2.
The matching area calculation unit D403 receives the 1H delay signal D0 (the pixel value of the line immediately below the interpolation pixel generation position). The matching area calculation unit D403 sets, as one of the three blocks, an area composed of a plurality of pixels on the line immediately below the interpolation pixel generation position. Then, the matching area calculation unit D403 outputs the pixel value of the 1H delay signal D0 in the set block to the block matching unit 409. That is, the pixel value of the pixel d (x) is output as the pixel value in the block.
The matching area calculation unit U404 receives the N-1 field delay signal FD1 (the pixel value of the line immediately above the interpolation pixel generation position). The matching area calculation unit U404 sets, as one of the three blocks, an area composed of a plurality of pixels on a line that is one level above the interpolation pixel generation position. Then, the matching area calculation unit U404 outputs the pixel value of the N−1 field delay signal FD1 in the set block to the block matching unit 409. That is, the pixel value of the pixel u (x) is output as the pixel value in the block.

The blocks (the x range of the pixel u (x), the x range of the pixel d (x), and the x range of the pixel d2 (x)) are P and the block size output from the cutout position determining unit 406. It is determined from the BS output from the determination unit 408. The x range (horizontal range) of the pixel u (x), the x range of the pixel d (x), and the x range of the pixel d2 (x) are calculated by the following calculation formulas.
u range: (P-int (BS / 2)) to (P + int (BS / 2))
Range of d: (-P-int (BS / 2)) to (-P + int (BS / 2))
d2 range: (-3P-int (BS / 2)) to (-3P + int (BS / 2))

Next, the processing flow of the oblique detection unit according to the present embodiment will be described with reference to the flowchart of FIG.
First, in step S <b> 202, the control unit 405 initializes i and outputs an initial value of i to the cut-out position determination unit 406. In the present embodiment, as in the first embodiment, a value of -15 to 15 is set as the value of i. In this embodiment, i = −15 is set as an initial value.

  Next, in S <b> 203, the cutout position determination unit 406 determines the cutout position P as in the first embodiment. The determined cutout position P is output to the matching region calculation unit 2D402, the matching region calculation unit D403, the matching region calculation unit U404, the matching angle determination unit 407, and the block matching unit 409. As shown in FIG. 14, the matching area calculation unit 2D402, the matching area calculation unit D403, and the matching area calculation unit U404 determine the center position of the block to be set from the determined P value. Specifically, in the matching area calculation unit U404, the position of the pixel u (P) is determined as the center position. In the matching area calculation unit D403, the position of the pixel d (−P) is determined as the center position. In the matching area calculation unit 2D402, the position of the pixel d (-3P) is determined as the center position. The pixel u (P) is point-symmetric with the pixel d (−P) with the generation position H of the interpolation pixel as the center. The pixel d (−3P) is a pixel on a straight line passing through the pixel u (P) and the pixel d (−P). FIG. 14 is an example of P = -1.

  In step S204, the matching angle determination unit 407 calculates the matching angle MA based on the cutout position P determined in step S203. The calculation method is the same as in the first embodiment.

  Next, in S205, the block size determination unit 408 calculates the block size BS based on the matching angle MA calculated in S204. The calculation method is the same as in the first embodiment. The calculated block size BS is output to the matching area calculation unit 2D402, the matching area calculation unit D403, the matching area calculation unit U404, and the block matching unit 409. Thereafter, in the matching region calculation unit 2D402, the matching region calculation unit D403, and the matching region calculation unit U404, the block having the determined block size BS is set, and the pixel value in the block is output to the block matching unit 409. .

  In step S206, the block matching unit 409 uses the cutout position P calculated in step S203, the block size BS calculated in step S205, and the pixel values output from the matching region calculation unit D403 and the matching region calculation unit U404. Perform block matching. Specifically, the SAD value is calculated in the same manner as in the first embodiment from the pixel value in the block set by the matching area calculation unit D403 and the pixel value in the block set by the matching area calculation unit U404. Is done. The SAD value calculated in this step is referred to as upSAD.

  Next, in S207, the block matching unit 409 uses the cutout position P calculated in S203, the block size BS calculated in S205, and the pixel values output from the matching region calculation unit 2D402 and the matching region calculation unit D403. Perform block matching. Specifically, the SAD value is calculated in the same manner as in the first embodiment from the pixel value in the block set by the matching area calculation unit 2D402 and the pixel value in the block set by the matching area calculation unit D403. Is done. The SAD value calculated in this step is referred to as downSAD.

  In step S208, the block matching unit 409 normalizes upSAD. In step S209, the block matching unit 409 normalizes downSAD. The reason for normalization and the normalization method are the same as in the first embodiment.

  Next, in S210, the control unit 405 determines whether i = + 15. In the case of i = + 15, the control unit 405 determines that the SAD values (upSAD, downSAD) have been calculated for all of the plurality of candidate directions, and this flow ends. When i is not +15, the control unit 405 determines that calculation of SAD values for all of the plurality of candidate directions has not been completed, and increments i by 1 in S211. Then, the process returns to S203, and the processes of S203 to S209 are repeated until i = + 15.

  The block matching unit 409 outputs a plurality of SAD values obtained by the above processing flow to the matching angle calculation unit 410. The plurality of SAD values include 31 upSADs corresponding to P = -15 to +15 (normalized values) and 31 downSADs corresponding to P = -15 to +15 (normalized values). It is. Then, the matching angle calculation unit 410 determines the edge direction from the 62 SAD values (details will be described later).

  The matching angle calculation unit 410 uses the 31 upSAD values (normalized values) calculated by the block matching unit 409 and the 31 downSADs (normalized values) at the interpolation pixel generation position. Determine the direction of the edge. Then, the matching angle calculation unit 410 outputs the determination result as angle information ANGLE. In this embodiment, it is determined that the value of P corresponding to the minimum value of upSAD or the value of P corresponding to the minimum value of downSAD is a value representing the direction of the edge at the generation position of the interpolation pixel. Then, the value of P determined to be a value representing the direction of the edge at the interpolation pixel generation position is output as angle information ANGLE. Further, when the difference between the value of P corresponding to the minimum value of upSAD and the value of P corresponding to the minimum value of downSAD is equal to or greater than a predetermined threshold, it is determined that the generation position of the interpolation pixel is not an edge portion. Then, 0 is output as the angle information ANGLE.

Hereinafter, the processing of the matching angle calculation unit 410 will be described using equations.
First, the minimum value of upSAD (upSADmin) and the minimum value of downSAD (downSADmin) are calculated by the following equations. In the following formula, P is −15 to 15, and MIN is a function for calculating the minimum value.
upSADmin = MIN (upSAD (P))
downSADmin = MIN (downSAD (P))
When the following conditional expression is satisfied, a value of P satisfying upSADmin = upSAD (P) or a value of P satisfying downSADmin = downSAD (P) is output as the angle information ANGLE. When the following conditional expression is not satisfied, 0 is output as the angle information ANGLE. In the following expression, upP is a value of P that satisfies upSADmin = upSAD (P), and downP is a value of P that satisfies downSADmin = downSAD (P).
| UpP-downP | <Threshold

  Next, a specific example of SAD value calculation when the matching angle MA is 30 °, 10 °, and 5 ° will be described. Assume that the relationship between the matching angle MA and the block size BS is as shown in FIG.

An example in the case of MA = 30 ° will be described in detail with reference to FIG.
In the example of FIG. 15A, each of the real line U, the real line D, and the real line D2 passes through a point of interest (interpolation pixel generation position) located on the interpolation line, and is on a straight line that satisfies MA = 30 °. A region corresponding to 5 pixels in the horizontal direction is set as a block centering on this pixel. A block set for the actual line U is referred to as a block 1D, a block set for the actual line D is referred to as a block 2D, and a block set for the actual line D2 is referred to as a block 3D.
Specifically, P = −3 is set in the cutout position determination unit 406, MA = 30 ° in the matching angle determination unit 407, and BS = 5 in the block size determination unit 408. Therefore, for the real line U, an area corresponding to 5 pixels in the horizontal direction centering on a pixel (pixel on the real line U) that is 3 pixels to the left of the attention point is set as a block 1D. For the real line D, an area corresponding to 5 pixels in the horizontal direction centering on a pixel (pixel on the real line D) that is 3 pixels to the right of the point of interest is set as a block 2D. For the real line D2, a region for five pixels in the horizontal direction centered on a pixel (pixel on the real line D2) that is 9 pixels to the right of the point of interest is set as a block 3D. The block 1D is in a point-symmetric position with respect to the block 2D with the attention point as the center. The center of the block 3D is located on a straight line passing through the center of the block 1D and the center of the block 2D (a straight line passing through the point of interest).
Then, in the block matching unit 409, the pixel value of the block 1D is compared with the pixel value of the block 2D, and upSAD is calculated. Further, the pixel value of the block 2D and the pixel value of the block 3D are compared to calculate the downSAD.

An example in the case of MA = 10 ° will be described in detail with reference to FIG.
In the example of FIG. 15B, each of the real line U, the real line D, and the real line D2 passes through a point of interest (interpolation pixel generation position) located on the interpolation line and is on a straight line that satisfies MA = 10 °. A region corresponding to 5 pixels in the horizontal direction is set as a block centering on this pixel. A block set for the actual line U is referred to as a block 1E, a block set for the actual line D is referred to as a block 2E, and a block set for the actual line D2 is referred to as a block 3E.
Specifically, P = −6 is set in the cutout position determination unit 406, MA = 10 ° in the matching angle determination unit 407, and BS = 9 in the block size determination unit 408. Therefore, for the actual line U, an area corresponding to 9 pixels in the horizontal direction centered on a pixel (pixel on the actual line U) that is 6 pixels to the left from the point of interest is set as a block 1E. For the real line D, an area for nine pixels in the horizontal direction centering on a pixel (pixels on the real line D) that is 6 pixels to the right of the point of interest is set as a block 2E. For the real line D2, an area for nine pixels in the horizontal direction centered on a pixel (pixel on the real line D2) that is 18 pixels to the right of the point of interest is set as a block 3E. The block 1E is point-symmetric with the block 2E around the point of interest. The center of the block 3E is located on a straight line passing through the center of the block 1E and the center of the block 2E (a straight line passing through the point of interest).
Then, in the block matching unit 409, the pixel value of the block 1E and the pixel value of the block 2E are compared to calculate upSAD. Further, the pixel value of the block 2E and the pixel value of the block 3E are compared, and downSAD is calculated. Similar to the first embodiment, the calculated upSAD and downSAD are normalized to a value corresponding to BS = 5.

An example in the case of MA = 5 ° will be described in detail with reference to FIG.
In the example of FIG. 15C, each of the real line U, the real line D, and the real line D2 passes through a point of interest (interpolation pixel generation position) located on the interpolation line and is on a straight line that satisfies MA = 5 °. A region corresponding to 5 pixels in the horizontal direction is set as a block centering on this pixel. A block set for the actual line U is referred to as a block 1F, a block set for the actual line D is referred to as a block 2F, and a block set for the actual line D2 is referred to as a block 3F.
Specifically, P = −12 is set in the cutout position determination unit 406, MA = 5 ° is set in the matching angle determination unit 407, and BS = 15 is set in the block size determination unit 408. Therefore, for the actual line U, an area for 15 pixels in the horizontal direction centered on a pixel (pixels on the actual line U) 12 pixels to the left of the point of interest is set as a block 1F. For the real line D, an area for 15 pixels in the horizontal direction centering on a pixel 12 pixels away from the target point on the right side (a pixel on the real line D) is set as a block 2F. For the real line D2, a region for 15 pixels in the horizontal direction centered on a pixel (pixel on the real line D2) that is 36 pixels to the right of the point of interest is set as a block 3F. The block 1F is point-symmetric with the block 2F around the point of interest. The center of the block 3F is located on a straight line passing through the center of the block 1F and the center of the block 2F (a straight line passing through the point of interest).
Then, in the block matching unit 409, the pixel value of the block 1F and the pixel value of the block 2F are compared to calculate upSAD. Further, the pixel value of the block 2F and the pixel value of the block 3F are compared to calculate the downSAD. Similar to the first embodiment, the calculated upSAD and downSAD are normalized to a value corresponding to BS = 5.

As described above, in the present embodiment, more blocks are set for one candidate direction than in the first embodiment, and the edge direction is determined by comparing the set blocks. Thereby, the edge direction can be determined with higher accuracy than in the first embodiment.
In this embodiment, an example is shown in which one block is set above the attention point and two blocks are set below the attention point. However, the position of the block to be set is not limited to this. For example, two blocks may be set above the attention point, and one block may be set below the attention point. Three blocks may be set on either the upper side or the lower side of the attention point. Specifically, a block may be set for each of three actual lines adjacent to the upper side of the interpolation pixel generation position.
Also, more than three blocks may be set. For example, two blocks may be set above the attention point, and two blocks may be set below the attention point.

<Example 3>
Hereinafter, an edge direction determination apparatus, an edge direction determination apparatus control method, and an image processing apparatus according to Embodiment 3 of the present invention will be described with reference to the drawings. In the present embodiment, the corresponding candidate direction is set according to the gradient of the change in the pixel value with respect to the change in the position in the predetermined direction at the position on the straight line (on the straight line in the candidate direction) that is the block setting position. An example of controlling (correcting) the size of n blocks will be described. In this embodiment, an example in which the predetermined direction is the horizontal direction will be described. Note that descriptions of functions and configurations similar to those of the first and second embodiments are omitted as much as possible.
Since the general configuration of the image processing apparatus according to the present embodiment is the same as that of the first embodiment (FIG. 1), description thereof is omitted.

In the present embodiment, the internal configuration of the oblique detection unit is different from those in the first and second embodiments. Specifically, the oblique detection unit according to the present embodiment further includes a gradient calculation unit 501 and a block size correction unit 502 in addition to the functional units shown in the first and second embodiments (FIGS. 4 and 12). FIG. 16 is a block diagram illustrating an internal configuration of the oblique detection unit according to the present embodiment. FIG. 16 shows an example in which the configuration that is the feature of the present embodiment is added to the first embodiment, but the configuration that is the feature of the present embodiment may be added to the second embodiment.
Hereinafter, the oblique detection unit according to the present embodiment will be described with reference to FIG. In addition, the same code | symbol is attached | subjected to the function part similar to Example 1, and the description is abbreviate | omitted.

For each candidate direction, the gradient calculation unit 501 calculates the gradient for at least one block among n blocks set for the candidate direction.
In the present embodiment, the pixel value output from each matching region calculation unit (matching region calculation unit D301 and matching region calculation unit U302) is input to the gradient calculation unit 501. In this embodiment, the matching area calculation unit outputs pixel values in an area larger than the block determined by the matching area calculation unit. For example, the matching area calculation unit D301 outputs all the pixel values of the line immediately below the point of interest (interpolation pixel generation position). The matching area calculation unit U302 outputs all the pixel values of the line immediately above the target point.
Then, the gradient calculation unit 501 calculates the gradient for each of the n (two) blocks to be set, and corrects the block size correction for a flag (gradient flag F) indicating the representative value of the two gradients calculated for the two blocks. Output to the unit 502.

  A specific example of the operation of the gradient calculation unit 501 will be described with reference to FIG. FIG. 17 is an example in the case of P = 3. In the case of P = 3, a block centered on the pixel u (P) (region consisting of pixels on the actual line U) and a block centered on the pixel d (-P) (region consisting of pixels on the actual line D) ) And are set.

The gradient calculation unit 501 calculates the gradient using the following formula for each of the two blocks. That is, in the present embodiment, the first difference and the second difference are calculated. Then, the larger difference between the first difference and the second difference is calculated as the gradient. The first difference is a difference between the pixel value at the set position of the block and the pixel value at a position away from the set position by a predetermined distance in the left direction. The second difference is a difference between the pixel value at the set position of the block and the pixel value at a position away from the set position in the right direction (the opposite direction to the left) by a predetermined distance. In the following expression, u (x) and d (x) represent pixel values (for example, luminance and color difference) of the pixel. MAX is a function for calculating the maximum value.
[Blocks set on the actual line U]
First difference hedge_u1 = | u (P) −u (P−2) |
Second difference hedge_u2 = | u (P) −u (P + 2) |
Gradient hedge_u = MAX (hedge_u1, hedge_u2)
[Block set on actual line D]
First difference hedge_d1 = | d (−P) −d (−P−2) |
Second difference hedge_d2 = | d (−P) −d (−P + 2) |
Gradient hedge_d = MAX (hedge_d1, hedge_d2)
In addition, although the said example is an example in case the said predetermined distance is a distance for 2 pixels, the said predetermined distance is not restricted to the distance for 2 pixels. For example, the predetermined distance may be a distance of 1, 3, 5 pixels.
The gradient calculation method is not limited to the above method. For example, either the first difference or the second difference may be calculated as the gradient. The gradient may be calculated using a least square method or the like from pixel values of three or more pixels including pixels at the set position of the block whose vertical positions are equal to each other.

Then, the gradient calculation unit 501 determines that there is a high possibility that the edge portion has a large angle with respect to the horizontal direction when the following conditional expression is satisfied, and outputs a gradient flag F = 0. Further, when the following conditional expression is not satisfied, the gradient calculating unit 501 determines that there is a high possibility that the edge portion has a small angle with respect to the horizontal direction, and outputs a gradient flag F = 1. In other words, in the present embodiment, of the two gradients calculated for the two blocks, the one having a larger value is used as the representative value. The gradient flag F = 0 when the representative value is equal to or greater than the predetermined value (greater than or equal to the threshold value), and the gradient flag F = 1 when the representative value is less than the predetermined value (less than the threshold value).
(Hedge_u ≧ threshold) or (hedge_d ≧ threshold)

The block size correcting unit 502 controls the size of n blocks set for the corresponding candidate direction so that the length in the horizontal direction becomes longer when the gradient is small than when the gradient is large. (to correct). Then, the block size correction unit 502 outputs the corrected block size BS2 (the number of pixels in the horizontal direction of the block) to the block matching unit 307.
In the present embodiment, the block size correction unit 502 controls the sizes of the two blocks to be set based on the representative values of the two gradients calculated for the two blocks. Specifically, the block size correction unit 502 controls the sizes of the two blocks set for the candidate direction represented by the position P according to the gradient flag F (P).

A specific example of the operation of the block size correction unit 502 will be described with reference to the flowchart of FIG.
First, in S302, the block size correction unit 502 calculates correction values α and β. The correction value α is a correction value for converting the block size BS to the second size. The correction value β is a correction value for converting the block size BS to the first size. The second size is larger than the first size. The correction values α and β are determined according to the value of the block size BS. For example, it is assumed that the block size is one of three values of X, Y, and Z (X <Y <Z). It is assumed that the first size is X and the second size is Z. In this case, α and β are calculated by α = Z−BS and β = BS−X. For example, when BS = Y, α = Z−Y and β = Y−X. Further, when BS = Z, α = Z−Z = 0 and β = Z−X. When BS = X, α = Z−X = 0 and β = XX = 0.

  In S303, the block size correction unit 502 determines whether or not the gradient flag F (P) = 1. If F (P) = 1, the block size correcting unit 502 sets the second size as the block size BS2 in S304. In S304, BS2 is calculated by adding α to BS. Therefore, when BS = X, Y, the block size is increased to Z. When BS = Z, the block size is left as it is. If F (P) = 0, the block size correction unit 502 sets the first size as the block size BS2 in S305. In S305, BS2 is calculated by subtracting β from BS. That is, when BS = Y, Z, the block size is reduced to X. When BS = X, the block size is left as it is.

  Based on BS2, the block matching unit 307 selects a pixel value in the block from a plurality of input pixel values, and performs block matching. Specifically, a pixel value in a block having the pixel d (P) as the center and a block size of BS2 is selected from the pixel values output from the matching region calculation unit D301. Further, from the pixel values output from the matching area calculation unit U302, pixel values in a block having the pixel u (P) as a center and a block size of BS2 are selected. Since the block matching method is the same as that of the first embodiment, description thereof is omitted.

As described above, according to the present embodiment, according to the gradient of the change in the pixel value with respect to the change in the position in the predetermined direction at the position on the straight line (on the straight line in the candidate direction) as the block setting position,
The size of n blocks set for the corresponding candidate direction is controlled. Specifically, the size of n blocks set for the corresponding candidate direction is controlled so that the length in the horizontal direction becomes longer when the gradient is small than when the gradient is large. When the gradient is large, there is a high possibility that the edge portion has a large angle with respect to the horizontal direction, and when the gradient is small, there is a high possibility that the edge portion has a small angle with respect to the horizontal direction. Therefore, with the above configuration, both the degree of possibility that the edge direction is a direction with a shallow angle and the degree of possibility that the edge direction is a direction with a deep angle can be calculated with higher accuracy. Can be determined with higher accuracy.

In the present embodiment, the gradient is calculated for each of a plurality of blocks for each candidate direction, and the block size is controlled based on the calculated representative values of the plurality of gradients. However, the present invention is not limited to this configuration. For example, the gradient of one block may be calculated for each candidate direction, and the block size may be controlled based on the calculated one gradient. In the present embodiment, the gradient is calculated for all the blocks set for one candidate direction (each of n blocks), but the present invention is not limited to this configuration. The gradient may be calculated for some of the n blocks. For example, when three blocks are set for one candidate direction as in the second embodiment, the gradient may be calculated for one or two of the three blocks.
In this embodiment, the example in which the representative values of the plurality of gradients are the maximum values of the plurality of gradients has been shown, but the representative value is not limited to this. For example, the representative value may be a minimum value, a mode value, an average value, or the like of a plurality of gradients.
In the present embodiment, an example in which the block size is set to the first size or the second size based on the gradient is shown, but the present invention is not limited to this configuration. For example, the block size may be changed continuously or stepwise such that the smaller the gradient, the longer the horizontal length.
In the present embodiment, an example in which the block size determined based on the calculated angle is corrected has been described, but the configuration is not limited thereto. For example, the block size may be determined based only on the gradient without calculating the angle and determining the block size based on the calculated angle.

<Example 4>
Hereinafter, an edge direction determination device and an edge direction determination device control method according to Embodiment 4 of the present invention will be described with reference to the drawings. In the first to third embodiments, an example in which the predetermined direction (the direction for defining the angle used when determining the block size BS; the direction of the long side of the block) is the horizontal direction has been described. In this embodiment, an example in which the predetermined direction is the vertical direction will be described. In the first to third embodiments, an example in which the input image signal is an interlaced image signal has been described. In this embodiment, an example in which the input image signal is a progressive image signal will be described.
Since the configuration of the oblique detection unit according to the present embodiment is the same as that of the other embodiments, the description thereof is omitted.

FIGS. 19A and 19B show examples of blocks set by the matching area calculation unit. In this embodiment, for each of a plurality of candidate directions, a block including a target point (a target block) and a block not including a target point (a reference block) are set, and block matching between the target block and the reference block is performed. Shall be done. Then, the edge direction at the point of interest is determined from the block matching result (SAD value) for each candidate direction.
In FIGS. 19A and 19B, the angle MA is a matching angle. In the present embodiment, the matching angle MA is an angle of the candidate direction with respect to the vertical direction. That is, the matching angle MA is an angle with respect to the vertical direction of a straight line passing through the center position of the target block and the center position of the reference block.
FIG. 19A shows an example when the matching angle MA is large, and FIG. 19B shows an example when the matching angle MA is small. In FIGS. 19A and 19B, a region surrounded by a broken line indicates a pixel. As shown in FIGS. 19A and 19B, the vertical length of the block is longer when the matching angle MA is larger than when the matching angle MA is small. Specifically, in FIG. 19A, the block size (the length in the vertical direction of the block) is a length of 3 pixels, and in FIG. 19B, the block size is a length of 5 pixels. Has been. The block size is determined by the same method as in the other embodiments. The block size is not limited to a length of 3 pixels and a length of 5 pixels.
In this way, by controlling the block size, the edge direction can be accurately determined as in the other embodiments.

As described above, in this embodiment, the length in the vertical direction is longer in the block set for the candidate direction having a small angle with respect to the vertical direction than in the block set for the candidate direction having a large angle with respect to the vertical direction. In addition, the block size is controlled. As a result, the edge direction can be accurately determined as in the other embodiments.
In the first to fourth embodiments, the case where the predetermined direction is the horizontal direction and the case where the predetermined direction is the vertical direction are illustrated, but the predetermined direction may be an oblique direction.

301,403 Matching area calculation unit D
302, 404 Matching area calculation unit U
304,406 Position determining unit 305,407 Matching angle determining unit 306,408 Block size determining unit 307,409 Block matching unit 308,410 Matching angle calculating unit 402 Matching region calculating unit 2D
501 Gradient calculation unit 502 Block size correction unit

Claims (13)

An edge direction determination device that determines the direction of an edge at a point of interest in an image,
For each of a plurality of directions that are edge direction candidates, setting means for setting n (n is an integer of 2 or more) blocks of the same size on a straight line in that direction passing through the point of interest;
Determination means for calculating the image similarity between the n blocks set by the setting means for each of the plurality of directions and determining the direction having the highest similarity as the edge direction at the point of interest; ,
Have
For each of the directions that are candidates for edge directions, the setting means
For at least one of the n blocks set for that direction, the gradient of the change in pixel value with respect to the change in the position in the predetermined direction at the position on the straight line, which is the set position of that block, is calculated. ,
An edge direction determination device that controls the size of the n blocks to be set so that the length in the predetermined direction is longer when the gradient is smaller than when the gradient is large . The setting means sets the size of the n blocks to be set as a first size when the gradient is greater than or equal to a predetermined value, and sets the size of the n blocks to be set when the gradient is less than the predetermined value. The edge direction determination apparatus according to claim 1 , wherein the second size is larger than the first size. The setting means includes
A gradient is calculated for each of a plurality of blocks among the n blocks to be set,
3. The edge direction determination device according to claim 1, wherein the size of the n blocks to be set is controlled according to representative values of a plurality of gradients calculated for the plurality of blocks. The edge direction determination device according to claim 3 , wherein the representative value is a maximum value of the plurality of gradients. The gradient is a difference between a pixel value at a setting position of a block and a pixel value at a position away from the setting position in the predetermined direction by a predetermined distance, a pixel value at a setting position of the block, and the predetermined value from the setting position. of the difference between the pixel value at the position spaced a predetermined distance in the direction opposite to the direction, the edge direction determination apparatus according to any one of claims 1 to 4, characterized in that a larger value. The block edge direction determination apparatus according to any one of claims 1 to 5, characterized in that a rectangular region having a long side along the predetermined direction. An image processing apparatus that converts an interlaced image signal into a progressive image signal by interpolation,
The edge direction determination device according to any one of claims 1 to 6 ,
Interpolation pixel generation means for generating an interpolation pixel by intra-field interpolation based on the edge direction determined by the edge direction determination device;
An image processing apparatus comprising: A control method of an edge direction determination device that determines the direction of an edge at a point of interest in an image,
A setting step of setting n (n is an integer of 2 or more) blocks of the same size on a straight line passing through the point of interest for each of a plurality of directions that are edge direction candidates;
A determination step of calculating an image similarity between the n blocks set in the setting step for each of the plurality of directions, and determining a direction having the highest similarity as an edge direction at the point of interest; ,
Have
In the setting step, for each direction that is a candidate for an edge direction,
For at least one of the n blocks set for that direction, the gradient of the change in pixel value with respect to the change in position in the predetermined direction at the position on the straight line, which is the set position of that block, is calculated. ,
Control of the edge direction determination apparatus, wherein the size of the n blocks to be set is controlled so that the length in the predetermined direction becomes longer when the gradient is small than when the gradient is large Method. In the setting step, when the gradient is equal to or larger than a predetermined value, the size of the n blocks to be set is set as a first size, and when the gradient is less than the predetermined value, the size of the n blocks to be set is set Is a second size larger than the first size
The control method of the edge direction determination apparatus according to claim 8. In the setting step,
A gradient is calculated for each of a plurality of blocks among the n blocks to be set,
The size of the n blocks to be set is controlled according to the representative values of the plurality of gradients calculated for the plurality of blocks.
The control method of the edge direction determination apparatus of Claim 8 or 9 characterized by the above-mentioned. The representative value is a maximum value of the plurality of gradients.
The control method of the edge direction determination apparatus according to claim 10. The gradient is a difference between a pixel value at a setting position of a block and a pixel value at a position away from the setting position in the predetermined direction by a predetermined distance, a pixel value at a setting position of the block, and the predetermined value from the setting position. The larger of the differences from the pixel value at a position that is a predetermined distance away in the opposite direction
The control method of the edge direction determination apparatus of any one of Claims 8-11 characterized by the above-mentioned. The block is a rectangular region having a long side along the predetermined direction.
The control method of the edge direction determination apparatus of any one of Claims 8-12 characterized by the above-mentioned.

Images