JP2010093730A - Apparatus and method for processing video signal, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the following problem: when specifying an area of an image obtained by a specific pulldown pattern from an interlaced image in which interlaced images obtained by different pulldown patterns are mixed, a detection error may be generated. <P>SOLUTION: A weave determination part 6 determines whether an area in a first composite image (present field) which is an interlaced image is a first area in which a progressive image is to be configured by combining the first composite image and a second composite image (preceding field), or a second area in which a progressive image is to be configured by combining the first composite image and a third composite image (succeeding field) (S106). When both first and second areas exist in the first composite image, a block detection part 7 specifies an area in which an interlaced image obtained from a progressive image of a second frame rate is composed, in accordance with the first and second areas (S214). <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for specifying a region of an image obtained by a specific pull-down pattern.

  As a format used when transmitting and storing moving image data, an interlaced image having 60 fields per second has been widely used. For transmission by digital broadcasting and storage of digital data by DVD, an interlaced image of 60 fields per second is compressed using an MPEG-2 moving image encoding method. Also in video camera shooting, shooting in an interlace format of 60 fields per second is common. On the other hand, progressive images of 24 frames per second are widely used for movie shooting and screening. Therefore, when a movie is broadcast or recorded on a DVD, a process of converting a progressive image of 24 frames per second into an interlaced image of 60 fields per second is generally performed. This conversion is generally called 2-3 pull down. It is. In addition, subtitles added to such movies may be generated as progressive images of 30 frames per second. Such subtitle data is converted into an interlaced image of 60 fields per second when a movie is broadcast or recorded on a DVD. As described above, a method of converting a progressive image of 30 frames per second into an interlaced image of 60 fields per second is generally called 2-2 pull-down.

  FIGS. 15A to 15A show images of the main movie of 24 frames per second, and FIGS. 15A to 15B show interlaced images of 60 fields per second. 2-3 pull-down is a sequence of 2-3, 2-3, A, B, C, D, continuous frames like Ae, Ao, Be, Bo, Be, Co, Ce, Do, De, Do. Divide into interlaced images by pattern. As with the interlaced image, FIGS. 15B to 15A show subtitle images of 30 frames per second, and FIGS. 15B and 15B show subtitles obtained by 2-2 pulldown, 60 seconds per second. An interlaced image of the field is shown. 2-2 pull down is an interlaced image of A, B, C, and D continuous frames in the pattern of 2-2-2 like Ae, Ao, Be, Bo, Ce, Co, De, and Do. Divide into

  Weave is generally known as a technique for converting an interlaced image obtained by pulling down 2-3 or 2-2 into a progressive image (hereinafter referred to as I / P conversion). Weave is a technique for generating a progressive image by combining the preceding and following fields. FIGS. 16A and 16B show how an interlaced image that has been pulled down 2-3 or 2-2 is subjected to I / P conversion by a weave. With regard to 2-3 pulldown, it is combined with each interlaced image in the order of back, front, back, front, and front, whereas with respect to 2-2 pulldown, back, front, back, front, back, in order. Combined with each interlaced image. In this way, when I / P conversion is performed according to one pull-down pattern in a video in which a plurality of pull-down pattern pixels are mixed, the image quality of a video with a different pull-down pattern may be reduced. For example, when I / P conversion is performed according to the pull-down pattern of the main part of the movie, the interlaced image in which the subtitle appears and the interlaced image in which the subtitle does not appear are weave at the moment when the subtitle appears or disappears on the screen Subtitles may be blurred. Further, for example, when the subtitle moves by scrolling, comb-like noise (hereinafter referred to as combing) may occur in the scanning line direction of the subtitle portion. Conversely, when I / P conversion is performed according to the subtitle pull-down pattern, the subtitles are normally displayed, but there may be a phenomenon that the main part of the movie is disturbed.

In order to solve such a problem, a method has been proposed in which captions are detected based on a luminance difference, and different I / P conversion is performed in the caption portion and other portions (for example, Patent Document 1).
JP 2007-074439 A

  However, depending on the video, there is a case where an area of an image obtained by a specific pull-down pattern is erroneously detected.

  That is, when an area of an interlaced image obtained by a specific pulldown pattern is specified from an interlaced image in which interlaced images obtained by different pulldown patterns are mixed, the area may be erroneously detected.

  In other words, in some cases, an interlaced image region obtained from a progressive image having a specific frame rate cannot be identified from an interlaced image in which interlaced images obtained from progressive images having different frame rates are mixed.

  For example, if a subtitle portion is to be specified by a luminance difference, there is a possibility that a non-subtitle portion may be erroneously detected as a subtitle depending on a luminance difference of a non-subtitle portion. Further, if a portion that is not a subtitle is detected as a subtitle, I / P conversion of the area is performed by a method according to the pull-down pattern of the subtitle, which may cause image quality degradation. Also, depending on the caption and the surrounding brightness, the caption area may not be detected as a caption.

  An object of the present invention is to accurately detect an area of an interlaced image obtained from a progressive image having a specific frame rate from an interlaced image in which interlaced images obtained from progressive images having different frame rates are mixed.

  In order to solve the above problems, for example, a video signal processing apparatus of the present invention has the following configuration.

  That is, an interlace in which an interlaced image at the third frame rate obtained from the progressive image at the second frame rate is combined with an interlaced image at the third frame rate obtained from the progressive image at the first frame rate. A video signal processing apparatus that identifies an area in which an interlaced image obtained from a progressive image at the second frame rate is synthesized from a first synthesized image that is an image, the first synthesized image being an interlaced image The region in the image is a first region in which a progressive image is to be formed by combining the first composite image and the second composite image input before the first composite image, or the first A progressive image is formed by combining a composite image and a third composite image input after the first composite image. A determination unit that determines whether the second region should be determined; and when the first region and the second region exist in the first composite image, the first region and the second region And specifying means for specifying an area in which an interlaced image obtained from the progressive image at the second frame rate is combined according to the area.

  The video signal processing method according to the present invention also includes a third frame obtained from a progressive image at a second frame rate in an interlaced image at a third frame rate obtained from a progressive image at a first frame rate. A video signal performed by a video signal processing device that identifies an area in which an interlaced image obtained from a progressive image at the second frame rate is synthesized from a first synthesized image that is an interlaced image obtained by synthesizing a rate interlaced image In the processing method, the region in the first composite image that is the interlaced image is progressive by combining the first composite image and the second composite image input before the first composite image. The first region to be included in the image, or the first region input after the first composite image and the first composite image A determination step of determining whether or not the second region is to form a progressive image by combining with the composite image, and when the first region and the second region exist in the first composite image And a specifying step of specifying a region in which an interlaced image obtained from a progressive image at the second frame rate is combined according to the first region and the second region.

  According to the present invention, it is possible to accurately detect a region of an interlaced image obtained from a progressive image having a specific frame rate from an interlaced image in which interlaced images obtained from progressive images having different frame rates are mixed.

  Furthermore, according to the present invention, it is possible to obtain a higher quality video by switching the progressive image generation method according to the detected area.

  Hereinafter, an embodiment according to the present invention will be described in detail according to the attached screen.

<First Embodiment>
FIG. 1 is a block diagram illustrating an example of a video signal processing apparatus according to the first embodiment. In order to simplify the description, an example in which the processing target is a luminance signal of a video signal will be described, but the present invention is not limited to this. That is, the video signal may include, for example, a color signal in addition to the luminance signal. In this case, the present invention can be applied to the color signal.

  The video signal processing apparatus shown in FIG. 1 includes an input terminal 1, a field memory 2, a field memory 3, a combing degree calculation unit 4, and a combing degree calculation unit 5. The video signal processing device also includes a weave determination unit 6, a block detection unit 7, a caption detection unit 8, a motion adaptive I / P conversion unit 9, a weave I / P conversion unit 10, a selector 11, and an output terminal 12.

  The input terminal 1 is an input terminal for inputting an interlaced video signal. In the present embodiment, the interlaced video signal input from the input terminal 1 is 60 interlaced video signals per second. The interlaced video signal is input as a composite image video signal including interlaced images generated by different pull-down patterns. That is, for example, 60 interlaced images per second obtained by pulling down a progressive image (main movie) of 24 frames per second by 2-3 pull down, and 60 interlaced images per second obtained by 2-2 pulldown of a progressive image (caption) of 30 frames per second. The composite image is input.

  The video signal processing apparatus according to the present embodiment specifies a region where an interlaced image obtained from, for example, a progressive image (caption) of 30 frames per second is synthesized from an input interlaced image (synthesized image).

  Each of the field memory 2 and the field memory 3 has a capacity for storing one field of the interlace signal input from the input terminal 1.

  Each of the combing degree calculation unit 4 and the combing degree calculation unit 5 calculates the combing degree in units of blocks based on the two input fields. Details of the combing degree calculation will be described later.

  The weave determination unit 6 determines, in block units, a weave pattern indicating whether the current field should be weighed with the previous or next field according to the combing degree calculated by the combing degree calculation units 4 and 5. That is, whether the weave determination unit 6 should generate a frame (progressive image) by combining the current field and the previous field, or should generate a frame by combining the current field and the subsequent field, depending on the degree of combing. Is determined for each block. The previous field is a field input before the current field. The subsequent field is a field input after the current field. In addition, the current field is an interlaced image that is a target of subtitle area detection in the present embodiment, the previous field is an interlaced image input before the current field, and the subsequent field is an interlaced image input after the current field. is there.

  That is, the weave determination unit 6 uses the first composite image and the second composite image input before the first composite image as a region in the composite image (first composite image) that is an interlaced image. It is determined whether the first region is to be combined to form a progressive image. In addition, the weave determination unit 6 includes a first image in which a region in the first composite image is a progressive image that is a combination of the first composite image and the third composite image input after the first composite image. 2 is determined. Then, the weave determination unit 6 counts the number of blocks determined for each weave pattern.

  Note that the determination result by the weave determination unit 6 of the present embodiment includes a pattern that should not be combined with either field and a pattern that can be combined with either field. Processing of weave determination by the weave determination unit 6 will be described later.

  The block detection unit 7 specifies a block indicating a weave pattern that occupies most of the entire frame as a main block, and further specifies a block indicating a reverse weave pattern as a sub-block with respect to the main block. The block detection unit 7 stores a range of the area that can be taken by the area to be specified in advance, and based on the range, the main block (most of the weave pattern blocks) and sub-blocks (small portions of the weave). Identify the block of patterns. That is, in this embodiment, the main block area is specified as the main movie area. On the other hand, the sub-block area is identified as a caption area. For example, the block detection unit 7 stores in advance a range of the area that the subtitle area can take in the entire frame. If the area of the weave pattern with a small number of determined blocks is within the range of the area that the pre-stored caption area can take, the area is sub-blocked (the block of the small portion of the weave pattern). ). Then, the block detection unit 7 specifies an area other than the sub-block as a main block (a block of most of the weave patterns). However, the main block area may be specified first.

  In other words, the block detection unit 7, when it is determined by the weave determination unit 6 that, for example, a frame (progressive image) to be reproduced in combination with the previous field is to be generated in combination with the previous field, that block is detected. The main block. Then, the block detection unit 7 identifies, as a sub-block, a block determined to generate a frame (progressive image) to be reproduced in combination with the subsequent field in the current field. The block detection unit 7 also detects a weave pattern that occupies most of the block. That is, the block detection unit 7 detects that the weave pattern occupying most is, for example, a combination pattern of the current field and the previous field.

  The caption detection unit 8 specifies the caption area by calculating the luminance difference between the target pixel and the surrounding pixels in the sub-block. Then, the caption detection unit 8 outputs a caption detection signal to the selector 11 based on the identified caption area.

  The motion adaptive I / P conversion unit 9 performs pixel interpolation by motion adaptive I / P conversion based on the field information. The motion-applied I / P conversion is a method for performing I / P conversion using interpolation pixel information calculated from motion information corresponding to frame data reproduced before and after. Here, I / P conversion is a technique for converting an interlaced image into a progressive image. However, the motion application I / P conversion unit 9 is not limited to motion application I / P conversion, and may generate a frame image by interpolating an interlaced image using data in the current field, for example. .

  The weave I / P conversion unit 10 performs pixel interpolation by I / P conversion based on the weave based on the field information. That is, the weave I / P conversion unit 10 generates a frame to be reproduced by combining the field corresponding to the weave pattern determined by the weave determination unit 6 and the current field.

  The selector 11 selects pixel data to be output to the output terminal 12 among the pixel data of the frame output from the motion adaptive I / P converter 9 and the pixel data of the frame output from the weave I / P converter 10. Switching is performed according to the caption detection signal output from the caption detection unit 8. That is, the selector 11 outputs the pixel output from the motion application I / P conversion unit 9 for the area determined to be the subtitle portion, and the weave I / P conversion for the area determined to be the movie main part. The pixels output from the unit 10 are output.

  The output terminal 12 outputs the progressive image output from the motion application I / P conversion unit 9 or the weave I / P conversion unit 10 to an output device (not shown) according to the switching by the selector 11. The output device in this embodiment is, for example, a digital television or a DVD recorder that supports progressive image reproduction.

  The outline of the processing of the video signal processing apparatus with the above configuration is as follows.

  An interlaced video signal input from the input terminal 1 (in the embodiment, for the sake of simplicity, a luminance signal having no color component) is supplied from the field memory 2, the combing degree calculation unit 4, the motion adaptive I / O. It is supplied to the P conversion unit 9 and the weave I / P conversion unit 10, respectively.

  The field memory 2 delays the input signal by one field time, and the delayed signal is a field memory 3, a combing degree calculation unit 4, a combing degree calculation unit 5, a motion adaptive I / P conversion unit 9, and a weave I / P conversion. Unit 10 and caption detection unit 8.

  The field memory 3 further delays the input signal delayed by the field memory 2 by one field and outputs the delayed signal to the combing degree calculation unit 5, the motion adaptive I / P conversion unit 9, and the weave I / P conversion unit 10.

  The combing degree calculation unit 4 constructs a progressive image by weaving the two input fields for each block, and calculates the combing degree in each block. Then, the calculated combing degree is output to the weave determination unit 6. However, the combing degree calculation unit 4 forms a frame by weaving two fields instead of weaving for each block, divides the frame into a plurality of blocks, and calculates the combing degree in each block. Anyway.

  Similarly, the combing degree calculation unit 5 divides a frame generated by the input two-field weave into a plurality of blocks, calculates the combing degree in each block, and sends the calculated combing degree to the weave determination unit 6. Output.

  Here, the combing degree is a value for evaluating whether a frame generated by weaving two fields has a sense of incongruity. In other words, in the current field, when there is a relationship between adjacent pixels in the top and bottom, combing occurs if there is no relationship between adjacent pixels in the frame generated by weaving the current field and other fields. It is highly possible that Therefore, in such a case, a high combing degree is calculated. On the other hand, in the current field, when there is a relationship between pixels adjacent vertically, if there is a relationship between pixels adjacent vertically in a progressive image generated by weaving the current field and other fields, It is unlikely that combing has occurred. Therefore, in such a case, a low combing degree is calculated.

  That is, when the combing degree calculation unit 4 determines that combing occurs in a block of the frame when a frame (progressive image) is configured by combining the current field and the subsequent field, the combing degree of the block is increased. calculate. Further, when a frame is configured by combining the current field and the previous field, the combing degree calculation unit 5 calculates a high combing degree of the block when it can be determined that combing occurs in the block of the frame. The rear field is a field input after the current field, and the front field is a field input before the current field. On the other hand, when it can be determined that no combing occurs in the generated frame block, the combing degree calculation units 4 and 5 calculate a low combing degree for the block. Details of the method for calculating the combing degree will be described later.

  The weave determination unit 6 determines a weave pattern for each block in the current field based on the two combing degrees calculated by the combing degree calculation units 4 and 5, respectively. The weave pattern determined by the weave determination unit 6 is a pattern in which the weave between the current field and the previous field is optimal (pattern 1), or a pattern in which the weave between the current field and the subsequent field is optimal (pattern 2). ) Is included. Furthermore, the weave pattern determined by the weave determination unit 6 includes a pattern in which neither weave is optimal (pattern 3), or a pattern in which the weave pattern cannot be determined based on the combing degree (pattern 4). The weave determination unit 6 outputs the determined weave pattern to the block detection unit 7.

  When most of the blocks in the current field indicate one of the patterns 1 and 2, the block detection unit 7 identifies the area indicating the other pattern as a sub-block, The information is output to the caption detection unit 8. At this time, the block detection unit 7 outputs information indicating the weave pattern (pattern 1 or pattern 2) indicated by most blocks to the weave I / P conversion unit 10.

  That is, when the first area and the second area exist in the first composite image, the block detection unit 7 specifies the caption area as follows. Note that the first region is a region that is a second composite image in which an interlaced image that is to form a progressive image in combination with the region of the first composite image is input before the first composite image. The second region is a region that is a third composite image in which an interlaced image that should form a progressive image in combination with the first composite image is input after the first composite image. In this case, an area (caption area) in which an interlaced image obtained from a progressive image at the second frame rate is combined is specified according to each area. The first composite image corresponds to the current field, the second composite image corresponds to the previous field, and the third composite image corresponds to the subsequent field.

  In addition, the block detection unit 7 according to the present embodiment specifies the narrower area (subblock area) as the caption area between the first area and the second area.

  The caption detection unit 8 identifies a caption based on the pixel of the current field and the sub-block area information output from the block detection unit 7, and outputs a caption detection signal to the selector 11. In other words, the caption detection unit 8 specifies a caption region in units of blocks from the sub-block region information detected by the block detection unit 7, and further narrows down the detailed position of the caption based on the pixel information in the block.

  In other words, the caption detection unit 8 synthesizes an interlaced image (caption) obtained from a progressive image at the second frame rate based on luminance information in the narrower one of the first region and the second region. The position of the selected pixel is specified. Note that the first region is a region in which the interlaced image that is to form a frame in combination with the first composite image is the second composite image (previous field). The second area is an area in which the interlaced image that is to form the frame by combining the first synthesized image is the third synthesized image (the subsequent field).

  The motion adaptive I / P conversion unit 9 performs motion adaptive I / P conversion based on the input three fields (the previous field, the current field, and the subsequent field), and outputs the calculated pixel to the selector 11. However, as described above, the motion application I / P conversion unit 9 generates a frame image by interpolating an interlaced image using, for example, data in the current field, not limited to motion application I / P conversion. You may do it.

  The weave I / P conversion unit 10 performs I / P conversion by the weave based on the input three fields and the weave pattern occupying most of the fields, and outputs the calculated pixels to the selector 11.

  The selector 11 switches the output from the motion adaptive I / P conversion unit 9 and the weave I / P conversion unit 10 according to the caption detection signal output from the caption detection unit 8, and outputs it from the output terminal 12.

  The subtitle detection process and the video signal correction process corresponding to the detection result performed by the video signal processing apparatus configured as described above will be described in more detail.

  The interlaced image input from the input terminal 1 is stored in the field memory 2. The field memory 2 has a memory for one field, and delays the interlaced image from the input terminal 1 by one field time. That is, the field memory 2 stores a certain interlaced image (first interlaced image) input from the input terminal 1 until the next interlaced image (second interlaced image) is input. The field memory 2 converts the first interlaced image into the field memory 3, the combing degree calculation units 4 and 5, the caption detection unit 8, the motion applied I / P conversion at the timing when the second interlaced image is input. Output to the unit 9 and the weave I / P conversion unit 10. That is, the interlaced image output from the field memory 2 is delayed by one field time from the interlaced image input from the input terminal 1.

  Further, the interlaced image output from the field memory 2 is stored in the field memory 3. The field memory 3 has a memory for one field, and delays the interlaced image from the field memory 2 by one field time. That is, the field memory 3 stores a certain interlaced image (second interlaced image) output from the field memory 2 until the next interlaced image (third interlaced image) is input. Then, the field memory 3 sends the second interlaced image to the combing degree calculation unit 5, the motion application I / P conversion unit 9, and the weave I / P conversion unit 10 at the timing when the third interlaced image is input. Output each. That is, the interlaced image output from the field memory 3 is delayed by 2 field times from the interlaced image input from the input terminal 1.

  In this embodiment, an interlaced image (first interlaced image) delayed by two field times is used as the previous field. Also, an interlaced image (second interlaced image) delayed by one field time is a current field, and an input interlaced image (third interlaced image) is a subsequent field. That is, a first interlaced image, a second interlaced image, and a third interlaced image are sequentially input from the input terminal 1. When the third interlaced image (back field) is input from the input terminal 1, the field memory 2 stores the third interlaced image (back field). At this time, the field memory 3 stores the second interlaced image (current field).

  Further, at this time, the combing degree calculation unit 4 stores the second interlaced image (current field) and the third interlaced image (rear field). The combing degree calculation unit 5 stores a first interlaced image (previous field) and a second interlaced image (current field). The motion applied I / P conversion unit 9 and the weave I / P conversion unit 10 store first, second, and third interlaced images (previous field, current field, and subsequent field), respectively. Yes. The subtitle detection unit 8 stores a second interlaced image (current field).

  The combing degree calculation unit 4 calculates the combing degree in units of blocks over one frame. That is, the combing degree calculation unit 4 divides a frame formed by combining the current field and the subsequent field into a plurality of blocks, and calculates the combing degree in each block. In this embodiment, the combing degree is calculated for each block of 240 × 135 pixels obtained by dividing an image (frame) of 1920 × 1080 pixels. However, the frame size and the block size are not limited to this.

  The combing degree calculation unit 4 receives a frame generated by combining (weaveing) the current field and the subsequent field as shown in FIG. This is divided into blocks of 240 × 135 pixels as described above. The combing degree calculation unit 4 calculates the combing degree of each block as follows.

  First, the combing degree calculation unit 4 uses a peripheral pixel including each pixel and its upper and lower pixels in one block of a frame generated by the weave of the two fields, and uses a small pixel centered on each pixel. Calculate the combing degree of the area. That is, the combing degree calculation unit 4 further divides the blocks constituting the frame into a plurality of small areas, and calculates the combing degree of the small areas. The combing degree calculation unit 4 determines that the central pixel of the small area is a combing pixel if the combing degree of the small area is larger than the threshold value. That is, the combing degree calculation unit 4 determines that the central pixel of the small area is the combing pixel when the correlation between the vertically adjacent pixels is low in the small area. Further, the combing degree calculation unit 4 calculates the combing degree of the block by counting the number of combing pixels in the block. The small area of this embodiment is assumed to be the current field 3 × 2 pixels and the rear field 3 × 3 pixels. However, the size of the small area is not limited to this. As shown in FIG. 3, the pixel values of the small area in the current field are C00-C12, and the pixel values of the small area in the subsequent field are N00-N22. Details of the processing will be described below.

First, as shown in FIG. 4, the combing degree calculation unit 4 forms a frame using the small area pixels in the current field and the small area pixels in the subsequent field, and further divides the frame into three small area blocks. . When each small area block is a block A, a block B, and a block C from the top of FIG. 4, the following calculation is performed in each small area block. However, min (a, b, c) calculates the smallest value of a, b, c, and max (a, b, c) calculates the largest value of a, b, c. Where | a | is the absolute value of a.
Block A:
A1 = min (| N00-C00 |, | N01-C01 |, | N02-C02 |)
A2 = min (| C00−N10 |, | C01−N11 |, | C02−N12 |)
A3 = min (| N00−N10 |, | N01−N11 |, | N02−N12 |)
SA = A1 + A2-A3
Block B:
B1 = min (| C00-N10 |, | C01-N11 |, | C02-N12 |)
B2 = min (| N10-C10 |, | N11-C11 |, | N12-C12 |)
B3 = min (| C00-C10 |, | C01-C11 |, | C02-C12 |)
SB = B1 + B2-B3
Block C:
C1 = min (| N10-C10 |, | N11-C11 |, | N12-C12 |)
C2 = min (| C10-N20 |, | C11-N21 |, | C12-N22 |)
C3 = min (| N10-N20 |, | N11-N21 |, | N12-N22 |)
SC = C1 + C2-C3

  That is, A1 is calculated such that the larger the difference between adjacent pixel values in the block A, which is a small area block of the frame combining the current field and the subsequent field, is larger. The same applies to A2. In A3, a larger value is calculated as the difference between pixel values adjacent in the vertical direction in the subsequent field increases. Accordingly, the SA is calculated such that the pixel value adjacent in the vertical direction is large in the frame in which the current field and the subsequent field are combined and the difference between the pixel values adjacent in the vertical direction in the subsequent field is small. The concept of SB and SC is the same as SA.

Further, the combing degree calculation unit 4 calculates the combing degree (score1) of the small area according to the following equation based on the calculation result of each small area block.
M1 = min (SA, SB)
M2 = min (SB, SC)
score1 = max (M1, M2)
If this score1 is larger than a threshold value (bad_block_num_threshold), the central pixel of this small area is determined as a combing pixel. That is, the combing degree calculation unit 4 determines that the pixel corresponding to the position of N11 is a combing pixel when the combing degree of the small area in FIG. 3 is higher than the threshold value.

  That is, the combing degree is a value based on a difference between pixel values adjacent in the vertical direction in a frame in which interlace images are combined.

Further, when it is determined that the central pixel of the small area is a combing pixel, the combing degree calculation unit 4 increments bad_block_count1 indicating the number of combing pixels in the block. If bad_block_count1 when one block is all combing pixels is bad_block_max1, the combing degree comb1 of this block is:
comb1 = bad_block_count1 / bad_block_max1
Is calculated by That is, the combing degree calculation unit 4 calculates the combing degree (comb1) of the block according to the number of pixels in the block and the number of pixels determined to be the combing pixel in the block. The combing degree (comb1) is a combing degree of a certain target block in a frame in which the current field and the subsequent field are combined.

  The combing degree calculation unit 4 calculates the combing degree of each block in the frame generated by the weave of the current field and the subsequent field by the method as described above. As a result, all the blocks in the frame in which the current field and the subsequent field are combined have a combing degree.

  Similarly, details of the combing degree calculation unit 5 will be described. The outline of the processing is the same as that of the combing degree calculation unit 4. As shown in FIG. 5, the combing degree calculation unit 5 receives a frame generated by combining (weave) the current field and the previous field. This is divided into blocks of 240 × 135 pixels as described above. The combing degree calculation unit 5 calculates the combing degree of each block as follows.

  The combing degree calculation unit 5 first calculates the combing degree of a small area centered on each pixel by using each pixel and surrounding pixels in one block of a frame generated by the weave of the two fields. calculate. That is, the combing degree calculation unit 5 further divides the blocks constituting the frame into a plurality of small areas and calculates the combing degree of the small areas. If the combing degree of the small area is larger than the threshold value, it is determined that the central pixel of the small area is a combing pixel. Further, the combing degree calculation unit 5 calculates the combing degree of the block by counting the number of combing pixels in the block. As described above, the small area of the present embodiment is an area composed of the current field 3 × 2 pixels and the rear field 3 × 3 pixels. As shown in FIG. 6, the pixel value of the small area in the current field is C00-C12, and the pixel value of the small area in the previous field is P00-P22. Details of the processing will be described below.

As shown in FIG. 7, the combing degree calculation unit 5 forms a frame using the small area pixels in the current field and the small area pixels in the previous field, and further divides it into three small area blocks. When each small area block is a block D, a block E, and a block F from the top of FIG. 7, the following calculation is performed in each small area block.
Block D:
D1 = min (| P00-C00 |, | P01-C01 |, | P02-C02 |)
D2 = min (| C00-P10 |, | C01-P11 |, | C02-P12 |)
D3 = min (| P00-P10 |, | P01-P11 |, | P02-P12 |)
SD = D1 + D2-D3
Block E:
E1 = min (| C00-P10 |, | C01-P11 |, | C02-P12 |)
E2 = min (| P10-C10 |, | P11-C11 |, | P12-C12 |)
E3 = min (| C00-C10 |, | C01-C11 |, | C02-C12 |)
SE = E1 + E2-E3
Block F:
F1 = min (| P10-C10 |, | P11-C11 |, | P12-C12 |)
F2 = min (| C10-P20 |, | C11-P21 |, | C12-P22 |)
F3 = min (| P10-P20 |, | P11-P21 |, | P12-P22 |)
SF = F1 + F2-F3

  That is, a larger value of D1 is calculated as the difference between adjacent pixel values in the block D, which is a small area block of a frame obtained by combining the current field and the previous field, is larger. The same applies to D2. Then, a larger value is calculated for D3 as the difference between the pixel values vertically adjacent in the previous field increases. Therefore, SD is calculated such that a pixel value adjacent to the upper and lower sides is larger in a frame in which the current field and the previous field are combined, and a larger value is obtained as the difference between the pixel values adjacent to the upper and lower sides in the previous field is small. The concept of SE and SF is the same as that of SD.

Further, the combing degree calculation unit 5 calculates the combing degree (score2) of the small area according to the following equation based on the calculation result of each small area block.
M3 = min (SD, SE)
M4 = min (SE, SF)
score2 = max (M3, M4)

If this score2 is larger than a threshold (bad_block_num_threshold), the central pixel of this small area is determined as a combing pixel. That is, the combing degree calculation unit 5 determines that the pixel corresponding to the position of P11 is a combing pixel when the combing degree of the small area in FIG. 6 is higher than the threshold value. Further, when the combing degree calculation unit 5 determines that the center pixel of the small area is a combing pixel, the combing degree calculation unit 5 increments bad_block_count2 indicating the number of combing pixels in the block. When bad_block_count2 when one block is all combing pixels is set to bad_block_max2, the combing degree comb2 of this block is
comb2 = bad_block_count2 / bad_block_max2
Is calculated by That is, the combing degree calculation unit 5 calculates the combing degree (comb2) of the block according to the number of pixels in the block and the number of pixels determined to be the combing pixel in the block. The combing degree (comb2) is a combing degree of a certain target block in a frame in which the current field and the previous field are combined.

  The combing degree calculation unit 5 calculates the combing degree of each block in the frame generated by the weave between the current field and the previous field by the method as described above. As a result, all the blocks in the frame combining the current field and the previous field have a combing degree.

  When the combing degree which is an output from the combing degree calculation unit 4 and the combing degree calculation unit 5 is input, the weave determination unit 6 determines the weave pattern of each block.

  FIG. 8 is a flowchart illustrating a weave pattern determination process performed by the weave determination unit 6. Here, the threshold of the combing degree is mb_th. That is, for example, when the calculated combing degree of a certain weave pattern is higher than the threshold mb_th, it is determined that there is a high possibility that combing will occur in the block. This threshold mb_th can be set to a value such as 0.6, for example. However, another value may be used.

  Further, the weave determination unit 6 determines that the combination of the current field and the previous field is optimal (pattern 1) when field_pair determined by comb1, comb2, and mb_th is pattern1. When field_pair is pattern2, it is determined that the combination of the current field and the subsequent field is optimal (pattern 2). When field_pair is pattern3, it is determined that the combination with any of the preceding and following fields is not optimal (pattern 3). Further, when the field_pair is pattern4, the weave determination unit 6 determines that combing does not occur in any combination with the preceding or following field (pattern 4). In the case of the pattern 4, it is determined which field should be combined with the current field based on the past pattern information. However, in this embodiment, the weave pattern in the case of pattern 4 is determined by the selector 11. The weave determination unit 6 outputs a signal based on the determined pattern to the block detection unit 7. Hereinafter, details of the processing by the weave determination unit 6 will be described with reference to FIG. Note that the weave determination unit 6 performs the process of FIG. 8 on all blocks in the current field for each block (target block) in the frame.

  First, the variable n is initialized to 0 (S101).

  Next, comb1 is compared with the threshold mb_th (S102). As described above, comb1 is the combing degree of a certain target block in a frame in which the current field and the subsequent field are combined. That is, when comb1 is larger than the threshold value mb_th, it means that it can be determined that combing has occurred in the target block of the frame in which the current field and the subsequent field are combined. This mb_th can be set to an arbitrary value. If comb1 is larger than the threshold value mb_th, the process proceeds to S107. On the other hand, if comb1 is smaller than the threshold value mb_th, the process proceeds to S103.

  In S107, n is incremented and the process proceeds to S103.

  In S103, the weave determination unit 6 compares comb2 with the threshold mb_th. As described above, comb2 is the combing degree of a certain target block in a frame in which the current field and the previous field are combined. That is, when comb2 is larger than the threshold mb_th, it means that it can be determined that combing has occurred in the target block of the frame in which the current field and the previous field are combined. If comb2 is larger than the threshold value mb_th, the process proceeds to S108. On the other hand, if comb2 is smaller than the threshold mb_th, the process proceeds to S104.

  In S108, n is incremented, and the process proceeds to S104.

  In S104, it is checked whether or not the value of n is 1. When n is 1, it means that either comb1 or comb2 is above the threshold mb_th and the other is below the threshold mb_th. That is, in the target block of the current field, combing has occurred due to the combination with the previous or subsequent field, but no combing has occurred with the combination with the other field. In such a case, the weave determination unit 6 determines that the combination with the field where combing has not occurred is optimal.

  When it is determined in S104 that n is 1, the process proceeds to S106, and when it is determined that n is not 1, the process proceeds to S105.

  In S106, the weave determination unit 6 compares the magnitude relationship between comb1 and comb2. If it is determined in S106 that comb1 is greater than comb2, the process proceeds to S111. If it is determined that comb1 is smaller than comb2, the process proceeds to S112.

  That is, in S106, the weave determination unit 6 determines whether the interlace image that is to form the progressive image in combination with the first composite image (current field) that is an interlace image is the second composite image (previous field) or the third It is determined whether it is a composite image (after field). Note that the second composite image is a composite image input before the first composite image, and the third composite image is a composite image input after the first composite image.

  In S111, the weave determination unit 6 determines that it is optimal to combine the current field and the previous field to form the target block of the reproduction frame (field_pair = pattern1), and sends a signal indicating this to the block detection unit 7 Output to. That is, the weave determination unit 6 determines that combing occurs when the current block and the subsequent field are combined to form the block of interest, and that combing does not occur when the current field and the previous field are combined. Then, a signal indicating that the combination with the previous field is optimal (pattern 1) is output to the block detector 7.

  On the other hand, the weave determination unit 6 determines in S112 that it is optimal to configure the target block of the playback frame by combining the current field and the subsequent field (field_pair = pattern2), and detects a signal indicating the block detection. Output to unit 7. That is, the weave determination unit 6 determines that combing occurs when the current block and the previous field are combined to form the block of interest, and that the combination of the current field and the subsequent field forms the block of interest does not generate combing. Then, a signal indicating that the combination with the subsequent field is optimal (pattern 2) is output to the block detector 7.

  That is, the weave determination unit 6 determines whether the interlaced image that is to form the progressive image in combination with the first composite image (current field) is the second composite image (previous field) or the third composite image (back field). Determined by the degree of combing. In addition, as described above, the combing degree is a value based on a difference between pixel values adjacent in the vertical direction in a frame in which interlace images are combined. In addition, the combing degree includes a frame obtained by combining the first composite image (current field) and the second composite image (previous field), and the first composite image and the third composite image (back field). Are calculated for each frame.

  If the weave determination unit 6 determines that n is not 1 in S104, the weave determination unit 6 determines whether or not n is 2 in S105. The fact that n is 2 indicates that both comb1 and comb2 are larger than the threshold value mb_th. In other words, when n is 2, combing occurs in the target block in both the combination of the current field and the previous field, and the combination of the current field and the subsequent field, and the combination with any of the preceding and subsequent fields is not suitable. Means. If it is determined in S105 that n is 2, the process proceeds to S110, and if it is determined that n is not 2, the process proceeds to S109.

  In S110, the weave determination unit 6 determines that the combination of both the previous and subsequent fields is not appropriate (field_pair = pattern3), and outputs a signal indicating this to the block detection unit 7. In other words, the weave determination unit 6 determines that the combination with either field is not appropriate when combing occurs in the target block combining the current field and the previous field, or in the target block combining the current field and the subsequent field. To do. Then, the weave determination unit 6 outputs a signal indicating this (pattern 3) to the block detection unit 7.

  In S104, the weave determination unit 6 determines that n = 0. The case where n is 0 indicates a case where it is determined that combing has not occurred in the target block in both the combination of the current field and the previous field and the combination of the current field and the subsequent field (field_pair = pattern4). ). This situation occurs, for example, when there is little motion in the entire image or when the middle field is the current field among the three fields obtained by 2-3 pulldown. In this case, it is not possible to determine the optimum composite field based on the combing degree. When such a pattern (pattern 4) occurs, the weave determination unit 6 outputs a signal indicating that to the block detection unit 7.

  The selector 11 can determine the weave pattern of the block determined as such a pattern (pattern 4) based on, for example, the weave pattern of another block in the current field. That is, for example, the weave pattern of the block determined as the pattern 4 can be determined according to the larger one of the pattern 1 and the pattern 2. However, without being limited to this method, for example, the weave pattern of the pattern 4 may be determined based on the past pattern detection result. That is, the selector 11 may determine the weave pattern of the current block of interest according to the weave pattern of the field input so far.

  The weave determination unit 6 determines the weave pattern of the block of interest as described above, and outputs the result to the block detection unit 7. When the weave determination unit 6 outputs the determined weave pattern to the block detection unit 7, the weave determination unit 6 shifts the target block by one and performs the process of FIG.

  The block detection unit 7 receives the weave pattern block_pattern output from the weave determination unit 6. And the block detection part 7 specifies the subtitle area | region in a frame per block according to the input weave pattern for every block.

  A specific flow of processing performed by the block detection unit 7 will be described with reference to FIG. Here, for explanation, the number of blocks in one field is block_max, and the threshold is mj_th. Further, p = pattern1, n = pattern2, and b = pattern4. It should be noted that the block detection unit 7 of the present embodiment does not count the number of patterns (pattern 3) that do not weave in either the front or back interlaced image.

  First, in S201, n_cout, p_count, and b_count, which are count values of the weave pattern in each block in one field, are initialized. Also, block_count indicating the position of the block in the field is initialized. Next, in S202, block_count is incremented. At this time, block_count = 1 is the first block in the field, and block_count = last_block is the last block in the field. Next, the processing is switched depending on what block_pattern indicating the optimum pattern in the block is.

  First, in S203, it is determined whether block_pattern is n. If n, n_count is incremented in S207. If not n, the process proceeds to S204. Here, “block_pattern” is n indicates that the weave pattern of the block corresponding to the current block_count is determined to be the pattern 1 (field_pair = pattern1) of the combination of the current field and the subsequent field.

  In S204, it is determined whether block_pattern is p. If p, p_count is incremented in S208. If not p, the process proceeds to S205. Here, “block_pattern” is p indicates that the weave pattern of the block corresponding to the current block_count is determined to be the pattern 2 (field_pair = pattern2) of the combination of the current field and the previous field.

  In S205, it is determined whether block_pattern is b. If b, b_count is incremented in S209. If not b, the process proceeds to S206. Here, “block_pattern” is b indicates that the weave pattern of the block corresponding to the current block_count is determined as the pattern 4 (field_pair = pattern4) that cannot be determined by the weave determination unit 6.

  In S206, it is determined whether block_count is the last_block indicating the last block. If block_count is last_block, that is, if weave pattern counting has been completed for all blocks in the current field, the process proceeds to S210; otherwise, the process returns to S202.

  In S210, the block detection unit 7 compares the values of n_count and p_count. If n_count is larger (No in S210), the process proceeds to S212, and if p_count is larger (Yes in S210), the process proceeds to S211.

  In S212, the block detection unit 7 adds b_count to n_count, sets m, and proceeds to S213. That is, when n_count is larger than p_count, it indicates that there are more blocks where combing does not occur when the current field and the previous field are combined than when the current field and the previous field are combined. Therefore, when n_count is larger, the block determined as the weave pattern (pattern 2) combining the current field and the subsequent field and the block determined as pattern 4 are the main blocks. At this time, a block determined as a weave pattern (pattern 1) combining the current field and the previous field becomes a sub-block (caption area).

  On the other hand, in S211, the block detection unit 7 adds b_count to p_count, sets m, and proceeds to S213. That is, when p_count is larger than n_count, it indicates that there are more blocks where combing does not occur when the current field and the previous field are combined than when the current field and the subsequent field are combined. Therefore, when p_count is larger, the block determined as the weave pattern (pattern 1) combining the current field and the previous field and the block determined as pattern 4 are the main blocks. At this time, a block determined as a weave pattern (pattern 2) combining the current field and the subsequent field becomes a sub-block (caption area).

  That is, it is determined that m can be combined with either of the number of blocks whose weave pattern is a combination with the previous field and the number of the blocks which is a combination with the subsequent field. Indicates the number of blocks added. In other words, it is determined that m can be combined with either field in the area of the wider area of the area where the weave pattern is a combination with the previous field and the area where the weave pattern is a combination with the subsequent field. It shows the size of the added area.

  In S213, it is determined whether m is larger than the threshold value mj_th. In S213, if m is larger than the threshold value mj_th, the process proceeds to S214, and if m is smaller than the threshold value mj_th, the process is ended. That is, in the composite image assumed in the present embodiment, a subtitle image that occupies a small number of blocks is combined with a main movie image that occupies a large number of blocks on the current field. Moreover, since the pull-down pattern is different between the main movie image and the subtitle image, the optimal weave pattern may be different. Therefore, when the weave pattern determined for each block in the current field can be divided into a large number of weave patterns and a small number of weave patterns, the block determined as the small number of weave patterns includes a subtitle image. Conceivable. Note that the threshold value mj_th may be set in accordance with the ratio of the areas of two synthesized videos (the main movie and subtitles in this embodiment).

  In S214, the block detection unit 7 outputs, as a different pattern block (sub-block), a block determined as a smaller pattern of p_count and n_count. As described above, this sub-block can be regarded as a caption candidate block.

  That is, the block detection unit 7 includes the first region in which the progressive image is to be combined with the second combined image in the first combined image and the first region in which the progressive image is to be combined with the third combined image. If it is determined that the second area exists, the subtitle area is specified as follows. That is, in S214, an area (caption area) in which an interlaced image obtained from a progressive image at the second frame rate is combined is specified according to each area. The first composite image corresponds to the current field, the second composite image corresponds to the previous field, and the third composite image corresponds to the subsequent field.

  As described above, when the difference between the previous field, the current field, and the subsequent field is small (the motion of the moving image is small), it is difficult to determine the weave pattern based on the combing degree. Also, when the current field is an intermediate field when three interlaced images are generated from one progressive image in the 2-3 pulldown pattern, it is difficult to determine the weave pattern based on the combing degree. However, the block detection unit 7 of the present embodiment can estimate the caption candidate block even when it is difficult to determine the weave pattern based on the combing degree. This will be described in detail with reference to FIG.

  FIG. 10 shows the state of the current interlaced image at a certain time T = 0 and the current interlaced image at T = 1 after one field period. That is, from the input terminal 1, an interlaced image with T = 1 is input after an interlaced image with T = 0.

  Further, p, b, and n displayed on the right side of FIG. 10 each indicate what the weave pattern is. That is, p indicates that the weave pattern is determined to be a combination with the previous field (pattern 1), n is a combination with the subsequent field (pattern 2), and b is a combination with either field (pattern 4). .

  First, at T = 0, the weave determination unit 6 determines the weave pattern in each block by the method described above. And the block detection part 7 specifies the area | region shown by n in a figure as a subtitle candidate block.

  Next, at T = 1, the block detection unit 7 specifies a caption candidate block as in T = 0. The block detection unit 7 obtains the subtitle candidate block specified in the current field of T = 1 by the above method of T = 0 and T = −1 which are one field time before and two field times ago. An area for subtitle candidate blocks is added. The T = −1 field is an interlaced image input from the input terminal 1 immediately before the T = 0 field. Also, the added caption candidate block is not added in the next field (T = 2 interlaced image).

  As described above, the block detection unit 7 of the present embodiment specifies the caption area to be specified in the next interlaced image using the caption area specified in the current interlaced image. By doing this, it is possible to specify a subtitle area even in an interlaced image in which it is difficult to determine a weave pattern based on the degree of combing.

  The caption detection unit 8 detects captions based on the caption candidate blocks output from the block detection unit 7. That is, the caption detection unit 8 detects a caption pixel in the caption candidate block, for example, using luminance difference information between the interpolation target pixel and surrounding pixels.

  When detecting the caption pixel, the caption detection unit 8 outputs the detected caption area to the selector 11.

  The motion adaptive I / P converter 9 performs motion adaptive I / P conversion from three fields (the previous field, the current field, and the subsequent field), and outputs pixel data of the generated frame. That is, since the subtitle portion has a pull-down pattern different from that of the main movie of the movie, combing may occur in the subtitle portion when a progressive image is generated with the same weave pattern as the main movie. Therefore, the motion application I / P conversion unit 9 generates an interpolated pixel for displaying a subtitle portion based on frame data and motion information reproduced before and after. Note that the motion application I / P converter 9 of this embodiment outputs the interpolated pixel data for one frame generated from the current field. That is, the motion application I / P conversion unit 9 obtains pixel information for constituting a frame in combination with the current field in the narrower one of the first area and the second area (subblock area). It is generated as follows. That is, the motion application I / P conversion unit 9 generates pixel information using frame data displayed before and after the current field or pixel information in the current field. However, as described above, the generation of the interpolation pixel of the subtitle portion is not limited to the motion applied I / P conversion. For example, by using the surrounding pixels in the current field, the interlaced image is interpolated and the progressive image is converted. You may make it produce | generate. In addition, for a region determined to be a subtitle portion, a progressive image may be generated by the motion application I / P conversion unit 9 by a method other than weave.

  The weave I / P conversion unit 10 converts the pixels subjected to I / P conversion by the current field and the previous field weave and the pixels subjected to the I / P conversion by the current field and the previous field weave to the reverse of the caption candidate block. Output along the weave pattern. That is, when the weave pattern of the block determined as a caption candidate block is a weave between the current field and the previous field, the weave I / P conversion unit 10 performs I / P conversion with the current field and the subsequent field. Pixel data is output to the selector 11. Further, the weave I / P conversion unit 10 performs I / P conversion on the weaves of the current field and the previous field when the weave pattern of the block determined as the caption candidate block is the weave of the current field and the subsequent field. Pixel data is output to the selector 11. The pixel data output at this time is for one frame generated from the current field.

  That is, the weave I / P conversion unit 10 responds to the determination of the weave pattern as pixel information for constituting a frame in combination with the first synthesized image (current field) in the wider area (main block area). The pixel information of the second or third composite image is used. Note that the pixel information of the second or third composite image corresponds to the pixel information of the previous or subsequent field.

  The selector 11 outputs the pixel data output from the motion adaptive I / P converter 9 and the pixel data output from the weave I / P converter 10 to the output device as a result of caption detection. Switch with. That is, the selector 11 outputs the output pixel data from the motion application I / P conversion unit 9 for the area detected as the caption part, and outputs from the weave I / P conversion unit 10 for the other areas. Output pixel data.

  As described above, in this embodiment, the weave determination unit 6 determines the weave pattern in units of blocks. Then, the block detection unit 7 identifies a block indicating a weave pattern opposite to the weave pattern occupying most as a caption candidate block. Further, the caption detection unit 8 specifies the caption area by narrowing down the caption area by using pixel information from the caption candidate block.

  In this way, subtitles can be detected with high accuracy. Further, by switching the progressive image generation method between the subtitle portion and the other portions, it is possible to obtain a higher-definition progressive image.

  In other words, for example, in the case of specifying a caption area in a composite image including a main movie area and a caption area generated at different frame rates, the following cases may occur when trying to specify only by luminance information: May be falsely detected. That is, for example, if an object (for example, a signboard) having a luminance difference similar to the luminance difference between the subtitle and the background is included in the main movie image, it may be erroneously detected as a subtitle area. However, as in the present invention, the weave pattern is determined for each block of the composite image, and the area of the weave pattern with the fewer determined blocks is identified as the candidate for the caption area, thereby making the caption area more accurate. It becomes possible to identify well.

  In the embodiment, the luminance of the video signal has been described as an example. However, a current signal such as RGB may be used, or a chromaticity signal may be targeted.

  Further, although an example of a 1: 2 interlaced image has been described, the present invention is not limited to this, and it is possible to deal with other ratios of interlace by adjusting each calculation unit.

<Second embodiment>
The first embodiment may be realized by a general-purpose information processing apparatus such as a personal computer and a computer program executed by the information processing apparatus. Hereinafter, the example will be described as a second embodiment, focusing on differences from the first embodiment.

  FIG. 11 is a block configuration diagram of an information processing apparatus according to the second embodiment.

  In the figure, reference numeral 301 denotes a central processing unit (hereinafter referred to as CPU) that controls the entire apparatus and performs various processes. A memory 302 includes a ROM that stores a BIOS and a boot program, and a RAM that the CPU 301 uses as a work area. An instruction input unit 303 includes a keyboard, a pointing device such as a mouse, and various switches.

  Reference numeral 304 denotes an operating system (OS) necessary for controlling the present apparatus, application programs used in the present embodiment which will be described later with reference to FIGS. ). The external storage device 304 stores moving image data captured by a video camera 306, which will be described later, for example.

  A storage device 305 accesses a portable storage medium (for example, a DVD-ROM or a CD-ROM disk) that stores moving image data.

  Reference numeral 306 denotes a video camera that captures a moving image in field units. In other words, the video camera 306 stores, for example, 60 interlaced images per second obtained from 24 progressive images captured per second for the movie in the external storage device 304. In this embodiment, subtitle data of 30 progressive images per second is converted into 60 interlaced images per second and synthesized with 60 interlaced images captured by the video camera 306 per second.

  Reference numeral 307 denotes a monitor, and reference numeral 309 denotes a communication circuit, which includes a LAN, a public circuit, a wireless circuit, and a broadcast wave. A communication interface 308 transmits and receives encoded data via the communication circuit 309. In the present embodiment, processing for detecting a caption area is performed on moving image data from the communication interface 308, the external storage device 304, the storage device 305, or the like.

  The video signal processing in such a configuration will be described.

  Prior to processing, when the information processing apparatus is powered on according to an instruction from the instruction input unit 303, the CPU 301 follows the boot program (stored in the ROM) of the memory 302 from the external storage device 304 to the memory 302 (RAM). Load the OS. Then, in accordance with an instruction from the user, the apparatus functions as a video signal processing apparatus by loading an application program from the external storage device 304 into the memory 302. The state of the memory 302 when this application program is loaded into the memory 302 is shown in FIG.

  The memory 302 stores an OS for controlling the entire apparatus and operating various software, and an application program (video processing software) for detecting a caption portion of an image. Furthermore, image input software for controlling the video camera 306 to input (capture) moving images as digital data field by field and image display software for displaying images on the monitor 307 are stored. In addition, there is an image area for storing image data and a working area for storing various parameters.

  FIG. 13 is a flowchart showing video signal processing executed by the CPU 301.

  In step S1, each part is initialized.

  In step S2, it is determined whether or not an instruction to end the program of the I / P conversion process is given. This determination is made based on whether or not the user inputs an end instruction from the instruction input unit 303. If it is determined in step S2 that the program has not ended, the process proceeds to step S3. If it is determined that the program has ended, the process ends.

  In step S3, an image is input in field units, and the process proceeds to step S4. As described above, in step S3, moving image data is input from the communication interface 308, the external storage device 304, the storage device 305, or the like.

  In step S4, subtitles are detected, and the process proceeds to step S5. The caption detection process in step S4 will be described later.

  In step S5, the pixel is corrected based on the detection result in step S4, and the process returns to step S2.

  Here, the caption detection process in step S4 will be described with reference to the flowchart of FIG.

  In S400, the combing degree is calculated for each block in the frame generated by the weave of the current field and the previous field and the frame generated by the weave of the current field and the subsequent field. The method for calculating the combing degree is as described in the first embodiment.

  In S401, the weave pattern is determined for each block based on the combing degree calculated in S400. The determination procedure of the weave pattern is the same as FIG. 8 in the first embodiment and the description thereof.

  In S402, areas of blocks (sub-blocks) determined to have fewer patterns among the weave patterns determined in S401 are specified as caption area candidates. The procedure for specifying a caption area candidate is the same as that in FIG. 9 and the description thereof in the first embodiment.

  In S403, the subtitle position is accurately specified by, for example, luminance information.

  That is, in S403, the pixel of the interlaced image (caption) obtained from the progressive image at the second frame rate is synthesized based on the luminance information in the narrower one of the first region and the second region. Identify the location. Note that the first region is a region in which the interlaced image that is to form the progressive image in combination with the first composite image is the second composite image (previous field). The second area is an area in which the interlaced image that is to form the progressive image by combining the first synthesized image is the third synthesized image (the subsequent field). This process corresponds to the process performed by the caption detection unit 8 in the first embodiment.

  In step S5 in FIG. 13, the pixel is corrected. As described above, in step S5, the region detected as the caption portion is interpolated by, for example, motion applied IP conversion, and the region other than the caption portion is based on the determined weave pattern. To generate a frame in combination with other fields.

  That is, pixel information for constituting a frame in combination with the first composite image (current field) in the narrower region (sub-block region) of the first region and the second region is displayed in the front and rear. Generated using the frame data or the pixel information in the first composite image. On the other hand, as pixel information for constituting a frame in combination with the first composite image in the wider region (main block region), the second or third composite image (before or after) according to the determination of the weave pattern Field) pixel information is used. This processing corresponds to the processing of the motion detection I / P conversion unit 9, the weave I / P conversion unit 10, and the selector 11 in the first embodiment.

  Thus, in the second embodiment, the caption detection processing described in the first embodiment can be realized by software.

  The computer program is stored in a computer-readable storage medium such as a CD-ROM, and can be executed by setting it in a reading device (CD-ROM drive) of the computer and copying or installing it in the system. . Therefore, such a computer readable storage medium falls within the scope of the present invention.

<Other examples>
In the first and second embodiments, description has been given of performing different interpolation methods for the detected caption and other portions, but this is not restrictive. In other words, for example, subtitle erasure, sharpness, blur, noise reduction, enlargement, and reduction may be performed on the detected subtitle range in addition to switching the interpolation method.

  In the first and second embodiments, the case where the combing degree is calculated over the entire frame for each block has been described. However, the present invention is not limited to this. In other words, when the image of the pull-down pattern to be specified (for example, the subtitle portion) is displayed in advance such as being displayed on the lower side or the upper side of the display screen, The calculation may be omitted. In this way, it is possible to detect a pull-down pattern image to be specified with a smaller load.

It is a block block diagram of the video signal processing apparatus in 1st Embodiment. It is a figure for demonstrating the combing degree detection in 1st Embodiment. It is a figure for demonstrating the combing degree detection in 1st Embodiment. It is a figure for demonstrating the combing degree detection in 1st Embodiment. It is a figure for demonstrating the combing degree detection in 1st Embodiment. It is a figure for demonstrating the combing degree detection in 1st Embodiment. It is a figure for demonstrating the combing degree detection in 1st Embodiment. It is a flowchart which shows the process sequence for determining the weave pattern in 1st Embodiment. It is a flowchart for demonstrating the different pattern detection in 1st Embodiment. It is a figure for demonstrating the different pattern detection in 1st Embodiment. It is a block block diagram of the video signal processing apparatus in 2nd Embodiment. It is a figure which shows the storage condition of the memory in 2nd Embodiment. It is a flowchart which shows the process sequence of the video signal processing apparatus in 2nd Embodiment. It is a flowchart which shows the detection procedure of the caption in 2nd Embodiment. It is a figure for demonstrating 2-2 pulldown and 2-3 pulldown in embodiment of this invention. It is a figure for demonstrating I / P conversion in embodiment of this invention.

Explanation of symbols

4, 5 Combing degree calculation unit 6 Weave determination unit 7 Block detection unit 8 Subtitle detection unit 9 Motion application I / P conversion unit 10 Weave I / P conversion unit 11 Selector

Claims (8)

In the interlaced image at the third frame rate obtained from the progressive image at the first frame rate,
An interlaced image obtained from a progressive image at the second frame rate from a first synthesized image, which is an interlaced image obtained by synthesizing an interlaced image at a third frame rate obtained from a progressive image at the second frame rate. Is a video signal processing device that identifies the synthesized region,
A region in the first composite image, which is the interlaced image, is to form a progressive image by combining the first composite image and the second composite image input before the first composite image. Determining means for determining whether the region is a first region or a second region in which a progressive image is to be formed by combining the first composite image and the third composite image input after the first composite image;
When the first area and the second area exist in the first composite image, the progressive of the second frame rate is set according to the first area and the second area. A video signal processing apparatus comprising: specifying means for specifying an area in which an interlaced image obtained from an image is combined. 2. The video signal processing apparatus according to claim 1, wherein the progressive image at the second frame rate is a subtitle image. The specifying unit specifies a narrower one of the first region and the second region as a region in which an interlaced image obtained from a progressive image at the second frame rate is combined. 2. The video signal processing apparatus according to claim 1, wherein The determination means determines whether an interlaced image that should form a progressive image in combination with the first composite image is the second composite image or the third composite image, and the first composite image and the second composite image. Based on the difference between pixel values adjacent in the vertical direction in the progressive image combined with the composite image and the difference in pixel values adjacent in the vertical direction in the progressive image combined with the first composite image and the third composite image The video signal processing apparatus according to claim 1, wherein the video signal processing apparatus determines whether or not the video signal is processed. The specifying unit is configured to calculate a pixel obtained by synthesizing an interlaced image obtained from a progressive image at the second frame rate based on luminance information in a narrower one of the first region and the second region. 4. The video signal processing apparatus according to claim 1, wherein the position is specified. Of the first region and the second region,
Generating pixel information for constructing a progressive image in combination with the first composite image in the narrower region using data of frames displayed before and after or pixel information in the first composite image;
Interpolation means using pixel information of the second or third composite image according to the determination as pixel information for constructing a progressive image in combination with the first composite image in the wider region The video signal processing device according to claim 1, wherein the video signal processing device is a video signal processing device. In the interlaced image at the third frame rate obtained from the progressive image at the first frame rate,
An interlaced image obtained from a progressive image at the second frame rate from a first synthesized image, which is an interlaced image obtained by synthesizing an interlaced image at a third frame rate obtained from a progressive image at the second frame rate. A video signal processing method performed by a video signal processing device that identifies a region where
A region in the first composite image, which is the interlaced image, is to form a progressive image by combining the first composite image and the second composite image input before the first composite image. A determination step of determining whether the first region is a second region in which a progressive image is to be configured by combining the first composite image and the third composite image input after the first composite image;
When the first area and the second area exist in the first composite image, the progressive of the second frame rate is set according to the first area and the second area. And a specifying step of specifying a region where interlaced images obtained from the images are combined. In the interlaced image at the third frame rate obtained from the progressive image at the first frame rate,
An interlaced image obtained from a progressive image at the second frame rate from a first synthesized image, which is an interlaced image obtained by synthesizing an interlaced image at a third frame rate obtained from a progressive image at the second frame rate. To the computer that identifies the area where
A region in the first composite image, which is the interlaced image, is to form a progressive image by combining the first composite image and the second composite image input before the first composite image. A determination procedure for determining whether the region is a first region or a second region in which a progressive image is to be formed by combining the first composite image and the third composite image input after the first composite image;
When the first area and the second area exist in the first composite image, the progressive of the second frame rate is set according to the first area and the second area. A program for executing a specific procedure for specifying a region where an interlaced image obtained from an image is synthesized.

Images