JPH11239363A - Device for extracting character area in video image, its method and medium for recording its method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To extract a deteriorated color part in characters with high luminance and to suppress extraction of an area with a low contrast with respect to its surrounding areas in the case of extracting a character area from an image. SOLUTION: A plural frames averaging section 2 receives a plurality of color images and obtains a mean value of each of RGB components of pixels corresponding to each other between images to generate a mean color image. A similar luminance value area generating section 3 sets a plurality of adjacent pixels as a connection area whose luminance values are similar in the means value color image. A similar saturation value area generating section 4 sets a plurality of adjacent pixels as a connection area whose saturation values are similar in the connection area in which the luminance is also similar. A similar hue value area generating section 5 sets a plurality of adjacent pixels as a connection area whose hue values are similar in the connection area in which the luminance and the saturation area similar. A background area elimination section 6 eliminates a connection area not satisfying a character area characteristic with respect to the connection area where the luminance value, the saturation value and the hue value are similar to obtain a character area.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a plurality of frames forming a color moving image such as a video image of a television broadcast, for example, by connecting a character portion from a frame image in which characters such as telop characters are displayed to a pixel connection area. The present invention relates to a character area extraction technique for extracting a character region.

[0002]

2. Description of the Related Art With respect to a frame image in which characters in a plurality of frames constituting a color moving image are displayed or a still color image in which characters are displayed, a character portion is stably connected to a pixel with a small amount of calculation. A great deal of research has been conducted on character region extraction technology for extracting a region. Among them, reference 1 "H. Kuwa
no, S.M. Kurakake, K .; Odaka, "Te
lop Character Extraction
from Video Data ", Proc. of
IEEE International Works
hop on Document Image Ana
lysis, pp. 82-88 (1997-06) "
There is a character region extraction method proposed in US Pat.

In this method, a one-dimensional color space is divided into an input color image on which characters are displayed in the order of luminance, saturation, and hue in the image, thereby obtaining luminance, saturation, and hue. Are formed, adjacent pixel connection regions are formed in the image space, and a region that does not satisfy the character region feature is removed from the formed regions.

[0004] In the conventional method, a process of dividing a luminance space is performed on a luminance space within a local rectangular area in an image.
Since the image is divided using the threshold value obtained in the rectangular area, a good character area extraction result can be obtained even when there is local luminance variation in the image.

[0005]

However, in the above-mentioned conventional method, when the input character display color image is an image in a video in the NTSC signal format used in television broadcasting or the like, the character of color-degraded characters is not displayed. There is a problem that a character area in which the color deterioration portion is interrupted is extracted.

Normally, in a moving image of the NTSC signal format, the color of each pixel is blurred along each scanning line in the image, and the color of the background is changed at the right and left borders of the character in the image. There is a feature that the original color deteriorates by bleeding inside. The color degradation occurs only at the left and right edges of the horizontal component in the character, the central part is not affected, and the vertical component has a narrow character width, resulting in overall deterioration and lowering of the luminance. And the luminance contrast of the vertical component may increase (see FIGS. 12 and 13). For this reason, in the above-described conventional method, when a threshold value is determined in a rectangular area including a connection portion between a horizontal component and a vertical component of a character whose color in a moving image in the NTSC signal format has deteriorated, the vertical component whose color has deteriorated is , And as a result, the character area is cut off and extracted (see FIG. 14).

FIG. 12 is a diagram showing an example of color deterioration occurring in characters displayed in an NTSC format video. Acoustic Echo Canceller, Acoustic with Black White Background
c Echo Canceller ", and the vertical component in the character is degraded to gray.

FIG. 13 is a schematic diagram of color deterioration in a character.
FIG. 9 is a schematic diagram of color deterioration in which a black background color is blurred inside a telop character called a white character “t” in a black background, and the color inside the character is partially changed to gray. Regarding the horizontal component in the character, color deterioration occurs only on the left and right ends, and the central part is not affected (region A in FIG. 13). As for the vertical component, since the character width is narrow, the whole is deteriorated and the luminance is reduced (region B in FIG. 13), and the luminance contrast between the horizontal component and the vertical component in the character is increased. In such a case, according to the conventional binarization method [Reference Document 1], the area B may be regarded as the background.

FIG. 14 shows a conventional method [Ref. 1] and FIG.
This is the result of extracting the character area from the color image No. 2 and the result is that the color deteriorated portion of each character is interrupted.

[0010] In general, characters such as telop characters displayed in an image have a feature that the color contrast with surrounding portions is very high. In the above-described conventional method, the formation of the pixel connection region is performed only by the color space division process, and the characteristics of the color distribution in the image space are not taken into account. There is also a problem in that extraction is performed, and many areas other than characters are extracted.

Therefore, the present invention provides a character region extracting device for an image which can extract a color-degraded portion in a high-luminance character when extracting a character region from an image and can suppress extraction of a region having low contrast with the surroundings. It is an object to provide a method.

[0012]

In order to solve the above-mentioned problems, a character region extracting apparatus for a video according to the present invention uses a predetermined method from a plurality of frame images in a color moving image. And a plurality of color images in which the same characters as the characters in the frame image are displayed among the frame images displayed at the time after the frame image and the first frame image. Means for generating an average color image having average values of luminance, saturation, and hue of pixels corresponding to the positions of the plurality of images input by the first means as values of the pixels; A third means for forming a plurality of pixels having similar luminance values and adjacent pixels in an image space as a connected area with respect to the average color image obtained by the second means; Brightness obtained by means For a connected area image having similar values, a fourth means for forming a plurality of pixels having similar saturation values in each area and forming a plurality of adjacent pixels in an image space as a connected area, and a fourth means for obtaining a fourth area. Hue values are similar in each area for connected area images with similar luminance and saturation values,
A fifth unit for forming a plurality of pixels adjacent in the pixel space as a connected region, and a luminance value obtained by the fifth unit.
For the connected area in the image where both the saturation value and the hue value are similar,
Sixth means for removing a connected area that does not satisfy the character area feature, seventh means for accumulating the connected area image obtained by the sixth means as a character area extraction result, and first to seventh means Eighth means for controlling the execution order.

The third means extracts a temporary character area by binarizing the luminance within each horizontal line in the image and integrating the binarization results for each horizontal line. A ninth means, a tenth means for performing a labeling process for giving a serial number as a label to each area in the temporary character area image obtained by the ninth means, and a tenth means obtained by the tenth means An eleventh means for selecting a character area pixel in the temporary character area by binarizing the luminance distribution in the vertical direction in the temporary character area to which the same label is assigned in the label image; And a twelfth means for controlling the execution order of the eleventh means.

Further, the ninth means is to examine a luminance distribution in a certain horizontal line in an image, and to locally compare the luminance value in the horizontal line with a peripheral portion by a predetermined value. And the absolute value of the luminance gradient in the horizontal direction at both the left and right ends of the connected pixel area obtained by the thirteenth means on a certain horizontal line in the image. If both are larger than a preset value, the connected pixel area is set as a temporary high-brightness character area.
And means for examining the luminance distribution in a certain horizontal line in the image, and extracting a connected pixel region in a range where the luminance value is locally lower than the peripheral portion by a predetermined value in the horizontal line. Means on a horizontal line in the image,
If the absolute value of the horizontal luminance gradient at both the left and right ends of the connected pixel region obtained by the means in 5 is larger than a preset value at both the left and right ends, the connected pixel region is set as a temporary low-luminance character region. This is realized by using a sixteenth means and a seventeenth means for controlling the execution order of the thirteenth to sixteenth means.

[0015] Further, by binarizing the luminance distribution in the vertical direction in the provisional character area to which the same label in the label image is assigned, the character area pixels in the provisional character area are determined.
Means is calculated for each horizontal line in the same label area in the image, from the left and right ends of the area by a predetermined pixel width, excluding pixels inside the area by a predetermined pixel width. In the eighteenth means, the vertical distribution of the average luminance value for each horizontal line obtained by the eighteenth means in the same label area in the image is examined, and the horizontal lines are locally detected in the same label area. A nineteenth means for extracting an area on a plurality of connected horizontal lines in which the average luminance value for each is higher than a peripheral part by a predetermined value, and a nineteenth means in the same label area in the image When the vertical gradient value of the average luminance value for each horizontal line obtained by the eighteenth means at the upper and lower ends of the region for each of the plurality of connected horizontal lines is larger than a preset value at both the upper and lower ends. , The connected horizontal line area In the twentieth means for determining the area, and within the same label area in the image, the vertical distribution of the average luminance value for each horizontal line obtained by the eighteenth means is examined. A twenty-first means for extracting regions on a plurality of connected horizontal lines in which the average luminance value for each horizontal line is lower than a peripheral portion by a predetermined value; The vertical gradient value of the luminance average value for each horizontal line obtained by the eighteenth means at the upper and lower ends of the region on the plurality of horizontal lines to be connected obtained by the means is higher than the preset value at both the upper and lower ends. Is larger, a twenty-second means for determining the connected horizontal line area as a low-luminance character area and a twenty-third means for controlling the execution order of the eighteenth to twenty-second means are realized. Features.

Similarly, in order to solve the above-mentioned problem, a method for extracting a character area in an image according to the present invention comprises displaying a character from a plurality of frame images in a color moving image by using a predetermined method. A first procedure of inputting and storing a plurality of frame images and a plurality of color images in which characters identical to the characters in the frame images are displayed among frame images at times later than the frame images; and A second procedure of creating an average color image having average values of luminance, saturation, and hue of pixels corresponding to positions among a plurality of images input by the first procedure as values of respective pixels; A third procedure in which a plurality of pixels having similar luminance values and adjacent in the image space are formed as a connected area with respect to the average color image obtained in the second procedure, and a third procedure obtained by the third procedure Brightness value And a luminance value obtained by the fourth procedure in which a plurality of pixels having similar saturation values in each area and forming a plurality of adjacent pixels in the image space as a connected area are obtained. A fifth procedure of forming a plurality of pixels having similar hue values in each area and adjacent pixels in a pixel space as a connected area with respect to a connected area image having similar saturation values. A sixth procedure for removing a connected area that does not satisfy a character area feature from a connected area in an image having similar luminance values, saturation values, and hue values, and a connected area image obtained by the sixth procedure And accumulating as a character region extraction result.

In the third procedure, a temporary character area is extracted by binarizing the luminance within each horizontal line in the image and integrating the binarization results of each horizontal line. A ninth procedure, a tenth procedure of performing a labeling process of assigning a serial number as a label to each area in the temporary character area image obtained by the ninth procedure, and a tenth procedure. An eleventh step of selecting a character area pixel in the temporary character area by binarizing the luminance distribution in the vertical direction in the temporary character area to which the same label is assigned in the label image. It is characterized by.

In the ninth procedure, the luminance distribution in a certain horizontal line in the image is examined, and the luminance value is locally increased in the horizontal line by a predetermined value compared with the peripheral portion. And the absolute value of the luminance gradient in the horizontal direction at both the left and right ends of the connected pixel area obtained by the thirteenth procedure on a certain horizontal line in the image If both are larger than a preset value, the connected pixel area is set as a temporary high-brightness character area.
And the luminance distribution in a certain horizontal line in the image is examined, and a range in which the luminance value is locally lower in the horizontal line by a predetermined value than the peripheral portion is extracted as a fifteenth step. Procedure, and on a horizontal line in the image, the first
If the absolute value of the horizontal luminance gradient at both the left and right ends of the connected pixel region obtained by the procedure of 5 is larger than a preset value at both the left and right ends, the connected pixel region is set as a temporary low-luminance character region. And a sixteenth procedure.

Further, the eleventh procedure is performed for each horizontal line in the same label area in the image within a range excluding pixels inside the area by a preset pixel width from both left and right ends of the area. An eighteenth procedure for calculating the average value of the luminance values, and in the same label area in the image, the vertical distribution of the average luminance value for each horizontal line obtained in the eighteenth procedure is examined, and the same label area is determined. A nineteenth procedure for extracting a region on a plurality of connected horizontal lines in which the average luminance value of each horizontal line is locally higher by a predetermined value compared with the peripheral portion, and The vertical gradient value of the average luminance value for each horizontal line determined in the eighteenth step at the upper and lower ends of the region for each of the plurality of connected horizontal lines determined in the nineteenth step is determined in advance for both upper and lower ends. If it is larger than the set value, A twentieth procedure for determining the line area as a high-brightness character area, and, within the same label area in the image, examining the vertical distribution of the average luminance value for each horizontal line obtained in the eighteenth procedure, A twenty-first procedure for extracting an area on a plurality of connected horizontal lines in which the average luminance value of each horizontal line is locally lower than a peripheral part by a predetermined value in the label area, and the same label in the image Within the region, the vertical gradient values of the average luminance value for each horizontal line determined in the eighteenth step at the upper and lower ends of the region on the plurality of connected horizontal lines determined in the twenty-first step are the upper and lower ends. When both are larger than a preset value, the connection horizontal line area is determined as a low-luminance character area, and a twenty-second procedure is performed.

Further, the procedure in the above-described method of extracting a character region in a video is recorded on a computer-readable recording medium as a program to be executed by a computer.

As described above, in a color image in an NTSC format video, color bleeding occurs in the horizontal line direction, and color degradation also occurs in a character portion such as a telop. However, when only the luminance distribution on one horizontal line is examined even when the color is deteriorated, the change in the luminance in the character is small, and the contrast between the character and the background is often high.
FIG. 10 is a diagram illustrating an example of a luminance distribution on a certain scanning line in a color-deteriorated character. The luminance distribution on a horizontal line where color deterioration occurs in the character includes a high-contrast part in the character. In other words, it shows that the color degraded portion also has a feature that there is a sufficient difference in luminance from the surrounding background portion. Therefore, in the present invention, by setting a connected pixel having a locally high brightness value or a locally connected pixel having a low brightness value in a horizontal line unit in an image as a character area, high brightness which is difficult with the conventional method is obtained. It is possible to extract a color deteriorated portion in a character. Further, by determining the character area only when the absolute value of the luminance gradient at the area boundary is large, it is possible to suppress the extraction of an area having low contrast with the surrounding area, which is a problem of the conventional method. I do. FIG. 11 shows the result of character region extraction from the image of FIG.
This is shown in comparison with the result of extraction according to the prior art.

[0022]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a block diagram showing the configuration and processing flow of an apparatus according to an embodiment of the present invention.

In FIG. 1, reference numeral 1 designates a color moving image input storage means for inputting a plurality of designated color image data in which characters to be processed are displayed and storing them in a memory.

Reference numeral 2 denotes a plurality of frame averaging unit, which has an average value of RGB components of pixels corresponding to positions among a plurality of images input by the color moving image input storage means 1 as a value of each pixel. Create a value color image.

Reference numeral 3 denotes a luminance value similar region forming unit which uses a plurality of pixels having similar luminance values in the average color image obtained by the plural frame averaging unit 2 and adjacent pixels in the image space as a connection region. Form.

Reference numeral 4 denotes a saturation value similar area forming unit which has similar saturation values in the respective regions in the connected area image having similar luminance values obtained by the luminance value similar area forming unit 3, To form a plurality of adjacent pixels as a connection region.

Reference numeral 5 denotes a hue value similar region forming unit which has similar hue values in each region with respect to the connected region image obtained by the saturation value similar region forming unit 4 and having similar luminance values and saturation values. A plurality of pixels adjacent in the pixel space are formed as a connection region.

Reference numeral 6 denotes a background region removing unit which connects a connected region in the image having similar luminance values, saturation values, and hue values obtained by the hue value similar region forming unit 5 without satisfying the character region characteristics. Remove the area.

Reference numeral 7 denotes a character area image storage unit, which stores the character area extraction result image obtained by the background area removal unit 6.

Reference numeral 8 denotes a processing control unit which controls the execution of each of the above-mentioned units 1 to 7.

FIG. 2 is a diagram showing a luminance value similar area forming unit 3 in FIG.
FIG. 3 is a block diagram showing an example of a processing flow together with an example of the configuration of FIG.

In FIG. 2, reference numeral 9 denotes a horizontal line unit binarizing unit, which binarizes the luminance within each horizontal line in the image, and integrates the binary results for each horizontal line to create a provisional image. Extract the character area.

Reference numeral 10 denotes a binarization result labeling unit, which performs a labeling process for assigning a serial number as a label to each area in the provisional character area image obtained by the horizontal line unit binarization unit 9.

Reference numeral 11 denotes a luminance-similar character area determining unit which binarizes the luminance distribution in the vertical direction in a temporary character area to which the same label is added in the label image obtained by the binarization result labeling unit 10. By doing so, the character area pixels in the temporary character area are determined.

Reference numeral 12 denotes a processing control unit, which controls the execution of each of the units 9 to 11 described above. Also, 2, 3, and 4 are the same as the same-numbered portions in FIG.

FIG. 3 is a block diagram showing an example of the processing flow along with an example of the configuration of the horizontal line unit binarizing section 9 in FIG.

In FIG. 3, reference numeral 13 denotes a luminance maximum value area detection unit on a horizontal line, which examines the luminance distribution in a certain horizontal line in the image, and determines the luminance value locally within the horizontal line by a preset value. A high connected pixel area is extracted.

Reference numeral 14 denotes a provisional high-luminance character area extracting unit which outputs a horizontal luminance at both left and right ends of a connected pixel area obtained by a horizontal-line maximum luminance area detecting unit 13 on a certain horizontal line in the image. If the absolute value of the gradient is greater than a preset value at both the left and right ends, the connected pixel area is set as a temporary high-luminance character area.

Numeral 15 is a luminance minimum value area detecting unit on the horizontal line, which examines the luminance distribution in a certain horizontal line in the image.
A connected pixel region whose luminance value is locally lower by a preset value is extracted locally in the horizontal line.

Reference numeral 16 denotes a provisional low-luminance character area extraction unit which outputs a horizontal luminance at both left and right ends of a connected pixel area obtained by a horizontal minimum luminance area detection unit 15 on a certain horizontal line in the image. If the absolute value of the gradient is larger than the preset value at both the left and right ends, the connected pixel area is set as a temporary low-luminance character area.

Reference numeral 17 denotes a processing control unit, which controls execution in the processing units 13 to 16.

FIG. 4 is a block diagram showing an example of the flow of processing along with an example of the configuration of the luminance similar character area determination section 11 in FIG.

In FIG. 4, reference numeral 18 denotes a horizontal line luminance average value calculation unit, and for each horizontal line in the same label area in the image, pixels inside a predetermined pixel width from the left and right ends of the area are set. The average of the luminance values is calculated in a range excluding.

Reference numeral 19 denotes a vertical luminance maximum horizontal line area detecting unit which checks the vertical distribution of the average luminance value on each horizontal line obtained by the horizontal line average luminance value calculating unit 18.
A region on a plurality of connected horizontal lines whose luminance value is locally higher by a preset value is extracted in the region.

Reference numeral 20 denotes a high-brightness character area determination unit, which is the horizontal line brightness at the upper and lower ends of a plurality of connected horizontal lines in the same label area obtained by the vertical brightness maximum horizontal line area detection unit 19. When the vertical gradient value of the luminance average value for each horizontal line obtained by the average calculation unit 18 is larger than a preset value at both upper and lower ends, the connected horizontal line area is determined as a high luminance character area.

Numeral 21 denotes a vertical direction minimum luminance horizontal line area detecting section for examining the vertical distribution of the average luminance value on each horizontal line obtained by the horizontal line average luminance value calculating section 18.
A region on a plurality of connected horizontal lines whose luminance value is locally lower by a preset value is extracted in the region.

Reference numeral 22 denotes a low-luminance character area determination unit, which is the horizontal line luminance at the upper and lower ends of a plurality of connected horizontal lines in the same label area obtained by the vertical luminance minimum horizontal line area detection unit 19. If the vertical gradient value of the luminance average value for each horizontal line obtained by the average calculation unit 18 is greater than a preset value at both upper and lower ends, the connected horizontal line area is determined as a low luminance character area.

Reference numeral 23 denotes a processing control unit which controls the execution of each of the units 18 to 22.

FIG. 5 is a flowchart for explaining an embodiment of the processing of the horizontal line unit luminance binarizing section 9 in FIG.

In FIG. 5, step (30) is a process of reading the luminance value image into the memory.

Step (31) is a process of dividing the luminance image read into the memory in step (30) into horizontal lines.

Step (32) is a step of judging whether or not there is an unprocessed horizontal line in the luminance similar character area determination processing. If there is an unprocessed horizontal line, the process proceeds to step (33). If there is no unprocessed horizontal line, the process ends.

In a step (33), a luminance distribution is examined for each horizontal line in the image obtained in the step (31).
This is a process of obtaining all the pixels having the maximum value.

In a step (34), a luminance distribution is checked for each horizontal line in the image obtained in the step (31).
This is a process of obtaining all the pixels having the minimum value.

Step (35) is a step of judging whether or not there is an unprocessed maximum pixel in the provisional high-brightness character region extraction processing among the maximum pixels obtained in step (33). If there is an unprocessed local maximum pixel, the process proceeds to step (36). If there is no unprocessed local maximum pixel, the process proceeds to step (39).

In step (36), starting from the maximum value pixel obtained in step (33) as a starting point, the horizontal line is scanned one pixel at a time in the left and right directions, and left and right sides of the maximum value pixel. Detects the first pixel whose luminance value is lower than the maximum value by a preset threshold value, and extracts the two detected pixels and the pixel existing between the two pixels on the horizontal line as a connected pixel area. It is a process.

In step (37), the absolute value of the luminance gradient in the horizontal direction at both the left and right ends of the connected pixel area on the horizontal line obtained in step (36) is larger than the preset threshold value in both the left and right directions. The process proceeds to step (38) if it is determined to be large, and to step (35) if not.

Step (38) is a process in which the connected pixel area on the horizontal line obtained in step (36) is extracted as a temporary high-luminance character area.
Move to 5).

FIG. 6 shows steps (36) and (3).
7) is a diagram for explaining step (38), and shows whether or not a temporary high-luminance character region is extracted for the maximum luminance value pixels 1 to 4 on a certain horizontal line. The maximum luminance value pixels 1 and 3 are extracted as a provisional high luminance character area because both the luminance difference and the luminance gradient from the left and right pixels satisfy the conditions, but the maximum luminance value pixels 2 and 4 do not satisfy the conditions. Not extracted.

Step (39) is a step of judging whether or not there is an unprocessed minimum-value pixel in the provisional low-luminance character region extraction processing among the minimum-value pixels obtained in step (34). If there is an unprocessed minimal value pixel, the process proceeds to step (40). If there is no unprocessed minimal value pixel, the process proceeds to step (32).

In step (40), starting from the minimum value pixel obtained in step (34) as a starting point, the horizontal line is scanned left and right one pixel at a time, and the left and right sides of the minimum value pixel are scanned. Detects the first pixel having a luminance value higher than the minimum value by a preset threshold value, and extracts the detected two pixels and the pixel existing between the two pixels on the horizontal line as a connected pixel area. It is a process.

In step (41), the absolute value of the luminance gradient in the horizontal direction at both the left and right ends of the connected pixel area on the horizontal line obtained in step (40) is larger than the preset threshold value on both the left and right sides. The process proceeds to step (42) if it is determined to be large, and to step (39) if it is not large.

Step (42) is a process of extracting the connected pixel area on the horizontal line obtained in step (40) as a temporary low-luminance character area.
Move to 9).

FIG. 7 is a flowchart for explaining an embodiment of the processing of the luminance similar character area determination section 11 in FIG.

In FIG. 7, step (51) is a process of reading the luminance value image and the temporary high luminance character area image into the memory.

Step (52) is a step of judging whether or not there is an unprocessed provisional high-luminance area in the high-luminance character area determination processing. If there is an unprocessed provisional high-luminance area, step (53) is performed. Then, if there is no unprocessed provisional high-luminance area, go to step (59).

The step (53) is performed for each horizontal line in the same label area in the provisional high-brightness character area image within a range excluding pixels inside by a preset pixel width from both left and right ends of the area. This is the process of calculating the average of the brightness values.

In the step (54), the average value of the luminance values for each horizontal line obtained in the step (53) in the same label area in the image is examined in the vertical direction, and the horizontal value having the maximum luminance value is determined. This is the process of detecting all the lines.

Step (55) is a step of judging whether or not there is an unprocessed maximum luminance horizontal line in the high luminance character area determination processing. If there is an unprocessed maximum luminance horizontal line, step (56) is performed. If there is no unprocessed maximum luminance horizontal line, the process proceeds to step (52).

In step (56), the maximum brightness horizontal line obtained in step (54) is used as a start line, and the same label area is referred to one horizontal line at a time in the upward and downward directions. , The first horizontal line having a lower luminance average value by a preset threshold value than the luminance average value of the start line on the lower side is detected, and the pixels on the two detected horizontal lines and the two pixels within the same label area are detected.
In this process, pixels on a horizontal line existing between two horizontal lines are integrated and extracted as a connected pixel region. At this time, if the horizontal line at the upper end or the lower end of the same label area is reached before two horizontal lines are detected, the horizontal line is detected as the upper or lower end of the connected pixel area.

Step (57) is a step of judging whether the absolute value of the vertical luminance gradient value is greater than a predetermined threshold value at the upper and lower ends of the connected pixel area extracted in step (56), If it is determined that it is larger, the process proceeds to step (58). If it is determined that it is not larger, the process proceeds to step (55).

In step (58), if the absolute value of the vertical luminance gradient at both the upper and lower ends obtained in step (57) is larger than a predetermined threshold value in both the upper and lower directions, the connected pixel area is set to a high-luminance character. This is the process of determining the area.

FIGS. 8 and 9 show steps (53) and (5).
FIG. 8 is a diagram for explaining (6), (57), and (58). FIG. 8 uses an example in which white “snow” is displayed on a black background with color deterioration at both left and right ends and displayed. Obtaining the average value of the luminance in a range excluding the pixels at both ends for each horizontal line indicates that a value excluding the color deterioration portion can be obtained. FIG.
In this example, an example in which a noise region is integrated on the character “den” will be used. The maximum value of the average luminance value of each horizontal line is detected from the distribution in the vertical direction, and when a horizontal line in which the luminance value decreases by a certain value in the upper and lower parts is detected, the character "den" can be separated from the noise region.

Step (59) is a process of reading the temporary low-luminance character area image into the memory.

Step (60) is a step of judging whether or not there is an unprocessed temporary low-luminance area in the low-luminance character area determination processing. If there is an unprocessed temporary low-luminance area, step (61) is performed. Then, if there is no unprocessed provisional high luminance area, the processing of the luminance similar character area determination unit is terminated.

The step (61) is performed for each horizontal line in the same label area in the provisional low-luminance character area image in a range excluding pixels inside by a predetermined pixel width from both left and right ends of the area. This is the process of calculating the average of the brightness values.

In the step (62), the average value of the luminance values for each horizontal line obtained in the step (61) in the same label area in the image is examined in the vertical direction, and the horizontal value having the minimum luminance value is determined. This is the process of detecting all the lines.

Step (63) is a step of judging whether or not there is an unprocessed minimal luminance horizontal line in the low luminance character area determination processing. If there is an unprocessed minimal luminance horizontal line, the process proceeds to step (64). If there is no unprocessed minimum luminance horizontal line, the process proceeds to step (60).

In the step (64), the minimum luminance horizontal line obtained in the step (62) is set as a start line, and the same label area is referred to one horizontal line at a time in an upward direction and a downward direction. , The first horizontal line having a luminance average value higher than the luminance average value of the start line by a preset threshold value on the lower side is detected, and the pixels on the two detected horizontal lines and the two pixels within the same label area are detected.
In this process, pixels on a horizontal line existing between two horizontal lines are integrated and extracted as a connected pixel region. At this time, if the horizontal line at the upper end or the lower end of the same label area is reached before two horizontal lines are detected, the horizontal line is detected as the upper or lower end of the connected pixel area.

Step (65) is a step of judging whether the absolute value of the vertical luminance gradient value is greater than a preset threshold value at the upper and lower ends of the connected pixel area extracted in step (64), If it is determined that it is larger, the process proceeds to step (66), and if it is determined that it is not larger, the process proceeds to step (63).

In step (66), when the absolute value of the vertical luminance gradient at both the upper and lower ends obtained in step (65) is larger than a predetermined threshold value in both the upper and lower parts, the connected pixel area is set to a low-luminance character. This is the process of determining the area.

The present invention relates to a reading device for reading a recording medium, a memory for storing a color image to be recognized as a character, a program read from a recording medium, an extracted character area image, and the like, and capable of freely reading them. A device and a buffer for holding data necessary for performing various processes and devices similar thereto, and for displaying necessary information and monitoring color images, character recognition results, etc. in the process of the process. An output device such as a display, and input devices such as a keyboard and a mouse for giving necessary instructions are provided, and the memory device, buffer, output device, and input device are arranged in accordance with a procedure predetermined by the program. A computer or a device equivalent thereto controls the processing in each unit of the embodiment of the present invention described with reference to FIGS. The order or algorithm can be executed as appropriate, and a program for causing a computer or the like to execute the procedure or algorithm can be read by a recording medium such as a floppy disk, memory card, MO, CD, or DVD. It is possible to record it and distribute it.

[0084]

As described above, according to the present invention,
By setting a connected pixel having a high or low luminance value locally as a character region in units of horizontal lines in an image, it is possible to extract a region of a color-degraded character. Further, at this time, since the region where the absolute value of the luminance gradient at the region boundary is high is extracted as the character region, the effect of suppressing the excessive extraction of the surrounding region and the region with low contrast can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration and a processing flow of an apparatus for extracting a character region in a video according to an embodiment of the present invention.

FIG. 2 is a block diagram showing an example of a processing flow together with an example of a configuration of a luminance value similar area forming unit 3 in FIG. 1;

FIG. 3 is a block diagram showing an example of a processing flow together with an example of a configuration of a horizontal line unit binarizing section 9 in FIG. 2;

4 is a block diagram showing an example of a processing flow together with an example of a configuration of a luminance similar character area determination unit 11 in FIG. 2;

FIG. 5 is a flowchart showing an example of processing contents of a horizontal line unit binarization unit 9 in FIG. 2;

FIG. 6 is a capture diagram for explaining an example of a process performed by the horizontal line unit binarization unit 9 in FIG. 2;

FIG. 7 is a flowchart showing an example of processing contents of a luminance similar character area determination unit 11 in FIG. 2;

FIG. 8 is a capture diagram (part 1) for explaining an example of processing performed by the luminance similar character area determination unit 11 in FIG. 2;

FIG. 9 is a capture diagram (part 2) for explaining an example of a process performed by the luminance similar character area determination unit 11 in FIG. 2;

FIG. 10 is a diagram illustrating an example of a luminance distribution on a certain horizontal line in a color-deteriorated character.

FIG. 11 is an image showing a result of extracting a character area from the color-degraded image of FIG. 12 according to the present invention.

FIG. 12 is an image showing an example of color degradation of characters in a video in NTSC format.

FIG. 13 is a schematic diagram of a color-deteriorated character.

14 is an image showing a result of extracting a character area from the color-degraded image of FIG. 12 by a conventional method.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Color moving image input storage means 2 ... Multiple frame averaging unit 3 ... Luminance value similar region forming unit 4 ... Saturation value similar region forming unit 5 ... Hue value similar region forming unit 6 ... Background region removing unit 7 ... Character region Image storage unit 8 Processing control unit 9 Horizontal line unit binarization unit 10 Binarization result labeling unit 11 Luminance similar character area determination unit 12 Processing control unit 13 Maximum luminance on-horizontal line area detection unit 14 ... Temporary high-brightness character region extracting unit 15... Luminance on-horizontal-minimum value region detecting unit 16... Temporary low-luminance character region extracting unit 17... Processing control unit 18... Horizontal-line luminance average calculating unit 19. Area detecting section 20: High luminance character area determining section 21: Vertical luminance minimum horizontal line area detecting section 22: Low luminance character area determining section 23: Processing control section

 ──────────────────────────────────────────────────の Continued on the front page (72) Inventor Kenji Ogura 3-19-2 Nishi-Shinjuku, Shinjuku-ku, Tokyo Nippon Telegraph and Telephone Corporation

Claims (9)

[Claims] 1. A frame image in which characters are displayed using a predetermined method from a plurality of frame images in a color moving image, and a frame image at a time later than the frame image. First means for inputting and storing a plurality of color images in which the same characters as the characters in the image are displayed, and brightness between pixels corresponding to positions in the plurality of images input by the first means Means for creating an average color image having an average value of each pixel as a value of each pixel for each of the saturation, the hue, and the average color image obtained by the second means.
Third means for forming a plurality of pixels having similar luminance values and adjacent to each other in the image space as connected areas; and connecting area images having similar luminance values obtained by the third means, in each area. A fourth means for forming a plurality of pixels adjacent to each other in an image space as a connected area, and a connected area image having similar luminance and saturation values obtained by the fourth means. A fifth means for forming a plurality of pixels having similar hue values in each area and adjacent in a pixel space as a connected area; and a luminance value, a saturation value, and a hue value obtained by the fifth means. A sixth means for removing a connected area that does not satisfy the character area feature from a connected area in an image having similar values, and a seventh means for storing the connected area image obtained by the sixth means as a character area extraction result. Means for controlling the execution order of the first to seventh means That the eighth means, the image in the character area extraction apparatus characterized by having a. 2. The apparatus according to claim 1, further comprising a third unit that forms a plurality of pixels having similar luminance values in the color image and adjacent to each other in the image space as a connected region. Ninth means for performing binarization of luminance within each horizontal line in the image and extracting a temporary character area by integrating the binarization results for each horizontal line, and ninth means A tenth means for performing a labeling process for assigning a serial number as a label to each area in the provisional character area image, and a provisional character to which the same label is provided in the label image obtained by the tenth means. Eleventh means for selecting a character area pixel in a temporary character area by binarizing a vertical luminance distribution in the area, and twelfth means for controlling the execution order of the ninth to eleventh means And is realized by using A device for extracting character areas in video. 3. The image character area extracting apparatus according to claim 2, wherein ninth means for binarizing the luminance within each horizontal line in the image and extracting a temporary character area is provided. A thirteenth means for examining a luminance distribution in a certain horizontal line and extracting a connected pixel region in a range where the luminance value is locally higher than a peripheral portion by a predetermined value in the horizontal line, and If the absolute value of the horizontal luminance gradient at both left and right ends of the connected pixel region obtained by the thirteenth means on a certain horizontal line is larger than a preset value at both the left and right ends, the connected pixel region is temporarily set. A fourteenth means for making a high-luminance character area, and a luminance distribution within a certain horizontal line in the image is examined, and a luminance value is locally reduced within the horizontal line by a preset value compared with the peripheral portion. 15th means for extracting a connected pixel area If the absolute value of the horizontal luminance gradient at both the left and right ends of the connected pixel area obtained by the fifteenth means on a certain horizontal line in the image is larger than a preset value at both the left and right ends, the connection is made. 16. A character area in a video, which is realized by using: a sixteenth means for setting a pixel area as a provisional low-luminance character area; and a seventeenth means for controlling the execution order of the thirteenth to sixteenth means. Extraction device. 4. The apparatus according to claim 2, wherein the luminance distribution in the vertical direction is binarized in the temporary character area to which the same label in the label image is assigned. The eleventh means for determining a character area pixel is obtained by removing pixels inside the area by a preset pixel width from the left and right ends of the area for each horizontal line in the same label area in the image. An eighteenth means for calculating the average value of the luminance values in the range; and, in the same label area in the image, a vertical distribution of the average luminance value for each horizontal line obtained by the eighteenth means is examined. A nineteenth means for extracting a region on a plurality of connected horizontal lines in which the average luminance value of each horizontal line is locally higher than a peripheral portion by a predetermined value in the label region, and the same label in the image Within the area, the connection determined by the nineteenth means When the vertical gradient value of the average luminance value for each horizontal line obtained by the eighteenth means at both the upper and lower ends of the region for each of the plurality of horizontal lines is greater than the preset value at both the upper and lower ends, the connection horizontal A twentieth means for determining the line area as a high-brightness character area; and, in the same label area in the image, examining the vertical distribution of the average luminance value for each horizontal line obtained by the eighteenth means, A twenty-first means for extracting an area on a plurality of connected horizontal lines in which the average luminance value of each horizontal line is locally lower than a peripheral part by a preset value in the label area, and the same label in the image In the area, the vertical gradient value of the average luminance value for each horizontal line obtained by the eighteenth means at the upper and lower ends of the area on the plurality of connected horizontal lines obtained by the twenty-first means is equal to the upper and lower ends. And a preset value If the second horizontal line area is larger than the second horizontal line area, a twenty-second means for determining the connected horizontal line area as a low-luminance character area, and a twenty-third means for controlling the execution order of the eighteenth to twenty-second means are realized. A character region extracting device in a video characterized by the following. 5. A frame image in which characters are displayed by using a predetermined method from a plurality of frame images in a color moving image, and a frame image of a frame image at a time later than the frame image. A first procedure of inputting and storing a plurality of color images in which the same characters as the characters in the image are displayed, and a luminance of pixels corresponding to positions in the plurality of images input in the first procedure , A second procedure for creating an average color image having the average value of each of the saturation and the hue as the value of each pixel, and the average color image obtained by the second procedure,
A third procedure in which a plurality of pixels having similar luminance values and adjacent in the image space are formed as connected areas; and a connected area image in which the luminance values obtained in the third procedure are similar to each other, A fourth procedure of forming a plurality of pixels adjacent to each other in the image space as a connected area, and a connected area image having similar luminance and saturation values obtained by the fourth procedure. In contrast, a fifth procedure of forming a plurality of pixels having similar hue values in each area and adjacent in a pixel space as a connected area, and a luminance value, a saturation value, and a hue obtained by the fifth procedure. A sixth procedure for removing a connected area that does not satisfy the character area feature from a connected area in an image having similar values, and a seventh procedure for storing the connected area image obtained by the sixth procedure as a character area extraction result. And a video sentence comprising: Area extraction method. 6. The method according to claim 5, wherein a third step of forming a plurality of pixels having similar luminance values in a color image and adjacent to each other in an image space as a connected region is performed. A ninth procedure for extracting a temporary character area by binarizing the luminance within each horizontal line in the image and integrating the binarization results for each horizontal line, and a ninth procedure A tenth procedure of performing a labeling process of assigning a serial number as a label to each area in the temporary character area image, and a temporary character having the same label in the label image obtained by the tenth procedure. An eleventh procedure of selecting a character area pixel in a temporary character area by binarizing a luminance distribution in a vertical direction in the area, and a method of extracting a character area in a video image. 7. A method according to claim 6, further comprising the step of: performing binarization of luminance within each horizontal line in the image to extract a temporary character region; A thirteenth procedure of examining a luminance distribution in a certain horizontal line, extracting a connected pixel area in a range where the luminance value is locally higher than a peripheral part by a predetermined value in the horizontal line, and If the absolute value of the horizontal luminance gradient at both the left and right ends of the connected pixel region obtained by the thirteenth procedure on a certain horizontal line is larger than the preset value at both the left and right ends, the connected pixel region is temporarily set. A fourteenth procedure for determining a high-luminance character area, and a luminance distribution in a certain horizontal line in the image is examined, and a luminance value is locally reduced in the horizontal line by a predetermined value in comparison with the peripheral portion. 15th procedure for extracting a connected pixel area If the absolute value of the horizontal luminance gradient at both the left and right ends of the connected pixel area obtained by the fifteenth procedure on a certain horizontal line in the image is larger than a preset value at both the left and right ends, the connection is performed. A sixteenth procedure in which a pixel area is set as a provisional low-luminance character area, and the following steps are performed. 8. The method according to claim 6, wherein the luminance distribution in the vertical direction is binarized in a temporary character area in the label image to which the same label is assigned. The eleventh procedure for determining the character region pixels is performed by removing pixels inside the region by a preset pixel width from the left and right ends of the region for each horizontal line in the same label region in the image. An eighteenth procedure of calculating an average value of luminance values in the range; and, in the same label region in the image, examining a vertical distribution of the average luminance value of each horizontal line obtained in the eighteenth procedure, and A nineteenth procedure for extracting a region on a plurality of connected horizontal lines in which the average luminance value of each horizontal line is locally higher than a peripheral portion by a predetermined value in the label region, and the same label in the image Within the area, connect the links determined in the nineteenth procedure. If the vertical gradient value of the average luminance value of each horizontal line at the upper and lower ends of the region for each of the plurality of horizontal lines is larger than a preset value at both the upper and lower ends, the connection horizontal A twentieth procedure for determining the line area as a high-brightness character area; and, in the same label area in the image, examining the vertical distribution of the average luminance value for each horizontal line obtained in the eighteenth procedure. A twenty-first procedure for extracting a region on a plurality of connected horizontal lines in which the average luminance value for each horizontal line is locally lower by a preset value compared to the peripheral portion in the label region, and the same label in the image Within the region, the vertical gradient values of the average luminance value for each horizontal line determined in the eighteenth step at the upper and lower ends of the region on the plurality of connected horizontal lines determined in the twenty-first step are the upper and lower ends. And a preset value Again large Ri, 22 steps and, in the video in the character region extracting method characterized by performing for determining the ligation horizontal line area as a low luminance character area. 9. A program for causing a computer to execute the method for extracting a character region in a video according to any one of claims 5 to 8, and recorded on a computer-readable recording medium. A recording medium on which a method for extracting a character area in a video is described.