JP3533050B2 - Image area dividing device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention stably stabilizes a frame in which a character is displayed among a plurality of frames constituting a color moving image or a still color image in which a character is displayed with a small amount of calculation. The present invention relates to an image area division technique for forming a character portion as a pixel connection area.

[0002]

2. Description of the Related Art With respect to a frame in which characters are displayed in a plurality of frames forming a color moving image, an image area dividing technique for forming a character portion as a pixel connecting area is conventionally known as a still color image area dividing method. Many studies have been conducted as technology.

As one of them, a method of dividing a three-dimensional color space composed of three components of a color system and generating a pixel connected region having similar three component values as a character region (for example, " Ueha, Takeda, Okada: "Extraction of Character Area from Color Image by Color Matching", IEICE Technical Report, PRU9
4-28, pp. 9-16 (1994.9) "," Senda, Mino, Ikeda: "A method of extracting character patterns from color images that focuses on the monochromaticity of character strings", IEICE Technical Report, PRU94-
29, pp. 17-24 (1994.9) "," Yu
Zhong, Kale Karu and Anil
K. Jain: "Locating Text in
Complex Color Images ”, IE
EE, ICDR, pp. 146-149 (199
5) ”) In this case, the method of dividing the three-dimensional color space is a method of using clustering by the K-means method, a method of dividing a certain range centered on the maximum value of the frequency in the color space as one class, or the entire color space. There is also a method of dividing into equal intervals with an appropriate interval width.

A method is also proposed in which a three-dimensional color space is first divided, and then, for each divided area, when the pixel values between adjacent areas in the image are smooth, the areas are integrated. (For example, "Horita, Miyahara, Otani:" Region segmentation of color image based on uniform color space "", Theological theory, D
-II, Vol. J74-D-II, No. 10, pp. 1
370-1378 (1991.10) ").

[0005]

However, in the above-mentioned conventional image area dividing method for forming a character portion as a pixel connected area, the three areas composed of three components of the color system are used.
In the method of region division by dividing the three-dimensional color space,
Since the three-dimensional color space, which is an enormous amount of data, is divided, there is a problem that the amount of processing becomes very large. Like this 3
The method of performing the integration processing of the formed areas after the division processing of the dimensional color space has a problem that the processing amount becomes further huge.

Further, even if the color space is divided using the selected threshold value, the above-mentioned conventional method does not necessarily select the threshold value which reflects the characteristics of the character display form, as described above. There is a problem that a good character area formation result cannot be obtained.

Further, in the above-mentioned conventional method, since only one frame in which characters are displayed is to be processed, when the character color is similar to the background color in the frame, the character area and the background area are not separated and are the same. However, even if the character region and the background region are separated, the accurate contour of the character region cannot be extracted as the boundary between the character region and the background region, which is a problem.

The present invention provides an image area dividing device for forming an accurate character area as a connected area while reducing the enormous amount of calculation which has been a problem in the conventional method in the area division of a color image containing characters. The purpose is to do.

[0009]

[Means for Solving the Problems ]
, The image area dividing device according to a first aspect of the present invention, color
Luminance for binarizing an image by dividing the luminance space by an automatically set threshold for the luminance value for an image on which characters of the HSI color system of phase, saturation, and luminance are displayed. Space division processing means and a high saturation part and a low saturation part by dividing the saturation space by a threshold value set in advance or a threshold value set automatically with respect to the saturation in the area determined to be processed. And a saturation space division processing unit that divides the hue space into areas by dividing the hue space according to a preset threshold value or an automatically set threshold value for the hue in the area determined to be processed. The division processing means and, for each area determined to be processed, determine whether or not the condition that the number of pixels with high hue edge strength exists in the area is equal to or more than a preset number threshold value is determined. , Article The color space component to the region that satisfies
The hue change area determination means that determines to perform processing by the split processing means and the number of pixels with high saturation edge strength are preset in each area of the image binarized by the luminance space division processing means. It is determined whether or not the condition that the number of thresholds is greater than or equal to the condition is satisfied, and it is determined that the saturation space division processing unit performs processing on the region that satisfies the condition. And a saturation change area determination means for determining that the hue change area determination means performs processing on an area satisfying the condition that the saturation average value is equal to or greater than a preset threshold value and the saturation variance value is equal to or less than a preset threshold value. Whether the condition that the saturation average value in the area is equal to or greater than a preset threshold value and the saturation variance value is equal to or less than the preset threshold value with respect to the area divided and formed by the saturation space division processing unit Judgment If it is determined that the hue change area determination means performs processing on the area satisfying the condition, the saturation space division processing means performs processing again on the area where the saturation variance value is larger than a preset threshold value. High-saturation region determination means for determining to perform, and the saturation change region determination means,
The high-saturation area determination means and the hue change area determination means each include an area which is not determined to be processed and a means for outputting the image area division result by the hue space division processing means. . here
Then, like the image area dividing device according to claim 2, the characters are
Change the color system of the displayed image from the RGB color system to the hue and color.
Color system conversion processing means for converting to the HSI color system of degrees and brightness
The display system image converted to the HSI color system is provided.
It is used for image processing in the luminance space division processing means.
You may ask .

An image area dividing device according to claim 3
The image area dividing device according to claim 2 , wherein a frame in which a character having a designated frame number is displayed from a color moving image and a plurality of character image frames for inputting a designated number of frames at a time after the frame are input. comprising an input means, to create a RGB values averaged image of multiple frames the input, and a plurality of frames averaging means for inputting an image in which a character is displayed on the color system conversion processing unit, the
It is characterized by

According to the first and second aspects of the invention , the color system of the input image in which the characters are displayed is changed from the RGB color system to the HSI.
(Hue, Saturation, Luminance) The image converted into the HSI color system is divided into two areas by dividing the brightness space by an automatically set threshold value with respect to the brightness value. For each region formed by binarization and luminance space division,
It is determined whether or not the condition that the number of pixels with high saturation edge strength is equal to or greater than a preset number threshold value is present in the region, and the saturation in the region that satisfies the condition is
The area is divided into a high saturation area and a low saturation area by dividing the saturation space according to a preset threshold value or an automatically set threshold value, and for each area, the average saturation value in the area is calculated in advance. It is determined whether or not a condition of being equal to or greater than a set threshold value and equal to or less than a threshold value of a saturation saturation value is satisfied, and the number of pixels having a high hue edge strength is preset in the region for the condition. It is determined whether or not the condition that the number of thresholds is greater than or equal to the condition is satisfied, and for the region that satisfies the condition, the hue space in the region is divided by a plurality of preset hue value threshold values to divide the region. The region formed by the above processing is output as the image region division result.

According to the third aspect of the present invention , a frame in which a character having a designated frame number is displayed from a color moving image and a designated number of frames at a time after the frame are input, and a plurality of input multiple frames are input. RGB value averaged image is created and the created RGB value averaged image is contracted.
It is processed by the invention according to claim 1 or 2 .

Usually, a character portion in a moving image such as a television image has similar luminance, saturation, and hue values, or the luminance is similar, and the saturation and hue values change smoothly. In many cases, a pattern of display color is used, but in either case, there is a property that the brightness contrast between the character portion and the surrounding background portion is very high. Therefore, by using the luminance space division processing means for a color image containing a character, the character and the background can be efficiently separated and the character area can be formed regardless of the hue and saturation of the character portion. However, for a character whose brightness is partially similar at the boundary with the background, it is difficult to form a character area by the brightness space division processing unit.Therefore, among these, characters with low saturation in a background with high saturation, etc. With regard to a character having a high saturation edge strength at the boundary portion of, the character and the background are separated by the saturation space division processing unit that divides into a high saturation portion and a low saturation portion, so that the character area can be accurately formed. Here, a character with a high saturation variance value but a low saturation edge strength, such as a character whose saturation changes smoothly, is not subjected to area division processing by saturation, and the hue within the area is set in advance. By dividing the hue space by the plurality of hue value thresholds described above, it is possible to form a character area as one area. Further, when the saturation is similar to the background at the boundary between the character and the background, the character and the background are both highly saturated, and the hue edge strength at the boundary between the background and the character is high. The division processing means separates the character from the background and enables the character area to be accurately formed. Here, a character with a large hue variance value but a low hue edge strength, such as a character whose hue changes smoothly, can be formed as a single area by the high-saturation area determination means without performing area division processing by hue. And As described above, according to the present invention, the one-dimensional color space division processing is performed in the order of the luminance binarization processing, the saturation division processing, and the hue division processing to form an area that reflects the characteristics of the character display form. Means for forming a stable character area and reducing the processing amount as compared with the conventional method.

That is, as described above, in the present invention, the area division of the color image is performed by the combination of the division processing of the one-dimensional color space which reflects the characteristics of the character display form in the image. The processing amount for determining the threshold value for dividing the color space is 3
Compared to the determination of the division threshold of the dimensional color space, the processing amount is reduced because the dimension of the color space is smaller. In the first division of the luminance space, for example, “Salt:“ Dynamic Binarization Processing Method for Detecting Characters in Scene ””, SI, Vol. J71-D, No.
5, pp. 863-873 (1988.5) ", a threshold value for division is automatically determined by using an average value in a partial rectangle or a discriminant analysis method. In the hue space division processing, pixel values in the vicinity of the high edge strength portion in each area are used, and their average value or a threshold for division is automatically determined using a discriminant analysis method. As described above, according to the present invention, the color space is divided by the division threshold that is automatically determined after reflecting the display form of the characters in the image, thereby forming an area having higher robustness than the conventional method. It is possible to do.

Further, in the invention according to claim 3, when the pixel value of the contour portion of the character is not expressed by a stable pixel value accurately reflecting the shape of the character in one frame, the character is displayed. A frame and a means for creating an averaged image from a plurality of frames at a later time based on the frame, by averaging a plurality of pixel values, a smooth and stable contour is obtained for the character area. A shape pattern can be formed. Furthermore, even with respect to the problem of the conventional method that the character area and the background area are not separated but integrated into the same area, over a plurality of frames,
When the color of the background portion is changed, the average value is calculated to cause a difference in pixel value between the character portion and the background portion, thereby improving the accuracy of separation between the character area and the background area as compared with the conventional method.

[0016]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings.

FIG. 1 is a block diagram showing a device configuration of an embodiment of the present invention. In FIG. 1, 1 is a character image multi-frame input processing unit, 2 is a multi-frame averaging processing unit, 3
Is a color system conversion processing unit, 4 is a luminance space division processing unit, 5 is a saturation gradation change region determination unit, 6 is a saturation space division processing unit, 7 is a high saturation region determination unit, and 8 is a hue gradation change region determination unit. , 9 is a hue space division processing unit, and 10 is an output unit of the region generation result.

In the character image multi-frame input processing unit 1,
From the color moving image, a frame in which a character having a designated frame number is displayed and a designated number of frames at a time after the frame are input, and the frames are passed to the multi-frame averaging processing unit 2.

The plural-frame averaging processing unit 2 obtains an average value of RGB values of corresponding pixels in a plurality of frames input from the character image plural-frame input processing unit 1, and creates an RGB averaged image. It is passed to the color system conversion processing unit of.

In the color system conversion processing section 3, the RGB of the RGB averaged image inputted from the plural frame averaging processing section 2 is converted into RGB.
The value is converted into the HSI color system that includes hue, saturation, and luminance. The conversion method is, for example, “Image Analysis Handbook, The University of Tokyo Press, pp. 485-491 (1991.
1) ”is used. The outline of the conversion method is as follows.

In the RGB orthogonal coordinate system, the main diagonal axis of the RGB cube contacting the three axes is the luminance I axis, and I = max
It is defined as {R, G, B}. Here, an RGB cube is projected in parallel on a plane orthogonal to the I axis to form a regular hexagon, and when a certain point P is given, the saturation S on that plane is WP / W.
E (value range [0, 1]), and hue H is defined by the counterclockwise direction angle between WR and WP (value range [0, 2π]).
Where W is the intersection of the plane and the I-axis, E is the intersection of the extension line of WP and the side of the regular hexagon, and R is the RG projected on the plane.
It is the R point of the B cube. First, I is converted by the above equation, and if I = 0, then S = 0 and H = undefined. If I ≠ 0, S = (I−i) / I, i = min {R, G, B}
(Min is a function that gives the minimum value), and r = (I-
R) / (I-i), g = (I-G) / (I-i), b =
(IB) / (I-i) is calculated, and when R = I, H
= Π (b−g) / 3, and when G = I, H = π (2+
r−b) / 3, and when B = I, H = π (4 + g−
r) / 3. However, when H <0, 2π is added to the value of H.

Of the HSI values thus obtained, the brightness I is to the brightness space division processing unit of 4, the saturation S is 5, the saturation gradation change region determination unit, the saturation space division processing unit of 6, and the high saturation of 7. The hue H is passed to the area determination unit and to the hue gradation change area determination unit 8 and the hue space division processing unit 9.

In the luminance space division processing unit 4, regarding the luminance obtained by the color system conversion processing unit 3, "Salt:" Dynamic Binarization Processing Method for Detecting Characters in Scene "", Theological Theory, D , Vol.
J71-D, No. 5, pp. 863-873 (198
8.5) ”, the binarization processing is performed, and the coordinate values of the pixels forming each formed area and the label value of the area to which the pixel belongs belong to the saturation gradation change area determination unit 5. The outline of the binarization process is as follows.

(1) Region division processing Input grayscale images g _i , _j (1 ≦ i ≦ I, 1 ≦ j ≦ J) are M ×
It is equally divided into N rectangular areas (P × Q pixels) (however,
I = P × M, J = Q × N). In the following, this rectangular area will be referred to as a partial area or simply an area.

(2) Threshold value determination process For each of the divided partial areas (m, n), the threshold value θ _m , _n is determined from the density histogram of the pixels in the area (1 ≦ m
≤ M, 1 ≤ n ≤ N).

(3) Inter-region Interpolation Processing In order to suppress background noise, the thresholds θ _m and _n are repeatedly corrected between adjacent regions by the weight value based on the contrast evaluation value. Here, the contrast evaluation values _cm and _n are grayscale images g
_{Using the} standard deviations σ _m , _n and the average μ _m , _n of the sub-regions (m, n) of _i , _j , it is defined as _cm , _n = σ _m , _n / μ _m , _n .

(4) Inter-Pixel Interpolation Process Discretely obtained thresholds θ _m , _n of each partial region (m, n) are given to the center of each partial region, and these are interpolated over the entire image to obtain a threshold image t. _{Find i} and _j .

(5) Threshold processing The input grayscale images g _i , _j and the threshold images t _i , _j are compared in size,
If g _i , _j ≧ t _i , _j, then b _i , _j = 1; otherwise, b _i , _j =
Binary patterns b _i , _j are obtained according to the conditional expression 0.

The saturation gradation change area determination section 5 calculates the edge strength of the saturation value obtained by the color system conversion processing section 3 for each pixel forming each area formed by the brightness space division processing section 4. Determining whether or not a condition that the number of pixels whose saturation edge strength is higher than a preset threshold value is greater than or equal to a preset number exists is satisfied, and passes the region that satisfies the condition to the saturation space division processing unit 6 In a region that does not satisfy the condition, the saturation average value is equal to or more than a preset threshold value,
In addition, an area that satisfies the condition that the saturation variance value is equal to or less than a preset threshold value is passed to the hue gradation change area determination unit 8. The other areas are passed to the output unit 10.

In the saturation space division processing unit 6, in the area input from the saturation gradation change area determination unit 5,
With respect to the saturation calculated by the color system conversion processing unit 3, a threshold value for saturation space division set in advance, or a saturation of a neighboring area centered on a pixel with high saturation edge strength in each area Using the average value or the value set using the discriminant analysis method as a threshold value, a high saturation area composed of pixels having a saturation value larger than this value, a pixel composed of pixels having a small saturation value The area is divided into the low-saturation area, and the formed area is passed to the high-saturation area determination unit 7.

The high-saturation area determination section 7 calculates an average value and a variance value for the saturation values obtained by the color system conversion processing section 3 for each area formed by the saturation space division processing section 6. ,
It is determined whether or not the condition that the average value is greater than or equal to a preset threshold value and the variance value is less than or equal to the preset threshold value is satisfied. The region that satisfies the condition is passed to the process of the hue gradation change region determination unit 8, and the region that does not satisfy the condition is larger than the threshold value set in advance for the variance value, the process of the saturation space division processing unit 6 is performed. The processing is performed again, and the other areas are passed to the output unit 10 without performing the subsequent processing.

In the hue gradation change area judging section 8, for the areas satisfying the conditions of the high saturation area judging section 7,
The edge strength of the hue value obtained by the color system conversion processing unit 3 in the area is calculated, and it is determined whether or not the number of pixels whose hue edge strength is higher than a preset threshold value is greater than or equal to the preset number. The area that satisfies the condition is passed to the processing of the hue space division processing unit 9, and the area that does not satisfy the condition is passed to the output unit 10 without performing the subsequent processing.

In the hue space division processing unit 9, when the number of pixels determined by the hue gradation change region determination unit 8 is a certain number or more, the hue value determined by the color system conversion processing unit 3 in the region is determined. Then, using a preset threshold for hue space division, or using the hue value of a neighboring area centered on a pixel with a high hue edge strength in each area, the average value or the discriminant analysis method is used. The hue space is divided using the value set as a threshold as a threshold, and the formed area is passed to the output unit of 10.

The output unit 10 outputs the processing results up to the above.

Experimental results of this embodiment will be described. The frame in which the characters in the moving image used in the experiment are displayed is shown in FIG. FIG. 3 shows an averaged image generated by using the processing of the plural-frame averaging processing unit 2 for 10 frames with this frame as the head frame. FIG. 4 shows a region image formed by binarizing the averaged image by the luminance by the processing of the luminance space division processing unit 4. For this area image, the processing of the saturation gradation change area determination unit 5, the saturation space division processing unit 6, the high saturation area determination unit 7, the hue gradation change area determination processing unit 8, and the hue space division processing unit 9 is used. FIG. 5 shows the result of region generation based on saturation and hue values while considering the edge strength within the region.
A threshold value for dividing the saturation and the hue space uses a preset value. The saturation threshold value is 0.4 and the hue threshold value is

[0036]

[Equation 1]

It was set in 6 sections. However, the range of saturation and hue is [0.0, 1.0] and [0, 2π], respectively. According to the present embodiment example, it was confirmed that stable character area generation with less influence of threshold value selection at the time of color space division can be realized with a small calculation amount.

In the above embodiment, the case where the character portion is formed as the pixel connecting area with respect to the frame in which the character is displayed in the plurality of frames forming the color moving image has been described. By omitting the input processing unit 1 and the multiple frame averaging processing unit 2,
In the case of a still color image as well, it is possible to form a character portion as a pixel connection area in the same manner.

[0039]

As described above, according to the present invention,
Color space division processing for region generation is performed in the order of luminance, saturation, and hue in a one-dimensional color space that reflects the character display form, so the character region is stable with a smaller amount of processing than conventional methods. It is possible to divide the image area to be formed in.

When the average value including a plurality of frames at a later time is used for the frame in which the characters in the moving image are displayed, the pixel values of the character portion and the background portion are Because of the difference, it is possible to improve the accuracy of separation between the character area and the background area as compared with the conventional method.

[Brief description of drawings]

FIG. 1 is a block diagram showing a device configuration of an embodiment of the present invention.

FIG. 2 is a drawing-substituting photograph showing an example of a frame of a moving image used in an experiment according to the embodiment example.

FIG. 3 is a drawing-substituting photograph showing an example of an averaged image subjected to an averaging process in the above embodiment.

FIG. 4 is a drawing-substituting photograph showing an example of a region image formed by binarization by the luminance space division processing in the above embodiment.

FIG. 5 is a drawing-substituting photograph showing an example of a result of region generation based on a saturation and a hue value in consideration of an edge strength in the region in the above embodiment.

[Explanation of symbols]

1 ... Character image multi-frame input processing unit 2 ... Multiple frame averaging processor 3 ... Color system conversion processing unit 4 ... Luminance space division processing unit 5 ... Saturation gradation change area determination unit 6 ... Saturation space division processing unit 7 ... High saturation area determination unit 8 ... Hue gradation change area determination processing unit 9 ... Hue space division processing unit 10 ... Output section

Continuation of front page (56) Reference JP-A-5-22463 (JP, A) JP-A-5-60616 (JP, A) JP-A-5-68262 (JP, A) JP-A-7-93554 (JP , A) (58) Fields investigated (Int.Cl. ⁷ , DB name) H04N 9/74 H04N 1/40 H04N 1/46 H04N 1/60

Claims (3)

(57) [Claims] 1.Characters of HSI color system of hue, saturation, and brightness are
Displayed imageThe brightness value is automatically set for
Binarize the image by dividing the luminance space by the threshold
Luminance space division processing means, Regarding the saturation in the area decided to be processed in advance
Color is set according to the set threshold or the automatically set threshold.
Dividing the saturation space divides the region into high-saturation and low-saturation areas.
A saturation space division processing means for dividing, Regarding the hue in the area decided to be processed in advance
Color according to the set threshold or automatically set threshold
Hue space division processing that performs region division by dividing the phase space
Reasoning means, For each area that was decided to be processed, within the area
The number of pixels with high hue edge strength is preset
Judgment whether or not the condition that more than the threshold value exists is satisfied
To the area that meets the conditionThe hue space division processing
ReasonHue change area decision hand that decides to perform processing in
Dan, Each area of the image binarized by the luminance space division processing means
The number of pixels with high saturation edge strength is preset in
Judgment as to whether or not the condition that there is more than the number threshold is satisfied
And the area of the saturation space is divided into
Satisfies the conditions by deciding to perform processing with the split processing means
The average saturation value in a non-existing area is greater than or equal to the preset threshold, and
Meets the condition that the saturation variance value is less than or equal to a preset threshold
For the areaThe hue change area determination meansProcess with
Saturation change area determination means for determining that, In the area formed by dividing by the saturation space division processing means
On the other hand, the saturation average value in the area is equal to or greater than the preset threshold value, and
Meets the condition that the saturation variance value is less than or equal to a preset threshold
It is determined whether or not to
Determined to perform processing in the hue change region determination means,
For areas where the saturation variance is greater than a preset threshold
Again decided to perform processing with the saturation space division processing means
High saturation area determination means to The saturation change area determination means, the high saturation area determination means,
And processing in each of the hue change area determination means.
Areas that were not decided toFor the hue space
Split processing meansAnd a means for outputting the image area division result in
An image area dividing device characterized by the above. 2. A color system of an image in which characters are displayed is an RGB table.
Table for converting from color system to HSI color system of hue, saturation and luminance
A color system conversion processing unit is provided to convert to the HSI color system.
The image of the display system is displayed in the brightness space division processing means.
The image according to claim 1, which is provided for image processing.
Image area dividing device. 3. A frame number is designated from a color moving image.
The frame in which the displayed characters are displayed, and after the frame
Character image to enter the specified number of frames at time
A plurality of frame input means and an RGB value averaged image of the plurality of input frames are generated.
3. The image area dividing device according to claim 2 , further comprising: a plurality of frame averaging means configured to input as an image in which characters are displayed in the color system conversion processing means .