JP4870721B2 - Image feature extraction apparatus, image feature extraction method, program, and recording medium - Google Patents

Description

  The present invention relates to a technique for extracting features from an image, and more particularly to a technique for extracting a color edge pair from an image.

  As a technique that can be used to detect a line figure drawn by thin lines such as a map, a design drawing, and a character string from an image and separate it from the background of the image, there is a technique described in Patent Document 1, for example.

In this technique, an edge that is a feature whose luminance changes discontinuously (hereinafter referred to as “luminance edge”) is extracted from the image as a feature quantity, and an ascending edge and a descending gradient edge are paired (hereinafter referred to as “ It is used as a line figure extraction.
JP-A-11-205679

  However, in the above conventional technique, since a luminance edge is extracted as a feature amount from an image, the intensity of the luminance edge is small in a line figure that is different in color from the background but has similar luminance. Therefore, in such a case, it becomes difficult to create a luminance edge pair, and sufficient performance cannot be exhibited.

  The present invention has been made in view of the above points, and provides a technique that makes it possible to appropriately extract an edge pair (color edge pair) even when the luminance is similar between a line figure and its background. The purpose is to provide.

In order to solve the above-described problems, the present invention provides an image feature extraction device for extracting a color edge pair from an image including a line figure, the image acquisition unit acquiring the image from an image storage unit, and the image Color edge extracting means for extracting color edges as absolute values from the color edge pair color candidate extraction set from the color edges extracted by the color edge extracting means, and the line figure has a locally similar color Color edge pair extraction means for extracting a color edge pair candidate satisfying a color edge pair extraction condition based on the color edge pair from the color edge pair candidate set as a color edge pair , and the color edge pair extraction condition comprises a first color edge pair candidate. In one color edge and the second color edge, among the pixels around the first color edge, the pixel is on the side closer to the second color edge And color-containing, among pixels around the second color edge, the image first is the color of the pixel on the side close to the color edges characterized in that it is a condition that it is similar Configured as a feature extraction device.
The present invention also provides an image feature extraction device for extracting a color edge pair from an image including a line figure, an image acquisition unit for acquiring the image from an image storage unit, and a color for extracting a color edge from the image A color that satisfies a color edge pair extraction condition based on a property of extracting a color edge pair candidate set from the color edges extracted by the edge extraction means and the color edge extraction means, and the line figure has a locally similar color Color edge pair extraction means for extracting an edge pair candidate as a color edge pair from the color edge pair candidate set, wherein the color edge pair extraction condition includes a first color edge and a second color edge constituting the color edge pair candidate And the color of the pixel on the side closer to the second color edge among the pixels around the first color edge and the circumference of the second color edge. Among the pixels in the pixel, the condition that the color of the pixel closer to the first color edge is similar, the color change vector in the vicinity of the first color edge, and the second color edge The condition that the color change vector in the vicinity is substantially opposite, and the amount of color change between the plurality of pixels between the first color edge and the second color edge is small. The color edge pair extracting unit extracts a color edge pair candidate that satisfies at least one of the three conditions as a color edge pair from the color edge pair candidate set. You may comprise as an image feature extraction apparatus.

  The present invention may also be configured as a method, a program corresponding to the processing operation of the image feature extraction apparatus, and a computer-readable recording medium on which the program is recorded.

  According to the present invention, it is possible to appropriately extract a color edge pair even when the luminance is similar between the line figure and its background. This makes it possible to extract a line figure region in the image with high accuracy.

Embodiments of the present invention will be described below with reference to the drawings. In the embodiment, the color information is high-dimensional information, for example, three-dimensional information. For example, RGB, HSV, HSI, YUV, LUV, LAB, XYZ, and the like are models for expressing colors with three-dimensional information, but the present invention can be applied to any color expression method. .
(First embodiment)
<Device configuration>
FIG. 1 shows a functional configuration diagram of an image feature extraction apparatus 1 according to the present embodiment. As shown in FIG. 1, the image feature extraction apparatus 1 according to the present embodiment includes an image storage unit 11, an image acquisition unit 12, a color edge extraction unit 13, a color edge pair extraction unit 14, a pairness determination unit 15, and an input / output Part 16.

  The image storage unit 11 stores image data to be processed in the present embodiment (simply referred to as an image), receives an image request from the image acquisition unit 12, and displays an image related to the image request as an image. This is a functional unit returned to the acquisition unit 12.

  The image storage unit 11 includes a magnetic disk device, an optical disk device, a memory, and the like in a computer used as the image feature extraction device 1. Further, instead of providing the image storage unit 11 in the image feature extraction device 1, it may be arranged on the network as a device different from the image feature extraction device 1. For example, as the image storage unit 11, a server device connected to the image feature extraction device 1 via a network can be used.

  In the present embodiment, the “image” may be a video (moving image) or a still image. Also, the content of the image is not particularly limited. Examples of the content include a natural image, an artificial image, a face image, and data obtained by imaging a document. In addition, the image stored in the image storage unit 11 may be pre-processed in advance.

  FIG. 2 shows an example of a method for storing image data in the image storage unit 11. As shown in FIG. 2, in this example, color component values are stored continuously for each pixel (pixel position).

  The image acquisition unit 12 is a functional unit that sends an image request signal to the image storage unit 11, acquires an image from the image storage unit 11, and outputs the acquired image to the color edge extraction unit 13. The color edge extraction unit 13 is a functional unit that performs color edge extraction processing on the image received from the image acquisition unit 12 and passes the result to the color edge pair extraction unit 14. The color edge pair extraction unit 14 specifies a color edge pair candidate from the color edge extraction processing result, and uses the pairness determination unit 15 to extract a color edge pair from the color edge pair candidate and pass it to the input / output unit 16 It is.

  The image feature extraction apparatus 1 is realized by causing a general computer including a CPU and a storage device such as a memory or a hard disk to execute a program corresponding to the processing described in the present embodiment. Each functional unit described above is a functional unit realized by executing the program on a computer. Therefore, for example, the exchange of information between the functional units is actually performed via a storage device such as a memory. The above program may be stored in a recording medium such as a memory and then installed in a computer from there, or may be downloaded from a server on a network.

<Operation of Image Feature Extraction Apparatus 1>
Hereinafter, the operation of the image feature extraction apparatus 1 will be described with reference to the flowchart shown in FIG.

  First, the image acquisition unit 12 sends an image request signal for requesting an image to be processed to the image storage unit 11 based on information input from the input / output unit 16, for example. And the image is passed to the color edge extraction unit 13 (step 1).

  In addition, when the image acquired here is a moving image, the image acquisition unit 12 performs a process of extracting each frame from the moving image, and outputs each extracted frame to the color edge extraction unit 13. This process is performed for each frame (image).

  Subsequently, the color edge extraction unit 13 performs a process of extracting a color edge from the image received from the image acquisition unit 12 and passing it to the color edge pair extraction unit 14 as color edge information (step 2).

  As a color edge extraction technique, a technique that can extract a portion where color information changes discontinuously as a feature is used. As such a technique, many conventionally proposed edge extraction techniques can be used. For example, edge extraction technology using simple difference and its absolute value, edge extraction technology using DoG (differential Gaussian) operator, edge extraction technology using edge detection operators such as Sobel and Prewitt, edge extraction using Laplacian operator Any of a technology, an edge extraction technology using a MAX-MIN operator, and the like can be used. Of course, techniques not listed here may be used.

  As an example, a processing example of the color edge extraction unit 13 in the case where vertical stripe edge extraction is performed when a method combining a Sobel operator (3 × 3) and an absolute value is employed will be described.

  When pixels (component values a to i) in the vicinity of a certain pixel of interest (component value e) are arranged as shown in FIG. 4A in the sub-block, the color edge extracting unit 13 includes the sub-block. On the other hand, by applying the 3 × 3 Sobel operator shown in FIG. 4B and taking the absolute value, | c−a | +2 | f−d | + | i−g | (| · | (Absolute value) is calculated, and this is used as the value of the edge of the specific component at the position of the target pixel. The component here is, for example, an R component of color information.

  The color edge extraction unit 13 applies an edge extraction technique for each color component of each pixel of the image, and calculates a color edge value for each color component for each pixel. Further, the color edge extraction unit 13 obtains, for each pixel, the sum of the color edge values of the respective color components or the length of a vector composed of the color edge values of the respective color components, and uses this as the color edge value to be output. Note that the process of setting the color edge value of each color component to one color edge value is not limited to the above.

  Then, the color edge extraction unit 13 passes the combination of the pixel position and the color edge value to the color edge pair extraction unit 14 as color edge information.

  The color edge pair extraction unit 14 generates a set of color edge pair candidates that are sets of two color edges that are candidates for color edge pairs from the color edge information received from the color edge extraction unit 13, and stores the set in a storage device such as a memory. (Step 3). It should be noted that the original image information (pixel position and color component information) is held in a memory or the like, and can be appropriately acquired if necessary during processing.

  In the process of generating the color edge pair candidate, the color edge pair extraction unit 14, for example, determines that the edge values of the two color edges are equal to or greater than a threshold value and are in a predetermined positional relationship, and the distance between the two color edges Is extracted as a color edge pair candidate. The predetermined positional relationship is, for example, that two color edges are arranged side by side on the same Y coordinate position, or are arranged vertically on the same X coordinate position.

  Next, the color edge pair extraction unit 14 proceeds to a pairing determination process using the pairing determination unit 15.

  Here, first, the color edge pair extraction unit 14 checks whether or not there is an unjudged pair of color edge pairs generated in step 3 (step 4 in FIG. 3). If there is no undetermined one (No in step 4), the pairness determination for all color edge pair candidates has been completed, and the process is terminated (step 8).

  If there is an undetermined one (Yes in step 4), one color edge pair candidate (information including the position of the color edge pixel) is selected from the set of undetermined color edge pair candidates, and the color edge pair candidate is determined as a pair property Then, the pairing determination unit 15 determines the pairing of the color edge pair candidate (step 5).

  The pair property determination unit 15 determines whether the color edge pair candidate received from the color edge pair extraction unit 14 satisfies the pair property determination condition of the color edge pair.

  As the pairing determination conditions, the following pairing determination conditions 1 to 3 are used. In the present embodiment, it is determined that a color edge pair satisfies at least one of these conditions. These conditions are conditions that focus on the property that the color of a line figure such as a character is locally constant. Further, the pairing determination conditions are not limited to the following.

  Pairability determination condition 1: The colors of pixels on the other color edge side are similar for two color edges (pixel points of the color edge portion) constituting a color edge pair candidate. In other words, this condition is that, among the pixels around the first color edge in the first color edge and the second color edge constituting the color edge pair candidate, the pixel closer to the second color edge is selected. This is a condition that the color and the color of the pixel near the first color edge among the pixels around the second color edge are similar.

  Examples of color similarity determination criteria for determining that colors are similar include the following.

  -The sum of the absolute values of the difference between the component values of the color information of each pixel is not more than a certain value. This is, for example, that (absolute value of difference in R component value) + (absolute value of difference in G component value) + (absolute value of difference in G component value) is equal to or less than a certain value.

  The distance between colors of each pixel (for example, Euclidean distance in RGB space) is not more than a certain value.

  The pairness determination condition 1 will be further described with reference to FIG. When the color edge pixels 101 and 104 constituting the color edge pair candidate exist as shown in FIG. 5A, the pixel on the color edge (pixel 104) side of the other side of the pixel 101 is the pixel 102. A pixel 103 on the color edge (pixel 101) side of 104 is a pixel 103. These pixels 102 and 103 are referred to as pixels on the other color edge side. In the example shown in FIG. 5A, in each color edge, the pixel of the color edge and the pixel closer to the other color edge are adjacent to each other, but are not necessarily adjacent to each other. For example, one to several pixels may be separated between the pixel 101 and the pixel 102 and between the pixel 103 and the pixel 104, respectively.

  In the example shown in FIG. 5A, when the colors of the pixels 102 and 103 are similar, the pairing determination unit 15 determines that the color edge pair candidates 101 and 104 satisfy the pairing determination condition. .

  When the target line figure is a character, by using this pairing determination condition 1, color edge pairs are concentrated and extracted on the outline of the line constituting the character. Here, in the selection of color edge pair candidates, by appropriately setting the distance between the color edge pair candidates according to the size of the character and the thickness of the character line, it is possible to suppress the color edge pairs detected outside the periphery of the character. it can.

  Pairing determination condition 2: The color change vector in the pixel in the vicinity of each color edge of the color edge pair candidate is substantially in the reverse direction. That is, this condition is that the color change vector in the vicinity of the first color edge and the change vector in the vicinity of the second color edge are substantially the same at the first color edge and the second color edge constituting the color edge pair candidate. The condition is that the direction is opposite.

  For example, in the RGB color space, a color change vector from a pixel in which R component, G component, and B component are all 0 to a pixel in which R component is 255, G component, and B component is 0 is R The component is a vector of 255, and the G component and B component are both zero. The vector and the inverse vector are vectors having an R component of −255, a G component, and a B component of 0.

  The pixel in the vicinity of the color edge is, for example, from one color edge (referred to as a start point color edge) of two color edge pairs constituting a color edge pair candidate to the other color edge (referred to as an end point color edge). Two pixels lined up in the direction of the vector, and on the start point color edge side, pixels existing in the vicinity of the start point color edge on the line drawn from the start point color edge in the direction perpendicular to the vector (start point color edge) And the pixels lined up in the direction of the vector of the pixel. On the end color edge side, on the line drawn from the end point color edge in the direction perpendicular to the vector, it is in the vicinity of the end color edge. There are two pixels: an existing pixel (including an end color edge) and a pixel arranged on the opposite side of the vector direction of the pixel. The color change vector of the pixel in the vicinity of the color edge is, for example, on the start point color edge side, from the color of the pixel existing on the start point color edge side in the vicinity pixel, It is a change vector to the color of the pixel existing on the end color edge side, and on the end color edge side, from the color of the pixel existing on the start point color edge side of the neighboring pixels, This is a change vector to the color of a pixel existing on the end edge side of.

  Further, as a color change vector of a pixel in the vicinity of the color edge, on the start point color edge side, a plurality of pixels (in the vicinity of the start point color edge on a line drawn from the start point color edge in the direction perpendicular to the vector) ( For example, from the average of the colors of the start point color edge and the two pixels above and below it, a plurality of pixels lined up in the direction of the vector for each of the plurality of pixels (for example, adjacent to the three pixels described above) (3 pixels arranged in the direction direction of the vector) using a change vector to the average color, and on the end color edge side, a plurality of pixels existing in the vicinity of the end color edge on the line drawn from the end color edge in the direction perpendicular to the vector From the average of the colors of pixels (for example, the end point color edge and the three pixels above and below it), each of the plurality of pixels is aligned in the opposite direction of the vector. A plurality of pixels may be used change vector obtained by subtracting the average color (adjacent to e.g. the 3 pixels 3 pixels aligned in the opposite direction side of the vector).

  In the example shown in FIG. 5A, the color of the pixel of the image changes from black to yellow in the vicinity of one color edge 101 of the color edge pair candidate, and in the vicinity of the other color edge 104 of the color edge pair candidate. Since the color has changed from yellow to black and satisfies the pairing determination condition 2, the pairing determination unit 15 determines that the color edge pair candidates 101 and 104 are color edge pairs. Of course, the color change vector from the pixel 101 to the pixel 102 and the color change vector from the pixel 103 to the pixel 104 may be used for the pairing determination in the pairing determination condition 2.

  Further, as described above, the change vector from the average color of the pixels 201, 101, and 301 to the average color of the pixels 202, 102, and 302 in FIG. 5B and the average of the colors of the pixels 203, 103, and 303 To the average vector of the colors of the pixels 204, 104, and 304 may be used for pairing determination.

Pairing determination condition 3: The amount of color change between two color edges of a color hedge pair candidate is small.
For example, this condition is that the difference between the maximum value and the minimum value of each color component of a pixel on a line segment connecting two color edges of a color edge pair candidate is not more than a predetermined value. Further, for example, the maximum value of the distance between the colors of the pixels on the line segment connecting the two color edges of the color edge pair candidate (such as the Euclidean distance in the RGB space) is not more than a predetermined value.

  The pairing determination unit 15 that has performed the pairing determination based on the above pairing determination condition returns the pairing determination result for the color edge pair candidate received from the color edge pair extraction unit 14 to the color edge pair extraction unit 14, and the color edge pair extraction unit 14 receives the pairedness determination result of the color edge pair candidate from the pairedness determination unit 15. When the pairing determination result is satisfied (Yes in step 6 in FIG. 3), the color edge pair extraction unit 14 passes the color edge pair to the input / output unit 16 as the color edge pair extraction result. The color edge pair extraction result includes two color edge positions (pixel positions), color component values, and color edge values included in the color edge pair.

  When the pairedness determination result is not satisfied (No in Step 6), the color edge pair extraction unit 14 returns to the process of Step 4, and if there is an undetermined color edge pair candidate, the pairedness described above with respect to that color edge pair candidate. Judgment processing is performed.

  The input / output unit 16 that has received the color edge pair extraction result from the color edge pair extraction unit 14 outputs the color edge pair extraction result as color edge pair information (step 7). This output may be visualized in the form of an image or the like, or may be input to another function unit in the form of a list and further processed. Then, the process returns to step 4.

  In the above example, the color edge pair extraction unit 14 performs the pairing determination process using the pairing determination unit 15 for each color edge pair candidate set, but all the color edge pair candidates are transferred to the pairing determination unit 15. It is also possible to receive information on all color edge pairs satisfying the pair property from the pair property determination unit 15 and generate and output all color edge pair extraction results based on the information.

(Second Embodiment)
Hereinafter, as an example of a method of using the feature amount (color edge of the color edge pair) extracted by the processing method according to the first embodiment, an example in which the feature amount is used for telop character string region extraction will be described. By using the configuration described here, it is possible to extract the telop character string region with high accuracy while effectively utilizing the image feature extracted in the first embodiment.

<Device configuration>
FIG. 6 shows a functional configuration diagram of the telop character string region extraction device 20 in the present embodiment. As shown in FIG. 6, the telop character string region extraction device 20 in the present embodiment includes a data storage unit 21, an image acquisition unit 22, a feature amount extraction unit 23, a telop character string region shape acquisition unit 24, and a density calculation unit. An information acquisition unit 25, a telop character string region extraction unit 26, and an input / output unit 27 are included.

  The data storage unit 21 is a functional unit that stores data used in the processing according to the present embodiment, receives a data request from each functional unit, and returns data related to the data request to the functional unit that is the request source. FIG. 7 shows an example of data stored in the data storage unit 21 in the present embodiment. As shown in FIG. 7, the data storage unit 21 stores image data (simply referred to as “image”) and telop character string region shape information in association with the image ID (FIG. 7A). )) And a table (FIG. 7B) that stores the density calculation area shape information and the weight coefficient distribution information in association with the telop character string area shape information. The contents of each information will be described when the apparatus operation is described.

  The data storage unit 21 includes a magnetic disk device, an optical disk device, a memory, and the like in a computer used as the telop character string region extraction device 20. Further, instead of providing the data storage unit 21 in the telop character string region extraction device 20, it may be arranged on the network as a device different from the telop character string region extraction device 20. For example, as the data storage unit 21, a server device connected to the telop character string region extraction device 20 via a network can be used.

  Also in this embodiment, the “image” may be a video (moving image) or a still image.

  The image acquisition unit 22 is a functional unit that sends an image request signal to the data storage unit 21, acquires an image from the data storage unit 21, and outputs the acquired image to the feature amount extraction unit 23. The feature amount extraction unit 23 is a functional unit that performs feature amount extraction processing on the image received from the image acquisition unit 22 and passes the result and the image ID to the telop character string region shape acquisition unit 24.

  In the present embodiment, the feature amount extraction unit 23 includes the color edge extraction unit 13, the color edge pair extraction unit 14, and the pairness determination unit 15 in the first embodiment, and color edge pair information is obtained as a result of the feature amount extraction processing. Is output.

  The telop character string region shape acquisition unit 24 is a functional unit that acquires telop character string region shape information from the data storage unit 21.

  The density calculation information acquisition unit 25 is a functional unit that acquires information necessary for density calculation from the data storage unit 21 and passes the information to the telop character string region extraction unit 26. The telop character string region extraction unit 26 is a functional unit that extracts the telop character string region from the feature amount extraction processing result using the information received from the density calculation information acquisition unit 25. Further, the telop character string region extraction unit 26 further has a function of performing precise extraction of the region end in the telop character string region using the information received from the density calculation information acquisition unit 25. The input / output unit 27 is a functional unit for inputting / outputting data.

  The telop character string region extraction device 20 is realized by causing a general computer including a CPU and a storage device such as a memory or a hard disk to execute a program corresponding to the processing described in the present embodiment. Each functional unit described above is a functional unit realized by executing the program on a computer. Therefore, for example, the exchange of information between the functional units is actually performed via a storage device such as a memory. The above program may be stored in a recording medium such as a memory and then installed in a computer from there, or may be downloaded from a server on a network.

<Operation of Telop Character String Region Extraction Device 20>
Next, the operation of the telop character string region extraction device 20 in the present embodiment will be described with reference to the flowchart shown in FIG.

  First, the image acquisition unit 22 receives an image ID indicating an image to be acquired from the input / output unit 27 as input information. Then, the image acquisition unit 22 acquires an image corresponding to the image ID from the data storage unit 21, and passes the image to the feature amount extraction unit 23 (step 11). If the acquired image is a moving image, the image acquiring unit 22 performs a process of extracting each frame from the moving image, and outputs each extracted frame to the feature amount extracting unit 23. This process is performed for each frame (image).

  The feature amount extraction unit 23 performs the feature amount extraction processing for acquiring the color edge pair information described in the first embodiment on the image passed from the image acquisition unit 22, and the feature amount extraction processing result (color edge pair) Information) and the image ID are passed to the telop character string region shape acquisition unit 24 (step 12).

  Subsequently, the telop character string region shape acquisition unit 24 acquires the telop character string region shape information corresponding to the image ID from the data storage unit 21 using the image ID received from the feature amount extraction unit 23, and the telop character string. The region shape information and the feature amount extraction result are passed to the density calculation information acquisition unit 25 (step 13).

  The telop character string region shape information is information indicating the shape of the telop character string region to be extracted. In the present embodiment, a rectangle (rectangle) is used as the telop character string region. In the telop character string region shape information, the rectangle that includes (circumscribes) the telop character string region to be extracted is either vertically long or horizontally long. It is information indicating whether or not. The telop character string region shape information is, for example, 1-bit information indicating “vertical” or “horizontal”.

  Subsequently, the congestion degree calculation information acquisition unit 25 uses the telop character string region shape information received from the telop character string region shape acquisition unit 24 to transmit the congestion corresponding to the telop character string region shape information from the data storage unit 21. The degree calculation area shape information is acquired, and the density calculation area shape information and the feature amount extraction result are passed to the telop character string area extraction unit 26 (step 14). The density calculation area shape information is information indicating the shape of a sub-block (local area) used for density calculation, which will be described later, and is information having the number of pixels in the vertical direction and the number of pixels in the horizontal direction, for example. . In this embodiment, when the telop character string area shape information is information indicating “horizontal”, the sub-block is a rectangle that is horizontally long, and the telop character string area shape information indicates “vertical”. In the case of information, it is assumed to be a vertically long rectangle. When the sub-block is a horizontally long rectangle, the rectangle is, for example, a rectangle having a ratio of 1 × 10 in the vertical direction, for example, a rectangle having 10 pixels in the vertical direction × 100 pixels in the horizontal direction. By adopting such a sub-block, it is possible to calculate the density with priority given to the continuity of the feature amount in the direction in which the telop character strings continue, and the extraction accuracy of the telop character string region can be improved. .

  Subsequently, the telop character string region extraction unit 26 performs a telop character string region extraction process (step 15). Here, the telop character string region extraction unit 26 applies a sub-block corresponding to the density calculation region shape information to each pixel of the feature amount extraction result (an image in which each pixel value is a color edge value). Then, the sum of the feature values (color edge values) of each pixel included in the sub-block is calculated. This sum is called the density for each pixel. That is, when attention is paid to a certain pixel, the sum of the feature values of each pixel included in the sub-block centered on the target pixel is the density of the target pixel.

  This density is obtained for all pixels of the feature amount extraction result. In addition, it is good also as calculating | requiring for every pixel of the predetermined space | interval instead of calculating | requiring a density about all the pixels. Alternatively, the density may be obtained only for a pixel at a predetermined position excluding a portion where it is known that a telop character string does not exist.

  Then, the telop character string region extraction unit 26 further performs clustering processing on the density calculation result, and detects a region where pixels having a density higher than a certain threshold are gathered in a certain area or more. When the number of pixels forming a cluster (a group of pixels having a density higher than a certain threshold) is equal to or greater than a predetermined threshold, the cluster is determined to be a telop character string region.

  Subsequently, the telop character string area extraction unit 26 performs a shaping process such as obtaining a circumscribed rectangle for the area determined to be a telop character string area, and the telop character string area after the shaping process is converted into a telop character string. This is the column area extraction result. The telop character string region extraction result includes information indicating the contour of the region after shaping, for example, the coordinate values of four vertices in the case of a rectangle (rectangle).

  Although this telop character string region extraction result may be output as it is, in this embodiment, in order to obtain the telop character string region more precisely, the region end precision extraction processing (hereinafter referred to as “precision”) is performed using multi-value gradient weights. (Referred to as extraction processing) (step 16). This precise extraction process will be described with reference to the flowchart of FIG.

  In the following, a case where the telop character string region shape information is information indicating “horizontal” will be described as an example. Further, it is assumed that the coordinate value of the upper left vertex of the telop character string area obtained in step 15 of FIG. 8 is (Xl, Yu) and the coordinate value of the lower right vertex is (Xr, Yd). At this time, the telop character string region extraction result, the feature amount extraction result, the telop character string region shape information, and the like are held in a storage device such as a memory. In the following processing, necessary data is appropriately stored from the storage device. It is read and processed.

  In the precise extraction process, first, the density calculation information acquisition unit 25 acquires weight coefficient distribution information corresponding to the telop character string region shape information from the data storage unit 21 (step 61).

  In the present embodiment, when the telop character string region shape information is information indicating “horizontal”, the left and right end portions of the telop character string region (horizontal rectangle) extracted in step 15 of FIG. When the telop character string region shape information is information indicating “vertical”, the region ending precise extraction processing is performed on the upper end portion and the lower end portion. Therefore, as shown in FIG. 7B, the data storage unit 21 stores the weight coefficient distribution information corresponding to the left end portion and the weight coefficient distribution corresponding to the right end portion with respect to the telop character string region shape information “horizontal”. Information is stored, and for the telop character string region shape information “vertical”, weight coefficient distribution information corresponding to the upper end portion and weight coefficient distribution information corresponding to the lower end portion are stored.

  In this example, since the telop character string region shape information is “horizontal”, in step 61, the density calculation information acquisition unit 25 performs weight coefficient distribution information corresponding to the left end portion and weight coefficient distribution corresponding to the right end portion. Information is acquired from the data storage unit 21.

  When the telop character string area shape information is “horizontal”, the same number of pixels as the vertical direction of the telop character string area is used as the vertical extraction pixel area, and the weighting factor is used as the horizontal pixel number. An area having the number of pixels specified by the distribution information is used. In other words, when the telop character string area shape information is “horizontal”, the weight coefficient distribution information includes the horizontal length (number of pixels, w) of the precision extraction processing area and the weight distribution in the horizontal direction. Contains information. For example, in the weighting factor distribution information, the length (number of pixels) in the horizontal direction of the precise extraction processing region is 5 pixels, and the weighting factor of the first pixel position (pixel 1) counted from the left is 5, It includes information such as a weighting factor of 4, a third weighting factor of 3, a fourth weighting factor of 2, a fifth weighting factor of 1, and the like.

Hereinafter, based on the example of the weighting factor distribution information, the region end precise extraction processing of the left end portion will be described. It should be noted that the area end precise extraction process at the right end part is also performed, but is the same as the area end precise extraction process at the left end part (only the direction is reversed). Explain.
The weight coefficient distribution information corresponding to the left end portion acquired by the density calculation information acquisition unit 25 in step 61 of FIG. 9 is passed to the telop character string region extraction unit 26.

  The telop character string region extraction unit 26 has the same number of pixels in the vertical direction as the vertical direction of the telop character string region using the weight coefficient distribution information, and is the same as the number of pixels specified by the weight coefficient distribution information. A precise extraction processing region having the number of pixels as the number of pixels in the horizontal direction is provided, and the weighting factor distribution in the precise extraction processing region is set using the weighting factor distribution information so that the left side weight becomes heavy (step 62). ). Note that when performing the right-side precision extraction, the weighting coefficient distribution in the precision extraction processing region is set so that the right-side weight becomes heavier. That is, weighting is performed so that the weight on the telop character string region end side becomes larger than that on the telop character string region center side.

  FIG. 10 is a diagram for explaining the precise extraction process in this example, and 602 in the upper part of FIG. 10 shows an example of the weight coefficient distribution in the precise extraction process region. Further, reference numeral 603 in FIG. 10 denotes a precision extraction processing area having a horizontal length of 5 in this example (in the precision extraction process, the weighted density is obtained while moving this area in the horizontal direction). Is shown. Reference numeral 601 denotes a set of color edge points constituting the color edge pair.

  When the number of horizontal pixels of the sub-block used for the extraction of the telop character string region is b, the telop character string region extraction unit 26 uses, for example, the coordinate value of the upper left corner of the precise extraction processing region as (Xl-b / 2, Yu) is the product of the feature value (color edge value) of each pixel in the feature value image to be processed and the weight value at the pixel position corresponding to the precise extraction processing region. It calculates with respect to all the pixels in the inside, and calculates the sum (step 63). This sum is called weighted congestion.

  The telop character string region extraction unit 26 performs weighting while moving the coordinate value of the upper left corner of the precise extraction processing region from (Xl−b / 2, Yu) to (Xl + b / 2, Yu) by the width of one pixel. The degree of congestion is calculated sequentially (step 63). In the above example, the starting point is (Xl−b / 2, Yu) and the ending point is (Xl + b / 2, Yu). For example, the weighted density from (Xl−b, Yu) to (Xl + b, Yu) may be calculated sequentially.

  Then, the telop character string region extraction unit 26 calculates the coordinate value of the upper left corner of the precise extraction processing region (X0, Yu) when the weighted density value exceeds a preset threshold value (or exceeds a threshold value). ), And the coordinate value on the left side of the telop character string area at this time is changed from Xl to X0 (step 64). That is, the left end of the telop character string area is set as the left end position of the precision extraction processing area at this time. In the lower part of FIG. 10, the weighted density calculated while shifting the precision extraction processing area to the side is displayed in the form of a graph, and it is shown that the weighted density exceeds the threshold at the position 605. ing.

  The telop character string region extraction unit 26 uses the telop character string region information and the image ID after performing the precise extraction of the top and bottom or left and right edges for all the telop character string regions obtained in step 15 of FIG. The data is passed to the input / output unit 27, and the input / output unit 27 outputs the information (step 17 in FIG. 8).

  In addition to performing precise extraction of the left and right edges and upper and lower edges, for example, when precisely detecting the lower left corner edge, the lower left weight is increased in the precision extraction processing area, and the weight is increased as the distance from the lower left is increased. By using a reduced weight distribution, it is possible to detect the change in the feature quantity at the lower left corner with high sensitivity and to detect the end of the lower left corner with high accuracy.

  In the above-described example, the telop character string area shape information is shown as an example of processing in the case where the data storage unit 21 stores the telop character string area shape information in advance. Area shape information may be acquired. For example, several directions (for example, a vertical direction and a horizontal direction) are determined in advance, and after extracting a telop character string region using a local region having a shape corresponding to each direction, a shape corresponding to which direction is determined. The telop character string region shape information may be acquired by determining whether the result extracted using the local region is the best result and outputting the direction as the telop character string region shape information.

(Regarding the effect of the embodiment of the present invention)
In the prior art, the luminance is used to extract the edge, and thus the extraction of the luminance edge pair may fail when the luminance of the background and the line figure is similar. In the embodiment of the present invention, the luminance is similar. Color information is actively used to detect color differences, color edges that are characteristic of color information discontinuously changing are extracted, and color conditions are applied by applying pairing criteria. Edge pairs can be extracted.

  That is, by actively using color information that is two-dimensional or higher (in many cases, three-dimensional) high-dimensional information, it is possible to separate the line figure and the background even when the luminance is similar.

  In addition, when creating a luminance edge pair in the prior art, processing was performed so as to pair an ascending edge and a descending edge using a luminance gradient direction, but color information that is high-dimensional information is used. Since it is difficult to define the gradient up and down, a conventional method cannot determine a pair. Therefore, in the present embodiment, a pairness determination condition for creating a color edge pair is newly introduced, whereby a pair can be created for a color edge.

  As described above, in this embodiment, by using color information instead of luminance, edge pairs (color edge pairs) can be extracted even in the case of backgrounds with similar luminance, and thereby it is possible to extract a line figure region. It is possible to separate the line figure area from the background area.

  The present invention is not limited to the above-described embodiments, and various modifications and applications are possible within the scope of the claims.

2 is a functional configuration diagram of the image feature extraction device 1. FIG. It is a figure which shows the example of the storage method to the image storage part 11 of image data. 4 is a flowchart for explaining the operation of the image feature extraction apparatus 1. It is a figure for demonstrating the example of edge extraction. It is a figure for demonstrating pair property determination conditions. 3 is a functional configuration diagram of a telop character string region extraction device 20. FIG. 4 is a diagram illustrating an example of data stored in a data storage unit 21. FIG. 4 is a flowchart for explaining the operation of the telop character string region extraction device 20. It is a flowchart of a precision extraction process. It is a figure for demonstrating the example of a precision extraction process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image feature extraction apparatus 11 Image storage part 12 Image acquisition part 13 Color edge extraction part 14 Color edge pair extraction part 15 Pair property determination part 16 Input / output part 20 Telop character string area extraction apparatus 21 Data storage part 22 Image acquisition part 23 Feature value Extraction unit 24 Telop character string region shape acquisition unit 25 Information acquisition unit for density calculation 26 Telop character string region extraction unit 27 Input / output unit

Claims (6)

An image feature extraction device for extracting a color edge pair from an image including a line figure,
Image acquisition means for acquiring the image from the image storage means;
Color edge extracting means for extracting a color edge from the image as an absolute value ;
A color edge pair candidate set is extracted from the color edges extracted by the color edge extraction means, and color edge pair candidates satisfying the color edge pair extraction condition based on the property that the line figure has a locally similar color comprising a color edge pair extracting means for extracting from the color edge pair candidate set as the edge pair, and
The color edge pair extraction condition is:
In the first color edge and the second color edge constituting the color edge pair candidate, among the pixels around the first color edge, the color of the pixel closer to the second color edge; 2. An image feature extraction apparatus according to claim 1, wherein the image feature extraction device has a condition that among the pixels around the second color edge, the color of the pixel closer to the first color edge is similar . An image feature extraction device for extracting a color edge pair from an image including a line figure,
Image acquisition means for acquiring the image from the image storage means;
Color edge extraction means for extracting a color edge from the image;
A color edge pair candidate set is extracted from the color edges extracted by the color edge extraction means, and color edge pair candidates satisfying the color edge pair extraction condition based on the property that the line figure has a locally similar color comprising a color edge pair extracting means for extracting from the color edge pair candidate set as the edge pair, and
The color edge pair extraction condition is:
In the first color edge and the second color edge constituting the color edge pair candidate, among the pixels around the first color edge, the color of the pixel closer to the second color edge; A condition that among the pixels around the second color edge, the color of the pixel on the side closer to the first color edge is similar,
A condition that a color change vector in the vicinity of the first color edge and a color change vector in the vicinity of the second color edge are substantially in opposite directions; and
A condition that the amount of change in color between a plurality of pixels between the first color edge and the second color edge is small.
Including the following three conditions:
The image feature extraction apparatus, wherein the color edge pair extraction unit extracts a color edge pair candidate satisfying at least one of the three conditions as a color edge pair from the color edge pair candidate set . An image feature extraction method executed by an image feature extraction apparatus for extracting a color edge pair from an image including a line figure,
An image acquisition step of acquiring the image from an image storage means;
A color edge extraction step for extracting a color edge from the image as an absolute value ;
A color edge pair candidate set is extracted from the color edges extracted in the color edge extraction step, and color edge pair candidates satisfying a color edge pair extraction condition based on the property that the line figure has a locally similar color comprising a color edge pair extracting from the color edge pair candidate set as the edge pair, and
The color edge pair extraction condition is:
In the first color edge and the second color edge constituting the color edge pair candidate, among the pixels around the first color edge, the color of the pixel closer to the second color edge; An image feature extraction method characterized in that the condition is that the color of a pixel near the first color edge among pixels around the second color edge is similar . An image feature extraction method executed by an image feature extraction apparatus for extracting a color edge pair from an image including a line figure,
An image acquisition step of acquiring the image from an image storage means;
A color edge extraction step of extracting a color edge from the image;
A color edge pair candidate set is extracted from the color edges extracted in the color edge extraction step, and color edge pair candidates satisfying a color edge pair extraction condition based on the property that the line figure has a locally similar color comprising a color edge pair extracting from the color edge pair candidate set as the edge pair, and
The color edge pair extraction condition is:
In the first color edge and the second color edge constituting the color edge pair candidate, among the pixels around the first color edge, the color of the pixel closer to the second color edge; A condition that among the pixels around the second color edge, the color of the pixel on the side closer to the first color edge is similar,
A condition that a color change vector in the vicinity of the first color edge and a color change vector in the vicinity of the second color edge are substantially in opposite directions; and
A condition that the amount of change in color between a plurality of pixels between the first color edge and the second color edge is small.
Including the following three conditions:
In the color edge pair extraction step, the image feature extraction device extracts a color edge pair candidate satisfying at least one of the three conditions as a color edge pair from the color edge pair candidate set. Image feature extraction method.   A program for causing a computer used as the image feature extraction apparatus to execute each step in the image feature extraction method according to claim 3 or 4.   A computer-readable recording medium storing the program according to claim 5.

Images