JP2016151978A - Image processing apparatus and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus that can more accurately specify partial images describing one character compared with a case of specifying the partial images by using information indicating only characterness in areas.SOLUTION: An image processing apparatus comprises: detection means that detects partial images including characteristics as a character from images as character candidate areas; extraction means that extracts the inclusion relationship between the partial images; and specification means that specifies the partial images describing one character by using information indicating characterness in areas in the partial images and the inclusion relationship.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus and an image processing program.

  Patent Document 1 aims to provide a method for determining a character area of an image that can accurately determine a character area even if the image data includes a background image other than a character string and the character quality is poor. , Input an image including a character string composed of characters of the same size and having a predetermined arrangement, and perform binarization processing, and among all the connected pixels included in the processing data, the characteristic is a predetermined condition And a circumscribed rectangular region including all the character candidates whose positional relationship between the character candidates forms the predetermined arrangement is determined as the character region of the image, It is disclosed that a character region can always be accurately determined by the above configuration.

  In Patent Document 2, a scene including characters existing in a two-dimensional plane or a three-dimensional space is input as a grayscale image, and it is possible to correspond to a character line by area division processing of the scene image and density difference evaluation of each area. A high part is detected as a region, a combination of regions existing in the vicinity is extracted as a character pattern candidate, and the character pattern candidate and a standard pattern of each recognition target character category stored in the character recognition dictionary The similarity is calculated from the distance of the character, and it is determined whether or not the character pattern candidate corresponds to a character based on whether or not there is a category that gives a similarity greater than a threshold value. If the character pattern candidate is a character, the character pattern candidate determined as a character is extracted as a high similarity pattern, and the category that gives the highest similarity in the high similarity pattern is taken as the character recognition result. In the character recognition processing method, for a set composed of high similarity patterns obtained from character pattern candidates obtained by combinations of the regions existing in the vicinity, categories corresponding to the high similarity patterns included in the set The degree of coincidence between the feature as a two-dimensional figure possessed by the standard pattern and the feature as the two-dimensional figure possessed by each item similarity pattern is compared between the high similarity patterns included in the set, and the degree of coincidence The high-similarity pattern with a high degree of similarity is increased, and the renewal process that lowers the degree of similarity is not performed repeatedly. Finally, the high-similarity pattern that gives the maximum similarity is extracted as a character pattern. It is disclosed that a category that gives the maximum similarity is output as a character recognition result.

  In Non-Patent Document 1, there are cases where characters in a scene have lost the image characteristics inherent to the characters due to the influence of shadows, reflections, etc., and it is sometimes difficult to determine the character / non-character in the candidate area alone. A character candidate is identified by combining identification using features between regions such as a candidate region triplet and a candidate region triplet following character / non-character determination in a single candidate region. It is disclosed that the determination is made in consideration of the inclusion relationship between the areas in the narrowing down and the identification by the candidate area pair.

JP 08-339421 A Japanese Patent No. 2979089

L. Neumann, J.M. Matas, “Text Localization in Real-World Images using Efficiently Pruned Exclusive Search”, in ICDAR 2011, p. 687-691

When specifying a partial image in which one character is described from an image, information indicating the character of a region is used.
The present invention provides an image processing apparatus and an image processing program that can be specified more accurately than when a partial image in which one character is described is simply specified using information indicating the character of an area. The purpose is to do.

The gist of the present invention for achieving the object lies in the inventions of the following items.
According to the first aspect of the present invention, there is provided a detecting means for detecting a partial image having character characteristics from an image as a character candidate area, an extracting means for extracting an inclusion relationship between the partial images, and a character in an area in the partial image. An image processing apparatus comprising: specifying means for specifying a partial image in which one character is described using information indicating the likelihood and the inclusion relation.

  According to a second aspect of the present invention, the extraction unit constructs a tree structure indicating the inclusion relation, and the specifying unit extracts a feature of a region in the partial image from the image and the partial image. Means for calculating information indicating the character likeness of the area in the partial image from the features extracted by the second extracting means, information indicating the character likeness calculated by the calculating means, and the tree The image processing apparatus according to claim 1, further comprising a second specifying unit that specifies a partial image in which one character is described using an inclusion relation in the structure.

  The invention according to claim 3 is characterized in that the second specifying means selects one of the nodes from the root to the leaf in the tree structure having the highest information indicating the character character of the region in the partial image corresponding to each node. The image processing apparatus according to claim 2, wherein the image processing apparatus is specified as a partial image in which characters are described.

  The invention according to claim 4 is characterized in that the second specifying means deletes the parent and child nodes of the node specified as a partial image in which one character is described. It is a processing device.

  According to a fifth aspect of the present invention, there is provided a computer for detecting, as a character candidate area, a partial image having a character feature from an image, an extracting unit for extracting an inclusion relationship between the partial images, An image processing program for functioning as a specifying unit that specifies a partial image in which one character is described, using information indicating the character of an area and the inclusion relation.

  According to the image processing apparatus of the first aspect, more accurate identification can be performed as compared with the case where the partial image in which one character is described is simply identified using the information indicating the character of the area.

  According to the image processing apparatus of the second aspect, it is possible to specify a partial image in which one character is described using the inclusion relation in the tree structure.

  According to the image processing apparatus of claim 3, in the nodes from the root to the leaf in the tree structure, one character is described that has the highest information indicating the character character of the area in the partial image corresponding to each node. Can be identified as a partial image.

  According to the image processing apparatus of the fourth aspect, the parent and child nodes of the node specified as the partial image in which one character is described can be deleted.

  According to the image processing program of the fifth aspect, more accurate identification can be performed as compared with the case where the partial image in which one character is described is simply identified using the information indicating the character of the area.

It is a conceptual module block diagram about the structural example of this Embodiment. It is explanatory drawing which shows the system configuration example using this Embodiment. It is explanatory drawing which shows the system configuration example using this Embodiment. It is a conceptual module block diagram about the structural example of this Embodiment (character area specific | specification module). It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is a flowchart which shows the process example by this Embodiment (inclusion relation correction module). It is a block diagram which shows the hardware structural example of the computer which implement | achieves this Embodiment.

Hereinafter, an example of a preferred embodiment for realizing the present invention will be described with reference to the drawings.
FIG. 1 shows a conceptual module configuration diagram of a configuration example of the present embodiment.
The module generally refers to components such as software (computer program) and hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Therefore, the present embodiment is a computer program for causing these modules to function (a program for causing a computer to execute each procedure, a program for causing a computer to function as each means, and a function for each computer. This also serves as an explanation of the program and system and method for realizing the above. However, for the sake of explanation, the words “store”, “store”, and equivalents thereof are used. However, when the embodiment is a computer program, these words are stored in a storage device or stored in memory. It is the control to be stored in the device. Modules may correspond to functions one-to-one, but in mounting, one module may be configured by one program, or a plurality of modules may be configured by one program, and conversely, one module May be composed of a plurality of programs. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include other modules. Hereinafter, “connection” is used not only for physical connection but also for logical connection (data exchange, instruction, reference relationship between data, etc.). “Predetermined” means that the process is determined before the target process, and not only before the process according to this embodiment starts but also after the process according to this embodiment starts. In addition, if it is before the target processing, it is used in accordance with the situation / state at that time or with the intention to be decided according to the situation / state up to that point. When there are a plurality of “predetermined values”, they may be different values, or two or more values (of course, including all values) may be the same. In addition, the description having the meaning of “do B when it is A” is used in the meaning of “determine whether or not it is A and do B when it is judged as A”. However, the case where it is not necessary to determine whether or not A is excluded.
In addition, the system or device is configured by connecting a plurality of computers, hardware, devices, and the like by communication means such as a network (including one-to-one correspondence communication connection), etc., and one computer, hardware, device. The case where it implement | achieves by etc. is included. “Apparatus” and “system” are used as synonymous terms. Of course, the “system” does not include a social “mechanism” (social system) that is an artificial arrangement.
In addition, when performing a plurality of processes in each module or in each module, the target information is read from the storage device for each process, and the processing result is written to the storage device after performing the processing. is there. Therefore, description of reading from the storage device before processing and writing to the storage device after processing may be omitted. Here, the storage device may include a hard disk, a RAM (Random Access Memory), an external storage medium, a storage device via a communication line, a register in a CPU (Central Processing Unit), and the like.

The image processing apparatus 100 according to the present embodiment specifies a partial image in which one character is described from an image. As shown in the example of FIG. 1, the image receiving module 110, character candidate area detection A module 120, a character area specifying module 130, and a character recognition module 140 are provided.
The image processing apparatus 100 is mainly for performing character recognition in a scene image. That is, a character area from a still image or a moving image (hereinafter also referred to as a scene image) taken by a camera (a camera built in a portable terminal (including a wearable terminal) including a digital camera or a mobile phone). This invention relates to a technique for identifying characters and performing character recognition. For example, the scene image includes a landscape image including a character image such as a signboard. In particular, the image processing apparatus 100 relates to a technique for specifying a character area from a scene image. In addition, you may make it use for processes (for example, image restoration process etc.) other than character recognition.
In addition, the processing result of the image processing apparatus 100 is used for, for example, navigation, content-based image search, translation of guidance in a scene image, assistance for the visually impaired, extraction of point information from the scene image, and semantic tagging of the scene image Further, it may be used for processing such as automation of visual confirmation work.
In addition, when performing character recognition in a scene image, the scene image as a target has the following unique properties. In particular, it mentions properties that are not in document character recognition.
-It is necessary to specify a character area from a complicated scene.
・ There may be shadows, reflections, reflections on glass, etc.
-There are various fonts and layouts.
-Perspectives and characters on curved surfaces such as cylinders may be targeted.
Because of such a property, a high character score (information indicating character character) is not always obtained from an original character region in a scene image.

In describing this embodiment, terms are defined.
The tree structure is a data structure having a tree structure of graph theory. In graph theory, a tree means an acyclic (no loop) graph.
The tree structure is represented by edges connecting nodes. In most cases, the tree used as a data structure is a rooted tree in which a root node (root) is determined, and the relationship between the nodes is expressed in terms similar to a family tree. Each node in the tree structure has zero or more child nodes, and the child nodes exist downward in the tree structure (generally, the growth direction of the tree structure is downward). A node having a child node is a parent node when viewed from the child node. A node has at most one parent node.

  The image reception module 110 is connected to the character candidate area detection module 120 and the character area identification module 130, and passes the scene image information 115 to the character candidate area detection module 120 and the character area identification module 130. The image reception module 110 receives an image including a character image and passes the image to the character candidate region detection module 120 and the character region identification module 130. Here, accepting an image means, for example, reading an image with a camera, a scanner, etc., receiving an image from an external device via a communication line by fax or the like, a hard disk (in addition to what is built in a computer, For example, reading out an image stored in a network (including those connected via a network). The image is a binary or multivalued image (including a color image). One image may be received or a plurality of images may be received. Further, the content of the image is mainly the above-described scene image.

The character candidate area detection module 120 is connected to the image reception module 110 and the character area identification module 130, and passes the character candidate area information 125 to the character area identification module 130. The character candidate area detection module 120 detects a partial image having a character feature from the image received by the image receiving module 110 as a character candidate area. The detection result is passed to the character area specifying module 130 as character candidate area information 125. Examples of characteristics as characters include high contrast with the background and small changes in shading within the character area. As a specific example, the present embodiment is a Maximal Stable Extreme Region (MSER) that has recently attracted particular attention as a robust technique for shadows, reflections, and gradation characters among detection techniques that use these features. ) Is used. MSER is defined as shown in Equation (1) (“J. Matas, O. Chum, M. Urban, T. Pajdla,“ Robest wide-baseline stereo maximum stable extent regi sions 22 ”. , P. 761-767, 2004).

MSER (contents of processing by the character candidate region detection module 120 (processing for generating the character candidate region information 125)) will be described with reference to FIGS.
(1) A target image 500 (grayscale image) as shown in the example of FIG. 5 is set as a processing target (scene image information 115) by the character candidate area detection module 120. The target image 500 is divided into three areas: a rectangular area 510 that is substantially black, a rectangular area 530 that is substantially white, and a gray gradation area 520 that is thinned from left to right.
(2) The target image 500 is binarized while moving the binarization threshold from the darkest pixel value toward the brightest pixel value.
(3) With the pixel value having the darkest threshold value, all the pixels have pixel values equal to or greater than the threshold value, so that no region is detected as shown in the example of FIG.

(4) The threshold value is moved, and when the threshold value exceeds the pixel value of the region 510 in the example of FIG. 5, the region 510 is detected. As shown in the example of FIG. 7, the first MSER (region 510A) Is detected.
(5) Further, when the threshold value is moved to the bright side, the vicinity of the darkest part of the region 520 is detected. However, if the threshold value is changed by ± Δ, the area of the detected region also changes greatly, and therefore the region near the darkest part of the region 520 is not detected from the definition of MSER.
(6) When the threshold value becomes brighter than the brightest part of the region 520, the area of the detected region does not change even if the threshold value is changed. Therefore, the region 520 is detected at this stage, and as shown in the example of FIG. The second MSER (area 520A), which is the sum area of the areas 510 and 520, is detected. The second MSER (area 520A) includes the first detected MSER (area 510A).
(7) The threshold value is further moved to the brighter side, and the region 530 is detected when it becomes brighter than the region 530, and the third MSER (target image 500A) is detected as shown in the example of FIG. . The third MSER (target image 500A) includes the first and second detected MSERs (region 510A, region 520A).

(8) Next, the same processing is performed while moving the threshold value from the brightest pixel value toward the darkest pixel value. That is, the process proceeds in the opposite direction to that described above. This is to detect white characters and the like.
(9) The detection results are shown in the examples of FIGS. Similar to the case where the threshold value is changed from dark to light, the detected MSERs are inclusive of each other.
The nature of MSER will be described.
The MSER when the binarization threshold is changed from dark to light and the MSER when the threshold value is changed from light to dark have an inclusive relationship.
The MSER detected when the threshold value is changed from dark to light does not have an inclusive relationship with any of the MSER values when the threshold value is changed from light to dark. However, the MSER in which the entire image is detected is excluded. The same applies to the reverse case.
A region with gradation can be detected as in the region 520 shown in the example of FIG.

  The character area specifying module 130 is connected to the image receiving module 110, the character candidate area detecting module 120, and the character recognizing module 140, and the scene image information 115 is received from the image receiving module 110 and the character candidate area is transferred from the character candidate area detecting module 120. The information 125 is received and the specific character area information 135 is passed to the character recognition module 140. The character region specifying module 130 extracts an inclusion relationship between the partial images, and uses the information indicating the character of the region in the partial image (hereinafter also referred to as a character score) and the inclusion relationship, The described partial image is specified. The candidate character region that is the processing result of the candidate character region detection module 120 includes regions other than characters. When MSER is used, a plurality of types of region candidate groups may be extracted from one character (described later with reference to FIGS. 14 to 16). Character / non-character determination is performed for each of such character candidate areas to identify the character area. The module configuration, processing, and the like in the character area specifying module 130 will be described later with reference to FIG.

  The character recognition module 140 is connected to the character area specifying module 130 and receives specific character area information 135 from the character area specifying module 130. The character recognition module 140 performs character recognition on the character area image (specific character area information 135) specified by the character area specifying module 130. A known method may be used as the character recognition method.

FIG. 2 is an explanatory diagram showing a system configuration example using the present embodiment.
The image processing apparatus 100 is connected to a photographing apparatus 210 and an information processing apparatus 230. This system may be housed in an integrated housing or may be a separate housing. Communication between the devices may be wireless, wired, or a combination thereof.
The imaging device 210 is connected to the image processing device 100. The photographing device 210 is the above-described camera, for example, may be one that is photographed by a person, or may be mounted on a car or the like, or installed in a fixed place such as a surveillance camera. It may be what has been done.
The information processing device 230 is connected to the image processing device 100. The information processing device 230 performs processing using the processing result of the image processing device 100 (character recognition module 140). For example, this system is mounted on an automobile or the like, and processing such as navigation is performed as described above. Further, the information processing apparatus 230 may perform processing using the processing result by the character area specifying module 130. For example, as described above, processing such as image restoration is performed.

FIG. 3 is an explanatory diagram showing a system configuration example using the present embodiment.
The image processing apparatus 100, the user terminal 310A, the user terminal 310B, the user terminal 310C, and the information processing apparatus 330 are connected via a communication line 390, respectively. The communication line 390 may be wireless, wired, or a combination thereof, and may be, for example, the Internet or an intranet as a communication infrastructure.
The information processing device 330 performs processing equivalent to that of the information processing device 230 described above. The functions of the image processing apparatus 100, the information processing apparatus 330, and the combination of the image processing apparatus 100 and the information processing apparatus 330 may be realized as a cloud service.
The user terminal 310 performs processing equivalent to that of the above-described photographing apparatus 210. For example, an image captured by the user terminal 310 may be processed by the image processing apparatus 100. Further, processing may be performed by the image processing apparatus 100 and the information processing apparatus 330 according to an instruction from the user terminal 310A, and the processing result may be transmitted to the user terminal 310B or the like.

FIG. 4 is a conceptual module configuration diagram of a configuration example of the present embodiment (character area specifying module 130).
The character area specifying module 130 includes an inclusion relation construction module 410, a character area feature extraction module 420, a character score calculation module 430, and an inclusion relation correction module 440.

The scene image information 115 is information (which may be the scene image itself) related to the scene image output from the image receiving module 110, and is color (including a grayscale image) or monochrome raster image information. In the case of color, it may be RGB or an image in another color space such as L * a * b *. In the present embodiment, the case where only one specific channel (for example, only the R image of the RGB image) is used as an example of processing when the input image is in color will be described. The results may be integrated by specifying a region and finally performing a logical sum.
The character candidate area information 125 is a processing result by the character candidate area detection module 120 and is expressed by contour pixel position information, in-area pixel position information, and the like of each candidate area. In this embodiment, a list of pixel coordinates detected as individual MSERs is referred to as MSER information, and a set of all MSER information detected from the entire target image is character candidate area information. 125.

The inclusion relationship construction module 410 is connected to the inclusion relationship correction module 440 and receives the character candidate area information 125. The inclusion relationship construction module 410 extracts inclusion relationships between partial images. Further, the inclusion relationship construction module 410 may construct a tree structure indicating the inclusion relationship.
MSER is characterized in that a certain region includes another region and a tree structure is formed when such inclusion relationships are linked (described above with reference to FIGS. 5 to 13). Therefore, specifically, the inclusion relationship construction module 410 checks the inclusion relationship for all MSERs in the character candidate area information 125 and constructs a plurality of tree structures. Here, there are at least two tree structures as a plurality of tree structures. For example, as described above, there are a tree structure generated by changing the threshold value for binarization from dark to light, and a tree structure generated by changing the threshold value from light to dark.

An example of a tree structure will be described with reference to FIGS.
As the nature of MSER, a plurality of MSERs may be detected from one character. For example, an original character image (for example, a character image written on a signboard or the like) is assumed to be a binary image as shown in the example of FIG. A scene image shot by a camera is a multi-valued image as shown in the example of FIG. 15 even if it was originally a binary image due to the influence of shadows, reflections, uneven illumination, camera shake, blurring, etc. .
As a result, a plurality of MSERs may be detected from one character as shown in the example of FIG.
Since it is MSER, each detection area has an inclusive relationship represented by the tree structure shown in the example of FIG.

The character area feature extraction module 420 is connected to the character score calculation module 430 and receives the scene image information 115 and the character candidate area information 125. The character region feature extraction module 420 extracts the feature of the region in the partial image from the target image (scene image information 115) and the partial image (character candidate region information 125) that is the processing result of the character candidate region detection module 120. Extract. Specifically, the character area feature extraction module 420 determines the character of the area from the information (character candidate area information 125) regarding each MSER indicated by the character candidate area information 125 and the scene image information 115 corresponding to each MSER. A plurality of feature quantities reflecting the uniqueness are extracted, and a feature vector in which the feature quantities are arranged (a feature vector having the feature quantity as a vector element) is generated. As a feature of the character image,
(1) The change in color and shade in the character area is small (2) The contrast with the background is high (3) Simple outline (4) Consists of line segments of a certain width, etc. And as an example of the feature value that reflects the character of the area,
(1) aspect ratio
(2) compactness
(3) Convex hull area to surface ratio
(4) background color consistency
(5) relative segment height
(6) number of holes
(7) character color consistency
(8) skeleton length to perimeter ratio
(See L. Neumann, J. Matas, “A Method for Text Localization and Recognition in Real-World Images”, in ACCV 2010, p. 770-783, 2010).
The character area feature extraction module 420 sets a vector in which feature amounts (real values) are arranged as a feature vector. In the above example, an 8-dimensional vector is generated.
The feature vector may include not only the feature quantity extracted from the attention area as described above but also the feature quantity extracted based on the relationship with the neighboring area. For example, an average color difference from the nearest region, a size ratio, or the like may be used as the feature amount.

ただし、式（２）において、σはシグモイド関数、ｈ_ｊはｊ番目の中間ノード出力、ｕ_ｊはｊ番目の中間ノードと出力ノードのリンクの重み、ｗ_ｉｊはｉ番目の入力ノードとｊ番目の中間ノードのリンクの重み、Ｘ_ｉはベクトルＸのｉ番目の成分、Ｉは入力次元数、Ｊは中間ノード数を表す。
パラメータｕ及びパラメータｗは、学習により求める。具体的には、文字領域の特徴ベクトル（正例）と非文字領域の特徴ベクトル（負例）を学習データとして多数用意し、ニューラルネットワークが、それぞれｙ＝１、ｙ＝０になるべく近い値を出力するように最尤学習を行う。 The character score calculation module 430 is connected to the character region feature extraction module 420 and the inclusion relationship correction module 440. The character score calculation module 430 calculates information (character score) indicating the character character of the region in the partial image from the features extracted by the character region feature extraction module 420.
Specifically, the character score calculation module 430 receives the feature vector generated by the character region feature extraction module 420 and calculates a score whose corresponding MSER is a character region. For example, the character score may be a posterior probability of a character, and the posterior probability may be calculated by a neural network. More specifically, as an example of the character score, the posterior probability p (y | x) regarding the label y of the corresponding region when the feature vector x generated by the character region feature extraction module 420 is given is used. Can do. The label is binary (y = {0, 1}), where y = 0 represents a non-character and y = 1 represents a character.
The posterior probability p (y | x) can be obtained as an output of a neural network (multilayer perceptron having one intermediate layer) as shown in the example of FIG.
A feature vector is a d-dimensional real vector x = (x ₁ , x ₂ ,..., X _d ), and a vector X = (x ₁ , x ₂ ,..., X _d , 1) with a constant element 1 added thereto is neuralized. As input to the network. At this time, the posterior probability p (y | x) is obtained as shown in the equation (2) as an output of the neural network.

In Equation (2), σ is a sigmoid function, h _j is the j-th intermediate node output, u _j is the link weight of the j-th intermediate node and the output node, w _ij is the i-th input node and the j-th , X _i is the i-th component of vector X, I is the number of input dimensions, and J is the number of intermediate nodes.
The parameter u and the parameter w are obtained by learning. Specifically, a large number of character vector feature vectors (positive examples) and non-character region feature vectors (negative examples) are prepared as learning data, and the neural network sets values as close as possible to y = 1 and y = 0, respectively. Perform maximum likelihood learning to output.

The inclusion relationship correction module 440 is connected to the inclusion relationship construction module 410 and the character score calculation module 430, and outputs specific character region information 135. The inclusion relationship correction module 440 specifies a partial image in which one character is described, using information indicating the character likeness calculated by the character score calculation module 430 and the inclusion relationship in the tree structure.
In addition, the inclusion relationship correction module 440 has a portion in which one character is described in the node from the root to the leaf in the tree structure that has the highest information indicating the character character of the region in the partial image corresponding to each node. It may be specified as an image. In addition to the “highest” as a condition, the top n partial images may be specified. For example, if the number of characters is known in advance, the number of characters may be n.
Further, the inclusion relationship correction module 440 may delete the parent and child nodes of the node specified as the partial image in which one character is described.
Specifically, the inclusion relationship correction module 440 identifies the MSER having the highest character score output from the character score calculation module 430 as the character region. Here, “the highest character score” refers to the highest character score among the nodes of the entire tree structure. An example of including another character in a certain character is extremely rare. Therefore, the MSER corresponding to the parent and child of the MSER specified as the character region in the tree structure representing the inclusion relation is excluded from the character candidate region, and the tree Remove from structure. The parent excluded here includes the parent of the parent, further includes the parent, and includes the route in the route. The excluded children include the children of the child, and further include the child, including the leaves in the path. That is, the nodes other than the target node are deleted from the nodes on the route from the root to the leaf including the target node. Thereby, one tree is divided into a plurality of subtrees. The above processing is repeated until all trees are only a single node.

This will be described with reference to FIGS.
For each tree constructed by the inclusion relationship construction module 410, the MSER having the highest character score is selected and specified as a character region. The example shown in FIG. 18 describes the character score (number) calculated by the character score calculation module 430 for each node (MSER) in the example (tree structure) shown in FIG. Here, the higher the value of the character score, the more likely it is to be a character. Among these, the MSER 1810 surrounded by a frame line indicates the highest score in this tree structure.
Since an example including another character in a certain character is very rare, MSER corresponding to the parent and child of the MSER specified as the character region is excluded from the character candidate region. In the example of FIG. 18, the parent of MSER 1810 (character score: 0.91 MSER), its parent (character score: MSER of 0.26, root), and child of MSER 1810 (character score: 0.11 MSER and character score) : 0.13 MSER, 2 leaves). By this deletion processing, as shown in the example of FIG. 19, one tree is divided into a plurality of subtrees (three in the example of FIG. 19).
Again, for each tree, the MSER with the highest character score is selected and specified as the character region. As in the example of FIG. 20, MSER 2010 is selected for the left tree, and MSER 2020 is selected for the right tree.
The MSER corresponding to the parent and child of the MSER specified as the character area is excluded from the character candidate area. Specifically, the child of MSER 2010 (character score: 0.21 MSER and character score: 0.03 MSER, two leaves) is deleted, and MSER 2020 parent (character score: 0.83 MSER), MSER 2020 Are deleted (character score: 0.58 MSER and character score: 0.32 MSER, two leaves), and the state shown in the example of FIG. 21 is obtained.
The above processing is repeated until all trees are only a single node.

FIG. 22 is a flowchart illustrating a processing example according to the present exemplary embodiment (the inclusion relationship correction module 440).
In step S2202, a tree having a plurality of nodes is selected.
In step S2204, the MSER having the maximum character score is specified as the character region.
In step S2206, the parent and child in the tree structure are deleted.
In step S2208, it is determined whether or not the number of nodes of all trees is 1. If the number of nodes of all trees is 1, the process is terminated (step S2299). Otherwise, step S2202 is performed. Return to.

  The specific character area information 135 is an MSER in which the node set extracted by the inclusion relationship correction module 440 is specified as a character. In the above example, the state is as shown in FIG. This is called a specific character area. The specific character area information 135 is expressed in the same format as the character candidate area information 125 or in the index information of the character candidate area information 125.

The characters in the scene image may have lost the image characteristics inherent to the characters due to the influence of shadows and reflections, and a high character score is not always obtained from the character region. Accordingly, when character / non-character determination is performed based only on the character score, a character region having a low character score may not be detected.
The inclusion relationship correction module 440 can also detect a region with a low character score by using the inclusion relationship between regions.
Character regions are specified in order from the highest character score. Since a region having a high character score has high reliability as a character region, the processing is highly reliable as a whole.
A character area having a low character score due to the influence of shadows, reflections, etc. belongs to a subtree composed of nodes having a low character score. However, since the area having the highest character score in the partial tree is specified as the character area, the reliability of the result is still high.

  A hardware configuration example of the image processing apparatus according to the present embodiment will be described with reference to FIG. The configuration illustrated in FIG. 23 is configured by, for example, a personal computer (PC), and illustrates a hardware configuration example including a data reading unit 2317 such as a scanner and a data output unit 2318 such as a printer.

  A CPU (Central Processing Unit) 2301 includes various modules described in the above-described embodiments, that is, the image reception module 110, the character candidate area detection module 120, the character area identification module 130, the character recognition module 140, and the inclusion relation construction module. 410, a character area feature extraction module 420, a character score calculation module 430, an inclusion relationship correction module 440, and the like. The control unit executes processing according to a computer program that describes an execution sequence of each module.

  A ROM (Read Only Memory) 2302 stores programs, calculation parameters, and the like used by the CPU 2301. A RAM (Random Access Memory) 2303 stores programs used in the execution of the CPU 2301, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 2304 including a CPU bus or the like.

  The host bus 2304 is connected to an external bus 2306 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 2305.

  A keyboard 2308 and a pointing device 2309 such as a mouse are input devices operated by an operator. The display 2310 includes a liquid crystal display device or a CRT (Cathode Ray Tube), and displays various types of information as text or image information.

  An HDD (Hard Disk Drive) 2311 includes a hard disk (may be a flash memory or the like), drives the hard disk, and records or reproduces a program executed by the CPU 2301 and information. The hard disk stores a target image, scene image information 115, character candidate area information 125, specific character area information 135, a character recognition result, and the like. Further, various other data, various computer programs, and the like are stored.

  The drive 2312 reads data or a program recorded on a removable recording medium 2313 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and the data or program is read out to the interface 2307 and the external bus 2306. , The bridge 2305, and the RAM 2303 connected via the host bus 2304. The removable recording medium 2313 can also be used as a data recording area similar to the hard disk.

  The connection port 2314 is a port for connecting the external connection device 2315 and has a connection unit such as USB or IEEE1394. The connection port 2314 is connected to the CPU 2301 and the like via the interface 2307, the external bus 2306, the bridge 2305, the host bus 2304, and the like. A communication unit 2316 is connected to a communication line and executes data communication processing with the outside. The data reading unit 2317 is, for example, a scanner, and executes document reading processing. The data output unit 2318 is a printer, for example, and executes document data output processing.

  Note that the hardware configuration of the image processing apparatus illustrated in FIG. 23 illustrates one configuration example, and the present embodiment is not limited to the configuration illustrated in FIG. 23, and the modules described in the present embodiment are executed. Any configuration is possible. For example, some modules may be configured with dedicated hardware (for example, Application Specific Integrated Circuit (ASIC), etc.), and some modules are in an external system and connected via a communication line In addition, a plurality of systems shown in FIG. 23 may be connected to each other via communication lines so as to cooperate with each other. In particular, in addition to personal computers, portable information communication devices (including mobile phones, smartphones, mobile devices, wearable computers, etc.), information appliances, copiers, fax machines, scanners, printers, multifunction devices (scanners, printers, copiers) Or an image processing apparatus having two or more functions such as a fax machine).

The above-described embodiment may be performed as follows.
-You may add the character specific area | region display module which displays the image of the area | region which has the largest character score with the parent / child.
Furthermore, a character identification result correction module for changing the character identification result may be added to either the parent or child area displayed by the character identification area display module by the operation of the operator.
The correction by the operation of the operator is possible, and it is only necessary to select from the region having the maximum character score and its parent / child.
If the maximum character score in the tree structure is less than or less than a predetermined value, an inclusion relationship correction module that excludes all candidate regions in the tree structure may be added.
For example, if the character score is extremely low, there is a high possibility of noise (non-character region), and this process can reduce the false detection rate.
In the present embodiment, the character area is detected, but may be applied to, for example, area labeling of an image. The image area labeling is, for example, assigning labels such as sky, clouds, sea, ground, and trees to each area of a landscape image.

The program described above may be provided by being stored in a recording medium, or the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording the program”.
The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum, such as “DVD-R, DVD-RW, DVD-RAM,” and DVD + RW. Standard “DVD + R, DVD + RW, etc.”, compact disc (CD), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), Blu-ray disc ( Blu-ray (registered trademark) Disc), magneto-optical disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM (registered trademark)) )), Flash memory, Random access memory (RAM) SD (Secure Digital) memory card and the like.
The program or a part of the program may be recorded on the recording medium for storage or distribution. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, or a wireless communication It may be transmitted using a transmission medium such as a network or a combination of these, or may be carried on a carrier wave.
Furthermore, the program may be a part of another program, or may be recorded on a recording medium together with a separate program. Moreover, it may be divided and recorded on a plurality of recording media. Further, it may be recorded in any manner as long as it can be restored, such as compression or encryption.

DESCRIPTION OF SYMBOLS 100 ... Image processing apparatus 110 ... Image reception module 115 ... Scene image information 120 ... Character candidate area detection module 125 ... Character candidate area information 130 ... Character area identification module 135 ... Specific character area information 140 ... Character recognition module 210 ... Shooting apparatus 230 ... Information processing device 310 ... User terminal 330 ... Information processing device 390 ... Communication line 410 ... Inclusion relation construction module 420 ... Character area feature extraction module 430 ... Character score calculation module 440 ... Inclusion relation correction module

Claims (5)

Detecting means for detecting a partial image having a feature as a character from the image as a character candidate region;
Extraction means for extracting an inclusion relationship between the partial images;
An image processing apparatus comprising: specifying means for specifying a partial image in which one character is described using information indicating the character-likeness of an area in the partial image and the inclusion relation. The extraction means constructs a tree structure indicating the inclusion relationship,
The specifying means is:
Second extraction means for extracting features of a region in the partial image from the image and the partial image;
Calculating means for calculating information indicating the character likeness of the area in the partial image from the features extracted by the second extracting means;
2. A second specifying unit that specifies a partial image in which one character is described using information indicating the character likeness calculated by the calculating unit and an inclusion relation in the tree structure. Item 8. The image processing apparatus according to Item 1. The second specifying means includes a portion in which one character is described in the node from the root to the leaf in the tree structure that has the highest information indicating the character of the region in the partial image corresponding to each node. The image processing apparatus according to claim 2, wherein the image processing apparatus is specified as an image. The image processing apparatus according to claim 3, wherein the second specifying unit deletes a parent node and a child node of a node specified as a partial image in which one character is described. Computer
Detecting means for detecting a partial image having a feature as a character from the image as a character candidate region;
Extraction means for extracting an inclusion relationship between the partial images;
An image processing program for functioning as specifying means for specifying a partial image in which one character is described, using information indicating the character-likeness of an area in the partial image and the inclusion relation.

Images