JP2017117340A - Character attribute estimation device and character attribute estimation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately estimate the attribute of characters in a scene image on the basis of information other than the characters.SOLUTION: A character attribute estimation device comprises: a character area detection part that detects an area of characters from an image; an object area detection part that detects, of pixels adjacent to the area of characters detected by the character area detection part, a pixel satisfying a condition according to at least part of a pixel value, as an area of an object on which the characters are described; and an attribute estimation part that performs machine learning of an image group according to the attribute of the details represented by the characters as training data, and estimates the attribute of the details represented by the characters from a result of the machine learning and the shape of the area of the object detected by the object area detection part.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a character attribute estimation device and a character attribute estimation program.

  2. Description of the Related Art Conventionally, a technique for recognizing characters in an image captured by a camera is known. In general, when recognizing a character, a character region is first extracted from an image, and then a character is recognized based on a pattern such as the shape and color of the character (see, for example, Patent Documents 1 to 3).

  In the above-described method, the shape, color, and the like of characters in the image cannot be extracted in a complete state, and part of information is missing or noise is mixed. Therefore, conventionally, there is a method of correcting recognition errors of some characters when a character string is recognized by correcting using a dictionary created in advance (for example, Patent Document 4, Patent) Reference 5).

JP 2001-331803 A JP 2005-018176 A JP 2002-230551 A Japanese Patent Publication No. 04-074756 Japanese Patent Publication No. 07-066423

  However, in the prior art, there is no method for estimating character attributes based on information other than characters in order to perform highly accurate character recognition.

  Therefore, an object of the present invention is to provide a character attribute estimation device and a character attribute estimation program capable of estimating the attribute of a character in a scene image with high accuracy based on information other than characters.

  In one aspect of the present invention, in the character attribute estimation device, at least one of a character region detection unit that detects a character region from an image and pixels adjacent to the character region detected by the character region detection unit. Machine learning using, as training data, an object region detection unit that detects a pixel that satisfies a condition corresponding to a pixel value of a part as an object region in which the character is described, and an image group according to the attribute of the content represented by the character An attribute estimation unit that estimates an attribute of the content represented by the character from the result of the machine learning and the shape of the region of the object detected by the object region detection unit.

  In one aspect of the present invention, in the character attribute estimation device, a character region detection unit that detects a character region from an image, and a pixel adjacent to the character region detected by the character region detection unit. An object region detection unit that detects pixels satisfying a condition corresponding to at least a part of pixel values as an object region in which the characters are described, and machine learning an image group according to the type of the object as training data, From the result of machine learning and the shape of the region of the object detected by the object region detection unit, the object type determination unit for determining the type of the object, and the type of the object determined by the object type determination unit An attribute estimation unit that estimates an attribute of content represented by the character.

  According to the disclosed technology, it is possible to estimate the attribute of a character in a scene image with high accuracy based on information other than the character.

It is a figure for demonstrating the outline | summary of the character attribute estimation apparatus which concerns on this embodiment. It is a functional block diagram which shows the function structure of the character attribute estimation apparatus which concerns on this embodiment. It is a figure which shows an example of an attribute estimation table. It is a flowchart which shows an example of the process which estimates the attribute of the character in a scene image. It is a flowchart which shows an example of an object area | region detection process. It is a figure explaining an object area | region detection process. It is a flowchart which shows an example of an attribute estimation process. It is a figure explaining an attribute estimation process. It is a functional block diagram which shows the modification of the function structure of a character attribute estimation apparatus. It is a flowchart which shows the modification of the process which estimates the attribute of the character in a scene image.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Outline of character attribute estimation device>
FIG. 1 is a diagram for explaining the outline of the character attribute estimation apparatus according to the present embodiment. A character attribute estimation apparatus 1 shown in FIG. 1 inputs an image (scene image 2), and estimates the attribute of one or more characters appearing in the image. The “character” includes, for example, information described for transmitting predetermined information such as a pictograph, a symbol, and a graphic as well as kanji, kana, and alphanumeric characters. Here, the “character attribute” is information indicating contents represented by characters such as “person name”, “place name”, “company name (organization name)”, but is limited to the above example. is not.

  The character 3 shown in the scene image 2 is written on some object 4 (for example, a name tag or a signboard), and it can be expected that the character 3 and the object 4 have some relationship. For example, there is a relation such as “character attributes that are largely described near the center of a name tag are often personal names”.

  The character attribute estimation device 1 according to the present embodiment detects a character area that is a set of pixels of the character 3 shown in the scene image 2, and uses information around the character area to determine the attribute of the character 3 Is estimated. Thereby, the attribute of the character in a scene image can be estimated with high precision. In the present embodiment, not only the character attributes but also the character data recognized using the character attributes may be output.

<Functional configuration of character attribute estimation device>
Next, the functional configuration of the character attribute estimation apparatus according to the present embodiment will be described with reference to the drawings. FIG. 2 is a functional block diagram showing a functional configuration of the character attribute estimation apparatus according to the present embodiment.

  The character attribute estimation device 1 includes an image acquisition unit 11, a character region detection unit 12, an object region detection unit 13, an object type determination unit 14, an attribute estimation unit 15, a dictionary selection unit 16, and a character recognition unit 17. And have.

  The image acquisition unit 11 acquires image (scene image) data including characters. For example, the image acquisition unit 11 may be an image taken by an imaging device such as a camera or a video camera, or may be an image stored in advance. Moreover, the imaging device mentioned above may be provided inside the character attribute estimation apparatus 1, and may be provided outside. Further, the above-described image may be a still image or an image frame included in a moving image.

  The character region detection unit 12 detects a character region that is a region in which characters are captured from the image acquired by the image acquisition unit 11. For example, when a black character appears on a white signboard or the like, the character region is an area of the black portion. The detection of the character region can be detected based on the color and shape of a pixel set having similar characteristics, the shape of an edge, and the like, but is not limited thereto. The character area can be represented by, for example, a set of pixels (coordinates) constituting the character.

  The object region detection unit 13 detects an object region that is an object region in which a character is described from an image acquired by the image acquisition unit 11 based on the character region detected by the character region detection unit. Process. In the object region detection process, the object region detection unit 13 sets at least some of the pixels adjacent to the character region detected by the character region detection unit 12 as an initial range of the object region. A pixel that satisfies a predetermined condition with respect to a pixel outside is detected as an object region.

  That is, the object region detection unit 13 sets at least a part of pixels adjacent to the character region detected by the character region detection unit 12 as an initial range of the object region in the object region detection process. In addition, the object region detection unit 13 determines whether there is a pixel that is adjacent to the object region and satisfies the condition corresponding to the pixel value of the object region, and sequentially includes the pixels that satisfy the condition in the object region. The object region detection unit 13 performs the above-described processing sequentially outward with respect to the initial range of the object region, thereby expanding the object region and detecting the final object region. Details of the object region detection process will be described later. Note that the object region detection unit 13 may detect a predetermined object region from the image using a known image processing technique such as pattern recognition.

  The object type determination unit 14 determines the type of an object on which characters are described based on the object region detected by the object region detection unit 13. The object type determination unit 14 performs machine learning in advance using an image group corresponding to the type of the object to be detected as training data. Then, the object type determination unit 14 determines the type of the object on which the character is described from the result of the machine learning and the shape of the input object region. In addition, the object type determination unit 14 detects the color of the object area, and also determines the type of the object on which the character is written, together with the color of the input object area.

  The attribute estimation unit 15 estimates the attribute of the content represented by the character in the character area detected by the character area detection unit 12 from the type of the object determined by the object type determination unit 14. The attribute estimation unit 15 detects the position of the character in the object region, the range of the character, the relative size of the character whose attribute is to be estimated, the number of characters, and the color of the background of the character. The attribute estimation unit 15 uses, for example, a preset attribute estimation table to determine the type of the object, the position of the character in the object region, the character range, and the relative size of the character whose attribute is to be estimated with respect to other characters. Attribute estimation processing for estimating the attribute (category) of the character described in the object is performed in correspondence with at least one of the number of characters and the background color of the character. Thereby, the attribute estimation part 15 can select appropriately the dictionary (word dictionary) for character recognition based on the attribute of a character, for example. Moreover, the attribute estimation part 15 may output the attribute information, after estimating the 1 or more attribute with respect to a character by the attribute estimation process mentioned above. Details of the attribute estimation process will be described later.

  The dictionary selection unit 16 selects a dictionary corresponding to the character attribute estimated by the attribute estimation unit 15. For example, if the character attribute is “person name”, a dictionary in which characters that are easily used as a person name are registered is selected.

  The character recognition unit 17 recognizes characters in the character region detected by the character region detection unit 12 based on the dictionary data selected by the dictionary selection unit 16. For example, pattern matching between each character registered in the dictionary and the detected character region is performed, and the character in the detected character region is the character with the highest similarity among the characters registered in the dictionary. Recognize that

  The character attribute estimation apparatus 1 shown in FIG. 2 is configured to perform character recognition using the estimation result after estimating the attribute of the character included in the image or the like, but is not limited to this. For example, a configuration up to outputting a character attribute may be used. In that case, the character attribute estimation device 1 includes the image acquisition unit 11, the character region detection unit 12, the object region detection unit 13, the object type determination unit 14, and the attribute estimation unit 15 described above. Character attribute information obtained from the unit 15 is output.

<Example of attribute estimation table>
FIG. 3 is a diagram illustrating an example of the attribute estimation table. Examples of items in the attribute estimation table include, for example, “object type”, “character position or range”, “relative size relative to other characters”, “number of characters”, “character background color” ”And“ character attributes ”, but are not limited thereto.

  The “object type” is the type of the object in which characters are described, determined by the object type determination unit 14.

  “Character position or character range” is the character position or character range in the region of the object in which the character is described. As the “character position”, for example, “upper part”, “central part”, “lower part” and the like are set. In the present embodiment, a predetermined condition for determining the position may be set for each “character position” of each “object type”. For example, the condition for determining that the “object type” is “name tag” and the “character position” is “upper” is that “the center of the character area divides the object area into three equal parts in the vertical direction. In this case, the condition is not limited to this.

  Further, as the “character range”, for example, “entire surface” or the like is set. In the present embodiment, a predetermined condition for determining the range may be set for each “character range” of each “object type”. For example, the condition for determining that “object type” is “signboard” and “character range” is “entire surface” is “a range including all character regions described in the object (for example, each character region An ellipse with the smallest area including the shape (a figure such as 620 and 621 in FIG. 8 in which the left and right sides of the rectangle are replaced with straight lines and each formed by a semicircle is also referred to as an “ellipse” for the sake of convenience. ) Or rectangle) is, for example, 80% or more of the area of the object region. However, the conditions are not limited thereto.

  “Relative size with respect to other characters” refers to, for example, when a plurality of characters are described in a region of an object, the size of the character that is the object of character attribute determination, It is expressed as relative size information for the characters. As “relative size with respect to other characters”, for example, “large”, “medium”, “small”, “equal”, etc. are set. In the present embodiment, for each “object type”, a predetermined condition for determining the size may be set for each “relative size with respect to other characters”. For example, the condition for determining that “type of object” is “name tag” and “relative size with respect to other characters” is “large” is that the size of a circle included in one group and the other group Is a case where the ratio of the sizes of the circles included in is equal to or greater than a predetermined value, etc., but the conditions are not limited thereto.

  The “number of characters” is the number of character regions included in the object region. The “character background color” is the color of the part other than the character area in the object area.

  In the example of FIG. 3, the attribute of the object is associated with “name tag”, and “person name” and “company name” are stored as character attributes. “Company name” is stored in association with “signboard”. This is because, for example, when recording a news video, a signboard installed in front of a building or conference room is first photographed. In addition, “person name” is stored in association with “name plate”. This is because, for example, when recording a video of a press conference, a name plate placed on a desk is first photographed.

<Process for estimating character attributes in a scene image>
Next, with reference to FIG. 4, a process for estimating the attributes of characters in a scene image will be described. FIG. 4 is a flowchart illustrating an example of processing for estimating the attributes of characters in a scene image.

  The image acquisition unit 11 acquires a scene image (step S101).

  The character region detection unit 12 detects a character region that is a region where characters are captured from the acquired scene image (step S102).

  Based on the detected character area, the object area detection unit 13 performs an object area detection process for detecting an object area that is an object area in which characters are described (step S103).

  The object type determination unit 14 determines the type of the object on which the character is written based on the object region detected by the object region detection unit 13 (step S104).

  The attribute estimation unit 15 performs attribute estimation processing for estimating the attribute of the character described in the object based on the determined type of the object (step S105).

<Object area detection processing>
Next, with reference to FIG. 5, the details of the object region detection process for detecting the object region, which is the region of the object in which the characters are described, by the object region detection unit 13 in step S103 of FIG. 4 will be described. FIG. 5 is a flowchart illustrating an example of the object region detection process.

  The object area detection unit 13 determines the initial range of the object area based on the pixel values at the outer edge of the character area (step S201).

  Next, the object region detection unit 13 selects one unselected pixel as a candidate for the object region among the pixels adjacent to the object region (step S202). Next, the object region detection unit 13 determines whether the selected pixel has a feature close to a pixel group included in the object region (step S203).

  If there is no close feature (NO in step S203), the process proceeds to step S205 described later. In addition, when it has a close feature (YES in step S203), the object region is sequentially expanded by including the pixel in the object region (step S204).

  The object region detection unit 13 determines whether there is an unselected pixel as a candidate for the object region among the pixels adjacent to the object region (step S205).

  If there is no unselected pixel (NO in step S205), the object region detection unit 13 ends the process. If there is an unselected pixel (YES in step S205), the object region detection unit 13 returns to the process in step S202.

<Example of object area detection>
Next, an example of object region detection according to the present embodiment will be described with reference to FIG. FIG. 6 is a diagram for explaining the object region detection process.

  FIG. 6A shows an example of an object region in which one character “1” is included, which is included in the scene image. In FIG. 6A, the pixel value of each pixel included in the character area 601 is assumed to have a luminance of about 0, for example. A region 602 is a region having characteristics close to each pixel in contact with the character region 601. The pixel value of each pixel included in the area 602 is assumed to have a luminance of about 200, for example. The region 603 is a region that does not have a feature close to each pixel in contact with the character region 601. The pixel value of each pixel in the region 603 is assumed to have a luminance of about 125, for example.

  By the object region detection process, a character region 601 and a region including the region 602 are detected as object regions.

  FIG. 6B is a diagram illustrating an example in which the initial range of the object region is determined based on the value of the outer edge that is each pixel in contact with the character region 601 in step S201 of FIG. In the example of FIG. 6B, all the pixels at the outer edge of the character area 601 are determined as the initial range 604b of the object area. In addition, for example, a configuration may be adopted in which a part of the pixels in the outer edge portion is determined as the initial range, such as selecting a pixel having a value closest to the average value or median value from the pixels in the outer edge portion of the character area.

  Next, in step S204 in FIG. 5, the object region is sequentially expanded. First, each pixel in contact with the initial range 604b of the object area (each pixel to which “x” in FIG. 6B is added) is selected. Then, among the selected pixels, for example, the process of including the pixel having the feature closest to the current object region in the object region is sequentially repeated. In this case, for example, the process of including the pixel closest to the average value or median value of the pixels included in the current object region in the object region is repeated. In FIG. 6C, the pixel near the lower left of the character area 601 is relatively close to the average value or the median value of the pixels included in the current object area, and is therefore captured first in the object area 604c. An example is shown. By sequentially repeating this process, the object region is changed from 604b in FIG. 6B to 604c in FIG. 6C, 604d in FIG. 6D, 604e in FIG. 6E, and FIG. As in 604f, it is expanded sequentially.

  Or, when the value of the pixel adjacent to the object region is between the maximum value and the minimum value of the pixels included in the object region, the object region is sequentially expanded by including the adjacent pixel in the object region. It is good also as composition to do. In that case, when there are no more pixels to be included in the object region, the maximum value of the pixels to be newly included in the object region is increased by 1, the minimum value is decreased by 1, and the values of adjacent pixels are included in the new object region. As long as it is between the maximum value and the minimum value of the existing pixels, the process of including the adjacent pixels in the object region may be repeated.

  FIG. 6F is a diagram illustrating an example in which NO is determined in step S205 in FIG. The object region 604f is a region including a character region 601 and a region 602. Since the feature between the pixel adjacent to the object region and the pixel group included in the object region are distant (for example, the difference in feature is equal to or greater than a predetermined threshold value), the process ends when the extension of the object region converges. For example, the processing ends when pixels that are within a predetermined threshold from the average value or median value of the current object region no longer exist outside the current object region.

  Alternatively, the processing may be terminated when a pixel that is within a predetermined threshold value between the maximum value and the minimum value of the pixels included in the object region does not exist outside the current object region.

  In addition, it is good also as a structure which uses each three pixel values of RGB as a pixel value. In that case, for example, only a pixel value of one primary color (for example, R) selected from RGB may be used, or an average value of a plurality of primary colors may be used.

<Object type determination process>
Next, processing for determining the type of an object by the object type determination unit 14 will be described. The object type determination unit 14 performs machine learning in advance using an image group corresponding to the type of the object to be detected as training data. Then, the object type determination unit 14 performs general object recognition or specific object recognition on the object region detected by the object region detection unit 13 based on, for example, a shape or color feature, so that an object on which characters are described The type of is determined.

<Attribute estimation processing>
Next, attribute estimation processing by the attribute estimation unit 15 will be described with reference to FIG. FIG. 7 is a flowchart illustrating an example of attribute estimation processing.

  The attribute estimation unit 15 determines the position of the character in the object region based on the character region detected by the character region detection unit 12 and the object region detected by the object region detection unit 13 (step S301).

  Next, the attribute estimation unit 15 determines the range of the area including characters in the object area (step S302). Next, the attribute estimation unit 15 determines a relative size with respect to other characters in the object region (step S303). Next, the attribute estimation unit 15 determines the number of characters in the object area (step S304). Next, the background color of the character in the object area is determined (step S305).

  Next, based on the attribute estimation table, the attribute estimation unit 15 is based on the type of object, the position and range of characters in the object region, the relative size with respect to other characters, the number of characters, and the background color. The attribute of the content represented by the character is estimated (step S306).

<An example of attribute estimation>
Next, an example of attribute estimation according to the present embodiment will be described with reference to FIG. FIG. 8 is a diagram for explaining an example of attribute estimation according to the present embodiment.

  FIG. 8A is a diagram for explaining preprocessing for determining characters in the region (object region) of the object 4 detected by the object region detection unit 13. For example, the attribute estimation unit 15 generates circles 610 to 619 including (inscribed) each character area, and the distance (for example, the distance between the centers of the nearest circles) is within a predetermined range, and the size (for example, a circle) One or more circles having a difference in radius) within a predetermined range are determined as the same group. In the example of FIG. 8A, the circles 610 to 613 are determined to be the same group, and the circles 614 to 619 are determined to be the same group. In addition, it is good also as a structure which uses polygons, such as a square and a rectangle, instead of a circle.

  FIG. 8B is a process of determining the character position, the character range, the relative size with respect to other characters, the number of characters, and the background color in the object region in steps S301 to S305 in FIG. FIG.

  For example, the attribute estimation unit 15 generates ellipses 620 and 621 including the circles of each group and having the smallest area, and determines the positions and ranges of the generated ellipses 620 and 621 in the object region.

  The position of the character in the object area is determined according to whether or not a predetermined condition for determining each position is set in the attribute estimation table of FIG. In the example of FIG. 8B, the circles 614 to 619 are included in the circles 614 to 619 in order to satisfy the condition that the centers of the circles 614 to 619 are located in the uppermost region when the object region is divided into three equal parts in the vertical direction. It is determined that the position of each character is “upper”.

  The character range in the object region is determined according to whether or not a predetermined condition for determining each range is set in the attribute estimation table of FIG.

  The relative size with respect to other characters in the object region is determined according to whether or not a predetermined condition for determining each size is set in the attribute estimation table of FIG. In the example of FIG. 8B, the ratio of the size of each circle included in the ellipse 620 to the size of each circle included in the ellipse 621 is equal to or greater than a predetermined value. Is determined to be “large”. Similarly, the relative size of each character included in the ellipse 621 with respect to other characters is determined to be “medium”.

  The number of characters in the object area is determined based on, for example, the number of circles included in each group. In the example of FIG. 8B, since the number of circles 614 to 619 included in the group of ellipses 620 is 6, the number of characters is determined to be 6.

  The background color of the character in the object area is determined based on the average value of the color of the part other than the character area in the object area. In the example of FIG. 8B, since the average value of portions other than the character area is within a predetermined range for determining “white”, the background color of the character is determined to be “white”.

  Using the example of the attribute estimation table shown in FIG. 3, the process for estimating the attribute of the content represented by the character in step S306 in FIG. 7 will be described.

  The characters of the circles 610 to 613 have the object type “business card”, the character position in the object region “center”, the relative size with respect to other characters “large”, and the number of characters in the object region 4 The background color of the character in the object area is “white”. Therefore, it is presumed that the attribute of the content represented by the characters is “person name”. Similarly, since the characters of the circles 614 to 619 have the object type “business card” and the relative size with respect to other characters “medium”, the attribute of the content represented by the characters is “company name”. Presumed.

  Note that the values in the predetermined range and the predetermined threshold in each of the attribute estimation processes described above may be configured to use values according to the type of the object. In that case, the value may be configured to be set by the user.

<Modification>
The object region detection unit 13 may be configured to detect a region that meets a condition by expanding the entire scene image until the entire region image is included in the object region while preserving the feature of the object region each time the object region is expanded.

For example, every time the object region is expanded, the object region detection unit 13 expands the pixel value variation q _internal (for example, the difference between the minimum value and the maximum value, variance, etc.) in the object region before the expansion. The pixel value variation q _external in the previous object area is calculated and stored. Then, the object region detection unit 13 expands the state image so that all the pixels of the scene image are included in the object region, and then the variation ratio q _internal / q _external when the object region is expanded a certain number of times is A configuration may be _adopted in which a region that is smaller than the ratio q _internal / q _external when the number of variations before and after the number of times of expansion is expanded is detected as the object region.

Alternatively, a configuration may be _{adopted in} which a region where the variation ratio q _internal / q _external is the smallest when the number of times the object region is expanded is within a predetermined range (for example, 50 to 150 times) is detected as the object region. Here, the region that is the smallest within the predetermined range is the region that is the smallest within the entire range. If the object to be detected is in a slightly larger object (for example, “person”) This is because there is a possibility that another object may be detected when there is a region where “name tag” is reflected in the region where the name appears.

  When a plurality of regions are detected as object regions by the above processing, only one of the plurality of regions may be used for the subsequent processing, or one or more regions of the plurality of regions are used. Thus, the attribute of the character for each area may be estimated, and the obtained attribute of the character for each area may be used for character recognition or the like.

  In addition, since the attribute estimation unit 15 has a plurality of determination results of the object type determination unit 14, if a plurality of character attributes are estimated, the attribute estimation unit 15 confirms the determination result obtained from the object type determination unit 14 for each attribute. A score indicating the likelihood may be output, or a character attribute associated with the most probable attribute may be output.

  Moreover, the attribute estimation part 15 is good also as a structure which estimates the attribute of a character based on the area | region of the object detected by the object area | region detection part 13, for example. Such a case will be described below. FIG. 9 is a functional block diagram showing a modification of the functional configuration of the character attribute estimation device. Compared with the functional block diagram of FIG. 2, the object type determination unit 14 is not present. The attribute estimation unit 15 of the modification machine-learns in advance an image group corresponding to the attribute of the content represented by the detection target character as training data. For example, a “person name” attribute is added to a plurality of images in which characters of a person's name are shown, and the plurality of images are machine-learned as training data. And the attribute estimation part 15 of a modification estimates the attribute of the content which the said character represents from the result of the said machine learning, the shape of the area | region of the object detected by the object area | region detection part 13, etc. FIG. FIG. 10 is a flowchart illustrating a modification of the process for estimating the attribute of a character in a scene image. Steps S401 to S403 in FIG. 10 are the same as steps S101 to S103 in FIG. In the attribute estimation process of step S404, the attribute estimation unit 15 of the modified example estimates the character attribute based on the shape of the object region instead of the object type. Thereby, it is not necessary to determine the object type by the object type determination unit 14.

  The object type determination unit 14 may be configured to output information on the determined object type to the character recognition unit 17. As a result, the character recognition unit 17 can recognize a character using features such as a large amount of distortion and a small amount of linear components when the type of the object is a cloth such as a banner.

  The character attribute estimation device 1 may be realized by cloud computing constituted by one or more computers, for example. In that case, for example, the dictionary selection unit 16 and the character recognition unit 17 may be configured to be realized by another computer.

  The character attribute estimation device 1 can be applied to a system that switches a dictionary used for character recognition, for example, according to a character attribute. For example, from a video database that accumulates videos of interview contents or videos of past broadcast programs, characters recognized in images in the video are recognized as characters, and the recognized character string is used as a search tag for the video. The present invention can be applied to an archive system that manages in association with a moving image. Moreover, the character attribute estimation apparatus 1 can also be applied to, for example, a smartphone that recognizes characters from an image captured by a camera.

<About the character attribute estimation program>
The character attribute estimation apparatus 1 according to this embodiment includes, for example, a CPU (Central Processing Unit), a volatile storage medium such as a RAM (Random Access Memory), a nonvolatile storage medium such as a ROM (Read Only Memory), a mouse, and the like. Or a keyboard, an input device such as a pointing device, a display unit for displaying images and data, and a computer having an interface for communicating with the outside.

  In that case, each function of the character attribute estimation device 1 can be realized by causing the CPU to execute a program (character attribute estimation program) describing these functions. The program can also be stored and distributed on a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), semiconductor memory, or the like. That is, the above-described processing can be realized by installing a program for causing a computer (hardware) to execute the processing in each configuration described above, for example, on a general-purpose PC or server.

  Moreover, you may implement | achieve part or all of the character attribute estimation apparatus 1 in embodiment mentioned above as integrated circuits, such as LSI (Large Scale Integration). Each functional block of the character attribute estimation apparatus 1 may be individually made into a processor, or a part or all of them may be integrated into a processor.

<Summary>
According to the embodiment described above, the character attributes are estimated using information around the character area. Thereby, the attribute of the character in a scene image can be estimated with high precision. Character attributes can be used for selecting a word dictionary for improving the recognition accuracy of characters in a scene image. If the dictionary used for character recognition is switched according to the character attributes, the recognition speed can be improved by reducing the size of the dictionary to be collated, and erroneous recognition of characters (words) can be reduced.

  Further, according to the above-described embodiment, the area of the object is detected using information around the character area. Therefore, compared to the case where the object region is detected based on the luminance or the like from the scene image as in the conventional technique, the object region in which the characters are described can be detected with high accuracy.

  As described above, the embodiments have been described in detail with reference to the drawings. However, the specific configuration is not limited to that described above, and various design changes and the like can be made without departing from the scope of the invention. Is possible. Moreover, it is also possible to combine a part or all of each embodiment mentioned above.

DESCRIPTION OF SYMBOLS 1 Character attribute estimation apparatus 11 Image acquisition part 12 Character area detection part 13 Object area detection part 14 Object type determination part 15 Attribute estimation part 16 Dictionary selection part 17 Character recognition part

Claims (6)

A character area detection unit for detecting a character area from an image;
Object region detection for detecting pixels satisfying a condition corresponding to at least some pixel values of pixels adjacent to the character region detected by the character region detection unit as an object region in which the character is described And
Machine learning is performed as training data on an image group corresponding to the attribute of the content represented by the character, and the content represented by the character is determined from the result of the machine learning and the shape of the region of the object detected by the object region detection unit. An attribute estimation unit for estimating attributes;
A character attribute estimation device comprising: A character area detection unit for detecting a character area from an image;
Object region detection for detecting pixels satisfying a condition corresponding to at least some pixel values of pixels adjacent to the character region detected by the character region detection unit as an object region in which the character is described And
Object type determination in which an image group corresponding to the object type is machine-learned as training data, and the object type is determined from the result of the machine learning and the shape of the object area detected by the object area detection unit And
An attribute estimation unit that estimates an attribute of the content represented by the character from the type of the object determined by the object type determination unit;
A character attribute estimation device comprising: The object area detection unit uses at least a part of pixels adjacent to the character area as an initial range of the object area, and is adjacent to the object area and is in accordance with a pixel value of the object area. Sequentially include pixels in the region of the object,
The character attribute estimation apparatus according to claim 1 or 2. The attribute estimation unit is based on the position of the character in the region of the object, the range of the character, the relative size of the character with respect to other characters, the number of characters, or the background color of the character, Estimating the attributes;
The character attribute estimation apparatus as described in any one of Claims 1 thru | or 3. A dictionary selection unit that selects a dictionary according to the attribute estimated by the attribute estimation unit;
A character recognition unit for recognizing the character based on the dictionary selected by the dictionary selection unit;
The character attribute estimation apparatus according to claim 1, comprising: Computer
The character attribute estimation program for functioning as a character attribute estimation apparatus as described in any one of Claims 1 thru | or 5.

Images