KR102125056B1 - System and Method for Generating Character and Book for Mixed Character Automatic Recognition and System and Method for Searching using same - Google Patents

Abstract

The present invention relates to a character and book generation system for mixed character automatic recognition which increases the efficiency of mixed character recognition and search through deep learning including the formation of individual characters variation and virtual page formation for automatic recognition of mixed characters, to a method thereof, and to a search system and method using the same. The system comprises: a mutation forming unit which learns a variation of individual characters or one side and learns a combination of one side based on an actual letter, and generates all characters which can be combined with the combination of the side; a virtual page forming unit randomly filling in pages of different sizes and shapes with various languages and types of characters and noise generated by the mutation forming unit and generating a book having virtual pages; and a network learning unit learning text regions of pages generated by the virtual page forming unit, classifying and learning what language the characters generated in the mutation forming unit, and learning a location and type of lower components.

Description

System and Method for Generating Characters and Books for Automatic Recognition of Mixed Characters and Searching System and Method Using the Same

The present invention relates to automatic recognition of mixed characters, and specifically, deep learning that includes variation of individual characters and formation of virtual pages to generate characters and books for automatic recognition of mixed characters to increase the efficiency of recognition and search of mixed characters. The present invention relates to a system and method and a search system and method using the same.

Currently, it is required to improve the character recognition technology in various industries.

For example, more advanced character recognition technology is required to rapidly and accurately process large amounts of documents, such as banks, card companies, government offices, or to improve automatic control capabilities in robot application technology.

In particular, the development of a precise character recognition technology for the construction of a document recognition system for realization of a specialized search service for image documents and high documents has been developed.

The document recognition system for realizing a professional search service for high-document documented image data needs to be combined with character recognition technology. Character recognition technology extracts features from input text data and matches them with several predefined character models. It consists of a process of classifying into a character model.

The character model that is pre-built in the training process is the same as the criterion that corresponds to a single concept through recognition, and thus the data to be recognized is limited or affects the performance of character recognition depending on the nature of the character model.

Various methods, such as pattern matching, correlation calculation, feature point extraction and matching, and neural network-based deep learning, are known as such character recognition-related technologies.

In these technologies, character recognition performance greatly depends on a scale or orientation change of a test image compared to a reference image. In particular, orientation correction greatly affects the improvement of character recognition rate.

As an example of the prior art, the technique for statistical pattern recognition based on the relative position of the inclination and the character of the character has the following problems. (Registration No. 10-1900903, Republic of Korea)

First, an area division error in which characters are placed is directly related to an error in character recognition, and there is a problem in that numerous variations due to differences in personal habits or characteristics cannot be identified.

For pattern recognition based on the inclination and relative position, in the case of Chinese characters that are close to 60,000 or more than 160,000 Korean characters (when combined consonants and vowels), the number of similar characters and personal characteristics in the interior is too large. There are many problems in that accuracy is deteriorated in recognizing a specific character.

Also, in the case of tortures or books in which several characters are mixed together, the number of cases in this case is almost unlimited, making it difficult to accurately recognize characters.

Also, as another example of the prior art, in the case of a character recognition system using deep learning, there is a limited sample collection problem that takes a lot of time, such as dividing characters individually in a specific book or writing a number of handwritten characters, and limiting the number of learning samples There is a problem that the recognition rate decreases due to (Republic of Korea Patent Publication No. 10-2011-0039900) (Republic of Korea Patent Publication No. 10-2008-0076433)

As described above, the character recognition technology for constructing the document recognition system of the prior art has difficulty in qualitative and quantitative aspects depending on the difficulty of the character because it cannot fully algorithmize the human information processing ability.

For example, in the case of printed characters, difficulties arise due to inconsistencies such as various font shapes, sizes, slants, and the difficulty in quantitative aspects of character recognition due to the vast amount of various characters There is a problem in that it is difficult to construct a system having a large storage capacity engineeringly, and a problem takes a long time to recognize a specific character on a system having a large storage capacity.

The qualitative and quantitative difficulty of such character recognition is a factor that makes it difficult to implement general-purpose character recognition with constant performance for all data.

Therefore, in order to solve the problem of efficient text dataization of large image data documents, it is more accurate by constructing a specialized database for each field according to the field of the document that requires technology and text to minimize the difficulty of character recognition. Efficient system construction is required.

In particular, if the image of a document that has mixed characters or needs to be processed, such as an old document, has a long collection period, the document image quality is low and includes various document structures, and the performance of character recognition is high for large data generated by various fonts. Because of the further decline, the development of new technologies for character recognition technology is required to realize the optimal professional search service.

Republic of Korea Registered Patent No. 10-1900903 Republic of Korea Patent Publication No. 10-2011-0039900 Republic of Korea Patent Publication No. 10-2008-0076433

The present invention is to solve the problems of the prior art mixed character automatic recognition technology, deep character recognition, including the formation of variations and virtual page formation of individual characters to improve the efficiency of recognition and search of mixed characters automatic recognition of mixed characters The purpose is to provide a system and method for generating text and books, and a search system and method using the same.

According to the present invention, the composition and position of each element constituting the characters are organized into a database, and various variations of each digitized individual character or one side are learned with VAE (Variational AutoEncoder) or GAE (Generative Adversal Network) to generate a new image. The purpose of the present invention is to provide a system and method for generating text and books for automatic recognition of mixed text, and a search system and method using the same, so as to increase the efficiency of searching and searching for mixed text.

The present invention randomly fills the generated languages and types of characters and noise into pages of various sizes and shapes to create a book with pages close to infinity that are difficult to distinguish from actual books, and the area and shape of each character The purpose is to provide a system and method for generating text and books for automatic recognition of mixed characters, and a search system and method using the same.

The present invention uses a network that learns whether to include sub-components by using a classification network in case of characters with sub-components (number of Chinese characters, phonology of Hangul), or faster- with region proposal network. Provides a system and method for generating text and books for automatic recognition of mixed characters to improve the accuracy by learning the location of sub-components using deep learning networks such as RCNN, mask-RCNN, etc., and provides a search system and method using the same It has a purpose.

In the present invention, in the case of a document having sub-components, each character is recognized by retrieving the combination (structure) and structure (location) of the sub- (higher) components in the database of each character, and after identifying the components and their relations Provides a system and method for generating texts and books for automatic recognition of mixed characters, and a search system and method using the same, which enables quick recognition and search of characters by re-searching the relations between the components in the database after first searching for the component There is a purpose.

The present invention is a system and method for generating text and books for automatic recognition of mixed texts to improve the accuracy and efficiency of high document recognition and search by constructing a wall input system that is not commonly used in the process of forming variations and forming virtual pages, and using the same The aim is to provide a search system and method.

Other objects of the present invention are not limited to those mentioned above, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

The character and book generation system for automatic recognition of mixed characters according to the present invention for achieving the above object is to learn the variation of individual characters or one side to learn the combination of one side based on the actual character, and to combine the combinations of one side A variation forming unit that allows all possible characters to be generated; randomly filling characters and noise of various languages and types generated in the variation forming unit into pages of different sizes and shapes to generate a book having virtual pages A virtual page forming unit; a network learning unit learning a character area of the pages generated by the virtual page forming unit, classifying and learning what language the characters generated by the variation forming unit are, and learning the location and type of sub-components; It is characterized by including.

Here, the variation forming unit learns the variation of individual characters or one side to learn the combination of one side based on the actual character, and generates all the characters that can be combined with the combination of one side to form the lower component variation to form the lower component variation. Characterized in that it comprises a character variation forming unit that forms a character variation by learning the test images to include the formation unit and possible variations of different sizes and shapes, thickness, slope, and typeface.

And the virtual page forming unit, the size and spacing variation generating unit to randomly change the space between the character size and the next character within a certain range to generate a variation similar to the actual document, and the noise extracted from the actual document on the page itself and An area that generates documents similar to the real environment by generating noise, blocks, pictures, and tables of various variations with a deep learning network based on non-character areas including blocks, figures, and tables. It characterized in that it comprises a base document generator.

In addition, the network learning unit, a character area detection unit that detects a character area by learning a character area of pages generated by the virtual page formation unit, and a language type classification that learns which language is generated by the variation formation unit In case of characters with sub- and sub-components, the location of the sub-components is studied using a classification network to learn the location of the sub-components, and to learn the types and inclusions of the sub-components. Characterized in that it comprises a kind learning unit.

In addition, if the language has no sub-elements, it is characterized by directly learning the character areas of the generated pages and detecting and segmenting them into individual characters.

The method for generating characters and books for automatic recognition of mixed characters according to the present invention for achieving other purposes is to learn variations of individual characters or one side and learn combinations of one side based on real characters, so that all combinations of one side can be combined. Forming a character component variation by generating letters and learning the test images to include possible variations of different sizes, shapes, thicknesses, inclinations, and fonts; forming character variations; multiple languages created in the variation formation unit, and A virtual page forming step of randomly filling characters of different types and noise into pages of different sizes and shapes to generate a book having virtual pages; detecting a character area of pages generated in the virtual page forming step, and forming variations Characterized in that it comprises; a network learning step of learning the classification of the characters generated by the language, and learning the location and type of sub-components.

Here, the step of forming the virtual page includes a step of generating a size and spacing variation to generate a variation similar to the actual document by randomly changing the character size and the spacing between the characters next to the character, and the page itself extracted from the actual document Create a document similar to the real environment by generating noise, blocks, pictures, and tables of various variations with a deep learning network based on non-character areas with multiple noises, blocks, figures, and tables It characterized in that it comprises a step of generating an area-based document.

In the network learning step, a text area detection step of learning a text area of pages generated by the virtual page forming unit to detect a text area, and a language type classification for classifying and learning what language the characters generated in the variation forming unit are. And a step of learning the position of the lower component using a classification network in the case of a character having a lower component, and learning the type and inclusion of the lower component. .

The search system using text and book generation for automatic recognition of mixed characters according to the present invention for achieving other objects learns variations of individual characters or one side to learn combinations of one side based on real characters, and thus to a combination of one side. A variation forming unit that allows all combinable characters to be generated; randomly filling the characters and noise of various languages and types generated in the variation forming unit into pages of different sizes and shapes to create a book with virtual pages The virtual page forming unit; learning the character area of the pages generated by the virtual page forming unit, detecting the character area, and learning the classification of the characters generated by the variation forming unit, and determining the location and type of sub-components. A network learning unit that learns; By learning the character area of the pages generated using the results learned from the network learning unit, divides individual characters, recognizes the language, recognizes the configuration of sub-components, or locates them Characterized in that it comprises; a mixed character recognition and search unit for recognizing and searching for characters.

Here, the mixed character recognition and retrieval unit uses a character region detection unit for learning the character regions of the generated pages and classifying them into individual characters, a character region detection unit for detecting the character regions separated by the character region division units, and a network. A language type classification unit to check which language is used, and if there is a language with sub-components, a sub-element location recognition unit that recognizes or recognizes the configuration of sub-components using a network, and a document with sub-components. In one case, the sub-component type recognition unit for recognizing each character by retrieving the combination and structure of sub-components from the database of each character is identified, and the components and their relations are searched first. Then, there is characterized in that it includes a component-based retrieval unit that enables recognizing and retrieving characters by re-searching the relationships between the components therein.

In the case of characters without sub-components, it is recommended to recognize the characters directly using either a network learning whether the sub-components are included or a deep learning network including a faster-RCNN or a mask-RCNN with a region proposal network. It is characterized by.

In addition, when the lower component is the upper element used as one element, the upper component is recognized first in the text and classified based on the relationship between the components.

The search method using the generation of characters and books for automatic recognition of mixed characters according to the present invention for achieving other objects is performed by performing a variation formation step, a virtual page formation step, and a network learning step to generate characters and books for automatic recognition of mixed characters. A step of separating individual characters by learning a character area of pages generated using the results learned in the network learning step; in the case of a character without a sub-component, recognizes the character directly, and the language with the sub-component Recognizing the configuration of the lower component or determining its location using a ramen network; if the lower component is a parent element that is used as one element, the upper component is recognized first in the text and the relationship between the components is established. It characterized in that it comprises; sorting by criteria; in the case of a document with sub-components, retrieving each character by retrieving the combination and structure of sub-components from the database of each character and outputting a search result; do.

Here, the step of forming the virtual page includes a step of generating a size and spacing variation to generate a variation similar to the actual document by randomly changing the character size and the spacing between the characters next to the character, and the page itself extracted from the actual document Create a document similar to the real environment by generating noise, blocks, pictures, and tables of various variations with a deep learning network based on non-character areas with multiple noises, blocks, figures, and tables It characterized in that it comprises a step of generating an area-based document.

In the network learning step, a text area detection step of learning a text area of pages generated by the virtual page forming unit to detect a text area, and a language type classification for classifying and learning what language the characters generated in the variation forming unit are. And a step of learning the position of the lower component using a classification network in the case of a character having a lower component, and learning the type and inclusion of the lower component. .

The character and book generation system and method for automatic recognition of mixed characters according to the present invention as described above, and the search system and method using the same have the following effects.

First, it is possible to increase the efficiency of mixed character recognition and search by deep learning including variation of individual characters and virtual page formation.

Second, the composition and position of each element constituting the characters are organized into a database, and various variations of each digitized individual character or one side can be learned with VAE (Variational AutoEncoder) or GAE (Generative Adversal Network) to create a new image. By doing so, it is possible to build a highly accurate and efficient mixed character search system.

Third, randomly fill the generated languages and types of characters and noise into pages of various sizes and shapes to create a book with pages close to infinity that are difficult to distinguish from the actual books. It is possible to realize the optimal professional search service by learning deep learning and recognizing characters while knowing the answers such as meaning.

Fourth, in the case of characters with sub-elements (number of Chinese characters, phonology of Hangul), use a network that learns whether to include sub-components using a classification network, or faster-RCNN with a region proposal network. In order to improve accuracy by learning the location of the lower component using any one of deep learning networks including mask-RCNNs.

Fifth, in the case of a document with sub-components, each character is recognized by retrieving the combination (structure) and structure (location) of the sub- (parent) components from the database of each character, and then recognizing the relations with the components and then database To search the component first, and then re-search the relationship between the components in it so that characters can be quickly recognized and searched.

Sixth, it is possible to increase the accuracy and efficiency of high document recognition and retrieval by constructing a wall entry system that is not commonly used in the process of forming variations and forming virtual pages.

1 is a block diagram of a system for generating characters and books for automatic recognition of mixed characters according to the present invention
Figure 2 is a flow chart showing a method for generating text and books for automatic recognition of mixed characters according to the present invention
3A and 3B are diagrams illustrating an example of character data processed by a character and book generation technique for automatic recognition of mixed characters according to the present invention.
4A and 4B are diagrams illustrating an example of classification result characteristic graph and classification result character data for generating characters and books for automatic recognition of mixed characters according to the present invention.
5 is a configuration diagram of a generated virtual page of text and book generation for automatic recognition of mixed characters according to the present invention
6 is a block diagram showing an example of a network learning tool for generating characters and books for automatic recognition of mixed characters according to the present invention
7A and 7B are configuration diagrams of detection results according to the generation of characters and books for automatic recognition of mixed characters according to the present invention.
8 is a block diagram showing the result of character recognition in Dongguk Jungwoon according to the present invention
9 is a block diagram showing the concept of a single shot detector (SSD) according to the present invention
10A and 10B are wall data and variation configuration diagrams for wall input processing according to the present invention.
11 is a block diagram of a search system using text and book generation for automatic recognition of mixed characters according to the present invention.
12 is a flow chart showing a search method using text and book generation for automatic recognition of mixed characters according to the present invention

Hereinafter, a description will be given of a preferred embodiment of a system and method for generating text and books for automatic recognition of mixed characters and a search system and method using the same, as follows.

Characters and advantages of a character and book generation system and method for automatic recognition of mixed characters according to the present invention and a search system and method using the same will become apparent through detailed description of each embodiment below.

1 is a block diagram of a system for generating characters and books for automatic recognition of mixed characters according to the present invention.

The system and method for generating texts and books for automatic recognition of mixed characters according to the present invention and the search system and method using the same are organized into a database for the composition and location of each element constituting characters, and various variations of each individual character or one side digitized. It is possible to build a highly accurate and efficient mixed character search system by learning a VAE (Variational AutoEncoder) or a GAE (Generative Adversal Network) to create new images.

To this end, the present invention learns a combination of squares based on actual characters, so that all combinations of characters can be generated with a combination of multiple squares, and possible variations of various sizes and shapes, thickness, slope, typeface, etc. It may include a configuration to learn and generate various types of test images to be included.

The present invention randomly fills the generated languages and types of characters and noise into pages of various sizes and shapes to create a book with pages close to infinity that are difficult to distinguish from actual books, and the area and shape of each character And learning to perform deep learning while knowing the answer such as the meaning in advance so as to enable character recognition.

The present invention uses a network that learns whether to include sub-components by using a classification network in case of characters with sub-components (number of Chinese characters, phonology of Hangul), or faster- with region proposal network. It may include a configuration for learning the location of the sub-component using any one of the deep learning network including RCNN, mask-RCNN.

In the present invention, in the case of a document having sub-components, each character is recognized by retrieving the combination (structure) and structure (location) of the sub- (higher) components in the database of each character, and after identifying the components and their relations It may include a component that searches for a component in the database first, and then re-searches the relationships between the components in it to quickly recognize and search characters.

The present invention may include a configuration for increasing the accuracy and efficiency of high document recognition and retrieval by constructing a wall entry system that is not commonly used in the process of forming variations and forming virtual pages.

The character and book generation system for automatic recognition of mixed characters according to the present invention, as shown in FIG. 1, learns the variation of individual characters or one side to learn the combination of one side based on the actual character, and can combine all of the combinations in one side. Having a virtual page by randomly filling characters and noise of various languages and types generated in the variation forming unit 100 and the variation forming unit 100 to generate characters in pages of different sizes and shapes A virtual page forming unit 200 for generating a book, a character area of pages generated by the virtual page forming unit 200, a character area is detected to detect the characters generated in the variation forming unit 100. And a network learning unit 300 that performs cognitive classification learning and learns the location and type of sub-components.

Here, the variation forming unit 100 learns the variation of individual characters or one side to learn the combination of one side based on the actual character, generates all the characters that can be combined with the combination of one side to form a lower component variation. It includes an element variation forming unit 10 and a character variation forming unit 11 that learns test images so as to include possible variations of different sizes, shapes, thicknesses, inclinations, and fonts to form character variations.

In addition, the virtual page forming unit 200 randomly changes the character size and the spacing between the next characters within a certain range to generate a variation similar to the actual document, and a variation document generating unit 20 that generates a variation similar to the actual document, and the actual document on the page itself. Based on non-character areas such as noise, blocks, figures, and tables extracted from the deep learning network, it generates noise, blocks, pictures, and tables of various variations, similar to the real environment. And an area-based document generation unit 21 for generating a document.

In addition, the network learning unit 300 learns the character area of the pages generated by the virtual page forming unit 200 to detect the character area, and the characters generated by the variation forming unit 100 are used. In the case of a character having a language type classification unit 32 for language recognition classification learning and a sub-element (the number of Chinese characters, the phonology of Hangul), the location of the sub-elements is learned by using a classification network, It includes a lower component position learning unit 33 and a lower component type learning unit 34 for learning the type and inclusion of the lower component.

Here, the network learning unit 300 directly detects the character area of the generated pages in a language without a sub-component, and detects and segments into individual characters.

The method of generating characters and books for automatic recognition of mixed characters according to the present invention is as follows.

2 is a flow chart showing a method for generating text and books for automatic recognition of mixed characters according to the present invention.

First, by learning the variation of individual characters or one side, the combination of the one side is learned based on the actual character, and all the characters that can be combined with the combination of one side are generated to form the lower component variation (S201).

Subsequently, the test images are trained to form the character variations so that possible variations of different sizes and shapes, thickness, slope, and typeface are included (S202).

3A and 3B are diagrams illustrating an example of character data processed by a character and book generation technique for automatic recognition of mixed characters according to the present invention.

Then, the virtual page forming step of generating a book having a virtual page is performed by randomly filling characters and noise of various languages and types generated in the variation forming unit 100 into pages of different sizes and shapes. (S203 )

Learning by deep learning while knowing the answer, such as the area, shape, and meaning of each character in advance (S204), generates a document similar to the real environment (S205).

Such a virtual page forming step is a step of generating a size and spacing variation that randomly changes the character size and the spacing between adjacent characters within a certain range to generate a variation similar to a real document, and various noises extracted from the actual document on the page itself. And an area that generates documents similar to the real environment by generating noise, blocks, pictures, and tables of various variations with a deep learning network based on non-character areas such as blocks, figures, and tables. It includes the steps of generating the base document.

5 is a configuration diagram of a generated virtual page of text and book generation for automatic recognition of mixed text according to the present invention, and FIG. 6 is a network learning tool for text and book generation for automatic recognition of mixed text according to the present invention. It is a block diagram showing an example.

Then, the character area detection and classification learning of the generated characters are performed, and a network learning step of learning the location and type of the sub-elements is performed (S206).

4A and 4B are diagrams illustrating an example of classification result characteristic graphs and classification result character data for generating characters and books for automatic recognition of mixed characters according to the present invention.

Here, the network learning step is a character area detection step of learning a character area by learning a character area of pages generated by the virtual page forming unit 200, and classifying and learning what language the characters generated by the variation forming unit 100 are. Classifying the type of language, and learning the position of the sub-element using a classification network in case of characters with sub-elements (the number of Chinese characters, the phonology of Hangul), and the type of the sub-elements And learning whether to include.

Such a network learning step is to perform a classification network classifying the generated characters into which language.

In the case of characters with sub-components (number of Chinese characters, phonology of Hangul), use a network to learn whether the sub-components are included by using a classification network, or deeper such as faster-RCNN or mask-RCNN with region proposal network. Learning the location of sub-elements using the learning network, and in the case of a language without sub-elements, learn each character directly in the same way.

Sub-components can generate a large number of characters to improve learning accuracy.

A network that learns the character area of pages created in the virtual page formation step by using a deep learning network such as RCNN, which has a region proposal network that can learn the text area first, and detects and segments into individual characters. Learn.

7A and 7B are configuration diagrams of detection results according to the generation of characters and books for automatic recognition of mixed characters according to the present invention, and FIG. 8 is a diagram illustrating the results of character recognition in Dongguk Jungwoon according to the present invention.

And Figure 9 is a block diagram showing the concept of a single shot detector (SSD) according to the present invention.

The present invention may include a configuration for increasing the accuracy and efficiency of high document recognition and retrieval by constructing a wall input system that is not commonly used in the process of forming variations and forming virtual pages.

10A and 10B are wall data and variation configuration diagrams for wall input processing according to the present invention.

The construction of a search system using text and book generation for automatic recognition of mixed characters according to the present invention will be described in detail as follows.

11 is a block diagram of a search system using text and book generation for automatic recognition of mixed characters according to the present invention.

The search system using text and book generation for automatic recognition of mixed characters according to the present invention, as shown in FIG. 11, learns variations of individual characters or one side to learn combinations of one side based on actual characters, and thus to a combination of one side. It is possible to randomly fill in various languages and types of characters and noise generated by the variation forming unit 100 and the variation forming unit 100 to generate all combinable characters in pages of different sizes and shapes. A virtual page forming unit 200 for generating a book with pages, a character area of pages generated by the virtual page forming unit 200, and a character area detected to detect a character area. The network learning unit 300 for classifying and learning the location and type of sub-components, and learning the character areas of the pages generated using the results learned by the network learning unit 300, thereby learning individual characters. And a mixed character recognition and search unit 400 for recognizing and retrieving characters by recognizing the location of the sub-components by identifying the language, and recognizing the location of the sub-components.

Here, the mixed character recognition and search unit 400 learns the character region of the generated pages and detects the character region divided by the character region division unit 41 and the character region division unit 41. The character area detection unit 42, a language type classification unit 43 for confirming which language is used by using a network, and, if a language having a sub-element, uses a network to recognize the configuration of the sub-elements or determine its location The sub-component position recognition unit 44 and the sub-component type recognition unit 45 for retrieving a combination of sub-components and structure relations in a database of each character to recognize each character in the case of a document having sub-components. ), and the component-based search unit 46 for recognizing and retrieving characters quickly by retrieving the relations between the components after retrieving the components from the database after recognizing the components and their relations. do.

Here, in the case of a character without a sub-component, the character is directly recognized using a network learning whether the sub-component is included or a deep learning network such as faster-RCNN or mask-RCNN with a region proposal network.

In the case of a parent element that is used as a single element by combining lower components, the upper component is recognized first in the text and the relationship between the components is classified as one of the structural relations (4-20).

And, in the case of a document with sub-components, each character is recognized by searching for the combination of the sub-(high) components and the structure relation (location) in the database of each character, and using image captioning network (deep learning) After grasping the components and their relationships, the database is searched for the components first, and then the relationships between the components are re-searched so that characters can be quickly recognized and searched.

The search method using text and book generation for automatic recognition of mixed characters according to the present invention will be described in detail as follows.

12 is a flow chart showing a search method using text and book generation for automatic recognition of mixed characters according to the present invention.

First, characters and books are generated for automatic recognition of mixed characters by performing a variation formation step, a virtual page formation step, and a network learning step. (S1201)

Subsequently, the character area of the pages generated using the result learned in the network learning step is learned to distinguish individual characters. (S1202)

In the case of characters without sub-components, the characters are directly recognized using a network learning whether the sub-components are included or using a deep learning network such as faster-RCNN or mask-RCNN with a region proposal network (S1203). )

Subsequently, if the language has the sub-components, the configuration of the sub-components is recognized by using a network or its location is determined (S1204).

In the case of a higher element used as a single element by combining lower components, the upper component is recognized first in the text and the relationship between the components is classified as one of the structural relations (4-20) (S1205).

Subsequently, in the case of a document with sub-components, the combination of the sub-(high) components and the structure relation (location) of each character database is searched to recognize each character and output the search result. (S1206)

At this time, using the image captioning network (deep learning), the components and their relationships are identified, and then the components are first searched in the database and then the relationships between the components are re-discovered to quickly recognize and search characters.

The character and book generation system and method for automatic recognition of mixed characters according to the present invention described above, and the search system and method using the same are organized into a database for each element constituting characters and digitized for each individual character or edition. It is possible to build a highly accurate and efficient mixed character search system by learning various variations of the room with VAE (Variational AutoEncoder) or GAE (Generative Adversal Network) to create new images.

It will be understood that the present invention is implemented in a modified form without departing from the essential characteristics of the present invention as described above.

Therefore, the specified embodiments should be considered in terms of explanation rather than limitation, and the scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the equivalent range are included in the present invention. Should be interpreted.

100. mutation formation
200. Virtual page forming unit
300. Network Learning Department
400. Mixed character recognition and search unit

Claims (15)

A variation forming unit configured to learn individual characters or variations of one side and learn combinations of one side based on actual characters to generate all characters that can be combined with a combination of sides;
A virtual page forming unit that randomly fills various languages and types of characters and noise generated in the variation forming unit into pages of different sizes and shapes to generate a book having virtual pages;
And a network learning unit that learns a character region of pages generated by the virtual page forming unit, classifies and learns what language the characters generated by the variation forming unit are, and learns the location and type of sub-components. Text and book generation system for automatic recognition of mixed characters. The method of claim 1, wherein the mutation forming unit,
A lower component variation forming unit that learns individual characters or variations of one side to learn combinations of one side based on actual characters to generate all characters that can be combined with a combination of one side to form lower component variations;
Character and book generation system for automatic recognition of mixed characters, characterized in that it comprises a character variation forming unit to form a character variation by learning test images to include possible variations of different sizes and shapes, thickness, slope, and typeface. According to claim 1, The virtual page forming unit,
A size and spacing variation generation unit that randomly changes the spacing between the character size and the next character within a certain range to generate a variation similar to a real document;
Deep learning network generates noise, blocks, pictures, and tables of various variations based on non-character areas including noise, blocks, figures, and tables extracted from actual documents on the page itself Characters and books generation system for automatic recognition of mixed characters, characterized in that it comprises an area-based document generation unit for generating a document similar to the real environment. According to claim 1, The network learning unit,
A text area detection unit for learning a text area of pages generated by the virtual page forming unit to detect a text area;
A language type classification unit that classifies and learns what language the characters generated in the mutation forming unit are,
In the case of characters with sub-components, the sub-component location learning unit and the sub-component type learning unit for learning the location of the sub-components using a classification network and learning the types and inclusions of the sub-components. Character and book generation system for automatic recognition of mixed characters, characterized in that it comprises. [5] The method of claim 4, wherein in the case of a language without sub-components, characters and books are generated for automatic recognition of mixed characters, characterized in that the character areas of the generated pages are directly learned and detected and segmented as individual characters. system. Learning individual characters or variations of one side to learn a combination of sides based on actual characters, thereby generating all characters that can be combined with a combination of sides to form lower component variations;
Forming text variations by learning test images to include possible variations of different sizes and shapes, thickness, slope, and typeface;
A virtual page forming step of randomly filling characters and noise of various languages and types generated in the variation forming unit into pages of different sizes and shapes to generate a book having virtual pages;
And a network learning step of detecting a character area of pages generated in the virtual page forming step, classifying and learning what language the characters generated by variation formation are, and learning the location and type of sub-components. How to generate texts and books for automatic recognition of mixed texts. The method of claim 6, wherein the virtual page forming step,
A size and spacing variation generation step of randomly changing the size of the text and the spacing between the adjacent characters within a predetermined range to generate a variation similar to a real document;
Generates noise, blocks, pictures and tables of multiple variations with a deep learning network based on areas of non-characters with various noises, blocks, figures, and tables extracted from the actual document on the page itself Characterized in that it comprises a region-based document generation step for generating a document similar to the real environment Character and book generation method for automatic recognition of mixed characters. The method of claim 6, wherein the network learning step,
A text area detection step of learning a text area of pages generated by the virtual page forming unit to detect the text area;
A language type classification step of classifying and learning what language the characters generated in the mutation forming unit are,
Learning the location of the sub-element using a classification network if the character has the sub-element;
And learning the type and inclusion of sub-components. A variation forming unit that learns individual characters or variations of one side to learn combinations of one side based on actual characters, thereby generating all characters that can be combined with a combination of sides;
A virtual page forming unit for randomly filling characters and noise of various languages and types generated in the variation forming unit into pages of different sizes and shapes to generate a book having virtual pages;
A network learning unit that learns a character area of pages generated by the virtual page forming unit, detects a character area, classifies and learns what language the characters generated in the variation forming unit are, and learns the location and type of sub-components;
By learning the character area of the pages generated by using the results learned from the network learning unit, the individual characters are classified, and in which language, the composition of the lower component is recognized or the location is recognized to recognize and search the characters. A search system using text and book generation for automatic recognition of mixed characters, comprising: a mixed character recognition and search unit. The method of claim 9, wherein the mixed character recognition and search unit,
A character area divider for learning the character areas of the generated pages and classifying them into individual characters,
A character area detection unit that detects a character area separated by the character area division unit;
Language type classification unit to check the language using the network,
If the language with the sub-components, the sub-component location recognition unit for recognizing the location of the sub-elements using a network or determining its location,
In the case of a document having sub-components, a sub-component type recognition unit for recognizing each character by searching in a combination and structure relation of sub-components in the database of each character;
A character-based search unit that includes a component-based search unit that recognizes and searches characters by retrieving the relations between the components after retrieving the components from the database after recognizing the components and their relationships. Search system using text and book generation for recognition. 10. The method according to claim 9, in the case of a character without a sub-component, directly using a network learning whether the sub-component is included or a deep learning network including a faster-RCNN with a region proposal network or a deep learning network with mask-RCNNs. A search system using text and book generation for automatic recognition of mixed characters, which recognizes characters. 10. The method of claim 9, If the lower component is combined to be used as one element, the upper element,
A search system using text and book generation for automatic recognition of mixed characters, characterized by first recognizing the upper components from the characters and classifying them based on the relationship between the components. Generating characters and books for automatic recognition of mixed characters by performing a variation forming step, a virtual page forming step, and a network learning step;
Classifying individual characters by learning a character region of pages generated using the results learned in the network learning stage;
In the case of a character without a sub-component, directly recognizing a character, and in a language having a sub-component, recognizing the configuration of the sub-component or using a network to recognize a location thereof;
If the lower component is the upper element that is used as one element by combining, recognizing the upper component in the text first and classifying it based on the relationship between the components;
In the case of a document having sub-components, searching for a combination and structure relation of sub-components in the database of each character, recognizing each character, and outputting a search result. Search method using text and book generation. The method of claim 13, wherein the virtual page forming step,
A size and spacing variation generation step of randomly changing the size of the text and the spacing between the adjacent characters within a predetermined range to generate a variation similar to a real document;
Generates noise, blocks, pictures and tables of multiple variations with a deep learning network based on areas of non-characters with various noises, blocks, figures, and tables extracted from the actual document on the page itself A search method using text and book generation for automatic recognition of mixed characters, characterized by including an area-based document generation step of generating a document similar to a real environment. The method of claim 13, wherein the network learning step,
A text area detection step of learning a text area of pages generated by the virtual page forming unit to detect the text area;
A language type classification step of classifying and learning what language the characters generated in the mutation forming unit are,
Learning the location of the sub-element using a classification network if the character has the sub-element;
And learning the type and inclusion of sub-components. A search method using text and book generation for automatic recognition of mixed characters.

Images