DE102018119908A1 - Optical Character Recognition (OCR) system - Google Patents

Abstract

A system (1) for optical character recognition (OCR) from an image containing text is intended to minimize transcription errors and to improve the quality of the resulting machine-coded text. It includes:
- A text area dispatcher module (6) which is connected to at least one OCR engine (aTR1, aTR2, ... aTRn) and is configured such that it uses the image as input data to the OCR engine (aTR1, aTR2 , ... aTRn) and receives machine-coded texts as output data from the OCR engine (aTR1, aTR2, ... aTRn), and
- a text information management system module (14) configured to segment the image and machine-coded text into pairs of corresponding image segments and text segments, assign the pairs to a plurality of users (mTR) for review, and corrected text segments from those Users (mTR) receives.

Description

The invention relates to a system for optical character recognition (OCR) from an image that contains text.

Optical character recognition (also Optical Character Reader, OCR) is the mechanical or electronic conversion of images from typed, handwritten or printed text into machine-coded text, be it from a scanned document, a photo of a document, a scene photo (e.g. the text on signs and Posters in a landscape photo) or from a subtitle text that is superimposed on a picture (e.g. from a television program).

OCR is a common way to digitize printed text so that it is electronically edited, searched, stored more compactly, displayed online and used in machine processes such as cognitive calculations, machine translation, (extracted) text-to-speech, key data and text mining can be. OCR is a research area in the fields of pattern recognition, artificial intelligence and computer vision.

Current OCR systems are able to achieve high recognition accuracy for most fonts and support a large number of digital image file formats. Some systems are able to reproduce formatted output that comes very close to the original page with images, columns and other non-textual components.

Although the technology has advanced a lot in recent years, transcription errors are still inevitable. This applies in particular to exotic fonts or poor scan quality. Handwriting can only be processed via OCR with restrictions with a high learning effort and a high error rate. Transcription errors are sometimes simply accepted because manual correction is a lengthy, time-consuming and expensive task.

The object of the present invention is therefore to provide a system for optical character recognition from an image with text, which minimizes transcription errors and improves the quality of the resulting machine-coded text.

• - ein Textbereich-Dispatcher-Modul, das mit zumindest einer OCR-Engine verbunden ist und so konfiguriert ist, dass es das Bild als Eingangsdaten zu der OCR-Engine zuweist und maschinencodierte Texte als Ausgangsdaten von der OCR-Engine empfängt, und
• - ein Textinformations-Verwaltungssystem-Modul, das so konfiguriert ist, dass es das Bild und den maschinencodierten Text in Paare von entsprechenden Bildsegmenten und Textsegmenten segmentiert, die Paare einer Vielzahl von Benutzern zur Überprüfung zuweist und korrigierte Textsegmente von den Benutzern empfängt.

According to the invention, this object is achieved in that the system comprises:

• a text area dispatcher module which is connected to at least one OCR engine and is configured such that it assigns the image as input data to the OCR engine and receives machine-coded texts as output data from the OCR engine, and
• a text information management system module configured to segment the image and machine-coded text into pairs of corresponding image segments and text segments, assigning pairs to a plurality of users for review, and receiving corrected text segments from the users.

The invention is based on the consideration that manual processing of automatically recognized OCR results should take place. However, the problem arises here that the manual post-processing of a complete document is of high quality, but is complex and expensive. Using voluntary users is cheaper, but the quality will be worse. There are also problems of copyright and confidentiality. To solve the problems mentioned above, segmentation of the image and text segments is carried out, i.e. Image and text are divided into parts that belong together, with each pair containing a section of the pictorial text and the machine-readable transcription of this section. These segmented pairs are then distributed to and corrected by a large number of users for manual review.

In an advantageous embodiment, the segmentation takes place at the sentence level. Such a segmentation is technically easy to do, on the one hand, and on the other hand it makes it easier for the distributed users to check it.

In a further advantageous embodiment, the system further comprises a dictionary module, the text information management system module being further configured to check the corrected text segments in the dictionary module. This achieves a double check of the text recognition: the segments automatically recognized by the OCR engine are checked manually by the users, and the corrections of the users are checked again using a comprehensive dictionary, i.e. it is compared whether the words determined by the users exist in the dictionary. If this is not the case, a further check, e.g. be triggered by another, different user.

In a first embodiment, the system comprises an interface module which provides a human-machine interface on which the users can display pairs assigned to them and enter corrections. Such an interface module can be installed on a desktop computer, for example, receives the image and text segments and makes them available to the user on the locally generated human-machine interface. It is particularly advantageous the interface module is a web server module and the human-machine interface is a web interface. The system is therefore designed for online access, so that users can log on to the web interface with their name and password, for example, and can display the image and text segments there online. Corresponding input fields are provided for the correction.

In a second, alternative or additional embodiment, the system comprises an interface module, which is configured in such a way that it transmits a plurality of pairs, each assigned to a user, to a mobile terminal of the user and receives corrected text segments generated on the mobile terminal. In this embodiment, the correction can be done offline, i.e. on the user's terminal, i.e. a smartphone, tablet, laptop or similar a suitable app is installed that receives and stores a plurality of text segments from the interface module. The user can then carry out the functions described above for the human-machine interface offline in the app, the app generating the human-machine interface and in this case the interface module taking over the communication with the app. In this case, too, the communication advantageously takes place via the Internet, so that the interface module here is also advantageously designed as a web server module. The corrections are saved in the app and later transferred to the interface module. This allows users to work even when there is no continuous online connectivity.

Accordingly, the above-mentioned object is still achieved by a computer program product, i.e. For example, an app for a smartphone, tablet or laptop that can be loaded into the memory of at least one mobile terminal and contains software code sections that cause the mobile terminal to display a plurality of pairs of image segments and text segments assigned to the user of the terminal by an interface. Receive module and display on a display element of the terminal, receive corrections to the text segment through a user interface of the terminal and send corrected text segments to the interface module.

• - ein Textbereich-Vorverarbeitungsmodul, das konfiguriert ist, um dem Bild eine Anzahl von Attributen zuzuweisen,
• - ein Benutzerverwaltungsmodul, in dem Profile einer Mehrzahl von Benutzern gespeichert sind,

wobei das Textbereichsvorbereitungsmodul ferner so konfiguriert ist, dass es die Paare auf Basis eines Abgleichs der Attribute mit den Profilen zuweist. Das heißt, es wird eine Voranalyse des Textes durchgeführt, die Attribute z.B. zu Typ, Stil, Sprache, Layout etc. des Textes im Bild zuordnet. Zusätzlich werden in Profilen persönliche Informationen zu jedem Benutzer gespeichert, die eine Einschätzung erlauben, für welchen Typ, Stil, Sprache, Layout etc. der jeweilige Benutzer besonders geeignet ist. Durch einen Abgleich dieser Informationen kann dann ein Benutzer ermittelt werden, der für die Korrektur des Textes besonders geeignet ist.In a further advantageous embodiment, the system according to one of the preceding claims, further

• a text area preprocessing module configured to assign a number of attributes to the image,
• a user management module in which profiles of a plurality of users are stored,

wherein the text area preparation module is further configured to assign the pairs based on a matching of the attributes with the profiles. This means that a pre-analysis of the text is carried out, which assigns attributes to the type, style, language, layout, etc. of the text in the image. In addition, personal information about each user is stored in profiles, which allows an assessment of the type, style, language, layout, etc. for which the respective user is particularly suitable. A comparison of this information can then be used to determine a user who is particularly suitable for correcting the text.

The profiles advantageously contain one or more information from the following group: language level, ability to read special fonts, preferred working hours and quality rating. In this way, users can be identified who speak the appropriate language of the text (even at a suitable level), who can read the font of the text well, whose working hours are such that a quick check is guaranteed, etc. In addition, users who deliver a better quality of transcriptions, with preference given to assignment.

In a further advantageous embodiment of the system, the text information management system module is further configured such that it assigns a pair to a plurality of users and changes a quality rating in the profile on the basis of a comparison of the plurality of corrected text segments of the plurality of users. By assigning a pair to several users, on the one hand, for example, the confidence regarding the transcription can be further increased. If several users confirm that a transcription is correct, the probability increases that this is actually the case. In addition, the quality of users can be checked by multiple distribution: If users have confirmed the correct transcription determined in this way or corrected an incorrect transcription correctly, their quality level in the profile can be increased. Conversely, the quality level of a user can be reduced if the user has not recognized an incorrect transcription.

A computer program product is advantageously loadable into the memory of at least one computer and contains pieces of software code that cause the computer to perform the function of a system described above.

The advantages of the invention are, in particular, that by distributing an automated task, namely the OCR transcription over several systems and / or persons, errors of individual systems can be eliminated and a confidence level of the resulting voice can be deducted. The quality of the transcription is increased.

Embodiments of the invention are described in more detail with reference to drawings, in which the only one

FIG shows a schematic representation of an exemplary system for optical character recognition.

The diagram shown in the FIG shows a system for optical character recognition 1 which uses a dynamic mix of automatic and manual transcription. The modules of the system 1 are realized by computer processing, ie they represent modules of a computer program that is loaded into the memory of one or more hardware devices, so that the hardware device is responsible for each module of the system 1 performs the automated steps described. The hardware can be localized or distributed, stationary or mobile, or even implemented as a cloud-based system.

With the system 1 the input is received in the form of a scanned text document. Common file formats are pdf, jpg, tiff, jp2, etc. The scanning of texts is an industry-standard procedure and is outside the scope of this description. Those skilled in the art know how to perform scans to obtain digital graphic representations of documents in the above or other graphic file formats.

In a first step of text recognition, a pre-analysis is carried out in a text area analyzer module 2 (Text Area Analyzer (TAA) module). The TAA module 2 generates attributes that apply to both the entire document and parts of the document. Attributes that relate to the entire document are, for example, the document type (book, newspaper, etc.). In addition, the TAA module leads 2 document segmentation, ie it divides the image of the page into different areas with different properties such as: text areas, illustration areas, mathematical / chemical formula areas, handwritten text areas etc. These characteristics are shown as attributes and assigned to the individual areas as metadata.

The TAA module is an alternative 2 implemented as a combination of automatic analysis and manual (human) quality assurance and correction. Ie the TAA module 2 is designed for human interaction through a user interface to the quality of analysis by the TAA module 2 to improve. This is advantageous in the case of very heterogeneous layouts (eg magazine covers) or poor scan quality (eg old micro fiches).

In the embodiment shown in the FIG, a subsequent text area preprocessing module 4 (Text Area Pre-Processor (TAPP) module) used to filter and preprocess areas in the TAA module 2 identified as text information. This preprocessing can include image enhancement, contrast adjustment or binarization and improve the quality of text recognition. The TAPP module 4 is optional, however, and may not be available in other versions.

A subsequent text area dispatcher module 6 (Text Area Dispatcher (TAD) module) then uses OCR (Optical Character Recognition) engines to transcribe the text contained in the text areas. The TAD module chooses 6 an automated system for optical character recognition for each text area ( ATR1 . ATR2 ... ATRN ) from a collection of optical character recognition engines 8th , For this purpose, the image areas and metadata of the respective documents from the TAD module 6 for optimized assignment of the transcription to the subsequent automated systems for optical character recognition ( ATR1 . ATR2 ... ATRN ) used. The TAD module uses a self-learning algorithm (e.g. neural networks) to gradually improve the assignment based on the previous individual aTR (automatic text recognition) results.

The aTR engines are various industrial quality OCR engines. There are several systems on the market that cover a wide range of fonts, languages and layouts (e.g. Tesseract, ABBYY Fine Reader). Other systems have been developed for very specific input (input data), e.g. Chinese, Arabic, Medieval Latin, etc. Typically, the OCR engines themselves are complex systems with their own strengths and weaknesses. Therefore, processing the same input by different systems does not usually lead to an identical output (output data).

After text recognition in one or more OCR engines from the collection 8th has taken place, receives and evaluates an OCR validation engine module 10 (OCR Validation Engine (OVE) module) the results. The evaluation takes place in two steps, which are described below.

In an assessment step of the OVE module 10 the quality of the transcription (either from the aTR systems of the collection described above 8th or from the mTR described below using the dictionary search in a dictionary module 12 (DICT module) and the confidence information of the various aTR systems. Typically, such aTR systems together with the transcription result provide a confidence indication. The confidence indications of the individual aTRs are in the OVE module 10 standardized with a common reference system for the resulting confidence level. Transcriptions are additionally made by a dictionary search in the dictionary module 12 Verified. The confidence level of transcriptions with a dictionary comparison is increased.

The OVE module 10 selects the transcription with the best rating for the second step of the validation. Based on their respective confidence level, transcriptions are accepted or marked for post-processing.

In an alternative embodiment, the OVE module uses 10 the so-called alternative word voting, which is provided by some aTR implementations. In this case, an aTR provides several possible transcriptions for a word with individual confidence levels. The OVE module 10 combines the results of several aTRs and selects the option with the highest combined confidence. All word votes by individual aTRs and alternative word votes (if available) are processed and the confidence levels are calculated. The transcription with the highest combined confidence level is then accepted or taken into account for post-processing.

In a further alternative embodiment of the system 1 become transcriptions with satisfactory confidence from the OVE module 10 randomly selected for post-processing in order to increase the overall quality and promote self-learning. The random selection can be weighted with the confidence level in order to prefer transcriptions with a confidence level just above the threshold for postprocessing.

In another variant of the OVE module 10 and DICT module 12 updates the OVE module 10 the dictionary module 12 or the dictionaries (DICT) with new words based on transcriptions with a very high level of confidence. In another variant, updated aTR engines are provided with updated dictionaries to improve aTR performance.

The OVE module 10 evaluates the individual performance of the aTRs in relation to the text attributes contained in the metadata (e.g. language, font). The OVE module 10 provides such assessments to the TAD module 6 to improve the assignment decision. If, for example, five different aTRs are available and the coordination is only ever between three aTRs and the results of the other two aTRs are regularly rejected, the TAD module orders 6 assign the three most appropriate aTRs for a particular language / font based on the previous assessment.

Transcriptions marked for manual post-processing become a text information management system module 14 (Text Information Management System (TIMS) module). The original image and the transcription results are segmented there, typically at the sentence level. A corresponding pair, consisting of an image segment and a transcription segment, forms a post-processing task. All tasks that are created from multiple transcriptions are processed in a random order in order to recompile larger text segments by mTRs (manual text recognition) in a group 16 to prevent.

The TIMS module 14 assigns the above tasks to the mTRs (ie people) based on their profile, which is stored in a user management module (UM module 18 ) is saved. Profiles can include information about language levels, the ability to read special fonts, preferred working hours, and quality ratings (where permitted by law). In the exemplary embodiment, these mTR users use a web interface 20 by a web server (WS) module 22 is provided to see the image contained in a task and the transcription result. The transcription results are confirmed or corrected by the respective mTR. The people in the group 16 Those involved in the manual correction process (mTR) can be volunteers or receive a financial reward for their work.

In an advantageous embodiment, the mTRs work mobile, for example from home, while traveling or to use short waiting times. A batch of sets is provided per mTR and stored locally in a mobile application while connectivity is available. An mTR can work on the transcription of the entire batch in an offline environment. The results are sent to the TIMS module 14 sent as soon as connectivity is restored. This variant allows the area of application of people working as mTRs to be extended to regions with less developed infrastructure.

The TIMS module 14 collects the results of mTRs via the WS module 22 and applies a dictionary search in the DICT module 12 to evaluate corrections provided by mTRs. From the results of the dictionary search, the user profile data from the UM module 18 and the previous confidence level from the OVE module 10 a new level of confidence is derived. The TIMS module 14 plans tasks with an unsatisfactory confidence level for further post-processing through additional mTRs until a satisfactory confidence level is reached.

In an alternative embodiment, tasks are initially assigned to several mTRs in order to enable a better evaluation by comparing the results of several mTR corrections.

If a satisfactory level of confidence is reached for a task, the TIMS module gives 14 a feedback to the UM module 18 which mTRs provided correct or incorrect transcriptions. The UM module 18 then updates the quality level in the respective mTR profile to include future task assignments and task evaluations by the TIMS module 14 to improve. The TIMS module 14 then reinserts the tasks into the original context of a transcription.

The UM module 18 alternatively uses the feedback of the TIMS module 14 to the rewards for users (mTR) of the group 16 to calculate.

LIST OF REFERENCE NUMBERS

1

Optical character recognition system

2

Text Area Analyzer Module

4

Text Area preprocessing

6

Text Area dispatcher module

8th

Collection of optical character recognition engines

10

OCR validation engine module

12

Dictionary module

14

Text information management system module

16

Group of manual translators

18

User management module

20

Web interface

22

Web server module

Claims (10)

System (1) for optical character recognition (OCR) from an image containing text, comprising: - A text area dispatcher module (6) which is connected to at least one OCR engine (aTR1, aTR2, ... aTRn) and is configured such that it uses the image as input data to the OCR engine (aTR1, aTR2 , ... aTRn) and receives machine-coded texts as output data from the OCR engine (aTR1, aTR2, ... aTRn), and - a text information management system module (14) configured to segment the image and machine-coded text into pairs of corresponding image segments and text segments, assign the pairs to a plurality of users (mTR) for review, and corrected text segments from those Users (mTR) receives. System (1) according to Claim 1 , with segmentation taking place at the record level. System (1) according to Claim 1 or 2 further comprising a dictionary module (12), the text information management system module (14) being further configured to check the corrected text segments in the dictionary module (12). System (1) according to one of the preceding claims, comprising an interface module (22) which provides a human-machine interface (20) on which the user (mTR) can display pairs assigned to them and enter corrections. System (1) according to one of the preceding claims, comprising an interface module (22) which is configured such that it transmits a plurality of pairs each assigned to a user (mTR) to a mobile terminal of the user (mTR), and receives corrected text segments generated on the mobile terminal. System (1) according to one of the preceding claims, further comprising a text area preprocessing module (4) configured to assign a number of attributes to the image, - a user management module (18) in which profiles of a plurality of users (mTR) are stored, the text area preparation module (6) being further configured to assign the pairs on the basis of a comparison of the attributes with the profiles. System (1) according to Claim 6 The profiles contain one or more information from the following group: language level, ability to read special fonts, preferred working time and quality rating. System (1) according to Claim 7 , wherein the text information management system module (14) is further configured to assign a pair to a plurality of users (mTR), and based a comparison of the plurality of corrected text segments of the plurality of users (mTR) changes a quality rating in the profile. Computer program product which can be loaded into the memory of at least one computer and contains software code sections which cause the computer to perform the function of a system (1) according to one of the preceding claims. Computer program product that is loadable into the memory of at least one mobile terminal and contains software code sections that cause the mobile terminal to receive a plurality of pairs of image segments and text segments assigned to the user (mTR) of the terminal from an interface module (22) and to be displayed on a display element of the terminal, to receive corrections of the text segment by a user interface of the terminal and to send corrected text segments to the interface module (22).

Images