BE1025134B1 - Method of identifying a character in a digital image - Google Patents

Abstract

The invention relates to a method for combining a first optical character recognition (OCR) (12) and a second OCR (100). The first OCR (12) operates first on a string image (11). Its output (13) (first identified characters, character positions and character likelihood parameters) is used to generate a first graph (16). Segmentation points related to the positions of the first identified characters (14) are used as input by the second OCR (100) by performing combined segmentation and classification on the character string image (11). The output (17) (second identified characters, character positions and character likelihood parameters) of the second OCR (100) is used to update (20) the first graph (16) to generate a second graph (21). ) which combines the output (13) of the first OCR (12) with the output (17) of the second OCR (100). Decision models are then used to modify (22) the path weights in the second graph (21) to generate a third graph (23). A better path is determined (24) on the third graph (23) to obtain identification (25) of the characters present in the character string image (11).

Description

(30) Priority data:

(73) Holder (s):

IRIS.

1435, MONT-SAINT-GUIBERT

Belgium (72) Inventor (s):

COLLET Frédéric 1040 ETTERBEEK

Belgium

HAUTOT Jordi 4122 NEUPRE

Belgium

DAUW Michel

1831 MACHELEN

Belgium

DE MUELENAERE Pierre 1490 COURT-SAINT-ETIENNE Belgium (54) Method for identifying a character in a digital image (57) The invention relates to a method for combining a first optical character recognition (OCR) (12) and a second OCR (100). The first OCR (12) operates first on a character string image (11). Its output (13) (first identified characters, positions of the characters and likelihood parameters of the characters) is used to generate a first graph (16). Segmentation points related to the positions of the first identified characters (14) are used as input by the second OCR (100) by performing combined segmentation and classification on the character string image (11). The output (17) (second characters identified, character positions and character likelihood parameters) of the second OCR (100) is used to update (20) the first graph (16) in order to generate a second graph (21 ) which combines the output (13) of the first OCR (12) with the output (17) of the second OCR (100). Decision models are then used to modify (22) the path weights in the second graph (21) in order to generate a third graph (23). A better path is determined (24) on the third graph (23) in order to obtain the identification (25) of the characters present in the character string image (11).

... image of chains as caramàres

Prsmlère

OCR

Promtem candie, -sis characters ”Γ.

Generation of the graph prernsr First graph

..5

....

_z 100 u

S & grneniaiien classification comfoinioa

Second caïiiiirfai • character with s? position '' ____ eUa f.-rr »ab> lité Ξ1 .............. 7 .............

Update to Graph Day

Der.i.xlèmë ~ „____ DD.rap.te ____

Addliior · ai - ^î; ™ -.:.7

Trorstèrne

tewnjicabon clos caracferas

.. 17

BELGIAN INVENTION PATENT

FPS Economy, SMEs, Middle Classes & Energy Publication number: 1025134 Deposit number: BE2017 / 5263 Intellectual Property Office International Classification: G06K 9/34 G06K 9/62 G06K 9/68 Date of issue: 16/11/2018

The Minister of the Economy,

Having regard to the Paris Convention of March 20, 1883 for the Protection of Industrial Property;

Considering the law of March 28, 1984 on patents for invention, article 22, for patent applications introduced before September 22, 2014;

Given Title 1 “Patents for invention” of Book XI of the Code of Economic Law, article XI.24, for patent applications introduced from September 22, 2014;

Having regard to the Royal Decree of 2 December 1986 relating to the request, the issue and the maintenance in force of invention patents, article 28;

Given the patent application received by the Intellectual Property Office on 04/13/2017.

Whereas for patent applications falling within the scope of Title 1, Book XI of the Code of Economic Law (hereinafter CDE), in accordance with article XI. 19, §4, paragraph 2, of the CDE, if the patent application has been the subject of a search report mentioning a lack of unity of invention within the meaning of the §ler of article XI.19 cited above and in the event that the applicant does not limit or file a divisional application in accordance with the results of the search report, the granted patent will be limited to the claims for which the search report has been drawn up.

Stopped :

First article. - It is issued to

I.R.I.S., Parc-Scientifique de Louvain-la-Neuve, Rue du Bosquet 10, 1435 MONT-SAINT-GUIBERT Belgium;

represented by

GEVERS PATENTS, Holidaystraat 5, 1831, DIEGEM;

a Belgian invention patent with a duration of 20 years, subject to the payment of the annual fees referred to in article XI.48, §1 of the Code of Economic Law, for: Method of identifying a character in a digital image.

INVENTOR (S):

COLLET Frédéric, Rue Ernest Havaux 27, 1040, ETTERBEEK;

HAUTOT Jordi, Rue Fine Pierre 14, 4122, NEUPRE;

DAUW Michel, Calenbergstraat 26, 1831, MACHELEN;

DE MUELENAERE Pierre, Rue des Couteliers 24, 1490, COURT-SAINT-ETIENNE;

PRIORITY (S):

DIVISION:

divided from the basic request:

filing date of the basic application:

Article 2. - This patent is granted without prior examination of the patentability of the invention, without guarantee of the merit of the invention or of the accuracy of the description thereof and at the risk and peril of the applicant (s) ( s).

Brussels, 11/16/2018,

By special delegation:

B E2017 / 5263

The present invention relates to methods and programs for identifying characters in a digital image.

Background technique

There are known optical character recognition (OCR) methods which convert text present in an image into machine readable code.

US Patent 5,519,786 describes a method of implementing a weighted voting scheme to accurately read and recognize characters in a scanned image. A plurality of optical character recognition processors scan the image and read the same image characters. Each OCR processor delivers a matched character corresponding to each character read. For a particular character reading, the characters reported by each OCR processor are grouped into a set of candidate characters.

For each coach character roidat, a weighting is generated according to ur ie confusion matrix which stores the probabilities a CCR particuH era to Identify characters precisely. The weights if then they can be compared to determine which candidate

see

Second, a preprocessor must quantify the strengths and weaknesses of the OCR processors to generate the cells of the confusion matrix which contain the probability that the character read by an OCR processor is the character reported by the OCR processor. This step can take time. Furthermore, if this

B E2017 / 5263 step is not well executed, for example because the training set used for this purpose is not suitable for a given type of character, a probability may be low for a character which is in fact well recognized and the OCR process can provide worse results than your independent OCR processors.

better and / or faster identification. This objective is achieved according to the invention by a process for selecting character candidates in a process for identifying characters in a digital image, the process comprising your steps consisting in:

a) apply a first character identification process to determine the first character candidates and a list of segmentation points for the first character candidates,

b) generate a list of character widths corresponding to a segmentation point of the list of segmentation points, the list of character widths,

d) apply a character classification process on the part of the digital image to obtain an hypothesis ID of a character possibly present in the part of the digital image and a likelihood parameter which relates to the likelihood that

e) select the hypothesis ID as the second character candidate in the digital image if the likelihood parameter

fulfills a first predetermined criterion.

B E2017 / 5263

In the method according to the invention, your segmentation points determined from the first character identification process (first OCR) are used as input for a combined segmentation and classification process (which includes steps b, c, d and e), which is a second OCR. As a result, the second OCR is able to perform better segmentation than if it operated alone without the entry of the first OCR. This allows the second OCR to use segmentation points that it would not have determined alone and therefore to recognize certain characters, for example 10 Asian characters, that it would not have recognized alone.

In addition, the second OCR works faster with this entry from the first OCR, because its initial estimate of the width of the next character is better than without this entry.

□ In the embodiments of the invention, the method 15 further comprises a step f) of updating the list of segmentation points with another segmentation point determined on the basis of the part of the digital image. corresponding to the second candidate character selected. This other segmentation point is expected to be useful for the next character you just identified.

In one embodiment of the invention, step f) comprises:

checks whether the other segment point is already in the segment point list and whether the other segment point is added to the segment point list if the other segment point is not not already present in the list of segmentation points.

In one embodiment of the invention, in step b), the list of character widths is generated on the basis of at least the list of segmentation points of the first character candidates. The list of character widths can therefore be generated quickly and easily.

B E2017 / 5263 reliable, since its determination is based on data from the first OCR.

Advantageously, steps b, c, d el e are performed for each segmentation point in the list of segmentation points.

In one embodiment of the invention, steps a, d and e are repeated for another character width from the list of character widths if the likelihood parameter does not meet the first predetermined criterion. If the likelihood parameter does not meet the first predetermined criterion, it indicates that the hypothesis ID may not be correct. It is therefore better to consider another part of the digital image starting at the same segmentation point and which therefore covers at least in part your same connected components.

In one embodiment of the invention, steps c, d

and e are repeated for a other character width from the list of character widths if the likelihood parameter satisfied first predetermined criterion and does not meet a second criterion predetermined. In a mode of realization of the invention, the process further includes the steps consists in : generate a data structure from the first

character candidates and their segmentation points and update the data structure with the second selected character candidate and a next segmentation point calculated from the character width of the second selected character candidate and the segmentation point used to determine te part of the digital image corresponding to the second character candidate selected.

B E2017 / 5263

A first goal of the data structure is to combine the results of the character classification process with your results of the first character identification process.

In an embodiment of the invention, step a) comprises the determination of first likelihood parameters of the first candidate characters providing an indication of the chance that the candidate character is correct, the data structure includes your first likelihood parameters first character candidates and the method further comprises the steps of;

change the scale of the first likelihood parameters of the first character candidates and / or the likelihood parameters of the second character candidates to make them comparable to each other and update the data structure with your likelihood parameters of the second character candidates .

character candidates

Advantageously, the method comprises the step of applying a decision modeling method to the updated data structure. Decision models improve the accuracy of the process.

In one embodiment of the invention, the method further comprises the steps consisting in:

determine a better route in the updated data structure and

B E2017 / 5263 of a

Preferably, the step of applying a decision modeling method comprises favoring the candidate characters which have the first likelihood parameters first character candidates and the second likelihood parameters of second character candidates satisfying the third predetermined criterion .

In one embodiment of the invention, the first character identification method provides first character candidates for characters of a first type, in which the character classification method provides second character candidates for characters of a second type, and the decision modeling method comprises the advantage of the character candidates of the first type if the character candidates are provided by the first character identification process and the fact of favoring the character candidates of the second type if the character candidates are

B E2017 / 5263 positions of first candidate characters by combined segmentation and classification or by an independent step preferably carried out between steps a) and b) and possibly between steps b) and c).

Another object of the present invention is to provide a method of identifying characters in a digital image.

This objective is achieved according to the invention by a process comprising:

a) the application of a first character identification method to determine first character candidates and first segmentation points corresponding to the first character candidates,

b) the generation of a first graph based on the first character candidates and the first segmentation points, in which your first segmentation points are represented by vertices and the first character candidates by edges,

c) the application of a second character identification process on at least part of the digital image to generate at least one second character candidate,

d) updating the graph by adding the at least one second character candidate to the first graph and

e) selecting from the updated graph of candidate characters as characters of the digital image.

In this process, the graph, when it is generated, includes as vertices the segmentation points from the first

OCR and as edges your ID from the first OCR. The graph is then updated with new vertices and new edges from the second OCR. During the selection of character candidates (step e), all of your edges are considered and none of them is prion discarded. There is no need to test the OCRs against one another. Other to determine their weaknesses, because all IDs

B E2017 / 5263 from all your OCRs are considered even if a decision rule favoring certain edges under certain conditions could be introduced in the graph. There is no need for a structure like the confusion matrix either.

Preferably, the second character identification method generates at least one second segmentation point and ie at least one second segmentation point is added to the graph in step d).

In an embodiment of the invention, the first character identification method provides first likelihood parameters providing a likelihood that your first character candidates are correct, the second character identification method provides second likelihood parameters providing a likelihood that the at least one second character candidate is correct, and the method further comprises the steps of:

change the scale of the first likelihood parameters and / or second likelihood parameters to make them comparable to each other, add your first likelihood parameters in the first graph and add the second likelihood parameters in the set graph up to date.

Advantageously, the method comprises the step of applying a decision modeling method to the updated graph.

In one embodiment of the invention, the decision modeling method comprises a rule which favors the candidate characters which have the first likelihood parameter and the second likelihood parameters meeting a third predetermined criterion.

B E2017 / 5263 or of a second type and the decision modeling method includes a rule which favors the character candidates corresponding to the first type of characters if these character candidates have been determined by the first character identification method and favors the character candidates corresponding to the second type of characters if these character candidates have been determined by the second character identification method.

segmentation from the first character identification process. With this method, the second character identification method is capable of performing better segmentation than if it transient readable on computer storing a program leads computer to execute a method to identify characters in

b)

c) nor one of a corresponding to the segmentation point provided by the position of a first candidate initial character and to a character width coming from the list of character widths.

B E2017 / 5263

d) apply a character classification process on the part of the digital image to obtain an hypothesis ID of a character possibly present in the part of the digital image and a likelihood parameter which relates to the likelihood that the hypothesis ID is correct and

e) select the iD hypothesis as the character in the digital image if the likelihood parameter meets a first predetermined criterion.

The method according to the invention is preferably designed to run on a computer.

Brief description of the drawings

The invention will be better understood by means of the following description and the attached figures.

Fig. 1 illustrates a functional diagram of an OGR process according to the invention.

essi

BE2017 / 5263 claims. The drawings described are only schematic and are not limiting. In the drawings, the size of some of the items may be exaggerated and not to scale for illustrative purposes. The dimensions and the relative dimensions therefore do not necessarily correspond to the actual reductions in order to practice the invention.

In addition, your terms first, second, third, etc.

in the description and the claims are used to distinguish between similar elements and not necessarily to describe a sequential or chronological order. The terms are interchangeable under appropriate circumstances and the embodiments of the event can operate in sequences other than those described or illustrated here.

In addition, the terms above, below, over, under and the like in the description and claims are used for descriptive purposes and not necessarily to describe relative positions. The terms thus used are interchangeable under appropriate circumstances and the embodiments of the invention described here can operate in other orientations than those described or illustrated here.

) Π

which the invention can be implemented rather than limiting the scope of the invention.

The term "comprising" used in your claims should not be interpreted as being limited to the elements or steps listed below; it does not exclude other elements or other steps. It must be interpreted as specifying the presence of the characteristics, whole numbers, stages or components mentioned to which we refer, but does not prevent the presence or addition of one or more other characteristics, whole numbers, stages or components or their groups. As a result, the scope of the expression "a device

BE2017 / 5263 comprising A and B "should not be limited to devices consisting only of components A and B rather in relation to the present invention, the only listed components of the device are A and B and, in addition, the claim should be Interpreted as including equivalents of these components.

The term "character", as used here, refers to a symbol or sign used in writing such as a grapheme, a logogram, an alphabetical letter, a typographic ligature, a numeric number or a punctuation mark .

The expressions "Identification, identifier and ID", as they are used lai, refer to one or more characters recognized in machine readable code to obtain an explorable string of

The term "position", as used herein when referring to a particular position, refers to data that helps locate the character. For example, the position can be provided by the coordinates of the pixels of a bounding box which surrounds the character.

character can,

the leftmost black pixel of a character bounding box or the leftmost black pixel of the character.

The expression "segmentation part", as used herein, refers to a part of an image of a character string which undergoes processes to determine whether it represents a character, a group of characters, a pattern, etc. The leftmost point of a segmentation part is preferably a segmentation point.

The term "classification", as used herein, refers to the generation of at least one hypothesis on the identification of one or more

B E2017 / 5263 several characters. Each identification hypothesis or ID hypothesis is associated with a likelihood parameter.

The term "optical character recognition (OCR)", as used herein, refers to any conversion to text present in an image in readable code over a black and white image is intended refer to a group of black pixels which are connected to each other by black pixel guards.

B E2017 / 5263

Fig. 1 shows a block diagram of a process

OCR 10 according to the invention.

erence of the segmentation 14 of the image characters and of the weights 18 of the characters, the weights being parameters of likelihood. The first character candidates 13 with the segmentation points 14 of the characters in the image can be referred to as first segmentation points 14. the plurality of segmentation points 14 can be considered as a list of

The first OCR may or may not use models of the first segmentation points originating from the segmentation points of candidate characters 14. If necessary, the generation 15 of the first graph converts the weights 18 into first likelihood parameters so that they correspond to another

B E2017 / 5263 ends first segmentation points are represented in the form of vertices, the first character candidates being represented as edges and the first likelihood parameters of the first character candidates are represented in the form of corresponding edge values.

A graph typically starts with a first vertex which corresponds to an initial segmentation point which is the leftmost segmentation point of the image of the character string

11. The first vertex is connected to another vertex which corresponds to another segmentation point by an edge corresponding to the character identified between the initial point and the other segmentation points. The graph continues with a vertex for each segment point and an edge for each character between the segment points. The character likelihood parameters are represented by edge marks at the corresponding edges. The graph is by a vertex corresponding to a point on the right of the first e in the image of the character string 11. This point on the 5th of the image of the character string 11 is preferably not a point of segmentation.

Vane types of data structures that a graph can be used to order the outputs of the OCR without departing from the scope of the invention.

Fig. 2 illustrates an image of a character string 11. In this example 201, the first OCR 12 has been identified by a b starting from a segmentation point 202 with a likelihood parameter of 80% and by a t starting from a segmentation point 203 with a likelihood parameter of 95%.

Fig. 3 illustrates the first graph 301 generated in step 15 from the output 13 of the first OCR 12. The vertex 302 corresponds to the segmentation point 202. The edge 304 corresponds to the letter b and the associated parameter of 8Q% 305 is also Indus in the right caracre of the last carao

B E2017 / 5263

“I π s V first graph 301. The vertex 303 corresponds to the segmentation point 203. The edge 306 corresponds to the letter t and the associated parameter of 95% 307 is also included in the first graph 301.

Referring now to FIG. 1, the first segmentation point 14 and the image of the character string 11 are used as input for the combined segmentation and classification 100. In an embodiment of the invention, in which your positions are delivered by the first OCR 12 instead of the first segmentation points 14, the first segmentation points 14 are determined at another step of method 10, for example, during combined segmentation and classification 100. For example, if the first OCR 12 delivers the limitation boxes for the first characters, the segmentation points can be determined as the points of the leftmost black pixel in each bounding box.

The combined segmentation and olassificatlon 100 constitute a second OCR and will be described later with reference to FIG. 4. The combined segmentation and classification 100 preferably generates at least once for each character in the image of the character string 11 at least one second candidate

second segmentation point. The probability of the second character candidate is a second likelihood parameter.

In one embodiment of the invention, there may be one or more second candidate characters each having its corresponding position and preferably each having its probability determined by the combined segmentation and classification 100 for each character of the image of the character string 11.

B E2017 / 5263

The second character candidate, its position and its probability are, each time they are generated, used as input for an update 20 of the graph which updates the first graph 16. the end of the combined segmentation and classification 100, the global output of the updates 20 of the graph is a second graph 21 which combines the output of the first OCR 12 and the output of the combined segmentation and classification 100. If necessary, the update 20 of the graph converts the probabilities in second likelihood parameters having the same scale as the first likelihood parameters to make them comparable. Update 20 of the graph adds a new edge for each second ID. Update 20 of the graph adds a new vertex at the end of each new edge which does not end on a vertex already present in the graph.

The second graph 21 includes an edge for each of the first IDs and for each of the second IDs associated with their respective likelihood parameter, and a vertex for each of the first segmentation points and for each of the second segmentation points. An example of the second graph is illustrated in FIG. 7 and

the first OCR or from the

combined 100.

The second graph 21 is then used as input to an optional step of adding decision models 22, which generates a third decision graph 23. Step 22 can add the decision models to the graph by using a weighted finite state transducer (WFST) algorithm. This step 22 is in fact an application of a decision modeling method. The addition of decision models 22 modifies the parameters of likelihood of the edges, that is to say the characters or the group of edges, that is to say groups of characters in order to favor those which are a priori most

B E2017 / 5263 likely, eg due to context. The decision models modify the path weights. The decision models are preferably related to the contextual decision. Decision models greatly improve identification accuracy.

Decision models can involve bigrams, typographic metrics, lists of words such as dictionaries, n grams of characters, punctuation rules, and spacing rules.

A first type of decision model is a linguistic model. If the word "ornate" is present in the image of the character string to be identified, the first or second OCR can for example find the word "ornate" and the word "ornate" as ID with similar likelihood parameters of the makes the letters rn taken together look like the letter m. A linguistic model using a dictionary is able to detect that the word "ornate" does not exist, while the word "ornate" exists.

In one embodiment of the present invention, the linguistic model uses a model of n ~ grams. If the word "TRESMEUR" is present in the image of the character string 11, "i RE5MEUR" and the word ^f <TRESMbUR "as ID with similar parameters of likelihood since the letters" S "can resemble the letter "5" in printed text. A linguistic model using a bi-gram model (n-gram with n ~ 2) would prefer "TREASURE" if "ES" and "SM" had better probabilities of occurrence than "E5" and "SM s.

Another type of template used in an embodiment of the present invention is a typographic template. If the word "Loguivy" is present in the image of the character string 11, the first or second OCR can for example find the word "Loguivy" and the word "Loguivv" as ID with parameters of

B E2017 / 5263 similar likelihood in that the letters “y” may resemble the letter “v” in Printed text. A typographic model using font metrics would prefer "Loguivy" because the position of the background of the final character is more likely to correspond to the lower position of a "y" (in its model) than a "v".

In one embodiment of the present invention, the typographic model considers the position of the character in the image to verify whether the sizes and the positions are expected or not.

In a decision model involving punctuation and spacing rules, certain combinations of punctuation and / or spaces are disadvantaged.

In one embodiment of the invention, a decision model is added in step 22 which favors the IDs which have the first likelihood parameter and the second likelihood parameter meeting a predetermined criterion. In fact, it is expected that if the same ID is discovered by the first OCR 12 and the segmentation and classification combined 100, DD will be more credible. Therefore, if for an ID, both the first likelihood parameter and the second likelihood parameter are above a threshold, for example, $ 10%, the ID can be favored.

the invention, a decision model is added in step 22, which is equivalent to an OCR vote because it favors the first OCR in some cases and the second OCR in other cases. For example, if the first OCR is known to be extremely precise for a first type of characters, such as katakana characters, while the combined segmentation and classification 100 are known to be extremely precise for a second type of characters, such as kanji, a decision model can be added which favors your IDs corresponding to the first type of 30 characters (katakana) if these IDs have been determined by the first OCR and favors IDs corresponding to the second type of characters (kanji) if

B E2017 / 5263

these were determined by the combined segmentation and classification 100.

determination of the best path 24 determines, between your possible paths defined by your edges joined by vertices in the third weighting of the path to find the a functional diagram for combined 100 according to a mode

This functional scheme includes some segmentation girdles which

During this step 101, the character data 13 could

The combined segmentation and classification 100 begins to derate on your stacking segmentation points with a segmentation point index i which is set equal

B E2017 / 5263 to a first value, for example 1. A corresponding segmentation point 121 coming from stack 102 is considered. The first time a segmentation point 121 is considered in the combined segmentation and classification 100, the segmentation point 121 is preferably one of the first segmentation points 14 which has been determined by the first OCR. The segmentation point 121 is preferably removed from the stack 102 so that the stack 102 contains only the segmentation points which must still be considered. The segmentation point 121 is preferably placed in a list of segmentation points already processed.

Then, the combined segmentation and classification 100 determine in step 130 a list of widths 131. Preferably, this determination 130 does not use as input a list of widths determined for another segmentation point. In one embodiment of the invention, the list 131 of widths is generated in step 130 as described in US patent application n ^{<: 14/} 254,096 which is incorporated in the present application by reference, in which the first segmentation points 14 are used as input. The image of the character string 11 and / or the first character candidates 20 can also be used as inputs. In one embodiment of the invention, the determination 130 uses a database 132 storing reference character widths, storing an average width and possibly storing other character statistics. The determination 130 is preferably based on this average width provided by the database 132 and the size or sizes of the component or components connected around the segmentation point i, the size or sizes being provided by the image 11 of the chain of characters. The determination 130 is based even better on this average width provided by the database 132 and the size (s) of the component (s) connected according to the segmentation point i, the size (s) being provided by the image 11 of the chain of characters.

B E2017 / 5263

5 _:

U. ' V

Then, the combined segmentation and classification 100 begin to iterate over the widths of the width list 131, a width index j being set equal to a first value, for example 1. A width 124 from the width list 131 is considered. The combination of the segmentation point i and the width j provides a segmentation part 125 which is a part of the image of the character string 11. A classification of characters 126 is performed on the segmentation part 125. The classification of characters 126 is preferably a classification of unique characters. Character classification 126 may include a character classification designed for a first type of characters, for example Asian characters, and a character classification designed for a second type of characters, for example Latin characters. Such a classification of characters is described in US Patent Application No. 14/299 205 ^s which is incorporated herein by reference.

Character classification 126 provides hypothesis ID 127a with a likelihood of error P _err 127b. The hypothesis ID 127a is a hypothesis on the character possibly present on the segmentation part 125. The likelihood of the error P _aï 127b is a likelihood parameter, The likelihood of the error P _efr 127b is a percentage which decreases with the probability that classification 126 has

B E2017 / 5263 correctly identified the content of the segmentation part 125. The probability that classification 126 has correctly identified the content of the segmentation part 125 is equal to 1-P _efr . P _err and the probability equal to 1 ~ P _{e (T} can both be called "second likelihood parameter", because they both provide an indication of the probability that the identification assumption 127a is correct for character classification 126.

The combined segmentation-classification 100 then checks, in step 128, whether the likelihood of the error P _err meets a first predetermined criterion which is that P _{f ::} - is less than a threshold to have a high likelihood Th _erf error. Th _erf is preferably chosen to be close to 100%, for example equal to 99.9%. Thus, the first predetermined criterion rejects the hypotheses which are really implausible. If the first predetermined criterion is not satisfied, the index j is incremented and a next width from the list of widths 131 is considered.

If the first predetermined criterion is satisfied, it indicates that the hypothesis ID could have a chance of being correct. If the first predetermined criterion is satisfied, an update 110 of the stack is carried out. This update 110 of the stack is described below with reference to FIG. 5.

If the first predetermined criterion is satisfied, an update 150 of the average list is carried out and the updated average width replaces the average width in the database 132. In one embodiment of the invention, the updating day 150 of the average width is carried out as described in the patent application US rr 14/254,096 which is incorporated into the present application by reference.

In one embodiment of the invention, the updating 150 of the average width is followed by the determination 130 of the list of widths which generates the list of widths 131 to be used for the same

B E2017 / 5263 segmentation point I in the next iteration over widths by combined segmentation and classification 100.

If the first predetermined criterion is satisfied, the second character candidate, its position and its probability 17 ’are added to the graph during the corresponding update of the graph (Fig. 1). The second character candidate, its position and its probability 17 cannot be updated as such by the combined segmentation and classification 100, but just added to the graph. The second character candidate is equal to hypothesis ID 127a. In other words, the assumption

D is chosen as the second character candidate. The position of the second character candidate is provided by the segmentation part where the second character candidate has been identified, that is, by the segmentation point i and the width j which have been considered. The probability of the second character candidate is equal to i “P _in -, with the value of P _sn · of 127b.

predetermined 128.

Character classification 126 can provide a plurality of hypotheses ID 127a each with their respective likelihood of error P _s „127b, all corresponding to the same segmentation part 125. In such a case, checks 128 and 140 are performed on the ID hypothesis with the _lowest P _8rf , that is to say the ID hypothesis which is most expected to be correct. The "second character candidate with position and probability" 17 of the figures then includes the plurality of hypotheses ID, each with their respective position and probability. They are all included in the graph

B E2017 / 5263 directly before moving to a new segmentation segment 125.

In the combined segmentation and classification 100, any segmentation part 125 is considered only once.

The first predetermined criterion provides an adjustable parameter for determining the probability threshold from which the hypotheses ID 127a are added in the graph. A very high Th _err provides a second very large graph 21 which gives high precision, but could slow down the OCR process 10. A lower Th _sn · provides a second smaller graph 21, which could give less precision, but speeds up the process OCR 10,

In one embodiment of the invention, different <A .‘- S.

sr

B E2017 / 5263

If the stack 102 still contains at least one segmentation point, the index I of segmentation points is increments and a next segmentation point 121 is considered.

If the stack 102 does not contain a segmentation point, the combined segmentation-classification 100 preferably performs an end of stacking step 142. The end of stacking step 142 includes verification that the end of the image of the character string 11 has been reached, for example by checking that HD of the rightmost character in the second graph 21 includes the 10 rightmost black pixel of the image of the character string 11.

If the end of the image of the character string 11 has not been reached, an empty character with a probability equal to 0 is inserted into the graph starting from the vertex corresponding to the rightmost segmentation point which has been considered, a segmentation point corresponding to the end of this empty character is placed in the stack 102 and the combined segmentation and classification 100 resume with this segmentation point.

If the end of the image of the character string 11 has been reached, an empty character with a probability equal to G can be added 20 to the second graph 21 between each vertex which is not connected to a next vertex (each point segmentation for which no second candidate character 17 has been selected) and the next vertex.

Fig. 5 shows the update 110 of the stack according to an embodiment of the invention. When it is carried out, the updating of the stack 110 takes as inputs the segmentation part 125 considered at this stage in the combined segmentation and classification 100 and the image of the character string 11. The updating 110 of the stack comprises a determination 111 of another segmentation point. This other segmentation point 112 is supposed to correspond to the start of a character to the right of the character which has just been identified in the segmentation part 125 by the classification of

B E2017 / 5263 characters 126. The other segmentation point 112 is preferably determined as the leftmost black pixel to the right of the segmentation part 125. The space between characters can have a predetermined value or be calculated at course of combined segmentation and classification 100.

Alternatively, the other segmentation paint 112 can be calculated, for example, during character classification 126 and provides the update 110 of the stack

In one embodiment of the invention, the determination 111 of another segmentation point uses information coming from the database 132.

The update 110 of the stack then checks in step 113 whether the other segmentation point 112 is currently present in the stack 102 or in the list of segmentation points already processed. Otherwise, the other segmentation point 112 is placed in the stack 102.

Since a character is supposed to start a bit after the previous character, this update 110 of the stack generates segmentation points which are probable starting points for a next character. The verification in step 113 prevents a segmentation point from appearing twice in the stack 102.

Fig. 6 illustrates a very simple example of the combined segmentation and classification 100 according to the invention. Fig. 7 illustrates the second graph 21 at the end of the combined segmentation and classification 100 illustrated in FIG. 6.

Fig. 6a shows an image of a character string 11 with the word batch and in which the letters I and o are so close that it is difficult to determine whether it is I and o or b. The first OCR 12 provided the IDs b and t with their position and their probability (P ~ 80% for b and P ~ S5% for t). From their position, the segmentation points 601 and 602 can be determined (Fig. 6a). The graph which is the

B E2017 / 5263 first graph at this stage includes the vertices 711, 712 and 713 and the edges 701 and 702 (Fig. 7).

The stack contains the segmentation points 601 and

602. For the segmentation point 601. the determination 130 of the list of widths determines possible widths including a width

603. The width 603 is placed in the list 131 of widths.

Fig. 6b shows the width 603 which is considered the first in conjunction with the segmentation point 601 in classification 126. The classification 126 finds that the segmentation portion provided by the segmentation point 601 and the width 603 represented a b with a probability of 70%. Like P _S! ·, (30%) is less than Th _gfr (99.9%), a corresponding edge 793 is added to the graph 21 during an update 20 of the graph. The update 110 of the stack determines the other segmentation point (step 111). We discover in step 102 that this other segmentation point which corresponds to point 602 is already in the stack 102. Consequently, this other segmentation point is not added to the stack 102. The update 15G of the average width is performed, which updates the database 132. The determination 130

not

less than Ti _0rf (20%), another width 604 of the list 131 is tested.

Fig. 6c shows the width 604 which is considered together with the segment point 601 in the classification 126. The classification 125 finds that the segmentation part provided by the segment point 601 and the width 604 represents an I with a probability of 85%. As P _grr (15%) is Less than Th _sn · (99.9%), a corresponding edge 704 is added to graph 21 during an update of the graph and a vertex 714, which corresponds to a point segmentation 605 at the end of I, is added to graph 21.

B E2017 / 5263

Another segmentation point 605 is then determined in step 111 of updating 110 of the stack. Since point 605 is not in the stack (step 113), it is added to the stack. update 150 of the average width and the determination 130 of the list of widths are also carried out.

As P _efr (15%) is less than ΤΙ _{3! Τ} (20%), there is a check (step 141) to check if there is any segmentation point left in stack 102. The points of segmentation 602 and 605 still being in the stack 102, the next segmentation point 602 is considered.

Fig. 6d shows the segmentation point 602 considered together with the width 606 in the classification 126, The classification 126 finds that the segmentation part provided by the segmentation point 602 and the width 606 represents a t with a probability of 99.5%. As P _err (0.5%) is less than Th _efr (99.9%), a corresponding edge 705 is added to the graph 21 during an update 20 of the graph. The updating 110 of the stack is not carried out by updating the checking widths (step 141) to check whether there is any segmentation point left in the stack 102. The segmentation 605 still being in the stack 162, this segmentation point 605 is considered.

Fig. 6th shows the segmentation point 605 considered together with a width 607 in the classification 126. The classification 126 finds that the segmentation part provided by the segmentation point 605 and the width 607 represents an o with a probability of 85%. As P _err (15%) is less than Th _erf (99.9%), a corresponding edge 706 is added to graph 21 during the updating of characters has been reached. The are done to me

B E2017 / 5263 day 20 of the graph. Update 110 of the stack determines the other segmentation point (step 111). It can be seen in step 112 that this other segmentation point which corresponds to point 602 is found in the list of segmentation points which have already been processed. Consequently, this other segmentation point is not added to the stack 102. The updating 150 of the average width and the determination 130 of the list of widths are also carried out.

As P _eiT (15%) is less than ΤΙ _{Θ! Τ} (20%), there is a check (step 141) to check if there is any segmentation point left in stack 102. As stack 102 is empty, the combined segmentation and classification 100 move to the steps 142 of end of stack.

The end of stacking steps verify that the rightmost pixel of the image of the character string 11 is in fact a part of the t. All the vertices of graph 21 are connected to a next vertex

Indicated in the graph.

A model of

example at

lot "by the character string" bt ".

The present invention comprises a combination of a first OCR 12 and a second OCR, which is the combined segmentation and classification 100, at various stages: at the entry of the first segmentation points 14 of the stack 102 of the second OCR 100, upon updating 20 of the graph generated by the first OCR 12 which generates the second graph 21 and in your decision models 22.

The invention does not require an overview of how the first OCR 12 is performed. It only requires obtaining from the first OCR 12 the sequence of ID of

B E2017 / 5263 characters, their position in the image of the character string and, if possible, their likelihood parameters, which are the usual outputs of any OCR. As a result, the invention is able to combine almost any commercial OCR with the second OCR.

In addition, the invention takes the better of the two OCRs than the output of the first OCR 12 and better than the output of the second OCR 100.

segmentation ns b ·:

If points ":

• mirrored points course

According to one embodiment of the invention, at least one

B E2017 / 5263

In other words, the invention relates to a method and a program for combining a first optical character recognition (OCR) 12 and a second OCR 100. The first OCR 12 operates first on an image of a character string 11. Its output 5 13 (first identified characters, positions of the characters and parameters of likelihood of the characters) is used to generate a first graph 16. Segment points related to the positions of the first identified characters 14 are used as input by the second OCR 100 by performing a combined segmentation and classification 10 on the character string image 11. The output 17 (identified second characters, positions of the characters and character likelihood parameters) of the second OCR 100 is used to update the first graph 16, generate a second graph 21 which combines the output 13 of the first OCR 12 with the output 17 d e the second OCR 100. Decision models are then used to modify the weightings 22 of the paths in the second graph 21 in order to generate a third graph 23. A better path 24 is determined on the third graph 23 to obtain the identification 25 of the characters present in the image of character string 11.

B E2017 / 5263

Claims (4)

Claims 1. Method for selecting character candidates in a method for identifying characters in a digital image, the method comprising the steps consisting in: a) applying a first character identification process to determine first character candidates and a list of segmentation points for the first character candidates, b) generate a list of character widths corresponding to a segmentation point from the list of segmentation points, c) determine a part of the digital image corresponding to the segmentation point and to a character width from the list of character widths, d) apply a character classification process on the part of the digital image to obtain an hypothesis ID of a character possibly present in the part of the digital image and a likelihood parameter which relates to the likelihood that the hypothesis ID be correct and e) select the hypothesis ID as the second character candidate in the digital image if the likelihood parameter meets a first predetermined criterion. 2. Method according to claim 1, the method comprising the list of segmentation points with another segmentation point determined on the basis of the part of the digital image corresponding to the second selected candidate character. 3. Method according to claim 2, in which step f) comprises: checking that the other segmentation point is had B E2017 / 5263 B E2017 / 5263 comprises the first likelihood parameters of the first character candidates and the method further comprises the steps of: change the scale of the first likelihood parameters of the first character candidates and / or the likelihood parameters of the second character candidates to make them comparable to each other and update the data structure with the likelihood parameters of the second character candidates. 10. The method of claims, wherein the B E2017 / 5263 your character candidates are provided by the first character identification process and makes you favor character candidates of the second type if the character candidates are provided by the classification process. 15. The method of claim 1, wherein the first character identification method comprises a decision modeling method. 16. The method of claim 1, wherein the first character identification process provides positions of first character candidates and further comprises a step of determining the list of segmentation points of the first character candidates from the positions of the first character candidates. 17. Method of identifying characters in a digital image, proceed comprising your steps consisting in: a) applying a first character identification method to determine first character candidates and first segmentation points corresponding to the first character candidates, b) generating a first graph on the basis of the first character candidates and the first segmentation points, in which the first segmentation points are represented by methods of identifying characters on at least part of the digital image to generate at minus a second character candidate, d) update the graph by adding at least one second character candidate to the first graph and e) selecting in the updated graph candidates characters as characters of the digital image, 18. The method of claim 17, wherein the second character identification method generates at least one B E2017 / 5263 second segmentation point and in which ie at least one second segmentation point is added to the graph in step d). 19. The method of claim 18, wherein the first character identification method provides first likelihood parameters providing a likelihood that the first character candidates are correct, ie the second character identification method provides second likelihood parameters providing a likelihood that you at least one second character candidate is correct, and the method further comprises the steps of: change the scale of the first likelihood parameters and / or the second likelihood parameters to make them comparable to each other, add your first likelihood parameters in the first graph and add the second likelihood parameters in the set graph up to date. 20. The method of claim 19, further comprising the step of applying a decision modeling method on te your second likelihood parameters meeting a third predetermined criterion. 22. The method of claim 21, wherein the first character identification method provides character candidates for characters of a first type, wherein the second character identification method provides character candidates for characters of a second type and in which the decision modeling method comprises a rule which favors your candidate characters corresponding to the first type of characters if these candidates B E2017 / 5263 characters were determined by the first character identification method and favors the character candidates corresponding to the second type of characters if these character candidates were determined by the second character identification method. 23. The method of claim 17, wherein the second character identification method uses as input, in step c), the first segmentation points from the first character identification method. 24. Non-transient medium readable on a computer storing a B E2017 / 5263 B E2017 / 5263 B E2017 / 5263 ist of widths J I i ί i I ΐ I whatever \ '' ^. 'VA'.VAWAWttW. aonnees F ^ segmentation edge ™ segmentation i + width j Second character candidate with his josition and his probability B E2017 / 5263 B E2017 / 5263 BE2017 / 5263 Method dOdentlflcatlon d ^j a character dang a digital image The invention relates to a method for combining a first optical character recognition (OCR) (12) and a second 5 OCR (100). The first OCR (12) operates first on a character string image (11). Its output (13) (first identified characters, positions of the characters and likelihood parameters of the characters) is used to generate a first graph (16). Segmentation points related to the positrons of the first 10 characters identified (14) are used as input by the second OCR (100) by performing combined segmentation and classification on the character string image (11). The output (17) (identified second characters, character positions and character likelihood parameters) of the second OCR (100) is used to update (20) the first graph (16) in order to generate a second graph ( 21) which combines the output (13) of the first OCR (12) with the output (17) of the second OCR (100). Decision models are then used to mod-Her (22) the path weights in the second graph (21) in order to generate a third graph (23). A better path is determined (24) 20 on the third graph (23) in order to obtain the identification (25) of the characters present in the character string image (11).