FR3139644A1 - METHOD FOR THE AUTOMATED PROCESSING OF A DOCUMENT FOR ITS USE BY PEOPLE WITH DISABILITIES - Google Patents

Abstract

METHOD FOR THE AUTOMATED PROCESSING OF A DOCUMENT FOR ITS USE BY PEOPLE WITH DISABILITIES Process for automated processing of a structured digital document (F1) to generate an augmented digital object (OB1): Acquisition of a first digital file (F1) in portable document format; Segmentation of said first digital file (F1) into a plurality of digital objects: Implementation of a first algorithm (ALG1) making it possible to generate digital objects defining paragraphs; Implementation of 'a second algorithm (ALG2) making it possible to generate digital objects relating to character strings having metadata; Implementation of a third algorithm (ALG3) making it possible to generate digital objects defining character string organization structures ;Labeling (LAB1) of the first digital objects (OB1);Scheduling of digital objects (OB1, OB2, OB3) from a third software component (MOD3);Generation of a second digital file (F2) comprising all ordered digital objects. Figure for abstract: Fig.1

Description

METHOD FOR THE AUTOMATED PROCESSING OF A DOCUMENT FOR ITS USE BY PEOPLE WITH DISABILITIES Field of the invention

The field of the invention is that of methods for automatically processing documents in portable document format (PDF) so as to make them accessible to disabled people. In particular, the field of the invention relates to methods integrating algorithms aimed at automatically labeling documents to make these documents accessible and interpretable by third-party applications.

State of the art

There are different techniques for scanning and labeling a document in portable document format (PDF) automatically. However, current annotation and labeling techniques do not allow full use of the document for individuals with disabilities, particularly the blind or visually impaired. Indeed, when using such a document, in order to transcribe the contextualized information such as that contained in tables or such as certain notes or even information listed in lists, it is necessary that an interpreter, such as a voice interpreter, can present the information in context. However, current methods do not allow the exploitation of each digital object to contextualize the automated reading of a document.

There are solutions based on artificial intelligence, but most often with the objective of discriminating or filtering relevant information or even restoring a layout without creating digital objects that can be effectively exploited by an interpreter.

There is therefore a need to define a solution making it possible to generate a document that can be fully activated by a third-party application to exploit each digital object of a document according to their structure and facilitate access by a visually impaired or blind individual.

• Acquisition d’un premier fichier numérique au format portable document format ;
• Segmentation dudit premier fichier numérique en une pluralité d’objets numériques à partir d’un premier composant logiciel comportant :
• Mise en œuvre d’un premier algorithme comportant :
  • Extraction d’un premier ensemble d’objets numériques comportant chacun une séquence de symboles discrets dans une langue naturelle définissant un paragraphe ; ladite extraction comportant l’application d’un premier ensemble de règles et d’une première base de connaissance pour générer au moins une première caractéristique symbolique dudit paragraphe ;
  • Génération d’une pluralité de couples de positions 2D dans le premier fichier numérique, chaque couple de positions définissant une première zone rectangulaire comprenant l’affichage d’un premier objet et association desdits couples de positions auxdits premiers objets ;
• Mise en œuvre d’un second algorithme comportant :
  • Extraction d’un second ensemble d’objets numériques comportant chacun une séquence de symboles discrets dans une langue naturelle et au moins une seconde caractéristique symbolique représentative d’une métadonnée associée à l’objet numérique ; ladite extraction comportant l’application d’un second ensemble de règles et d’une seconde base de connaissance ;
  • Génération d’une pluralité de couples de positions 2D dans le premier fichier numérique, chaque couple de positions définissant une seconde zone rectangulaire comprenant l’affichage d’un second objet et association desdits couples de positions auxdits seconds objets
• Mise en œuvre d’un troisième algorithme comportant :
  • Extraction d’un troisième ensemble d’au moins un troisième objet numérique comportant chacun une séquence de symboles discrets dans une langue naturelle et au moins une troisième caractéristique symbolique représentative de l’agencement d’une pluralité de sous-objets au sein dudit au moins un objet numérique, ladite extraction comportant l’application d’un troisième ensemble de règles et d’une troisième base de connaissance ;
  • Génération d’une pluralité de couples de positions 2D dans le premier fichier numérique, chaque couple de positions définissant une troisième zone rectangulaire comprenant l’affichage d’un troisième objet et association desdits couples de positions auxdits troisièmes objets ;
• Labellisation des premiers objets numériques au moyen d’un second composant logiciel comportant la mise en œuvre d’un moteur d’inférence ;
• Ordonnancement des objets numériques à partir d’un troisième composant logiciel prenant en compte en entrée :
  • Les positions des premiers objets, des seconds objets et des troisièmes objets ;
  • Les labels de chaque premier objet ;
  • Les premières caractéristiques symboliques, les secondes caractéristiques symboliques et les troisièmes caractéristiques symboliques ;
• Génération d’un second fichier numérique comportant l’ensemble des objets numériques ordonnés.

According to a first aspect, the invention relates to a method for automated processing of a digital document to generate an augmented digital object:

• Acquisition of a first digital file in portable document format;
• Segmentation of said first digital file into a plurality of digital objects from a first software component comprising:
• Implementation of a first algorithm comprising:
  • Extraction of a first set of digital objects each comprising a sequence of discrete symbols in a natural language defining a paragraph; said extraction comprising the application of a first set of rules and a first knowledge base to generate at least one first symbolic characteristic of said paragraph;
  • Generation of a plurality of pairs of 2D positions in the first digital file, each pair of positions defining a first rectangular area comprising the display of a first object and association of said pairs of positions with said first objects;
• Implementation of a second algorithm comprising:
  • Extraction of a second set of digital objects each comprising a sequence of discrete symbols in a natural language and at least one second symbolic characteristic representative of metadata associated with the digital object; said extraction comprising the application of a second set of rules and a second knowledge base;
  • Generation of a plurality of pairs of 2D positions in the first digital file, each pair of positions defining a second rectangular area comprising the display of a second object and association of said pairs of positions with said second objects
• Implementation of a third algorithm comprising:
  • Extraction of a third set of at least one third digital object each comprising a sequence of discrete symbols in a natural language and at least one third symbolic characteristic representative of the arrangement of a plurality of sub-objects within said at least a digital object, said extraction comprising the application of a third set of rules and a third knowledge base;
  • Generation of a plurality of pairs of 2D positions in the first digital file, each pair of positions defining a third rectangular area comprising the display of a third object and association of said pairs of positions with said third objects;
• Labeling of the first digital objects by means of a second software component comprising the implementation of an inference engine;
• Scheduling of digital objects from a third software component taking into account as input:
  • The positions of the first objects, second objects and third objects;
  • The labels of each first object;
  • The first symbolic characteristics, the second symbolic characteristics and the third symbolic characteristics;
• Generation of a second digital file containing all of the ordered digital objects.

One advantage is that it makes it possible to generate a document that can be activated by a reader such as a screen reader in order to use all the elements of a document. An advantage is to allow intelligible restitution of each element of a document, particularly for a blind person.

• Identification d’une ligne du premier fichier numérique comportant une première chaine de caractères ;
• Identification de l’orientation de ladite ligne selon au moins une première règle ;
• Identification d’une information symbolique caractéristique de la chaine de caractère selon une seconde règle et d’une information caractérisant l’espace entre chaque chaine de caractères de ladite ligne selon une troisième règle ;
• Catégorisation de ladite ligne,
• Réitération des opérations d’identification de chaque ligne et de leur catégorisation ;
• Génération de ladite première caractéristique symbolique d’un paragraphe lorsque la catégorisation d’un ensemble de lignes est identique.

According to one embodiment, the first algorithm comprises the steps of:

• Identification of a line of the first digital file comprising a first character string;
• Identification of the orientation of said line according to at least a first rule;
• Identification of symbolic information characteristic of the character string according to a second rule and of information characterizing the space between each character string of said line according to a third rule;
• Categorization of said line,
• Reiteration of the operations to identify each line and their categorization;
• Generation of said first symbolic characteristic of a paragraph when the categorization of a set of lines is identical.

An advantage is to associate metadata with each line of characters in order to reconstruct paragraphs. This characteristic allows better interpretation of the content when a screen reader automatically processes the content of a document to generate a reading for a visually impaired person, for example.

• Paragraphe de pied de page, correspondant à un paragraphe agencé dans une zone d’une page située en bas de page et comportant une chaine de caractères répétée au sein d’une pluralité de pages ou une séquence de nombres incrémentée à chaque page ;
• Paragraphe de haut de page, correspondant à un paragraphe agencé dans une zone d’une page située en haut de page et comportant une chaine de caractères répétée au sein d’une pluralité de pages ou une séquence de nombres incrémentée à chaque page ;
• Paragraphe standard, correspondant à un paragraphe défini par un style, une police et une taille de caractères donnés et étant le plus représentatif sur l’ensemble de pages du document numérique ;
• Paragraphe distingué, correspondant à un paragraphe défini par un style, une police et une taille de caractères donnés différents du paragraphe standard ;
• Paragraphe titre, correspondant à un paragraphe portant le nom d’une section ou d’une sous-section et suivi d’au moins un paragraphe structuré différent d’un paragraphe de pied de page ou d’une note.

According to one embodiment, the labeling step makes it possible to associate a label with a paragraph from one of the following labels:

• Footer paragraph, corresponding to a paragraph arranged in an area of a page located at the bottom of the page and comprising a string of characters repeated within a plurality of pages or a sequence of numbers incremented on each page;
• Top-of-page paragraph, corresponding to a paragraph arranged in an area of a page located at the top of the page and comprising a string of characters repeated within a plurality of pages or a sequence of numbers incremented on each page;
• Standard paragraph, corresponding to a paragraph defined by a given style, font and character size and being the most representative on all pages of the digital document;
• Distinguished paragraph, corresponding to a paragraph defined by a given style, font and character size different from the standard paragraph;
• Title paragraph, corresponding to a paragraph bearing the name of a section or subsection and followed by at least one structured paragraph other than a footer paragraph or a note.

An advantage is to differentiate the different types of paragraphs to allow a contextualized reading of the information by a screen reader in order to make the reading of the different paragraphs of a document intelligible.

According to one embodiment, the second software component is implemented from an inference engine executed from a set of rules and a fact base taking into account the first symbolic characteristics.

An advantage is to allow the processing of all possible scenarios based on rules dealing with different possible scenarios.

• un réseau de type « Recurrent neural network », noté RNN ou ;
• un réseau de type « Long short-term memory », noté LSTM ou ;
• un transformeur de type BERT ou GPT, tel que GPT-3.

According to one embodiment, the second software component is implemented from an inference engine executed from a learning function whose input vectors comprise at least the first symbolic characteristics, said learning function implemented work being:

• a “Recurrent neural network” type network, denoted RNN or;
• a “Long short-term memory” type network, denoted LSTM or;
• a BERT or GPT type transformer, such as GPT-3.

According to this embodiment, the implementation of a learning function makes it possible to automatically determine new rules used by the inference engine. One advantage is to enrich the model on which the inference engine infers.

• détection d’un premier symbole typographique prédéfini ;
• détection d’un second symbole typographique prédéfini et d’un enchainement d’un symbole typographique donné ayant une graphie prédéfinie,
• génération d’une catégorie définissant une métadonnée lorsqu’au moins une détection est réalisée ;
• génération d’une seconde caractéristique symbolique comportant ladite chaine de caractères comportant le symbole détecté et la catégorie générée.

According to one embodiment, the second algorithm comprises:

• detection of a first predefined typographic symbol;
• detection of a second predefined typographic symbol and a sequence of a given typographic symbol having a predefined spelling,
• generation of a category defining metadata when at least one detection is carried out;
• generation of a second symbolic characteristic comprising said character string comprising the detected symbol and the generated category.

An advantage is to allow detection of notes, emails or other elements associated with particular typographic elements. An advantage is to allow the automatic reading of information intelligible by a screen reader to be contextualized.

• Identification d’une pluralité de symboles définissant une répétition d’agencement d’un ensemble de données du premier fichier afin de définir une troisième caractéristique symbolique ;
• Catégorisation d’un troisième objet numérique à partir de la génération de la troisième caractéristique symbolique.

According to one embodiment, the third algorithm comprises:

• Identifying a plurality of symbols defining an arrangement repeat of a data set of the first file to define a third symbolic characteristic;
• Categorization of a third digital object from the generation of the third symbolic characteristic.

• Sélection de chaque première zone géographique associée à un premier objet numérique ;
• Contrôle des labels de chaque premier objet définissant chacun un paragraphe avec un label d’un premier objet à une position adjacente précédente,
• Génération d’un premier objet enrichi lorsque les labels de l’ensemble des premiers objets adjacents répondent à au moins une règle dudit troisième algorithme.

According to one embodiment, the extraction of the third objects comprises:

• Selection of each first geographical area associated with a first digital object;
• Control of the labels of each first object each defining a paragraph with a label of a first object at a previous adjacent position,
• Generation of a first enriched object when the labels of all of the first adjacent objects meet at least one rule of said third algorithm.

One advantage is that it makes it possible to organize the automatic reading of intelligible information within a table, for example by a screen reader.

• Sélection d’une troisième zone de la page du premier fichier ;
• Sélection de chaque première zone géographique associée à un premier objet numérique située dans la troisième zone sélectionnée ;
• Contrôle des labels de chaque premier objet définissant chacun un paragraphe,
• Génération d’un troisième objet lorsque les labels de l’ensemble des premiers objets répondent à au moins une règle dudit troisième algorithme.

According to one embodiment, the extraction of the third objects comprises:

• Selection of a third area of the page of the first file;
• Selection of each first geographical area associated with a first digital object located in the third selected area;
• Control of the labels of each first object each defining a paragraph,
• Generation of a third object when the labels of all of the first objects meet at least one rule of said third algorithm.

According to one embodiment, the first enriched objects, the second objects, and the third objects are ordered according to a display sequence on a page, said first objects being ordered within an area associated with a first enriched object.

An advantage is to order and sequence the reading of information on a page according to their arrangement of elements within this page. One advantage is to generate a sequence that maintains the best intelligibility during automatic reading of a document containing many elements of different natures such as titles, tables, notes, etc.

According to one embodiment, the second digital file produced as an output is made activatable by object by means of an application aimed at exploiting the content of said second digital file from an interface of a user terminal intended for the visually impaired or blind, for example comprising a screen reader.

According to another aspect, the invention relates to a system comprising a user's electronic terminal comprising at least one calculator, a memory, a display and a communication interface for transmitting messages on a data network to at least one first server comprising hardware resources for executing the first algorithm, the second algorithm, the third algorithm and the second software component in order to execute the steps of the method of the invention, said second digital file being transmitted via the data network to said electronic terminal of the user to display the second digital file and to access content that can be activated via the interface of said electronic terminal.

According to one embodiment, the system comprises a screen reader comprising a voice synthesizer for generating an audio stream of information, said information being extracted from the digital objects, said information further comprising information generated from the metadata of the second file .

Brief description of the figures

Other characteristics and advantages of the invention will emerge on reading the detailed description which follows, with reference to the appended figures, which illustrate:

: a diagram according to one embodiment of the main stages of the invention;

: an example of representation of a document comprising different blocks of information comprising different structures.

: an example of a system of the invention making it possible to execute the method of the invention.

There represents an example of carrying out a process of the invention and in particular an example of implementing the steps of said process. The method of the invention makes it possible to generate a document in portable format, activatable document format denoted PDF _N. By “activatable” we mean the characteristic according to which the digital file obtained comprises different digital elements, called digital objects, which can be processed by a computer program in a differentiated manner. For example, a reader such as a voice synthesizer or a braille sequence generator can interpret the document in parts or contextualize the reading of certain objects such as tables or a table of contents. The usable elements can for example be tables, lists, notes, text inserts, titles, images or any other element that can be interpreted individually in the document.

The invention therefore makes it possible to take into account an input PDF file ₁ whose different elements cannot be used individually into a document presenting elements which can be used individually. The input document is an unstructured PDF document which a priori is not annotated for use by a third-party application.

The invention relates more particularly to the acquisition of a file in PDF format whose symbols are identified and usable due to an ocerization for example or a conversion of a previous document.

However, the method of the invention may include a preliminary step aimed at converting a file into a "pdf" format file whose symbols such as alphabets make it possible to identify words.

There illustrates a first software component MOD ₁ which makes it possible to take as input a document denoted PDF ₁ and to process in particular the different sequences of discrete symbols in a natural language and graphic objects such as lines, blocks or images integrated into the PDF document ₁ in order to to generate structured paragraphs and usable digital objects.

The input to the system can be obtained by means of reception of the file on a communication interface, or by recording a file in a memory or even following processing by software of a file allowing exploitation automatic display of symbols and images contained in each page of the file.

It is recalled that a PDF input document ₁ in the context of the invention is a “non-activatable” document and therefore not accessible, for example, to visually impaired or blind people. The non-accessible document is characterized in that it does not include a tag allowing metadata to be associated with the different elements of a page allowing these elements to be used by a software interpreter. In other words, a non-accessible PDF document is a document comprising images and sequences of sequences of discrete symbols in a natural language arranged in different arrangements which are digitally inseparable from each other so that the different elements of the page cannot be treated independently.

In the remainder of the description, we call “an algorithm”: a computer-implemented software function configured to produce a digital output from a digital input. Functions comprise a set of instructions interpretable by a software interpreter to perform operations such as mathematical operations, data conversions or comparisons of extracted data with threshold values. At a minimum, a software function is executed within a hardware environment having at least one computer and one memory.

A first software component MOD ₁ is implemented by the method of the invention with the objective of extracting each element of the page in the form of digital objects, denoted OB ₁ , OB ₂ and OB ₃ . An advantage of this MOD ₁ software component is to exploit specific symbolic characteristics of said objects to extract said objects and to make it possible to generate digital objects having characteristics allowing activations of said objects independently.

According to one embodiment, the first software component comprises a so-called SDK component. According to an example, an SDK component, designating in the English literature “Software Development Kit” and denoted SDK ₁ on the is implemented to exploit the data from the PDF document ₁ . The SDK component ₁ includes a set of functions allowing you to interpret elements of a non-activatable .pdf format file so as to extract words, images and graphic elements contained in the pdf format file. Implementing an SDK ₁ configuration therefore makes it possible to extract sets of words and different graphic blocks from the file.

1st algorithm of component MOD1

A first algorithm ALGO ₁ of the first component MOD ₁ makes it possible to model the different sets of words in the form of paragraphs. To this end, the first ALGO ₁ algorithm extracts a set of characteristics such as the orientation of the sequence of discrete symbols on the page, the spacing between the words of said sequence, the font sizes of the symbols, and possibly the style of the symbols. symbols and the inclination of the cursive of the symbols and the thickness of the symbols. All of these characteristics define a first symbolic characteristic, denoted K ₁ . This ALGO ₁ algorithm can be configured to exploit sequences of symbols according to orientation lines formed by the sequence within the page. An advantage of the invention is to make it possible to reconstruct certain structured paragraphs comprising several lines from the positions of said symbol lines and the graphic properties of said symbols.

A “structured paragraph” is a paragraph comprising a plurality of lines and having for each line similar or coherent properties of positions and shapes. Shape properties can relate to font, character size, character slant, indents at line start positions, leading spaces, etc. These properties can in particular be encoded in a symbolic characteristic hereinafter designated K ₁ .

The extracted sets of words can come from a structured paragraph of several lines, titles, footers, headers, tables, lists, captions, notes, quotes, tables of contents, etc. An advantage of this first ALGO ₁ algorithm is to extract a set of words arranged according to a linear portion, that is to say most often a line comprising a set of words. These word lines are associated with a position pair POS ₁ and a symbolic characteristic K ₁ . The positions can correspond to those of the start of the line and the end of the line. The analysis of inter-symbol spaces can notably lead to delimiting the lines.

We note more generally in the rest of the description a "paragraph" or a "structured paragraph" a set of lines whose symbolic characteristics K ₁ are substantially identical to each other whose pairs of positions have a coherence of juxtaposition or proximity between them .

In order to extract the different characteristic values of the first symbolic characteristic K ₁ , the method of the invention implements the execution of a first set of predefined rules R ₁ and a first knowledge base BC ₁ . According to one example, the R ₁ rules can include comparisons of spaces between each word, font sizes or even operations aimed at detecting a style or an inclination of incursive. For this purpose, the BC ₁ knowledge base may include a library of styles or fonts.

• une première position sur la page dont :
  • une première coordonnée X1 correspond à la coordonnée de la position du premier symbole discret de la séquence de symboles discrets dans le sens de la ligne de la séquence et
  • une seconde coordonnée Y1 correspond à la partie supérieure du premier symbole de la séquence de symboles considérée dans la ligne formée par la séquence ; et
• une seconde position sur la page dont :
  • une première coordonnée X2 correspond à la coordonnée de la position du dernier symbole discret de la séquence de symboles discrets dans le sens de la ligne de la séquence et ;
  • une seconde coordonnée Y2 correspond à la partie inférieure du dernier symbole de la séquence de symboles considérée dans la ligne formée par la séquence.

Furthermore, the first ALGO ₁ algorithm executes a function making it possible to generate at least:

• a first position on the page including:
  • a first coordinate X1 corresponds to the coordinate of the position of the first discrete symbol of the sequence of discrete symbols in the direction of the line of the sequence and
  • a second coordinate Y1 corresponds to the upper part of the first symbol of the sequence of symbols considered in the line formed by the sequence; And
• a second position on the page including:
  • a first coordinate X2 corresponds to the coordinate of the position of the last discrete symbol of the sequence of discrete symbols in the direction of the line of the sequence and;
  • a second coordinate Y2 corresponds to the lower part of the last symbol of the sequence of symbols considered in the line formed by the sequence.

A pair of positions, denoted POS ₁ , is formed to define a first zone in which the sequence is written on the page in .pdf format.

The positions can be defined according to an absolute metric expressed in a unit such as the centimeter or a relative metric such as a number of points, said positions being defined in relation to a frame of reference on the page in which an origin can be defined as a from the four corners of the page. In another example, positions can be defined according to a proportion of the page without the need for a metric. In the latter case, the unit can be taken as a dimension of the page such as the width or length of the page. This definition makes it possible to encode positions in a vector way.

According to an exemplary embodiment, a plurality of positions such as 4 positions can be produced so as to define the 4 corners of a rectangle or a polygon in which the sequence fits. Note that two positions are enough to reconstruct other positions of the form framing a line of a paragraph. This last remark applies to any other digital object in a rectangular shape, such as a paragraph, a table, etc.

According to one embodiment, each paragraph and the generated data which are associated with said paragraph are annotated for example according to a paragraph number and/or a paragraph identifier.

Each paragraph associated with the data produced, that is to say the symbolic characteristic K₁and the pair of POS positions₁is transmitted to a second software component denoted MOD₂in order to label the paragraph excerpted.

2nd algorithm of component MOD1

A second ALGO ₂ algorithm of the first software component MOD ₁ is implemented to detect and extract sets of words corresponding to notes, email addresses, URL links or quotes. The input of the second ALGO ₂ algorithm can be fed by the processing of the first ALGO ₁ algorithm which makes it possible to identify lines of symbols forming words or ideograms or even numbers.

To this end, the method of the invention implements the execution of a second set of predefined rules R ₂ and a second knowledge base BC ₂ . According to an example, the rules may include searches for predefined symbols such as the "@" symbol or the "http" symbol or the "www" symbol or a note of type "(1)", i.e. a number within parentheses or the symbol “(*)”.

Certain R ₂ rules are also applied so as to detect and extract sets of words arranged consecutively to an image or a graphic block identified by the SDK component ₁ . This function allows you to compare positions extracted from blocks of images or graphics with positions from sets of words.

Each R ₂ rule can be associated with other rules in order to discriminate certain elements such as legends or notes.

According to a first example of combination of rules R ₂ , an image block followed by a set of words having no other sets of words in an identical font can be annotated by a “legend” type identity, which allows to generate an attribute or category to said set of words. This set of words is also considered in the present invention as a single line or a paragraph. This generated data, such as an attribute, allows it to be encoded with the paragraph to be used by an interpreter in particular thanks to the TAG ₁ function of the third MOD ₃ software component detailed below.

According to a second example, within a paragraph identified by the first algorithm, a set of words in parentheses containing the symbol “www” can be identified as a URL.

According to a third example, a set of words at the bottom of the page whose start of the sequence begins with the symbol “(1)” can be a note.

The method of the invention makes it possible to generate at the output of this second ALGO ₂ algorithm a set of second digital objects OB ₂ . The second digital objects OB ₂ each comprise a set of words and a second symbolic characteristic K ₂ and a pair of positions POS ₂ calculated in the same way as in the execution of the first algorithm ALGO ₁ . The second characteristic K ₂ may include an attribute, which may define a label, directly calculated in the first software component MOD ₁ such as one of the following labels: {links, citations, notes}. The links can themselves be typed according to a URL or EMAIL type subcategory. Furthermore, according to one embodiment, the second characteristic K ₂ may include data identifying a font, a font size, a style or even an orientation of the cursive of the font of a word. Such data can also be extracted from rules R ₂ and a knowledge base BC ₂ such as those used in the execution of the first ALGO ₁ algorithm.

The data produced by the second algorithm ALGO ₂ are transmitted to a third software component MOD ₃ . The third software component MOD ₃ makes it possible in particular to order all of the outputs of the first and second software components MOD ₁ and MOD ₂ with the aim of regenerating an ordered sequence of objects in the processed page.

Scheduling allows you to regenerate a coherent PDF _N document while respecting the reading direction while allowing certain objects on the page to be activated independently of others.

3rd algorithm of component MOD1

The method of the invention comprises the execution of a third algorithm ALGO ₃ by the first software component MOD ₁ . The third ALGO ₃ algorithm allows you to extract table-type objects, such as the table of contents, list and page numbers.

This ALGO ₃ algorithm also includes the reconstruction of a digital object OB ₃ from at least one paragraph and a structure, an organization or an arrangement of said at least one paragraph. The structure of a paragraph can correspond to its geometric characteristics, or its number of lines or even its orientation. The paragraph corresponds to a sequence of symbols in natural language, it can be a letter, a number, a number, a symbol or a set of words, or even a combination of these last elements.

In the description, the third generic digital object OBG ₃ is called the table, list, table or table of contents type structure independently of the sequences of symbols it contains. These third OBG ₃ generic digital objects may include digital sub-objects of the cell, line, enumeration type, or even a hierarchical structure of lines having links between them, such as titles and sub-titles and sub-sub-titles. securities.

The third digital object OB ₃ is called the reconstruction of the association of the third generic digital objects OBG ₃ and the labeled paragraphs which are reinjected from the second component MOD ₂ within the structure.

In the context of the invention, the third generic digital object OBG ₃ can be encoded according to a symbolic characteristic denoted K ₃ .

According to a first example, a first table detection is implemented by the detection of lines and/or cells and/or borders or any other element characteristic of the geometry of a structure such as a table. A pair of positions is extracted in order to identify a rectangular block in which the table fits. The table positions are calculated either by taking into account the start and end of a sequence of one or more paragraphs, or by taking into consideration the border of the table, for example two corners of the table. Additionally, the algorithm includes a function to calculate the number of rows and/or the number of columns in the table.

In order to identify a table, the algorithm can implement a third set of predefined rules R ₃ and/or a third knowledge base BC ₃ . The R ₃ rules can correspond to the identification of intersections of linear plots delimiting cells, rows or columns of a table for example. The intersection of 3 line plots refers to a table cell at the edge of the table, the intersection of 4 line plots indicates the presence of a cell inside the table, the intersection of two line plots refers to the corner or end of the board.

The method of the invention also makes it possible to calculate the dimensions of the cells which can be extracted as well as their position on the page or inside the table.

Thus the third symbolic characteristic K ₃ includes a set of data making it possible to characterize the shape of the table, its dimensions as well as its organization in the form of rows and columns and cells.

According to a second example, a second “list” detection can be implemented in the method of the invention. In the context of the invention, by list we mean a list of lines of symbols forming, for example, words, ideograms or even numbers. Lists typically include bullet points that may correspond to identifiable graphic symbols. For example, symbols such as hyphens “-”, squares or stars or even numbers can identify the presence of a list. A dimension of a list can be inferred between the first paragraph and the last paragraph of the list. Here we understand as paragraph the sequence of discrete symbols in natural language arranged consecutively to the detected symbol.

A set of third rules R ₃ and the third knowledge base BC ₃ then include searches for characteristic symbols and alignments between paragraphs or lines. Thus, the pairs of positions of the symbol lines can be used to infer or corroborate the presence of a list.

Page numbers can also be detected by the third ALGO ₃ algorithm using R ₃ rules and a BC ₃ knowledge base. For example, the position of a digit within a page corresponding to the position of another digit in another page makes it possible to deduce the presence of an object having identical properties from one page to another. Finally, the identification of an ordered sequence on a set of pages makes it possible to deduce the identification of a page number.

For example, if a page number is located in a header or footer identified in the algorithm to automatically analyze page footers and page headers, the page number of the following pages is automatically detected 'one page to another.

According to one embodiment, the third algorithm ALGO ₃ receives as input a label LAB(i) of a paragraph PARA ₁ calculated by the second software component MOD ₂ . The LAB(i) label of a paragraph allows you to reorder the paragraphs within a table or list type structure. In addition, the LAB(i) label makes it possible to check that paragraphs of a table are generally homogeneous. Thus, the method of the invention makes it possible to avoid errors in the identification of objects. In particular, a page number cannot be found in a table or the title can be excluded from a list, etc.

Second MOD2 software component

• un moteur d’inférence inférant à partir de règles R₄et une base de connaissance BC₄;
• un moteur d’inférence inférant à partir de règles R₄et une base de connaissance BC₄combinées avec la mise en œuvre d’une intelligence artificielle telle qu’un modèle de type LSTM, TRANSFORMEUR ou tout autre réseau de neurones.

According to one embodiment, the second software component MOD ₂ comprises a function implementing an inference engine MI ₁ . This function can be implemented in different ways such as:

• an inference engine inferring from rules R ₄ and a knowledge base BC ₄ ;
• an inference engine inferring from R ₄ rules and a BC ₄ knowledge base combined with the implementation of artificial intelligence such as an LSTM, TRANSFORMER or any other neural network type model.

We recall the acronym in the Anglo-Saxon terminology of LSTM: “Long-Short-Term-Memory” and that a Transformer is called in the Anglo-Saxon terminology “Transformer”.

In the latter case, artificial intelligence can be implemented to automatically generate new rules or to enrich the knowledge base.

• Paragraphe de pied de page, correspondant à un paragraphe agencé dans une zone d’une page située en bas de page et comportant une chaine de caractères répétée au sein d’une pluralité de pages ou une séquence de nombres incrémentée à chaque page ;
• Paragraphe de haut de page, correspondant à un paragraphe agencé dans une zone d’une page située en haut de page et comportant une chaine de caractères répétée au sein d’une pluralité de pages ou une séquence de nombres incrémentée à chaque page ;
• Paragraphe standard, correspondant à un paragraphe défini par un style, une police et une taille de caractères donnés et étant le plus représentatif du document analysé ;
• Paragraphe distingué, correspondant à un paragraphe défini par un style, une police et une taille de caractères donnés différents du paragraphe standard.
• Paragraphe titre, correspondant à un paragraphe portant le nom d’une section ou d’une sous-section et suivi d’au moins un paragraphe structuré différent d’un paragraphe de pied de page ou d’une note

An interest is to identify a LAB(i) label of a paragraph among the following labels:

• Footer paragraph, corresponding to a paragraph arranged in an area of a page located at the bottom of the page and comprising a string of characters repeated within a plurality of pages or a sequence of numbers incremented on each page;
• Top-of-page paragraph, corresponding to a paragraph arranged in an area of a page located at the top of the page and comprising a string of characters repeated within a plurality of pages or a sequence of numbers incremented on each page;
• Standard paragraph, corresponding to a paragraph defined by a given style, font and character size and being the most representative of the document analyzed;
• Distinguished paragraph, corresponding to a paragraph defined by a given style, font and character size different from the standard paragraph.
• Title paragraph, corresponding to a paragraph bearing the name of a section or subsection and followed by at least one structured paragraph other than a footer paragraph or a note

The function allowing you to label paragraphs is noted LAB ₁ on the . According to a first example, it can correspond to a statistical function making it possible to classify probabilities obtained from the inference engine if the latter is configured to take into account a statistical model. This method makes it possible in particular to elect the best choice between possible outputs of the inference engine. Depending on one case, the statistical model may be a Bayesian network. The latter can be trained to create the inference engine model.

According to another example, the inference engine does not take into account a statistical model and determines, using the knowledge base BC ₄ , a label for all the paragraphs OB ₁ which are given to it as input. The other digital objects OB ₂ and OB ₃ are labeled from the BC ₂ and BC ₃ knowledge bases in the MOD ₁ software component. One advantage is not implementing specific training of the inference engine.

In the case of several possible outputs, that is to say several possible labels for the same object, the invention can include the generation of a notification accessible from an interface of a computer, a tablet or a Smartphone for a user to determine the label. Thus, the label assigned by a user in the event of ambiguity of several automatic labeling possibilities allows the learning function to generate a new rule which will be used by the inference engine.

The learning function is preferably configured to generate new rules or propose new rules to a user who can validate them or not.

Thanks to the output of the second software component MOD ₂ and the third algorithm ALGO ₃ of the first software component MOD ₁ , the invention therefore makes it possible to reconstruct a table or a list or a table comprising different paragraphs, that is to say a set of consecutive words, within each cell of the table or list.

Each third object OB ₃ comprises a third symbolic characteristic K ₃ characterizing the structure of the digital object. According to one embodiment, all of the labels of the paragraphs included within a third digital object OB ₃ can be encoded within the third symbolic characteristic to then be transmitted to the third software component MOD ₃ .

For this purpose, the third algorithm ALGO ₃ of the first software component MOD ₁ includes a function making it possible to recover a set of labeled paragraphs and to regenerate a digital object corresponding to a table or a list in which the different paragraphs are segmented within rows of a list or cells of a table.

Finally, the third ALGO ₃ algorithm makes it possible to carry out a consistency check between, on the one hand, certain paragraph labels and, on the other hand, the presence of said paragraphs within a digital object produced such as a table or a list. The third ALGO ₃ algorithm outputs data comprising the third digital objects OB ₃ , each third digital object OB ₃ being associated with a third symbolic characteristic K ₃ and a position pair POS ₃ . These data OB ₁ , OB ₂ and OB ₃ are transmitted to the third software component MOD ₃ to order these three objects in the overall reading sequence of the digital objects produced by each algorithm ALGO ₁ , ALGO ₂ and ALGO ₃ .

The third software component MOD ₃ therefore comprises a first function ORD ₁ making it possible to order the different digital objects OB ₁ , OB ₂ , OB ₃ produced by the first software component MOD ₁ . To this end, each digital object is ordered according to a reading sequence taking into account the nature of the digital object and its position. Its nature can be a table, a list, an image, a line of words, ideograms or numbers, a structured paragraph or even a quote, a note, a link, etc.

The third software component includes a function noted TAG ₁ on the making it possible to generate a PDF _N digital document comprising a structure which makes it possible to make the document accessible to visually impaired people in particular using an interpreter capable of reading the different objects and of generating multimedia outputs which can be accessible to these people.

The TAG ₁ function therefore makes it possible to associate the metadata, that is to say the symbolic characteristic data and the positions of the digital objects with said content of said digital object.

According to one embodiment, the method of the invention comprises a processing step making it possible to activate certain objects according to a reading direction or according to a user interaction. Activation of the object can, for example, result in the generation of a vocal reading of the content of a paragraph or a table from a voice synthesizer or more generally from a screen reader. An advantage of the invention is to make it possible to contextualize voice reading, for example with predefined semantics which depends on the object interpreted. Thus, the reading of a table by a voice synthesizer can integrate context sentences specifying, for example, that a table is going to be read. It can be specified a reading direction of the table and language elements indicating for example: “the table has 3 lines, in the first line, the first column, it is indicated “Paris”, in the second line, first column, it says “Dublin”, etc.

Thus, the invention makes it possible to give a disabled person access to complex content of a document which is not easily reproducible even in electronic form.

There illustrates an example of a page of a document F ₁ or F ₂ depending on whether we consider the input document or the document generated by the method of the invention. The F ₂ file is a document in .pdf format structured in the form of a plurality of digital objects generated and arranged using the method of the invention. In this illustration, the page comprises a plurality of first digital objects OB ₁ , each being associated with a symbolic characteristic K ₁ , a plurality of third digital objects OB3 and in this example a second digital object OB ₂ associated with a second symbolic characteristic K ₂ .

There represents an exemplary embodiment of a system of the invention comprising a user's terminal T ₁ making it possible to display and/or read the content of a digital file in .pdf format.

A first server SERV ₁ comprises at least one memory and a calculator making it possible to execute the algorithms of the first software component MOD ₁ . According to one embodiment, the second software component MOD ₂ is executed on a second server SERV ₂ comprising calculation means and at least one memory. An advantage of this distributed architecture is that it allows you to benefit from artificial intelligence dedicated to the calculation of labels. Data exchanges are carried out via a NET ₁ data network such as the Internet.

Claims (13)

Process for automated processing of a digital file (F₁) to generate an augmented digital object (OB₁):

• Acquisition of a first digital file (F ₁ ) in portable document format;
• Segmentation of said first digital file (F ₁ ) into a plurality of digital objects from a first software component (MOD1) comprising:
• Implementation of a first algorithm (ALG ₁ ) comprising:
  • Extraction of a first set (ENS ₁ ) of digital objects (OB ₁ ) each comprising a sequence of discrete symbols in a natural language defining a paragraph (PARA ₁ ); said extraction comprising the application of a first set of rules (R ₁ ) and a first knowledge base (BC ₁ ) to generate at least one first symbolic characteristic (K ₁ ) of said paragraph (PARA ₁ );
  • Generation of a plurality of pairs of 2D positions in the first digital file (F ₁ ), each pair of positions defining a first rectangular zone (Z ₁ ) comprising the display of a first object (OB ₁ ) and association of said pairs of positions to said first objects (OB ₁ );
• Implementation of a second algorithm (ALG ₂ ) comprising:
  • Extraction of a second set (ENS ₂ ) of digital objects (OB ₂ ) each comprising a sequence of discrete symbols in a natural language and at least one second symbolic characteristic (K ₂ ) representative of metadata associated with the object digital (OB ₂ ); said extraction comprising the application of a second set of rules (MR ₂ ) and a second knowledge base (BC ₂ );
  • Generation of a plurality of pairs of 2D positions in the first digital file (F ₁ ), each pair of positions defining a second rectangular zone (Z ₂ ) comprising the display of a second object (OB ₂ ) and association of said pairs of positions (POS ₂ ) to said second objects (OB ₂ );
• Implementation of a third algorithm (ALG ₃ ) comprising:
  • Extraction of a third set (ENS ₃ ) of at least one third digital object (OB ₃ ) each comprising a sequence of discrete symbols in a natural language and at least one third symbolic characteristic (K ₃ ) representative of the arrangement of a plurality of sub-objects within said at least one digital object (OB ₃ ), said extraction comprising the application of a third set of rules (R ₃ ) and a third knowledge base (BC ₃ );
  • Generation of a plurality of pairs of 2D positions in the first digital file (F ₁ ), each pair of positions defining a third rectangular zone (Z ₃ ) comprising the display of a third object (OB ₃ ) and association of said pairs of positions (POS ₃ ) to said third objects (OB ₃ );
• Labeling (LAB ₁ ) of the first digital objects (OB ₁ ) by means of a second software component (MOD ₂ ) comprising the implementation of an inference engine (MI ₁ );
• Scheduling (ORD ₁ ) of digital objects (OB ₁ , OB2, OB3) from a third software component (MOD ₃ ) taking into account as input:
  • The positions of the first objects, second objects and third objects;
  • The labels of each first object (OB ₁ );
  • The first symbolic characteristics (K ₁ ), the second symbolic characteristics (K2) and the third symbolic characteristics (K ₃ );
• Generation of a second digital file (F ₂ ) comprising all of the ordered digital objects.

Method according to claim 1, characterized in that the first algorithm (ALG₁) includes the steps of:

• Identification of a line of the first digital file comprising a first character string;
• Identification of the orientation of said line according to at least a first rule (R ₁ );
• Identification of symbolic information characteristic of the character string according to a second rule (R ₂ ) and of information characterizing the space between each character string of said line according to a third rule (R ₃ );
• Categorization of said line,
• Reiteration of the operations to identify each line and their categorization;
• Generation of said first symbolic characteristic of a paragraph when the categorization of a set of lines is identical.

Method according to any one of claims 1 to 2, characterized in that the labeling step (LAB₁) allows you to associate a label with a paragraph from one of the following labels:

• Footer paragraph, corresponding to a paragraph arranged in an area of a page located at the bottom of the page and comprising a string of characters repeated within a plurality of pages or a sequence of numbers incremented on each page;
• Top-of-page paragraph, corresponding to a paragraph arranged in an area of a page located at the top of the page and comprising a string of characters repeated within a plurality of pages or a sequence of numbers incremented on each page;
• Standard paragraph, corresponding to a paragraph defined by a given style, font and character size and being the most representative on all pages of the digital document (F ₁ );
• Distinguished paragraph, corresponding to a paragraph defined by a given style, font and character size different from the standard paragraph;
• Title paragraph, corresponding to a paragraph bearing the name of a section or subsection and followed by at least one structured paragraph other than a footer paragraph or a note.

Method according to any one of claims 1 to 3, characterized in that the second software component (MOD ₂ ) is implemented from the inference engine (MI ₁ ) executed from a set of rules and 'a basis of facts taking into account the first symbolic characteristics (K ₁ ). Method according to any one of claims 1 to 2, characterized in that the second software component (MOD₂) is implemented from the inference engine (MI₁), said inference engine (MI₁) being executed from a learning function whose input vectors include at least the first symbolic characteristics (K₁), said learning function implemented being:

• a “Recurrent neural network” type network, denoted RNN or;
• a “Long short-term memory” type network, denoted LSTM or;
• a BERT or GPT type transformer, such as GPT-3,

The implementation of a learning function making it possible to automatically determine new rules used by the inference engine (MI₁). Method according to any one of claims 1 to 5, characterized in that the second algorithm (ALG₂) understand :

• detecting a first predefined typographic symbol;
• the detection of a second predefined typographic symbol and a sequence of a given typographic symbol having a predefined spelling,
• generating a category defining metadata when at least one detection is carried out;
• generating a second symbolic characteristic (K ₂ ) comprising said character string comprising the detected symbol and the generated category.

Method according to any one of claims 1 to 6 characterized in that the third algorithm (ALG₃) includes the steps of:

• Identification of a plurality of symbols defining a repetition of arrangement of a set of data of the first file (F ₁ ) in order to define a third symbolic characteristic (K ₃ );
• Categorization of a third digital object (OB ₃ ) from the generation of the third symbolic characteristic (K ₃ ).

Method according to any one of claims 1 to 7, characterized in that the extraction of at least one third digital object (OB₃) understand :

• the selection of each first rectangular geographical area (Z ₁ ) associated with a first digital object (OB ₁ );
• Controlling the labels of each first object (OB ₁ ) each defining a paragraph (PARA ₁ ) with a label of a first object (OB1) at a previous adjacent position,
• generating a first enriched object when the labels of all of the first adjacent objects (OB ₁ ) meet at least one rule of said third set of rules (R ₃ ).

Method according to any one of claims 1 to 8, characterized in that the extraction of at least one third object (OB₃) understand :

• Selection of a third area of the page of the first file (F ₁ );
• Selection of each first geographical rectangular zone (Z ₁ ) associated with a first digital object (OB ₁ ) located in the third selected zone (Z ₃ );
• Control of the labels of each first object (OB ₁ ) each defining a paragraph (PARA ₁ ),
• Generation of a third object (OB ₃ ) when the labels of all the first objects (OB ₁ ) meet at least one rule of said third algorithm (ALG ₃ ).

Method according to claim 8, characterized in that the first enriched objects (OB ₁ '), the second objects (OB ₂ ), and the third objects (OB ₃ ) are ordered according to a display sequence on a page, said first objects (OB ₁ ) being ordered within an area associated with a first enriched object (OB ₁ '). Method according to any one of claims 1 to 10, characterized in that the digital objects of the second digital file (F ₂ ) produced as output are made activatable by means of an application aimed at exploiting the content of said second digital file (F ₂ ) from an interface of a user terminal intended for the visually impaired or blind, for example comprising a screen reader. System comprising a user's electronic terminal (T ₁ ) comprising at least one calculator, a memory, a display and a communication interface for transmitting messages on a data network to at least a first server (SERV ₁ ) comprising hardware resources for executing the first algorithm (ALG ₁ ), the second algorithm (ALG ₂ ), the third algorithm (ALG ₃ ) and the second software component (MOD ₂ ) in order to execute the steps of the method of one any of claims 1 to 10, said second digital file being transmitted via the data network to said electronic terminal (T ₁ ) of the user to display the second digital file and to access content that can be activated via the interface of said electronic terminal (T ₁ ). System according to claim 12 comprising a screen reader comprising a voice synthesizer for generating an audio stream of information, said information being extracted from the digital objects, said information further comprising information generated from the metadata of the second file (F ₂ ).