EP4256822A1

EP4256822A1 - Method for processing information, electronic device, and corresponding computer program product and recording medium

Info

Publication number: EP4256822A1
Application number: EP21835805.9A
Authority: EP
Inventors: Sonia Laurent; Laetitia HERTAULT; Cédric Floury
Original assignee: Orange SA
Current assignee: Orange SA
Priority date: 2020-12-04
Filing date: 2021-11-29
Publication date: 2023-10-11
Also published as: FR3117298A1; US20240004604A1; WO2022117936A1

Abstract

The invention relates to a method implemented in an electronic device, comprising: - at least partial playback, on a user interface of the electronic device, of items obtained from computing applications and associated with at least a first class of a plurality of item classes; - acquiring at least a first command to be executed, which command relates to at least one of the items that has been at least partially played back. The invention also relates to the electronic device and the corresponding computer program product and medium.

Description

DESCRIPTION

Title of the invention: Information processing method, electronic device, computer program product and corresponding recording medium

1. Technical area

This application relates to the field of collaborative exchanges in several terminals, for example in the context of the use of heterogeneous tools by said terminals. It relates in particular to a method for processing information provided by at least one computer application and adapted to be played, at least partially, on a user interface of an electronic device, as well as an electronic device, a computer program product and a corresponding recording medium.

2. State of the art

Many communication tools allow exchanges between users of communication terminals (fixed and/or mobile) such as personal computers, tablets, connected objects. These exchanges can be synchronous or asynchronous depending on the case and take multiple forms such as textual, audio and/or video.

The same event between several users of communication terminals can implement a plurality of tools, which can differ according to the users of the terminals or over time and can themselves be more or less complex. For example, in the context of a videoconference, exchanges by electronic messaging (email) or by so-called "instant" messaging (called "chat" or "Chat" according to the English terminology) can take place before or after the holding of a videoconference.

The rise of teleworking, and more generally of remote communications, and the proliferation and heterogeneity of these communication tools are increasingly confronting users with information overload (sometimes referred to as "infobesity"). Thus, users receive more and more information, more or less important, which can sometimes be forgotten, or not processed (or with delay) by some users.

The purpose of this application is to propose improvements to at least some of the drawbacks of the state of the art.

3. Disclosure of Invention

The present application aims to improve the situation using a process implemented in at least partially in an electronic device and comprising:

• a restitution, at least partial, on a user interface of said electronic device, of items obtained from computer applications and associated with at least a first class of a plurality of classes of items;

• an acquisition of at least a first command to be executed concerning at least one of said at least partially returned items.

For example, the present application relates in particular to a method implemented in an electronic device comprising:

• a selection, from among a plurality of classes of items, of at least a first class of items to be returned;

• a restitution (350), at least partial, on a user interface of said electronic device, of items obtained from a plurality of computer applications, said restitution taking into account a membership of said items obtained in said first class of items;

• an acquisition (370) of at least a first command to be executed concerning at least one of said at least partially restored items.

In certain embodiments, said selection takes into account parameterization data, and/or user profile, of at least one storage area accessible to said device, and/or data obtained via at least one user interface of said device.

In at least one embodiment, said items are obtained by extraction from output elements of said applications.

In at least one embodiment, at least one of said computer applications runs at least partially on said communication device. In at least one embodiment, the method includes obtaining said plurality of classes implementing an artificial intelligence technique.

In at least one embodiment, said acquisition of at least one first command comprises a selection of said at least one item concerned by said first command.

In at least one embodiment, said first command comprises an action relating to the restitution of said at least one item. In at least one embodiment, said first command comprises an action to be performed on said at least one item concerned and/or a second command to be transmitted to at least one application accessible from said electronic device. In at least one embodiment, the method comprises an update of said restitution after an execution of said first command.

In at least one embodiment, one of said elements and/or said at least one complementary element is a textual, audio, image and/or video element.

The characteristics, presented separately in the present application in connection with certain embodiments of the method of the present application can be combined with each other according to other embodiments of the present method. According to another aspect, the present application also relates to an electronic device suitable for implementing the method of the present application in any one of its embodiments.

For example, the present application thus relates to an electronic device comprising at least one processor configured to:

The present application relates in particular to an electronic device comprising at least one processor configured to:

The present application also relates to a computer program comprising instructions for the implementation of the various embodiments of the above method, when the program is executed by a processor, and a recording medium readable by an electronic device and on which the computer program is recorded.

For example, the present application thus relates to a computer program comprising instructions for the implementation, when the program is executed by a processor of an electronic device, of a method comprising:

For example, the present application relates to a computer program comprising instructions for the implementation, when the program is executed by a processor of an electronic device, of a method comprising:

For example, the present application also relates to a recording medium readable by a processor of an electronic device and on which is recorded a computer program comprising instructions for the implementation, when the program is executed by the processor, of a method comprising:

For example, the present application relates to a recording medium readable by a processor of an electronic device and on which is recorded a program computer comprising instructions for the implementation, when the program is executed by the processor, of a method comprising:

The programs mentioned above may use any programming language, and be in the form of source code, object code, or intermediate code between source code and object code, such as in partially compiled form, or in n any other desirable shape.

The information carriers mentioned above can be any entity or device capable of storing the program. For example, a medium may comprise a storage means, such as a ROM, for example a CD ROM or a microelectronic circuit ROM, or even a magnetic recording means.

Such a storage means can for example be a hard disk, a flash memory, etc. On the other hand, an information medium can be a transmissible medium such as an electrical or optical signal, which can be conveyed via an electrical or optical cable, by radio or by other means. A program according to the invention can in particular be downloaded from an Internet-type network.

Alternatively, an information medium may be an integrated circuit in which a program is incorporated, the circuit being adapted to execute or to be used in the execution of any one of the embodiments of the method which is the subject of the present application for patent.

4. Brief description of drawings

Other characteristics and advantages of the invention will emerge more clearly on reading the following description of particular embodiments, given by way of simple illustrative and non-limiting examples, and the appended drawings, among which: [Fig 1] presents a simplified view of a system, cited by way of example, in which at least certain embodiments of the method of the present application can be implemented, [Fig 2] presents a simplified view of a device suitable for implementing at least certain embodiments of the method of the present application,

[Fig 3] presents an overview of the processing method of the present application, in some of its embodiments

[Fig 4] presents an example of a user interface of an electronic device implementing at least certain embodiments of the method of the present application.

5. Description of embodiments

The present application aims to help a user (or a group of users) of at least one electronic device, such as a communication terminal, to process an information flow, by identifying in this information items which appear to be importance (referred to here as "significant") and by grouping these items together to provide them to the user, and thus facilitate their processing. A restitution in grouped form of the items can in fact facilitate their subsequent use by a user and/or the monitoring (for example over time) of these items.

By significant item, or item to be highlighted (“highlight” according to English terminology), we mean information (word, group of words, image, audio and/or video sequence for example) identified as potentially having a certain importance ( ie potentially of interest) for a user and with which it is likely to be associated with processing. It may be, for example, the statement of a problem to be addressed, a question to be answered, important information to be memorized, and/or a task to be performed.

The items may come from information received from different applications, local to a terminal or implementing several communication devices (such as communication tools such as electronic messaging tools, for example instant messaging, tools linked to public social networks or private).

By application is meant here a computer application (or tool), comprising in particular a software component.

In this way, the present application can for example help a user (such as an organizer of a meeting) confronted with a large flow of information not to forget to answer certain questions or to remember the context of a question (e.g. where, when, by whom, and/or on what occasion the question was asked). Indeed, embedded in the mass and diversity of applications, some information could be forgotten and not processed by the user. A grouping (or aggregation) of items can in particular bring together items originating from information originating from various applications and having for example a “semantic” link. Such a grouping can help a user to have an "overview" of a group of items, independently of the applications from which they come (directly or indirectly) and therefore favor processing of items by theme rather than by origin, which can help a user to gain efficiency.

Certain embodiments can also help a user to follow up on these items, in particular those that he has not yet processed, and allow him to perform one or more actions in relation to their processing.

Thus, by assisting in the processing of information flows by users, the process of the present application can, for example, when it is implemented in a professional context, help companies to improve the well-being and productivity of their collaborators.

The present application will now be described in connection with FIG. 1 in more detail. Figure 1 depicts a telecommunications system 100 in which certain embodiments of the invention may be implemented. The system 100 comprises one or more electronic devices, at least some of which can communicate with each other via one or more communication networks 120, possibly interconnected, such as a local network or LAN (Local Area Network, 1) and/or a wide area type network , or WAN (Wide Area Network, 2). For example, the network may include a corporate or domestic LAN network and/or a WAN network of the internet or cellular type, GSM - Global System for Mobile Communications, UMTS - Universal Mobile Telecommunications System, Wifi - Wireless, etc.

As illustrated in FIG. 1, the system 100 can also include several electronic devices, such as a terminal (such as a laptop computer 110, a smartphone 130, 132, 134, a tablet 136), and/or a server 140, for example an applications server, a storage device 150. The system may also include management and/or network interconnection elements (not shown). These electronic devices can be associated with at least one user 160, 162, 164, 166 (for example through a user account accessible by login), some of the electronic devices 110, 130 being able to be associated with the same user 160. FIG. 2 illustrates a simplified structure of an electronic device 200 of system 100, for example device 100, 130, 136 or 140 of FIG. 1, adapted to implement the principles of the present application. According to the embodiments, it can be a server, and/or a terminal.

The device 200 notably comprises at least one memory M 210. The device 200 can notably comprise a buffer memory, a volatile memory, for example of the RAM type (for "Random Access Memory" according to the English terminology), and/or a non-volatile memory. volatile (for example of the ROM type (for "Read Only Memory" according to the English terminology). The device 200 can also comprise a processing unit UT 220, equipped for example with at least one processor P 222, and controlled by a program computer PG 212 stored in memory M 210. On initialization, the code instructions of the computer program PG are for example loaded into a RAM memory before being executed by the processor P. Said at least one processor P 222 of the processing unit UT 220 can in particular implement, individually or collectively, any of the embodiments of the method of the present application (described in particular in relation to FIG. 3), according to the instructions s of the computer program PG.

The device can also comprise, or be coupled to, at least one I/O input/output module 230, such as a communication module, allowing for example the device 200 to communicate with other devices of the system 100, via wired or wireless communication interfaces, and/or such as an interfacing module with a user of the device, also called more simply in this application “user interface”.

By user interface of the device, we mean for example an interface integrated into the device 200, or part of a third-party device coupled to this device by wired or wireless communication means. For example, it may be a secondary screen of the device or a set of speakers connected by wireless technology to the device.

A user interface can in particular be a so-called “output” user interface, suitable for rendering (or controlling a rendering) of an output element of a computer application used by the device 200, for example an application running at least partially on device 200 or an "online" application running at least partially remotely, such as on server 140 of system 100. Examples of device output user interfaces include one or more screens, in particular at least one graphic screen (touchscreen for example), one or more loudspeakers, a connected headset. The interface of the device 200 can for example be adapted to the rendering of the dashboard illustrated in figure 4. By rendering, we mean here a restitution (or “output” according to the English terminology) on at least one user interface, in any form, for example comprising textual, audio and/or video components, or a combination of such components.

Furthermore, a user interface may be a so-called “input” user interface, suitable for acquiring a command from a user of the device 200. It may in particular be a command corresponding to an action to be perform in connection with a returned item, and/or a command to be transmitted to a computer application used by the device 200, for example an application running at least partially on the device 200 or an "online" application s' running at least partially remotely, such as on server 140 of system 100. Examples of device 200 input user interfaces include a sensor, audio and/or video acquisition means (microphone, camera (webcam) example), a keyboard, a mouse.

Said at least one microprocessor of device 200 can in particular be adapted for:

For example, said at least one microprocessor of device 200 can in particular be adapted for:

Some of the above input-output modules are optional and may therefore be absent from device 200 in some embodiments. In particular, if this application is sometimes detailed in connection with a device communicating with at least one second device of the system 100, the method can also be implemented locally by a device, for example using application output elements that do not require exchanges between devices (such as an electronic calendar for example).

On the contrary, in some of its embodiments, the method can be implemented in a distributed manner between at least two devices 110, 130, 132, 134, 136, 140 and/or 150 of the system 100.

The term "module" or the term "component" or "element" of the device here means a hardware element, in particular wired, or a software element, or a combination of at least one hardware element and at least one software item. The method according to the invention can therefore be implemented in various ways, in particular in wired form and/or in software form.

Figure 3 illustrates some embodiments of the method 300 of the present application. The method 300 can for example be implemented by the electronic device 200 illustrated in FIG. 2.

As illustrated in Figure 3, the method 300 may include obtaining 310 outputs related to one or more applications. This may include a heterogeneous plurality of applications. The applications can be diverse according to the embodiments of the present application and in particular according to the activities of a user of the device 200. For example, it can be an electronic messaging application, instant for example, a online application, accessible from a WEB portal, a textual, audio and/or video acquisition or rendering application, or a collaborative tool. The collaborative tool can for example allow exchanges between users, via a messaging interface, and/or offer possibilities for voice or video calls, videoconferencing, sharing of multimedia documents (text, audio and/or video, link internet, ..) archiving of exchanges, etc.

Obtaining 310 output elements can for example comprise reception on a communication interface of at least some of these elements and/or read access to a storage zone, local or remote, of these elements (for example a database (as illustrated by the element 150 of FIG. 1), such as a database common to several users or dedicated to one or more activities).

An output element can for example include a functional element, resulting from the implementation of the functionalities of the application (for example a more or less long text generated by a word processing or online messaging application) and/or data associated with this functional element. The data associated with this functional element can for example include an identifier of the application that produced (or provided) the functional element and/or an identifier of the functional element in a storage zone for this functional element, or even other data presented in more detail with the detailed rendering step below.

In the embodiment of FIG. 3, the method can also include an extraction 320 of significant items from the output elements obtained. From an output item of an application can be extracted zero, one or more significant items. The extraction 320 of such significant items can for example be based on conversion algorithms from voice to text called “STT” (Speech To Text), from text to voice called “TTS” (Text To Speech), or techniques of artificial intelligence, such as natural language processing techniques known by the acronym NLU (for Natural Language Understanding). In particular, a user context (for example his professional activities) may be taken into account.

For example, an instant messaging exchange of polite phrases, or "banalities" (such as "Hello", Are you okay?), with little semantic content, may not give rise to any meaningful item, while an exchange richer in meaning ("Our client XX has not been delivered this morning. Could you please call him back? Oh, by the way, could you provide me with a forecast of the team's holidays for Christmas?") could give rise to one or more significant items.

In the example illustrated, the method can further comprise an association 330 of at least one of the significant items extracted with at least one class (or category) of significant items. This association (or classification) may for example use artificial intelligence techniques, such as natural language processing techniques known by the acronym NLU (for Natural Language Understanding), and/or classification techniques by neural networks . In certain embodiments, these techniques can make it possible to obtain at least one set of hierarchical classes (a class comprising for example several subclasses, or the same class that can be attached to one or more "mother" classes according to different models of classification). The candidate classes for an association can for example correspond to the classes used to train a neural network used during the association.

In certain embodiments, the method may comprise a rendering (or restitution) 350 of at least a part of the significant items extracted on at least one electronic device output user interface. The user interface rendering environment can be more or less complex. It can for example include a rendering space dedicated to item aggregations (sometimes called

“Dashboard” (or “Highlight Dashboard” in the present application).

Figure 4 illustrates an example of restitution (or rendering) of significant items. In this illustration, the rendering is performed in visual form, in at least one rendering window. Of course, other types of restitution can be used in other embodiments.

Significant items can be grouped (or in other words aggregated) together according to one or more grouping criteria. Thus, a grouping criterion can be a criterion for belonging to one or more classes. The rendering can also be limited to the rendering of significant items associated with certain classes. Thus, in certain embodiments, the classes of items to be restored can be selected according to parameterization data, and/or a user profile, stored for example in a storage zone accessible to the device 200, or data obtained 340 via a user interface. In the example of figure 4, only the items relating to questions to which no answer has been provided (by the user of the device for example) are returned.

The total number of significant items rendered, like the number of significant items rendered for a given class, may vary depending on the embodiments. Likewise, the number and choice of classes rendered may vary according to the embodiments. For example, it can be a rendering of a constant total number of significant items, a rendering of the significant items of a constant number of classes, or even a rendering of a fixed percentage of significant items extracted or a rendering of significant items from a fixed percentage of the total number of classes.

In certain embodiments, the number of items and/or group(s) of items rendered can take into account the type of rendering (visual, sound, etc.), the user interface of the restitution (the size of a rendering screen for example), of a priority assigned to at least one class of significant items (according to a criterion of relevance for a user for example), or of rules defined in at least one file configuration or via a user interface, allowing for example a selection of significant items to be returned according to one or more classes associated with them). Within the same class, the order of aggregation of the significant items may vary according to the embodiments. For example, the order of aggregation of significant items may take into account their chronology and/or their semantics. In the example of FIG. 4, a dedicated interface 410 thus allows acquisition of a scheduling criterion within a group. Thus, account can be taken, for example, of a generation chronology of the output elements from which the items are extracted by the corresponding applications (as illustrated in figure 4), or of a date at the latest for the processing of an item ( so as to restore first the items requiring an urgent action of the user). It can also be taken into account that items of a restored class belong to other classes (items belonging to a given class being grouped together) or that they come from the same application.

FIG. 4 illustrates a rendering of a dashboard 400 comprising a rendering zone 420 of an item, in which the rendering of at least a part 422 of an item (here textual), can be accompanied by a rendering of additional data, such as at least one identifier 424 of the application (for example Skype ©) from which the item comes and/or at least one identifier 426 of a user (person or group of people) of the application from which the item comes and/or a date and/or time 428 of generation (creation and/or modification) of the output elements from which the items are extracted.

As illustrated in FIG. 4, the items originating from several interlocutors and/or from several applications can be grouped together.

In the embodiment illustrated in FIG. 3, the method can also comprise an optional step of modifying 360 at least one aggregation during restitution. For example, the 360 modification may include the addition of at least one significant item. This item is then associated with the class of items of this aggregation. A user can thus add to the aggregation an item that would not have been extracted from an output element during the extraction step. It can also be an item added freely by a user having at least partial access to at least one class of items (for example during the at least partial restitution of a class of items) For example, an “Add” function can allow the user to add an item to the significant items returned. Additional information can also be added in addition to this item (such as an identifier of the user at the origin of the addition, a date and/or time of addition of the item, a mention of the addition "free", a classification, a reason for the addition, etc.) (Example: Addition (By: Didier Dupont, On: 04 Dec. 2020 11.15, Text: Request the entry of my time spent. Channel: free addition and not via an application) - how: via informal exchange). Conversely, a significant item can be deleted from the restitution (for example because it is considered irrelevant) or its classification can be modified.

The method can also comprise an optional step of acquiring feedback on a current or past restitution. This feedback can take the form of an evaluation (a rating for example) of the relevance of the significant items returned. It can also take the form of the creation of a new class (or category) of significant item(s), and its association with at least one significant item rendered (for example a significant item added during the step optional modification described above).

The modifications made to the restitution, such as any acquired feedback, can be used to modify the rules for extracting and/or classifying significant items. Specifically, in some embodiments, the occurrence of changes and/or the acquisition of feedback may trigger an update of a model used for extraction or classification. It can for example be a question of a refinement of the learned model (“fine tuning” according to the English terminology) by training it using the feedback data obtained, used as learning data. Such an improvement can help to obtain a model better adapted to the expectations of a user, or to an evolution of the applications he uses, or to correct errors in the model.

According to FIG. 3, the method can also include a 360 modification (before or during the 350 rendering) of at least one of the returned items, in particular to make it more understandable or more concise. For example, a textual sequence can be slightly modified to increase its clarity, or replaced by a shorter sequence of the same semantics. A vocal sequence can be replaced by a vocal sequence of equivalent semantics but with a less pronounced accent or a slower rhythm. For speed of access or memory saving, an image can also be replaced by a portion of this image.

According to FIG. 3, the method further comprises an optional step 370 of acquisition of a command linked to at least one of the returned items. It can be for example at least one action to be performed on an item, on its rendering and/or on the output element from which it comes (for example the deletion or the archiving of an item or of the output element from which it is taken). It may be for example at least one command, in connection with this item, to be transmitted to an application, (for example a command to generate a voice and/or text message to another device or to a user from a third-party terminal, to transmit the item and/or the output element from which it originated, or to comment on it (for example, provide an answer to a question)). Thus, an order may require the deletion of items representative of tasks to be performed once they have been completed. The application to which to send a command linked to an item can correspond to the application from which the item originates or it can be a different application. Thus, it can be answered by voice message to an item from a text messaging application. Other command examples may include adding an event to a calendar, in particular electronic, setting a reminder concerning an action to be performed later, scheduling an appointment or a meeting. An item and the action performed on this item can also be used to enrich a document (multimedia file, minutes of a meeting, report, etc.).

A command can relate to a single item among the returned items or a command can relate to several items. Group processing of items can indeed promote faster and/or more consistent processing of certain items. For this purpose, the user interface may include interface elements suitable for acquiring a selection of the item(s) concerned by a command.

Such interface elements can for example comprise means of explicit or implicit designation of at least one returned item. For example, these designation means can allow a selection of a plurality of items rendered for a class (for example all the items rendered for the class, or only some of the items rendered for the class), certain items can also be excluded explicitly or implicitly. Implicit designation of certain items can indeed help in faster acquisition of the order.

In certain embodiments, as illustrated in FIG. 4, the actions that can be performed on an item 430 can be indicated by at least one interface element 432, 434, possibly operable by a user (for example by click).

The acquisition of a command can include the acquisition of at least one parameter necessary for an action linked to the command (for example an access path to a file where to archive one or more items, elements allowing the generation of 'a message automatic (such as a designation of one or more recipients of a reply message, a voice or text message to be transmitted, etc.)

The process detailed above can also be applied to elements provided by various applications. Thus, according to a first example of implementation, a user can receive via an electronic messaging application a document that can be viewed via a text editing application, such as a document in PDF format or a Word© document. This can be a meeting report, for example. When the user opens the document via the text editing application, the content of the document can be retrieved via an API of the text editor for example and significant items can then be extracted from this content and then classified.

Thus, according to a second example of implementation, an analysis of the content of an electronic message, or of a post on a social network (for example a corporate social network such as the Jive © tool), for example a redirection via a post of an electronic message can make it possible to extract significant items associated with different classes.

Claims

CLAIMS Method implemented in an electronic device comprising:

• an acquisition (370) of at least a first command to be executed concerning at least one of said at least partially restored items. Electronic device comprising at least one processor configured to:

• an acquisition (370) of at least a first command to be executed concerning at least one of said at least partially restored items. Method according to Claim 1 or device according to Claim 2, in which the said selection takes account of parameterization and/or user profile data of at least one storage zone accessible to said device and/or of data obtained via at least one user interface of said device. A method according to claim 1 or 3, or a device according to claim 2 or 3, wherein at least one of said computer applications runs at least partially on said device. A method according to any of claims 1, 3 or 4 comprising, or apparatus according to any of claims 2 to 4 wherein said at least one processor is configured for, classifying said items according to said plurality of classes putting implement an artificial intelligence technique. Method according to any one of claims 1 or 3 to 5, or device according to any one of claims 2 to 5, wherein said acquisition of at least a first command comprises a selection of said at least one item concerned by said first command. Method according to any one of claims 1 or 3 to 6, or device according to any one of claims 2 to 6, wherein said first command comprises an action relating to the restitution of said at least one item. Method according to any one of claims 1 or 3 to 7, or device according to any one of claims 2 to 7, wherein said first command comprises an action to be performed on said at least one item concerned and/or a second command to transmit to at least one application accessible from said electronic device. Method according to any one of claims 1 or 3 to 8 comprising, or device according to any one of claims 2 to 8 wherein said at least one processor is configured for, an update of said restitution after an execution of said first ordered. Method according to any one of Claims 1 or 3 to 9 or device according to any one of Claims 2 to 9, in which at least one of the said elements and/or the said at least one complementary element is a textual, audio, image and /or video. Computer program comprising instructions for implementing, when said computer program is executed by a processor of an electronic device, a method comprising:

• an acquisition (370) of at least a first command to be executed concerning at least one of said at least partially restored items. Recording medium readable by at least one processor of an electronic device and on which is recorded a computer program comprising instructions for the implementation, when the computer program is executed by said at least one processor, of a method comprising: a selection, from among a plurality of classes of items, of at least a first class of items to be rendered; • a restitution (350), at least partial, on a user interface of said electronic device, of items obtained from a plurality of computer applications, said restitution taking into account a membership of said items obtained in said first class of items; • an acquisition (370) of at least a first command to be executed concerning at least one of said at least partially restored items.