WO2022122165A1

WO2022122165A1 - Methods, system and apparatus for providing mental state data as an on-demand service

Info

Publication number: WO2022122165A1
Application number: PCT/EP2020/085648
Authority: WO
Inventors: Johan Eker; Anders Berkeman; Pex TUFVESSON; Mikael Johansson
Original assignee: Telefonaktiebolaget Lm Ericsson (Publ)
Priority date: 2020-12-10
Filing date: 2020-12-10
Publication date: 2022-06-16

Abstract

A method, system and apparatus for providing mental state data of an individual is provided. The method comprises receiving a service request for mental state data for the individual from a requesting application. The method further comprises processing the service request to determine a usage context for the requested mental state data. The method further comprises determining whether the service request can be fulfilled based on at least one user setting for sharing the mental state data of the individual and the determined usage context. Each user setting comprises at least an indication of a mental state classification and an allowed usage context. The method further comprises providing data representing the available at least one mental state classification to the requesting application based on a determination that user settings allow sharing of the mental state data for determined usage context of the service request.

Description

METHODS, SYSTEM AND APPARATUS FOR PROVIDING MENTAL STATE DATA AS AN ON- DEMAND SERVICE

TECHNICAL FIELD

The present disclosure relates to processing and sharing of mental state data and to related aspects. More particularly, the present disclosure relates to a method, system and apparatus for providing mental state data as an on-demand service to third party applications and/or devices.

BACKGROUND

Many applications are also known in the art which can process a brainwave data obtained from an electroencephalogram, EEG, sensor array. For example, applications are known in the art for an individual to visualize their brain activity or to provide a brain-machine interface, BMI, where a user's mental activity provides input to a device or machine which may result in a response from a real or virtual object.

Mental state data is of great interest in a variety of real and virtual situations involving, for example, entertainment, retail, finance, transport, and health. However, whilst someone may want to share certain classifications or categories of their mental state with an application, they may not want to share other classifications or categories of their mental activity. For example, in order to better learn a language, a user may choose to share their emotional and cognitive mental states with an educational application and an on-line instructor, but they may not want to continue to share their emotional mental state data with the application and the instructor if, from the user's perspective, the context of the activity for which they are sharing their mental state changes. As an example, if that user were to take a break from actively engaging in the language learning task to make a cup of tea or answer the phone and talk to a friend and did not remove their headset. In such circumstances, the user might want to no longer share their emotional mental state data or to not share any mental state data.

At present, many individual applications need to be individually configured to use brainwave data feeds from an individual. These brainwave data feeds may comprise more information derived from a user's brain activity usually than is necessary or appropriate to share with an application. Currently, however, there is no mechanism to police what mental state information an application might be extracting from a user's brain activity or if this extracted information is being used as that user intended. For example, some headsets are being sold for public use which are mobile and which could be worn all day long by a user. If a user wearing such a headset enters the premises of a retail outlet, their mental state data feed, if available to the retail outlet, could be used for neuro-marketing, which is something that some users would object to. Another problem arises if a user wishes to use the same or a different headset for multiple applications, as many applications also require a user to (re)train and sometimes to also (re)configure and/or (re)ca librate their EEG headset prior to use. This can mean that every time a user engages with a different application, time is lost before their mental state data is available to that application if they may have to repeat the training process, even if it is the same headset they are using.

Therefore, there is a need for an improved system for processing and sharing the mental state information of the individual.

SUMMARY

It is an object of some embodiments to solve or mitigate, alleviate, or eliminate at least some of the above or other disadvantages. Various aspects of some of the embodiments of the disclosed technology seek to obviate and/or mitigate problems associated with providing mental state data for an individual as a service to third party applications.

Some embodiments of the disclosed technology relate to providing a service to allow various applications to access a user's mental state data as an on-demand service. The service is provided in a way that allows a user to configure various settings which limit or filter what mental state data is shared with an application. The disclosed technology also enables a user to share their mental state data with different applications in a seamless manner without necessarily needing to retrain their headset to locate features relevant for each new application by making their mental state data available on-demand to applications. The applications are configured to generate service requests to access the user's mental state data which are sent to a server system configured to provide that user's mental state on demand to applications.

A first aspect of the disclosed technology is a computer-implemented method for providing mental state data. The method comprises receiving a service request for mental state data for an individual from a requesting application, determining a usage context for the requested mental state data, associating the service request with a brainwave activity data feed from the individual, determining mental state data for the individual based on the brainwave activity data feed, filtering the determined mental state data based on at least one user setting associated with a user account of the individual, and providing the filtered mental state data to the requesting application based on the determined usage context.

In some embodiments, the associating comprises extracting an identifier from the service request and determining the extracted identifier is associated with the user account of the individual.

In some embodiments, the user account of the individual includes information representing the at least one user setting for filtering the mental state data of the individual comprising one or more sharing conditions, wherein the one or more sharing conditions for sharing mental state data of the individual are configurable by the individual as one or more user settings for the user account.

In some embodiments, at least one sharing conditions for sharing mental state data of the individual comprises a user-configurable setting in the user account of the individual to share or not share a mental state classification of the mental state data for the individual in one or more user-selected usage contexts.

In some embodiments, at least one sharing condition for sharing mental state data of the individual comprises a user-configurable setting in the user account of the individual to share or not share for a usage context one or more user-selected mental state classifications of the mental state data for the individual. Examples of such a usage context include a possible use of the mental state data by the application which could be inferred based on a current or anticipated user activity and/or based on one or more characteristics or meta-data of the requesting application. For example, a requesting application could be a music application, but, based on a sports watch data feed also being activated when the music application generates the request for the individual's mental state data, the server may determine the likely usage of the mental state data is a sports-related usage context and not a relaxation related usage context.

In some embodiments, based on the sharing conditions for sharing mental state data, the filtering comprises removing mental state data having a mental state classification for which a sharing condition is not met from the mental state data sent to the requesting application.

In some embodiments, the filtering retains as filtered mental state data for the individual at least one mental state classification for which a sharing condition is met in the user account of the individual and the filtered mental state data comprises an indication of the at least one mental state classification.

In some embodiments, the filtering retains as filtered mental state data for the individual at least one mental state classification for which a sharing condition in the user account of the individual is met and the filtered mental state data comprises a filtered brain activity data feed associated with the at least one mental state classification of the individual.

In some embodiments, the filtered mental state data comprises current mental state data of the individual derived by processing the brainwave activity data feed for of the individual in real-time to determine a set of brain-wave activity features.

In some embodiments, the mental state data is associated with a mental state classification by using an artificial intelligence, Al, system configured to determine a set of brain-wave activity features for the individual from their brain-wave activity data feed and to associate the determined brain-wave activity features with a mental state classification for the individual. Examples of Al models which can be configured in this manner include any suitable Al model computer program comprising a mathematical model that can be used to classify data and form a decision and includes machine learning classifier models where the algorithms improve on their performance as they are exposed to more data over time and deep learning models in which the machine learning models are multi-layered and learn from large amounts of data, for example, which are suitable for wavelet transform for classification of EEG signals may be used which include support vector machines, SVMs, and/or artificial neural networks ANNs, for example ANNs with auto-regression, AR, maximum-likelihood estimation, MLE, or long-short-term memory, LSTM, classifiers. Some examples of suitable Al models are disclosed, for example, in Wavelet Transform for Classification of EEG signal using SVM and ANN by Nitendra Kumar, Khursheed Alam and Abul Hasan Sissiqi, published in Biomedical and Pharmacology Journal Vol. 10 (4), 2061-2069 (2017), https://dx.doi.org/10.13005/bpj/1328.

In some embodiments, the determined mental state data comprises a mental state classification of at least one of: an attentional state, an emotional state, a cognitive state, and an arousal state.

In some embodiments, a usage context comprises one or more of: a task-based context, a geographic environmental context, a situational environmental context, and a physiological context.

In some embodiments, the usage context is determined based on one or more of: explicit information provided by the application in the request; a usage context stored in the user account association with the identifierforthe application; and one or more inferences of activity ofthe individual based on one or more of: at least one data feed or a fused plurality of data feeds from one or more sensors associated with the individual at the time the request was received by the server.

In some embodiments, the usage context is dynamic, and the method further comprises monitoring the usage context and adapting the filtered mental state data based on the monitored usage context.

In some embodiments of the method, the method further comprises: monitoring the at least one current mental state classification data of the individual for a duration of time, determining that at least one mental state classification has changed to a new mental state classification; and based on at least one a sharing condition indicating the new mental state classification is not to be shared for the usage context, removing the mental state classification data from the mental state data provided to the requesting application.

In some embodiments, at least one user setting in the user account of the individual comprises at least one condition for performing an action based on a determined mental state classification.

In some embodiments, the action comprises pushing information associated with the usage context and the current mental state classification to at least one device, wherein the at least one device is configured to perform an action responsive to receiving the pushed information. For example, an alarm may be provided if the mental state indicates, given particular usage context an escalation in a risk of harm to the user, such as, for example, a mental state of drowsiness increasing above a threshold for safe driving.

In some embodiments, the at least one condition for performing the action based on a current mental state classification comprises determining at least one feature of the brain activity data feed meets a threshold triggering the action. In some embodiments, the threshold is a context-based threshold which is associated with a particular usage context.

In some embodiments, the user account of the individual is associated with at least one of: an account-holder identifier for the individual account holder and a source identifier for a source device of the brainwave activity data feed of the individual, and determining if the extracted identifier is associated with a user account of the individual comprises determining if the extracted identifier comprises a source identifier or an account-holder identifier which matches a corresponding source identifier or an account-holder identifier for the user account associated with the individual.

In some embodiments, the user account of the individual is associated with one or more other sensed activity data feeds associated with the individual. For example, a user may be wearing a pulse oximeter which could provide a pulse and /or blood oxygen data feed(s) indicating a user is sleeping or at rest, or have just activated a running application from which the server could infer the use context of the user is related to running, even if the application requesting the user's mental state information is a music related application.

Advantageously, some embodiments of the disclosed technology provide real-time filtered mental state data for an individual to be shared as a stream of events with one or more applications. Advantageously some embodiments of the disclosed technology allow a user to wear the same headset for a variety of different applications, and in some embodiments, the user can set settings so that one or more different cla ssif ication (s) or the same classification(s) of the mental state data can be obtained concurrently for different applications.

A second aspect of the disclosed technology is a computer-implemented method of configuring an individualized mental state on-demand service, the method comprising: providing, for display on a user device, a prompt for configuring one or more usage contexts and one or more mental state classifications as one or more user settings of a user account for an individual whose mental state is the individualized mental state provided by the on-demand service, receiving at least one configured user setting, each configured user setting associating a mental state classification of the individual with a usage context for which the mental state classification is to be made available to one or more requesting applications, for example, service requesting applications, and storing, in the user account, each received at least one configured user setting, an identifier for the user account, and a brain activity identifier associated with a source device of sensed brain activity data for the individual.

In some embodiments, the method of the second aspect further comprises receiving a registration request to establish the user account from a device and associating the user account with the brain activity identifier associated with the source device of the sensed brain activity data for the individual.

A third aspect of the disclosed technology comprises a computer-implemented method of training an artificial intelligence, Al, system to provide data representing at least one individualized mental state classification for a mental state of an individual, the method comprising: receiving, by the Al system, training data comprising a sensed brain activity data feed for the individual, training the Al system by using the Al system to analyze the training data to find sets of features for classifying mental states of the individual, cross-training the trained Al system using sets of features classifying the same mental states of at least one other individual; and generating sets of features using the cross-trained Al system, each set of features classifying a mental state of the individual.

In some embodiments, the Al system is trained using labelled data and cross-trained using labelled data, wherein the labelled data is generated by user input based on at least one feedback stimuli.

In some embodiments, the Al system is trained using unlabeled data and cross-trained using labelled data.

In some embodiments, the cross-trained Al system is configured to output a current mental state classification as an external stimulus to the individual.

In some embodiments, the current mental state classification is determined by associating features derived from a current time-segment of a brain wave activity data stream sensed from individual with a set of features previously associated with a mental state classification of the individual.

In some embodiments, a feature comprises one or more power spectrum peaks in one or more frequency bands associated with brain wave activity in a current time-segment of the brain wave activity data stream for one or more channels of brain wave activity data.

In some embodiments, the feature further comprises an association between one or more power spectrum peaks in one or more frequency bands associated with brain wave activity in a current time-segment of the brain wave activity data stream for one or more channels of brain wave activity data and the location of a brain wave activity sensor providing a channel of brain activity data on the scalp of the individual.

In some embodiments, the feature further comprises an association of the features for brain wave activity sensed in the current time-segment and with the features for brain wave activity sensed in one or more previous time-segments.

In some embodiments, the training data includes sensed brain activity data and other activity data associated with the individual from at least one sensor configured to detect the other activity associated with the individual concurrently with the brain activity of the individual. For example, in some embodiments, a fused data feed could be provided by another device associated with the user for both the mental state data and the activity data, or the data could be fused appropriate by the server in some embodiments instead.

In some embodiments, the other activity data of the individual comprises data generated using at least one of the following sensors: a heart-rate sensor, a blood oxygen sensor, a skin-temperature sensor, a gaze tracking sensor, and a hydration sensor.

In some embodiments, the one or more frequency bands comprise one or more: a brain activity alpha wave, beta wave, gamma wave, delta wave or theta wave, frequency bands.

Advantageously, some embodiments provide real-time filtered mental state data for an individual to be shared as a stream of events with one or more applications.

Advantageously this allows a user to wear the same headset for a variety of different applications, and in some embodiments, the user can set settings so that one or more different classification(s) or the same classification(s) mental state data concurrently for different applications.

A fourth aspect is a computer-implemented method for determining at least one individualized mental state classification of a mental state of an individual. The method comprises receiving sensed brain wave activity data for the individual, determining features in the received sensed brain wave activity data, associating the features in the received sensed brain wave activity data with one or more feature sets, each feature set comprising at least one feature, associated by an Al system with one or more mental state classifications of the individual, and generating at least one mental state classification for the sensed brain wave activity data based on the association of the determined features with the at least one mental state classification meeting a classification condition.

In some embodiments, the determining of the features in the received sensed brain wave activity data uses an Al system trained using a method according to the third aspect or any of its disclosed embodiments.

In some embodiments, the receiving and the generating are performed in real-time, as in online rather than off-line, and the mental state classification comprises a current mental state classification.

In some embodiments, the method according to the fourth aspect of any of its disclosed embodiments further comprise: providing the at least one mental state classification to an apparatus configured to perform a method according to the first aspect or second aspect or any of the disclosed embodiments of the first or second aspects. The method may further comprise providing data representing a mental state classification to an application, for example, an application which has requested the mental state data of the individual.

In some embodiments, the method is performed by a user equipment or a server, and the method further comprises: receiving the one or more feature sets received from an Al system configured to perform a method according to the third or fourth aspects, wherein the user equipment or the server performs the associating by associating the features in the received sensed brain wave activity data with the received one or more feature sets to determine the one or more mental state classifications of the individual to a requesting application. A fifth aspect of the disclosed technology is an apparatus or control circuitry for providing a mental state of at least one individual as a service, the apparatus or control circuitry comprising a memory comprising machine-executable instructions and one or more processors or processing circuitry. The machine-executable instructions are configured, when loaded from the memory and executed by the one or more processors or processing circuitry, to cause the apparatus or control circuitry to process a service request for mental state data for an individual received from a requesting application by causing the apparatus or control circuitry to: determine a usage context for the requested mental state data; associate the service request with a brainwave activity data feed from the individual; determine the mental state data for the user based on the brainwave activity data feed; filter the determined mental state data based on a user setting; and transmit the filtered mental state data to the requesting application based on the determined usage context.

In some embodiments of the apparatus or control circuitry of the fifth aspect, the machineexecutable instructions further cause the apparatus or control circuitry to perform one of the embodiments of the first method aspect.

Some embodiments of the apparatus or control circuitry of the fifth aspect are configured to configure an individualized mental state on-demand service for an individual as a service, wherein the machine-executable instructions are configured, when loaded from the memory and executed by the one or more processors or processing circuitry, to cause the apparatus or control circuitry to provide, for display on a user device, a prompt for configuring one or more usage contexts and one or more mental state classifications as one or more user settings of a user account for an individual whose mental state is the individualized mental state provided by the on-demand service; receive at least one configured user setting, each configured user setting associating a mental state classification of the individual with a usage context for which the mental state classification is to be made available to one or more requesting applications, and store, in the user account, each received at least one configured user setting, an identifier for the user account of the individual and a brain activity identifier associated with a source device of sensed brain activity data for the individual.

A sixth aspect of the disclosed technology is an apparatus or control circuitry comprising: a memory comprising machine-executable instructions; and one or more processors or processing circuitry. The machine-executable instructions are configured, when loaded from the memory and executed by the one or more processors or processing circuitry, to cause the apparatus or control circuitry to: receive sensed brain wave activity data for the individual; determine features in the received sensed brain wave activity data; associate the features in the received sensed brain wave activity data with one or more feature sets, each feature set comprising at least one feature, associated by an artificial intelligence, Al, model, or machine-learning model, with one or more mental state classifications of the individual; and generate at least one mental state classification for the sensed brain wave activity data based on the association of the determined features with the at least one mental state classification meeting a classification condition.

In some embodiments of the apparatus or control circuitry of the sixth aspect, the machineexecutable instructions further cause the apparatus or control circuitry to perform one of the embodiments of the fourth method aspect.

A seventh aspect of the disclosed technology comprises an apparatus or control circuitry for training an artificial intelligence, Al, system comprising: a memory comprising machine-executable instructions; and one or more processors or processing circuitry. The the machine-executable instructions are configured, when loaded from the memory and executed by the one or more processors or processing circuitry, to cause the apparatus or control circuitry to receive training data comprising a sensed brain activity data feed for an individual, train the Al system by using the Al system to analyze the training data to find sets of features for classifying mental states of the individual, cross-train the trained Al system using sets of features classifying the same mental states of at least one other user, and generate sets of features using the cross-trained Al system, each set of features classifying a mental state of the individual.

In some embodiments of the apparatus or control circuitry of the seventh aspect, the machineexecutable instructions further cause the apparatus or control circuitry to perform one of the embodiments of the third method aspect.

An eighth aspect of the disclosed technology comprises an apparatus or control circuitry comprising; a data communications transceiver; a memory comprising machine-executable instructions; and one or more processor(s), wherein the machine-executable instructions, when loaded from the memory and executed by the one or more processor(s), are configured to cause an application hosted by the apparatus or control circuitry to generate a request for mental state data for an individual, wherein the machine-executable instructions are further configured to cause the apparatus or control circuitry to send the request to the apparatus of the fifth aspect or a disclosed embodiment of the fifth aspect.

A ninth aspect of the disclosed technology comprises an apparatus or control circuitry (8), for example, a headset or the like, comprising a plurality of sensors, for example, a sensor array, configured to detect brainwave activity of an individual, a data communications transceiver, a memory comprising machine executable instructions, one or more processor(s) or processing circuitry, wherein the instructions, when loaded from the memory and executed by the one or more processor(s) or processing circuitry, are configured to cause the brain wave activity data of the individual detected by the plurality of sensors to be transmitted to the apparatus of the fifth aspect or a disclosed embodiment of the fifth aspect.

A tenth aspect of the disclosed technology comprises a server system for providing mental state data for an individual on demand to a plurality of requesting applications, the server system comprising at least the apparatus or control circuitry of the fifth aspect or a disclosed embodiment of the fifth aspect and the apparatus or control circuitry of the sixth aspect or a disclosed embodiment of the sixth aspect.

It will be apparent to anyone of ordinary skill in the art that in some embodiments, the apparatus of the fifth aspect may also comprise the apparatus of the sixth aspect, in other words, the server system which handles the requests may also include the Al system which classifies the mental state data (and the server system may also then handle training the Al system as well in some embodiments).

In some embodiments, the server system further comprises the apparatus or control circuitry of the seventh aspect.

In some embodiments, the server system further comprises at least one apparatus according to the eighth aspect and/or at least one apparatus according to the ninth aspect.

An eleventh aspect of the disclosed technology comprises a computer program product comprising a non-transitory computer readable medium, having thereon machine-executable instructions which, when loaded and executed by an apparatus or control circuitry comprising one or more processors or processing circuitry, are configured to cause the method of the first aspect or a disclosed embodiment of the first aspect to be performed.

Another aspect of the disclosed technology comprises a computer program product comprising a non-transitory computer readable medium, having thereon machine-executable instructions which, when loaded and executed by an apparatus or control circuitry comprising one or more processors or processing circuitry, are configured to cause the method of the second aspect or a disclosed embodiment of the second aspect to be performed.

Another aspect of the disclosed technology comprises a computer program product comprising a non-transitory computer readable medium, having thereon machine-executable instructions which, when loaded and executed by an apparatus or control circuitry comprising one or more processors or processing circuitry, are configured to cause the method of the third aspect or a disclosed embodiment of the third aspect to be performed.

Another aspect of the disclosed technology comprises a computer program product comprising a non-transitory computer readable medium, having thereon machine-executable instructions which, when loaded and executed by an apparatus or control circuitry comprising one or more processors or processing circuitry, are configured to cause the method of the fourth aspect or a disclosed embodiment of the fourth aspect to be performed.

In some embodiments, the machine-executable instructions of the above computer-program product aspects comprise one or more computer code modules or computer circuitry configured to implement the methods. Other aspects of the disclosed technology comprise an apparatus comprising means configured to implement any one of the above method aspects. The machine-executable instructions may be provided in the form of one or more modules which correspond to code configured to implement one or more of the elements of each of the method aspects.

In some embodiments, any of the above aspects may additionally have features identical with or corresponding to any of the various features as explained above for any of the other aspects.

Accordingly, in some embodiments, an individualized mental state on-demand service is provided by a server or server system which is configured to receive user registration requests and facilitate user configuration of settings for the on-demand service and associate these with a user account, store the user account information in association with a unique account identifier. At least one data source identifier for at least one data feed comprising brainwave activity, for example, provided by an EEG sensor array worn by that user, and is associated with the user account. Optionally, any other the sources for data feeds related to other sensors associated with the user can also be linked to the account so that the other sensor data can be combined with EEG sensor data for the purposes of assessing the user's current contextual activity and/or for mental state classification purposes.

In some embodiments, the server system is configured to respond to service requests received from a plurality of different third party applications for mental state data for the individual with an indication or indications of the user's mental state which are configured according to the one or more settings the individual has configured for a usage context associated with that application or with their own activity at the time the application is receiving their mental state data. The mental state data may be provided as a mental state classification label and/or as a data feed associated with the mental state classification label.

The server system also includes an Artificial Intelligence, Al, server or server system which has been trained at least in part on that user's individual brain wave activity data to classify features in the brainwave activity based on the user's mental state and/or to apply a classification label to the user's current mental state. The server system forwards each user's brainwave activity data feed to the Al system for classification and then monitors the output from the Al system and optionally stores this and/or one or more current classifications for a user's mental state in order to service any incoming or ongoing service requests. The classification of a user's mental state as determined by the Al system is allows the server system to apply filters based on the user settings to the user's mental state data output by the Al system when responding to service requests from applications which request the user's mental state.

The aspects and embodiments disclosed herein and as set out in the claims may be combined in any suitable manner apparent to someone of ordinary skill in the art.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing will be apparent from the following more particular description of the example embodiments, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the example embodiments.

Figure 1A shows a flowchart illustrating steps in a method for providing mental state data of an individual according to at least some embodiments of the disclosed technology;

Figure IB illustrates a block diagram of a system for providing individualized mental state data on demand according to at least some embodiments of the disclosed technology;

Figure 2 illustrates schematically how the system of Figure IB can be configured to provide mental state data to a plurality of different applications according to at least some embodiments of the disclosed technology;

Figure 3 illustrates schematically how of the system of Figure IB can be configured for multiple users according to at least some embodiments of the disclosed technology.

Figure 4 comprises a flowchart which illustrates how the Al system illustrated in Figures IB to 3 is cross-trained according to at least some embodiments of the disclosed technology;

Figure 5A represents schematically a brain-wave detection device according to at least some embodiments of the disclosed technology;

Figure 5B represents schematically an example of a posterior scalp topography of an individual;

Figure 5C illustrates an example of seven channels of brain wave activity data of the individual;

Figure 5D illustrates an example frequency decomposition of one of the channels providing brain wave activity data shown in Figure 5C;

Figure 5E illustrates an example of data segment of brain wave activity comprising alpha waves;

Figure 6 is a flowchart illustrating a method for classifying the brain activity of the individual according to at least some embodiments of the disclosed technology;

Figure 7 is a sequence diagram illustrating schematically how user settings are configured for an individualized mental state on-demand service accordingto at least some of the disclosed embodiments;

Figure 8 is a sequence diagram illustrating schematically how an Al system is trained to classify a current mental state of a user according to at least some of the disclosed embodiments;

Figure 9 comprises a sequence diagram illustrating schematically how users mental state is remotely monitored using the server system according to at least some of the disclosed embodiments;

Figure 10 comprises a sequence diagram illustrating schematically how an application can access a user's mental state data on demand according to at least some of the disclosed embodiments;

Figure 11-13 schematically illustrates example apparatuses according to some embodiments;

Figure 14 illustrates an example computer readable medium comprising machine-executable instructions according to some embodiments; and

Figure 15 illustrates schematically an example of a machine learning model showing how the cross-training of the Al system shown in Figure 4 is implemented according to some embodiments. . DETAILED DESCRIPTION

Aspects of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. The system and method disclosed herein can, however, be realized in many different forms and should not be construed as being limited to the aspects set forth herein. Like numbers in the drawings refer to like elements throughout.

The terminology used herein is for the purpose of describing particular aspects of the disclosure only, and is not intended to limit the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

Some of the example embodiments presented are directed towards a server or server system providing an individualized mental state data service on demand to third party applications and devices. The server system includes a server associated with an artificial intelligence, Al, system. Various methods performed by the server and the Al system for providing real time mental state of an individual and for training the Al system are also disclosed.

Aspects of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. The apparatus disclosed herein can, however, be realized in many different forms and should not be construed as being limited to the aspects set forth herein. Like numbers in the drawings refer to like elements throughout.

Figure 1A shows a flowchart illustrating an example embodiment of a method 100 for providing mental state data 24 of an individual, e.g. a user A, as an on-demand service to a requesting application shown schematically in Figure IB as application 12, for example, a method comprising a first aspect of the disclosed technology. Figure 1A will be described in more detail with reference to Figure IB of the accompanying drawings, which illustrates schematically an embodiment of a server system 10 configured to implement an embodiment of the method 100 illustrated in Figure 1A.

The sequence of steps of the method 100 may not be necessarily executed in the same order as they are presented in Figure 1A. Further, one or more steps may be grouped together and performed in form of a single step, or one step may have several sub-steps that may be performed in parallel or in sequential manner.

Some embodiments of the method 100 illustrated in Figure 1A provide mental state data for an individual on demand responsive to requests from one or more different applications. As shown in Figure 1A, the method 100 comprises first receiving 102 a service request 14 for mental state data for an individual from a requesting application 12, determining 104 a usage context for the requested mental state data 24, associating 106 the service request 14 with a brainwave activity data feed 20 from the individual, determining 108 mental state data for the user based on the brainwave activity data feed, filtering 110 the determined mental state data based on a user setting, and providing 112 the filtered 110 mental state data to the requesting application 12 based on the determined usage context. Some embodiments of the method may comprise receiving a service request for mental state data for the individual from a requesting application, processing the service request to determine a usage context for the requested mental state data and determining whether the service request can be fulfilled based on at least one user setting for sharing the mental state data of the individual and the determined usage context. Each user setting comprises at least an indication of a mental state classification and an allowed usage context. The method further comprises providing data representing the available at least one mental state classification to the requesting application based on a determination that user settings allow sharing of the mental state data for determined usage context of the service request. If a data feed is being provided to the requesting application 12 and either the usage context or the mental state classification for the individual's mental state changes, the data feed provided to the application 12 may be modified or cease depending on the user settings for the new combination of mental state classification or usage context.

The method 100 may, for example, be implemented by the server 16 shown in Figure IB. By performing the method 100, the server 16 is able to provide the mental state data for the individual as an on-demand service to one or more requesting applications. This further allows a plurality of requests for the same user's mental state data to be concurrently fulfilled by the server 16. The server 16 may perform method 100 for a plurality of different individuals or users.

In some embodiments of the method 100, the associating 106 comprises extracting an identifier from the service request 14 and determining the extracted identifier is associated with a user account of the individual. The user account includes information representing the user setting. Each user account includes one or more conditions for sharing mental state data which are configurable by a user as one or more user settings for that user's user account.

Any suitable form of data storage and/or record facility or registry may be used to store user account information which the server 16 can access. In order to associate a received brain wave activity data feed with a user account, and to associate a request for an individual's mental state data with that individual's user account, in some embodiments, each user account is associated with an account-holder identifier for the individual account holder and/or a source identifier for the source of the brainwave activity data feed of the individual. This allows the method 100 to determine if the extracted identifier is associated with the user account of the individual by determining if the extracted identifier comprises the source identifier or the account-holder identifier which matches the account-holder identifier or a source identifier for the user account.

In some embodiments, the user account is associated with one or more other sensed activity data feeds associated with the individual. For example, an activity watch of a user may provide positioning, speed, and activity data and also heart-rate data. This information may be taken into account by the server 16 when determining the usage context and/or used to help determine a user's mental state in some embodiments. In some embodiments, the user account is associated with one or more other sensed activity data feeds associated with the individual and, for example, the server system 10 is configured so that the one or more other sensed activity data feeds are taken into account by the Al system when classifying the mental state of the individual and/or by the server to determine the usage context.

The user interface may enable a user to configure user settings in a variety of different ways. For example, a condition for sharing mental state data comprises a setting to share or not share mental state data associated with a mental state classification for one or more user selected usage contexts in some embodiments. However, it is also instead or in addition possible for a user account to be configured so that a sharing condition for sharing mental state data comprises a setting to share or not share for a usage context mental state data assigned to one or more user selected mental state classifications.

Based on the sharing conditions for sharing mental state data, the filtering 110 which, for example, the server 16 performs, comprises removing mental state data having a mental state classification for which a sharing condition is not met from the mental state data sent to the requesting application 12. The remaining filtered mental state data is sent to the requesting application 12, in other words, the filtered mental state data comprises mental state data with one or more mental state classifications for which a sharing condition is met according to that individual's user settings.

The shared mental state data may be provided in the form of a label for the individuals' current mental state or in the form of a filtered data feed for the individual's mental state. For example, in some embodiments, the filtered mental state data represents at least one mental state classification 24 for which a sharing condition is met and the shared filtered mental state data comprises an indication of the at least one mental state classification. In some other embodiments, instead or in addition, the filtered mental state data represents at least one mental state classification 24 for which a sharing condition is met and the form of the filtered mental state data comprises a filtered brain activity data feed associated with the at least one mental state classification 24. In some embodiments, the filtered 110 mental state data comprises current mental state data of the individual derived by processing the brainwave activity data feed of the individual in real-time to determine a set of brain-wave activity features. In this way the user's current mental state can be shared in real-time with a plurality of different applications. For example, a user may be driving a car whilst listening to educational audio to learn a new language. A car application may request access to the user's mental state to determine their driving performance and level of focus on the driving activity. A language application may request access to the user's mental state to determine how well they have understood the teaching and/or to assess their level of focus on the educational language learning activity.

The mental state data is associated with a mental state classification or classifications by using an Al system. In some embodiments, mental state data is associated with a mental state classification by using an Al system configured to determine a set of brain-wave activity features for the individual from their brain-wave activity data feed and to associate the determined brain-wave activity features with a mental state classification for the individual. The way the Al system is trained and operates is described in more detail later on.

In some embodiments of the method 100 for providing mental state data, the method 100 comprises receiving 102, for example by the server 16, a service request 14 for mental state data for an individual from a requesting application 12.

The method further comprises processing 104 the service request 14, for example, at the server by using one or more processors or processing circuitry 33 (see Figure 2), to determine a usage context for the requested mental state data 24 and determining 106, for example, using the one or more processors or processing circuitry 32, whether the service request 14 can be fulfilled based on at least one user setting for sharing mental state data of the individual and the determined usage context, for example by checking a user setting stored in a user account for that individual. Each user setting comprises at least an indication of a mental state classification and an allowed usage context.

The method further comprises providing 108 data representing an available at least one mental state classification 24 to the requesting application 12 based on a determination that the user settings allow sharing of the user's current or historic mental state data for a determined usage context associated with the service request. Examples of mental state classifications include, for example, mental states which can be classified based on one or more of an individual's attentional state, emotional state, cognitive state and/or arousal state. An individual's mental state brain activity data feed may include characteristics or features indicative of a combination of two or more of the above states or features or characteristics which are predominantly associable with just one of the above mental state classifications.

Examples of a usage context include a usage context comprises one or more of a task-based context, a geographic environmental context, a situational environmental context, and a physiological context. The usage context may be determined by the server 16 based on one or more of: explicit information provided by the application 12 in the request 14, a usage context stored in the user account association with an identifier for the application 12, one or more inferences based on user activity of the user using one or more of: at least one data feed or a fused plurality of data feeds from one or more sensors associated with the individual at the time the request 14 was received by the server 16. The usage context is often (but not always) dynamic during the lifetime of a data feed.

In some embodiments, method 100 further comprises monitoring the usage context, and adjusting the filtered mental state data provided to the requesting application accordingly so the filtered mental state data is based on the currently determined usage context. For example, an update may be provided to indicate a user's mental state has changed, and the mental state classification label provided if this is permitted by the user settings. If the application is receiving a brainwave activity data feed, then the filter settings will be adjusted based on the user settings for the new usage context to add or remove mental state data from the data feed provided to the requesting application.

In some embodiments, the method 100 further comprises: monitoring 56 the at least one current mental state classification data 24 of the individual for a duration of time; determining that at least one mental state classification has changed to a new mental state classification; and based on at least one a sharing condition indicating the new mental state classification is not to be shared for the usage context, removing the mental state classification data from the mental state data provided 108 to the requesting application 12.

In some embodiments, the method 100 filters the data based on user setting of a user account which comprises at least one condition for performing an action based on a determined mental state classification. Examples of an action include causes a device to vibrate, or play a sound, to act as an alert or an alarm, or to provide information to be audibly or visually displayed to the user, for example to provide some form of feedback, which may, in some embodiments, be used to configure the user's headset. In some examples, the action comprises pushing information associated with the usage context and the current mental state classification to at least one device, wherein the device is configured to perform an action responsive to receiving the pushed information, for example, to move a real or virtual object or to provide control information such as may be used to control a device via a brain computer interface, BCI.

In some examples, the at least one condition for performing an action based on a current mental state classification comprises determining at least one feature of the brain activity data feed meets a threshold triggering the action. Such a threshold may be a context-based threshold, in which different threshold levels are associated with different usage contexts. For example, if the usage context is yoga, and the individual's mental state is "sleepy" the user's activity watch may be caused to gently vibrate just before the individual drifts off to sleep from a state of deep relaxation. If the usage context is driving, a large audible alert may be played on the vehicles sound system, as soon as a very low threshold indicative of drowsiness is detected.

The service request 14 includes in the request an identifier for the individual and/or at least one identifier for a device capable of providing EEG sensor data in order for the server 16 to provide mental state data 24 of the individual to the application 12. In some embodiments, the service request is generated and sent to the server by an application which may be a third-party application that a user has installed and configured to run on an item of user equipment such as a mobile phone or personal computer or the like. In some embodiments, the requesting application 12 may be hosted by a server, including in some embodiments, the server 16, and may run on a user equipment, which user A may be operating. The service request may be received directly from a device hosting the application or be received indirectly via one or more other devices.

At step 104, the service request 14 is processed to determine a usage context for the requested mental state data 24. The usage context may be static or dynamic and if dynamic will relate to the context at the time the EEG data is being supplied. The usage context may be determined by the server 16 in one or more various ways. For example, it can be determined by the server based on explicit information provided by the application in the request, and/or from input by the user stored in association with an entry for the application with their user account settings, and/or be from a registry entry look-up for the application in some embodiments. In other embodiments, instead or in addition, the usage context may comprise a contextual use inferred by the server or by an intermediate device based on user activity of the user. The user activity may be inferred using one or more data feeds or a fused data feed from one or more sensors associated with the user at the time a request is received by an application to access the user's mental state.

As an example, a user may be wearing a sports smart-watch which generates a data feed for the user's heart-rate. If this heart-rate data feed is shared from this device at the time a request is made by an application to access the user's mental state, the intended usage context for the request for mental data state may be a sports-activity usage context. This may be confirmed by usage context information meta-data included in the request. The server 16 may also provide the heart-rate data feed to the Al system to assist in the classification of the user's mental state.

As another example, in some embodiments, the usage context of mental state data may be to learn a new language, which is associated with an application 12 to learn French installed on user equipment. In this case, the application 12 may include meta-data indicating that it is an educational application, any may provide further educational task-specific meta-data to indicate it is a languagelearning application in the request, or responsive to a query from the server 12 having sent the request to provide such meta-data.

In some embodiments, a service request 14 includes meta-data indicating an intended usage context, however, in some embodiments, the service request 14 may provide an indication where the intended usage context can be obtained, for example to a registry entry. In some embodiments, the server 16 repeatedly or continuously infers usage context from one or more usage characteristics associated with the application and/or based on other data feeds from devices associated with the user. In some embodiments, a user may label a particular type of application with a default usage context as part of the settings for their user account.

In this case, server 16 will perform a look-up operation based on a received request which includes an identifier for the application to determine if that identifier has been associated with a default usage in the user settings. In some embodiments, once the server 16 has determined the usage context for an application, this is stored by the server, either as part of a user account or in a central database to facilitate subsequent usage context determinations when the same application or type of application requests access to the mental state data of that user or of other users as appropriate. In this manner, the application which requests access to a user's mental state which was inferred to be sports related due to the heart-rate data feed from one user's sports watch may at first be considered sports-related with a particular confidence score. If the same type of application was associated with a number of different users, the confidence score of this usage context may be increased.

In some embodiments, a usage context may be inferred and/or a confidence score in the usage context increased by inference by the Al system based on the user's brainwave activity being consistent with a server determined or meta-data indicated usage context. At step 106, it is determined whether the service request 14 can be fulfilled based on a user setting 26 for sharing mental state data of the user A and the determined usage context. The user settings are associated with an individual's account, which is associated with one or more EEG data feeds. The service request 14 may include an identifier for the user whose mental state data is to be obtained by and/or an identifier for a brainwave activity device and/or an identifier for a data feed comprising brainwave activity associated with the user. One or more of these identifiers is provided in the request or is provided later on to the server 16 in order for the server 16 to match to a corresponding identifier for a user account in order to locate the correct user account and mental state information to share.

Any suitable data structure may be used as a record to store information for each individual's user settings and account information. The data may be stored in a database on or accessible by the server 16 or in a distributed form. In some embodiments, each user's account is stored on a registry server or servers which may be distributed and/or cloud-based for example, which the server accesses by generating a request for user account information responsive to receiving a request from application 12.

Each user account accordingly comprises at least an account identifier and information which enables an application to obtain mental state information as requested for an individual. For example, each user account may in some embodiments include an identifier for the brain wave activity data source (for example, an identifier such as a Media Access Control identifier or some other form of network address for the user's headset or similar device) which can be stored in that user's account settings. A user account may also include one or more identifiers for other equipment, which may provide sensor related data used by the server to determine a usage context or by the Al server to determine a mental state classification in conjunction with the brainwave activity data of the individual.

Each user account comprises one or more user settings which the server uses to configure what mental state data is shared and under what circumstances with a requesting application. These user settings 26 each comprise an indication of at least one mental state classification and at least one allowed usage context for each of the at least one mental state classifications. The user settings which are stored in a user's account accordingly comprises usage context preferences or controls or configurations for sharing mental state data 24 with one or more applications 12. In some embodiments, at least one of the applications 12 is a user-indicated application, being a named application in that user's account. However, in some embodiments, application 12 may not be previous associated with a particular user's account.

Examples of a mental state classification 22 include a classification of one or more of an attentional state, an emotional state, a cognitive state and/or an arousal state, among others. Examples of an attentional state include task-focused, distracted etc. Examples of an emotional state include happy, sad, bored, relaxed, angry etc. Examples of a cognitive state include undecided, alert etc.; and finally, examples of a user's physiological state include "pre-race", "stressed" etc.

As mentioned above, in some embodiments, the server receives a request from an application 12 which has not previously requested access to a particular user's mental state data.

In order to determine whether a service request from an unknown application can be fulfilled on-demand, in other words, in real-time, the server 16 first checks what the user's mental state currently is, and then determines if this can be shared with the requesting application 12 by determining the usage context of the user's mental state data (although as someone of ordinary skill in the art will appreciate, these two conditions can be checked concurrently rather than sequentially, and it is possible also to check the usage context first and then the user's current mental state). The usage context can be determined by the server 16 based on information provided by the application 12 itself, for example as meta-data in the request, by querying another server (not shown) which contains meta-data describing the application, and/or by inferring a context of use based on one or more sensors also associated with the user indicating a particular usage context is likely.

At step 108 data representing an available mental state classification 22 is provided to the requesting application 12 based on a determination that user settings 26 allow sharing of mental state data 24 for determined usage context of the service request 14. The data representing the available mental state classification 22 can be, but is not limited to, an indication such as a label for the mental state classification 22 and/or in some embodiments, comprises a filtered data stream of brainwave activity received from the Al system. The filtered data stream comprises a sequence of permitted mental state labels in some embodiments.

The steps 102-108 will be described in detail with examples in association with system components with reference to Figure IB and Figure 2.

Figure IB illustrates a block diagram of the server system 10 for providing mental state data 24 of an individual, e.g. user A. The mental state data 24 of user A is sent to the application 12 which may be located on a different device such as an item of user equipment, a third-party device or server, or hosted by the server 16. The user A may be operating a user equipment (user equipment 44 shown in Figure 7) in some embodiments and may want to access the application 12 on that user equipment. The user equipment may include a mobile phone, a vehicle navigation and infotainment device, a personal computer, health or medical condition monitoring device or any similar electronic devices capable of processing and communicating data with one or more servers via one or more communication networks.

The terms "individual" and "user" will be used interchangeably throughout the description where they refer to the same entity.

In Figure IB, the server system 10 can be a client-side brainwave detection system 10 that comprises a wearable device 8 (shown as headset #A for user #A). The wearable device 8 may, as an example, be a headset (for e.g., intracranial electroencephalography or any other related EEG measurement tool(s)) wearable around the individual's scalp. Various forms of headset or wearable device are possible, for example, in some embodiments the headset may be part of a helmet, a set of headphones, goggles or glasses, or as a hat, or a tattoo. In some embodiments, the headset or wearable device 8 may have another purpose, such as functioning as a safety device or as a near-eye display. The term headset or wearable device are used equivalently herein, and a headset may in some example embodiments comprise a device capable of providing intracranial EEG recordings.

Figure IB also shows the server 16 and an artificial intelligence, Al, system 28 on the right-hand side receiving the brain-wave data. The server 16 is configured to make brainwave data for one or more individuals including user #A available as an on-demand service to one or more requesting applications such as the application 12 shown in Figure IB (also shown as applications 12A, 12B in Figure 2).

In some embodiments, the wearable device 8 comprises a control circuitry 2, an input/output, I/O, unit 4, a transmitter/receiver, Tx/Rx, unit 6, a memory 11, and one or more electroencephalogram, EEG, sensors 18. Also shown in Figure IB are is an optional other device 9, which may receive the brainwave activity of the user for example over a short-range wireless communications link and which may then use another, in some embodiments, different type of communications link to transmit the brainwave activity to remote server 16. Application 12 is hosted on a device 13, which may comprise a communications-enabled device such as a mobile phone or a remote server device. As shown schematically in Figure IB, the host device comprises a memory 11, a data communications transceiver, Tx/Rx 6, a data interface shown as I/O 4, at least one processor or processing circuitry shown as processor(s) 2, and memory 11. In some embodiments, the host device 13 of application 12 may comprise the other device 9, and the other device 9 may comprise a device associated with the user or with another user. In some embodiments, the host device 13 could also comprise the server 16. The host device 13 of the requesting application 12 may communicate using a suitable communications protocol over wired and/or wireless communications channels with the server 12 via the Tx/Rx 6 and/or data interface 4.

The EEG sensors 18 included in the wearable device 8 are in some embodiments provided such that they are capable of being in contact with the scalp of the user A (e.g. via electrodes) such that electrical-signals from the brain (e.g., cerebral cortex) of the individual can be detected using the EEG sensors 18. In some embodiments, the electrodes may penetrate the skin surface and be invasive, enabling invasive EEG measurements to be provided. In some embodiments, the electrodes may be printed on the user's scalp and the wearable device 8 comprise the printed array of electrodes on the user's skin. In some embodiments, intracranial EEG, iEEG, and other techniques such as electrocorticography, ECoG, using subdural grid electrodes, and stereotactic EEG, sEEG, are supported by using depth electrodes.

The EEG sensors 18 sense the brainwave activity of user A and generate raw electrical signals. In some embodiments, these may be locally processed by the processor or processing circuitry 2 of the wearable device 8, for example, to locally process the signals and/or enhance or suppress signal at one or more wavelengths. The processing results in brain wave data 20 comprising bio-signals for a sensed mental state or states of the user A. The brain wave data 20 comprises data associated with a voltage and frequency of electrical activity from neurons in a cerebral cortex of the user A wearing the headset wearable device 8.

As discussed above, the EEG data 20 is provided to the server 16 according to the disclosed technology so that the user A mental state can be made available as the on-demand service to applications such as the application 12. However, instead of simply sharing all of the raw EEG data 20 with the application 12, the server 16 provides one or more indications such as a label for one or more classifications of the user's current mental state(s) in some embodiments. In other embodiments, a filtered subset of the raw EEG data 20 which the Al system 28 has associated with one or more classifications of the mental state of the user A. What mental states are made available as a service via the server 16 depends on the application's intended use of the mental state, what user settings have been established to share one or more mental state classifications for that usage context.

This mental state usage context may be provided by the application 12 and/or determined by the server 16 based on certain indications associated with the application 12, for example, any metadata describing the function of the application 12 or in some case the server 16 may infer the usage context. For example, if the server 16 determines that the application 12 associated with one or more sensor devices such as a GPS sensor and/or a heart-rate sensor, the server may determine a usage context of fitness. The usage context, i.e. the context of use of the EEG data 20 and the user's own settings for what mental states are to be shared for particular contexts of use are store by the server 16 and are checked every time a service request is received from the application 12.

Once the usage context associated with the service request has been determined, the server 16 then checks what particular classifications of the user's mental state are indicated in that user's settings to be shared for that usage context and/or with that requesting application 12. Providing the current mental state of the user matches a classification for which the indicated usage context permits sharing with the requesting application 12, the user's current mental state data is shared. The form of mental state data which is shared comprises a label for the user's current mental state classification in some embodiments. In other embodiments, a data feed comprising all or part of the EEG data 20 which the Al system 28 has determined to be associated with a particular mental state classification is shared. What form of mental state data is to be shared may differ for different usage contexts and/or be configured as a user setting.

In order to provide mental state data responsive to the received service request, the raw EEG data 20 which the server receives from the user is forwarded to the Al system 28 which processes it and provides the user's current mental state data to the server 16 so this can be stored and/or streamed or indicated to any requesting application 12.

In order to analyzing the mental state of user A, in some embodiments, other sensor information for user A is captured. For example, background activities by the user such as their heart rate, their running or walking speed, the time since they last stood up, sat down, or otherwise changed their pose and/or body position, their gaze and/or pupil size, and what other types of activity they are engaged are from suitable sensors, for example, inertial measurement sensors, accelerometers, and/or global positioning sensors and/or gaze or eye tracking optical sensors. It is also possible to infer some usage context related data just based on the context of a newly opened application or other content displayed on a screen of a device associated with the user if this information is remotely determinable.

To effectively analyze the background activity of the user A, the recorded EEG data 20 is communicated to the server 16 located remotely to the wearable device 8. The EEG data 20 may be communicated with the server 16 via any suitable communication network (not shown), described in detail below with reference to Figure 2 using the RX/TX unit 6.

In some embodiments, the recorded EEG data 20 may also be communicated to user equipment of the user A, for example to a smartphone or personal computer. The user equipment may then transmit the received recorded EEG data 20 to the server 16 via another suitable communication network, which may be similar to or different from the communication network discussed above.

The server 16 may be a remote virtual server such as a cloud server or an application server hosted by a third party entity. The server 16 provides an addressable data interface, such as a web-page universal resource locator, port, or other suitable access point for an application hosted on remote device to use to access the mental state service on demand. In some embodiments, the application 12, is capable of running on user equipment. The application 12 may be a mobile application or a web page that can be accessed on the user equipment and interacted with using I/O units of the user equipment. An example of an application 12 is a driver assistance application which may be hosted on a vehicle being driven by the user. Another example of application 12 is a teaching application which the user is using to learn to drive the vehicle in some embodiments. More than one application can concurrently request mental state data of the same individual in some embodiments from the server 16.

The application 12, may, for example be a document creating application, a platform for learning soft skills, an infotainment content browsing and viewing application, a driving assistance application, a multimedia player application, a gaming application, a chat application or even a phone calling application, to name a few examples.

In some other embodiments, the server 16 hosts the application 12 and also stores information corresponding to the application 12 in a database (not shown). The application 12 may be the requesting application that requests mental state data of the user A.

In some other embodiments, the server 16 does not host the application 12. Another server (not shown) may host the application 12 and the server 16 may be a dedicated server configured only to perform one or more steps required to provide mental state data of the user A to the application 12 by communicating with the server that hosts the application 12.

In some embodiments, a plurality of different hosts, such as different devices or user equipment, servers or other apparatus each host an application 12. In some embodiments, a server or other apparatus may host a web-based application 12 that is accessed via user equipment.

Although various client-side and server-side architectures are possible, an embodiment will be described where the client-side system 10 comprises a user equipment which accesses setting information via a server 16. The server 16 hosts a registration or setting application which allows the user to access and configure. The registration or setting application is accessed via and/or executed on the user equipment, a headset is also provided and one or more sensors for other user activities, or situational or environmental conditions may also be available in some embodiments.

A suitable data store associated with the server 16, such as, for example, a database, is used to store user account information. Each user account is associated with one or more user settings 26. The server 16 is configured to provide a suitable user interface to each user so they can configure the one or more user settings 26 on to the user equipment. In some embodiments, the user equipment may be the same equipment as the equipment or apparatus on which the application 12 runs.

In some embodiments, the one or more user settings 26 comprise settings for an indicated application such as the application 12. In some embodiments, the one or more user settings 26 comprises settings indicating for a menu of possible classifications of mental states what usage contexts the states are allowed to be shared. In some embodiments, at least one user setting indicates at least one condition for performing at least one action based on a current mental state. In some embodiments, the one or more user settings 26 include default settings indicating the default conditions for performing the at least one action based on the current mental state.

In some embodiments of the disclosed technology, the server 16 and the Al system 28 are separate, in other words, the Al system may be remote from server 16 meaning that the server 16 communicates the EEG data 20 to the Al system 28 using a suitable communication network, as discussed above. The Al system 28 may be a part of the server 16 in some embodiments. In some embodiments, the Al system 28 may be a standalone system communicating with the server 16.

The Al system 28 comprises an Al model which is trained in two stages. The first stage is trained specifically for the individual user based on a training routine where the user shares their EEG data 20 and information about their mental state and any background activity they are engaged in. The second training stage is a cross-training or transfer learning stage in which the Al system 28 trained on the individual user's EEG data 20 is then cross trained using a dataset that includes historical recorded EEG data 20 of various users (user B, user C... user N) that the Al was trained to classify.

In some embodiments, the recorded EEG data 20 corresponding to various previous users of the mental state as a service is used to cross train the Al system 28 and further to classify mental state data of the new user A. The classified mental state data of the user A along with a mental state classification 22 is communicated to the server 16.

Thus, the Al system 28 can be used to classify the mental state data 24 of the user A in a manner that allows a user to control what mental state data is provided in order to fulfil the service request generated by the requesting application 12. The use of a cross-trained Al system in some embodiments allows a faster and more accurate classification of a live stream of EEG data from the user, which enables server 16 to provide a live individualized mental state data on-demand service to a requesting application 12.

Further, the present disclosure aims at efficiently training the Al system 28 by collecting and utilizing the vast amount of EEG data 20 from multiple individuals and enabling the cross-referenced training to increase the learning and improve accuracy of the mental state classification 22. This is explained in conjunction with Figure 4.

In some aspects, the user settings 26 can be updated using the mental state classification 22 communicated by the Al system 28. The mental state classification 22 may be presented to the user via the user equipment accessing the application 12. In some aspects, the user settings 26 may be triggered during the initial set-up of the application 12 or may be presented while placing the request for providing the mental state data of the user using the application 12.

Figure 2 illustrates a block diagram showing a more detailed example of the server system 10 for providing mental state data 24 of an individual on demand. In Figure 2, EEG sensors 18A of user, A, are configured to send information to the server 16. User A may, in some examples such as that illustrated in Figure 2, being interacting with one or more applications (such as the application 12A, and 12B) and is wearing a headset such as the headset 8 shown in Figure IB. The server 16 which receives the user's EEG data 20 is configured to perform the various steps as described in reference with Figure 1A to provide each requesting application 12A, 12B with shared mental state data for user A based on the user settings 26 for each application's determined usage context.

The client-side system 10 comprises the one or more EEG sensors 18A and one or more additional sensors/other sensors 32, for example, sensors 32A for user#A. As already described earlier, the EEG sensors 18A (similar to EEG sensor 18) records brainwaves of user A and transmits the EEG data 20 to the server 16. The one or more other sensors 32A may include sensors such as a heart-rate sensor, a blood oxygen sensor, a skin-temperature sensor, a gaze-tracking sensor and a hydration sensor, among others. The sensors 32A may be built in sensors implemented in the user equipment. Additionally, the sensors 32A may be implemented in available activity/fitness tracker devices such as for example, a smart watch, that user A may be wearing.

The server system 10 further comprises the server 16 and the Al system 28 configured to assist the server 16 by classifying the mental state data 24 of the user A. The server 16 may be a remote virtual server such as a cloud server. In some aspects, the server 16 may host applications 12A and 12B and obtain context of applications 12A and 12B. The applications 12A and 12B may be recited at the user equipment and is accessed by the user A.

The application 12A is associated with a usage context #1, for example a task-based usage context, and may be referred to as context #1 application 12A. Likewise, the application 12B may represent another usage context, for example, usage context #2 which may comprise, for example, a physiological context. Application #2 may be referred to as a context #2 application 12B.

Also shown in Figure 2 are additional sensors 32A which are configured to record additional bio signals 30 from the user #A. Examples of additional bio-signals include blood pressure, heart rate, skin temperature, and blood oxygen level, of the user A, which may enable accurate classification of mental state data of the user A at the Al system 28. As an example, an elevated heart rate may indicate that the user A may be excited and therefore a possible mental state data of the user A may be an arousal state.

The EEG sensors 18A and the other sensors 32A may directly communicate with the server 16 via a communication network (not shown) for transmission of the EEG data 20 and any bio-signals 30 to the server 16. In some embodiments, the EEG sensors 18A may transmit the EEG data 20 to the user equipment via the communication network. The user equipment may then transmit the EEG data 20 received from the EEG sensors 18A to the server 16. Further, in cases where the other sensors 32A are implemented as part of the user equipment, the server 16 receives the recorded bio-signals 30 from the user equipment. In cases where the other sensors 32A are implemented in the activity/fitness tracker devices, the bio-signals 30 are directly transmitted to the server 16.

Any suitable embodiments of a communication network can be used to convey the EEG sensor data 20 to the remote server 16. For example, in some embodiments, the communication is via a wireless or cellular communication network. Accordingly, in some example embodiments, to facilitate communication with the server 16, the EEG sensors 18 are equipped with a wireless module such as a Subscriber Identity Module, SIM. The SIM module may also facilitate receiving signals from the server 16. The SIM module is in communication with the Tx/Rx unit 6 via a communication interface.

Examples of the communication network include any type of wireless communication network including cellular-based communication protocols such as code division multiple access, CDMA, time division multiple access, TDMA, Global System for Mobile communications, GSM, Integrated Digital Enhanced Network, iDEN, General Packet Radio Service, GPRS, Enhanced Data rates for GSM Evolution, EDGE, Universal Mobile Telecommunications System, UMTS, Wideband Code Division Multiple Access, WCDMA, and variants, second generation, 2G, third generation, 3G, fourth generation, 4G, long term evolution, LTE, and fifth generation, 5G, and later generations, for example, sixth generation, 6G, wireless and broadcast communication network communications standards. In various embodiments, network link may further include, or alternately include, a variety of communication channels and networks such as Wireless Local Area Network, WLAN/Wireless Fidelity, Wi-Fi, WiMAX, Wide Area Networks, WANs, and also ad-hoc and shorter range network communications, for example, Bluetooth.

The server 16 may comprise a memory 36 and a processor 32. The memory 36 may be configured to provide one or more suitable data stores or databases for the user settings and/or any pushed EEG data. The one or more databases may store information corresponding to the context 1 application 12A and the context 2 application 12B. The databases may also store EEG sensor data 20, other sensors data 30 and the mental state data 24 of an individual.

The processor 33 is configured to execute instructions that may be stored in, for example, but not limited to, the memory 36. The processor 33 is configured to associate the EEG sensor data 20, other sensors data 30 and the mental state data 24 of an individual with a user account of the individual, which will be discussed later with reference to Figure 7. The processor 33 is configured to perform the steps 102-108 described above in reference with Figure 1A.

Figure 2 shows schematically how the server may receive a number of different service requests 14A, 14B from different applications 12A, 12B, either sequentially or concurrently in time. As illustrated in Figure 2, just two applications 12A, 12B are shown, each sending a service request 14A, 14B. The service requests 14A and 14B may comprise requests to provide the same or different mental state data 24A and 24B for the user A, however, to better illustrate, different mental state data will be assumed to be requested in the following embodiment.

The mental state data 24A, for example, may be associated with a usage context by application 12A and mental state data 24B, for example, may be associated with a usage context of the different application 12B. For example, a usage context of application 12A, which may be an infotainment application, would be to determine user engagement with the presented context. The user A could watch news or other infotainment content using the application 12A with an attentive state of mind if they found it interesting but may be distracted and not engage fully with the content if it was boring. The mental state, therefore, in this case may be an attentive mental state, as an example. The application 12A may request from the server 16 access to the user's mental state. If the user has indicated that for the usage context "content viewing" they are not happy to share their mental state of engagement in their account setting however, application 12A would not have access to their mental state. Application 12B, however, could be a driving assistance application in the user's car. A user should be paying attention when they are driving, and the same mental state information "attentive/distracted" may be requested by application 12B. In this case, for the usage context "driving" the user is happy to share their mental state information with the application 12B, and this is accordingly shared by the server. However, if the same application 12B, requested the user's mental state and the user's current mental state was instead "angry", this would not be shared by the server unless the user's settings indicated that they were also happy to share their emotional state with the driving application 12B.

A default usage context for the mental state data may be presented by an application 12 requesting the mental state, but in some embodiments, the server 16 will determine the usage context from other sensors associated with the user. This allows more sophisticated sharing to be performed as the same application may be allowed or denied access to a user's mental state based on a presumed usage derived from the user's activity. For example, if the user has a heart rate monitor and a sports GPS watch monitor configured to provide a sensor stream which is sent or otherwise shared or accessible to the server 16, the server 16 can deduce the user is about to engage in a sports activity, and if a request is received from an application, even if the application was not related to a sports activity, the server could assign the application request to a sports usage context.

In some embodiments, the server 16 determines the usage context for the requested mental state data 24A, 24B based on various sensed data. Examples of usage contexts, including usage contexts that can be determined by the server 16 using information obtained from other sources, include taskbased usage contexts, geographic environmental usage contexts, situational environmental contexts and physiological usage contexts. An example of a task-based usage context is an educational activity usage context. Examples of educational activities include learning a language, driving a car, learning to bake, doing yoga. These may be deduced in part from the EEG data 20 and/or other data sources, including the application 12 requesting the mental state data.

For example, if a "learn-a-foreign-language" application 12 requests access to a user's mental state data, the server 16 can deduce the usage context will be educational, and in some embodiments, deduce more specifically, learning a language. Other examples of task usage context may be driving a car, viewing content, writing a letter, or playing a game, cooking a meal, or any other form of manual task requiring mental and/or physical attention by the user to complete.

Examples of a geographic environmental context include a geographic location, which may be relative to a particular place, such as how far the user is located away from the location they have set as their home, which could be determined, for example, by a user's GPS location. A geographic environment could also include certain places, and may include shops, and other commercial establishments, or countryside location, which may be derived from map-data.

Examples of a situational environmental context comprise a situation such as a hospital waiting room situation, a pre-surgery situation, a classroom activity situation, where a particular location is associated with a particular situational context. Examples of a physiological context include specific relaxation or stress related contexts such as a "pre-race" context, for when the user is about to run a race, undergo a surgery, and may also in some embodiments include medical conditions which are long term such as a dementia related usage context

The server 16, for example, processor or processing circuitry 32, is configured to determine whether the service requests 14A, 14B can be fulfilled based on one or more user settings 26 for sharing mental state classification data 24A, 24B and a determined usage context for each mental state classification. This setting information is captured from the user and stored in a suitable user account format which comprises a unique user identifier, at least one identifier for the brainwave activity data source(s), i.e. the EEG sensor data 20, for that user, zero or more identifiers for any other data source(s) associated with the user, such as a heartrate monitor, GPS device, etc., and that user's mental state sharing settings.

The information may be configured in an initial setting configuration as part of the user opening an account and registering for the service, and it may be also updated later, for example, responsive to a prompt from the server that a new usage context has been determined for the first time. Such a prompt which requests the user to configure what mental state classifications are allowed for the new usage context may be generated when a new application 12 first sends in a service request to the server 16.

The user interface for the account settings may be configured so that a menu of usage context classifications is presented, where each usage context can be assigned one or more allowed mental state classifications for which the user's mental state data can be shared by that user. Alternatively, or in addition, the user interface may present a menu of mental state classifications, where each mental state classification is assigned one or more allowed usage contexts by the user. Additional user settings may also be presented such as an alert etc.

User setting 26 may encompass user preferences for sharing mental state data 24A, 24B with a specific application, such as applications 12A and 12B shown in Figure 2.

In some embodiments, the server 16 causes a user interface, Ul, to be presented to user equipment of user A responsive to a request from the user equipment to configure and personalize user settings. The Ul may further enable the user to select setting indicating what category of mental state data the user wishes to share and what type of data, the user wishes not to share with the application. The Ul can include options (e.g. "yes/no" button) to provide user settings 26. Additionally, user settings can be provided in form of text. Further, user settings 26 can be selected from a drop down menu, etc. Based on user settings 26, mental state data may or may not be provided to the application.

The processor 33 of the server 16 provides data representing an available mental state classification 22 to the requesting applications 12A, 12B based on a determination that user settings 26 allow sharing of mental state data 24A, 24B for determined usage context of the service requests 14A, 14B. The data representing the available mental state classification 22 can be, but not limited to, an indication of the mental state classification 22. For example, additional sensor information such as the user's heart rate etc. may also be shared with the requesting application if this is permitted by the user settings.

To ensure only user designated mental states are shared with the requesting application 12 for certain usage contexts, the server 16, in other words processor or processing circuitry 33 is configured to monitor the mental state classification data generated by the Al system 28 and applies a filter 34 based on the user settings to remove or cease to provide, a current mental state data, if the usage context changes to one where the user's current mental state is not allowed to be shared. A usage context may change either because the user has indicated this, because the server 16 has deduced this from an aggregation of sensor data that forms a user's activity feed, and/or because the application 12 itself has notified the server of a change of usage context.

The user's mental state may also change whilst the usage context remains stable. In either case, the processor 33 applies the filter 34 configured to filter mental state data 24A, 24B of the user A based on the user settings 26 and usage context and provides only filtered mental state data 24A to the requesting application 12A and filtered mental state data 24B to the requesting application 12B. The filtered mental state data may comprise a data feed or a classification label, the latter being updated if the mental state of the user changes.

The filter 34, when applied, allows sharing of only selected mental state data to the applications 12A, 12B. For example, a user A can provide user settings 26 to not share emotional state data and only to share cognitive state data while engaging in a task via the applications 12A, 12B. The processor 33 can then apply the filter 34 to the emotional state data feed so that the emotional state data is not shared.

In an example, the application 12A may be a language learning application, where a language instructor offers classes to applicants enrolled for the classes. The application 12A requests the user's mental state data. The usage context in this case may be included in the request, for example, application 12A may request mental state data for an educational usage context task-based on learning a language. The User A indicate they are happy to share their focus when learning a language. If the user A becomes distracted during a class, the mental state data captured by the EEG sensors 18A and 32A is shared via the server 16 with Al system 28 and classifies the mental state of that user as distracted. This could allow the instructor of that class to be aware of the distracted state of user A. The user A could also choose to share a cognitive state reflecting their mental understanding for such a usage context as well. This mental state data could allow the class instructor to see that they now fully understand that teaching point, and move on to another. Another user, not shown in Figure 2, could choose to not share an attention state (mental state) with the application 12A as the user B does not want the language instructor to know that he/she is not being attentive but is being distracted during the class. Therefore, the user B provides appropriate user settings 26 to the attention state but may still share cognitive state and/or an emotional state. Based on the received user setting and the usage context for user B, that user's attention state data is not shared with their version of application 12A.

In another example embodiment, another application 12B is an application to view entertainment content. The application 12B presents a horror movie or perhaps a sports program. The heart rate of the user A will be elevated in both contexts. The other sensors 32A captures signals corresponding to an elevated heart rate for user A. The EEG sensors 18A captures the brain waves. The user A may choose to not share an emotional state with the application 12B as the user A does not want the application 12B to know that he/she is scared. Therefore, the user A provides appropriate user settings 26 to the emotional state data but may choose to share an attention state data in this case.

Based on the received user setting and the usage context, the emotional state is not shared with the context 2 application 12B. The user's responds when watching the sports program may be the same and/or may be very different, but the user may decide that they are happy to share their emotional state when watching the sports program. The usage context in this case may be more granular than just the context of use that the application indicates, a current state of operation of the application may be relevant to determine the context of use, in other words, not just that content will be presented to the user, but the type of content presented may be categorized and a user may configure their user settings accordingly. This allows, for example, a user to change channel using the application 12B from the horror movie to the sports program, and the server 16 will automatically stop filtering the user's emotional mental state responsive to detecting the change of usage context.

The data representing the available mental state data 24A, 24B comprises filtered brain activity data associated with the available mental state classification 22. The data representing the mental state classification 22 comprises a current mental state of the user A derived by processing a set of characteristics detected in current sensed brain activity of the individual to associate the detected set of characteristics with a set of features previously associated with sensed brain activity of the individual for that mental state classification 22.

The processor 33 may include one or more processing units (e.g., in a multi-core configuration). The processor 33 is operatively coupled to a communication interface such that processor 33 is capable of communicating with the headset, the user equipment and the Al system 28. The processor 33 may also be operatively coupled to a database comprised in the memory 36.

The data-store comprises a database in some embodiments which may be any computeroperated hardware suitable for storing and/or retrieving data. The database stores the EEG sensor data/brainwave data 20 and the mental state classification 22. The database may include multiple storage units such as hard disks and/or solid-state disks, for example arranged in a redundant array of inexpensive disks, RAID, configuration. Distributed or cloud based storage may also be used by the database in some embodiments, for example, the data base may be hosted on a storage area network, SAN, and/or a network attached storage, NAS, system. In some embodiments, the database is physically hosted on one or more: magnetic storage devices (such as hard disk drives, floppy disks, magnetic tapes, etc.), optical magnetic storage devices (e.g., magneto-optical disks), and semiconductor memories (such as mask ROM, PROM, programmable ROM, EPROM, erasable PROM, Phase-change memory, flash ROM, and RAM, random access memory), etc.

In some embodiments, the database may be accessed by the processor or processing circuitry 33 using a storage interface. The storage interface may include, for example, an Advanced Technology Attachment, ATA, adapter, a Serial ATA, SATA, adapter, a Small Computer System Interface, SCSI, adapter, a RAID controller, a SAN adapter, a network adapter, and/or any suitable component providing the processor or processing circuitry 33 with access to the content of the database.

It may be noted that the server 16 may include fewer or more components than those depicted in FIG. 2. As explained above, the server 16 may be included within or embody an electronic device. Moreover, the server 16 may be implemented as a centralized system, or, alternatively, the various components of server 16 may be deployed in a distributed manner while being operatively coupled to each other.

Figure 3 is a block diagram of the server system 10 for providing an individualized mental state of the individual on demand which shows how the server system 10 may be trained for each individual, of which just two, user#A and user#B are shown for clarity. Figure 3 shows schematically how server 16 can be set up to provide individualized mental state services for two users A and B, wherein user A's domain is shown on the left-hand side and user B's domain is shown in the right-hand side.

In some embodiments, for each user, the Al server 28 provides one or feedback loops, 38, 39, which are shown schematically in Figure 3 as feedback loops 38A, 39A for user A and 38B, 39B for user B. The first optional feedback loop 38 presents a user with one or more external stimuli so that their mental state response to each one or a combination of the stimuli can be assessed by the Al system 28. The second optional feedback loop 39 provides feedback based on the characteristics of the EEG sensor data 20 received by the Al system 28. The second feedback is provided directly to the user's EEG headset device 8, for example, to the processor 2 of the headset, which processes the feedback to configure the sensors or sensor array to fine-tune the EEG data 20 sent to the Al system 28, via server 16, based on the feedback received. For example, feedback 39 may be used to configure the sensor array to enhance the signal quality of the EEG data 20 at certain frequencies, either by controlling the sensitivity of the EEG sensors 18 detecting the brainwave activity or the by controlling the processing of the sensor data prior to its transmission as the sensor data 20 to the server 16.

Also shown in Figure 3 are the EEG sensors 18A, other sensors 32A and the processor 2A of user A's headset 8 which includes the EEG sensors 18A (other headset components are not shown in Figure 3 for clarity). On the right-hand side of the figure, the EEG sensors 18B, other sensors 32B and the processor 2B of user B's headset which includes the EEG sensors 18B are shown schematically (other components of user B's headset are not shown in Figure 3 for clarity). Sensors 18A, 18B may be each referred to herein or shown in the drawings as sensors 18 and sensors 32A, 32B may also be referred to individually or shown in the drawings as sensors 32.

The Al system 28 is configured to receive sensor data 20, 30 from the server 16 and classify the mental state of each individual. In some embodiments, by receiving, for example, sensor data 20A, 30A from user A's domain and sensor data 20B, 30B from user B's domain, the Al system 28 is able to train each user's equipment in some embodiments and is first trained on the calibrated sensor data from each user. The Al system 28 is configured to store the mental state data for each individual user in the database of the server 16 comprising the sensor data 20, 30 of the multiple users. The stored sensor data 20 and 30 for all users can be collectively utilized for cross-training the Al system in some embodiments but is also used by the server 16 to provided classifications of each individual user's mental state.

In order to classify each user's mental state, the Al system 28 is trained to recognize features in the EEG sensor data. In embodiments where the Al system 28 is configured to provide the first feedback loop 38 and the second feedback loop 39, the features may be determined based in part on the feedback process.

For example, in some aspects, the first feedback loop 38 provides an external stimulus 40 to the user A and the second feedback loop provides feedback 39 for training and/or calibration of the headset sensors 18A of user A.

In some embodiments of the first feedback loop, the Al system 28 is configured to output the current mental state classification 22 as an external stimulus 40 to the user A. The external stimulus 40 can be provided, for example, to a device or display in the proximity of or attached to the headset 8 (i.e. comprising the EEG sensors 18) worn by the user A and/or the user equipment of user A. In some aspects, the one or more external stimuli 40 are the one or more stimuli provided to the user A by a display system (e.g. in the form of images, graphical objects, data objects, and/or sounds presented to the user A for purposes other than providing the mental state data). In some other aspects, the external stimulus 40 can be presented in the form of augmented reality objects using any augmented output device. Advantageously, the Al system 28 is configured to passively collect data about the user A (using any display/audio system) in order to classify the mental state data of the user A. Further, the data can also be used to cross-train Al system 28 using the features from other user i.e., user B for classifying the mental state.

In the second feedback loop 39 shown in Figure 3, the trained Al system 28 may find sets of features for classifying mental states of the individual. The Al system 28 uses the sets of features to further enhance the the data received from the EEG sensors 18A and the other sensors 32A, for example by providing indications or configuration feedback in order to improve the sensitivity of the Al system 28for identifying any changes of the current mental state. Based on an indication that identifies a change in the current mental state, the Al system 28 determines and/or monitors a threshold for triggering an action, as detailed below.

The server 16 in association with the Al system 28 monitors the current mental state classification 22 of the individual for a duration of time and may in some embodiments store this information. The server 16 determines if the mental state classification 22 has changed. While monitoring, if it is determined that the current mental state has changed, the user setting indicates that the changed mental state classification is available for the usage context. Based on this indication, the server 16 provides the data representing the updated mental state classification 22 to the requesting application 12. The provided data comprises a stream of mental state classification data.

In some embodiments, the user settings 26 include at least one condition for performing an action based on the current mental state classification 22. The action comprises, for example, pushing information associated with the allowed usage context and the current mental state classification to at least one device such as the server 16 or the user equipment, wherein the device is configured to perform an action responsive to receiving the pushed information. In some embodiments, the action responsive to receiving the pushed information comprises applying filter 34 when sharing the current mental state data. In another example, the action is performed by a device in the user domain, for example, a vehicle may generate an audible alert is a user's mental state changes to indicate the user is experiencing drowsiness or another mental state indicative of driver distraction.

In some embodiments, the at least one condition for performing the action based on the current mental state classification 22 comprises determining at least one feature associated with current brain activity characterizing a current mental state meets a threshold triggering the action.

The threshold is also a context-based threshold, which is associated with a particular usage context. For example, in a usage context, such as, playing a game on mobile or watching entertainment content, on a computer, etc., detecting that an individual's mental state is associated with drowsiness is acceptable and may be well within a set threshold. As such, the set threshold does not trigger any action in this case. In another example, when the usage context is associated with driving, and if it is detected that the individual's mental state is associated with drowsiness, it may be outside a set threshold, in which case, an alarm may be generated.

In some embodiments the threshold may comprise a logical choice (e.g. this or that). In some embodiments, the threshold may comprise an arbitrary decision criterion (as descried in the previous paragraph). In some embodiments, the threshold is an absolute value or a change in a value from previous mental state.

The Al system 28 uses the external stimulus 40 to prompt user input (user settings 26) to accept or reject a detected change of mental state classification 22 as a threshold changes setting for subsequent changes of that detected mental state classification 22. The mental state classification 22 is associated with at least one usage context. The Al system 28 further causes the threshold change setting for the current mental state classification 22 and a usage context to be stored in a user account associated with the user.

In some embodiments, the calibration of sensor data/mental state classification and the threshold may be configured in the user equipment. The Al system 28 may encode the data and the changes in the classification or the mental state data and send it to the user equipment. The user equipment may be configured to decode the encoded data and calibrate the threshold based on the usage context as described earlier.

Some embodiments of a method of remotely training an artificial intelligence, Al, system 28 to provide data representing at least one individualized mental state classification for a mental state of an individual comprise the Al system 28 receiving (step 202) continuous training data comprising sensed brain activity data 20 for the individual, training (step 240) the Al system by analyzing the training data to find sets of features for classifying mental states of the individual, cross-training (step 206) the trained Al system using sets of features classifying the same mental states of at least one other user; and generating (step 208) sets of features using the cross-trained Al system, each set of features classifying a mental state of the individual.

Figure 4 illustrates schematically how method 200 performed by the Al system 28 includes cross training the individually trained Al system 28 to allow for faster and more accurate data to be provided which represents at least one individualized mental state classification of a mental state of the individual. The method 200 is performed for each individual who has registered to use the service and/or set up a user account. The method 200 trains the artificial intelligence, Al system 28, for example, the Al system described herein as being used by the server to provide each individual's mental state as the on-demand service to third party applications.

One example of such a method provides data representing at least one individualized mental state classification for a mental state of an individual for which the Al system performs a training method comprising: receiving 202 training data comprising a sensed brain activity data feed 20 for the individual, training 204 the Al system by analyzing the training data to find sets of features for classifying mental states of the individual, cross-training 206 the trained Al system using sets of features classifying the same mental states of at least one other user, and generating 208 sets of features using the cross-trained Al system, each set of features classifying a mental state of the individual.

Figure 4 illustrates schematically an example of how the cross-training can be implemented when there are four users: user A, user B, user C and user D. It is assumed that the Al system 28 accesses the EEG sensor data 20 and may have access to other sensor data 30 corresponding to each user.

In Figure 4, the server 16 provides a training data stream or data set of sensor data 20, 30 for user B (step 202A) which is used to individually train the Al system for user B (step 204A) by determining a set of features in user B's EEG data which are associated with various categories of mental states. The individually trained Al system is then cross-trained using a collective user Al system model which has associated features for at least one other user with mental state classifications for the at least one other user. In this first example, shown on the left-hand side of Figure 4, the collectively trained Al system model comprises just the individual trained Al system model for user A. By cross-training the Al system model individually trained for user B with the features associated with various mental state classifications found by the collectively trained Al system model for all other users, the cross-trained Al system can generate a set of features (step 208A) which are more likely to provide better results for the individualized mental state classifications 22 of the mental state of the user B.

Figure 4 also shows how the cross-training adapts as more users subscribe to the mental state as a service. Figure 4 shows how another user C at first provides EEG data to the Al system model so this can be individually trained (step 204B) to classify their mental state, for example, by using training data generated (step 202B) as a result of providing feedback using one or both of the feedback loops shown in Figure 3. The individually trained Al system associates certain features in the EEG data stream from user C with their mental state. The trained Al system is then cross-trained using the features associated with the mental state data found by the collectively cross-trained system for users A and B in step 206B. The cross-trained system generates a set of features for classifying the mental state of user #C which is based both on the individual training and on the collective training it has received. Similarly, for the next user D, the Al system is first individually trained to generate a set of features which classify user D's mental state (step 202c), and the trained Al system for user D is then cross-trained using the features found by the cross-trained Al system for the group of users A, B, C to be associated with various mental states for that group of users A, B, C, and the cross-trained Al system can then be used to generate a better set of features for classifying the mental state of user D.

In one embodiment, the cross-training comprises a transfer learning process, which reuses some or all of the training data, feature representations, neural-node layering, weights, training method, loss function, learning rate, and other properties of the earlier Al system model. In some embodiments of the method 200 of training an Al system to provide a classification of an individualized mental state is provided which comprises the step 202 of receiving continuous training data comprising sensed brain activity data 20 for the user. The method 200 comprises the step 204 of training the Al system 28 by analyzing the training data to find sets of features for classifying mental states of the user A. At step 206 the Al system is cross trained using data corresponding to one or more other users (user B) from the database that stores historic EEG data corresponding to plurality of other users (such as user B). The method 200 finally comprises the step 208 of generating sets of features using the cross-trained Al system 28. Each set of features classifies a mental state of the user A. The set of features classifying the mental state of the user A is further stored in the database of the server 16 and may be used to cross train the Al system 28 when a service request 14 to share mental state data of another user will be received at a later stage.

In some embodiments, the method 200 trains the Al system using unlabeled data for the individual. For example, the Al system may learn what features should be labelled as being associated with a user's mental state, for example based on feedback as part of the training process for the user's headset (see Figure 3 for example, where the headset training feedback is labelled as calibration feedback 39). In other embodiments, the Al system is first trained using labelled data, for example, the user may upload to their user account a labelled data set or other configuration data or may restore an old account or the user's brain-wave data may be collected from the user and labelled as part of the training process. The Al system is also cross-trained using labelled data for features known to be associated with the previous users in some embodiments. In some embodiments, the Al system is trained using unlabeled and/or labelled data and is then cross-trained using labelled data.

In some embodiments, the individually trained and/or the cross-trained Al system is configured to output a current mental state classification as an external stimulus to the individual as part of a feedback process. In some embodiments, the individual responds to the feedback with brain wave activity that improves the identification of features by the trained and/or cross-trained Al system.

In some embodiments, a current mental state classification for a user, where current, for example, may mean contemporaneous with the time the request was received or close to the time when the request was received, is determined, for example, in real-time, by associating features derived from a current time-segment 42 of a brain wave activity data stream sensed from individual with a set of features previously associated with a mental state classification of the individual.

In some embodiments, the characteristics or features in the brain-wave activity data feed of an individual comprises a plurality of features associated with particular frequencies. In some embodiments, the Al system is configured to determine a feature based on one or more power spectrum peaks in one or more frequency bands associated with brain wave activity in a current time-segment of the brain wave activity data stream for one or more channels of brain wave activity data. In some embodiments, a feature further comprises an association between one or more power spectrum peaks in one or more frequency bands associated with brain wave activity in a current timesegment of the brain wave activity data stream for one or more channels of brain wave activity data and the location of a brain wave activity sensor providing a channel of brain activity data on the scalp of the individual.

In some embodiments, a feature may be determined over a plurality of time-segments, for example, the Al system may be trained to find an association of the features for brain wave activity sensed in a current time-segment and with the features for brain wave activity sensed in one or more previous time-segments.

As mentioned earlier referring to method 100, in some examples, the user account will include identifiers for other data feeds associated with the user's activity, such, as for example, a smart-watch etc. This data can also be included in the training data provided to the Al system in some embodiments, in which can the training data includes the sensed brain activity data and other user activity data 30 from at least one sensor 32A configured to detect other user activity concurrent with brain activity. The data feeds from EEG sensors and at least one other sensor for the user's activity may be fused in some example embodiments.

Some examples of other user activity data 30 include data generated using at least one of the following sensors: a heart-rate sensor, a blood oxygen sensor, a skin-temperature sensor, a gaze tracking ; and a hydration sensor.

The one or more frequency bands may be any suitable frequency band or sub-frequency band for which brain-wave activity is generated, for example, a frequency of electrical brain-wave activity that an EEG sensor is capable of detecting. Examples of such frequency bands include one or more of: a brain activity gamma wave frequency band, a brain activity beta wave frequency band, a brain activity alpha wave frequency band, a brain activity theta wave frequency band, and a brain activity delta wave frequency band.

Figure 5A illustrates schematically how an array of EEG sensors 18 are located over a scalp of a user such as user A when they are wearing the wearable device 8 such that the EEG sensors 18 can record the brain waves 20 of user at various locations of that user's brain when he/she is engaged in an activity for which their mental state is to be shared with application 12 as described above.

Figure 5B depicts the posterior scalp topography of the brain wave activity data 20 of an individual for certain brain wave frequencies recorded by the EEG sensors 18, in which the shading and contour lines illustrate the variation of particular detected frequencies by a sensor array, for example, a sensor array such as Figure 5A illustrates where sensors are arranged over the user's scalp. Figure 5B shows schematically an example of power peaks in brain wave activity at certain frequencies in a posterior scalp tomography. Figures 5C-5D illustrate brain wave activity data (EEG data) 20 of an individual (user A) as received from the EEG sensors 18 for training the Al system 28. Figure 5C depicts the brain wave activity data 20 of the individual when he/she is engaged in an activity and illustrates the brain wave activity data 20 for a seven channel EEG sensor array (in other words where seven EEG sensors are arranged over a user's skull). Figure 5D shows how each channel captures neural oscillations at different frequencies that characterize various mental states, such as attentional state, the emotional state, the cognitive state or the arousal state. In the example embodiment illustrated schematically in Figure 5D these are shown as brain waves at various ranges of different wavelengths or frequencies, for example, a gamma wave frequency band, a beta wave frequency band, an alpha wave frequency band, a theta wave frequency band and a delta wave frequency band.

Figure 5E shows how a segment of the brain wave activity is captured for the purposes of training and using the Al system 28. In Figure 5E, a window or sliding window 42 is used within which the brain wave activity data 20 on one or more possible frequency bands can be used for the training of the Al system 28, for example. The brain waves activity data may be raw data streams captured by the EEG sensors 18. The sampling frequency for sampling the brain wave activity data stream is chosen suitably for accommodating all frequencies from the above frequency bands.

In Figure 5E, the brain wave activity data stream sensed from an individual in a current timesegment 42 comprises various features. A current mental state classification is determined by processing the features derived from the current time-segment 42 of the brain wave activity data stream with a set of features previously associated with a mental state classification 22 of the individual. A feature for classifying a mental state of the individual comprises one or more power spectrum peaks in one or more frequency bands associated with brain wave activity in a current time-segment of the brain wave activity data stream for one or more channels of brain wave activity data. A feature further comprises a processing between one or more power spectrum peaks in one or more frequency bands associated with brain wave activity in a current time-segment of the brain wave activity data stream for one or more channels of brain wave activity data and the location of a brain wave activity sensor providing a channel of brain activity data on the scalp of the individual. A feature further comprises processing of the features for brain wave activity sensed in the current time-segment and with the features for brain wave activity sensed in one or more previous time-segments. The training data includes sensed brain activity data and other user activity data 30 from at least one sensor 32A configured to detect other user activity concurrent with brain activity for user A.

Figure 6 is a flowchart illustrating an example embodiment of a method 300 for classifying brain activity. The method 300 comprises one or more steps that may be performed by Al system 28. The sequence of steps of the method 300 may not be necessarily executed in the same order as they are presented. Further, one or more steps may be grouped together and performed in form of a single step, or one step may have several sub-steps that may be performed in parallel or in sequential manner.

The method comprises the step 302 of receiving sensor data stream or sensed brain wave activity data for the individual user #A. As described previously in the description, the sensor data stream may refer to the brain wave data 20 captured by the EEG sensors 18A as well as data 30 captured by the other sensors 32A. The method comprises the step 304 of detecting features received sensed brain wave activity data. The Al system 28 is trained to detect features that can indicate a mental state in the input data. The method comprises the step 306 of processing the features in the received sensed brain wave activity data with one or more feature sets. Each feature set comprises at least one feature, associated by the Al system 28 trained as described in steps 204 and 206 with reference to Figure 4 above with one or more mental state classifications of the individual. The processing of the features in the received sensed brain wave activity data is performed at the Al system 28.

The method further comprises the step 308 of generating at least one mental state classification for the sensed brain wave activity data based on the processing 306 of the determined features with a mental state classification meeting a classification condition. The receiving and the generating are performed in real-time and the mental state classification comprises a current mental state classification. The classified mental state data can be shared with the server 16.

Figure 7 illustrates schematically a sequence flow between a user equipment 44 and the server 16 for configuring an individualized mental state on-demand service. The user equipment may be a user equipment operated by the individual whose mental state data is requested by the application 12. The user equipment 44 and the server 16 may communicate using a communication network as described previously in this description. The sequence flow comprises one or more step representing the communication between the user equipment 44 and the server 16.

In some embodiments, such a flow arises as a result of the server 16 performing a method of configuring the individualized mental state on-demand service, the method comprising: providing, for display on the user device 44, a prompt for configuring one or more usage contexts and one or more mental state classifications as one or more user settings of the user account for the individualized mental state on-demand service, receiving at least one configured user setting, each configured user setting associating a mental state classification of the individual with a usage context for which the mental state classification is to be made available to one or more requesting applications 12, and storing the configured user setting with an identifier for the user account and a brain activity identifier associated with a source 18 of sensed brain activity data 20 for the individual. In some embodiments, the method further comprises receiving a registration request to establishing the user account from a device; and associating the user account with the brain activity identifier associated with the source of sensed brain activity data 20 for the individual.

For example, as illustrated in Figure 7, the user equipment 44 sends 46 a request to register with the server 16. Any suitable registration process can be followed, for example, in some embodiments, the registration request is sent via an application running on a user device such as a smart phone or computer or other device configured or configurable to send a registration request. In other embodiments, the request for registration is processed via a web-server. In some embodiments the registration associates the data 20 and 30 from the EEG sensors 18A with the mental state classifications found by the processor/processing circuitry of the server 16. Once a suitable user account is established and the user registered 48, the user can identify the EEG headset sensors and/or any the other sensors 32A that may be providing data for which that user's settings should be assigned and receive (50) a prompt for configuration settings. The user account accordingly is configured to associate EEG data with any other sensor data received forthat user. In some embodiments, registration may be important when a change in the mental state data of the user has to be notified to the user. The server 16 receives the registration request to establishing a user account from the user equipment 44.

In some embodiments, a user registers an account with the server 16 using input fields provided in one or more Uls or displays provided by the server 16 to the user equipment 44. Each user account is associated with a unique user identifier and may include personal information and also information about any the user equipment 44, such as model number, among others.

In some embodiments, a Ul may be provided to facilitate the user linking their account to one or more social media accounts to which the user and/or the user equipment has been associated. A registered user/user equipment can, during subsequent sessions, log in to the digital platform of the server 16 by providing his/her credentials, which the user had provided while registering. The server 16 stores the details of the user and the user equipment in the memory 36.

The server 16 subsequently registers 48 the user/user equipment 44. The server 16, sends the prompt 50 to the user equipment 44 for configuring one or more usage contexts and one or more mental state classifications as one or more user settings of a user account for the individualized mental state on-demand service. In some embodiments, the server 16 provides a plurality of user-setting options for presentation 52 to the user on a suitably displayed Ul so the user can select various settings and options. The user setting selected are detected 54 by the user equipment 44. The configured user settings are communicated 56 to the server 16. Each configured user setting associates a mental state classification with a usage context for which the mental state classification is to be made available to one or more requesting applications. The server 16 stores 58 the configured user settings with an identifier for the user account and a brain activity identifier associated with a source (e.g. EEG sensor 18) of sensed brain activity data 20 for the individual. The server 16 associates the user account with the brain activity identifier associated with the source of sensed brain activity data 20 for the individual.

In an example, the sequence flow for registration may occur simultaneously when the user is interacting with the server 16 via the application 12 while engaged in a task-based context activity. For example, the configured user settings may include permission to share cognitive state data for a certain task-based context but not to share arousal state data for the same task -based context.

Figure 8 illustrates an example embodiment of a sequence flow between the EEG sensors 18, 18A and the other sensors 32A, the user equipment 44, the server 16 and the Al system 28 for an individual user #A. The sequence flow comprises one or more steps representing the communication between the sensors 18, 18A, 32A, the user equipment 44, the server 16 and the Al system 28.

The data streams 20, 30 captured by the sensors 18, 18A, and 32A are transmitted to the server 16. The server 16 makes the data stream 20, 30, accessible to the Al system 28 or sends the data stream 20, 30, to the Al system 28. This step corresponds to the step 202 described with reference to Figure 4.

The method steps performed by the Al system 28 as seen in Figure 8 are already described with reference to Figure 4. The Al system 28 is trained to find the mental state classification features using the sensor data stream data, at step 204 which may include the second training feedback loop 39 shown previously in Figure 3. At step 206, the trained Al system 28 is cross-trained using sets of features classifying the same mental states of other users. At step 60, the cross-trained Al system 28 classifies the current mental state of the user. At step 62, the one or more external stimuli feedback is provided to the user equipment 44. The feedback can include information corresponding to the mental state data classification of the individual and/or one or more external stimuli to provoke a particular mental response. The feedback can be provided in the form of an audio/voice feedback such as an alarm. In some embodiments, the feedback can be provided in the form of a text or a notification displayed on the Uls of the application 12 (or the other digital platforms).

Figure 9 illustrates a sequence flow between the EEG sensors 18, 18A and the other sensors 32A, the server 16 and the Al system 28. The sequence flow comprises one or more steps representing the communication between the sensors 18, 18A, 32A, the server 16 and the Al system 28.

The data streams 20, 30 captured by the sensors 18, 18A, and 32A for user #A are transmitted to the server 16. The server 16 makes the data stream 20, 30, accessible to the Al system 28 or sends the data stream 20, 30, to the Al system 28. The Al system 28 performs steps 204 and 206 as described above with reference to Figure 4 and Figure and determines 78 a mental state or classifies a mental state as representing an attentional state or an emotional state or a cognitive state or an arousal state.

The Al system 28 communicates the current mental state classification 22 determined in the previous step to the server 16. The server 16 may monitor 56 the current mental state classification data 24 of the individual for a duration of time to determine if a mental state classification has changed. While monitoring, it may be determined that the mental state classification has changed. If it is determined that the current mental state has changed, the server will check if the user setting indicates that the changed mental state classification is available for the given usage context. Based on the indication, the server 16/AI system 28 provides data representing the updated mental state classification 24 to the requesting application 12. The provided data comprises a stream of mental state classification data.

Figure 10 illustrates a sequence flow between the application 12A, the server 16 and the Al system 28. The sequence flow comprises one or more steps representing the communication between the application 12A, the server 16 and the Al system 28.

As already described, the requesting application 12A may be an app which has a component hosted on the server 16 and a component running on the user equipment 44 in some embodiments, but in other embodiments it does not need to be configured in this manner. The application 12A may be an example of learning/educational application, content creating application, infotainment content providing application and a gaming application, among others. The application 12A sends a service request 14 to the server 16, at step 104, to access a mental state data of the individual interacting with the application 12A on the user equipment 44. The steps performed at the server 16 upon receiving the service request 14 are already described in detail in reference with Figure 1A. Therefore, the steps 104- 108 will not be described in much detail here for the sake of brevity.

The server 16 determines a usage context for the requested mental state data at step 104. A usage context may comprise one or more of task-based context, geographic environmental context, situational environmental context and physiological context. At step 106, the server 16 checks user settings for sharing the mental state data and determines whether the service request 14 can be fulfilled based on the user settings for sharing mental state data of the individual and the determined usage context. The server 16 receives mental state classification 22 from the Al system 28. At step 108, the server 16 filters mental state data 24 by applying filter 34 based on the user settings. The server 16 then provides the filtered mental state data to the application 12A.

Figure 11 schematically illustrates an example apparatus 1110 according to the fifth aspect of any of the disclose embodiments of the fifth aspect. In some embodiments, for example, apparatus configured to execute a method according to any embodiments of the first method aspect. The apparatus 1110 comprises a controller (CNTR; e.g., controlling circuitry or a control module) 1100.

The controller 1100 is configured to cause the apparatus to receive, or to cause reception of, the service request 14 for mental state data for the individual from the requesting application 12. To this end the controller 1100 comprises or is otherwise associated with (e.g., connected, or connectable, to) a transceiver (TX/RX; e.g., transceiving circuitry or a transceiver module) 1101. The transceiver 1101 may be configured to receive the service request 14 for mental state data for the individual from the requesting application 12.

The controller 1100 is also configured to cause the apparatus to determine, or to cause determination of, the usage context for the requested mental state data 24. To this end the controller 1100 comprises or is otherwise associated with (e.g., connected, or connectable, to) a determiner (DET; e.g., determining circuitry or a determination module) 1102. The determiner 1102 may be configured to determine the usage context for the requested mental state data 24.

The controller 1100 is also configured to cause the apparatus to associate, or cause association of, the service request 14 with the brainwave activity data feed 20 (or the EEG data) from the individual. To this end the controller 1100 comprises or is otherwise associated with (e.g., connected, or connectable, to) an associator (ASC; e.g., associating circuitry or association module) 1103. The associator 1103 may be configured to associate the service request 14 with the brainwave activity data feed 20 (or the EEG data) from the individual.

The controller 1100 is also configured to cause the apparatus to determine, or cause determination of, the mental state data for the user based on the brainwave activity data feed. To this end the controller 1100 comprises or is otherwise associated with (e.g., connected, or connectable, to) the determiner (DET; e.g., determining circuitry or a determination module) 1104. The determiner 1104 may be configured to determine the mental state data for the user based on the brainwave activity data feed.

The controller 1100 is also configured to cause the apparatus to filter, or cause filtration of, the determined mental state data based on the user setting. To this end the controller 1100 comprises or is otherwise associated with (e.g., connected, or connectable, to) a filtration circuitry (FLT; e.g., filtration module) 1105. The filtration circuitry 1105 may be configured to filter the determined mental state data based on the user setting.

The controller 1100 is also configured to cause the apparatus to transmit, send or otherwise communicate, or cause transmission of, the filtered mental state data to the requesting application 12 based on the determined usage context. To this end the controller 1100 comprises or is otherwise associated with (e.g., connected, or connectable, to) the transceiver 1101. The transceiver 1101 may be configured to cause the transmission of the filtered mental state data to the requesting application 12 based on the determined usage context.

Figure 12 schematically illustrates an example apparatus 1210 according to some embodiments, for example, apparatus configured to execute a method according to any embodiments of the second method aspect disclosed herein. The apparatus 1210 comprises a controller (CNTR; e.g., controlling circuitry or a control module) 1200.

The controller 1200 is configured to cause the apparatus to receive, or cause reception of, a sensed brain wave activity data for the individual. To this end the controller 1200 comprises or is otherwise associated with (e.g., connected, or connectable, to) a transceiver (TX/RX; e.g., transceiving circuitry or a transceiver module) 1201. The transceiver 1201 may be configured to receive the sensed brain wave activity data for the individual.

The controller 1200 is configured to cause the apparatus to determine, or cause determination of, features in the received sensed brain wave activity data. To this end the controller 1200 comprises or is otherwise associated with (e.g., connected, or connectable, to) a determiner (DET; e.g., determining circuitry or a determination module) 1202. The determiner 1202 may be configured to determine the features in the received sensed brain wave activity data.

The controller 1200 is configured to cause the apparatus to associate, or cause association of, the features in the received sensed brain wave activity data with one or more feature sets, each feature set comprising at least one feature, associated by an Al system with one or more mental state classifications of the individual. To this end the controller 1200 comprises or is otherwise associated with (e.g., connected, or connectable, to) an associator (ASC; e.g., associating circuitry or an association module) 1203. The associator 1203 may be configured to associate the features in the received sensed brain wave activity data with one or more feature sets, each feature set comprising at least one feature, associated by an Al system with one or more mental state classifications of the individual.

The controller 1200 is configured to cause the apparatus to generate, or cause generation of, at least one mental state classification 24 for the sensed brain wave activity data based on the correlation of the determined features with the at least one mental state classification meeting a classification condition. To this end the controller 1200 comprises or is otherwise associated with (e.g., connected, or connectable, to) a generator (GEN; e.g., generating circuitry or a generation module) 1204. The generator 1204 may be configured to generate at least one mental state classification 24 for the sensed brain wave activity data based on the correlation of the determined features with the at least one mental state classification meeting a classification condition.

Figure 13 schematically illustrates an example apparatus 1310 according to some embodiments, for example, for example, apparatus configured to execute a method according to any embodiments of the third method aspect. The apparatus 1310 comprises a controller (CNTR; e.g., controlling circuitry or a control module) 1300.

The controller 1300 is configured to cause the apparatus to receive, or cause reception of, continuous training data comprising a sensed brain activity data feed 20 for the individual. To this end the controller 1300 comprises or is otherwise associated with (e.g., connected, or connectable, to) a transceiver (TX/RX; e.g., transceiving circuitry or a transceiver module) 1301. The transceiver 1301 may be configured to receive continuous training data comprising a sensed brain activity data feed 20 for the individual.

The controller 1300 is configured to cause the apparatus to train, or cause training of, the Al system 28 by causing the Al system 28 to analyze the training data to find sets of features for classifying mental states of the individual. To this end the controller 1300 comprises or is otherwise associated with (e.g., connected, or connectable, to) a trainer (TRN; e.g., training circuitry/module) 1302. The trainer 1302 may be configured to training of the Al system 28 by analyzing the training data to find sets of features for classifying mental states of the individual. The controller 1300 is configured to cause the apparatus to cross-train, or cause cross-training of, the trained Al system using sets of features classifying the same mental states of at least one other user. To this end the controller 1300 comprises or is otherwise associated with (e.g., connected, or connectable, to) the trainer 1302. The trainer 1302 may be configured to cross-train the trained Al system using sets of features classifying the same mental states of at least one other user.

The controller 1300 is configured to cause the apparatus to generate, or cause generation of, sets of features using the cross-trained Al system, each set of features classifying a mental state of the individual. To this end the controller 1300 comprises or is otherwise associated with (e.g., connected, or connectable, to) a generator (GEN; e.g., generating circuitry or a generation module) 1303. The generator 1303 may be configured to generate sets of features using the cross-trained Al system, each set of features classifying a mental state of the individual.

Figure 14 illustrates an example set of machine-executable instructions 1400, which are stored on a computer readable medium. The machine-executable instructions are loadable into one or more data processor(s) (PROC; e.g., data processing circuitry or a data processing unit) 1410, which may, for example, be comprised in the server 16 or Al system 28 of server system 10. When loaded into the data processor(s), the machine-executable instructions may comprise a computer program stored in a memory, MEM, 1420 associated with or comprised in the data processor. According to some embodiments, the computer program may, when loaded into and run by the data processor, cause execution of method steps according to, for example, any of the first to fourth method aspects or any of their disclosed embodiments, for example, such as those described hereinabove and illustrated in Figures 1-3, or otherwise described herein.

Aspects of the disclosure are described with reference to the drawings, e.g., block diagrams and/or flowcharts. It is understood that several entities in the drawings, e.g., blocks of the block diagrams, and combinations of entities in the drawings, can be implemented by computer program instructions, which instructions can be stored in a computer-readable memory, and loaded onto a computer or other programmable data processing apparatus. Such computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer and/or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer and/or other programmable data processing apparatus, create means for implementing the functions/acts specified in the block diagrams and/or flowchart block or blocks.

The described embodiments and their equivalents may be realized in software or hardware or a combination thereof. The embodiments may be performed by general purpose circuitry. Examples of general purpose circuitry include digital signal processors, DSP, central processing units, CPU, coprocessor units, field programmable gate arrays, FPGA, and other programmable hardware. Alternatively or additionally, the embodiments may be performed by specialized circuitry, such as application specific integrated circuits, ASIC. The general purpose circuitry and/or the specialized circuitry may, for example, be associated with or comprised in an apparatus such as a wireless communication device or a network node.

Embodiments may appear within an electronic apparatus (such as a wireless communication device or a network node) comprising arrangements, circuitry, and/or logic according to any of the embodiments described herein. Alternatively or additionally, an electronic apparatus (such as a wireless communication device or a network node) may be configured to perform methods according to any of the embodiments described herein.

According to some embodiments, a computer program product comprises a tangible, or nontangible, computer readable medium such as, for example a universal serial bus, USB, memory, a plugin card, an embedded drive or a read only memory ROM, flash memory, non-volatile memory such as, for example, electrically erasable programmable read-only memory, EEPROM, programmable ferroelectric RAM, FeRAM or F-RAM, metalization cell memory, for example, conductive bridging RAM or CBRAM™, parallel random-access machine, PRAM, a shared-memory abstract machine, spin-transfer torque memory, STT-RAM or STT-MRAM, silicon-oxide-nitride-oxide-silicon, SONOS, memory, resistive random access memory, ReRAM or RRAM, domain-wall memory, DWM, also referred to as racetrack memory, nano-RAM, NRAM, 3D XPoint non-volatile memory, and millipede non-volatile memory, and may in some embodiments comprise a form of semi-volatile memory. SST-RAM may be non-volatile or semi-volatile. Figure 14 illustrates an example of a computer readable medium storing machineexecutable instructions 1400 which can be transferred to or form part of a computer readable medium shown as MEM 1420. MEM 1420 may form part of the apparatus shown in Figures 11 to 13 and has stored thereon a computer program comprising the machine executable instructions 1400. The computer program is loadable into a data processor (PROC; e.g., data processing circuitry or a data processing unit) 1410, which may, for example, be comprised in the server system 10. The computer program may be stored in a memory (MEM) 1420 associated with or comprised in the data processor. According to some embodiments, the computer program may, when loaded into and run by the data processor 1410, cause execution of method steps according to, for example, any of the methods illustrated in Figures 1-3 or otherwise described herein. The machine-executable instructions may be provided and loaded into the MEM 1420 from an external source as shown in Figure 14 comprising for example, using a tangible or non-tangible medium as disclosed herein. The machine-executable instructions are provided in a form in MEM 1420 which enables them to be executed by the processor 1420 when loaded from MEM 1420 and may be some embodiments be implemented as circuitry or otherwise hardcoded.

Figure 15 of the accompanying drawings shows schematically how a method of training or cross- training an Al model which the Al system hosts, for example, the cross-training process shown in Figure 4 can be implemented according to some embodiments, using for example, the system shown in Figure 3.

Several such Al models are known to those of ordinary skill in the art, which are suitable for Al classifier models, for example, machine learning models which can use a mix of different techniques with training occurring in two phases. For example, initially there will be a need to create a first mapping of EEG data to mental states. For this supervised learning will be applied, typically using neural networks. This can be done either in a controlled environment or using external input, e.g. from cameras or questionnaires. This must be done for the initial set of users. For the second phase when new users are onboarded in some embodiments unsupervised learningto cluster EEG data from new users to the initial data set may be used at least in part. There could be only one "cluster" or more than one or as many as there are users, depending on specific circumstances. If there are many, some unsupervised technique could be used to select the "best" match. After this, in some embodiments cross-training in the form of individual transfer learning is performed if required (this may again benefit from external inputs, but at much lower rate).

In the embodiment illustrated in Figure 15, the Al or ML model of Al system 28 uses a training process which occurs in two phases or stages. In the first stage, a classifier model 1530 is trained based on raw EEG data from each individual user. In the second stage, a classifier combiner model 1630 is used for classifying each individual user's EEG data.

As shown in Figure 15, a single user training 1500 occurs for each individual user #A, #B, #C in one or more controlled usage context(s). For example, when a user #A is wearing their EEG headset 8A in a particular usage context, they may train the classifier and/or their headset using the system shown schematically in Figure 3 where, for individual user #A stimuli feedback 40A and training feedback 39A is provided. The feedback may be used by that user's headset 8A and/or the processor/control circuitry 2A of that user's headset 8A and/or to the other user device 9 of that user #A to adjust the EEG electrode(s) so that when the raw EEG data 1510 undergoes spectrographic analysis the resulting spectrogram 1520 generates features which, when input into classifier 1530, result in better classification of that user's mental state.

The spectrogram 1520 plots as a function of the input frequencies at a point in time the peak power of the wavelets in the input raw EEG data feed signal from some arbitrary time zero (which may be a time zero indicating the time-stamp for the start of a data segment such as a frame or window or buffer of data from the user's EEG data feed). In some embodiments, the spectrogram may be generated using a data segment of the EEG data feed, for example, by using a sliding window such as Figure 5E shows, in which case time zero may represent a start of a segment of data in that user's EEG data feed. Any suitable technique known to those of ordinary skill in the art for wavelet transform for classification of EEG signals using ML models such as support vector machines and/or artificial neural networks may be used however, such as, for example, the techniques disclosed in Wavelet Transform for Classification of EEG signal using SVM and ANN by Nitendra Kumar, Khursheed Alam and Abul Hasan Sissiqi, published in Biomedical and Pharmacology Journal Vol. 10 (4), 2061-2069 (2017), https://dx.doi.org/10.13005/bpj/1328.

As shown in Figure 15, the features identified from each spectrogram are fed into an Al model classifier or machine learning model classifier 1530 which is configured to process the input spectrogram features to associate combinations of features with one or more classifications of a mental state of the user #A. The user #A is able to confirm their mental state in this training phase and/or may confirm the context of that mental state. Once trained, the output of the first training phase 1500 is a trained classifier model which is then stored in memory 1540 (of the Al system 28). Examples of suitable classifiers 1530 include a linear discriminant analysis model classifier, or a support vector machine classifier, or another classifying neural network model known to those of ordinary skill in the art to be suitable for classifying spectrographic features. In some embodiments, the classifier model and/or any preprocessing of the raw EEG data is configured to occur sufficiently quickly to prevent the EEG data feed from the user #A overflowing system memory or buffer space.

A similar individual training phase or process 1500 for a controlled usage context is also provided for the other individual users #B, #C as shown in Figure 15. For example, the process 1500 is also used to generate an individually trained classifier 1530 for user #B as shown in Figure 3. The configuration of each trained classifier 1540 is then stored in memory shown as stored trained classifier #A 1540 in Figure 15 until a suitable number of individually trained classifiers have been generated, at which point the second phase of training 1600 for each individual user starts.

In the second training phase, cross-training phase 1600, new raw EEG data 1610 of each user (user #A only is shown) undergoes similar spectrographic analysis to generate spectrographic features such as may be provided in the form of a spectrogram 1620 plotting that new raw EEG data wavelet peak power as a heat map of frequency vs. time from an arbitrary time zero. The spectrogram features are then input a ML model 1640 comprising a classifier model combiner 1630.

The classifier combiner 1630 comprises a plurality of trained classifiers 1540 (each classifier having been trained for a different one of a plurality of individual users, in this case, users #A, #B, #C). The combined classifier 1630 may also be trained using calibration information or feedback from the user #A. The combined classifier 1630 combines the plurality of individually trained classifiers 1540 which are extracted or exported (1550) from the classifier store 1540 using a suitable ensemble technique. Examples of suitable ensemble techniques known in the art included but are not limited to a concatenation ensemble technique, an average ensemble technique, and a weighted ensemble technique. The output of each of the individually trained classifiers is input into a combiner function. The combiner function is trained for every new user so that once the classifier combiner 1630 has been trained (and also possibly calibrated), its output 1650 is one or more mental state classifications(s) of for user #A based on the new EEG data 1610 for the user #A. By combining trained classifiers from several individual users the learning time for a new user is decreased and the reduction in training time can be achieved by using either ensemble techniques or transfer learning.

Examples of the output of the combiner 1630 comprise accordingly one or more mental state classifications based on the raw EEG data received from user #A, such as for example, a "happy" emotional mental state and a "confused" cognitive mental state may be concurrently classified in some examples.

In some embodiments, the individually trained classifiers which are combined by the particular classifier combiner 1630 comprise multiple individually trained classifiers 1530 which are selected based on one or more criteria, such as, for example, characteristics of the trained classifiers 1540. In some embodiments, one or more of the plurality of trained classifiers are selected for combining, based, for example on input or feedback from the user, which may be done off-line, for example, via a questionnaire. Examples of classifier combiner 1630 include a linear function model or a neural network model. The output from each of the individual trained classifiers 1540 are treated as features to a new combining classifier.

In some embodiments, going from the raw EEG time series data 1510 to the spectrogram 1520, a short-term Fourier transform is used to generate the wavelets from the raw EEG time series data. For example, a Gabor transform can be used to generate Morlet wavelets. Alternatively, another windowed short-time Fourier transform, or a Hilbert transform on bandpass filtered data, could be used in other embodiments.

In some embodiments, the dimensionality of the spectrogram may be reduced using a suitable spatial filtering method to improve the signal to signal and noise ratio, SSNR, of the event-related potentials, ERPs, in the EEG data. The term ERP refers to the relevant waveform which remains after may trials have been averaged together to cause random brain activity to be averaged out. For example, a trainable XDAWN spatial filter may be applied to reduce the dimensionality of each trained classifier exported to the classifier combiner.

Some embodiments may use in addition/or instead other types of preprocessing on the raw EEG data, for example, preprocessing such as bandpass filtering and downsampling and the like, before the pre-processed data is input into the classifier model 1530. For example, downsampling the raw EEG data signal may be useful as this reduces the number of time-points before classification, which can be advantageous for a live data feed where it may be helpful if it results in a faster classification. Spatial filtering techniques can both enhance the signal and reduce the number of features which are fed into the classifier, which can also reduce the computational complexity of the classifier required, using less processing power and also potentially allowing a faster classification of the user's mental state.

Such pre-processing may be performed in some embodiments where there is no prior information about the spatial distribution of a brain response, for example, if all electrodes in an electrode array covering a user's scalp were used, and spatial filtering may also be performed in some embodiments on the raw EEG data before this is input into the classifier to enable a less computationally complex classifier to be used than might be required if there was no spatial filtering. As an example, in some embodiments, a classifier may be used on bandpass filtered EEG as input when the number of individual training processes is large enough, for example, more than 50 or so.

In some embodiments, where the cross-training comprises transfer learning and classification, a Riemannian geometry mean can be used as a reference matrix for performing an affine transformation of each user's data. One way of doing this is by calculating a symmetric positive definitive covariance matrix calculated from the XDAWN signal, and use Riemannian transport, to make inter-session and inter-person transfer learning possible.

The use of different ensemble training techniques for combining the individually trained classifier models enables multiple, slightly different, processing steps similar to the one described above to be performed in order to do calculations and filtering of EEG data. There are multiple approaches, and some of them are better than the others in some aspects like persistence to noise, accuracy, amount of seconds of EEG signals needed, training data required for this session, training data required for other sessions, etc., while being slightly worse in other aspects and various models can be selected as would be apparent to someone of ordinary skill in the art. More information can be found in the article entitled "Transfer Learning Algorithm of P300-EEG Signal Based on XDAWN Spatial Filter and Riemannian Geometry Classifier", published on 5^th March 2020 in Appl. Sci. 2020, 10, 1804; DOI: 10.3390/appl0051804 also available to download from https://www.mdpi.com/2076- 3417/10/5/1804/pdf on 7^th December 2020. Some examples of suitable python code can be obtained from the Python library pyRiemann https://pyriemann.readthedocs.io/en/latest/auto_examples/ERP/plot_classify_EEG_tangentspace.htm I or from the article entitled: SIMPLE DEEP NEURAL NETWORKS SHOW STATE-OF-THE-ART PERFORMANCE IN ERP-BASED BCI, by Delobel et al, published in the Proceedings of the 8^th Graz Brain Computer Interface Conference 2019, DOI: 10.3217/978-3-85125-682-6.49 also available to download as of 7^th December 2020 from https://diglib.tugraz.at/download.php?id=5d808fd804079&location=browse.

Generally, all terms used herein are to be interpreted according to their ordinary meaning in the relevant technical field, unless a different meaning is clearly given and/or is implied from the context in which it is used.

Reference has been made herein to various embodiments. However, a person skilled in the art would recognize numerous variations to the described embodiments that would still fall within the scope of the claims.

For example, the method embodiments described herein discloses example methods through steps being performed in a certain order. However, it is recognized that these sequences of events may take place in another order without departing from the scope of the claims. Furthermore, some method steps may be performed in parallel even though they have been described as being performed in sequence. Thus, the steps of any methods disclosed herein do not have to be performed in the exact order disclosed, unless a step is explicitly described as following or preceding another step and/or where it is implicit that a step must follow or precede another step.

In the same manner, it should be noted that in the description of embodiments, the partition of functional blocks into particular units is by no means intended as limiting. Contrarily, these partitions are merely examples. Functional blocks described herein as one unit may be split into two or more units. Furthermore, functional blocks described herein as being implemented as two or more units may be merged into fewer (e.g. a single) unit.

Any feature of any of the embodiments disclosed herein may be applied to any other embodiment, wherever suitable. Likewise, any advantage of any of the embodiments may apply to any other embodiments, and vice versa.

Hence, it should be understood that the details of the described embodiments are merely examples brought forward for illustrative purposes, and that all variations that fall within the scope of the claims are intended to be embraced therein.

Claims

53

1. A computer-implemented method (100) of providing mental state data, the method comprising: receiving (102) a service request (14) for mental state data for an individual from a requesting application (12); determining (104) a usage context for the requested mental state data (24); associating (106) the service request (14) with a brainwave activity data feed (20) for the individual; determining (108) mental state data for the individual based on the brainwave activity data feed; filtering (110) the determined mental state data based on at least one user setting associated with a user account of the individual; and providing (112) the filtered (110) mental state data to the requesting application (12) based on the determined usage context.

2. The computer-implemented method of claim 1, wherein the associating (106) comprises: extracting an identifier from the service request (14); and determining the extracted identifier is associated with the user account of the individual.

3. The computer-implemented method of claim 1 or 2, wherein the user account of the individual includes information representing the at least one user setting for filtering the mental state data of the individual comprising one or more sharing conditions, wherein the one or more sharing conditions for sharing mental state data of the individual are configurable by the individual as one or more user settings for the user account.

4. The computer-implemented method of claim 3, wherein at least one sharing conditions for sharing mental state data of the individual comprises a user-configurable setting in the user account of the individual to share or not share a mental state classification of the mental state data for the individual in one or more user-selected usage contexts.

5. The computer-implemented method of claim 3 or 4, wherein at least one sharing condition for sharing mental state data of the individual comprises a user-configurable setting in the user account of the individual to share or not share for a usage context one or more user-selected mental

RECTIFIED SHEET (RULE 91) ISA/EP 54 state classifications of the mental state data for the individual.

6. The computer-implemented method of any one of claims 4 to 5, wherein based on the sharing conditions for sharing mental state data, the filtering (110) comprises removing mental state data having a mental state classification for which a sharing condition is not met from the mental state data sent to the requesting application (12).

7. The computer-implemented method of claim 6, wherein the filtering (110) retains as filtered mental state data for the individual at least one mental state classification (24) for which a sharing condition in the user account of the individual is met and wherein the filtered mental state data comprises an indication of the at least one mental state classification.

8. The computer-implemented method of claim 6, wherein the filtering (110) retains as filtered mental state data for the individual at least one mental state classification (24) for which a sharing condition in the user account of the individual is met and wherein the filtered mental state data comprises a filtered brain activity data feed associated with the at least one mental state classification (24) of the individual.

9. The computer-implemented method of any one of previous claims, wherein the filtered mental state data comprises current mental state data of the individual derived by processing the brainwave activity data feed for of the individual in real-time to determine a set of brain-wave activity features.

10. The computer-implemented method of claim 4, wherein the mental state data is associated with a mental state classification by using an artificial intelligence, Al, system (28) configured to determine a set of brain-wave activity features for the individual from their brain-wave activity data feed and to associate the determined brain-wave activity features with a mental state classification for the individual.

11. The computer-implemented method of any one of the previous claims, wherein the determined mental state data comprises a mental state classification of at least one of: an attentional state; an emotional state; a cognitive state; and an arousal state.

RECTIFIED SHEET (RULE 91) ISA/EP 55

12. The computer-implemented method of any one of the previous claims, wherein a usage context comprises one or more of: a task-based context; a geographic environmental context; a situational environmental context; and a physiological context.

13. The computer-implemented method of any previous claims, wherein the usage context is determined based on one or more of: explicit information provided by the application (12) in the request (14); a usage context stored in the user account association with the identifier for the application (12); and one or more inferences of activity of the individual based on one or more of: at least one data feed or a fused plurality of data feeds from one or more sensors associated with the individual at the time the request (14) was received.

14. The computer-implemented method of any previous claims, wherein the usage context is dynamic, and the method further comprises monitoring the usage context and adapting the filtered mental state data based on the monitored usage context.

15. The computer-implemented method of claim 14, further comprising: monitoring the at least one current mental state classification data of the individual for a duration of time; determining that at least one mental state classification has changed to a new mental state classification; and based on at least one a sharing condition indicating the new mental state classification is not to be shared for the usage context, removing the mental state classification data from the mental state data provided to the requesting application (12).

16. The computer-implemented method of claim 4, wherein the at least one user setting in the user account of the individual further includes at least one condition for performing an action based on a determined mental state classification.

RECTIFIED SHEET (RULE 91) ISA/EP 56

17. The computer-implemented method of claim 16 wherein the action comprises pushing information associated with the usage context and the current mental state classification to at least one device, wherein the device is configured to perform an action responsive to receiving the pushed information.

18. The computer-implemented method of any one of claims 15 or 16, wherein the at least one condition for performing the action based on a current mental state classification comprises determining at least one feature of the brain activity data feed meets a threshold triggering the action.

19. The computer-implemented method of claim 18, wherein the threshold is a contextbased threshold which is associated with a particular usage context.

20. The computer-implemented method of claim 2 or of any preceding claim dependent on claim 2, wherein the user account of the individual is associated with at least one of: an account-holder identifier for the individual as the account holder; and a source identifier for a source device (8) of the brainwave activity data feed of the individual, wherein determining if the extracted identifier is associated with the user account of the individual comprises determining if the extracted identifier comprises a source identifier or an account-holder identifier which matches a corresponding source identifier or an account-holder identifier for the user account associated with the individual.

21. The computer-implemented method of claim 20, wherein the user account of the individual is associated with one or more other sensed activity data feeds associated with the individual.

22. A computer-implemented method of configuring an individualized mental state on- demand service, the method comprising: providing, for display on a user device (44), a prompt (50) for configuring one or more usage contexts and one or more mental state classifications as one or more user settings of a user account for an individual whose mental state is the individualised mental state provided by the on-demand service; receiving at least one configured user setting (56), each configured user setting associating a mental state classification of the individual with a usage context for which the mental state classification is to be made available to one or more requesting applications (12); and storing in the user account (58), each received at least one configured user setting, an identifier for the user account, and a brain activity identifier associated with a source device (18) of sensed brain activity data (20) for the individual.

RECTIFIED SHEET (RULE 91) ISA/EP

23. The computer-implemented method of claim 22, further comprising: receiving a registration request (46) to establish the user account from a device (11); and associating the user account with the brain activity identifier associated with the source device (18) of sensed brain activity data (20) for the individual.

24. A computer-implemented method of training an artificial intelligence, Al, system (28) to provide data representing at least one individualized mental state classification for a mental state of an individual, the method comprising: receiving (202), by the Al system (28), training data comprising a sensed brain activity data feed (20) for the individual; training (204) the Al system (28) by using the Al system (28) to analyse the training data to find sets of features for classifying mental states of the individual; cross-training (206) the trained Al system (28) using sets of features classifying the same mental states of at least one other individual; and generating (208) sets of features using the cross-trained Al system (28), each set of features classifying a mental state of the individual.

25. The computer-implemented method of claim 24, wherein the Al system is trained using labelled data and cross-trained using labelled data, wherein the labelled data is generated by user input based on at least one feedback stimuli.

26. The computer-implemented method of claim 24, wherein the Al system is trained using unlabelled data and cross-trained using labelled data. 1. The computer-implemented method of any one of claims 24 to 26, wherein the crosstrained Al system is configured to output a current mental state classification as an external stimulus to the individual.

28. The computer-implemented method of any one of claims 24 to 27, wherein the current mental state classification is determined by associating features derived from a current time-segment (42) of a brain wave activity data stream sensed from individual with a set of features previously associated with a mental state classification of the individual.

29. The computer-implemented method of claim 28, wherein a feature comprises one or

RECTIFIED SHEET (RULE 91) ISA/EP more power spectrum peaks in one or more frequency bands associated with brain wave activity in a current time-segment of the brain wave activity data stream for one or more channels of brain wave activity data.

30. The computer-implemented method of claim 29, wherein the feature further comprises an association between one or more power spectrum peaks in one or more frequency bands associated with brain wave activity in a current time-segment of the brain wave activity data stream for one or more channels of brain wave activity data and the location of a brain wave activity sensor providing a channel of brain activity data on the scalp of the individual.

31. The computer-implemented method of claim 29 or 30, wherein the feature further comprises an association of the features for brain wave activity sensed in the current time-segment and with the features for brain wave activity sensed in one or more previous time-segments.

32. The computer-implemented method of any one of claims 24 to 31, wherein the training data includes sensed brain activity data and other activity data (30) associated with the individual from at least one sensor (32) configured to detect the other activity associated with the individual concurrently with the brain activity of the individual.

33. The computer-implemented method of claim 32, wherein the other activity data (30) of the individual comprises data generated using at least one of the following sensors: a heart-rate sensor; a blood oxygen sensor; a skin-temperature sensor; a gaze tracking sensor; and a hydration sensor.

34. The computer-implemented method of any one of claims 29 to 33, wherein the one or more frequency bands comprise one or more of: a brain activity gamma wave frequency band; a brain activity beta wave frequency band; a brain activity alpha wave frequency band; a brain activity theta wave frequency band; and a brain activity delta wave frequency band.

RECTIFIED SHEET (RULE 91) ISA/EP 59

35. A computer-implemented method of determining at least one individualised mental state classification of a mental state of an individual, the method comprising: receiving (302) sensed brain wave activity data for the individual; determining (304) features in the received sensed brain wave activity data; associating (306) the features in the received sensed brain wave activity data with one or more feature sets, each feature set comprising at least one feature, associated by an Al system with one or more mental state classifications of the individual; and generating (308) at least one mental state classification (24) for the sensed brain wave activity data based on the association (306) of the determined features with the at least one mental state classification (24) meeting a classification condition.

36. The computer-implemented method of claim 35, wherein the determining (306) of the features in the received sensed brain wave activity data uses an Al system trained using a method according to any one of claims 24 to 34.

37. The computer-implemented method of claim 35 or 36, wherein the receiving (302) and the generating (308) are performed in real-time and the mental state classification (24) comprises a current mental state classification (24).

38. The computer-implemented method of any one of previous claims 35 to 37, further comprising: providing the at least one mental state classification (24) to an apparatus (16) configured to perform a method according to any one of claims 1 to 21 or 22 to 23 to provide data (24) representing a mental state classification to an application (12).

39. The computer-implemented method of any one of previous claims 35 to 37, wherein the method is performed by a user equipment (44) or a server (16), wherein the method further comprises receiving the one or more feature sets received from an Al system (28) configured to perform a method according to any one of claims 24 to 34, wherein the user equipment (44) or the server (16) performs the associating (306) by associating (306) the features in the received sensed brain wave activity data with the received one or more feature sets to determine the one or more mental state classifications of the individual to a requesting application (12).

40. An apparatus (16, 1110) or control circuitry (1100) for providing a mental state of at least one individual as a service, the apparatus or control circuitry (16, 1100, 1110) comprising:

RECTIFIED SHEET (RULE 91) ISA/EP 60 a memory comprising machine-executable instructions; and one or more processors or processing circuitry; wherein the machine-executable instructions are configured, when loaded from the memory and executed by the one or more processors or processing circuitry, to cause the apparatus or control circuitry to process a received service request (14) for mental state data for an individual from a requesting application (12) by causing the apparatus or control circuitry (16) to: determine a usage context for the requested mental state data (24); associate the service request (14) with a brainwave activity data feed (20) from the individual; determine the mental state data for the user based on the brainwave activity data feed; filter the determined mental state data based on a user setting; and transmit the filtered mental state data to the requesting application (12) based on the determined usage context.

41. The apparatus of claim 40, wherein the machine-executable instructions further cause the apparatus or control circuitry (16) to perform the method of any one of claims 2 to 21.

42. An apparatus (28, 1210) or control circuitry (28, 1200) comprising: a memory comprising machine-executable instructions; and one or more processors or processing circuitry; wherein the machine-executable instructions are configured, when loaded from the memory and executed by the one or more processors or processing circuitry, to cause the apparatus or control circuitry to: receive sensed brain wave activity data for the individual; determine features in the received sensed brain wave activity data; associate the features in the received sensed brain wave activity data with one or more feature sets, each feature set comprising at least one feature, associated by an artificial intelligence, Al, model with one or more mental state classifications of the individual; and generate at least one mental state classification (24) for the sensed brain wave activity data based on the association of the determined features with the at least one mental state classification (24) meeting a classification condition.

43. The apparatus or control circuitry of claim 42, wherein the machine-executable instructions further cause the apparatus or control circuitry (18) to perform the method of any one of claims 36 to 39.

RECTIFIED SHEET (RULE 91) ISA/EP 61

44. An apparatus (16, 28) or control circuitry (16, 28, 1300, 1310) for training an artificial intelligence, Al, system (28) comprising: a memory comprising machine-executable instructions; and one or more processors or processing circuitry; wherein the machine-executable instructions are configured, when loaded from the memory and executed by the one or more processors or processing circuitry, to cause the apparatus or control circuitry to: receive training data comprising a sensed brain activity data feed (20) for an individual; train the Al system (28) by using the Al system to analyse the training data (20) to find sets of features for classifying mental states of the individual; cross-train the trained Al system using sets of features classifying the same mental states of at least one other user; and generate sets of features using the cross-trained Al system, each set of features classifying a mental state of the individual.

45. The apparatus or control circuitry of claim 44, wherein the machine-executable instructions further cause the apparatus or control circuitry (16, 28) to perform the method of any one of claims 25 to 34.

46. An apparatus or control circuitry (13) comprising; a data communications transceiver; a memory comprising machine-executable instructions; and one or more processor(s), wherein the machine-executable instructions, when loaded from the memory and executed by the one or more processor(s), are configured to cause an application (12) hosted by the apparatus or control circuitry (13) to generate a request (14) for mental state data for an individual, wherein the machine-executable instructions are further configured to cause the apparatus or control circuitry(13) to send the request (14) to the apparatus (16) of any one of claims claim 40 or 41.

47. A server system (10) for providing mental state data for an individual on demand to a plurality of requesting applications (12), the server system (10) comprising at least: the apparatus or control circuitry (16, 1100, 1110) of claim 40 or 41; and the apparatus or control circuitry (28, 1200, 1210) of claim 42 or 43.

48. The server system of claim 47, further comprising:

RECTIFIED SHEET (RULE 91) ISA/EP 62 the apparatus or control circuitry (16, 28, 1300, 1310) of any one of claims 44 or 45.

49. A computer program product comprising a non-transitory computer readable medium (1420), having thereon machine-executable instructions (1400) which, when loaded and executed by an apparatus or control circuitry (16, 1100, 1110) comprising one or more processors or processing circuitry (1410), are configured to cause the method of anyone of claims 1 to 23 to be performed.

RECTIFIED SHEET (RULE 91) ISA/EP