EP3757950B1 - Procédé et système de selection des paramètres d'un élément de conception ou de sécurité de billets de banque basés sur l'analyse neuronale - Google Patents

Procédé et système de selection des paramètres d'un élément de conception ou de sécurité de billets de banque basés sur l'analyse neuronale Download PDF

Info

Publication number
EP3757950B1
EP3757950B1 EP20181632.9A EP20181632A EP3757950B1 EP 3757950 B1 EP3757950 B1 EP 3757950B1 EP 20181632 A EP20181632 A EP 20181632A EP 3757950 B1 EP3757950 B1 EP 3757950B1
Authority
EP
European Patent Office
Prior art keywords
banknote
indicator
security
neurometric
biometric
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP20181632.9A
Other languages
German (de)
English (en)
Other versions
EP3757950A1 (fr
Inventor
María Carmen TORRECILLA MORENO
Mariano Luis ALCAÑIZ RAYA
Jaime Guixeres Provinciale
Javier MARÍN MORALES
Diego ÁLVAREZ RODRÍGUEZ
Fernando LEÓN MARTÍNEZ
José María SÁNCHEZ ECHAVE
Miguel Vicente LÓPEZ SOBLECHERO
Rubén ORTUÑO MOLINERO
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Banco de Espana
Original Assignee
Banco de Espana
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Banco de Espana filed Critical Banco de Espana
Publication of EP3757950A1 publication Critical patent/EP3757950A1/fr
Application granted granted Critical
Publication of EP3757950B1 publication Critical patent/EP3757950B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q90/00—Systems or methods specially adapted for administrative, commercial, financial, managerial or supervisory purposes, not involving significant data processing
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07D—HANDLING OF COINS OR VALUABLE PAPERS, e.g. TESTING, SORTING BY DENOMINATIONS, COUNTING, DISPENSING, CHANGING OR DEPOSITING
    • G07D7/00—Testing specially adapted to determine the identity or genuineness of valuable papers or for segregating those which are unacceptable, e.g. banknotes that are alien to a currency
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07D—HANDLING OF COINS OR VALUABLE PAPERS, e.g. TESTING, SORTING BY DENOMINATIONS, COUNTING, DISPENSING, CHANGING OR DEPOSITING
    • G07D7/00—Testing specially adapted to determine the identity or genuineness of valuable papers or for segregating those which are unacceptable, e.g. banknotes that are alien to a currency
    • G07D7/003—Testing specially adapted to determine the identity or genuineness of valuable papers or for segregating those which are unacceptable, e.g. banknotes that are alien to a currency using security elements

Definitions

  • the present invention relates to the technical field of the neuroanalysis of objects through the processing of biometric signals of users exposed to said objects and, more specifically, to the characterization of banknotes and communication materials based on quantifiable information extracted from biometric signals, which allows banknotes and communication materials relating to banknotes to be classified according to an objective perception of the users of certain parameters concerning the design and security elements.
  • banknotes must correspond to aesthetic and functional criteria providing for the easy recognition thereof, for detection of their authenticity detected or simplified handling, while at the same time meeting a number of technical requirements of the public, of manufacturers and of the issuing authorities.
  • the banknote is a communication medium by itself, whereby the message expressed in the design thereof is intended to be communicated, among the requirements of the public, the intellectual and emotional processes of the public relating to how they perceive said aesthetic and functional aspects must be taken into account so as to ensure that the message integrated in the design of the banknotes is received by the public in a manner that is true to the purpose of the communication of the design.
  • the communication materials must also be produced considering the intellectual and emotional processes of the public so that they generate the greatest communicative impact possible therein, to thus maximize the guarantee that the public has received the educational message and, furthermore, has understood it as it was expected to. For that purpose, it is just as important to assess the content incorporated in the communication materials as in the materials themselves (brochures, images, videos, advertisements, etc.), and as in the distribution of said materials over the different channels of communication (web, radio, TV, written press, etc.).
  • Implicit measurements refer to the methods and techniques capable of capturing or tracking implicit mental processes or the results thereof, including brain images, behavioural monitoring and psychosomatic results.
  • Neuroscience has shown that most of the brain processes that regulate human emotions, attitudes, behaviours and decisions do not involve human consciousness. That is, these implicit processes are brain functions that occur automatically and out of conscious control or awareness; in contrast, explicit processes occur through conscious executive control.
  • the present invention describes a method according to claim 1 for classifying banknotes based on neuroanalysis, comprising the steps of: providing a user with visual information of a banknote; acquiring, by means of a sensor of an input module, at least one biometric signal of the user as a response to the visual information of the banknote; segmenting the acquired biometric signals into predetermined periods of time in a process module; comparing each of the segments with pre-established patterns; identifying certain events as a result of the comparison of each of the segments with the pre-established patterns; obtaining at least one biometric variable based on the identified events; analyzing the biometric variables in the process module according to previously known results stored in a database; establishing a neurometric indicator in the process module based on the preceding analysis; and classifying the banknote in an output module according to the established neurometric indicator.
  • the visual information of the banknote is provided physically, virtually or by means of a combination of the two in a tangible interface on which virtual elements added to a physical banknote by means of augmented reality technology are represented.
  • the biometric signal according to the invention comprises information of at implicit process of the user, being gesture analysis for banknote-hand interaction, eye tracking and facial expression analysis.
  • the biometric signal according to the invention comprises information from a physiological response of the user, to be selected from: a brain response, heart rate variation and skin conductance.
  • biometric variables obtained by the present invention based on the identified events are contemplated to comprise quantifiable information of said identified events, to be selected from: amount of identified events, average duration of the identified events, frequency of each identified event in a pre-established time, sequence of the identified events and number of visits to one same predefined area.
  • the invention contemplates defining at least one area of interest in the banknote and associating the biometric variable acquired from the user with said area of interest.
  • the visual information comprises a security element arranged in the banknote
  • the area of interest is larger than or equal to the area of the banknote occupied by the security element, and wherein the area of interest includes the area of the banknote occupied by said security element. It is thus advantageously possible to assess each of the elements of the banknote separately.
  • analyzing the biometric variables according to previously known results comprises training a supervised learning system of the process module according to the following steps:
  • the possibility of analyzing the biometric variables by means of the supervised learning module of the process module is contemplated, following the steps of: providing the initial value of the neurometric indicator assigned to each banknote in an input of the learning system; applying, through the supervised learning system, a predictive model to the biometric variables obtained by the process module and the assigned initial value; and validating the predictive model, by means of a cross-validation process, with a number of previously determined iterations.
  • neurometric indicators represent one or more of the following cognitive processes in the brain of the user: visual interest, attention, evoked emotions, motivation, mental load, stress and level of arousal.
  • a second aspect of the invention relates to a system according to claim 7 for classifying banknotes based on neuroanalysis, comprising the following elements:
  • the output module has display means configured to visually represent the neurometric indicators of the banknote and a final classification metric, based on the neurometric indicators, which is associated with the visual information of each banknote.
  • the present invention therefore involves a series of advantages over the state of the art.
  • the neuroanalysis carried out by the present invention is highly advantageous for the design and incorporation of security elements in the banknotes, because unlike the known studies in the state of the art, the invention contemplates the integration of metrics which quantify the gestural behaviour in the interaction of the public with the banknote, integrates eye tracking techniques to map fixations on the banknote, brain measurement equipment synchronized with the assessment of each banknote, integrates the heart rate variability signal as an indicator of the impact on the level of valence and arousal of the design of the banknote and, definitively, produces the classification of banknotes based on a precise objective characterization of human perception.
  • the classification of banknotes performed by the method and the system of the present invention follows a process which ensures the reproducibility thereof and comparison between studies that are conducted with the same equipment anywhere around the world.
  • the present invention contemplates the generation of a classifier with a neurobehavioural impact, taking into account metrics coming from the ocular analysis system, physiological metrics and voluntary responses, in order to provide design impact indicators which aid in the comparison of different design parameters for the purpose of determining the design and the security elements that will make up part of a banknote.
  • the present invention discloses a method and a system for classifying banknotes based on neuroanalysis techniques. It thus allows for determining which design and security elements must be integrated in the manufacture of a new banknote and its optimal configuration based on the monitoring of certain conscious and unconscious processes of the public exposed to such elements.
  • the present invention may also be applied to the communication material relating to banknotes, which allows the communication materials to be produced efficiently, emphasizing the main features of the banknotes in informative brochures that are both printed out and can be found on web pages from the issuing authorities (accessible through the web page of the Banco de Espa ⁇ a (Bank of Spain), for example).
  • each of the security features of a banknote (such as, for example, relieves, watermarks, security threads, windows with a portrait, holograms, colours, infrared properties, microtexts or a standard or special response to ultraviolet light) is identified and highlighted both visually and by means of descriptive texts which indicate to the user how to recognize it; therefore, similarly to the case of evaluating a banknote, the application of the present invention relating to said communication material allows for determining the effectiveness thereof of communicating to the public the design and security features integrated in a banknote based on the monitoring of certain conscious and unconscious processes of the public exposed to such communication material.
  • the neurodesign of banknotes according to the present invention may be applied to only one or to all the elements currently integrated in a banknote or communication material. It is preferably applied to security elements because the security of any of the elements implemented in a banknote to ensure the authenticity thereof does not only come from the technical features that are typical of said elements, which prevent or hinder imitation, but also influences the level of security the public perceives.
  • the perception of the public of a security element is essential in improving its efficacy because if a security element, such as a holographic sheet, for example, even in the hypothetical case that the imitation thereof was impossible, were to be integrated in a banknote such that it goes completely unobserved by the user, the effectiveness of said element in the overall integrity of the banknote would be zero.
  • the present invention therefore, increases the efficiency of the elements making up a banknote and communication materials providing an assessment of such elements based on several implicit measurements of the user, which are obtained as a result of a quantification of detected events by means of comparison with certain pre-established patterns of the biometric signals of the user captured by the corresponding sensors arranged in the system.
  • the quantification of conscious and unconscious responses is performed based on neuroscience and behavioural measurement techniques, which are used for inferring, from events detected in the biometric signals, various biometric variables which characterize said signals during the time of exposure of a user to a visual stimulus. These biometric variables are used to check for the existence of patterns in the unconscious responses and the correlations thereof with the assessment of the elements of the banknote under study, obtaining neurometric indicators which classify these elements based on cognitive responses, such as visual interest or workload.
  • the classification of the set of biometric variables into neurometric indicators is performed by supervised learning techniques such as neural networks. Each of these neurometrics is then weighted and fused in a single final metric, based on weights defined by a group of experts, which will allow the design or security element or the entire banknote or communication material being analyzed to be characterized on a general level, which thus enables it to be determined if said element is suitable for being integrated in the banknote, or for being put into circulation in the case of analyzing the entire banknote, or for being disclosed to the public if it is communication material.
  • Figure 1 shows a block diagram including the methodology followed in a complete embodiment of the invention.
  • a configurable neuro-assessment module 2 with a certain configuration which defines the context 21 to be extracted (context may not be provided 211, a real context may be provided 212 or a virtual context may be provided 213 ), the mode of presentation 22 of the banknote (which may be a physical mode of presentation 221 or a virtual mode of presentation 222 ), users 23 who are going to be exposed to the samples and the modes 24 for obtaining responses (contemplating the human behaviour response 241, the physiological response 242 and the voluntary responses 243 ).
  • the metrics obtained at the output of the process module depend entirely on the selected techniques and modes of obtaining user responses.
  • the following responses are contemplated:
  • biometric signals are obtained for each of the users at the input 31 of the neurometric process module 3. Therefore, the input of the neurometric process module groups together the synchronized signals obtained for each user by the corresponding sensors, the signals relating to human behaviour, the signals relating to their physiological response and the signals relating to their voluntary response.
  • conditioning process 32 which may comprise techniques for eliminating noise that may have been generated during the measurement process (particularly relevant in physiological signals), techniques for discarding possible atypical values and techniques for normalising the signal if needed. Conditioning is a necessary process, except for the voluntary responses, prior to extracting relevant metrics from the signals.
  • Each of the signals used receives a specific conditioning as described in detail below.
  • the conditioning comprises four sub-processes mainly consisting of detecting faces, identifying characteristics, identifying actions and identifying emotions.
  • First, all the different frames making up the video obtained are analyzed so as to identify the face of the user by applying computer vision techniques, such as the "Viola Jones Cascaded Classifier” algorithm, for example.
  • a detection of the features of the face is performed using facial coding algorithms, for example the FACS ("Facial Action Coding System") system, which can identify features such as vectors of the eyelids, corners of the mouth, tip of the nose, etc.
  • a point mesh which represents the face of the user is thus created.
  • the so-called “Action Units” are then identified by the FACS system, wherein the fundamental actions of the face are characterized (such as “brow lowering”, “nose wrinkling”, “lip tightening”, “outer brow raising”, etc.).
  • a classifier is applied providing the statistical probability in each instant that one of the basic emotions is being experienced, giving a signal of 0 to 1.
  • the emotions that are comprised include joy, anger, surprise, disgust, fear and sadness.
  • These signals are later corrected using an individualized baseline, using the response of the user to a neutral stimulus, thus minimizing individual biases. Therefore, the facial expression signal is finally made up of six independent signals, individually corrected with a baseline, wherein each of them represents the probability that the subject is experiencing each of the basic emotions in one instant of time.
  • the conditioning is mainly concentrated on the detection of the gestures of the user in the video signal, wherein the hands and interaction of the user with the banknote or communication material are observed.
  • the video is segmented for each of the banknotes or communication materials presented to each user. Then each video segment is analyzed so as to detect one or several events, for example: “the user flips the banknote over”, “the user touches the banknote searching for a distinctive texture”, “the user turns the banknote”, “the user looks at the banknote against the light”, “the user moves the banknote in search of a distinctive sound”, “the user folds the banknote”.
  • the detection of these gestures is preferably performed by means of a semi-manual process which, supported in open source code libraries such as "OpenPose", for characterizing the position of the hands and their phalanges and for performing an initial identification of the gestures described above, adds a manual review to confirm the correct identification of the gestures detected and processed by the algorithms.
  • open source code libraries such as "OpenPose”
  • the signals and measurements related to the brain response 2421 of the user are based on an electroencephalogram (EEG) signal made up of one power signal for each of the electrodes making up the data acquisition hardware.
  • EEG electroencephalogram
  • the data from each channel is analyzed so as to identify damaged channels using the fourth standardized moment (kurtosis) of the signal of each electrode.
  • the channel is also considered damaged if the signal is flatter than 10 % of the total duration thereof. If a channel is considered to be damaged, it can be interpolated from the neighboring electrodes thereof.
  • the baseline of the electroencephalogram signal is eliminated by subtracting it from the mean and setting a bandpass filter between 0.5 and 40 Hz.
  • the resulting signal is then segmented into periods with duration of one second.
  • An automatic detection is applied to reject periods wherein more than two channels contain samples exceeding an absolute threshold, for example, of 100.00 ⁇ V and a gradient of 70.00 ⁇ V between the samples.
  • an independent component analysis ICA is performed in order to identify and eliminate components due to blinking, eye movements and/or muscle movements. Said components are analyzed by means of visual inspection by a trained expert in order to confirm the effectiveness of the algorithms used.
  • conditioning of the signal comprises analyzing an electrocardiogram (ECG) signal, for example through the Pan-Tompkins algorithm for the detection of the QRS interval.
  • ECG electrocardiogram
  • This detection allows a new time series to be obtained which characterizes the electrocardiogram with the time that passes between beats.
  • the detection performed by the Pan-Tompkins algorithm is revised so as to detect ectopic beats and artifacts, and, finally, obtains a series of RR pulsations which include the time difference between two consecutive pulsations and allows the heart rate variability analysis to be performed.
  • the conditioning consists of a visual inspection for the diagnosis and correction of artifacts that may be incorporated in the signal. These artifacts are corrected by first- or second-order linear interpolations. Then the phase component of the clean signal is extracted the signal, which is what is affected by unconscious changes derived from occasional stimuli and is not affected by other changes such as temperature, for example. Lastly, this signal with the phase component is standardized using Venables and Christie formulas in order to eliminate inter-subject differences.
  • the neurometric process module 3 applies, signal by signal, algorithms for the extraction of numerical biometric variables of interest 33 in each of the conditioned signals.
  • the individual biometric variables of each user are obtained and synchronized with the phases of the neuro-assessment of the stimuli considered. This process is repeated by each of the users of the complete sample and by each of the signals recorded in the test analyzed in each case.
  • the values of the biometric variables obtained in this phase generate a metrics database which is the one used in the following phase to extract the neurometric indicators resulting from the neuro-assessment of the banknote.
  • the process follows the diagram of Figure 2 .
  • basic parameters for eye tracking 70 are extracted.
  • the main parameters for the eye tracking which are distinguished between fixations and saccadic eye movements, are extracted.
  • Fixations are understood as the instants wherein the eye is focusing on the visual scene to cause the visual information to reach the brain.
  • Saccadic eye movements are understood as the movement of the eyes with the aim of refocusing on another new point of visual interest.
  • an algorithm for detecting the raw signal is applied in order to extrapolate if the analyzed sample is part of a fixation or a saccadic eye movement.
  • the most widely-used algorithm is based on eye speed.
  • each piece of raw data of the present embodiment is classified as one of the following two states: ⁇ ⁇ ⁇ ⁇ 100 ° / sec ⁇ the piece of data is part of a saccadic eye mov . ⁇ ⁇ ⁇ 100 ° / sec ⁇ the piece of data is part of a fixation
  • the corrections are calculated through the groups of samples defined by the fixation, provided that the duration reaches a minimum established, for example, 100 ms.
  • the fixation position is defined by the average position of the samples associated with that fixation.
  • the lengths of the saccadic eye movements are defined by the distance between continuous fixations.
  • a division of the saccadic eye movements is contemplated, which is applicable to the viewing of the banknote or communication material which divides these movements of the eye into ambient saccadic eye movements (which scan the banknote entirely) or focal saccadic eye movements (which move around a specific area of the banknote of interest).
  • the following thresholds are applied for differences between ambient and focal movements: ⁇ ⁇ ⁇ 4.8 ° ⁇ ambient saccadic eye mov . ⁇ ⁇ 4.8 ° ⁇ focal saccadic eye mov .
  • a semi-automatic method is generated which helps transfer the coordinate system for fixations and saccadic eye movements from a 3D system to a 2D system, particularly centring on the banknote being assessed in each instant of the test.
  • Said method applies algorithms for segmenting and identifying objects by image analysis in order to identify the banknote under study in space, such that the position of the banknote in the 3D coordinate system is known at all times.
  • the position of the eye of the user is monitored in that same 3D coordinate system, whereby it is possible to perform the pairing of both values in a 2D space wherein the banknote can be represented as an image on both faces onto which the obtained fixations and saccadic eye movements can be projected.
  • the method for extracting metrics from the signal for eye tracking contemplates an extraction 73 of metrics relative to the visual attention of the user on the entire banknote or communication material in general. These metrics will take one or more of the following events into account: “fixations on the whole banknote (on both faces)”, “saccadic eye movements over the banknote”. “blinking when viewing the banknote (measurement usually provided by the eye tracking equipment)”, “size of the pupil (measurement usually provided by the eye tracking equipment)”.
  • the identification of events in the signal for eye tracking promotes the application of a series of mathematical operations to translate those events into quantifiable information, comprising for example: counting the amount of events (fixations, saccadic eye movements, blinking) taking place within the entire banknote (on both faces); counting the average time each event lasts; counting the frequency of these events in a defined period of time; or obtaining the sequence of these events.
  • the method for extracting metrics from the signal for eye tracking contemplates an extraction 74 of metrics relative to the visual attention of the user on said predefined areas of interest.
  • an additional basic parameter associated with the areas of interest which is the term "visit" is previously calculated. "Visit' is understood as a type of event which includes more than one continuous fixation and that the time between fixations does not exceed a pre-established time threshold, for example one second.
  • the extraction of numerical biometric variables of interest 33 for the specific case of the signal for facial expression 2412 comprises characterizing the response of each user to each of the banknotes shown from several independent identified and processed emotions.
  • the signals are segmented according to the presentation time of the stimuli, extracting several independent signals which characterize each banknote.
  • Three types of variables are obtained from these signals: the first ones are general metrics, computing the mean of the signal in the stimulus (e.g.
  • the second ones are metrics based on thresholds, wherein there a function is applied to each signal which analyses if the probability of feeling a particular emotion is greater than X, in order to subsequently calculate the percentage of time that the subject has been above said threshold, wherein said threshold can be defined in two levels, for example 0.5 to detect the percentage of time that the subject has been experiencing that emotion, regardless of the intensity, and 0.8 to calculate the percentage of time that the subject has been intensely experiencing that emotion;
  • the third type of metrics are ratio metrics, such as the ratio between positive and negative emotions, for example.
  • the number of times a gesture is made during the viewing of a banknote, and the percentage it represents with respect to the total number of gestures is counted from the conditioned signal.
  • the extraction of numerical biometric variables of interest comprises, from the conditioned signals after the conditioning process 32, a spectral analysis of the encephalogram signal to estimate the spectral power in each second, in the conventional frequency band: ⁇ (4-8 Hz), ⁇ (8-12 Hz), ⁇ (13-25 Hz), ⁇ (25-40 Hz).
  • those metrics which characterize cognitive states are also calculated.
  • These variables use previously trained classifiers which, from the initial tasks the user must perform to calibrate the classifier, allow the level of "engagement” and of "workload” to be predicted. "Engagement' reflects the general level of engagement, commitment, attention and concentration during the visual scanning of the user to gather information, whereas "workload” is understood as any cognitive process involving an executive process, such as analytical reasoning, problem-solving or working memory, for example.
  • the extraction of numerical biometric variables of interest 33 for the specific case of the signal for heart rate variability 2422 comprises three types of variables: variables derived from the time domain, variables derived from the frequency domain and variables which quantify non-linear dynamics.
  • the analysis in the time domain includes the following characteristics: mean and standard deviation of RR intervals, the root mean square of the sum of squares of the differences between adjacent RR intervals (RMSSD), the number of successive differences of intervals differing by more than 50 ms (pNN50), the triangular interpolation of the heart rate variability (HRV) histogram and the baseline width of the RR histogram assessed by means of triangular interpolation (TINN).
  • Frequency domain characteristics are calculated using the power spectral density (PSD), applying the fast Fourier transform.
  • the analysis is performed in three bands: VLF (very low frequency, ⁇ 0.04 Hz), LF (low frequency, 0.04-0.15 Hz) and HF (high frequency, 0.12-0.4 Hz).
  • VLF very low frequency
  • LF low frequency
  • HF high frequency, 0.12-0.4 Hz
  • the maximum value corresponding to the frequency having the maximum magnitude
  • the normalized power n.u.
  • the LF/HF ratio is calculated to quantify sympathovagal balance and to reflect sympathetic modulations. Furthermore, total power is calculated.
  • a Poincaré plot analysis is applied, which is a visual and quantitative technique in which the shape of a frame is classified into functional classes, providing summarized information about the behaviour of the heart.
  • a transverse axis (SD1) is associated with a rapid, beat-to-beat variability and a longitudinal axis (SD2) analyses long-term R-R variability.
  • An entropy analysis is further included, using methods existing in the state of the art such as "Sample entropy" (SampEn), "Approximate entropy” (ApEn) and DFA correlations.
  • EDA electrodemal activity
  • the first type is made up of the average of the signal in the segment of each stimulus, whereas the second type of variable analyses the peaks experienced by the user during the viewing of the banknote. These peaks will be characterized by the number of peaks per minute and the average amplitude thereof.
  • Examples of interviews and questionnaires that are conducted include following: after the user views each banknote (front and back) on the monitor, there are questions about certain semantic axes such as aesthetics, quality, design, durability, pleasure or emotional aspects, in addition to an assessment and unconscious association of open attributes for each of the banknotes; after the user views all the banknotes, a questionnaire is completed comprising questions to know which banknotes and security elements are remembered, in which part of the banknote a certain security element is located, or what content the communication material incorporates, and recognition questions showing images of banknotes, asking the user whether or not they were shown during the test; after the user physically interacts with each banknote, a questionnaire is completed to assess the medium of the banknote (paper, plastic or variants thereof) or of the communication material and attributes similar to the previous phase, but adding attributes related to the feel of the banknote such as the geometry, texture, sound and/or relief.
  • certain semantic axes such as aesthetics, quality, design, durability, pleasure or emotional aspects, in addition to an assessment and unconscious association of open attributes for
  • the neurometric process module 3 of the present invention applies a classification algorithm in a predictive module 34 in order to obtain at the output a set of neurometric indicators 4 of the neuro-assessment of the user.
  • Figure 3 comprises a block diagram which represents the two parts into which the calibration is divided: first the generation of a ground truth 300 , and then the creation of the predictive model 310.
  • the biometric variables 33 obtained for a set of banknotes for example one hundred banknotes, will be used.
  • the set of banknotes comprises the broadest possible range of responses on a cognitive, emotional and behavioural level. This set is preferably chosen by a multidisciplinary team of experts selected from different fields/sectors (such as banking, psychology or neuroscience) and contains both real banknotes and ad-hoc designs which guarantee a wide range of responses.
  • the group of experts only selects 301 the biometric variables related to the neurometric indicator, from the set of neurometric indicators 4, being generated at all times (some examples of the relationship between the selected biometric variables and the different neuro-assessment indicators are included below).
  • an unsupervised clustering-type (k-means) machine learning algorithm is applied for grouping together 302 the banknotes based on their responses.
  • the one hundred banknotes are thereby divided into different groups according to the response thereof in the different metrics forming the indicators.
  • the mean of each group which represents the average response in each group is then calculated.
  • the team of experts validates 308 the groups and analyses 303 the responses of each group in depth from the mean thereof and assigns a value 304 of the indicator to this group of banknotes, for example following a Likert scale from 1 to 5.
  • the classification model is created 310.
  • a dataset is created in which the inputs are the biometric variables selected 301 and the output is the value already assigned 304 to the corresponding neurometric indicator.
  • the predictive model 306 is designed with this dataset based on artificial neural networks.
  • the training 305 of the neural network which is fed with the selected metrics 301 and the assigned values 304, is validated 307 by applying a cross-validation algorithm of k-iterations with a k of 10, and the model is then tested with 15% of the sample, which was previously extracted from the validation process.
  • the predictive model is validated and tested 306, it may be applied to the biometric variables of any banknote, providing an assessment in each of the neurometric indicators.
  • the output of the predictive module 34 comprises the indicators generated according to the obtained predictive models which are applied to the numerical biometric variables of interest 33 and produce as a result a value for each of the indicators of the neuro-assessment of each banknote for each user.
  • Figure 4 shows a diagram with the measured signals of each user to be taken into account for the calculation of certain indicators.
  • a first visual interest indicator 41 BVIS
  • the human behaviour responses 241 represented by the eye tracking signals 2411 and facial expression analysis 2412 are considered relevant; none of the physiological responses 242 is necessary, and voluntary responses in the form of an interview 2433, questionnaires 2434 and response to tasks 2431 are indeed taken into account;
  • a second engagement indicator 42 BEI
  • the human behaviour response 241 represented by the eye tracking signals 2411, the physiological responses 242 represented by the brain response 2421 and the heart rate variability 2422, as well as the voluntary responses in the form of questionnaires 2434 are considered relevant;
  • a third workload indicator of 43 BWI
  • the human behaviour responses 241 represented by the eye tracking signals 2411, for facial expression analysis 2412 and for user behaviour tracking 2413 the physiological responses 242 represented by the brain response 2421 and the voluntary responses in the form of response to tasks
  • the visual interest indicator 41 is a metric related to the visual interest the design of the banknote arouses. This high level metric is centred on a non-linear model establishing a visual interest score which the perception of the design of the banknote generates and which allows for comparison between different design types.
  • the indicator is calculated through supervised learning techniques applied to the biometric variables of interest 33, extracted from the selected conditioned signals which contain quantifiable information specifically comprising in this embodiment:
  • some values relative to the voluntary response are contemplated as a global assessment of the design of the assessed banknotes; recall of the banknotes and of areas of interest of the banknote; and times allocated for performing the tasks of assessing the banknote.
  • BEI Banknote Engagement Index
  • the workload indicator 43 refers to the cognitive load or mental effort involved for the subject in the process of perceiving and assessing certain attributes of the banknote or communication material. It is very important because a high cognitive load may mean that there is a saturation of information, which leads to rejection, but at the same time a low value may indicate boredom of the subject, which is also negative.
  • a cognitive indicator combining the two aforementioned indicators, i.e., engagement indicator 42 BEI and workload indicator 43 BWI, is contemplated.
  • the emotional indicator 44, BEll ( "Banknote Emotional Induction Index” ) , used in one of the embodiments of the invention is a metric relative to the capacity of emotional induction of the banknote or communication material.
  • the indicator BEI is based on the calculation and representation of a point on a two-dimensional spatial axis in which the capacity of emotional excitation ( arousal ) and the capacity to generate a positive or negative emotion (valence) is extracted.
  • the processing of the signal from behavioural measurements micro facial expressions during banknote viewing
  • the physiological response cerebral hemisphere asymmetry, cardiac variability and skin conductance
  • the security indicator 45 used in one of the embodiments of the invention is a metric relative to banknote security. Namely, this indicator reflects the capacity of the design and security elements of the banknote for being authenticated by the public. The calculation thereof is based on several parameters relative to the behavioural signal (e.g. eye tracking of the security elements of the banknote, automatic tracking of the gestures of the participant interacting with the banknote) and voluntary response values of the subject. Through the modelling of these parameters, an absolute index can be obtained that allows for the comparison of new security elements and designs in a single banknote or the comparison of current security elements and designs of different types of banknotes.
  • BSCI "Banknote Security Capacity Index”
  • the neurometric indicators 4 are statistically treated in order to satisfactorily characterize a banknote or communication materials.
  • the general response of the banknote or communication materials is measured using data aggregation techniques (for example the arithmetic mean or standard deviation), and on the other hand, based on specific conditions and cases, different additional analyses are carried out in order to determine if there are significant differences that may allow final conclusions to be inferred relative to the objective of the neuro-assessment study.
  • correlation techniques and clustering techniques can be used. All this statistical analysis is implemented automatically, ensuring reproducibility and the comparison of the same studies contemplated on several dates and in several locations. Therefore, the statistical inference analysis extracts the significant differences in the biometric variables with numerical metrics of interest 33.
  • different models such as analysis of variance or the Kruskal-Wallis test, for example, the indicators calculated according to different clusters are compared.
  • the output module 5 calculates a final metric which encompasses all the calculated indicators and offers a snapshot of the performance of the banknote or communication material, allowing for a rapid assessment, comparison and classification compared to other assessed banknotes.
  • This final metric is based on a score of 1 to 10 through a mathematical equation in which each of the calculated neurometric indicators has an influence with a specific weight.
  • the model recalculates the value by cancelling out the impact of the value of that neurometric indicator.
  • the indicator is thereby dynamic and only reflects the indicators that are of interest in each specific case (for example, the preceding final score may be recalculated so that it only reflects the impact of the visual and cognitive indicators or even just one of them).
  • One of the embodiments contemplates graphic representation, for example by means of heat maps, two-dimensional axes, curves or percentages, of all the biometric variables, neurometric indicators and statistical inferences obtained during the process carried out by each of the modules of the invention.
  • Figure 5 represents one of these particular views, wherein a face of a banknote is represented, and associated with each of the defined areas of interest, the values of the indicators (not shown in the figure) obtained for said areas of interest are represented.
  • a defined area of interest to comprise a security element incorporated in the banknote, such as a hologram 52, a watermark 53, a special printing ink 54 or a window 55
  • the represented indicators code the neuro-assessment obtained from the perception of the users of that security element.
  • each of the areas of interest is associated with a percentage score of the visit time, visitors and revisits, which is furthermore complemented by a heat map and the sequence of visits of the different areas of interest. For example, after the analysis of the area of interest including the hologram 52, a visit time of 14.92 % of the total time spent on inspecting the banknote, 86.53 % of users who have observed it and 78.72 % of users who have revisited it is obtained. This type of measurements are what make it possible to construct the indicators for comparison between banknotes, comparison of elements and classification.
  • the present invention classifies in the output module 5 a complete sample of banknotes according to the obtained indicators associated with the areas of interest comprising the security elements.
  • the security level of the security elements is determined by the perception of the public and is a determining factor for assessing the incorporation thereof in future legal banknotes.
  • the classification of banknotes based on the perception of the users of the security elements allows security elements to be selected that are acceptable and unacceptable for being incorporated in legal currency, establishing a minimum threshold in the indicators for determining that the perception of the public of the security element is sufficient for it to be incorporated in the banknote.
  • These minimum thresholds may be calibrated using modified security elements and analyzing how the perception of the users varies with respect to the modifications of different security elements.
  • the modified security elements that obtain a better classification in the perception of the users will thus be the security elements that are most suitable for being incorporated in legal banknotes.
  • the eye tracking signals for example, the number of revisits of the user to the security element or the time used in viewing said element with respect to the rest of the banknote is a determining factor.
  • the colour of the banknote with respect to the perceived security of a certain security element. If the objective is to determine the colour of the banknote providing the most security, the set of banknotes that will be subjected to neuroanalysis will differ only in the colour of the design thereof, but the security elements will be kept intact.
  • the neuroanalysis of the perception of the users will allow it to be determined if colour variations have an influence on the perception of the security elements, characterizing the different banknotes based on the perception of the users and finally classifying them in an orderly and objective manner, with the best classified banknote being the banknote corresponding to the colour that is most suitable for security of the banknote.
  • a grey colour for the banknote could largely cancel out the security of a hologram element or a security thread element with a metallic appearance, which would be virtually camouflaged and go unnoticed for a user.
  • the classification will indicate how each of the test colours disturbs the perception of the security elements integrated in the banknote, whereby the final classification determines the colour to be included in the banknote to be manufactured.
  • the banknote samples and the areas of interest are selected so that precisely those parameters are what vary from one banknote to another, and similarly to the preceding case, the characterization of the perception of the users indicates in an objective manner the influence that said parameters have on the banknote. For example, by defining an area of interest 56 including the value of the banknote (50 Euros for example), it is interesting to compare the influence that different sizes and positions have compared to the perception of the design and security elements of the banknote.
  • the perceived security of the watermark 53 may be affected starting from a certain size of the representation of the value of the banknote, or a position that is too close, because it attracts the visual attention of the user in excess or would cancel out or reduce the perception of the watermark, which reduces the security of the banknote in the opinion of the user.
  • Even other elements of the banknote which, outwardly, have no more than a merely aesthetic function, such as the decoration included in the area of interest 57, are also important in the global assessment of the banknote, and the colour, size or position thereof may influence the security it has, for which reason in one of the embodiments the analysis of absolutely all the elements of the banknote is contemplated.
  • Figure 6 schematically shows of the possibilities of presenting objects for the neuro-assessment of the present invention, preferably banknotes or communication materials, both in a real format and in a virtual format.
  • the samples of banknotes or communication materials to be analyzed comprise different security features, design features or contents of the communication materials according to, among others, different materials, designs, sizes and positions, which influence the perception which the public has of the banknote.
  • the context of the samples of banknotes can be presented to the user by means of different techniques 21, which include not providing any context 211, adding real context 212 or adding a virtual context 213 wherein, by using computer and digital graphics techniques, different scenarios are reproduced, among which the following are contemplated: a virtual reality scenario, wherein the assessment configuration is used in laboratory conditions under a virtual replica of the real world, which may consist of two-dimensional (2D) models of the real context; an augmented reality scenario, wherein the configuration of the assessment is used in real life conditions, but completed with some virtual elements in 3D; and an augmented reality scenario, wherein the configuration of the assessment is used in laboratory conditions, but an augmented virtual replica of the real context is presented to the user.
  • a virtual reality scenario wherein the assessment configuration is used in laboratory conditions under a virtual replica of the real world, which may consist of two-dimensional (2D) models of the real context
  • an augmented reality scenario wherein the configuration of the assessment is used in real life conditions, but completed with some virtual elements in 3D
  • the context may be provided by means of one or a combination of the following immersive interfaces: visual devices (such as conventional monitors, vertically positioned monitors with stereoscopic 3D vision and 3D tracking of the position of the main user ( "fish tank " interface), horizontally positioned monitor with stereoscopic 3D vision and 3D tracking of the position of the main user ( “workbench” interface), surround displays made up of large displays based on projection and/or large monitors, hemispherical exhibits, or virtual reality headsets (HMD-Head Mounted Displays) and/or augmented reality and/or mixed reality); audio displays (wherein, for example, contextual sounds are reproduced using 3D sound generation techniques with headphones and/or external speakers); olfactory displays (wherein aromas are delivered through electronic noses and/or any commercial olfactory display); or haptic displays (where tactile and kinesthetic signals are provided through a tactile haptic display device, such as land references, body references, tactile
  • the present invention also contemplates several alternatives shown in Figure 6 .
  • two techniques are used based on the reliability thereof for reproducing real-life situations: using a physical banknote 221, wherein a real physical model of the banknote is presented to the user; or using a digital banknote 222, wherein a digital replica of the banknote is presented using a virtual banknote model which reproduces, in two or 3 dimensions, a digital image of the real banknote, or in a virtual banknote model based on a tangible interface which the user can feel with his or her hands.
  • This tangible interface may represent in three dimensions the graphic elements in the physical paper using spatial augmented reality techniques.
  • the final result of the overlay techniques can be presented to the user by means of a virtual reality headset or devices of this type may alternatively be dispensed with and digital projectors showing the information directly on the physical banknote may be chosen.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Engineering & Computer Science (AREA)
  • Economics (AREA)
  • General Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Inspection Of Paper Currency And Valuable Securities (AREA)

Claims (8)

  1. Procédé de sélection de paramètres d'au moins un élément de sécurité ou de conception à incorporer dans la fabrication d'un billet de banque en vue de faciliter la reconnaissance du billet de banque et d'augmenter par conséquent la sécurité, caractérisé en ce qu'il comprend les étapes ci-dessous consistant à :
    a) fournir à un utilisateur une pluralité de billets de banque (1) qui diffèrent par au moins un paramètre d'au moins un élément de sécurité ou de conception (52, 53, 54, 55, 56, 57) du billet de banque ;
    b) pour chaque billet de banque, définir au moins une zone d'intérêt qui inclut la zone du billet de banque occupée par ledit au moins un élément de sécurité ou de conception ;
    c) acquérir, par le biais d'un moyen de capteur intégré dans un module d'entrée, des signaux biométriques de suivi oculaire (2411), de comportement gestuel (2413), d'analyse d'expression faciale (2412), de réponse cérébrale (2421), de variabilité de rythme cardiaque (2422) et de conductance cutanée (2423), en réponse à une stimulation visuelle, tactile et sonore de l'utilisateur avec la pluralité de billets de banque, ainsi que des réponses explicites à des questionnaires soumis ;
    d) conditionner (32) les signaux biométriques acquis, et notamment segmenter les signaux biométriques acquis en des périodes de temps prédéterminées dans un module de processus (3), dans lequel lesdits signaux biométriques sont associés à au moins une zone d'intérêt définie ;
    e) comparer chacun des segments de signaux biométriques à des motifs préétablis et identifier certains événements conséquemment à la comparaison ;
    f) obtenir au moins une variable biométrique (33) sur la base des événements identifiés ;
    g) analyser les variables biométriques dans le module de processus (3), selon des résultats connus précédemment qui sont stockés dans une base de données, en faisant appel à des techniques d'apprentissage automatique, et générer des indicateurs neurométriques partiels (4) associés à chaque billet de banque, comprenant un indicateur d'intérêt visuel d'utilisateur (41), un indicateur d'engagement (42), un indicateur de charge de travail (43), un indicateur émotionnel (44) et un indicateur de sécurité (45) ;
    h) établir un indicateur neurométrique global en conséquence d'une pondération des indicateurs neurométriques partiels ; et
    i) sélectionner les paramètres dudit au moins un élément de sécurité ou de conception à incorporer dans la fabrication du billet de banque selon l'indicateur neurométrique global établi pour chaque billet de banque.
  2. Procédé selon la revendication 1, dans lequel la pluralité de billets de banque est fournie sous une forme physique, sous une forme virtuelle ou au moyen d'une combinaison de ces deux formes, dans une interface tangible sur laquelle sont représentés des éléments virtuels ajoutés aux billets de banque physiques au moyen d'une technologie de réalité augmentée.
  3. Procédé selon l'une quelconque des revendications précédentes, dans lequel ladite au moins une variable biométrique comprend des informations quantifiables concernant les événements identifiés à sélectionner parmi : une quantité d'événements identifiés, une durée moyenne des événements identifiés, une fréquence de chaque événement identifié au cours d'un temps préétabli, une séquence des événements identifiés et un nombre de visites dans une même zone prédéfinie.
  4. Procédé selon l'une quelconque des revendications précédentes, dans lequel l'étape d'analyse des variables biométriques selon des résultats connus précédemment comprend en outre l'étape consistant à entraîner un système d'apprentissage supervisé du module de processus selon les étapes ci-dessous consistant à :
    - répéter les étapes a) à c) de la revendication 1 pour une pluralité de billets de banque différents et d'utilisateurs différents ;
    - pour chaque billet de banque, regrouper les événements identifiés de chaque utilisateur selon un nombre de groupes préalablement établi ; et
    - attribuer une valeur initiale de l'indicateur neurométrique à chaque billet de banque, dans laquelle ladite valeur est basée sur une analyse des groupes d'événements identifiés effectuée par un utilisateur expert.
  5. Procédé selon la revendication 4, dans lequel l'étape d'analyse des variables biométriques au moyen du système d'apprentissage supervisé comprend en outre les étapes ci-dessous consistant à :
    - fournir la valeur initiale de l'indicateur neurométrique attribuée à chaque billet de banque, dans une entrée du système d'apprentissage supervisé ;
    - appliquer, par l'intermédiaire du système d'apprentissage supervisé, un modèle prédictif aux variables biométriques obtenues par le module de processus et à la valeur initiale attribuée ; et
    - valider le modèle prédictif, au moyen d'un processus de validation croisée, avec un nombre d'itérations préalablement déterminées.
  6. Procédé selon l'une quelconque des revendications précédentes, dans lequel les indicateurs neurométriques représentent un ou plusieurs des processus cognitifs suivants dans le cerveau de l'utilisateur : l'intérêt visuel, l'attention, les émotions évoquées, la motivation, la charge mentale, le stress et le niveau d'éveil.
  7. Système destiné à sélectionner des paramètres d'au moins un élément de sécurité ou de conception à incorporer dans la fabrication d'un billet de banque en vue de faciliter la reconnaissance du billet de banque et d'augmenter par conséquent la sécurité, caractérisé en ce qu'il comprend :
    - un module d'entrée (2) comprenant un moyen de capteur, configuré de manière à acquérir des signaux biométriques de suivi oculaire (2411), de comportement gestuel (2413), d'analyse d'expression faciale (2412), de réponse cérébrale (2421), de variabilité de rythme cardiaque (2422) et de conductance cutanée (2423), en réponse à une stimulation visuelle, tactile et sonore d'un utilisateur avec une pluralité de billets de banque (1) qui diffèrent par au moins un paramètre d'au moins un élément de sécurité ou de conception (52, 53, 54, 55, 56, 57) du billet de banque, ainsi que des réponses explicites à des questionnaires soumis ;
    - un module de processus (3) configuré de manière à : définir au moins une zone d'intérêt sur chaque billet de banque qui inclut la zone du billet de banque occupée par ledit au moins un élément de sécurité ou de conception ; conditionner les signaux biométriques, et notamment segmenter des signaux biométriques en des périodes de temps prédéterminées, dans lequel lesdits signaux biométriques sont associés à au moins l'une des zones d'intérêt définies ; comparer chacun des segments des signaux biométriques à des motifs préétablis ; identifier certains événements conséquemment à la comparaison ; obtenir au moins une variable biométrique (33) sur la base des événements identifiés ; analyser les variables biométriques selon des résultats connus précédemment qui sont stockés dans une base de données, en faisant appel à des techniques d'apprentissage automatique ; générer des indicateurs neurométriques partiels (4) associés à chaque billet de banque, comprenant un indicateur d'intérêt visuel d'utilisateur (41), un indicateur d'engagement (42), un indicateur de charge de travail (43), un indicateur émotionnel (44) et un indicateur de sécurité (45) ; et établir un indicateur neurométrique global en conséquence d'une pondération des indicateurs neurométriques partiels ; et
    - un module de sortie (5) configuré de manière à sélectionner les paramètres dudit au moins un élément de sécurité ou de conception à incorporer dans la fabrication du billet de banque selon l'indicateur neurométrique global établi pour chaque billet de banque.
  8. Système selon la revendication 7, dans lequel le module de sortie comprend un moyen d'affichage configuré de manière à représenter visuellement les indicateurs neurométriques du billet de banque et une métrique de classification finale basée sur les indicateurs neurométriques résultant de l'analyse neuronale de chaque billet de banque.
EP20181632.9A 2019-06-26 2020-06-23 Procédé et système de selection des paramètres d'un élément de conception ou de sécurité de billets de banque basés sur l'analyse neuronale Active EP3757950B1 (fr)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
ES201930590A ES2801024A1 (es) 2019-06-26 2019-06-26 Metodo y sistema de clasificacion de billetes basado en neuroanalisis

Publications (2)

Publication Number Publication Date
EP3757950A1 EP3757950A1 (fr) 2020-12-30
EP3757950B1 true EP3757950B1 (fr) 2023-08-02

Family

ID=71138529

Family Applications (1)

Application Number Title Priority Date Filing Date
EP20181632.9A Active EP3757950B1 (fr) 2019-06-26 2020-06-23 Procédé et système de selection des paramètres d'un élément de conception ou de sécurité de billets de banque basés sur l'analyse neuronale

Country Status (4)

Country Link
EP (1) EP3757950B1 (fr)
DK (1) DK3757950T3 (fr)
ES (2) ES2801024A1 (fr)
WO (1) WO2020260735A1 (fr)

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1924941A2 (fr) * 2005-09-16 2008-05-28 Imotions-Emotion Technology APS Système et méthode de détermination de l'émotion humaine par analyse des propriétés de l'oeil
JP5249223B2 (ja) * 2006-09-07 2013-07-31 ザ プロクター アンド ギャンブル カンパニー 感情反応及び選択傾向を測定するための方法
WO2011133548A2 (fr) * 2010-04-19 2011-10-27 Innerscope Research, Inc. Procédé de recherche par tâche d'imagerie courte
US10430810B2 (en) * 2015-09-22 2019-10-01 Health Care Direct, Inc. Systems and methods for assessing the marketability of a product

Also Published As

Publication number Publication date
DK3757950T3 (en) 2023-11-06
ES2963861T3 (es) 2024-04-02
ES2801024A1 (es) 2021-01-07
WO2020260735A1 (fr) 2020-12-30
EP3757950A1 (fr) 2020-12-30

Similar Documents

Publication Publication Date Title
US20200050837A1 (en) System and method for detecting invisible human emotion
Palmer et al. Face pareidolia recruits mechanisms for detecting human social attention
Groen et al. From image statistics to scene gist: evoked neural activity reveals transition from low-level natural image structure to scene category
Wang et al. An EEG-based brain–computer interface for dual task driving detection
Nuamah et al. Support vector machine (SVM) classification of cognitive tasks based on electroencephalography (EEG) engagement index
Abouelenien et al. Human acute stress detection via integration of physiological signals and thermal imaging
RU2708807C2 (ru) Алгоритм комплексного дистанционного бесконтактного мультиканального анализа психоэмоционального и физиологического состояния объекта по аудио- и видеоконтенту
Jin et al. Driver sleepiness detection system based on eye movements variables
US20120084139A1 (en) Systems and methods to match a representative with a commercial property based on neurological and/or physiological response data
US20220133194A1 (en) Measuring and strengthening physiological/neurophysiological states predictive of superior performance
Georges et al. UX heatmaps: mapping user experience on visual interfaces
Tabassum et al. Non-intrusive identification of student attentiveness and finding their correlation with detectable facial emotions
Hossain et al. Observer’s galvanic skin response for discriminating real from fake smiles
Fabiano et al. Gaze-based classification of autism spectrum disorder
Moshel et al. Are you for real? Decoding realistic AI-generated faces from neural activity
Patt et al. Disentangling working memory processes during spatial span assessment: A modeling analysis of preferred eye movement strategies
CN116348042A (zh) 用於量化注意力的方法及系統
Masui et al. Measurement of advertisement effect based on multimodal emotional responses considering personality
Hu et al. First impressions: Integrating faces and bodies in personality trait perception
EP3757950B1 (fr) Procédé et système de selection des paramètres d'un élément de conception ou de sécurité de billets de banque basés sur l'analyse neuronale
Goovaerts et al. Advanced EEG Processing for the Detection of Drowsiness in Drivers.
Liu Towards practical driver cognitive load detection based on visual attention information
Keskinarkaus et al. Pain fingerprinting using multimodal sensing: pilot study
Mantri et al. Real time multimodal depression analysis
Kim et al. A-Situ: a computational framework for affective labeling from psychological behaviors in real-life situations

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20210602

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20230420

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

RIN1 Information on inventor provided before grant (corrected)

Inventor name: ORTUNO MOLINERO, RUBEN

Inventor name: LOPEZ SOBLECHERO, MIGUEL VICENTE

Inventor name: SANCHEZ ECHAVE, JOSE MARIA

Inventor name: LEON MARTINEZ, FERNANDO

Inventor name: ALVAREZ RODRIGUEZ, DIEGO

Inventor name: MARIN MORALES, JAVIER

Inventor name: GUIXERES PROVINCIALE, JAIME

Inventor name: ALCANIZ RAYA, MARIANO LUIS

Inventor name: TORRECILLA MORENO, MARIA CARMEN

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602020014780

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

Effective date: 20231101

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231202

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231102

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230802

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230802

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231202

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230802

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230802

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230802

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230802

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231204

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230802

REG Reference to a national code

Ref country code: AT

Ref legal event code: UEP

Ref document number: 1595691

Country of ref document: AT

Kind code of ref document: T

Effective date: 20230802

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2963861

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20240402

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230802

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230802

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230802

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230802

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230802

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602020014780

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT