WO2024071027A1 - 脳情報の分析によるレコメンデーション - Google Patents

脳情報の分析によるレコメンデーション Download PDF

Info

Publication number
WO2024071027A1
WO2024071027A1 PCT/JP2023/034704 JP2023034704W WO2024071027A1 WO 2024071027 A1 WO2024071027 A1 WO 2024071027A1 JP 2023034704 W JP2023034704 W JP 2023034704W WO 2024071027 A1 WO2024071027 A1 WO 2024071027A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
digital data
data
information processing
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/034704
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
望 窪田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Creator S Next
Creator's Next Inc
Original Assignee
Creator S Next
Creator's Next Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Creator S Next, Creator's Next Inc filed Critical Creator S Next
Publication of WO2024071027A1 publication Critical patent/WO2024071027A1/ja
Priority to US19/072,711 priority Critical patent/US20250201383A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/70—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
    • G16H10/20—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for electronic clinical trials or questionnaires
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/67—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for remote operation
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns

Definitions

  • the present invention relates to an information processing method, a storage medium, and an information processing device that can provide recommendations based on the analysis of brain information.
  • one of the objectives of the present invention is to provide a mechanism that uses brain data to more appropriately select or generate content that matches a user's preferences.
  • an information processing method includes one or more processors included in an information processing device, which execute the following steps: generate predetermined digital data using a generator that generates digital data; input bioinformation of a user stimulated using the predetermined digital data, the bioinformation being acquired by a bioinformation measuring device worn by the user, to a classifier that uses a learning model in which the emotion or state of the user based on the bioinformation of the user is learned using a neural network; obtain a classification result of the predetermined digital data; instruct the generator to generate digital data if the classification result indicates discomfort; and, if the classification result indicates comfort, output information indicating that the predetermined digital data is comfortable for the user.
  • the present invention provides a mechanism that uses brain data to more appropriately select or generate content that matches a user's preferences.
  • FIG. 1 is a diagram illustrating an example of a system configuration according to each embodiment.
  • FIG. 2 is a diagram illustrating an example of a physical configuration of an information processing device of a server according to each embodiment.
  • FIG. 2 is a diagram illustrating an example of a processing block of the information processing device according to the first embodiment.
  • FIG. 2 is a diagram showing a state of a user according to the first embodiment;
  • FIG. 4 is a diagram showing an example of associated data according to the first embodiment;
  • 5 is a flowchart illustrating an example of processing of the information processing device according to the first embodiment.
  • FIG. 11 is a diagram illustrating an example of a processing block of an information processing device according to a second embodiment.
  • 13 is a flowchart illustrating an example of processing by an information processing device according to a second embodiment.
  • Fig. 1 is a diagram showing an example of a system configuration according to each embodiment.
  • a server 10 and each of vital information measuring devices 20A, 20B, 20C, and 20D are connected so as to be able to transmit and receive data via a network.
  • the vital information measuring devices are not to be individually distinguished, they are also referred to as vital information measuring devices 20.
  • the server 10 is an information processing device capable of collecting and analyzing data, and may be composed of one or more information processing devices.
  • the bioinformation measuring device 20 is a measuring device that measures bioinformation such as brain activity, heart rate, pulse rate, and blood flow.
  • the electroencephalograph is a measuring device having invasive or non-invasive electrodes that sense brain activity.
  • the electroencephalograph may be any device that has electrodes, such as a head-mounted or earphone type.
  • the bioinformation measuring device 20 may be a device that includes this electroencephalograph and is capable of analyzing, transmitting, and receiving brain information.
  • the bioinformation measuring device 20 may also be a brain information measuring device capable of measuring single molecules, as described below.
  • the three types of neurotransmitters are identified by using machine learning to identify the radio wave waveforms obtained from single molecule measurements, and a classifier that has learned the single molecule waveforms of dopamine, noradrenaline, and serotonin is used to identify the signals of unknown samples.
  • serotonin which generally indicates the degree of composure and relaxation
  • noradrenaline which indicates the degree of brain arousal and has a stimulating effect in enhancing concentration and judgment.
  • serotonin and noradrenaline may also be measured from the user's blood, etc.
  • ⁇ Hardware Configuration> 2 is a diagram showing an example of a physical configuration of an information processing device 10 of a server according to each embodiment.
  • the server 10 has one or more central processing units (CPUs) 10a corresponding to a calculation unit, a random access memory (RAM) 10b corresponding to a storage unit, a read only memory (ROM) 10c corresponding to a storage unit, a communication unit 10d, an input unit 10e, and a display unit 10f.
  • CPUs central processing units
  • RAM random access memory
  • ROM read only memory
  • the information processing device 10 is described as being configured as a single information processing device, but the information processing device 10 may be realized by combining multiple computers or multiple calculation units. Also, the configuration shown in FIG. 2 is an example, and the information processing device 10 may have other configurations or may not have some of these configurations.
  • the CPU 10a is a control unit that controls the execution of programs stored in the RAM 10b or ROM 10c and calculates and processes data.
  • the CPU 10a is a calculation unit that executes a program (learning program) that learns using a learning model that estimates the user's emotions or state (for example, comfort level (or discomfort level)) from biometric information.
  • the CPU 10a receives various data from the input unit 10e and communication unit 10d, and displays the calculation results of the data on the display unit 10f or stores them in the RAM 10b.
  • RAM 10b is a storage unit that allows data to be rewritten, and may be composed of, for example, a semiconductor memory element.
  • RAM 10b may store data such as the program executed by CPU 10a, data related to brain activity, and associated data showing the correspondence between content and an index related to the user's discomfort level based on brain information. Note that these are merely examples, and RAM 10b may store data other than these, or some of these data may not be stored.
  • ROM 10c is a memory section from which data can be read, and may be configured, for example, with a semiconductor memory element. ROM 10c may store, for example, a learning program or data that is not rewritten.
  • the communication unit 10d is an interface that connects the information processing device 10 to other devices.
  • the communication unit 10d may be connected to a communication network such as the Internet.
  • the input unit 10e accepts data input from a user and may include, for example, a keyboard and a touch panel.
  • the display unit 10f visually displays the results of calculations performed by the CPU 10a, and may be configured, for example, with an LCD (Liquid Crystal Display). Displaying the results of calculations by the display unit 10f can contribute to XAI (eXplainable AI). The display unit 10f may also display, for example, learning results.
  • LCD Liquid Crystal Display
  • the learning program may be provided by being stored in a non-transitory storage medium readable by a computer, such as RAM 10b or ROM 10c, or may be provided via a communication network connected by communication unit 10d.
  • the CPU 10a executes the learning program to realize various operations described below with reference to Figures 3 and 7. Note that these physical configurations are examples and do not necessarily have to be independent configurations.
  • the information processing device 10 may include an LSI (Large-Scale Integration) in which the CPU 10a is integrated with the RAM 10b and ROM 10c.
  • the information processing device 10 may also include a GPU (Graphical Processing Unit) and an ASIC (Application Specific Integrated Circuit).
  • a brain information measuring device is used as the biological information measuring device 20, and the measured data includes first data related to serotonin and second data related to noradrenaline.
  • Serotonin and noradrenaline are neurotransmitters in the brain, and can more appropriately represent activity in the brain.
  • first data related to serotonin and second data related to noradrenaline are acquired, and the user's emotions or state are estimated using learning data including the first data and the second data.
  • the user's emotions or state include, for example, whether the user feels comfortable or pleasant.
  • the first data can be used to analyze whether the user is relaxed or calm
  • the second data can be used to analyze whether the brain is in an alert state.
  • the user defines a calm and alert state as being comfortable or pleasant.
  • the content is output to the user, thereby stimulating the user's brain.
  • the content includes, for example, sounds such as music, images including moving images and still images, smells, tactile sensations, and the like.
  • the bio-information measuring device 20 measures first data and second data. By inputting the measured first data and second data into a trained learning model, it becomes possible to estimate the user's emotions or state.
  • the trained learning model includes a learning model that is the result of machine learning a learning model that estimates the user's emotions or state using the first data and the second data as training data.
  • brain activity is estimated using brain neurosubstances, so it is possible to more appropriately estimate the user's brain state, i.e., the user's emotions or state. Furthermore, in the first embodiment, it is also possible to provide content to the user based on the estimated user's emotions or state.
  • ⁇ Processing configuration example> 3 is a diagram showing an example of a processing block of the information processing device 10 according to the first embodiment.
  • the information processing device 10 includes an acquisition unit 11, a learning unit 12, an output unit 13, an association unit 14, a selection unit 15, and a storage unit 16.
  • the learning unit 12, the association unit 14, and the selection unit 15 shown in FIG. 3 are realized by being executed by, for example, a CPU 10a
  • the acquisition unit 11 and the output unit 13 are realized by, for example, a communication unit 10d
  • the storage unit 16 can be realized by a RAM 10b and/or a ROM 10c.
  • the information processing device 10 may be configured by a quantum computer or the like.
  • the acquisition unit 11 acquires first data on serotonin and second data on noradrenaline based on a signal acquired by the bioinformation measuring device 20 worn by the user while content is being output to the user.
  • the bioinformation measuring device 20 acquires first data on serotonin and second data on noradrenaline classified by a trained classifier (learning model) using a radio wave waveform obtained by single molecule measurement.
  • the learning unit 12 inputs learning data including the first data and the second data into a learning model 12a that uses a neural network, and learns the user's emotions or state. For example, the learning unit 12 learns to output an index value that represents a normal and awake state using the first data and the second data.
  • the learning performed by the learning unit 12 may include supervised learning in which the user annotates emotions indicating comfort, pleasantness, discomfort, etc. while measuring the first data and the second data, and the training data labeled with the user's emotions is used.
  • FIG. 4 is a diagram showing the state of the user according to the first embodiment.
  • the first quadrant shown in FIG. 4 is defined as the user being comfortable.
  • the third quadrant shown in FIG. 4 is defined as being uncomfortable for the user.
  • the magnitudes of the first data and the second data may be determined to be large if they are equal to or greater than a threshold value, and small if they are less than the threshold value, using a threshold value set for each.
  • Each threshold value may be set by learning using emotion-labeled training data.
  • Quadrants other than the first quadrant may be defined as being uncomfortable for the user.
  • learning model 12a is a learning model that includes a neural network, for example, a sequence data analysis model, and specific examples may include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), DNN (Deep Neural Network), LSTM (Long Short-Term Memory), bidirectional LSTM, DQN (Deep Q-Network), etc.
  • CNN Convolutional Neural Network
  • RNN Recurrent Neural Network
  • DNN Deep Neural Network
  • LSTM Long Short-Term Memory
  • bidirectional LSTM Long Short-Term Memory
  • DQN Deep Q-Network
  • the learning model 12a also includes models obtained by pruning, quantizing, distilling, or transferring a learned model. Note that these are merely examples, and the learning unit 12 may perform machine learning using other learning models.
  • the loss function used in the learning unit 12 includes a function defined so as to reduce the user's discomfort level based on the first data and the second data.
  • the loss function is defined as a function that reduces the error between an index value indicating the user's comfort determined from the first data and the second data and an ideal index value or annotation result that falls in the first quadrant.
  • the user's comfort can be defined using the first data and the second data.
  • the first data is data related to serotonin, so the user's level of relaxation (normality) can be measured
  • the second data is data related to noradrenaline, so the user's level of alertness can be measured.
  • the loss function is set so that the index value indicating a normal and alert state based on the first data and the second data becomes large (so that the difference from the ideal index value becomes small).
  • the learning unit 12 may also learn the user's emotions or state when any content is output. For example, the learning unit 12 learns the first data and second data of a user who listens to various types of music, and learns what type of music the user finds comfortable. Specifically, the learning unit 12 learns which type of music the user listens to and the first data and second data of the user are included in the first quadrant shown in FIG. 4. As described above, if the first data and second data are classified into the first quadrant, it is estimated that the user finds the music comfortable. On the other hand, if the first data and second data are classified into the third quadrant (or the second or fourth quadrant), it is estimated that the user finds the music uncomfortable. The learning unit 12 adjusts the bias and weight of the learning model 12a using the backpropagation method so that the output value of the loss function can be minimized.
  • the learning unit 12 may also use different learning models 12a for each user. For example, the learning unit 12 identifies a user based on the user information when the user logs in to the system 1, and performs learning using the learning model 12a corresponding to this user. This makes it possible to perform learning according to the user's preferences by using the user's personal learning model 12a.
  • the output unit 13 outputs the results of learning by the learning unit 12.
  • the output unit 13 may output the learned learning model 12a, or may output a comfort index value estimated by the learning model 12a, or information indicating an emotion or state classified by learning.
  • the above processing makes it possible to provide a mechanism that uses brain data to more appropriately select or generate content according to a user's preferences. For example, it is possible to generate a learning model that uses brain data to more appropriately select or generate content according to a user's preferences. Specifically, because a learning model trained using data on serotonin and noradrenaline is used, it becomes possible to more appropriately estimate the user's emotions or state. Therefore, by using this learning model, it becomes possible to provide content that is more appropriately tailored to the user's state.
  • the associating unit 14 associates an index value indicating the user's comfort (or discomfort level) predicted by the learning of the learning unit 12 with the content that was being output to the user at that time. For example, when the index value indicating comfort included in the predicted value of the learning result is greater than a predetermined value, i.e., the user feels comfortable, the associating unit 14 associates information for identifying the content with the index value. In this way, by associating the index value indicating comfort based on information on the user's brain activity with the content, it is possible to create a content list, for example, in order of the index value indicating comfort.
  • FIG. 5 is a diagram showing an example of related data according to the first embodiment.
  • the related data is data that associates content identification information (e.g., data A) with an index value (e.g., S1).
  • the related data shown in FIG. 5 is an example, and it is sufficient that the content that the user finds comfortable is associated with the index value at that time.
  • the association unit 14 may include this content in the dataset. This makes it possible to generate a dataset that collects content that indicates comfort based on information about the user's brain activity.
  • the selection unit 15 may select at least one piece of content from among a plurality of pieces of content based on an index value or classification result indicating user comfort contained in the learning result of the learning unit 12. For example, when the index value or classification result indicates discomfort, the selection unit 15 selects one piece of content from a list of content that the user finds comfortable, associated by the association unit 14. Specifically, the selection unit 15 may select the content in descending order of index value (order of comfort) or randomly.
  • the output unit 13 may output at least one content selected by the selection unit 15.
  • the output unit 13 selects an output device depending on the content, and outputs the content to the selected output device. For example, if the content is music, the output unit 13 selects a speaker as the output device and causes the music to be output from the speaker. Also, if the content is a still image, the output unit 13 selects the display unit 10f as the output device and causes the still image to be output from the display unit 10f.
  • the storage unit 16 stores data related to the above-mentioned learning.
  • the storage unit 16 stores information on the neural network used in the learning model, hyperparameters, etc.
  • the storage unit 16 may also store biometric information 16a including the acquired first data and second data, a learned learning model, related data 16b shown in FIG. 5, a content list that the user finds comfortable, etc.
  • Fig. 6 is a flowchart showing an example of processing of the information processing device 10 according to the first embodiment.
  • serotonin and noradrenaline are detected and acquired using a known technique.
  • step S102 the acquisition unit 11 acquires first data related to serotonin and second data related to noradrenaline based on a signal acquired by a brain information measuring device worn by the user.
  • the first data indicates the amount of serotonin secreted
  • the second data indicates the amount of noradrenaline secreted.
  • step S104 the learning unit 12 inputs learning data including the first data and the second data acquired when the content is output to the user into a learning model 12a that uses a neural network, and performs learning.
  • the learning model 12a is a learning model that learns the user's emotions or state based on the first data and the second data.
  • step S106 the output unit 13 outputs the learning result by the learning unit 12.
  • the learning result may include an index value indicating the user's emotion or state.
  • the output unit 13 may also output the trained model.
  • the first embodiment by using neurotransmitters, it becomes possible to more appropriately estimate brain activity, and to generate a learning model that more appropriately estimates brain activity.
  • the acquisition unit 11 may acquire dopamine as the third data.
  • an area in which comfort or pleasantness is felt may be identified in the three-dimensional space of the first data to the third data, and the learning unit 12 may learn the user's emotions or state using the first data to the third data.
  • the acquisition unit 11 may acquire radio wave waveforms obtained by single molecule measurement, and the learning unit 12 may detect dopamine, noradrenaline, and serotonin using machine learning of PUC (Positive and Unlabeled Classification) described in "Time-resolved neurotransmitter detection in mouse brain tissue using an artificial intelligence-nanogap.”
  • the learning unit 12 may further learn the above-mentioned emotions or states of the user using at least the detected serotonin and noradrenaline.
  • the digital data currently being output to the user is regenerated so that the user feels more comfortable, using biological information measured by the biological information measuring device 20.
  • the biological information used in the second embodiment includes at least one of the first data related to serotonin and the second data related to noradrenaline used in the first embodiment, and data such as brain waves, blood flow, pulse rate, heart rate, body temperature, and electrooculography.
  • GANs generative adversarial networks
  • a generative model that generates digital data is used as the generator of the GAN, and a learning model that estimates the user's emotions or state, as described in the first embodiment, is used as the discriminator.
  • the classifier determines "true” if the user's emotion or state indicates comfort, and determines "false” if the user indicates discomfort.
  • the second embodiment it becomes possible to regenerate digital data until the user feels comfortable.
  • ⁇ Processing Configuration> 7 is a diagram showing an example of a processing block of the information processing device 30 according to the second embodiment.
  • the information processing device 30 includes an acquisition unit 302, a generation unit 304, a determination unit 310, an output unit 312, and a database (DB) 314.
  • the information processing device 30 may be configured by a quantum computer or the like.
  • the acquisition unit 302 and the output unit 312 can be realized, for example, by the communication unit 10d shown in FIG. 2.
  • the generation unit 304 and the determination unit 310 can be realized, for example, by the CPU 10a shown in FIG. 2.
  • the DB 314 can be realized, for example, by the ROM 10c and/or the RAM 10b shown in FIG. 2.
  • the acquisition unit 302 acquires the bioinformation measured by the bioinformation measuring device 20.
  • the bioinformation includes at least one of the following information: neurotransmitters such as dopamine, serotonin, and noradrenaline, brain waves, pulse rate, heart rate, body temperature, blood flow, and electrooculography.
  • the acquisition unit 302 also acquires the bioinformation of the stimulated user using predetermined digital data.
  • the acquisition unit 302 outputs the acquired bioinformation to the identifier 308.
  • the generator 304 generates predetermined digital data, for example, by executing a model similar to a generative adversarial network (GANs).
  • GANs generative adversarial network
  • the generator 304 uses a generative adversarial network (GANs) including a generator 306 and a classifier 308 to generate digital data including at least one of a digital space, an image including a still image or a moving image, music, a control signal for a robot or a home appliance device, etc.
  • GANs generative adversarial network
  • the generator 306 generates digital data using input noise, etc.
  • the noise may be random numbers.
  • the generator 306 may be a neural network having a predetermined structure such as GANs.
  • the generator 306 may also be a generating AI that generates digital data by inputting a prompt.
  • the generator 306 outputs the generated digital data to the identifier 308.
  • the identifier 308 acquires biometric information of the user whose digital data is output or provided from the acquisition unit 302.
  • the identifier 308 estimates the user's emotion or state from the digital data generated by the generator 306 using the acquired biometric information.
  • the identifier 308 learns and, if the estimated user's emotion or state indicates comfort, identifies the digital data as "true", which is a positive first result.
  • the identifier 308 learns and, if the estimated user's emotion or state indicates discomfort, identifies the digital data as "false", which is a negative second result.
  • the determination that the emotion or state indicates comfort or discomfort is made based on the classification result if the learning result indicates a classification result of the emotion, and based on a comparison between the index value and a threshold value if the learning result indicates an index value of the emotion or state.
  • the identifier 308 may be a learning model trained using learning data including the user's biometric information and a comfortable or uncomfortable label at the time of acquiring the biometric information.
  • the comfortable or uncomfortable labels may be labels of the user's opposing emotions or states, such as like or dislike, fun or boring, etc.
  • the judgment unit 310 instructs the generator 306 to regenerate the digital data, and if the classification result of the classifier 308 indicates "true” (first result), it outputs that fact to the output unit 312.
  • the judgment unit 310 may output the classification result to the output unit 312 regardless of the content of the classification result.
  • the judgment unit 310 may output an updated prompt to the generator 306 so that the digital data is generated.
  • the generating unit 304 may update the parameters of the generator 306 and the discriminator 308 based on the result of the discrimination of authenticity (positive or negative) by the discriminator 308. For example, the generating unit 304 may update the parameters of the discriminator 308 using backpropagation so that the discriminator 308 can more appropriately estimate the user's emotion or state. The generating unit 304 may also update the parameters of the generator 306 using backpropagation so that the discriminator 308 discriminates the digital data generated by the generator 306 as true. The generating unit 304 outputs the finally generated digital data to the output unit 312.
  • the output unit 312 outputs information indicating that the digital data is comfortable for the user. For example, the output unit 312 outputs to the user one of a sound, image, mark, etc. indicating comfort, allowing the user to understand his or her own condition.
  • the output unit 312 may also output to the user the digital data that the user ultimately finds comfortable. Through the above process, it becomes possible to regenerate digital data until the user finds it comfortable.
  • the determination unit 310 may also determine the classification result or instruct the generator 306 to generate digital data when a predetermined condition regarding the timing of the determination is satisfied. For example, if new digital data is generated immediately after the generator 306 outputs newly generated digital data to the user, the user may not have enough time to feel emotions toward a single piece of digital data. Therefore, the determination unit 310 may determine the classification result obtained from the classifier 308 a predetermined time after determining that the classification result is "true” or "false".
  • the determination unit 310 may also determine whether the digital data is "true” or "false” using multiple identification results obtained during a specified time. For example, the determination unit 310 may use the larger number of identification results obtained during the specified time, or the larger of the maximum absolute value of the index value indicating "true” or the maximum absolute value of the index value indicating "false.”
  • the above process allows the user to have a certain amount of time to process a single piece of digital data. It also prevents the user from feeling anxious, concerned, or suspicious due to unnecessary switching of digital data. It is also possible to reduce the processing load on the information processing device 30.
  • the generated digital data may also include at least one of data related to virtual space, data related to robot control, data related to autonomous driving, and data related to home appliance devices.
  • Data relating to the virtual space includes, for example, a metaverse space and data used in the metaverse space.
  • a metaverse space For example, when the generator 306 generates a metaverse space, the user is stimulated by the metaverse space, and the identifier 308 estimates the user's emotion or state, so that the generator 306 can generate the metaverse space until the user feels comfortable.
  • Data related to robot control includes, for example, robots that assist human movements and nursing care robots.
  • a user who receives a service provided by a robot's movements feels either comfortable or uncomfortable with the robot's movements. If the user finds robot movements uncomfortable, generator 306 regenerates control data that the user finds comfortable. This allows generator 306 to generate control data for the robot until the user feels comfortable.
  • Data related to autonomous driving includes, for example, speed data of the autonomous vehicle and content to be output inside the vehicle during autonomous driving.
  • the discriminator 308 estimates whether the user riding in the autonomous vehicle feels comfortable with the video. This allows the generator 306 to generate a video to be displayed inside the autonomous vehicle until the user feels comfortable.
  • Data related to home appliance devices includes, for example, temperature control data for an air conditioner.
  • identifier 308 estimates whether a user in a room where the air conditioner is located feels comfortable at that room temperature. This allows generator 306 to automatically adjust the temperature of the air conditioner until the user feels comfortable.
  • DB314 stores data processed by generator 306 and identifier 308.
  • DB314 may store digital content generated for each user.
  • Fig. 8 is a flowchart showing an example of the process of the information processing device 30 according to the second embodiment. The process shown in Fig. 8 shows an example in which digital data continues to be generated until the user feels comfortable.
  • step S202 the generator 306 of the information processing device 30 generates predetermined digital data.
  • step S204 the classifier 308 of the information processing device 30 inputs the user's biometric information stimulated using the specified digital data to a classifier that uses a learning model that learns the user's emotions or state, and obtains a classification result that includes the user's emotions or state in response to the specified digital data.
  • step S206 the determination unit 310 of the information processing device 30 determines whether the identification result indicates comfort. For example, if the identification result indicates comfort ("true") (step S206-YES), the process proceeds to step S210, and if the identification result indicates discomfort ("false") (step S206-NO), the process proceeds to step S208.
  • step S208 the determination unit 310 of the information processing device 30 instructs the generator 306 to generate digital data. Then, the process returns to step S202.
  • step S210 the output unit 312 of the information processing device 30 outputs information indicating that the digital data is comfortable for the user.
  • the generator 306 may be implemented in another device, not in the information processing device 30.
  • the information processing device 30 may output a generation instruction (e.g., a prompt) to the external generator 306 and obtain digital data from the generator 306.
  • a generation instruction e.g., a prompt
  • the second embodiment it is possible to provide a mechanism that enables more appropriate generation of content according to the user's preferences using biometric information including data related to the brain. Furthermore, according to the second embodiment, it becomes possible to regenerate digital data until the user feels comfortable.
  • 10...information processing device 10a...CPU, 10b...RAM, 10c...ROM, 10d...communication unit, 10e...input unit, 10f...display unit, 11...acquisition unit, 12...learning unit, 12a...learning model, 13...output unit, 14...association unit, 15...selection unit, 16...storage unit, 16a...biometric information, 16b...associated data, 302...acquisition unit, 304...generation unit, 306...generator, 308...classifier, 310...determination unit, 312...output unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • Theoretical Computer Science (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Child & Adolescent Psychology (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Neurology (AREA)
  • Dermatology (AREA)
  • Neurosurgery (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Psychology (AREA)
  • Developmental Disabilities (AREA)
  • Hospice & Palliative Care (AREA)
  • Psychiatry (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Pathology (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • User Interface Of Digital Computer (AREA)
PCT/JP2023/034704 2022-09-26 2023-09-25 脳情報の分析によるレコメンデーション Ceased WO2024071027A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US19/072,711 US20250201383A1 (en) 2022-09-26 2025-03-06 Recommendation based on analysis of brain information

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022152658A JP7297342B1 (ja) 2022-09-26 2022-09-26 脳情報の分析によるレコメンデーション
JP2022-152658 2022-09-26

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US19/072,711 Continuation US20250201383A1 (en) 2022-09-26 2025-03-06 Recommendation based on analysis of brain information

Publications (1)

Publication Number Publication Date
WO2024071027A1 true WO2024071027A1 (ja) 2024-04-04

Family

ID=86900455

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/034704 Ceased WO2024071027A1 (ja) 2022-09-26 2023-09-25 脳情報の分析によるレコメンデーション

Country Status (3)

Country Link
US (1) US20250201383A1 (https=)
JP (2) JP7297342B1 (https=)
WO (1) WO2024071027A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2026009291A1 (ja) * 2024-07-01 2026-01-08 株式会社Ｎｔｔドコモ 情報処理装置および情報処理方法

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2025049263A (ja) * 2023-09-21 2025-04-03 ソフトバンクグループ株式会社 システム
JP2025049250A (ja) * 2023-09-21 2025-04-03 ソフトバンクグループ株式会社 システム

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005352151A (ja) * 2004-06-10 2005-12-22 National Institute Of Information & Communication Technology 人間の感情状態に応じた音楽出力装置及び音楽出力方法
JP2017204216A (ja) * 2016-05-13 2017-11-16 Ｃｏｃｏｒｏ Ｓｂ株式会社 記憶制御システム、システム及びプログラム
JP2018504719A (ja) * 2014-11-02 2018-02-15 エヌゴーグル インコーポレイテッド スマートオーディオヘッドホンシステム
US20210124420A1 (en) * 2019-10-29 2021-04-29 Hyundai Motor Company Apparatus and Method for Generating Image Using Brain Wave

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008046691A (ja) 2006-08-10 2008-02-28 Fuji Xerox Co Ltd 顔画像処理装置及びコンピュータのプログラム
JP6351692B2 (ja) 2016-11-17 2018-07-04 Ｃｏｃｏｒｏ Ｓｂ株式会社 表示制御装置
JP7097012B2 (ja) 2017-05-11 2022-07-07 学校法人 芝浦工業大学 感性推定装置、感性推定システム、感性推定方法およびプログラム
JP7091643B2 (ja) * 2017-12-11 2022-06-28 日産自動車株式会社 違和感判別方法及び違和感判別装置
US20240161543A1 (en) * 2021-03-29 2024-05-16 Sony Group Corporation Biological information processing device and biological information processing system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005352151A (ja) * 2004-06-10 2005-12-22 National Institute Of Information & Communication Technology 人間の感情状態に応じた音楽出力装置及び音楽出力方法
JP2018504719A (ja) * 2014-11-02 2018-02-15 エヌゴーグル インコーポレイテッド スマートオーディオヘッドホンシステム
JP2017204216A (ja) * 2016-05-13 2017-11-16 Ｃｏｃｏｒｏ Ｓｂ株式会社 記憶制御システム、システム及びプログラム
US20210124420A1 (en) * 2019-10-29 2021-04-29 Hyundai Motor Company Apparatus and Method for Generating Image Using Brain Wave

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2026009291A1 (ja) * 2024-07-01 2026-01-08 株式会社Ｎｔｔドコモ 情報処理装置および情報処理方法

Also Published As

Publication number Publication date
JP7297342B1 (ja) 2023-06-26
JP2024047533A (ja) 2024-04-05
JP2024047181A (ja) 2024-04-05
US20250201383A1 (en) 2025-06-19

Similar Documents

Publication Publication Date Title
WO2024071027A1 (ja) 脳情報の分析によるレコメンデーション
US20130080185A1 (en) Clinical analysis using electrodermal activity
EP4087468A1 (en) Dynamic user response data collection method
WO2016111993A1 (en) Human performance optimization and training methods and systems
Islam et al. Personalization of stress mobile sensing using self-supervised learning
Tiwari et al. Classification of physiological signals for emotion recognition using IoT
Deng et al. A machine learning-based monitoring system for attention and stress detection for children with autism spectrum disorders
Onim et al. A review of context-aware machine learning for stress detection
Wang et al. Design and analysis of a Closed-Loop emotion regulation system based on multimodal affective computing and emotional Markov chain
Rincon et al. Detecting emotions through non-invasive wearables
WO2021014738A1 (ja) 快適性運転データ収集システム、運転制御装置、方法、および、プログラム
JP3523007B2 (ja) 満足感計測システムおよびフィードバック装置
JP2019072371A (ja) システム及び意思疎通を図るために行うアクションの評価方法
JP2009066186A (ja) 脳活動状態推定方法および情報処理システム
JP7607377B2 (ja) 情報処理方法
WO2024009944A1 (ja) 情報処理方法、記録媒体、及び情報処理装置
Hwang et al. Towards Personalized Adaptive Learning Using Artificial Intelligence and Physiological Signals
JP7435965B2 (ja) 情報処理装置、情報処理方法、学習モデルの生成方法、及びプログラム
JP7408103B2 (ja) 情報処理装置、情報処理方法及び情報処理プログラム
WO2020202958A1 (ja) 分類装置および分類プログラム
EP4215109B1 (en) Apparatus and computer-implemented method for providing information about a user's brain resources, non-transitory machine-readable medium and program
JP7534745B1 (ja) 発作予測プログラム、記憶媒体、発作予測装置および発作予測方法
JP2026036311A (ja) システム
Hegodaarachchi A Computational Model for Predicting Music Popularity: A Psychophysiological Study
JP2026033414A (ja) システム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23872250

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23872250

Country of ref document: EP

Kind code of ref document: A1