JP2018521381A - Emotion detection system - Google Patents

Abstract

The emotion detection system disclosed herein actively protects the privacy and security of monitored objects. This provision of protection distinguishes the system from conventional emotion detectors. In some examples, a self-contained emotion detection device protects the identity of the subject while simultaneously providing anonymous data describing the subject's emotional state to other devices in the emotion detection system. In such an example, the emotion detection device protects the identity of the subject by isolating the raw image data acquired by the device exclusively into an encapsulated storage location within the device. To provide anonymous data, the emotion detection device first processes the raw image data to identify landmark points in the target face image. Next, the emotion detection device normalizes the landmark point to a general-purpose face, and transmits an anonymous data set describing the normalized landmark point to another device for subsequent processing.

Description

  Conventional emotion detectors acquire and record data describing a person's response to an event. Conventional emotion detectors further analyze the recorded data to interpret a person's emotional response to the event. Responses recorded, analyzed and interpreted by emotion detectors include human speech, facial expressions, physiological signals, device usage patterns, and the like. To be effective, many emotion detectors require a person to present a threshold level of activity. For example, an emotion detector based on a person's utterance requires the person to emit one or more detectable expressions. Similarly, emotion detectors based on device usage patterns require a person to manipulate an object (eg, a mouse or keyboard) to establish a baseline usage pattern. Other emotion detectors require physical contact with a person for the sensor to be effective. For example, an emotion detector based on a person's physiological state requires that the person wear one or more sensors (eg, an electrocardiogram electrode, skin conductance sensor, etc.). Benefits provided through the use of conventional emotion detectors include the system's ability to continuously record and later analyze a person's response to events generated by the system.

Notification of materials subject to copyright protection Some of the materials in this patent document are subject to copyright protection under the copyright laws of the United States and other countries. The copyright owner has no objection to its facsimile reproduction by any person, so long as the patent document or patent disclosure appears in a publicly available file or record of the United States Patent and Trademark Office, otherwise All copyrights are reserved. The copyright owner hereby does not waive any of its rights to keep this patent document confidential, including without limitation its right to comply with 37 CFR §1.14.

1 illustrates an emotion detection device configured in accordance with an embodiment of the present disclosure. 7 illustrates emotion detection processing configured in accordance with an embodiment of the present disclosure. 1 illustrates an emotion detection device that performs an emotion detection process configured in accordance with an embodiment of the present disclosure. 1 illustrates an emotion detection system configured in accordance with an embodiment of the present disclosure. FIG. 6 illustrates emotion monitoring processing configured in accordance with an embodiment of the present disclosure. FIG. 1 illustrates an emotion detection system that performs emotion monitoring processing configured in accordance with an embodiment of the present disclosure. 1 illustrates a computing system configured in accordance with one embodiment of the present disclosure. 1 illustrates a mobile computing system configured in accordance with one embodiment of the present disclosure.

  The emotion detection system disclosed herein actively protects the privacy and security of monitored subjects. This provision of protection distinguishes the system described herein from conventional emotion detectors. In some exemplary embodiments, a self-contained emotion detector provides anonymous data describing the subject's emotional state to other devices in the emotion detection system while protecting the subject's identity To do. In these examples, the emotion detection device protects the identity of the subject by isolating the raw image data acquired by the device exclusively within an encapsulated storage location within the device. To provide anonymous data, the emotion detection device first processes the raw image data to identify landmark points in the target face image. Next, the emotion detection device normalizes the landmark point to a generic face, and transmits an anonymous data set describing the normalized landmark point to another device for subsequent processing. In some examples, the other device processes an anonymous data set to identify and record the emotional state of the subject in connection with information describing the stimulus presented to the subject. The stimuli can include non-interactive content (eg, books, television, movies, radio programs, music, advertisements, etc.). Stimuli may also include interactive content (eg, games, shopping, etc.). In some examples, other devices in the emotion detection system attempt to influence the subject's emotional state by adjusting stimuli (eg, adjusting prices, plots, music genres, etc.). While various exemplary embodiments provided herein refer to facial images, other target features can be used as well to assess a subject's emotional state, to name a few examples: For example, voice recording, body posture and / or gesture images, and biometric data such as heart rate and blood pressure. As will be appreciated, the techniques provided herein can be used to encapsulate any such target raw data within the device, identify landmark points within that data, Normalized to a model (for example, a general-purpose speech model that repeats the user's inflection and raised pitch using general-purpose speech, or a general-purpose torso model that repeats the user's body language and / or gesture using a general-purpose body) can do. In any of the above exemplary cases, the anonymous data set describing the normalized landmarks can be sent to other devices for subsequent processing.

  Still other aspects, embodiments, and advantages of the above exemplary aspects and embodiments are discussed in detail below. Moreover, both the foregoing information and the following detailed description are merely exemplary of various aspects and embodiments and provide an overview or framework for understanding the nature and characteristics of the claimed aspects and embodiments. Intended to be. “One Embodiment”, “Other Embodiments”, “Example”, “Some Embodiments”, “Some Examples”, “Alternative Embodiments”, “Various Embodiments”, “1 References to “one embodiment”, “at least one embodiment”, “another embodiment”, “this and other embodiments”, or the like are not necessarily mutually exclusive and relate to an embodiment or example. It is intended that any particular feature, structure, or characteristic described as such may be included in at least one embodiment or example. The appearances of such terms herein do not necessarily all refer to the same embodiment or example. Any embodiment or example disclosed herein may be combined with any other embodiment or example.

  Moreover, the terminology and terminology used herein is for the purpose of description and should not be considered limiting. Any reference to an example system, method, embodiment, component, element, or operation referred to herein in the singular may further include the plural embodiment and any embodiment herein. Any reference to a plurality of components, elements, or operations may further include embodiments that include the singular. References in the singular or plural are not intended to limit the presently disclosed system or method, its components, operations, or elements. The use of “include”, “include”, “have”, “contain”, “accompany”, and derivatives thereof herein includes the items listed thereafter and equivalents thereof as well as further items. Intended to be. References to “or” may be considered inclusive, and thus any term described using “or” may be any one, more than one, and all of the terms described. May indicate. In addition, in the event that the use of a term contradicts between this document and a document incorporated by reference herein, the use of the term in the incorporated reference is a supplement to that of this document and The use of terms in the literature controls.

Overview As explained before, conventional emotion detectors offer several benefits, but also suffer from drawbacks depending on the application. For example, with respect to privacy, most human subjects do not want to be exposed to a certain lookout. This is especially true if the lookout continuously records the exact portrait and behavior of the human object. In addition, most human subjects are concerned about the lookout records being misused, stolen, or otherwise used without their permission. For contrast purposes, most human users oppose having the camera continuously record all their movements, expressing themselves or analyzing their emotional responses anonymously. I do not object.

  Thus, according to some examples of the present disclosure, an emotion detection apparatus and system are provided, in which output data indicating a subject's emotional state is anonymous (ie, the output data does not indicate the subject's personal identity, and therefore For example, the actual identity of the subject is not included in the output data). The emotion detection device and system differs from at least a conventional emotion detector by generating anonymous data, for example, by normalizing the data to a general purpose model. In some such examples, this anonymous data is the only output provided by the emotion detection device. In the above example, the emotion detection device may use information that can be used to identify the subject (eg, the subject's face, voice, body posture and gesturing, or other detectable identification before providing anonymous data). Delete the information describing the attributes). Thus, the above example provides further protection for the privacy and security of the monitored object.

  In some examples, the emotion detection device is incorporated into an emotion detection system that includes additional features. For example, some examples include one or more emotion detection devices, one or more remote devices that analyze anonymous data transmitted by the emotion detection devices, and a network that couples the emotion detection devices to the remote devices. Including an emotion detection system. In the above example, the remote device receives anonymous data, mine the anonymous data, and classifies the emotional state of each subject. An emotion identifier indicating the individual emotion state is then securely stored for subsequent processing.

  In some examples, the emotion detection system further includes one or more stimulus controllers. In the above example, the remote device can be configured to automatically control the operation of one or more stimulus controllers. This control can be caused by analysis of anonymous data or emotion identifiers. For example, in some examples, the remote device analyzes the subject anonymous data set and identifies responsive actions to be performed that are associated with the anonymous data. In response to identifying the responsive action, the remote device performs the responsive action. For example, if the remote device detects a negative emotional state associated with the first stimulus, the remote device can identify a responsive action that specifies the second stimulus. In response to identifying such responsive action, the remote device can generate a stimulus message and send it to the at least one stimulus controller to generate a second stimulus. In this example, in response to receiving the stimulus message, the stimulus controller presents a second stimulus to the subject.

  The above and other examples disclosed herein provide various additional advantages over conventional emotion detectors. For example, in some examples described herein, actual shooting of an object is not performed, image data is not permanently stored, and raw image data is stored on a general purpose programmable device (eg, a personal computer, a wrap Analysis by a specialized device (for example, emotion detection device), not by a top, a smartphone, a tablet, or the like. For purposes of contrast to traditional emotion detectors, the above approach prevents general purpose programmable devices from analyzing raw image data in the upper layers of the software stack where the raw image data is exposed to privacy and security threats. . In another example, the emotion detection device deactivates a camera, microphone, or other input used to acquire input data describing the subject's identity after the input data is acquired. Thus, the above example increases the difficulty of obtaining unauthorized access to input data and output data. Moreover, according to some examples, the emotion detection system efficiently utilizes anonymous data to achieve many valuable actions, as will be described in more detail below.

  In a more specialized example, the emotion detection system uses the facial image to monitor the emotional state of the subject. This emotion detection system is convenient for the subject because it is less invasive and requires lower threshold object activity than other types of emotion detection systems. In addition, this emotion detection system captures, analyzes, and deletes image data within a self-contained unitary emotion detection device, thereby providing an object for unauthorized recording or use of their portraits. Address the concerns. One example of such an emotion detection device, a private emotion recognition sensor (PERS), is further described below.

  The emotion detection device can include a camera, a face image processing component, a data storage device, and a data transmission component. In some examples, the camera acquires image data describing the face of interest. The face image processing component processes the image data to generate an anonymous set of data describing the physical state of the subject's face. This anonymous data set can be further analyzed to determine details about the subject's emotional state, but cannot be used to determine the identity of the subject. This device can be easily extrapolated with respect to other emotion-indicating features. For example, in another embodiment, the emotion detection device can include a microphone, a voice processing component, a data storage device, and a data transmission component. In such an example, the microphone obtains audio data describing the oral utterance of the subject. The audio processing component processes the audio data to generate an anonymous set of data describing the state of the target audio (in terms of inflection, loudness, etc.). This anonymous data set can be further analyzed to determine details about the subject's emotional state, but cannot be used to determine the identity of the subject. In other examples, the emotion detection device can include a camera, a body posture and gesture image processing component, a data storage device, and a data transmission component. In such an example, the camera obtains body posture and / or gesture data describing the subject's body posture and gesturing state. The image processing component processes the image data to generate an anonymous set of data describing the physical state of the subject's body and / or gesturing. This anonymous data set can be further analyzed to determine details about the subject's emotional state, but cannot be used to determine the identity of the subject. Many other variations will be apparent in light of this disclosure. The selection of a specific target emotion indication feature depends on factors such as the desired computational load in processing the data to identify the target emotional state. Any such target emotion indication feature may be associated with other target emotion indication features as is, as variously provided herein, or in an effort to obtain a more comprehensive understanding of the emotional state of a given subject. (Eg, face and voice, or face and gesture, etc.) can be analyzed and used. In any of the above cases, the raw emotion instruction data is converted to an anonymous equivalent and the raw data itself is discarded to protect the identity of the subject.

Emotion Detection Device FIG. 1 shows one example of an emotion detection device, PERS100. The PERS 100 can be implemented using a programmable device and may be fabricated using any of the components of the system described below with reference to FIGS. 9 and 10, for example. As shown in FIG. 1, the PERS 100 includes a housing 102 in which a camera 104, a face image processor 106, and an output buffer 114 are disposed. In some examples, the housing is made of plastic, aluminum, or some other durable material. The housing 102 can include fasteners that allow the housing to be installed in various locations and orientations or in various structures. For example, the housing 102 includes a clamp, resin, hook-and-loop fastener, or other mechanism so that emotion detection devices can be installed on counters, computer monitors, walls, billboards, signs, chairs, security cameras, or other locations. Can make it possible. In some examples, camera 104 is a two-dimensional camera. In another example, the camera 104 is a three-dimensional camera. In some examples, PERS 100 collectively enables a network interface (eg, wired or wireless transceiver, connector, antenna, etc.) with circuitry and allows PERS 100 to communicate with other devices over a communications network. And a software stack.

  As shown in FIG. 1, the face image processor 106 includes an input frame buffer 108, a face detection module 110, and a landmark point extraction module 112. Input frame buffer 108 and output buffer 114 can be allocated from various data storage devices included within housing 102, such as on-chip memory or off-chip memory. The face detection module 110 and landmark point extraction module 112 can be implemented in hardware or a combination of hardware and software. Examples of processes configured to be performed by the face image processor 106 via the face detection module 110 and landmark point extraction module 112 are further described below with reference to FIGS.

  According to some examples, the emotion detection device executes a process of generating anonymous data indicating the emotional state of the target. FIG. 2 shows a detection process 200 according to such an example. As shown in FIG. 2, the detection process 200 is a combination of the emotion detection device providing a private and secure anonymous data set describing one or more facial expressions to other devices for subsequent processing. Includes several actions that make it possible.

  In operation 202, the device performing detection process 200 initializes one or more components of the device (eg, camera, memory, processor, networking interface, image processing module, etc.). In operation 204, the device polls the camera for image data. The image data can include one or more frames. These frames can include raw image data of the subject's face. In some examples, the raw image data includes 3D image data. In another example, the raw image data includes two-dimensional image data.

  In operation 206, the apparatus determines whether any of the one or more frames includes an image of the subject's face. If so, the device proceeds to operation 208. Otherwise, the device proceeds to operation 218. In operation 208, the device stores the frame for subsequent processing. Further, in at least one example, the device deactivates the camera within operation 208 as a security measure. By deactivating the camera in act 208, the device helps to ensure that no further image data is acquired by any authorized process running on the device.

In operation 210, the device analyzes the image to identify and store data indicative of the subject's emotional state. In some examples, the device performs Intel® Realsense ^™ technology to identify landmark points in the facial image. A landmark point is a set of points that specify facial features, such as lips, eyebrows, eyelids, and other identities and orientations. In one example, the device identifies 78 landmarks that indicate the subject's emotional state in combination.

  In operation 212, the device deletes the frame as a further security measure. By deleting the frame in operation 212, the device limits the amount of time that the frame is stored and the amount of time that the frame and landmark points coexist in the storage device of the device.

  In act 214, the device generates an anonymous data set by normalizing the landmark points identified in act 210 to the generic face. This normalization process can be accomplished by mapping each landmark point identified in the face image to a corresponding landmark point in the generic face model, so that the overall representation of the face is universally derived from the facial image. Converted to face. In some examples, the anonymous data set includes elements of data having fields storing values that identify points and point locations in the generic face model. The value stored in the location field can include two-dimensional or three-dimensional information. Next, in operation 212, the device stores the anonymous data set for subsequent processing.

  In operation 216, the device sends the anonymous data set to an address specified by a predefined configurable parameter value. This address may be a local address, ie the address of a component that is integral to the device, or a remote address, ie the address of a component that is not integral to the device. In operation 218, the device determines whether a shutdown is imminent. If so, the detection process 200 ends. Otherwise, the detection process 200 proceeds to operation 204.

  The process according to the detection process 200 enables the emotion detection device to privately and securely generate and communicate data describing the emotional state of interest.

  FIG. 3 is an input-output diagram illustrating the operation of one exemplary emotion detection device. More specifically, FIG. 3 represents the PERS 100 of FIG. 1 that performs the detection process 200 of FIG. Referring primarily to FIG. 3 in the combination of FIGS. 1 and 2, PERS 100 performs operation 204 to poll camera 104 for an image. The camera 104 acquires a frame including 3D image data, and transmits the frame to the face image processor 106. Face detection module 110 of face image processor 106 detects a face in the image by performing operation 206 and stores the frame in the input frame buffer by performing operation 208. Next, the landmark point extraction module 112 of the face image processor 106 performs operation 210 to analyze the image and identify landmark points as indicated by the point set 302.

  Continuing the above example, face image processor 106 performs operations 212 and 214 to delete the frame, generate anonymous data set 304, and store the anonymous data set in output buffer 114. At this point, PERS 100 may perform operation 216 to send the anonymous data set to another device for further processing.

  While the components described above process the image data to determine the emotional state of the subject, other examples use other identifiable attributes about the subject for the same purpose. For example, in one example, the emotion detection device implements an architecture similar to the architecture shown in FIG. 1 and determines the emotional state of the subject by processing voice generation performed by the subject. In such an example, the face image processor 106 is a speech processor configured to detect the generation of a human voice and extract an anonymous data set describing the emotional state of the subject as represented by the generation of the voice. Replaced. This anonymous data set can include data describing the spoken words and how the words were spoken (eg, volume, pitch, speed, etc.).

Emotion Detection System FIG. 4 shows an emotion detection system 400 configured to monitor the emotional state of the subject 410 and optionally affect the emotional state. As shown, emotion detection system 400 includes emotion detection device 402, remote device 404, stimulus controller 406, and network 408. Stimulus controller 406 can generate stimuli including stimuli 412 and stimuli 414. The emotion detection device 402 can include, for example, the PERS described above with reference to FIGS. The remote device can include any programmable device, such as any of the systems described below with reference to FIGS. The communication network 408 can include any communication network with which systems can exchange information. For example, network 408 may be a public network, such as the Internet, and may include other public or private networks, such as a LAN, WAN, extranet, and intranet. As shown in FIG. 4, emotion detection device 402, remote device 404, and optionally stimulus controller 406 are connected to network 408 and communicate data via network 408.

  Stimulus controller 406 can take the form of various devices that function to provide stimulation to subject 410. For example, according to some examples, the stimulus controller 406 is a programmable device, such as a gaming console, computer-compatible television, personal computer, laptop, or mobile phone. In such examples, stimuli 412 and 414 can include content such as online shopping web pages, music, video games, television programs, and movies.

  In other examples, the stimulus controller 406 may not be connected to the network 408. Examples of disconnected stimulus controllers include theater projectors, non-computer compatible televisions, signs, billboards, price tags, and consumer products. In such examples, the stimuli 412 and 414 may include content such as movies, advertisements, prices, information messages, and consumer product designs.

  According to some examples, emotion detection system 400 is configured to operate in a passive mode or an active mode. When operating in passive mode, emotion detection system 400 passively monitors the emotional state of subject 410 as stimulus controller 406 stimulates the subject according to commands received by the subject or any other user or system. When operating in active mode, emotion detection system 400 monitors the emotional state of the subject and attempts to influence the emotional state of the subject according to one or more system goals. Examples of processes configured to be performed by the emotion detection system 400 are further described below with reference to FIGS.

  According to some examples, the emotion detection system performs a process of monitoring the emotional state of the subject and optionally attempting to affect the emotional state of the subject. FIG. 5 shows a monitoring process 500 according to such an example.

  In operation 502, the system that executes the monitoring process 500 generates anonymous data by executing the process according to the detection process 200, for example. In some examples, one component of the system generates anonymous data and sends it to another component of the system, which receives the anonymous data and later processes it.

  In operation 504, the system records the anonymous data set in the data store, identifies stimuli associated with the anonymous data set, parses the anonymous data set and derives inferences therefrom. In some examples, the system identifies a stimulus associated with the anonymous data set by comparing one or more time stamps of the anonymous data set with one or more time stamps of the stimulus. If the stimulus timestamp is immediately before the anonymous dataset time stamp (eg, between 100 and 500 milliseconds), the system identifies the stimulus as being associated with the anonymous dataset. Other identification methods may be implemented without departing from the scope of the examples disclosed herein, including explicit identification values within an anonymous data set or stimulus.

  The specific mode of analysis and potential inferences drawn in operation 504 varies between illustrations. For example, according to some examples, various data mining techniques are employed to determine (eg, classify) the emotional state represented by one or more anonymous data sets. Examples of data mining techniques that can be performed for the above purpose include neural networks, k-neighbor processing, and vector machines. In another example, within operation 504, the system performs a data mining prediction process (eg, logistic regression) based on the history of anonymous data sets received and processed by the system to determine the target or group of targets. Predict emotional state. In such an example, the system can create one or more summaries that articulate the predictions for the user of the system.

  In operation 506, the system determines whether the anonymous data or analysis thereof triggers a system action defined by one or more configurable parameters. If so, the device proceeds to operation 508. Otherwise, the monitoring process 500 ends.

  In operation 508, the system adjusts the configurable parameters of the one or more stimulation controllers by sending a stimulation message to the one or more stimulation controllers. The stimulation message can include one or more requests to present additional stimulation to the subject. In operation 510, the one or more stimulus controllers generate a stimulus 514 according to the configurable parameter, thereby attempting to change the subject's emotional state from the first emotional state to the second emotional state, and the monitoring process 500. Ends.

  The process according to the monitoring process 500 allows the emotion detection system to monitor the emotional state of the subject and possibly its behavior and to influence in some instances.

  FIG. 6 is an input-output diagram illustrating the operation of one exemplary emotion detection system. FIG. 6 represents the emotion detection system 400 of FIG. 4 that performs the monitoring process 500 of FIG. Referring primarily to FIG. 6 in the combination of FIGS. 2-5, the emotion detection device 402 operates 204 at a time immediately after presentation of the stimulus 412 to the subject 410 (eg, between 100 and 500 milliseconds). To poll for the image of the target 410. Emotion detection system 402 then performs operations 206-216 to generate an anonymous data set 304.

  Continuing the above example, remote device 404 performs operation 502 to receive anonymous data set 304 and record the data in a data store local to the remote device. The remote device 404 then performs an operation 506 to determine whether the recording of anonymous data triggers further action by referencing the value of one or more configurable parameters. If the remote device 404 determines that no further action is triggered, the process ends here. However, in this example, remote device 404 determines that further action is triggered and performs operation 508 to send stimulation message 600 to stimulation controller 406. The stimulus message 600 may include a request to present additional stimuli that are predicted to transition the subject's emotional state to a new emotional state. In response to receiving stimulus message 600, stimulus controller 406 performs operation 510 to generate stimulus 414. At this point, the activities described above can be repeated by emotion detection 402 performing action 204 and polling for the image of the subject 410 at a time immediately after presentation of the stimulus 414 to the subject 410.

  A more specialized emotion detection system than that previously disclosed will now be described with reference to FIG. One example of a specialized emotion detection system is directed to online or traditional broadcast content (eg, stories, lectures, articles, movies, shows, amateur videos, etc.). In this example, emotion detection device 402 is positioned to view subject 410 when subject 410 receives stimulus 412 by reviewing or interacting with the content. The remote device 404 records an anonymous data set and further analyzes the anonymous data set to identify identified portions of online content (eg, news articles, songs, jokes, suspenseful scenes, dramatic scenes, Determine whether the horrible scene or the like) had its intended effect on the emotional state of the object 410. In this example, a summary of the analyzed anonymous data can be provided to the content creator to allow the content producer to improve future content. Alternatively, the summary may be provided to the content provider to allow the content provider to make better suggestions regarding which content may be most relevant to the subject in general or specifically to the subject 410. .

  Another example of a specialized emotion detection system is directed to video games. In this example, emotion detection device 402 is positioned to view subject 410 when subject 410 receives stimulus 412 by playing a video game. The remote device 404 records the anonymous data set, further analyzes the anonymous data set, and the identified portion of the game (eg, stage, puzzle, battle with boss, cut scene, etc.) has the intended effect. Is determined for the emotional state of the object 410. In this example, a summary of the analyzed anonymous data can be provided to the video game creator to allow the video game creator to improve future games. Alternatively, the above summary is provided to the video game provider so that the video game provider can make better suggestions regarding which games may be most relevant to the subject or specifically to the subject 410 in general. Also good.

  Another example of a specialized emotion detection system is directed to online shopping. In this example, emotion detection device 402 is positioned to view subject 410 when subject 410 receives stimulus 412 by browsing a website in search for an item to purchase. The remote device 404 records the anonymous data set and further analyzes the anonymous data set to determine which items or prices caused the positive and negative emotional responses in the subject 410. Further, in this example, stimulus controller 406 may present additional stimuli (eg, stimulus 414) in a predefined situation. For example, if the subject 410 expresses a positive emotion when presented with an item without pricing information and later expresses a negative emotion when presented with a price, the remote device 404 is designated The stimulus message 600 can be sent to discount the price by a percentage. In response to receiving the stimulus message 600, the stimulus controller 406 can transmit additional stimuli 414 in the form of a discounted price.

  Further, in the above example, the remote device 404 can analyze the anonymous data set to produce a summary that indicates which products were interested in the subject 410 and which products were not interested. Such a summary of the analyzed anonymous data can be provided to the item manufacturer to allow the item manufacturer to improve future items. Alternatively, the summary may be provided to the retailer to allow the retailer to make better suggestions regarding which items may be most relevant to the subject in general or specifically to the subject 410.

  Another example of a specialized emotion detection system is directed to a movie presented at a cinema. In this example, emotion detection device 402 is positioned to view subject 410 when subject 410 receives stimulus 412 by watching a movie in a movie theater. The remote device 404 records the anonymous data set and further analyzes the anonymous data set so that identified portions of the movie (eg, jokes, suspenseful scenes, dramatic scenes, horrible scenes, etc.) It is determined whether the intended effect has been given to the emotional state of the object 410. In this example, a summary of the analyzed anonymous data can be provided to the filmmaker to allow the filmmaker to improve future movies.

  Another example of a specialized emotion detection system is directed to advertisements presented on signs and billboards. In this example, emotion detection device 402 is positioned to view subject 410 when subject 410 receives stimulus 412 by viewing a sign or an advertisement on a billboard. The remote device 404 records the anonymous data set and further analyzes whether the advertisement caused positive and negative emotional responses in the subject 410 so that the advertisement can determine its intended effect. Determine if you have an emotional state. Further, in this example, the remote device 404 can analyze the anonymous data set to produce a summary that indicates which products were interested in the object 410 and which were not. Such a summary of the analyzed anonymous data can be provided to the item manufacturer to allow the item manufacturer to improve future items. Alternatively, the summary may be provided to the retailer to allow the retailer to make better suggestions regarding which items may be most relevant to the subject in general or specifically to the subject 410.

  Another example of a specialized emotion detection system is directed to actual shopping. In this example, emotion detection device 402 receives stimulus 412 by viewing items to be purchased at a brick-and-mortar retail store (eg, supermarket, department store, specialty store, etc.) for object 410. , Positioned to view the object 410. The remote device 404 records the anonymous data set and further analyzes the anonymous data set to determine which items or prices caused the positive and negative emotional responses in the subject 410. Further, in this example, stimulus controller 406 may present additional stimuli (eg, stimulus 414) in a predefined situation. For example, if the subject 410 expresses a positive emotion when presented with an item without pricing information and later expresses a negative emotion when presented with a price, the remote device 404 is designated The stimulus message 600 can be sent to discount the price by a percentage. In response to receiving the stimulus message 600, the stimulus controller 406 can transmit additional stimuli 414 in the form of a discounted price.

  Further, in the above example, the remote device 404 can analyze the anonymous data set to produce a summary that indicates which products were interested in the object 410 and which were not. Such a summary of the analyzed anonymous data can be provided to the item manufacturer to allow the item manufacturer to improve future items. Alternatively, the summary may be provided to the retailer to allow the retailer to make better suggestions regarding which items may be most relevant to the subject in general or specifically to the subject 410.

  Another example of a specialized emotion detection system is directed to automobile operation. In this example, emotion detection device 402 is positioned to view subject 410 when subject 410 receives a stimulus 412 by driving a car and reviewing the content provided by the in-vehicle entertainment system. The remote device 404 records the anonymous data set and further analyzes the anonymous data set so that any events (weather and traffic conditions, songs played on the radio, etc.) are positive and negative emotional responses in the subject 410. Determine what caused it. Further, in this example, stimulus controller 406 may present additional stimuli (eg, stimulus 414) in a predefined situation. For example, if the subject 410 expresses a negative emotion when presented with weather or traffic conditions, the remote device may send a stimulus message 600 to calm the subject 410. In response to receiving the stimulus message 600, the stimulus controller 406 can provide additional stimuli 414 in the form of a soothing music or soothing message to relax the subject 410. In another example, if the subject 410 expresses a lack of emotion for a period of time that exceeds the value of the configurable parameter, the remote device 404 can send a stimulus message 600 to energize the subject 410. In response to receiving the stimulus message 600, the stimulus controller 406 can provide additional stimuli 414 in the form of energizing music, messages, or alarms to energize the subject 410.

  In some examples, components disclosed herein (eg, PERS 100, emotion detection device 402, remote device 404, and stimulus controller 406) can read parameters that affect the operations performed by the components. it can. The parameters can be physically stored in any form of suitable memory including volatile memory (eg, RAM, etc.) or non-volatile memory (eg, magnetic hard drive, etc.). Further, the parameters are logically stored in a suitable data structure (eg, a database or file defined by a user mode application) or in a commonly shared data structure (eg, an application registry defined by the operating system). It may be stored. In addition, some examples provide both a system and user interface that allows external entities to modify parameters and thereby configure component behavior.

  Information in emotion detection device 100 or emotion detection system 400 can be stored in any logical and physical structure capable of holding information on a computer readable medium, including, among other things, linked lists, files Includes systems, flat files, indexed files, hierarchical databases, relational databases, or object-oriented databases. Data may be modeled using unique and foreign key relationships and indexes. Unique and foreign key relationships and indexes can be established between the various fields and tables to ensure both data integrity and data exchange performance.

  Information can flow between the components disclosed herein using various techniques. The techniques include, for example, passing information over a network using a standard protocol such as TCP / IP or HTTP, passing information between modules in memory, and files, databases, data stores, or some other non-volatile It includes passing information by writing to a data storage device. In addition, pointers or other references to information may be transmitted and received instead of in combination with or in addition to a copy of the information. Conversely, information may be exchanged in combination or addition instead of to information pointers or other references. Other techniques and protocols for communicating information can be used without departing from the scope of the examples and embodiments disclosed herein.

Exemplary System FIG. 7 illustrates a computing system 700 configured in accordance with one embodiment of the present disclosure. In some embodiments, system 700 may be a computing system that detects a subject's emotional state, although system 700 is not limited to this context. For example, the system 700 can be a personal computer (PC), laptop computer, ultra laptop computer, tablet, touchpad, portable computer, handheld computer, palmtop computer, personal digital assistant (PDA), cellular phone, combined cellular phone. / PDA, television, smart device (eg, smart phone, smart tablet, or smart television), mobile internet device (MID), messaging device, data communication device, set top box, game console, or perform graphics rendering operations And may be incorporated into other such computing environments that can display content.

  In some embodiments, system 700 includes a platform 702 coupled to display 720. The platform 702 can receive content from a content device, such as a content service device 730 or content distribution device 740 or other similar content source. A navigation controller including one or more navigation mechanisms can be used, for example, to interact with the platform 702 and / or the display 720 to supplement the user's navigation gestures. Each of these exemplary components is described in more detail below.

  In some embodiments, platform 702 includes any combination of chipset 705, processor 710, memory 712, storage device 714, graphics subsystem 715, application 716, and / or radio 718. be able to. Chipset 705 may provide intercommunication between processor 710, memory 712, storage device 714, graphics subsystem 715, application 716, and / or radio 718. For example, chipset 705 can include a storage device adapter (not shown) that can provide intercommunication with storage device 714.

  The processor 710 can be implemented, for example, as a compound instruction set computer (CISC) or reduced instruction set computer (RISC) processor, x86 instruction set compatible processor, multi-core, or any other microprocessor or central processing unit (CPU). In some embodiments, the processor 710 may include a dual core processor, a dual core mobile processor, and the like. The memory 712 can be implemented as, for example, a volatile memory device, such as, but not limited to, a random access memory (RAM), a dynamic random access memory (DRAM), or a static memory (SRA). The storage device 714 is, for example, a nonvolatile storage device such as, but not limited to, a magnetic disk drive, an optical disk drive, a tape drive, an internal storage device, an attached storage device, a flash memory, a battery backup SDRAM (synchronous DRAM), And / or can be implemented as a network accessible storage device. In some embodiments, the storage device 714 may include technology that increases the enhanced storage performance protection of digital media, for example, when multiple hard drives are included.

  The graphics subsystem 715 can perform processing of images, such as still or video, for display, as described variously herein, and in some embodiments, a light source for a direct view display. Configured to drive the array. The graphics subsystem 715 can be, for example, a graphics processing unit (GPU) or a visual processing unit (VPU). An analog or digital interface can be used to communicatively couple graphics subsystem 715 and display 720. For example, the interface may be any of a high-definition multimedia interface, a display port, a wireless HDMI, and / or a wireless HD compliant technique. Graphics subsystem 715 can be integrated into processor 710 or chipset 705. Graphics subsystem 715 may be a stand-alone card that is communicatively coupled to chipset 705. Graphics and / or video processing techniques, including the direct view projection techniques described herein, can be implemented in a variety of hardware architectures. For example, graphics and / or video functionality can be integrated into the chipset. Alternatively, discrete graphics and / or video processors may be used. As yet another embodiment, graphics and / or video functionality may be implemented by a general purpose processor including a multi-core processor. In a further embodiment, the functionality may be implemented in a consumer electronic device.

  Radio 718 may include one or more radios that can transmit and receive signals using various suitable wireless communication techniques. The above approach may involve communication across one or more wireless networks. Exemplary wireless networks include (but are not limited to) a wireless local area network (WLAN), a wireless personal area network (WPAN), a wireless metropolitan area network (WMAN), a cellular network, and a satellite network. In communicating across such networks, the radio 718 can operate according to one or more applicable standards in any version.

  In some embodiments, the display 720 can include any television or computer type monitor or display. Under the control of one or more software applications 716, the platform 702 can display a user interface 722 on the display 720.

  In some embodiments, the content service device 730 is hosted by, for example, any national, international, and / or independent service, and is thus accessible to the platform 702 via the Internet or other network. possible. Content service device 730 may be coupled to platform 702 and / or to display 720. Platform 702 and / or content service device 730 may be coupled to network 760 to communicate (eg, send and / or receive) media information to and from network 760. Content distribution device 740 may also be coupled to platform 702 and / or to display 720. In some embodiments, the content service device 730 is via a cable television box, personal computer, network, telephone, internet-enabled device or appliance capable of delivering digital information and / or content, and the network 760. Or any other similar device that can communicate content unidirectionally or bidirectionally between the content provider and the platform 702 and / or display 720. It will be appreciated that content can be communicated unidirectionally and / or bidirectionally from and to any one of the components and content providers in system 700 via network 760. . Examples of content can include any media information including, for example, video, music, graphics, text, medical and gaming content, and the like.

  The content service device 730 receives content, such as cable television programs that include media information, digital information, and / or other content. Examples of content providers may include any cable or satellite television or radio or Internet content provider. The provided examples are not intended to limit the present disclosure. In some embodiments, the platform 702 can receive control signals from a navigation controller 750 having one or more navigation mechanisms. The navigation mechanism of controller 750 can be used, for example, to interact with user interface 722. In some embodiments, the navigation controller 750 is a computer hardware component (specifically, a human interface device) that allows a user to enter spatial (eg, continuous and multi-dimensional) data into a computer. ) Can be a pointing device. Many systems, such as graphical user interfaces (GUIs), and televisions and monitors allow a user to control a computer or television using physical gestures, facial expressions, or sounds, and data for the computer or television Allow to provide.

  The movement of the navigation mechanism of controller 750 can be reflected on a display (eg, display 720) by movement of a pointer, cursor, focus ring, or other visual indicator displayed on the display. For example, a navigation mechanism located on the navigation controller 750 under the control of the software application 716 can be mapped to a virtual navigation mechanism displayed on the user interface 722, for example. In some embodiments, the controller 750 may be integrated into the platform 702 and / or the display 720 rather than a separate component. However, as will be appreciated, the embodiments are not limited to the elements shown or described herein or within the context.

  In some embodiments, when a driver (not shown), for example, is enabled, the user immediately turns the platform 702 on and off like a television with a touch of a button after initial activation. It can include technologies that make it possible. When platform 702 is turned “off”, program logic may allow platform 702 to stream content to a media adapter or other content service device 730 or content distribution device 740. Further, the chipset 705 may include hardware and / or software support for 5.1 surround sound audio and / or 7.1 high definition surround sound audio, for example. The driver can include a graphics driver for the integrated graphics platform. In some embodiments, the graphics driver may include a peripheral component interconnect (PCI) express graphics card.

  In various embodiments, any one or more of the components shown in system 700 can be integrated. For example, the platform 702 and the content service device 730 may be integrated, the platform 702 and the content distribution device 740 may be integrated, or the platform 702, the content service device 730, and the content distribution service 740 may be integrated. Good. In various embodiments, platform 702 and display 720 may be an integrated unit. For example, the display 720 and the content service device 730 may be integrated, or the display 720 and the content distribution device 740 may be integrated. The above examples are not intended to limit the present disclosure.

  In various embodiments, the system 700 can be implemented as a wireless system, a wired system, or a combination of both. When implemented as a wireless system, system 700 includes components and interfaces suitable for communicating over a wireless shared medium, such as one or more antennas, transmitters, receivers, transceivers, amplifiers, filters, control logic, And the like. An example of a wireless shared medium can include portions of the wireless spectrum, such as the RF spectrum and the like. When implemented as a wired system, the system 700 connects components and interfaces suitable for communicating over a wired communication medium, eg, input / output (I / O) adapters, I / O adapters, to corresponding wired communication media. Physical connectors, network interface cards (NICs), disk controllers, video controllers, audio controllers, and the like. Examples of wired communication media can include wires, cables, metal conductors, printed circuit boards (PCBs), backplanes, switch fabrics, semiconductor materials, twisted pair wires, coaxial cables, optical fibers, and the like.

  Platform 702 can establish one or more logical or physical channels to communicate information. The information can include media information and control information. Media information can refer to any data representing content directed to the user. Examples of content may include data from, for example, voice conversations, video conferencing, streaming video, email or text messages, voice mail messages, alphanumeric symbols, graphics, images, video, text and the like. it can. The control information can refer to any data representing commands, instructions, or control words directed to the automated system. For example, control information is used to route media information through the system or in a predetermined manner (eg, using hardware assisted for privileged access violation checking as described herein). The node can be instructed to process However, embodiments are not limited to the elements or context shown or described in FIG.

  As described above, system 700 can be implemented in various physical forms or form factors. FIG. 8 shows an embodiment of a small form factor device 800 in which the system 700 can be implemented. In some embodiments, for example, the device 800 can be implemented as a mobile computing device with wireless capabilities. A mobile computing device can refer to any device having, for example, a processing system and a mobile power source or supply such as one or more batteries.

  As previously described, examples of mobile computing devices include personal computers (PCs), laptop computers, ultra laptop computers, tablets, touchpads, portable computers, handheld computers, palmtop computers, personal digital assistants. (PDA), cellular phone, combined cellular phone / PDA, television, smart device (eg, smart phone, smart tablet, or smart television), mobile internet device (MID), messaging device, data communication device, and the like Things can be included.

  Examples of mobile computing devices include computers arranged to be worn by a person, such as wrist computers, finger computers, ring computers, eyeglass computers, belt clip computers, armband computers, shoe computers, clothing computers , And other wearable computers. In some embodiments, for example, a mobile computing device can be implemented as a smartphone that can perform computer applications and voice and / or data communications. Although some embodiments may be described using a mobile computing device implemented as a smartphone as an example, it is well understood that other embodiments can be similarly implemented using other wireless mobile computing devices. obtain. Embodiments are not limited within this context.

  As shown in FIG. 8, the device 800 can include a housing 802, a display 804, an input / output (I / O) device 806, and an antenna 808. The device 800 can further include a navigation mechanism 812. Display 804 can include any suitable display unit that displays information suitable for a mobile computing device, and display 804, as provided herein, in one exemplary embodiment. It is a pixelated display that can be viewed directly. The I / O device 806 may include any suitable I / O device that inputs information to the mobile computing device. Examples of I / O devices 806 include alphanumeric keyboards, numeric keypads, touchpads, input keys, buttons, cameras, switches, rocker switches, microphones, speakers, voice recognition devices and software, and the like Can do. Information may be entered into device 800 by a microphone. The information can be digitized by a voice recognition device. Embodiments are not limited within this context.

  Various embodiments may be implemented using hardware elements, software elements, or a combination of both. Examples of hardware elements include processors, microprocessors, circuits, circuit elements (eg, transistors, resistors, capacitors, inductors, and the like), integrated circuits, application specific integrated circuits (ASICs), programmable logic devices. (PLD), digital signal processor (DSP), field programmable gate array (FPGA), logic gate, register, semiconductor device, chip, microchip, chipset, and the like. Examples of software include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, applications A program interface (API), instruction set, computing code, computer code, code segment, computer code segment, word, value, symbol, or any combination thereof may be included. Whether hardware elements and / or software elements are used depends on any number of factors, for example, desired calculation rate, power level, thermal tolerance, processing cycle budget, input data rate, output data rate, It may vary from embodiment to embodiment according to memory resources, data bus speed, and other design or performance constraints.

  Some embodiments can be implemented using, for example, a machine-readable medium or article that can store instructions or a set of instructions, where the instructions or instruction set is executed by a machine. Can cause a machine to perform the method and / or action according to The machine can include, for example, any suitable processing platform, computing platform, computing device, processing device, computing system, processing system, computer, processor, or the like, of hardware and software It can be implemented using any suitable combination. A machine-readable medium or article includes, for example, any suitable type of memory unit, memory device, memory article, memory medium, storage device, storage article, storage medium, and / or storage unit, eg, memory, removable Or non-removable medium, erasable or non-erasable medium, writable or rewritable medium, digital or analog medium, hard disk, floppy disk, compact disk read only memory (CD-ROM), compact disk recordable (CD-R), Can include compact disc rewritable (CD-RW), optical disc, magnetic media, magneto-optical media, removable memory card or disc, various types of digital versatile disc (DVD), tape, cassette, or the like . The instructions may be any suitable type of executable implemented using any suitable high-level, low-level, object-oriented, visual, compiled, and / or interpreted programming language. Can contain code.

Further exemplary embodiments The following examples relate to further embodiments, from which numerous combinations and configurations will become apparent.

  Example 1 is an emotion detection system including an emotion detection device, wherein the emotion detection device is executable by a memory, at least one processor coupled to the memory, and the at least one processor. Receiving the image data representing the face of the user, analyzing the image data, identifying a plurality of points on the face, and storing an anonymous data set describing the plurality of points, the anonymous data set A face image processing module configured to send an anonymous data set to a remote device distinguishable from the emotion detection device and delete the image data, including an identifier field and a location field. ,including.

  Example 2 includes the configuration requirements of Example 1, and the image data includes at least one of two-dimensional image data and three-dimensional image data.

  Example 3 includes the configuration requirements of any of the previous examples, wherein the face image processing module analyzes the image data at least in part by normalizing the data set to a generic face, and By storing, the anonymous data is configured to be stored at least partially.

  Example 4 includes any of the above-described configuration requirements, and the image data includes a mark point.

  Example 5 includes any component of the previous examples, and the location of each element of the anonymous data set includes at least one of a two-dimensional value and a three-dimensional value.

  Example 6 includes any component of the previous examples, the emotion detection device further includes a camera configured to acquire the image data, and the face image processing module stores the image data. The camera is configured to be deactivated prior to analyzing, storing the anonymous data set, and transmitting the anonymous data set.

  Example 7 includes the constituent elements of Example 6, and the emotion detection apparatus further includes a housing including the camera, the memory, the at least one processor, and the face image processing module.

  Example 8 includes the requirements of any of the previous examples and further includes the remote device, wherein the remote device identifies a stimulus provided to the subject prior to receiving the image data. The anonymous data set is analyzed to determine the emotional state of the subject when presented with the stimulus.

  Example 9 includes the configuration requirements of Example 8, and the stimulus includes at least one of non-interactive content and interactive content.

  Example 10 includes a component of any of the previous examples, and the stimulus includes a portion of content provided to the subject.

  Example 11 includes the configuration requirements of Example 10, and some of the content includes a sequence of video frames.

  Example 12 further includes a stimulus controller configured to stimulate the subject, including any of the components of the previous examples.

  Example 13 includes the configuration requirements of Example 12, further including the remote device, wherein the remote device is configured to send a message to the stimulus controller, the stimulus controller receiving the message. It is configured to stimulate the subject in response.

  Example 14 includes the configuration requirements of any of the previous examples and further includes the remote device, wherein the remote device analyzes the anonymous data set to identify an estimated emotional state of the subject. Identifying the target emotional state of the target based on the estimated emotional state of the target, identifying a stimulus that transitions the target to the target emotional state, and stimulating the target using the stimulus Composed.

  Example 15 is a method for detecting an emotion using an apparatus, which includes receiving image data representing a target face and analyzing the image data to identify a plurality of points on the face. Storing an anonymous data set describing the plurality of points, wherein each element of the anonymous data set includes an identifier field and a location field, the anonymous data set, and the device Includes transmitting to a distinguishable remote device and deleting the image data.

  Example 16 includes the configuration requirements of Example 15, and receiving the image data includes receiving at least one of two-dimensional image data and three-dimensional image data.

  Example 17 includes the requirements of Example 15 or 16, wherein analyzing the image data includes normalizing the data set to a generic face, and storing the anonymous data set comprises: Storing.

  Example 18 includes the components of any of Examples 15-17, and the step of analyzing the image data includes identifying a landmark.

  Example 19 includes the configuration requirements of any of Examples 15-18, and the step of storing the anonymous data set stores at least one of a two-dimensional value and a three-dimensional value in the location field of each element. Including the steps of:

  Example 20 includes the constituent elements of any of Examples 15 to 19, and the step of acquiring the image data through a camera, the step of analyzing the image data, the step of storing the anonymous data set, and the above Deactivating the camera prior to transmitting the anonymous data set.

  Example 21 includes the configuration requirements of Example 20, and obtaining the image data includes obtaining the image data from a camera integrated with the device.

  Example 22 includes any of the requirements of Examples 15-21, and identifies the stimulus provided to the subject prior to receiving the image data; and analyzing the anonymous data set; Determining the emotional state of the subject when presented with the stimulus.

  Example 23 includes the configuration requirements of Example 22, wherein identifying the stimulus includes identifying at least one of non-interactive content and interactive content.

  Example 24 includes the components of any of Examples 15-23, and identifying the stimulus includes identifying a portion of content provided to the subject.

  Example 25 includes the configuration requirements of Example 24, and identifying the portion of the content identifies at least one of a sequence of video frames in the content and one or more time offsets in the content. Including the steps of:

  Example 26 includes the components of any of Examples 15 to 25 and further includes stimulating the subject.

  Example 27 includes the requirements of Example 26, and stimulating the subject includes sending a message from the remote device to a stimulus controller and generating a stimulus that stimulates the subject.

  Example 28 includes any of the constituent requirements of Examples 15 to 27, analyzing the anonymous data to identify the estimated emotional state of the target, and the estimated emotional state of the target Identifying a target emotional state of the subject based on, identifying a stimulus that transitions the subject to the target emotional state, and stimulating the subject using the stimulus.

  Example 29 is a non-transitory computer program product encoded with instructions that, when executed by one or more processors, causes an emotion detection process to be performed, the process comprising: Receiving the representing image data, analyzing the image data to identify a plurality of points on the face, and storing an anonymous data set describing the plurality of points, the anonymous data Each element of the set includes an identifier field and a location field, transmitting the anonymous data set to a remote device distinguishable from the device, and deleting the image data.

  Example 30 includes the configuration requirements of Example 29, and receiving the image data includes receiving at least one of two-dimensional image data and three-dimensional image data.

  Example 31 includes the configuration requirements of Example 29 or 30, wherein analyzing the image data includes normalizing the data set to a generic face, and storing the anonymous data set allows the data set to be Including remembering.

  Example 32 includes the configuration requirements of any of Examples 29-31, and analyzing the image data includes identifying landmark points.

  Example 33 includes the configuration requirements of any of Examples 29 to 32, and storing the anonymous data set stores at least one of a two-dimensional value and a three-dimensional value in the location field of each element. Including doing.

  Example 34 includes the constituent elements of any of Examples 29 to 33, and acquires the image data through a camera, analyzes the image data, stores the anonymous data set, and the above Deactivating the camera prior to sending the anonymous data set.

  Example 35 includes the configuration requirements of Example 34, and obtaining the image data includes obtaining the image data from a camera that is integral to the device.

  Example 36 includes the components of any of Examples 29-35, identifying a stimulus provided to the subject prior to receiving the image data, and analyzing the anonymous data set, Further determining the emotional state of the subject when presented with the stimulus.

  Example 37 includes the configuration requirements of Example 36, and identifying the stimulus includes identifying at least one of non-interactive content and interactive content.

  Example 38 includes the components of any of Examples 29-37, and identifying the stimulus includes identifying a portion of content provided to the subject.

  Example 39 includes the configuration requirements of Example 38, wherein identifying a portion of the content identifies at least one of a sequence of video frames within the content and one or more time offsets within the content. Including doing.

  Example 40 includes the components of any of Examples 29-39 and further includes stimulating the subject.

  Example 41 includes the requirements of Example 40, wherein stimulating the subject includes sending a message from the remote device to a stimulus controller and generating a stimulus that stimulates the subject.

  Example 42 includes any of the constituent requirements of Examples 29 to 41, and the anonymous data is analyzed to identify the estimated emotional state of the target, and to the estimated emotional state of the target Identifying the target emotional state of the subject based on, identifying a stimulus that transitions the subject to the target emotional state, and stimulating the subject using the stimulus.

  The foregoing description of the exemplary embodiments has been presented for purposes of illustration and description. The above description is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. Many modifications and variations are possible in light of this disclosure. It is intended that the scope of the disclosure be limited not by the above detailed description, but by the claims appended hereto. Applications filed in the future claiming priority over this application may claim the disclosed subject matter differently and will be variously disclosed or otherwise exemplified herein. May generally include any set of one or more limitations.

Claims (25)

An emotion detection system including an emotion detection device, wherein the emotion detection device includes:
Memory,
At least one processor coupled to the memory;
Receiving image data executable by the at least one processor and representing the face of the subject;
Analyzing the image data to identify a plurality of points on the face;
Storing an anonymous data set describing the plurality of points, each element of the anonymous data set including an identifier field and a location field;
Sending the anonymous data set to a remote device distinguishable from the emotion detection device,
A face image processing module configured to delete the image data;
Including emotion detection system.   The emotion detection system according to claim 1, wherein the image data includes at least one of two-dimensional image data and three-dimensional image data. The face image processing module includes:
Analyzing the image data at least in part by normalizing the data set to a generic face;
The emotion detection system of claim 1, configured to store the anonymous data set at least in part by storing the data set.   The emotion detection system according to claim 1, wherein the image data includes a landmark.   The emotion detection system according to claim 1, wherein the location field of each element of the anonymous data set includes at least one of a two-dimensional value and a three-dimensional value.   The emotion detection device further includes a camera configured to acquire the image data, and the face image processing module analyzes the image data, stores the anonymous data set, and the anonymous data The emotion detection system of claim 1, wherein the emotion detection system is configured to deactivate the camera prior to transmitting a set.   The emotion detection system according to claim 6, wherein the emotion detection device further includes a housing including the camera, the memory, the at least one processor, and the face image processing module. The remote device further comprising:
Identifying a stimulus provided to the subject prior to receiving the image data;
The emotion detection system according to claim 1, configured to analyze the anonymous data set and determine an emotional state of the subject when presented with the stimulus.   The emotion detection system according to claim 8, wherein the stimulus includes at least one of non-interactive content and interactive content.   The emotion detection system according to claim 8, wherein the stimulus includes a part of content provided to the subject.   The emotion detection system of claim 10, wherein the portion of content includes a sequence of video frames.   The emotion detection system according to claim 1, further comprising a stimulation controller configured to stimulate the subject.   The remote device, wherein the remote device is configured to send a message to the stimulus controller, and the stimulus controller is configured to stimulate the subject in response to receiving the message. The emotion detection system according to claim 12. The remote device further comprising:
Analyzing the anonymous data set to identify an estimated emotional state of the subject;
Identifying the target emotional state of the subject based on the estimated emotional state of the subject;
Identifying a stimulus that transitions the subject to the target emotional state;
The emotion detection system according to claim 1, wherein the emotion detection system is configured to stimulate the object using the stimulus. A method for detecting emotions using a device,
Receiving image data representing the subject's face;
Analyzing the image data to identify a plurality of points on the face;
Storing an anonymous data set describing the plurality of points, each element of the anonymous data set including an identifier field and a location field;
Transmitting the anonymous data set to a remote device distinguishable from the device;
Deleting the image data;
Including methods.   The method of claim 15, wherein analyzing the image data includes normalizing a data set to a generic face, and storing the anonymous data set includes storing the data set.   The method of claim 15, wherein analyzing the image data includes identifying landmark points. Obtaining the image data via a camera;
Deactivating the camera prior to analyzing the image data, storing the anonymous data set, and transmitting the anonymous data set;
16. The method of claim 15, further comprising: Identifying a stimulus provided to the subject prior to receiving the image data;
Analyzing the anonymous data set to determine the emotional state of the subject when presented with the stimulus;
The method according to any one of claims 15 to 18, further comprising:   The method of claim 19, wherein identifying the stimulus comprises identifying a portion of content provided to the subject.   21. The method of claim 20, wherein identifying the portion of content includes identifying at least one of a sequence of video frames in the content and one or more time offsets in the content. . A computer program for causing a processor to execute an operation,
Receiving image data representing the subject's face;
Analyzing the image data to identify a plurality of points on the face;
Storing an anonymous data set describing the plurality of points, wherein each element of the anonymous data set includes an identifier field and a location field;
Sending the anonymous data set to a remote device distinguishable from the processor;
Deleting the image data;
Including computer programs.   23. The computer program product of claim 22, wherein analyzing the image data includes normalizing a data set to a generic face, and storing the anonymous data set includes storing the data set. . The operation is
Identifying a stimulus provided to the subject prior to receiving the image data;
Analyzing the anonymous data set to determine the emotional state of the subject when presented with the stimulus;
The computer program according to claim 22 or 23, further comprising:   A computer readable storage medium storing the computer program according to any one of claims 22 to 24.

Images