JP2012243043A - Content evaluation device, method, and program thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a content evaluation device capable of accurately evaluating a content by determining presence/absence of a user's recollection behavior.SOLUTION: A content evaluation device 110 identifies a reaction of a user to an image content by analyzing a biological signal of the user viewing the image content. The content evaluation device 110 comprises: an acquisition part 120 for acquiring a cerebral blood flow and a sight line position of a user; a reaction identifying part 102 for determining whether or not the user shows a psychological reaction while the user is viewing an image content, on the basis of the cerebral blood flow; and a recollection determination part 106 for determining whether or not the user viewing the image content recalls other image content displayed before the currently displayed image content, on the basis of the sight line position of the user.

Description

  The present invention relates to a content evaluation apparatus, and more particularly to a content evaluation apparatus that identifies a user's response to image content by analyzing a biometric signal of the user viewing the image content.

  As a conventional technique, there is a method of evaluating content based on a biological signal such as a brain wave of a user who views the content (see, for example, Patent Document 1).

Special table 2010-520554 JP 2004-21870 A

  However, in the configuration of the prior art, since the user's recall state is not taken into consideration at all, it is difficult to accurately evaluate the content (Patent Document 1).

  In general, it is known that the cerebral blood flow in the frontal lobe changes when calculation or memory is performed in the head. It also changes when something is remembered (ie, recalled) or when some processing is performed in the head. Therefore, for example, when viewing content, cerebral blood flow may change even when the user actually remembers past content rather than feelings about the current content.

  Therefore, in a system that does not consider the user's recall state as in the prior art, when a change occurs in the cerebral blood flow, is the cause of the content viewed by the user at the time when the change occurred? Or, it cannot be evaluated whether the user who is in the recall state has the content viewed in the past. As a result, it is difficult for the system according to the prior art to accurately evaluate the content.

  An object of the present invention is to solve the conventional problems, and to provide a content evaluation apparatus capable of accurately evaluating content by determining the presence or absence of a user's recalling action.

  A content evaluation apparatus according to an aspect of the present invention is a content evaluation apparatus that identifies a user's reaction to the image content by analyzing a biological signal of the user who views the image content, the cerebral blood flow of the user And an eye-gaze position, a reaction identification unit for identifying whether or not the user has a psychological reaction when viewing the image content, and viewing the image content A recall determining unit that determines whether or not the user has recalled another image content displayed before the displayed image content based on the user's line-of-sight position.

  According to this configuration, the content evaluation device identifies the psychological reaction of the user who views the displayed content from the user's cerebral blood flow, and has the user recalled content other than the content currently displayed? It can be determined from the line-of-sight position. As a result, it is possible to evaluate whether the psychological reaction indicated by the user is for the content currently displayed. As a result, it is possible to accurately evaluate the content.

  The present invention can be realized not only as a content evaluation apparatus including such a characteristic processing unit, but also as a content evaluation method using a characteristic processing unit included in the content evaluation apparatus as a step. Also, the characteristic steps included in the content evaluation method can be realized as a program for causing a computer to execute. Such a program can be distributed via a recording medium such as a CD-ROM (Compact Disc-Read Only Memory) or a communication network such as the Internet.

  Furthermore, the present invention can be realized as a semiconductor integrated circuit (LSI) that realizes part or all of the functions of such a content evaluation apparatus, or can be realized as a content evaluation system including such a content evaluation apparatus.

  ADVANTAGE OF THE INVENTION According to this invention, the content evaluation apparatus which can evaluate a content accurately can be provided by determining the presence or absence of the recall action by a user.

It is a block diagram which shows the structure of the content evaluation apparatus in embodiment of this invention. It is a figure which shows the cerebral blood-flow detection method in embodiment and its modification. It is a figure which shows an example of the correspondence of viewing content and the variation | change_quantity of cerebral blood flow shown by experiment. It is a figure which shows an example of the change of the blood flow rate of the frontal lobe shown by experiment. It is a figure which shows the other example of the correspondence of viewing content and the variation | change_quantity of cerebral blood flow shown by experiment. It is a figure which shows the coordinate system of the screen in which a user views a content in embodiment and its modification. It is a figure explaining the recall information which the evaluation information storage part in an embodiment and its modification accumulates. It is a figure which shows the example of the user's eyes | visual_axis position at the time of recall behavior shown in experiment. It is a figure which shows an example of the correspondence of viewing-and-listening content shown by experiment, cerebral blood flow, and recall. It is a figure which shows the other example of the correspondence of viewing-and-listening content shown by experiment, cerebral blood flow, and a recall. It is a figure which shows an example of the content evaluation displayed on the evaluation display part in embodiment and its modification. It is a flowchart which shows the flow of a process of the content evaluation apparatus in embodiment of this invention. It is a detailed flowchart of the identification process of reaction in embodiment of this invention. It is a detailed flowchart of the calculation process of the residence time in embodiment of this invention. It is a detailed flowchart of the recall determination process in the embodiment of the present invention. It is a detailed flowchart of the specific process of reaction object in embodiment of this invention. It is a 1st figure explaining the recall degree determination process by the recall determination part in embodiment of this invention. It is a 2nd figure explaining the recall degree determination process by the recall determination part in embodiment of this invention. It is a 3rd figure explaining the recall degree determination process by the recall determination part in embodiment of this invention. It is a figure explaining the change etc. of the cerebral blood flow of the user who is viewing the content for movie advertisement in the experiment. It is a block diagram which shows the structure of the content evaluation apparatus which concerns on the modification 1 of embodiment of this invention. It is a figure which shows an example of the correspondence of viewing content and the variation | change_quantity of cerebral blood flow shown by the content evaluation apparatus which concerns on the modification 1 of embodiment of this invention. It is a block diagram which shows the structure of the content evaluation apparatus which concerns on the modification 2 of embodiment of this invention. It is a block diagram which shows the hardware constitutions of the computer system which implement | achieves the content evaluation apparatus which concerns on embodiment and its modifications 1 and 2 of this invention.

  A content evaluation apparatus according to an aspect of the present invention is a content evaluation apparatus that identifies a user's reaction to the image content by analyzing a biological signal of the user who views the image content, the cerebral blood flow of the user And an eye-gaze position, a reaction identification unit for identifying whether or not the user has a psychological reaction when viewing the image content, and viewing the image content A recall determining unit that determines whether or not the user has recalled another image content displayed before the displayed image content based on the user's line-of-sight position.

  According to this configuration, the content evaluation device identifies the psychological reaction of the user who views the displayed content from the user's cerebral blood flow, and has the user recalled content other than the content currently displayed? It can be determined from the line-of-sight position. As a result, it is possible to evaluate whether the psychological reaction indicated by the user is for the content currently displayed. As a result, it is possible to accurately evaluate the content.

  In addition, when the user shows a psychological reaction when viewing the image content, and the recall determination unit determines that the user is recalling the other image content, A reaction target specifying unit for specifying the image content as a target for which the user has shown a psychological reaction may be further provided.

  According to this structure, the reaction object specific | specification part with which a content evaluation apparatus is provided can specify the object for which the user showed psychological reaction.

  Further, the reaction identification unit determines in advance when the absolute value of the acquired cerebral blood flow exceeds a predetermined threshold, or the absolute value of the integrated value of the cerebral blood flow in a predetermined time interval is determined in advance. If the measured value is exceeded, the user may be identified as having a psychological reaction.

  According to this configuration, the reaction identification unit can determine whether or not the user has shown a psychological reaction to the content by comparing the value of the user's cerebral blood flow with a threshold value.

  In addition, the recall determination unit is configured so that a degree of deviation between a model formulated in advance as a general line-of-sight position when viewing the image content and the user's line-of-sight position is greater than a predetermined value. The user may determine that the user is in the recall state.

  According to this configuration, the recall determination unit compares a model determined in advance as a user's typical line-of-sight position during content viewing with the user's acquired line-of-sight position, and compares the degree of deviation with a threshold value. Thus, it can be determined whether or not the user is recalling.

  More specifically, a dwell time calculation unit that calculates a dwell time, which is a time during which the movement distance of the line-of-sight position is within a predetermined threshold, and display of a scene that is each time-divided of the image content A recall information storage unit that stores recall information that is information that associates a time with a display area of a provided image that is an image that is displayed as part of the image content at the display time; The unit determines the distance between the line-of-sight position in the display time of the scene and the display area associated with the display time, and the dwell time in the display time for each scene. It may be determined whether or not the threshold is equal to or greater than the threshold, and if the threshold is equal to or greater than the threshold, the user may be determined to be in the recall state.

  The recall determination unit may calculate the recall level as a ratio of the total number of scenes in which the user is determined to be in the recall state with respect to the total number of scenes included in the recall information.

  According to this, the recall level determination unit can calculate the user's recall level for the entire content including a plurality of scenes as a numerical value.

  In addition, when the acquired cerebral blood flow exceeds a first threshold, the reaction identifying unit identifies the type of psychological reaction of the user as an excited state, and the cerebral blood flow is If the threshold value is below 2, the type of psychological reaction of the user is identified as being in a concentrated state, and the first threshold value is larger than the cerebral blood flow of the user at the reference time. The second threshold value may be a value smaller than the cerebral blood flow of the user at the reference time.

  According to this, the reaction identifying unit can identify whether the type of reaction indicated by the user with respect to the content is an excited system or a concentrated system. As a result, the content evaluation apparatus can evaluate the content in more detail.

  In addition, the reaction identification unit, the time when the acquired absolute value of the cerebral blood flow exceeds the predetermined threshold, and the absolute value of the integrated value of the cerebral blood flow in the predetermined time interval The time that is included in the time interval from any time at which the value exceeds the predetermined value to a time that is back by a predetermined time, the user performs the psychological reaction. As the reaction time candidate that is a candidate for the indicated time, the reaction target specifying unit specifies the target on which the user has shown the psychological reaction based on the reaction time candidate and the recall level. Also good.

  According to this, even when the reaction target specifying unit included in the content evaluation device showed a psychological reaction when the user was in a recall state, the user showed a psychological reaction to which content Can be specified in more detail.

  Specifically, when the recall level is smaller than a predetermined value, the reaction target specifying unit indicates the psychological reaction of the image content viewed by the reaction time candidate. When it is specified as an object and the recall level is equal to or greater than a predetermined value, the image content viewed before the reaction time candidate is specified as an object for which the user has exhibited the psychological reaction. It is good.

  In addition, when the provided image represents character information, the recall determining unit determines that the user does not move when the amount of movement of the line-of-sight position within the display time of the provided image is less than a predetermined threshold. The recall state may be determined.

  According to this, the content evaluation device can more accurately determine the recall state for the character information included in the content.

  The recall determination unit may be configured such that when the line-of-sight position is located in the display area of the provided image and the line-of-sight position is located in the display area from before the display time of the provided image, It may not be determined that the user is in the recall state.

  According to this, when the user's line-of-sight position coincides with the display area of the provided image, the recall determination unit prevents the determination that the user is not in the recall state even though it is originally in the recall state. Can do.

  In addition, when the scene includes a plurality of display areas, the recall determination unit may calculate the distance using a degree of separation from the display area closest to the line-of-sight position.

  According to this, even when a plurality of provided images are displayed simultaneously, the recall determination unit can appropriately determine the recall state.

  Further, the recall determination unit determines in advance a distance between the line-of-sight position in the display time and the display area associated with the display time, and the dwell time in the display time for each scene. It is also possible to determine whether the user is recalling another scene displayed before the scene when each of the thresholds is equal to or greater than the threshold.

  According to this, the recall determination unit 106 can determine whether or not the user is in the recall state at finer time intervals.

  The recall information may have a weight for each scene, and the recall determination unit may add up the total number of scenes determined to be the recall state by multiplying the weight corresponding to the scene.

  In general, the degree of interest in the content of the user varies depending on the scene. Therefore, a more appropriate recall level can be calculated by assigning a weight to each scene according to the degree of interest of the user assumed in advance and counting the number of weighted scenes.

  The content evaluation apparatus displays a plurality of image contents, and stores a stimulus presentation order storage unit that stores a display order of the plurality of image contents, and the plurality of stimulus presentation order storage units. The stimulus presentation order changing unit that changes the display order of the image content, and the plurality of contents are displayed based on the order stored in the stimulus presentation order storage unit or the display order changed by the stimulus presentation order changing unit. A stimulus presentation unit may be further provided.

  Specifically, the stimulus presentation order changing unit may change the display order so that the other image content specified by the reaction target specifying unit is displayed again.

  More specifically, the reaction target specifying unit is configured to increase or decrease the cerebral blood flow when the other image content is displayed for the first time, and when the other image content is displayed for the second time or later. Based on the difference from the increase / decrease amount of cerebral blood flow, it may be determined whether or not the user has recalled the other image content.

  According to this, the intensity of the user's interest in the content can be measured from the change in cerebral blood flow for each number of times of viewing when the user views the same content a plurality of times. Here, it is considered that the user has a strong interest in the recalled content. Therefore, the reaction target specifying unit can specifically specify content having a large increase or decrease in cerebral blood flow despite being viewed multiple times as a target for which the user has exhibited a psychological reaction.

  Further, the content evaluation device displays a plurality of image contents, and stores a display order of the plurality of contents stored in the stimulus presentation order storage unit, and the order stored in the stimulus presentation order storage unit Each of the plurality of image contents using a stimulus presentation unit that displays a plurality of contents, a line-of-sight history storage unit that stores the user's line-of-sight position as a history, and the line-of-sight position stored in the line-of-sight history storage unit A typical recall information generation unit that identifies the line-of-sight position when “” is displayed and generates the recall information including the identified line-of-sight position as the display area may be further included.

  According to this, the content creator who uses the content evaluation apparatus can reduce the man-hours required for preparing the recall information in advance.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment)
A content evaluation apparatus according to an embodiment will be described.

  FIG. 1 is a block diagram showing a configuration of a content evaluation apparatus according to an embodiment of the present invention.

  In FIG. 1, a content evaluation apparatus 110 is a content evaluation apparatus that identifies a user's response to image content by analyzing a user's biological signal for viewing the image content, and includes an acquisition unit 120, a reaction identification unit 102, and the like. , A residence time calculation unit 104, a recall information storage unit 105, a recall determination unit 106, and a reaction target specifying unit 107.

  The acquisition unit 120 acquires a biological signal including the user's cerebral blood flow and line-of-sight position. Specifically, the acquisition unit 120 includes a cerebral blood flow detection unit 320 and a line-of-sight detection unit 103. As will be described later, the cerebral blood flow detection unit 320 is a sensor that detects cerebral blood flow by, for example, near infrared spectroscopy. The line-of-sight detection unit 103 is a sensor that measures a line-of-sight position that is a position where the user gazes from a pupil position acquired by a camera, for example.

  The cerebral blood flow detection unit 320 is a means for detecting a change in cerebral blood flow. FIG. 2A is a diagram illustrating an example of the cerebral blood flow detection unit 320 in the present embodiment. The cerebral blood flow detection unit 320 includes, for example, a near-infrared sensor (NIRS; Near-Infrared Spectroscopy sensor). In recent years, it has been known that changes in cerebral blood flow can be detected by using this NIRS sensor. The outline will be described below.

  Hemoglobin in the blood sends oxygen into the body by combining with oxygen. The NIRS sensor includes a light emitting unit 322 including a light emitting element that emits near infrared light (wavelength 700 nm to 1000 nm), and a light receiving unit 321 including a light receiving element that receives the emitted near infrared light. The NIRS sensor irradiates near-infrared rays from the light emitting unit 322 toward the inside of the body, and the light receiving unit 321 receives light transmitted through the body tissue. By examining the component of the received light, the NIRS sensor can detect the hemoglobin oxygenation state in the blood of the brain. For example, it is possible to detect the oxygen blood flow in the frontal lobe by wearing it on the forehead or the like.

  The reaction identifying unit 102 identifies whether the user has exhibited a psychological reaction when viewing the image content based on the cerebral blood flow. That is, it is a means for detecting the user's reaction from the change in the blood flow of the brain detected by the acquisition unit 120. In the present embodiment, the type of reaction such as a predetermined emotion to the user's product and CM based on a change in cerebral blood flow when the user views content such as a product description video or CM (Commercial Message). Specify the reaction type.

  FIG. 2B is an example when viewing content. The user who is the subject 250 is viewing the content 220 including the CM of the camera B.

  The recall determination unit 106 determines whether the user viewing the image content has recalled another image content displayed before the displayed image content, based on the user's line of sight. Details of the recall determination unit 106 will be described later.

  The reaction target specifying unit 107 is a case where the user shows a psychological reaction when viewing the image content, and when the recall determination unit 106 determines that the user has recalled another image content, The image content is identified as an object for which the user has shown a psychological reaction. Details of the reaction target specifying unit 107 will be described later.

  FIG. 3 is a diagram illustrating an example of a correspondence relationship between the viewing content and the amount of change in cerebral blood flow, which is shown by an experiment. FIG. 3B is a schematic diagram of content to be viewed. The content to be viewed is a movie related to the product description. For example, the product description content 222 of the single-lens camera of company A is displayed from 0 seconds to 30 seconds after the start of measurement. Next, a product description content 223 of the digital camera of company B is displayed from 30 seconds to 60 seconds. Next, a product description content 224 of the digital camera of company C is displayed from 60 seconds to 90 seconds. Next, the product description content 225 of the digital camera of company D is displayed from 90 seconds to 120 seconds.

  On the other hand, Fig.3 (a) is a figure which shows a user's cerebral blood flow at the time of viewing the content shown in FIG.3 (b). Specifically, the experiment is performed using an NIRS sensor to detect a change in the blood flow in the frontal lobe of the user when viewing the content, and is a graph.

  The sampling interval for detecting cerebral blood flow is 0.6 seconds, the horizontal axis is time (seconds), and the vertical axis is the reference time, the amount of change in hemoglobin from the start of measurement (mMol · mm (millimeter · millimeter)) It is said. Three graphs superimposed on FIG. 3A are a graph 402 showing the amount of change in oxygenated hemoglobin (oxy-Hb) concentration, which is hemoglobin bound to oxygen, and deoxidation, which is hemoglobin not bound to oxygen. Three types of increase / decrease amounts are shown by lines in a graph 404 showing the amount of change in hemoglobin (deoxy-Hb) concentration and a graph 403 showing the amount of change in total hemoglobin (total-Hb) concentration.

  Generally, when cerebral blood flow increases, oxyhemoglobin increases with a delay of several seconds. Therefore, the amount of change in cerebral blood flow can be acquired by measuring the amount of change in oxyhemoglobin concentration. Hereinafter, for the sake of simplicity, the value of oxyhemoglobin concentration at the reference time (for example, at the start of measurement of cerebral blood flow) is set to 0, and the amount of change from the reference time is referred to as cerebral blood flow. That is, when the cerebral blood flow is positive, it corresponds to an increase in oxyhemoglobin compared to the reference time, and when the cerebral blood flow is negative, it corresponds to a decrease in oxyhemoglobin than the reference time. Further, the fact that the oxyhemoglobin concentration exceeds the threshold value is expressed as the cerebral blood flow exceeds the threshold value, and the fact that the oxyhemoglobin concentration falls below the threshold value is expressed as that the cerebral blood flow falls below the threshold value.

  As shown in FIG. 3 (a), it can be seen that each graph of hemoglobin concentration maintains a substantially constant value although it slightly increases or decreases from 0 to 30 seconds and from 30 to 60 seconds. . On the other hand, it can be seen that the concentration of oxyhemoglobin and total hemoglobin gradually increased from around 65 seconds and then decreased again.

  Here, the relationship between a change in cerebral blood flow and a user's psychological change with respect to content will be described with reference to FIG.

  In general, when the brain imposes tasks such as memory (working memory), it consumes oxygen and acts, and in order to supplement it, more oxygen is supplied from the blood. It is known that hemoglobin and total hemoglobin increase (bold reaction). It is also known that cerebral blood flow increases due to excitement and impatience. For example, utilizing the characteristic that cerebral blood flow increases due to impatience when lying, it is also known to be applied to a lie detector (Non-Patent Document 1).

  On the other hand, it is also known that oxygenated hemoglobin in the frontal lobe decreases when performing tasks intensively such as in games. Therefore, in the NIRS sensor, it is common to pay attention to the increase / decrease amount of oxyhemoglobin or total hemoglobin, or a combination thereof. However, there has been no technology disclosed about the relationship between cerebral blood flow and a user's psychological change when viewing content. Therefore, the inventors conducted the following experiment.

  First, dozens of users, who are test subjects, watch multiple contents (product description videos or commercials that are continuously played back), and measure changes in cerebral blood flow during viewing did. In addition, they were asked to verbally answer the psychological state and impressions of the content they watched. With reference to FIG. 4, the relationship between the change in cerebral blood flow obtained from the experimental results and the psychological state will be reported.

  FIG. 4 is a graph showing a change in blood flow in the frontal lobe of the user during content viewing using an actual NIRS sensor. The sampling interval of detection was 0.6 seconds, the horizontal axis was time (seconds), and the vertical axis was the amount of change in hemoglobin concentration from the measurement start point (mMol · mm (mmol · millimeter)). For each graph, three types of increase / decrease amounts of oxygenated hemoglobin (oxy-Hb) which is hemoglobin bound to oxygen, deoxygenated hemoglobin (deoxy-Hb) which is not bound hemoglobin, and total hemoglobin (total-Hb) Are indicated by different line types.

  In the experimental results, there were many users who did not change cerebral blood flow for content or scenes that had no particular impression and content or scenes of low interest. Here, a scene is a part of content divided in time.

  For example, FIG. 4A shows a typical data transition in the case of a change in cerebral blood flow when a certain user is viewing a certain content and has no impression. It can be seen that the amount of increase / decrease in the respective concentrations of deoxygenated hemoglobin, oxygenated hemoglobin, and total hemoglobin is small, and all of them retain a substantially constant amount. In addition, the verbal answer by the user indicated that the user has no particular impression of this content and is not interested. The user is considered to be viewing the content indifferently. In addition, it was a situation in which the functions and products appealing in the content were indifferently viewed.

  On the other hand, in the experimental results, there are many users who see changes in cerebral blood flow for content or scenes that are interested, excited, surprised, or nervous. In particular, many users have increased oxyhemoglobin concentration and total hemoglobin concentration. FIG. 4B shows an example of a change in cerebral blood flow when a user is viewing a certain content, and shows an example of a change in cerebral blood flow when the user is interested in the viewing content. is there. Although the deoxyhemoglobin concentration is almost constant, it can be seen that the oxygenated hemoglobin concentration and the total hemoglobin concentration increase with the passage of time, and thereafter the oxygenated hemoglobin concentration decreases. In addition, the verbal response by the user was very impressive for this content, and it was answered that it was stunning. In particular, there are many cases where people or scenes such as actors and actresses appearing in the content have some feelings. On the other hand, because some emotional change has occurred to the person or scene, the function and product appealing in the content is not fully aware, not impressed, or not at all, Sometimes I didn't remember.

  In addition, in the experimental results, there are many users who can see changes in cerebral blood flow for content that they are interested in and content that has been focused on. In particular, many users have decreased oxyhemoglobin concentration and total hemoglobin concentration. FIG. 4C shows an example of a change in cerebral blood flow when a certain user views a certain content, and the user concentrates on viewing the content. Although the deoxygenated hemoglobin concentration is almost constant, it can be seen that the oxidized hemoglobin concentration and the total hemoglobin concentration decrease with time and increase thereafter to recover. The verbal response by the user is very impressive for this content, and it is answered that it was viewed in a concentrated manner. The situation is such that the user is calmly watching the content. In addition, he also paid attention to the functions and products appealing in the content, and received a response that left an impression.

  Based on the obtained knowledge, the reaction identification unit 102 included in the content evaluation apparatus 110 according to the present embodiment uses a determination method for determining the type of the user's psychological state by the following process.

  First, when a predetermined threshold is provided and the absolute value of the change in cerebral blood flow from the reference time is less than the threshold, it is assumed that the psychological state of the user content is a state of low interest or impression.

  In addition, when the absolute value of the change in cerebral blood flow is equal to or greater than the threshold, the psychological state of the user content is a state of high interest or impression.

  As a result of verifying the above determination method using data obtained from subject experiments, the correctness rate of the psychological state determined based on cerebral blood flow was 70% or higher compared to the correct answer actually obtained from the user. It was.

  In addition, there was a very high correlation between the psychological state of being excited, surprised, and nervous about the viewing content, and the change in cerebral blood flow in which the oxyhemoglobin concentration increased. In addition, the correlation between the psychological state of seeing or concentrating on the viewing content and the change in cerebral blood flow in which the oxyhemoglobin concentration decreases was very high.

  In fact, there are many unexplained parts about changes in human psychological state and blood flow in the brain, and it is based on the current experimental results to specify higher psychological states such as product likes and dislikes, content likes and dislikes, and preferences. It is difficult. However, based on the above experimental results, it was determined whether the content was viewed with excitement at least to leave an impression (defined as “reactive”) or not (defined as “non-reactive”). It can be identified based on the increase or decrease in cerebral blood flow.

  In addition, it is a content that has been reacted, and the way of interest is excited, surprised, or nervous (defined as "excited system"), looked at, or concentrated The type of response to content, which is a psychological state (defined as “concentrated system”), can also be broadly classified by distinguishing whether cerebral blood flow has increased or decreased.

  Therefore, in the present invention, the user's reaction to the content is any one of the above three types (non-reactive system, excitatory system, and concentrated system), and the reaction identification unit 102 selects one of the reactions based on the change in cerebral blood flow. And the time when the reaction of the reaction system is shown (hereinafter referred to as a reaction time candidate) is specified.

  In the present invention, attention is paid to oxyhemoglobin concentration in the cerebral blood flow, and in the following, for the sake of explanation, the reaction type is recognized using the increase / decrease amount of oxyhemoglobin concentration. However, the increase / decrease amount of the oxygenated hemoglobin concentration is not necessarily limited, and the increase / decrease amount of the total hemoglobin concentration or a combination thereof may be used. For example, if the oxyhemoglobin concentration and the total hemoglobin concentration both increase or decrease in the same direction, the reaction system is specified, or if the oxyhemoglobin concentration increases more than a predetermined ratio with respect to the increase in the total hemoglobin concentration, the reaction occurs. It may be specified as a system.

  In addition, the reaction identification unit 102 uses smoothing processing or deoxygenated hemoglobin for the measurement result of the cerebral blood flow acquired from the cerebral blood flow detection unit 320 as the cerebral blood flow used to identify the reaction type. Baseline correction may be performed. Since cerebral blood flow includes noise due to physiological phenomena, it is possible to remove noise by performing smoothing processing. In addition, by performing baseline correction based on deoxygenated hemoglobin, it is possible to remove the increase due to noise and physiological phenomena and grasp the increase or decrease in cerebral blood flow in detail.

  Again, a description will be given with reference to FIG. The reaction identification unit 102 provides a first threshold 412 and a second threshold 413 as predetermined thresholds. For example, the first threshold 412 is set to 0.05, and the second threshold 413 is set to −0.05. At this time, the reaction identification unit 102 recognizes a case where the absolute value of the increase / decrease amount of the cerebral blood flow exceeds a predetermined threshold as a “reactive system”. That is, when the cerebral blood flow exceeds 0.05 or falls below -0.05, the reaction identification unit 102 recognizes the reaction type as a reactive system.

  More specifically, when the cerebral blood flow exceeds the first threshold value 412 that is a positive threshold value in the increasing direction, the reaction identifying unit 102 specifies that the reaction type is “exciting system” and sets a negative threshold value. When a certain second threshold 413 is exceeded in the decreasing direction, it is specified that the reaction type is “concentrated”.

  On the other hand, when the cerebral blood flow is within a threshold value (for example, within ± 0.05), the reaction identifying unit 102 specifies the reaction type as “no reaction system”. Also, the time specified as the reaction type “reactive system” (that is, “excited system” or “concentrated system”) is specified as a reaction time candidate.

  That is, the reaction identifying unit 102 determines the absolute value of the integrated value of the cerebral blood flow in a predetermined time interval when the absolute value of the acquired cerebral blood flow exceeds a predetermined threshold value or If the value exceeds the value, the user is identified as having a psychological reaction.

  More specifically, when the acquired cerebral blood flow exceeds the first threshold, the reaction identifying unit 102 identifies the type of psychological reaction of the user as being in an excited state, and the cerebral blood flow is When the threshold value is below 2, the user's psychological reaction type is identified as being in a concentrated state. Here, the first threshold value is larger than the user's cerebral blood flow at the reference time, and the second threshold value is smaller than the user's cerebral blood flow at the reference time.

  In FIG. 3, the product description video of the single-lens camera of Company A from 0 to 30 seconds is displayed, and the product description video of the digital camera of Company B is displayed from 30 to 60 seconds. Between, since the cerebral blood flow measured as the oxygenated hemoglobin concentration is within the threshold, the reaction identifying unit 102 specifies the reaction type as “non-reactive system”. On the other hand, a movie describing the product description of the digital camera of company C is displayed from 60 seconds, and the amount of oxyhemoglobin increases above the threshold value in 65 seconds. Therefore, the reaction identification unit 102 identifies the reaction type in the reaction time candidate 420 when 65 seconds have elapsed after the start of measurement as “excited system”.

  In general, the increase or decrease in cerebral blood flow also depends on individual differences and the part of the brain to be detected. Therefore, for example, by watching a video such as a blank in advance and calculating the average value of the increase or decrease in cerebral blood flow during that time as a threshold for recognizing it as “non-reactive system”, calibration is performed for each individual before starting the measurement. Also good.

  FIG. 5 is a diagram illustrating another example of the correspondence relationship between the content to be viewed and the amount of change in cerebral blood flow, which is shown by experiments. Specifically, it is a diagram showing the cerebral blood flow of another user who has viewed the same content as in the case of FIG. FIG. 5B is a schematic diagram of content to be viewed.

  FIG. 5A is a diagram illustrating the cerebral blood flow of the user when viewing the content. That is, FIG. 3 is a graph showing a change in blood flow in the frontal lobe of a user when viewing content using an NIRS sensor. The horizontal axis of the graph represents time, and the vertical axis represents brain blood flow. Here, the threshold value 412 is set to 0.05, and the threshold value 413 is set to −0.05.

  In FIG. 5A, the product description content 222 of company A displayed between 0 seconds and 30 seconds, the product description content 223 of company B displayed between 30 seconds and 60 seconds, and 60 seconds. It can be seen that the blood flow is within the threshold during the content 224 of the product description of the company C displayed during 90 seconds from the start. On the other hand, at the point of 90 seconds after the start of measurement, the graph 402 showing the oxygenated hemoglobin concentration exceeds the threshold value 412 and it can be seen that the blood flow in the brain is increased. Thus, it can be seen that the cerebral blood flow reflects the psychological state of the user and increases or decreases depending on the psychological state such as concentration on content, increased interest, and excitement.

  Next, processing for specifying which content and scene the obtained response is for will be described in order.

  With reference to FIG. 1, the line-of-sight detection unit 103 detects the line of sight of the user who is viewing the content. The line-of-sight detection unit 103 detects, for example, coordinates on a screen on which content is displayed, such as a television screen, a PC screen, or a mobile screen, every predetermined time.

  With reference to FIG. 6, the gaze detection by the gaze detection unit 103 will be described. FIG. 6 is a diagram showing a coordinate system of a screen on which a user views content in the present embodiment. The horizontal axis of the screen is the X coordinate, the vertical axis is the Y coordinate, and the upper left is the origin (0, 0). One pixel is represented as one unit, the X axis represents the right as positive, and the Y axis represents the lower as positive. FIG. 6A is a diagram for explaining positions and areas with respect to the screen. In FIG. 6A, for example, the line-of-sight position when the user is gazing at the point A is (200, 300). Further, the line-of-sight position when the user is gazing at the point A ′ moved 600 pixels to the right and 100 pixels downward from the point A is (800, 400). In addition, a predetermined rectangular area such as a rectangular area B surrounded by the points A and A ′ (the X coordinate of the upper left point, the Y coordinate of the upper left point, the X coordinate of the lower right point, Y coordinate of the point). For example, the region B is (200, 300, 800, 400). The line-of-sight detection unit 103 detects the line-of-sight position every predetermined time, such as every 10 ms (millisec). Thereby, the content evaluation apparatus 110 can measure the movement of the user's line of sight, the staying time of the line of sight in a predetermined area, and the like. Note that various methods have been known for detecting the eye gaze position, such as taking an eyeball image of the subject and detecting the eye gaze of the subject using the position of the pupil in the eyeball (for example, Patent Documents). 2), the gaze detection unit 103 according to the present invention is also detected by the same method.

  The dwell time calculation unit 104 calculates a dwell time that is a time during which the movement distance of the line-of-sight position is within a predetermined threshold. Specifically, the stay position at which the line of sight stays and the stay time at the stay position are calculated from the transition of the line-of-sight position detected by the line-of-sight detection unit 103 every predetermined time. For example, in FIG. 6B, it is assumed that the line of sight is positioned at C (400, 380) from 3.5 seconds to 5.5 seconds after the start of measurement of the biological signal. In this case, the residence time calculation unit 104 calculates that the residence position is coordinates C (400, 380) and the corresponding residence time is 2 seconds (= 5.5-3.5). In this way, the residence time calculation unit 104 can calculate the residence time by calculating the time difference.

  Referring to FIG. 1 again, the recall determination unit 106 determines whether or not the user has a recall action such as a reminder.

  The content evaluation system according to the present embodiment evaluates the content by having the user view the content and estimating the user's psychological state based on the cerebral blood flow during viewing. Therefore, it is very important to specify which of the plurality of contents viewed by the user is the reaction obtained from the user.

  Even when a result showing a psychological reaction such as excitement or interest is obtained, for example, when the cerebral blood flow increases, the user actually thinks about the content that has just been viewed. It is very common to notice something and remember it. For example, in a system that allows a user to continuously view content and compare the evaluation of the content for each content based on the acquired biological signal, the psychological reaction indicated by the obtained biological signal It is difficult to identify whether the content corresponds to the content. In addition, in a system that allows a user to watch a drama or a movie in which a plurality of different scenes continue, and evaluates the scene and estimates emotion based on the acquired biological signal, It is difficult to specify whether the scene corresponds to the scene.

  In fact, the user who showed the change in the cerebral blood flow shown in FIG. 5 described above immediately after watching the video of the company C in the oral question after the experiment, “Since the product of the company C was unexpected, “I was watching and then watching”. On the other hand, there is a reply that the video of Company D is not impressive at all. In this way, the results of increase or decrease in cerebral blood flow are also obtained from other experiments when remembering or thinking. This is because the user's recall behavior is also considered to be a working memory behavior such as short-term memory, and changes in the blood flow of the brain occur.

  In the present embodiment, the action of performing such reminding is defined as “recall”, and the determination of the recall state by the user will be described below.

  Referring to FIG. 1 again, the recall information storage unit 105 displays a display time of a scene that is each time-divided image content, and a display area of a provided image that is an image displayed as part of the image content at the display time. Recall information, which is information that associates Specifically, “recollection information” that is information about the content and the time of the content displayed on the screen is stored.

  In general, the content generally has a scene or region that should be focused on every time. The content is created so that the user can gain an understanding of the content by following these areas with his gaze. FIG. 7 is a diagram for explaining the recall information.

  As shown in FIG. 7, in the recall information storage unit 105, as the recall information, the “name” of the content, the content to be noticed for each time (assumed to be an attribute), and the area where the content is displayed are “ It is stored as a “scene tag”.

  Specifically, the recall information storage unit 105 includes at least one or more recall information 1105. Each recall information 1105 includes, for example, a content name 1108 and one or more scene tags 1110. One scene tag 1110 corresponds to one scene.

  The scene tag 1110 associates an identifier, a display time during which the provided image is displayed in the content, a display area of the provided image, and an attribute of the provided image. One scene tag may include a plurality of provided images.

  Here, the provided image is an image selected from images included in the content because the content creator wants the user to pay attention. The attribute of the provided image may be, for example, the role (product description, function description) in the content of the provided image, and the type of information indicated by the provided image (logo that conveys character information, video information) Or an external photograph).

  More specifically, when showing the CMs of four companies, A company CM, B company CM, C company CM, and D company CM to the user, and obtaining an evaluation for each CM, the CM of each company is content, One recall information is defined in association with the content. Each content includes provided images such as an external photograph of a camera, a brand name logo, and a usage scene, and the provided image is specified by a scene tag.

  For example, referring to the recall information 1105 corresponding to the content 3, the name of the content is “Company C digital camera CM”. The scene tag with the identifier 1 indicates that the display time of the corresponding provided image is 1 to 2 seconds, the display area is (200, 300, 700, 400), and the attribute is an actress. Yes.

  This means that the CM talent actress is displayed in the display area (200, 300, 700, 400) for 1 to 2 seconds.

  The scene tag with the identifier 2 indicates that the display time of the corresponding provided image is 3 to 4 seconds, the display area is (200, 300, 700, 600), and the attribute is a camera. Has been.

  This means that the provided image representing the camera of company C is displayed in the display area (200, 300, 700, 600) for 3 to 4 seconds.

  As described above, the recall information storage unit 105 stores, as the recall information, the display area of the provided image that the content production side wants to pay attention to or the display area of the provided image that the user generally considers to pay attention to. ing. Also, the user side generally pays attention to these areas, and understands the contents by watching these areas over time. Then, paying attention to the display area and understanding the content, a psychological reaction for each user such as the presence or absence of interest is formed.

  However, when the user is in a recall state while viewing the content, the inventors' experiments indicate that the line of sight is located in a region that is different from the region that the user is likely to focus on in the currently displayed content. It was revealed. In this case, it has also been clarified that the line of sight tends to stay for a while. These reasons are considered as follows.

  Specifically, the region that is easy for the user to pay attention is a region in which an image that easily attracts the user's interest is displayed, such as a display region of a provided image that represents an external view of an actress, an actor, or a product. However, when the user is in the recall state, if the line of sight goes to these easy-to-focus areas, the user concentrates on that area, making it difficult to continue the recall action. Therefore, it is considered that the user unconsciously removes the line of sight from the region where attention is easily focused. Further, as described above, when the user is in the recall state, the amount of movement of the line of sight decreases, and the line of sight often stays for a while at a position other than the region that is easily noticed. The reason for this is thought to be that, as a result of the user moving his / her line of sight as described above, if attention is paid to a region that is easily noticed displayed in the current content, it is difficult to continue the recalling action.

  Therefore, the recall determination unit 106 determines the recall state based on the position of the line of sight with respect to the display area, which is an area that is easy for the user to focus on, and the dwell time of the line of sight position based on the recall information. In other words, the recall determination unit 106 determines that the user's line-of-sight position is larger than a predetermined value and a model that is formulated in advance as a general line-of-sight position when viewing image content. Is determined to have been recalled (ie, it was in a recall state).

  More specifically, the recall determination unit 106 determines, for each scene, the distance between the line-of-sight position in the display time of the scene and the display area associated with the display time, and the residence time in the display time. If it is equal to or greater than the threshold value, and if it is equal to or greater than the threshold value, it is determined that the user is in the recall state.

  For example, the recall determination unit 106 calculates the vertical distance from the line-of-sight position to the display area, and when the distance is equal to or greater than a predetermined threshold and the dwell time at the line-of-sight position is equal to or greater than the predetermined threshold, the user It is determined that Furthermore, the recall determination unit 106 may calculate the recall level by the ratio of scenes determined to be in the recall state among all scenes included in one content. Hereinafter, a specific example will be described.

  FIG. 6B is a diagram illustrating an example of a user's line-of-sight position at the time of the recall action, which is shown in the experiment. Specifically, some scenes of moving image content introducing products of company C are displayed on the screen. On the screen, as a description of the product, a provided image representing characters “with a camera shake correction function” is displayed in the display area (200, 300, 800, 400). On the other hand, the user's line of sight is located at line of sight C (400, 380) during this period. Since the line of sight is located in the display area, it can be considered that the user is paying attention to the product description.

  On the other hand, when the user is recalling, the user's line of sight often stays at a position outside the display area. FIG. 8 is a diagram illustrating an example of the line-of-sight position of the user during the recalling action, which is shown in the experiment. Specifically, FIG. 8A shows a scene of a part of the moving image content introducing the product of company C, as in FIG. 6B. Unlike the user who is the measurement target of the line of sight shown in FIG. 6B, the line of sight of the user who is the measurement target of the line of sight shown in FIG. 8A is located at the line of sight D (200, 100). I understand that. Also, this scene has 3 seconds, and the user's line-of-sight position stays at coordinates (200, 100) for 3 seconds.

  Originally, it is thought that the user focuses on the provided image representing the character “with camera shake correction function” related to the description of the product. However, since the user is recalling, the line of sight is greatly deviated from the display area of the provided image, and the same position. It turns out that it is staying in. The vertical distance to the area with respect to the coordinates (200, 100) of the line of sight is the distance from the coordinates (200, 100) to the coordinates (200, 300), and is 200 (= (0, 200)). For example, if the distance threshold is 100 and the residence time threshold is 2 seconds, both the distance and the residence time are equal to or greater than the threshold value, and the recall determination unit 106 takes the recall action (that is, in the recall state) It is determined.

  When the content is a moving image or the like, each content can include a plurality of provided images. In this case, even if the recall determination unit 106 determines only the user's recall state for some of the provided images, the recall state for the entire content cannot be accurately determined.

  Therefore, for example, the recall determination unit 106 totals the number of recall actions by the user while viewing each content, and calculates the recall degree of the user for the content at the ratio.

  For example, referring again to FIG. 7A, “C company digital camera CM” of content 3 includes a total of 10 scene tags. Here, for example, the user takes a recall action only for the provided image corresponding to the scene tag (scene tag 1) having the identifier 1 out of 10, and for the provided images corresponding to the other 9 scene tags. It is assumed that the user's line of sight is located within a predetermined display area. In this case, the recall determination unit 106 calculates the recall level of the user for the content 3 as 1/10 = 10%. That is, the recall determination unit 106 calculates the recall level as a ratio of the total number of scenes determined to be in the recall state by the user with respect to the total number of scenes included in the recall information.

  The recall determination unit 106 may calculate the recall level based on the total time determined as the recall state. This is because the longer the time the line of sight departs from the region, the higher the possibility of recalling. In this case, the recall determination unit 106, for example, the total display time corresponding to the scene tag determined to be in the recall state with respect to the total display time corresponding to all scene tags included in the recall information. As a ratio, the degree of recall may be calculated.

  9 and 10 are diagrams for explaining the recall level. Fig.9 (a) is a figure which shows the measurement result of the same user's cerebral blood flow as Fig.3 (a). FIG. 9B shows an outline of four contents viewed by the user. FIG. 9C shows the recall level calculated by the recall determination unit 106 for each content.

  For example, when viewing the video content for the product explanation of Company A from 0 to 30 seconds, the user's line-of-sight position is far from the display area by more than a threshold in two of the ten scenes. The recall determination unit 106 calculates the recall level as 20%. On the other hand, at the time of viewing the video content for product explanation of Company B, Company C, and Company D, the user's line of sight is located in the display area, and the recall level is calculated as 0%.

  Fig.10 (a) is a figure which shows the cerebral blood flow of the same user as Fig.5 (a). FIG. 10B shows an outline of four contents viewed by the user. FIG. 10C shows the recall level calculated by the recall determination unit 106 for each content. For example, when viewing the video content for product explanation of Company A from 0 to 30 seconds, the recall is calculated as 0%, and when viewing the video content for product description of Company B, the recall is calculated as 20%. In addition, the degree of recall is calculated as 0% when viewing the video content for product explanation of company C, and the degree of recall is calculated as 80% when viewing the video content for product description of company D.

  The reaction target specifying unit 107 is a content or scene that shows a psychological reaction by the user based on the reaction time candidate specified by the reaction identifying unit 102 and the recall level specified by the recall determining unit 106. Identify the reaction target that is the target of The reaction target specifying unit 107 may specify the time when the reaction target is displayed.

  In the present embodiment, the video contents for product explanation of each company of Company A, Company B, Company C, and Company D are set as one content, and which content the psychological reaction indicated by the user is due to? This will be described based on an example of specifying

  First, as described above, the reaction identification unit 102 determines the absolute value of the integrated value of the cerebral blood flow when the acquired absolute value of the cerebral blood flow exceeds a predetermined threshold value and in a predetermined time interval. When the value exceeds a predetermined value, it is identified that the user has exhibited a psychological reaction (that is, the reaction type is “reactive system”). At this time, the reaction identifying unit 102 determines in advance the time when the acquired absolute value of the cerebral blood flow exceeds a predetermined threshold and the absolute value of the integrated value of the cerebral blood flow in a predetermined time interval. Response that is a candidate for the time when the user showed a psychological response to the time included in the time interval from one of the times that exceeded the specified value to the time that went back by a predetermined time Identify as a time candidate.

  Next, the reaction target specifying unit 107 determines whether the psychological reaction indicated by the user is a response to the content or scene that the user is viewing at the time specified as the reaction time candidate, or a past scene or content. It is specified whether the reaction is caused by the user recalling.

  For example, when the recall level calculated by the recall determination unit 106 is less than a predetermined threshold (for example, 60%), the reaction target specifying unit 107 displays the target for which the user has shown a psychological reaction when viewing the content. To identify the content being As a result, the reaction target specifying unit 107 specifies the content including the reaction time candidate as the target content for which the user has shown a psychological reaction. On the other hand, when the recall level is 60% or more which is the threshold value, the reaction target specifying unit 107 specifies the content viewed by the user at a time before the response time candidate as the target for which the user has shown a psychological reaction. .

  That is, when the recall level is smaller than a predetermined value, the reaction target specifying unit 107 specifies the image content viewed as a response time candidate as a target that the user has shown a psychological response, and the recall level Is equal to or greater than a predetermined value, the image content viewed before the reaction time candidate is specified as a target for which the user has shown a psychological reaction.

  Specifically, the psychological reaction identified by the reaction identifying unit 102 is identified as a reaction that targets content or a scene that the user has viewed in the past.

  More specifically, the number of users whose cerebral blood flow is measured as shown in FIG. 9 (a) has increased so that the cerebral blood flow exceeds the threshold of 0.05 starting from 65 seconds. Reaction has occurred. Therefore, the reaction identification unit 102 specifies 65 seconds as the reaction time candidate 420. Next, referring to the recall level for the scene including the reaction time candidate 420, that is, the moving image content for product explanation of company C, the recall level is 0%. That is, the recall level is less than the threshold value of 60%. Therefore, the reaction target specifying unit 107 specifies that the psychological reaction indicated in the reaction time candidate 420 is a reaction to a scene including the reaction time candidate 420, that is, a reaction to a moving image content for product explanation of company C. .

  On the other hand, the number of users whose cerebral blood flow is measured as shown in FIG. 10B is increasing so that the cerebral blood flow exceeds the threshold value of 0.05 starting from 90 seconds. Therefore, the reaction identification unit 102 specifies 90 seconds as the reaction time candidate 421. Next, referring to the scene containing the reaction time candidate 421, that is, the recall level for the video content for product explanation of Company D, the recall level is 80%. That is, the recall level is equal to or greater than the threshold value. Therefore, the reaction target specifying unit 107 specifies the moving image content for product description of company C, which is the previous scene, as a target recalled by the user. Thereby, the reaction target specifying unit 107 specifies that the user's psychological reaction indicated in the reaction time candidate 421 is a reaction to the moving image content for product explanation of the C company.

  The content evaluation device 110 may display the user's evaluation of the content on the evaluation display unit 108 that is an external display device. The evaluation display unit 108 is a display device for displaying results such as evaluation of content, for example, presence or absence of reaction. The evaluation display unit 108 is, for example, a liquid crystal display or a CRT (Cathode Ray Tube). Note that the content evaluation apparatus 110 may include the evaluation display unit 108.

  FIG. 11 is an example of content evaluation results displayed on the evaluation display unit 108. Each evaluated content and a user response to each content are displayed as a result of the evaluation. As reference information, an actual measurement value of cerebral blood flow is also displayed here. For example, regarding the video content for product explanation of Company A displayed between 0 seconds and 30 seconds and the video content for product description of Company B displayed between 30 seconds and 60 seconds, The blood flow has not changed, and it is evaluated that there is no psychological reaction of the user.

  On the other hand, regarding the moving image content for product explanation of Company D displayed between 90 seconds and 120 seconds, there is a reaction in the cerebral blood flow that exceeds the threshold. However, since the user is in a recollection state, this reaction is specified not as a response to the content of Company D but as a response to the previous content of Company C. In other words, it is specified that there is a response to the moving image content for product explanation of company C (because it has increased, it is excited).

  On the other hand, about 90 seconds to 120 seconds, since the user is in the recall state, it is evaluated that there is no response (recall). In addition, since the evaluation considering the recall is shown in FIG. 11, the recall is not displayed as shown in FIG. 10, but the recall may be displayed together as one of the evaluations. Good.

  Next, the operation flow of the present invention will be described with reference to FIGS. 12, 13, 14, 15, 16, and 17. FIG.

  First, referring to FIG. 12, the cerebral blood flow detection unit 320 detects the cerebral blood flow of the user who is viewing the content (step S101).

  Next, based on the detected increase or decrease in cerebral blood flow, the reaction identification unit 102 identifies the user's psychological reaction type (step S102). The detection of cerebral blood flow (step S101) and the gaze detection (step S102) may be performed in parallel.

  FIG. 13 is a detailed flowchart of the reaction identification process (step S102 in FIG. 12). The reaction identification unit 102 provides a predetermined threshold (for example, ± 0.05) and determines whether or not cerebral blood flow (that is, oxyhemoglobin concentration or total hemoglobin concentration) has increased or decreased beyond the threshold, When the increase / decrease amount is equal to or greater than the threshold value (Yes in step S201), the user is identified as having a psychological reaction (that is, “reactive system”) (step S202). On the other hand, when the increase / decrease amount of the cerebral blood flow is less than the threshold (No in step S201), it is identified as no response (that is, “no response system”) (step S203).

  When the increase / decrease amount of the cerebral blood flow is equal to or greater than the threshold value, the reaction identifying unit 102 may identify the reaction type in more detail depending on whether the change in cerebral blood flow is increased or decreased. For example, the reaction identification unit 102 according to the present embodiment determines whether the change in cerebral blood flow is increased or decreased (step S204), and in the case of an increase (Yes in step S204), the reaction type is “excited system”. (Step S205), and in the case of a decrease (No in step S204), the reaction type is identified as a “concentrated” reaction (step S206). Furthermore, the reaction identification unit 102 specifies the time when the amount of change in cerebral blood flow is equal to or greater than the threshold as a reaction time candidate (reaction time candidate) (step S207).

  Referring to FIG. 12 again, the line-of-sight detection unit 103 detects the position of the line of sight of the user who views the content every predetermined time (step S103). Based on the obtained time-series data of the line-of-sight position, the dwell time calculation unit 104 calculates the dwell time of the line-of-sight position (step S104).

  FIG. 14 is a detailed flowchart of the residence time calculation process (step S104 in FIG. 12).

  The dwell time calculation unit 104 repeats the processing from step S301 to step S306 described below during gaze detection (that is, until the content evaluation device 110 acquires all gaze positions necessary for content evaluation). The dwell time at the line-of-sight position is calculated.

  First, the residence time calculation unit 104 accumulates coordinates indicating the current line-of-sight position in the buffer (step S302). Thereafter, the coordinates indicating the current line-of-sight position are referred to at the next sampling timing (step S303), and the difference between the coordinates stored in the buffer and the current coordinates is calculated (step S304). For example, a predetermined threshold value such as ± 10 is provided for both the X coordinate and the Y coordinate, and it is determined whether or not the difference is within the threshold value (step S305). If the difference is greater than or equal to the threshold (Yes in step S305), it can be determined that the line of sight is moving, so the dwell time calculation unit 104 performs loop processing again (step S306).

  On the other hand, when the line-of-sight position difference is less than the threshold (No in step S305), the residence time calculation unit 104 first accumulates the current time in a buffer in order to calculate the residence time (step S307). Then, while the line-of-sight position difference is less than the threshold value, a loop process is performed (steps S308 to S310) while referring to the current time at each sampling timing (step S309). When the line-of-sight position difference exceeds the threshold value, the loop is exited (step S310), and the difference between the exit time and the time stored in the buffer is calculated (step S311). After that, the residence time calculation unit 104 sets the obtained time difference as the residence time (step S312). As a result, the line-of-sight position and the staying time at the line-of-sight position are obtained sequentially.

  Referring to FIG. 12 again, next, the recall determination unit 106 refers to the recall information stored in the recall information storage unit 105 (step S105). Then, the recall determination unit 106 determines whether or not the user is in the recall state based on the user's line-of-sight position and the gaze retention time when viewing the content (step S106).

  FIG. 15 is a detailed flowchart of the recall determination process (step S106 in FIG. 12).

  Note that the recall determination unit 106 according to the present embodiment calculates a recall level for each content with respect to one content in which a plurality of scenes are collected. Therefore, the processing from step S403 to step S412 is repeated for the number of scene tags included in the recall information (from step S403 to S412).

  In each iterative process, first, the recall determination unit 106 refers to the time (display time) of the scene included in the scene tag (step S404). The recall determination unit 106 refers to the display area of the scene (step S405). In the scene tag 1 of the recall information shown in FIG. 7, the display time is 1 second to 2 seconds, and the display area is (200, 300, 700, 400).

  Next, the recall determination unit 106 determines whether or not the user's line-of-sight position within the display time is located within the display area (step S406). If the line-of-sight position is located within the display area (Yes in step S406), it is determined that the user is not in the recall state because the user is looking at the provided image (step S407).

  On the other hand, when not located in the display area (No in step S406), the recall determination unit 106 calculates the distance between the display area and the line-of-sight position (step S408). Next, the recall determination unit 106 determines whether or not the calculated distance is greater than or equal to a threshold (step S409). When the distance is less than the threshold (No in step S409), the recall determination unit 106 determines that the user is not in the recall state during the corresponding display time (step S407). Since the distance between the display area and the line-of-sight position is short, the reason why the line-of-sight position is not located within the display area is considered to be an error when detecting the line of sight.

  On the other hand, when the distance is greater than or equal to the threshold (Yes in step S409), the recall determination unit 106 further determines whether or not the residence time is greater than or equal to the threshold (step S410). If the residence time is less than the threshold (No in step S410), the recall determination unit 106 determines that the user is not in the recall state (step S407).

  On the other hand, when the residence time is equal to or greater than the threshold (Yes in step S410), the recall determination unit 106 specifies that the user is in the recall state during the corresponding display time (step S411).

  The recall determination unit 106 repeats the loop processing from step S403 to step S412 by the number of scene tags included in the recall information for one content, and then exits the loop processing (step S412).

  With the above processing, it is determined whether or not the user is in a recall state for each scene tag. Thereafter, the recall determination unit 106 calculates the recall level as a ratio of the number of scene tags determined to be in the recall state to the number of all scene tags included in the recall information for one content (step S413). ).

  Referring to FIG. 12 again, next, the reaction target specifying unit 107 includes the presence / absence of the user's psychological reaction and the reaction time candidate identified by the reaction identifying unit 102, and the recall degree calculated by the recall determining unit 106. Based on the above, the object to which the user actually responded is specified. The reaction target specifying unit 107 according to the present embodiment assumes that the psychological reaction indicated by the user is a reaction with respect to any content that the user has viewed. Whether it is a reaction is specified (step S107).

  FIG. 16 is a detailed flowchart of the process for specifying the reaction target (step S107 in FIG. 12). First, the reaction target specifying unit 107 refers to the reaction time candidate specified by the reaction identifying unit 102 (step S501). Then, the recall level of the scene or content including the time determined as the reaction time candidate is referred to (step S502).

  In the present embodiment, there is one content including the reaction time candidate. For example, in the case of FIG. 9, the reaction time candidate 420 is 65 seconds. The degree of recall corresponding to this is 0%, which is the degree of recall of content including 65 seconds, that is, the moving image content for product explanation of company C. In the case of FIG. 10, the reaction time candidate 421 is 90 seconds. The degree of recall corresponding to this is 80%, which is the degree of recall of content including 90 seconds, that is, the moving image content for product explanation of Company D.

  Next, the reaction target specifying unit 107 determines whether or not the recall level is less than a threshold value (the threshold value is set to 60%, for example) (step S503). When the recall level is less than the threshold value (Yes in step S503), the reaction target specifying unit 107 indicates that the user's psychological reaction target indicated in the reaction time candidate is currently the user (that is, the time of the reaction time candidate). Is identified as the viewed content (step S504). On the other hand, when the recall level is equal to or higher than the threshold (No in step S503), the reaction target specifying unit 107 determines that the user's recall behavior is occurring at the time of the reaction time candidate. In this case, the reaction target specifying unit 107 determines that the user's psychological reaction indicated by the reaction time candidate is a reaction that targets the scene or content that the user viewed immediately before the scene or content including the reaction time candidate. (Step S505).

  For example, in the case of FIG. 9, the recall level of the user who views the moving image content for product explanation of company C is 0%, and the recall level is less than the threshold. Therefore, the reaction target specifying unit 107 has a psychological reaction indicated by the user in the reaction time candidate 420, which is the content currently being viewed by the user (at the time of the reaction time candidate 420). Identify the response to content. In the case of FIG. 10, the recall level of the user who views the moving image content for explaining the product of company D is 80%, and the recall level is equal to or higher than the threshold. Accordingly, the reaction target specifying unit 107 indicates that the psychological reaction indicated by the user in the reaction time candidate 421 is a product of company C, which is the content immediately before the content currently being viewed by the user (reaction time candidate 421). It is specified that the response is to a video content for explanation.

  Referring to FIG. 12 again, the content evaluation apparatus 110 includes the presence / absence and type of the reaction identified by the reaction identification unit 102, the reaction target specified by the reaction target specifying unit 107, or the display time of the reaction target. May be displayed on the evaluation display unit 108 as the user's evaluation of the content (step S108).

  It should be noted that the recall determination unit 106 according to the present embodiment collects a scene in which the user is in a recall state for one content in which a plurality of scenes to be focused on (that is, specified by a plurality of scene tags). Although the ratio is calculated as the recall level, the presence or absence of recall may be determined for each scene, or may be calculated as the recall level. For example, for each scene, the recall determination unit 106 determines that the distance between the line-of-sight position at the display time and the display area associated with the display time, and the residence time at the display time are equal to or greater than a predetermined threshold value. If there is a threshold value or more, it may be determined that the user has recalled another scene displayed before the scene.

  Moreover, although the recall determination unit 106 according to the present embodiment has described the example of performing the recall state determination process by comparing the distance between the display region and the line-of-sight position and the dwell time of the line of sight with threshold values, The state determination is not limited to this, and various variations are conceivable. An example will be described below.

  The recall determination unit 106 according to the present embodiment uses the degree of gaze separation with respect to the region to be focused on (that is, the display region defined in the scene tag) in the determination of the recall level. For example, in FIG. 8, the recall determination unit 106 calculates the vertical distance between the line of sight D (200, 100) and the region B (200, 300, 800, 400) that is the region of interest as 200. However, depending on the scene, there may be a plurality of areas to be noted. This will be described in detail with reference to FIG.

  In FIG. 17, it is assumed that a region B (200, 300, 800, 400) and a region C (300, 50, 60, 150) are defined as regions to be noted. On the other hand, the line-of-sight position is the line of sight D (200, 100). In this case, for example, the recall determination unit 106 may calculate the vertical distance to each of the regions B and C and the line of sight D, and determine the recall state based on whether or not the closest distance is equal to or greater than a threshold value. That is, when the scene includes a plurality of display areas, the recall determination unit 106 may calculate the distance using the degree of separation from the display area closest to the line-of-sight position. Among the plurality of display areas, even for the display area closest to the line-of-sight position, the fact that the line-of-sight position is more than the threshold value means that the user is not paying attention to the content that is currently being viewed, This is because it is considered that there is a high possibility.

  Alternatively, the recall state may be determined by assigning a weight to the distance from each of the region B and the region C to the line of sight D and using the sum of the weighted distances. That is, the recall information may have a weight for each scene, and the recall determination unit 106 may add up the total number of scenes determined to be in the recall state by multiplying the weight corresponding to the scene.

  For example, as shown in (Formula 1), the recall determination unit 106 may multiply the distance l (for example, a vertical distance) between each display region included in the scene and the line-of-sight position by a weight w. Thereby, as the sum of the weighted distances, the degree of divergence of the line-of-sight position with respect to the region of interest (display region) is calculated, and when the degree of divergence is greater than or equal to the threshold, the user may be determined to be in the recall state. it can. The weight value can be arbitrarily determined. For example, as described later, it may be defined in the scene tag for each display area, and a value f () determined by a predetermined function depending on the size of the distance l. l) may be used.

  This is because when there are a plurality of areas to be focused on in one scene, the user's line of sight moves to all or some of these areas if the user is focused on that scene. However, when the user is recalling, the recall is not disturbed, so in order to avoid all areas where the provided image is displayed unconsciously, the line of sight goes to a position far from any display area. This is because the line of sight tends to stay. By considering each region in this way, the recall determination unit 106 can determine the recall level more accurately.

  Further, the recall determination unit 106 according to the present embodiment determines that the user is in the recall state when the residence time is equal to or greater than a threshold (for example, 1 second or more), but the present invention is not limited to this.

  For example, the longer the residence time, the greater the weight may be given to the recall state. That is, when the user takes a recall action such as recollecting, the line of sight does not move much and often stays within a certain range. Therefore, the recall determination unit 106 calculates the recall level by multiplying the distance between the line-of-sight position and the display area, for example, by using the residence time t as a weight so that the longer the stay time of the line of sight, the higher the degree of recall. May be. More specifically, the following (formula 2), which is obtained by adding the concept of time to the above (formula 1), can be considered. In (Equation 2), the total sum L for all display areas is calculated by multiplying the corresponding residence time for each weighted distance.

  Furthermore, when the user takes a recall action, the user's line of sight not only stays at a position distant from the region of interest, but is close to any of the four corners of the display screen on which the content is displayed. It may stay in place. In general, content is created so that a user can easily focus on the content. Therefore, the provided image representing the target that the user wants to see, such as a talent, a product, a function description, and characters, is arranged in the center of the screen. On the other hand, when these provided images enter the eyes, the user is hindered from recalling. Therefore, the user unconsciously moves his / her line of sight to one of the four corners of the screen and tends to stay there. Therefore, the recall determination unit 106 is in the recall state when the user's line-of-sight position remains in one of the four corners of the display screen for a predetermined time or more regardless of the position of the region to be noted. May be determined.

  In the inventors' experiments, no significant difference depending on the position is obtained as to which position on the display screen the user's line of sight stays. For example, the stay position of the line-of-sight position is significantly different for each user, such as “the line-of-sight position of a certain user is always located at the upper left of the display screen, but the line-of-sight position of another user is always located on the right side of the screen”. Experimental results have not been obtained so far. However, each individual has a habit. Therefore, the recall determination unit 106 may accumulate, for example, a line-of-sight history for each individual, calculate a position and time at the time of recall, and use the modeled information as the recall information.

  Based on the results of experiments conducted by the inventors, in the video for product explanation and commercials of products, the line of sight is most likely to be placed on people such as talents and explainers, followed by the appearance of the product and the function of the product. There is a tendency that the line of sight is located on the letters and the company name logo.

  Therefore, a weight is assigned to each display area, which is a region to be focused on, in advance according to the content of the provided image, and the recall determination unit 106 adds the weighted recall state to calculate the recall level. It may be calculated.

  In other words, for the sake of explanation, the recall determination unit 106 according to the present embodiment has identified the recall state when the line of sight stays away from the display area by more than a threshold. However, the weight may be defined for each display area in the scene tag according to the content of the provided image displayed in the display area, and the recall determination unit 106 may calculate the recall level by the sum of the weighted distances or the like. . For example, in the above (Equation 1) and (Equation 2), when the content of the provided image displayed in the display area that is the area of interest is a talent, 1.0 is designated as the weight w corresponding to this display area. In the case of a character or logo, by specifying 0.5 as the weight w corresponding to this display area, the degree of recall changes even if the distance between the line-of-sight position and the display area is the same.

  In other words, the recall determination unit 106 originally retains the user's line of sight at a position away from the provided image even though the provided image representing an actress or actor who is more likely to be focused than character information is displayed. If it is, the degree of recall may be calculated higher than when a provided image representing characters or the like is displayed.

  Furthermore, when the provided image to be displayed is a character, the recall determination unit 106 may consider whether or not the line of sight is following the character when determining the recall. For example, in the case of a character area, it is possible not only to locate but also to consider whether the area is being followed or moved within the area. That is, when the provided image represents character information, the recall determination unit 106 is in the recall state when the movement amount of the line-of-sight position within the display time of the provided image is less than a predetermined threshold. May be determined. Furthermore, the recall determination unit 106 may determine that the user is not in the recall state when the moving direction of the line-of-sight position matches the writing direction of the character represented by the provided image. This will be specifically described with reference to FIG.

  FIG. 18 is a diagram illustrating a method for calculating a user's gaze degree for character information. In FIGS. 18A and 18B, it is assumed that the line of sight stays at the position indicated by the line of sight D (150, 860). On the other hand, the display area which is the area to be noted is the area B (200, 300, 800, 400). Therefore, the difference in coordinates between the line-of-sight position and the display area is (50, 40), and the distance is less than a threshold value (for example, 100). Therefore, for example, as illustrated in FIG. 18B, when the determination is made based only on the distance between the display area and the line-of-sight position, the recall determination unit 106 determines that the user is not in the recall state.

  However, in FIG. 18A, the provided image representing the characters “with a camera shake correction function” is displayed in the region B. Therefore, if the user originally focuses on the content, the line of sight generally follows the characters in the writing direction (from left to right in this example). Therefore, when the provided image represents a character, the recall determination unit 106 can determine the recall more accurately by adding whether or not the user's line of sight follows the character. In the case shown in FIG. 18 (a), the user's line of sight stays in the line of sight D in the area B where characters are displayed. Therefore, even if the distance between the area B and the line of sight D is within the threshold value, The recall determination unit 106 determines that the user is in the recall state. When the provided image represents character information, the recall determination unit 106 includes the information indicating that the provided image represents character information in the attribute field of the corresponding scene tag, so that the recall image determination unit 106 can display the character information. You can know whether or not to represent it.

  Note that the recall determination unit 106 may determine the recall in consideration of the time when the line of sight starts to stay. Hereinafter, a description will be given using a specific example of FIG. FIG. 19 is a diagram for explaining the recall determination process in consideration of the time when the gaze retention starts. First, in the screen shown in FIG. 19A, the line of sight is located at the line of sight D (200, 100). The display area to be noted is the area B (200, 300, 600, 400). Therefore, the line-of-sight position and the display area are separated from each other by a threshold or more, and the user's line of sight has started to stay. In other words, since the user recalls and is in the recall state, the line of sight is removed from the region to be noticed, and the line of sight remains. Here, it is assumed that the screen is switched and the screen shown in FIG. 19B is displayed. In this case, a provided image representing the characters “beautiful even at night” appears at a position where the user's line of sight stayed. However, since the user has already started recalling, the line of sight remains at the same position regardless of the screen change.

  In the case of the screen shown in FIG. 19B, the region of interest is the region C (50, 50, 600, 200), and therefore the line of sight D (200, 100), which is the position of the line of sight, is within the region C. In. Therefore, the recall determination unit 106 may erroneously determine that the user is not in the recall state even though the user is actually recalling. Therefore, the recall determination unit 106 determines whether there is a recall state using whether or not the user's line-of-sight position is in the display area at the time when the stay starts, and then determines whether or not the line-of-sight position remains. The presence / absence of the recall state may be determined only by the above. More specifically, the recall determination unit 106 refers to the recall information, and when the line-of-sight position is located in the display area of the provided image, the line-of-sight position is located in the display area before the display time of the provided image. When doing so, it may not be determined that the user is in the recall state. In this way, it is possible to avoid erroneous determination when a region is accidentally entered at the line of sight due to scene switching.

  Note that the content evaluation apparatus 110 according to the present embodiment uses a video content for product explanation in which a plurality of notable scenes are collected as one content, and each content of a user who has viewed the plurality of content. Changes in cerebral blood flow were measured. In addition, when it is determined from the measurement result that the user's psychological reaction has occurred, the content evaluation device 110 identifies which of the product explanation video content is the reaction. However, the content evaluated by the content evaluation device 110 is not limited to this. For example, for a drama or a movie in which a plurality of different scenes are continuous, application to a system for estimating a user's evaluation or feelings based on a biological signal is also possible.

  The inventors conducted similar evaluation experiments not only on changes in the user's line of sight and cerebral blood flow when viewing video content for product explanation, but also on content such as movie descriptions, CMs, and 3D images. Is going. As a result, it has been confirmed from experiments that the user exhibits the same reaction as the excitatory reaction and the concentrated reaction described above for these contents.

  Therefore, using the present invention, for example, a system that extracts a scene in which a change in the user's psychology has occurred due to a reaction in the cerebral blood flow, such as a scene that is concentrated, excited, or highly emotional. Is possible.

  FIG. 20 is a diagram for explaining changes in cerebral blood flow and the like of a user who is viewing content for promoting a movie in an experiment. FIG. 20A is a diagram showing a change in the cerebral blood flow of the user when watching a movie advertisement. The horizontal axis represents time, and the vertical axis represents cerebral blood flow.

  FIG. 20B is an example of the recall information expressed in the tag format corresponding to the promotional content of the movie. When a movie advertisement or the like is regarded as one content, a plurality of scenes generally exist in this content. Here, a plurality of scenes are shown as “scene groups”. For example, the scene group 1 is a scene relating to “movie description”, and includes that two notable areas of the scene tag 1 and the scene tag 2 and the corresponding display time are included. Alternatively, the scene group 8 is configured as a “scene that remembers magic” and includes a plurality of regions to be focused on, such as a scene tag 86 and a scene tag 87, and their time. The scene group 9 is composed of a plurality of areas to be focused on, such as a scene tag 90 and a scene tag 91, as “escape scenes”, and combinations of the times.

  It can be seen that the cerebral blood flow increases so that the cerebral blood flow exceeds the threshold value 0.05 at 100 seconds while maintaining a constant value for a while. The reaction identification unit 102 uses threshold processing or the like to specify 100 seconds as a candidate for the time when the user showed a psychological reaction (excitement system). Then, the recall determination unit 106 calculates the recall level in each scene after 100 seconds that is a candidate for the reaction time, and the reaction target specifying unit 107 responds to the content viewed at the time when the recall level is less than the threshold value. To be identified.

  For example, in this case, if the recall level is less than the threshold value, the reaction identifying unit 102 identifies that the user is excited about the “escape scene” of the scene group 9.

  On the other hand, when the recall level is equal to or greater than the threshold value, the recall determination unit 106 identifies that the user is in the recall state. In that case, the reaction target specifying unit 107 determines that the user has taken a recollection action such as a thought or a reminder of the “scene that remembers magic” that is the previous scene. As described above, the content evaluation apparatus 110 not only selects a content that shows a psychological reaction from a plurality of contents, but also a specific time when the user shows a psychological reaction from a series of videos. It is also possible to specify the scene.

  Note that the reaction target specifying unit 107 according to the present embodiment specifies that the target of the psychological reaction indicated at the time is the previous scene or content when the user is in the recall state. . This is because, as described in this embodiment, when content is continuously displayed and the user evaluates the content, the recalling action may be for the previous content. This is due to the experimental results of many. However, in a situation where the user is free to watch a movie or television program and is not necessarily forced to evaluate, the recall target is not necessarily for the previous scene or content. Therefore, the content evaluation apparatus 110 may display, for example, that the user is recalling any of a plurality of scenes or contents viewed by the user before showing the reaction. Alternatively, the content evaluation apparatus 110 may simply display that there is a recall action without specifying a scene that is a recall target. This makes it possible to perform content evaluation according to the usage status.

(Modification 1)
The content evaluation apparatus 110 according to the embodiment recognizes a psychological reaction such as the interest of a user who is a subject based on a change in blood flow in the brain with respect to content displayed in a predetermined order. Furthermore, whether or not the subject's interest is related to the previously presented content by estimating whether or not the subject is in a recall state using information such as the dwell time on the screen of the line-of-sight information I also judge.

  Furthermore, when the content evaluation apparatus 110a according to the present modification determines that the subject is in the recall state, the content evaluation device 110a presents the user with the content that is presumed that the user who is the subject is interested. More accurately identify if you were interested in FIG. 21 shows a system configuration of the content evaluation apparatus 110a according to this modification.

  As shown in FIG. 21, the content evaluation apparatus 110 a according to the present modification includes a stimulus presentation order storage unit 2101, a stimulus presentation part 2102, and a stimulus presentation order in addition to the system configuration of the content evaluation apparatus 110 shown in FIG. 1. A change unit 2103 is added. Further, instead of the reaction target specifying unit 107, a reaction target specifying unit 107a is provided.

  The stimulus presentation order storage unit 2101 stores a display order of a plurality of image contents. That is, the stimulus presentation order storage unit 2101 stores a stimulus presentation order to be presented to a user who is a subject. For example, the stimulus presentation order storage unit 2101 according to the present embodiment stores the order in which the CM video is presented to the subject and the time between the CM videos.

  The stimulus presentation unit 2102 displays a plurality of contents based on the order stored in the stimulus presentation order storage unit 2101 or the display order changed by the stimulus presentation order change unit 2103. That is, the stimulus presentation unit 2102 presents the stimulus to the subject by displaying the contents in order according to the order of the stimuli stored in the stimulus presentation order storage unit 2101. For example, in the case of a CM video, the stimulus presentation unit 2102 is configured as a display device such as a liquid crystal display, and presents the CM video and audio to the subject.

  The stimulus presentation order changing unit 2103 changes the display order of the plurality of image contents stored in the stimulus presentation order storage unit 2101. For example, the stimulus presentation order change unit 2103 is stored in the stimulus presentation order storage unit 2101 so that the content displayed by the stimulus presentation unit 2101 at the time specified by the reaction target specifying unit 107 is displayed again. Change the presentation order. The stimulus presentation order changing unit 2103 may change the presentation order at random.

  The stimulus presentation order changing unit 2103 changes the display order so that another image content specified by the reaction target specifying unit 107 as content for which the user has shown a psychological reaction is displayed again.

  Here, when the content specified by the reaction target specifying unit 107a as the first image content is identified as a target that the user may have shown a psychological reaction, the reaction target specifying unit 107a includes a plurality of image contents. Based on the difference between the increase / decrease amount of cerebral blood flow when the first image content is displayed for the first time and the increase / decrease amount of cerebral blood flow when the first image content is displayed for the second time or later. Then, it is determined whether or not the user has recalled the first image content.

  For example, the reaction target specifying unit 107a increases the cerebral blood flow when the first image content is displayed for the first time from the increase amount of the cerebral blood flow when the first image content is displayed for the Nth time (where N ≧ 2). Only when the difference minus the amount is greater than or equal to a predetermined threshold may the user identify that he has recalled the first image content. Hereinafter, a more specific description will be given with reference to FIG.

  FIG. 22 is a diagram illustrating a continuation of the correspondence between the content to be viewed and the amount of change in cerebral blood flow for the same user as in FIG. As shown in FIG. 22, when the CM of company D is presented to the subject as a stimulus, the subject exhibits an excitatory response. At this time, it is assumed that the recall determination unit 106 determines that the subject is in the recall state. In addition, the reaction target specifying unit 107a targets the camera CM of the C company that the user recalled when presenting the CM of the D company as the response of the excitatory system shown when the CM of the D company was presented. Suppose you specify that However, the identification at this point is not deterministic. Next, the stimulus presentation order changing unit 2103 changes the presentation order of the contents stored in the stimulus presentation order storage unit 2101 so that the CM of company C is presented again after presenting the CM of company D. .

  In general, when the presented content is content that the user views for the first time and when the content has been viewed twice or more, the user is larger than the content presented for the first time. Often shows a psychological response. As a result, there is a high possibility that the amount of blood flow in the brain will increase when the content presented for the first time is viewed. However, if the user is in the recall state of the CM of company C when the CM of company D is presented, the response of the subject is significant when the CM of company C is presented again after the CM of company D is presented. Is likely to be high. Specifically, as shown in FIG. 22, when the CM of company C is presented again after the presentation of CM of company D, the brain blood flow of the subject is considered to increase beyond a predetermined threshold.

  Therefore, the reaction target specifying unit 107a compares the cerebral blood flow during the first CM presentation of Company C with the cerebral blood flow during the second CM presentation, and the second time has a larger value. In this case, it is determined that the reaction of the blood flow in the brain from 90 seconds to 120 seconds is caused by the user recalling the CM of company C.

  As described above, when the content evaluation apparatus 110a according to this modification determines that the recall reaction is for the previously presented content using the line-of-sight information, the previously presented content is again presented. , Change the order of the content presented to the subject. Furthermore, by measuring a change in the blood flow in the brain of the subject with respect to the content presented again, it is possible to accurately determine whether it is due to a recall reaction.

(Modification 2)
As shown in FIG. 7, the recall information storage unit 105 provided in the content evaluation apparatus 110 according to the embodiment relates to a display area, a display time, an attribute, and the like of the target image in association with each scene included in the content. Stores recall information including information. Here, the information included in the recall information such as the display area needs to be set in advance by the content designer (that is, the content creator), and it takes time to set the area. Further, when a general user views the content, the line of sight does not always stay in the area.

  Therefore, the content evaluation apparatus 110b according to the present modification generates the recall information stored in the recall information storage unit 105 using the history of the line of sight viewed by the user.

  The content evaluation device 110b continuously presents video stimuli (that is, image content) to the user and measures the response in order to reduce the subject's restraint time. For this purpose, it is determined using the line-of-sight information whether the response is to a previously presented stimulus. Therefore, when a single video is presented in advance, a general user indicates a model (for example, a display area and a dwell time corresponding to a display time) indicating what kind of line-of-sight movement is performed. Information) is stored as recall information, and when the acquired user's line-of-sight position deviates significantly from the model that is the recall information, the recall determination unit 106 determines that the user is in the recall state. The system configuration is shown in FIG.

  As shown in FIG. 23, the content evaluation apparatus 110b according to the present modification displays a plurality of image contents and stores a display order of the plurality of contents compared to the content evaluation apparatus 110. Unit 2101, a stimulus presentation unit 2102 that displays a plurality of contents based on the order stored in the stimulus presentation order storage unit, a line-of-sight history storage unit 2301 that stores a user's line-of-sight position as a history, and a line-of-sight history storage unit 2301 A typical recall information generation unit 2302 that identifies a line-of-sight position when each of a plurality of image contents is displayed using the line-of-sight position stored in the image, and generates recall information including the identified line-of-sight position as a display area; Is further provided. In addition, the same code | symbol is attached | subjected about the component same as the component with which the content evaluation apparatus 110a which concerns on the modification 1 is provided, and detailed description is abbreviate | omitted.

  The line-of-sight history storage unit 2301 stores the line-of-sight information detected by the line-of-sight detection unit 103.

  The typical recall information generation unit 2302 extracts information for specifying where the line of sight stays when a general user is presented only with the content from the line-of-sight history accumulated in the line-of-sight history accumulation unit 2301. . The typical line-of-sight information generated by the typical recall information generation unit 2302 is stored in the recall information storage unit 105 as recall information.

  Note that the stimulus presentation order (that is, the display order of image content) stored in the stimulus presentation order storage unit 2101 according to the present modification is different from that of the first modification, and sufficient time has elapsed after presenting one content. It is ordered so that the next content is presented after taking. This is to prevent a user who is viewing the content from recalling the previous content. This is because, in the present modification, it is preferable that history information of a general line of sight where no recall state occurs is extracted in advance and stored in the recall information storage unit 105 as recall information.

  With reference to the recall information generated by the typical recall information generation unit 2302, the recall determination unit 106 determines that the eye is in a recall state when the line of sight stays in a region different from the general line-of-sight history. It is determined that the change in the blood flow in the brain is caused by the content viewed by the user before that time.

  Although the content evaluation device according to the embodiments of the present invention has been described above, the present invention is not limited to these embodiments.

  For example, the recall determination unit 106 may calculate the recall level based on the display time corresponding to the scene determined to be in the recall state, instead of the number of scenes determined to be in the recall state by the user. Specifically, the recall determination unit 106 may calculate the recall level as a ratio of the total display time of the scene determined to be in the recall state with respect to the total of all display times included in the recall information.

  In addition, the acquisition unit 120 according to the embodiment and the modification thereof includes the cerebral blood flow detection unit 320 and the gaze detection unit 103. However, the acquisition unit 120 includes the cerebral blood flow detection unit 320 and the gaze detection unit 103. Even in the case of having no, the same effect of the invention is obtained. For example, the acquisition unit 120 may acquire the cerebral blood flow and the line-of-sight position by reading out the cerebral blood flow and the line-of-sight position acquired by an external device of the content evaluation apparatus from a recording medium or the like. The acquisition unit 120 may acquire the cerebral blood flow and the line-of-sight position acquired by an external device of the content evaluation apparatus via a wired or wireless communication interface.

  Moreover, although the content evaluation apparatus according to the embodiment and the modification thereof includes the residence time calculation unit 104, the same invention effect can be achieved without the residence time calculation unit 104. For example, the recall determination unit 106 may determine whether or not the user is in the recall state using only the distance between the display area and the line-of-sight position. In this case, the content evaluation apparatus calculates the residence time. The unit 104 may not be provided.

  Moreover, although the content evaluation apparatus according to the embodiment and the modification thereof includes the recall information storage unit 105, the same invention effect can be achieved without the recall information storage unit 105. For example, the recall information may be stored in an external storage medium (not shown), and the recall determination unit 106 may read the recall information from the external storage medium. In this case, the content evaluation device may not include the recall information storage unit 105.

  In addition, although the content evaluation device according to the embodiment and the modification thereof includes the reaction target specifying unit 107, the same invention effect can be achieved without the reaction target specifying unit 107. For example, when content is evaluated by allowing a user to compare two contents, if the presence / absence of the recall state can be determined by the recall determination unit 106, the user can have a psychological reaction without the reaction target specifying unit 107. The indicated content can be specified.

  In addition, the content evaluation device 110a according to the modification of the embodiment presents again to the user the content (for example, the most recently displayed content) that is estimated that the user would have had a psychological reaction. The content that the user has responded to is accurately specified, but conversely, the content that is estimated that the user will not exhibit a psychological reaction may be presented to the user again. Specifically, the stimulus presentation order changing unit 2103 stores the content estimated to be not recalled by the user in the stimulus presentation order storage unit 2101 so that the content is presented to the stimulus presentation unit 2102 as the next stimulus. The content display order may be changed. The content presumed that the user will not recall is, for example, content that the user first viewed (content with an old viewing history) or content that is least recalled by other users. .

  Note that the content evaluation apparatus described in the embodiment and its modifications 1 and 2 can also be realized by a computer. FIG. 24 is a block diagram illustrating a hardware configuration of a computer system that implements the content evaluation apparatuses 110, 110a, and 110b.

  The content evaluation apparatus reads a computer 34, a keyboard 36 and a mouse 38 for giving instructions to the computer 34, a display 32 for presenting information such as calculation results of the computer 34, and a program executed by the computer 34. A CD-ROM (Compact Disc-Read Only Memory) device 40 and a communication modem (not shown).

  A program that is a process performed by the content evaluation apparatus is stored in a CD-ROM 42 that is a computer-readable medium, and is read by the CD-ROM apparatus 40. Alternatively, the data is read by the communication modem 52 through a computer network.

  The computer 34 includes a CPU (Central Processing Unit) 44, a ROM (Read Only Memory) 46, a RAM (Random Access Memory) 48, a hard disk 50, a communication modem 52, and a bus 54.

  The CPU 44 executes a program read via the CD-ROM device 40 or the communication modem 52. The ROM 46 stores programs and data necessary for the operation of the computer 34. The RAM 48 stores data such as parameters at the time of program execution. The hard disk 50 stores programs and data. The communication modem 52 communicates with other computers via a computer network. The bus 54 connects the CPU 44, the ROM 46, the RAM 48, the hard disk 50, the communication modem 52, the display 32, the keyboard 36, the mouse 38, and the CD-ROM device 40 to each other.

  Furthermore, some or all of the constituent elements constituting each of the above-described devices may be configured by a single system LSI (Large Scale Integrated Circuit). The system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip, and specifically, a computer system including a microprocessor, ROM, RAM, and the like. . A computer program is stored in the RAM. The system LSI achieves its functions by the microprocessor operating according to the computer program.

  Furthermore, some or all of the constituent elements constituting each of the above-described devices may be configured from an IC card that can be attached to and detached from each device or a single module. The IC card or module is a computer system that includes a microprocessor, ROM, RAM, and the like. The IC card or the module may include the super multifunctional LSI described above. The IC card or the module achieves its function by the microprocessor operating according to the computer program. This IC card or this module may have tamper resistance.

  Further, the present invention may be the method described above. Further, the present invention may be a computer program that realizes these methods by a computer, or may be a digital signal composed of the computer program.

  Furthermore, the present invention provides a computer-readable recording medium such as a flexible disk, a hard disk, a CD-ROM, an MO, a DVD, a DVD-ROM, a DVD-RAM, a BD (Blu-ray Disc). (Registered trademark)), a memory card such as a USB memory or an SD card, or a semiconductor memory. Further, the digital signal may be recorded on these recording media.

  Further, the present invention may transmit the computer program or the digital signal via an electric communication line, a wireless or wired communication line, a network represented by the Internet, a data broadcast, or the like.

  The present invention may be a computer system including a microprocessor and a memory, wherein the memory stores the computer program, and the microprocessor operates according to the computer program.

  In addition, the program or the digital signal is recorded on the recording medium and transferred, or the program or the digital signal is transferred via the network or the like and executed by another independent computer system. You may do that.

  Furthermore, the above embodiment and the above modification examples may be combined.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  The present invention can be applied to a content evaluation device and the like, and in particular, can be applied to a content evaluation device and the like that specify a reaction of the user to the image content by analyzing a biological signal of the user who views the image content.

32 Display 34 Computer 36 Keyboard 38 Mouse 40 CD-ROM device 42 CD-ROM
44 CPU
46 ROM
48 RAM
50 Hard disk 52 Communication modem 54 Bus 102 Reaction identification unit 103 Gaze detection unit 104 Residence time calculation unit 105 Recall information storage unit 106, 106a Recall determination unit 107 Reaction target identification unit 108 Evaluation display unit 110, 110a, 110b Acquisition of content evaluation device 120 Unit 220 content 222, 223, 224, 225 content 250 subject 320 cerebral blood flow detection unit 321 light receiving unit 322 light emitting unit 402, 403, 404 graph 412, 413 threshold 420, 421 reaction time candidate 1105, 1105a recall information 1108 name of content 1110 Scene tag 2101 Stimulus presentation order storage unit 2102 Stimulus presentation unit 2103 Stimulus presentation order change unit 2301 Gaze history accumulation unit 2302 Typical recall information generation unit

Claims (20)

A content evaluation device that identifies a user's reaction to the image content by analyzing a biological signal of the user who views the image content,
An acquisition unit for acquiring the cerebral blood flow and the line-of-sight position of the user;
A reaction identifying unit for identifying whether or not the user showed a psychological reaction when viewing the image content based on the cerebral blood flow;
A recall determining unit that determines whether or not a user viewing the image content has recalled another image content displayed before the displayed image content based on the line-of-sight position of the user; A content evaluation apparatus. Further, when the user shows a psychological reaction when viewing the image content, and the recall determination unit determines that the user has recalled the other image content, the other The content evaluation apparatus according to claim 1, further comprising: a reaction target specifying unit that specifies image content as a target for which the user has exhibited a psychological reaction. In the reaction identification unit, when the absolute value of the acquired cerebral blood flow exceeds a predetermined threshold value, or the absolute value of the integrated value of the cerebral blood flow in a predetermined time interval is determined in advance. The content evaluation apparatus according to claim 2, wherein if the value exceeds the value, the user is identified as having exhibited a psychological reaction. If the degree of divergence between the model formulated in advance as a general gaze position when viewing the image content and the gaze position of the user is greater than a predetermined value, the recall determination unit The content evaluation device according to claim 3, wherein the user is determined to be in a recall state. A dwell time calculation unit that calculates a dwell time that is a time during which the movement distance of the line-of-sight position is within a predetermined threshold;
Recall information that is information that associates a display time of a scene that is each time-divided image content with a display area of a provided image that is an image displayed as part of the image content at the display time. And a recall information storage unit
For each of the scenes, the recall determination unit determines in advance a distance between the line-of-sight position in the display time of the scene and the display area associated with the display time, and the dwell time in the display time. The content evaluation apparatus according to claim 4, wherein it is determined whether or not the threshold value is equal to or greater than a predetermined threshold value, and if the threshold value is equal to or greater than the threshold value, the user is determined to be in a recall state. The content evaluation apparatus according to claim 5, wherein the recall determination unit calculates a recall level as a ratio of a total number of scenes that the user determines to be in the recall state with respect to a total number of scenes included in the recall information. When the acquired cerebral blood flow exceeds a first threshold, the reaction identifying unit identifies the type of psychological reaction of the user as being in an excited state, and the cerebral blood flow is If it falls below the threshold, the user's psychological response type is identified as being concentrated,
The first threshold value is larger than the cerebral blood flow of the user at the reference time,
The content evaluation apparatus according to claim 1, wherein the second threshold is a value smaller than the cerebral blood flow of the user at the reference time. The reaction identification unit includes the time when the acquired absolute value of the cerebral blood flow exceeds the predetermined threshold, and the absolute value of the integrated value of the cerebral blood flow in the predetermined time interval is The user showed the psychological response to the time included in the time interval from one of the times that exceeded the value set before the article to the time that went back by a predetermined time. Identify it as a reaction time candidate that is a time candidate,
The content evaluation device according to claim 6, wherein the reaction target specifying unit specifies a target on which the user has shown the psychological reaction based on the reaction time candidate and the recall level. The reaction target specifying unit specifies, when the recall level is smaller than a predetermined value, the image content viewed as the reaction time candidate as a target for which the user has shown the psychological reaction, 9. The image content viewed before the reaction time candidate is specified as a target for which the user has exhibited the psychological reaction when the recall level is equal to or greater than a predetermined value. Content evaluation device. When the provided image represents character information, the recall determination unit determines that the user recalls the recall if the movement amount of the line-of-sight position within the display time of the provided image is less than a predetermined threshold. The content evaluation device according to claim 5, wherein the content evaluation device is determined to be in a state. In the case where the line-of-sight position is located in the display area of the provided image and the line-of-sight position is located in the display area before the display time of the provided image, The content evaluation device according to claim 5, wherein the content evaluation device is not determined to be in the recall state. The content evaluation device according to claim 5, wherein, when the scene includes a plurality of display areas, the recall determination unit calculates the distance using a degree of separation from the display area closest to the line-of-sight position. . For each of the scenes, the recall determination unit determines in advance a distance between the line-of-sight position in the display time and the display area associated with the display time, and the dwell time in the display time. The content evaluation apparatus according to claim 5, wherein it is determined whether or not the threshold value is equal to or greater than the threshold value, and if each is equal to or greater than the threshold value, it is determined that the user has recalled another scene displayed before the scene. The recall information has a weight for each scene,
The content evaluation apparatus according to claim 6, wherein the recall determination unit totals the total number of scenes determined to be the recall state by multiplying the weight corresponding to the scene. The content evaluation device displays a plurality of image contents,
A stimulus presentation order storage unit storing a display order of the plurality of image contents;
A stimulus presentation order changing unit that changes a display order of the plurality of image contents stored in the stimulus presentation order storage unit;
The content evaluation according to claim 2, further comprising: a stimulus presentation unit that displays the plurality of contents based on an order stored in the stimulus presentation order storage unit or a display order changed by the stimulus presentation order change unit. apparatus. The stimulus presentation order changing unit
The content evaluation apparatus according to claim 15, wherein the display order is changed so that the other image content specified by the reaction target specifying unit is displayed again. The reaction target specifying unit includes an increase / decrease amount of cerebral blood flow when the other image content is displayed for the first time, and an increase / decrease amount of cerebral blood flow when the other image content is displayed for the second time or later. The content evaluation apparatus according to claim 16, wherein it is determined whether or not the user has recalled the other image content based on a difference between the content and the user. The content evaluation device displays a plurality of image contents,
A stimulus presentation order storage unit storing a display order of the plurality of contents;
A stimulus presentation unit that displays the plurality of contents based on the order stored in the stimulus presentation order storage unit;
A line-of-sight history storage unit that stores the line-of-sight position of the user as a history;
Using the gaze position accumulated in the gaze history accumulation unit, the gaze position when each of the plurality of image contents is displayed is identified, and the recall information includes the identified gaze position as the display area The content evaluation device according to claim 5, further comprising: a typical recall information generation unit that generates A content evaluation method for identifying a user's reaction to the image content by analyzing a biological signal of the user viewing the image content,
Obtaining the cerebral blood flow and line-of-sight position of the user;
A reaction identifying step for identifying whether or not the user has a psychological reaction when viewing the image content based on the cerebral blood flow;
A recall determination step of determining whether or not a user viewing the image content has recalled another image content displayed before the displayed image content based on the line-of-sight position of the user; Includes content evaluation methods. A program causing a computer to execute the content evaluation method according to claim 19.

Images