JP5941764B2 - Content evaluation data generation system, content evaluation data generation method, and program - Google Patents

Description

  The present invention relates to a content evaluation data generation system, a content evaluation data generation method, and a program for acquiring information for evaluating and correcting a subject's response to a questionnaire and a subject's response to content. Subject's answer or response by correlating the measurement result of the subject's physiological response with the subject's situation measured by the camera by shooting the situation of the subject who is answering or watching the content. The present invention relates to a technique for acquiring information for correcting the error.

  In recent years, online questionnaire surveys using personal computers, TVs, or tablets connected to the Internet, and measurement of effects on content viewing have become widespread. Such surveys and measurement of effects are considered to be increasingly used as the number of devices connected to the Internet increases.

  Conventionally, in interactive systems such as questionnaire surveys, test questions, or games, data input by a subject using a pointing device such as a mouse or an input device such as a keyboard is obtained as response data of the subject. However, if the data included in the response data that the subject entered easily without being interested in the content of the presented questionnaire or content is included in the acquired response data, This will reduce the reliability of the results.

  For this reason, it is necessary to select easily input data from the acquired response data, but there is no means for selecting the data input by the subject, and the data easily input by the subject and the reliable data are selected. That is, it is very difficult to extract only valid response data. Even if the response data is selected, the final judgment as to whether the selected response data is valid data requires a detailed evaluation by a psychological expert or the like.

  Conventionally, in order to perform evaluation by extracting only valid response data, a method in which a subject is equipped with a sensor, a marker, or the like and the physiological reaction of the subject is measured has been used. However, in this method, since the atmosphere of the environment to be measured and the situation of the subject are different from usual, the subject is given specific consciousness such as tension. For this reason, the reliability of the acquired response data may be impaired, or the subject may be refused to participate in the measurement. In this case, it is difficult to say that response data reflecting the condition of a normal subject is acquired.

  As a technique for solving such a problem, for example, as in Patent Document 1, when measuring a subject's response to content, there is a technique for measuring and analyzing the eye movement of the subject in conjunction with the response data. It is disclosed. By using the technique disclosed in Patent Document 1, it is possible to evaluate content viewing reflecting the condition of a normal subject, and to determine the validity of the response data of the subject.

Japanese Patent No. 4874332

  However, in the technique disclosed in Patent Document 1, it is necessary to accurately capture the image of the eyeball of the subject with a camera in order to effectively measure the eyeball movement of the subject. In recent years, it has become common to view a plurality of contents at the same time in daily life, such as televisions and personal computers. For this reason, when the subject is viewing a plurality of contents, it is difficult to measure the eye movement of the subject, and effective response data in the evaluation of content viewing cannot be obtained with high accuracy. This makes it difficult to evaluate and analyze the presented content. Therefore, the practical range in which the technique disclosed in Patent Document 1 can be used is limited to a very narrow range.

  The present invention has been made based on the above problem recognition, and more accurately obtains information for evaluating the reaction of a subject who is viewing (browsing) content without making the subject aware of it. An object of the present invention is to provide a content evaluation data generation system, a content evaluation data generation method, and a program.

In order to solve the above problem, the content evaluation data generation system of the present invention photographs a subject viewing the content displayed on the display unit and the display unit displaying the content, and the photographed subject an imaging unit for outputting a captured image, a detection unit for detecting a heartbeat of the subject based on the captured image, and heart data detected by the detecting unit, wherein the subject is viewed upon detecting the heartbeat The content that has been recorded, or data indicating the content, and a state acquisition unit that acquires a display state of the display unit when the subject is browsing the content; wherein the detection unit, the captured image is corrected based on the change in display of the display unit in which the state acquisition unit has acquired, based on the corrected captured image Detecting the heartbeat of the subject, and wherein the.
Moreover, the content evaluation data generation system of this invention image | photographs the test subject who is browsing the said content displayed on the display part and the said display part which display a content, and outputs the picked-up image of the said test subject An imaging unit, a detection unit that detects the heartbeat of the subject based on the captured image, heartbeat data detected by the detection unit, and the content that the subject was browsing when the heartbeat was detected, Or a recording unit that records data indicating the content in association with each other, and the detection unit is based on the content viewed by the subject when the subject's heartbeat is detected, or data indicating the content And an evaluation unit that evaluates the effectiveness of the heartbeat of the subject detected by the detection unit.

In the content evaluation data generation method of the present invention, the display step of displaying the content on the display unit, and the subject viewing the content displayed on the display unit are photographed by the photographing unit, and the photographing is performed. A photographing step for outputting a photographed image of the subject, a detecting step for causing the detection unit to detect the heartbeat of the subject based on the photographed image, a heartbeat data detected by the detection unit in the storage unit, and the heartbeat detected The recording step of associating and recording the content that the subject has been browsing or the data indicating the content, and the display unit when the subject is browsing the content in the state acquisition unit viewed contains a status acquisition step of acquiring the state of the display, wherein the detection step, the state in the state acquisition step The captured image is corrected based on the change in display of the display unit is acquired to obtain unit, detects heart of the subject based on the corrected captured image, characterized in that.
In the content evaluation data generation method of the present invention, the display step of displaying the content on the display unit, and the subject viewing the content displayed on the display unit are photographed by the photographing unit, and the photographing is performed. A photographing step for outputting a photographed image of the subject, a detecting step for causing the detection unit to detect the heartbeat of the subject based on the photographed image, a heartbeat data detected by the detection unit in the storage unit, and the heartbeat detected When the detection unit detects the heartbeat of the subject, the recording step of associating and recording the content that the subject was browsing or the data indicating the content when the detection unit detects the heartbeat of the subject. Based on the content being browsed or data indicating the content, the heart rate of the subject detected by the detection unit An evaluation step of evaluating the effectiveness, to include the features.

Further, the program of the present invention includes a display step of displaying content on a display unit, and a subject who is browsing the content displayed on the display unit is photographed by a photographing unit, and the photographed image of the subject that is photographed An imaging step for outputting a signal, a detection step for detecting a heartbeat of the subject based on the captured image, a heartbeat data detected by the detection unit in a storage unit, and the heartbeat when the heartbeat is detected A recording step for associating and recording the content that the subject was browsing, or data indicating the content, and a display state of the display unit when the subject is browsing the content in a state acquisition unit A state acquisition step of acquiring, and the detection step is acquired by the state acquisition unit in the state acquisition step The captured image is corrected based on the change in display in the serial display unit, characterized in that to execute processing for detecting the heart rate of the subject based on the corrected captured image to the computer.
Further, the program of the present invention includes a display step of displaying content on a display unit, and a subject who is browsing the content displayed on the display unit is photographed by a photographing unit, and the photographed image of the subject that is photographed An imaging step for outputting a signal, a detection step for detecting a heartbeat of the subject based on the captured image, a heartbeat data detected by the detection unit in a storage unit, and the heartbeat when the heartbeat is detected The subject that the subject was browsing when the detection unit detected the heartbeat of the subject and the recording step that records the content that the subject was browsing or data indicating the content in association with each other Based on the content or data indicating the content, the effectiveness of the heartbeat of the subject detected by the detection unit is evaluated. Characterized in that to execute a valence step, to the computer.

  ADVANTAGE OF THE INVENTION According to this invention, the content evaluation data generation system and content evaluation data which can acquire more accurately the information for evaluating the reaction of the subject who is viewing (browsing) the content without making the subject aware of it. An effect that a generation method and a program can be provided is obtained.

It is the block diagram which showed an example of schematic structure of the content evaluation data generation system in the 1st Embodiment of this invention. It is the flowchart which showed the process sequence of the heart rate detection process in the heart rate detection part with which the content evaluation data generation system of the 1st embodiment was equipped. It is the block diagram which showed an example of schematic structure of the content evaluation data generation system in the 2nd Embodiment of this invention. It is the block diagram which showed an example of schematic structure of the content evaluation data generation system in the 3rd Embodiment of this invention. It is the block diagram which showed an example of schematic structure of the content evaluation data generation system in the 4th Embodiment of this invention. It is the block diagram which showed an example of schematic structure of the content evaluation data generation system in the 5th Embodiment of this invention. It is the block diagram which showed an example of schematic structure of the content evaluation data generation system in the 5th Embodiment of this invention.

<First Embodiment>
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of a schematic configuration of a content evaluation data generation system according to the first embodiment. FIG. 1 shows an example of a basic configuration in the content evaluation data generation system of the present invention. The content evaluation data generation system 1 illustrated in FIG. 1 is a system that includes an imaging unit 10, a display unit 20, a heartbeat detection unit 30, and a recording unit 40. FIG. 1 also shows the subject H in the content evaluation data generation system 1, but the subject H is not an element constituting the content evaluation data generation system 1 of the first embodiment.

  The display unit 20 includes a display device such as an LCD (Liquid Crystal Display) that displays content, and presents the content to the subject H by displaying the content recorded in the recording unit 40. In the display unit 20, the timing at which content display is started is controlled by a control unit (not shown). When the start of content display is instructed by the control unit, the display unit 20 reads content data to be displayed from the recording unit 40 and displays it.

  The imaging unit 10 includes a solid-state imaging device that photoelectrically converts an optical image of a subject, and captures a moving image of the subject. In the photographing unit 10, the timing for starting moving image photographing is controlled by a control unit (not shown). When the start of moving image shooting is instructed by the control unit, the shooting unit 10 is a moving image (for example, 30 frames / second) of the face area of the subject H who is viewing (browsing) the content displayed on the display unit 20. And the image data (RAW data) of each frame, which is a captured image of the subject H, is output to the heartbeat detecting unit 30 for each frame of the captured moving image.

  The heartbeat detection unit 30 performs image processing (hereinafter referred to as “heartbeat detection processing”) for detecting the heartbeat of the subject H on the RAW data input from the imaging unit 10. In the heart rate detection unit 30, the timing at which the heart rate detection process is started is controlled by a control unit (not shown). When the start of the heart rate detection process is instructed by the control unit, the heart rate detection unit 30 performs the heart rate detection process using the RAW data input from the imaging unit 10 and detects the heart rate of the subject H at regular intervals. A detailed description of the heartbeat detection process in the heartbeat detection unit 30 will be described later.

  Further, when the heartbeat detection process starts, the heartbeat detection unit 30 reads content data from the recording unit 40 in synchronization with the timing at which the content is displayed on the display unit 20. Then, the heartbeat detection unit 30 outputs and records response data in which the detected information about the heartbeat of the subject H (hereinafter also referred to as “biological information”) is associated with the read content data.

  For example, when moving image content such as a movie or a drama is displayed on the display unit 20, the free space of the tag information included in the moving image data is synchronized with the moving image data of each frame in the moving image content. The biological information of the subject H detected for each frame is added. Then, the moving image content in which the biological information of the subject H is sequentially added is recorded in the recording unit 40 as response data. Thereby, the change in the heartbeat of the subject H when viewing the moving image content can be evaluated in association with the moving image content. For example, when the video content provider counts the audience rating of the video content, it extracts the heart rate information added to the tag information of each frame sequentially from the response data viewed by each subject, in order of time. The change in heart rate of each subject can be evaluated. In this way, it is possible not only to reproduce the moving image content but also to collect effective response data that is considered to be actually viewing the moving image content.

  Further, for example, when question content such as a questionnaire is displayed on the display unit 20, the timing at which the subject H answers the questionnaire question, that is, using a pointing device such as a mouse or an input device such as a keyboard. At the timing when the test subject H inputs the answer data, the biometric information of the detected test subject H is added to the response data in which the question number and the answer data are recorded, and recorded in the recording unit 40. As a result, the state and change of the heartbeat of the subject H when the subject H answers each question of the question content can be evaluated in association with the response data for the question content. For example, when the survey organizer sums up the responses to the questionnaire, each subject answers each question by extracting the heart rate information added to the response data answered by each subject for each question. It is possible to evaluate the state of heartbeat and the change of heartbeat with the progress of questions. As a result, it is possible to aggregate effective response data that is considered to be answers that are input with interest in the contents of the questions, rather than answers that are input easily without being interested in the contents of the questions.

  The recording unit 40 records content data to be displayed on the display unit 20. The recording unit 40 records response data in which the biological information of the subject H is associated by the heartbeat detecting unit 30.

  With this configuration, the content evaluation data generation system 1 records the biological information of the subject H synchronized with the timing at which the content is displayed on the display unit 20 in the response data. Thereby, in the content evaluation data generation system 1, it is possible to acquire effective response data for evaluating the reaction of the subject H.

  Next, a heartbeat detection process by the heartbeat detection unit 30 provided in the content evaluation data generation system 1 will be described. In the heart rate detection process by the heart rate detection unit 30, the subject H's position adjustment process in the moving image captured by the imaging unit 10, that is, the shake correction process and the heart rate of the subject H based on the moving image after the shake correction are performed. The calculation process is performed in parallel. FIG. 2 is a flowchart showing a processing procedure of a heartbeat detection process in the heartbeat detection unit 30 provided in the content evaluation data generation system 1 of the first embodiment.

  When the heartbeat detection unit 30 starts the heartbeat detection process in response to an instruction from the control unit, the heartbeat detection unit 30 first starts a shake correction process. In the shake correction process, in step S1, the heartbeat detection unit 30 performs image processing on the RAW data input from the imaging unit 10, and detects a shake detection target image for detecting the shake of the subject H included in the moving image. Create In creating the blur detection target image in step S1, bitmap data is created from the RAW data of the frame. The heartbeat detection unit 30 creates a shake detection target image every time RAW data is input from the imaging unit 10, that is, for each frame image of the moving image captured by the imaging unit 10.

  Subsequently, in step S2, the heartbeat detection unit 30 uses the created blur detection target image of the first frame as a reference, and determines the subject's position from the difference in the position of the subject H with respect to the blur detection target image of the subsequent frames. H blur is detected. More specifically, based on the blur detection target image of the first frame and the blur detection target image of the nth (n = 2 or more natural number) frame, the frame of the moving image shot by the shooting unit 10 is obtained. The amount of blur when the first frame is changed to the nth frame is detected. The blur detection processing method by the heartbeat detection unit 30 can be performed by using a known blur detection technique that is generally used in, for example, a digital video camera. .

  Subsequently, in step S3, the heartbeat detection unit 30 aligns the data of each pixel included in the RAW data of each frame input from the imaging unit 10 based on the shake amount detected in step S2. Correct the blur. For example, blurring of pixels of R, G, and B color components included in the RAW data is corrected by inverse affine transformation. Note that the blur correction processing method by the heartbeat detection unit 30 can be performed by using a known blur correction technique that is generally used in, for example, a digital video camera, and thus detailed description thereof is omitted. .

  The process so far is the blur correction process. Thereafter, the heartbeat detection unit 30 sequentially repeats the blur correction processing from step S1 to step S3 every time RAW data of each frame is input from the imaging unit 10.

  Then, when the blur correction process for the RAW data of the first frame is completed, the heart rate detection unit 30 subsequently starts a heart rate calculation process. In the heart rate calculation process, in step S4, the heart rate detection unit 30 sets (limits) an area (analysis area) ROI to be analyzed from the RAW data data. Here, the face area of the subject H is set as the analysis area ROI. Then, the heartbeat detecting unit 30 extracts the luminance signal of the analysis region ROI, that is, the face region of the subject H, based on the data of each pixel included in the RAW data of the first frame subjected to the shake correction. In the extraction of the luminance signal in step S4, the average value of the luminance signal is obtained for each of the R, G, and B color components included in the RAW data analysis region ROI.

  Thereby, the average value of the luminance signal having a length corresponding to the number of frames of the moving image photographed by the photographing unit 10 can be extracted for each color component. Here, when the average values of the luminance signals for the respective color components of R, G, and B in the analysis region ROI are shown in time order, they are x1 (t), x2 (t), and x3 (t). Note that t is a frame number of a moving image shot by the shooting unit 10.

  Subsequently, in step S5, the heartbeat detecting unit 30 normalizes the average value of the luminance signal for each of the R, G, and B color components extracted in step S4. At this time, the heartbeat detection unit 30 calculates the normalized luminance signal x′i (t) with the following formula (1) so that the average value of the luminance signals of R, G, and B is 0 and the variance is 1. ) To normalize.

  In the above formula (1), i represents each of the color components R, G, and B, 1 represents R, 2 represents G, and 3 represents B. Therefore, xi (t) represents the average value of the luminance signal for each color component of each frame. Μi represents the average value of all the frames of the average value xi (t) of the luminance signal, and σi represents the standard deviation of all the frames of the average value xi (t) of the luminance signal.

  Subsequently, in step S6, the heartbeat detection unit 30 uses the independent component analysis method to decompose the R, G, and B luminance signals after formalization in step S5 into independent components. At this time, the heartbeat detection unit 30 converts the luminance signals x′1 (t), x′2 (t), and x′3 (t) into three independent components s1 (t), s2 (t), and s3. Decompose into (t).

  In the independent component analysis in step S6, the heartbeat detection unit 30 first makes a hypothesis that is expressed by the following equation (2).

  In the above equation (2), A represents a 3 × 3 matrix, and each element of A is aij (i = 1, 2, 3 j = 1, 2, 3). Further, x ′ (t) represents luminance signals of all color components, and s (t) represents independent components of all color components, and is represented by the following expression (3).

  However, there is no matrix A in which the above equation (2) holds as it is. Therefore, the heartbeat detecting unit 30 calculates an evaluation value close to the evaluation value of the independent component s (t) as s ′ (t) and calculates a matrix W close to the inverse matrix of the matrix A by the independent component analysis method.

  Subsequently, in step S7, the heartbeat detection unit 30 performs a fast Fourier transform on the evaluation value s' (t) calculated in step S6 for each color component. At this time, the heartbeat detection unit 30 performs fast Fourier transform on the evaluation values s′1 (t), s′2 (t), and s′3 (t) for each of R, G, and B color components. Conversion is performed to calculate frequency domain representations F (S1), F (S2), and F (S3) for each color component.

  Subsequently, in step S8, the heartbeat detecting unit 30 obtains the power spectrum P in the frequency domain expression F calculated in step S7. At this time, the heartbeat detecting unit 30 performs power spectrums P (S1), P (F (S1), F (S2), and F (S3) in the frequency domain representations F (S1), F (S2), and F (S3) for the respective color components. S2) and P (S3) are obtained respectively.

  Subsequently, in step S9, the heartbeat detection unit 30 selects a power spectrum P in which the characteristics of the heartbeat often appear from the power spectrum P obtained in step S8. For example, the power spectrum P (S2) is selected.

  Subsequently, in step S10, the heartbeat detection unit 30 extracts a frequency representing the heartbeat from the power spectrum P selected in step S9, and further calculates a heart rate based on the extracted frequency. For example, when the imaging frame rate of the imaging unit 10 is 60 frames / second and a heart rate frequency with a power spectrum between 0.75 and 4 is extracted, a value between 45 bpm and 240 bpm is calculated as the heart rate. To do.

  The process so far is the heart rate calculation process. Thereafter, the heart rate detection unit 30 sequentially repeats the heart rate calculation process from step S4 to step S10 every time the blur correction process for the RAW data of the frame input from the imaging unit 10 is completed.

  As described above, in the content evaluation data generation system 1, the heartbeat detection unit 30 performs the heartbeat detection process for each RAW data of the frame input from the imaging unit 10, and detects the heart rate of the subject H for each frame. Thereby, the content evaluation data generation system 1 can detect the heart rate of the subject H when viewing the content without making the subject H aware of it. At this time, in the content evaluation data generation system 1, it is not necessary to accurately capture the image of the eyeball of the subject H as in the conventional technique, that is, the subject H does not have to face almost in front. If the photographing unit 10 can photograph the face area, the heart rate of the subject H can be detected with high accuracy regardless of the direction of the subject H.

  In the content evaluation data generation system 1, the heart rate detection unit 30 synchronizes the response data associated with the content by the display unit 20 using the detected heart rate of the subject H as biometric information in synchronization with the timing at which the content is displayed. Recording is performed by the recording unit 40. Thereby, in the content evaluation data generation system 1, the response data including the information for evaluating the response of the subject H can be recorded. When the recorded response data is aggregated, the data reflecting the response of the subject H is recorded. Can be aggregated.

  Note that, in the content evaluation data generation system 1, the case where the imaging unit 10 outputs RAW data of each frame obtained by imaging the subject H to the heart rate detection unit 30 has been described. However, for example, the image capturing unit 10 may output image data subjected to some gamma correction to the heartbeat detecting unit 30. In this case, in step S4 shown in FIG. 2, before setting the analysis region ROI, the heart rate detection unit 30 performs inverse gamma correction on the image data, thereby performing subsequent heart rate calculation processing. Can be done as well.

  In the content evaluation data generation system 1, the case where the display unit 20 and the heart rate detection unit 30 read the content data recorded in the recording unit 40, that is, the same data has been described. However, the content data read to the heart rate detection unit 30 is not limited to the same data as the content data read to the display unit 20, and the timing at which the content is displayed on the display unit 20 Any data may be used as long as the data can be recognized by 30. That is, the content data read to the heartbeat detection unit 30 is data for synchronizing the timing at which the content is displayed on the display unit 20 and the timing at which the heartbeat detection unit 30 detects the heart rate of the subject H. That's fine. For example, when moving image content is displayed on the display unit 20, only the time data that the heartbeat detection unit 30 can recognize the elapsed time (display time) of the moving image content displayed on the display unit 20. May be. Further, for example, when displaying the question content on the display unit 20, only the question number data that can be recognized by the heartbeat detection unit 30 can be the question of the question content displayed on the display unit 20. Good.

<Second Embodiment>
Next, a content evaluation data generation system according to the second embodiment will be described. FIG. 3 is a block diagram illustrating an example of a schematic configuration of a content evaluation data generation system according to the second embodiment. The content evaluation data generation system 2 illustrated in FIG. 3 is a system that includes an imaging unit 10, a display unit 20, a heartbeat detection unit 30, and a recording unit 40. In FIG. 3, the subject H in the content evaluation data generation system 2 is also shown, but the subject H is not an element constituting the content evaluation data generation system 2 of the second embodiment.

  The components of the content evaluation data generation system 2 of the second embodiment are the same as the components of the content evaluation data generation system 1 of the first embodiment. However, in the content evaluation data generation system 2 of the second exemplary embodiment, as illustrated in FIG. 3, the recording unit 40 includes a data recording unit 41, a content storage unit 42, and a content server 100. The content storage unit 42 and the content server 100 are connected via the Internet 200. Accordingly, in the following description, the same components as those in the content evaluation data generation system 1 of the first embodiment are denoted by the same reference numerals and detailed description thereof is omitted, and different components are described. In the following description, the case where the content is moving image content will be described as an example.

  The content server 100 is a server that stores various content such as content for Internet TV, content for smartphone, content for PC, content for net game, and provides content requested via the Internet 200. In the content evaluation data generation system 2, the content server 100 uses the moving image data of each frame in the moving image content displayed on the display unit 20 and the elapsed time (display time) of the moving image data displayed on the display unit 20 as a heart rate. Time data (hereinafter referred to as “time data”) for the detection unit 30 to recognize is stored. Then, the content server 100 outputs the moving image data and the time data to the content storage unit 42 in response to a read request for moving image content data.

  The content storage unit 42 temporarily stores content data acquired from the content server 100 via the Internet 200 in accordance with control by a control unit (not shown). In the content evaluation data generation system 2, the content storage unit 42 acquires moving image data and time data from the content server 100 and temporarily stores them. Then, when the moving image content data is read from the display unit 20, the content storage unit 42 outputs the temporarily stored moving image data. In addition, when the moving image content data is read from the heartbeat detection unit 30, the content storage unit 42 outputs temporally stored time data.

  The data recording unit 41 records time data associated with the biological information of the subject H as response data by the heartbeat detection unit 30.

  With such a configuration, the content evaluation data generation system 2 can display video data of video content provided via the Internet 200 on the display unit 20 even when the content server 100 is located at a location away from the subject H. Can be displayed. Then, the content evaluation data generation system 2 records the response data in which the biological information of the subject H is associated with the time data indicating the timing at which the moving image data is displayed on the display unit 20. By recording the heart rate of the subject H synchronized with the timing of displaying the response data as the biometric information in the response data, it is possible to acquire response data effective for evaluating the response of the subject H.

  As described above, in the content evaluation data generation system 2 as well, as in the content evaluation data generation system 1 of the first embodiment, the heart rate of the subject H detected by the heart rate detection unit 30 performing the heart rate detection process is used as the content. The data is recorded in the recording unit 40 (data recording unit 41) in synchronization with the displayed timing. Thereby, the content evaluation data generation system 2 can also acquire response data effective for evaluating the reaction of the subject H.

  In the content evaluation data generation system 2, the moving image data and time data acquired from the content server 100 via the Internet 200 are temporarily stored in the content storage unit 42, and the display unit 20 and the heart rate detection unit are transferred from the content storage unit 42. The case where moving image data or time data is output to 30 has been described. However, the method of outputting moving image data or time data to the display unit 20 and the heart rate detection unit 30 is not limited to the above-described example. For example, the display unit 20 may directly acquire moving image data from the content server 100 via the Internet 200. In this case, the content storage unit 42 is configured to temporarily store only the time data acquired from the content server 100 via the Internet 200 and output it to the heartbeat detection unit 30.

  In the content evaluation data generation system 2, the case has been described in which the heart rate detection unit 30 records time data associated with the heart rate of the subject H in the data recording unit 41 as response data. However, the method of recording the response data associated with the heart rate of the subject H in the data recording unit 41 is not limited to the above-described example. For example, the configuration may be such that the heart rate detection unit 30 outputs only the detected heart rate of the subject H to the data recording unit 41. In this case, the content storage unit 42 outputs the temporarily stored time data to the data recording unit 41, and the data recording unit 41 is input from the heartbeat detection unit 30 in accordance with control by a control unit (not shown). The heart rate of the subject H and the time data input from the content storage unit 42 are associated and recorded as response data.

  In the content evaluation data generation system 2, the case has been described where response data in which the heart rate of the subject H detected in the time data is associated is recorded in the recording unit 40 (data recording unit 41). However, the data associated with the response data recorded in the recording unit 40 is not limited to the heart rate of the subject H, and any information that is effective for evaluating the response of the subject H is any information. Also good. For example, the content displayed on the display unit 20 is evaluated in advance, and a heart rate index representing the response of the subject assumed from the evaluation result is recorded in the content in advance, and this index and the detected subject H The result of evaluating the validity of the heart rate information of the subject H associated with the response data by comparing with the heart rate may be recorded in association with the response data.

<Third Embodiment>
Next, a content evaluation data generation system according to the third embodiment will be described. FIG. 4 is a block diagram showing an example of a schematic configuration of the content evaluation data generation system in the third embodiment. FIG. 4 shows an example of a basic configuration in a content evaluation data generation system that records the result of evaluating the effectiveness of detected biological information in association with response data. The content evaluation data generation system 3 shown in FIG. 4 is a system that includes an imaging unit 10, a display unit 20, a heartbeat detection unit 30, a recording unit 40, and an evaluation unit 50. In FIG. 4, the subject H in the content evaluation data generation system 3 is also shown, but the subject H is not an element constituting the content evaluation data generation system 3 of the third embodiment.

  The content evaluation data generation system 3 according to the third embodiment has a configuration in which an evaluation unit 50 is added to the content evaluation data generation system 1 according to the first embodiment. Other components are the same as the components of the content evaluation data generation system 1 of the first embodiment. Accordingly, in the following description, the same components as those in the content evaluation data generation system 1 of the first embodiment are denoted by the same reference numerals and detailed description thereof is omitted, and different components and operations will be described. . In the following description, a case where the content is question content will be described as an example.

  When the display unit 20 starts displaying the question content in response to control by a control unit (not shown), the question content data (hereinafter, “question data”) is read from the recording unit 40 one by one and sequentially. indicate.

  The imaging unit 10 shoots a moving image of the face area of the subject H who is an answerer who answers the question of the question content displayed on the display unit 20 according to control by a control unit (not shown). The image data (RAW data) of each frame of H is sequentially output to the heartbeat detecting unit 30.

  When the heartbeat detection unit 30 starts heartbeat detection processing in accordance with control by a control unit (not shown), data for recognizing the question number of the question data displayed on the display unit 20 (hereinafter referred to as “question number data”). ) Is read from the recording unit 40. Then, the heart rate detecting unit 30 reads out the heart rate of the subject H detected at the timing when the subject H inputs the answer data to the question using a pointing device such as a mouse (not shown) or an input device such as a keyboard. The data is output to the evaluation unit 50 in association with the number data.

  The recording unit 40 records question data and question number data. In addition, in the recording unit 40, the heart rate of the subject assumed in advance for each question by the question content provider is recorded as index data.

  The evaluation unit 50 evaluates the effectiveness of the heart rate of the subject H detected by the heart rate detection unit 30 according to control by a control unit (not shown). The evaluation of the effectiveness by the evaluation unit 50 is performed by comparing the question number data associated with the heart rate of the subject H input from the heart rate detection unit 30 with the index data read from the recording unit 40. And the evaluation part 50 outputs the new question number data which reflected the result of having evaluated effectiveness to the question number data input from the heartbeat detection part 30, and outputs it to the recording part 40 as response data, and records it.

  More specifically, the index data includes a heart rate value assumed when the subject answers the question, which is preset by the question content provider. The evaluation unit 50 has a heart rate value when each question included in the question number data input from the heart rate detection unit 30 is answered, and a heart rate value assumed in the corresponding question included in the index data. Compare Then, the evaluation unit 50 detects the subject H detected for the question when the difference between the heart rate value included in the question number data and the heart rate value included in the index data is within a predetermined range. Determine that your heart rate is valid. Conversely, when the difference between the values of the respective heart rates is outside the predetermined range, it is determined that the heart rate of the subject H detected with respect to this question is invalid. Thereafter, the evaluation unit 50 evaluates the validity of the information indicating whether or not the heart rate is valid, that is, the detected heart rate information of the subject H, and the question number associated with the heart rate of the subject H. It adds to data and outputs to the recording part 40 as response data. In addition, the evaluation part 50 adds the flag showing whether the associated heart rate is effective as an evaluation result of effectiveness for every question, for example.

  The recording unit 40 records the response data to which the result of evaluating the validity of the heart rate information of the subject H by the evaluation unit 50 is added.

  With such a configuration, the content evaluation data generation system 3 displays the content evaluation data on the display unit 20 in the same manner as the content evaluation data generation system 1 of the first embodiment and the content evaluation data generation system 2 of the second embodiment. The heart rate of the subject H corresponding to the content being read is detected. Then, the content evaluation data generation system 3 evaluates the effectiveness of the detected heart rate of the subject H based on an index assumed in advance for the content displayed on the display unit 20, and returns the evaluation result as a response. Record in data. Thereby, in the content evaluation data generation system 3, the response data for evaluating the reaction of the subject H can be corrected, and more effective response data can be acquired. Thus, for example, when the questionnaire implementer sums up the answers to the questionnaire, among the answers to each question entered by the subject H, it is effective that the answer is entered with an interest in the contents of the question. Only response data can be aggregated.

  In the content evaluation data generation system 3, the basic configuration in the content evaluation data generation system having the configuration in which the evaluation unit 50 that evaluates the effectiveness of the detected biological information is added has been described. In the content evaluation data generation system having a simple configuration, the effectiveness of the detected biological information can be evaluated in the same manner.

<Fourth Embodiment>
Next, a content evaluation data generation system according to the fourth embodiment will be described. FIG. 5 is a block diagram showing an example of a schematic configuration of a content evaluation data generation system according to the fourth embodiment. The content evaluation data generation system 4 illustrated in FIG. 5 is a system including an imaging unit 10, a display unit 20, a heartbeat detection unit 30, a recording unit 40, and an evaluation unit 50. In FIG. 5, the subject H in the content evaluation data generation system 4 is also shown, but the subject H is not an element constituting the content evaluation data generation system 4 of the fourth embodiment.

  The components of the content evaluation data generation system 4 of the fourth embodiment are the same as the components of the content evaluation data generation system 3 of the third embodiment. In the content evaluation data generation system 4 of the fourth embodiment, the recording unit 40 includes a data recording unit 41 and the content evaluation data generation system 2 of the second embodiment shown in FIG. The content storage unit 42 and the content server 100 are configured such that the content storage unit 42 and the content server 100 are connected via the Internet 200. Therefore, in the following description, the same components as those in the content evaluation data generation system 3 of the third embodiment and the content evaluation data generation system 2 of the second embodiment are denoted by the same reference numerals and detailed. Detailed description will be omitted, and different components and operations will be described. In the following description, the case where the content is moving image content will be described as an example.

  In the content server 100, index data is recorded in addition to moving image data and time data. In this index data, the video content provider evaluates the subject's reaction to the scene of the video data in advance, and the heart rate of the subject assumed based on the evaluation result corresponds to the elapsed time (display time) of the video data The data recorded as The content server 100 outputs moving image data, time data, and index data to the content storage unit 42 via the Internet 200 in response to a read request for moving image content data.

  The content storage unit 42 obtains each of the moving image data, the time data, and the index data from the content server 100 via the Internet 200 and temporarily stores them according to control by a control unit (not shown). Then, the content storage unit 42 outputs the temporarily stored moving image data according to the reading of the moving image content from the display unit 20, and temporarily stores it according to the reading of the moving image content from the heartbeat detection unit 30. Time data is output to the heartbeat detection unit 30 and index data is output to the data recording unit 41.

  When the heart rate detection unit 30 starts the heart rate detection process in response to control by a control unit (not shown), the heart rate of the subject H detected at a timing synchronized with the moving image data displayed on the display unit 20 is stored in the content storage unit. In association with the time data read from 42, the data is recorded and output to the data recording unit 41.

  The data recording unit 41 records time data associated with the heart rate information of the subject H by the heart rate detecting unit 30. The data recording unit 41 records the index data output from the content storage unit 42.

  The evaluation unit 50 compares the time data associated with the heart rate of the subject H with the heart rate detection unit 30 stored in the data recording unit 41 and the index data in accordance with control by a control unit (not shown). The effectiveness of the heart rate of the subject H detected by the heart rate detection unit 30 is evaluated. Then, when the data is read from the data recording unit 41, the evaluation unit 50 outputs new time data reflecting the result of evaluating the validity to the time data as response data to the data recording unit 41 for recording again.

  In addition, the evaluation method of the effectiveness of the heart rate of the subject H in the evaluation unit 50 is the same as the evaluation method of the effectiveness of the heart rate of the subject H in the evaluation unit 50 in the content evaluation data generation system 3 of the third embodiment. Detailed explanation is omitted because it can be considered.

  The data recording unit 41 records the response data to which the result obtained by evaluating the effectiveness of the heart rate information of the subject H by the evaluation unit 50 is added.

  With such a configuration, in the content evaluation data generation system 4, the heart rate of the subject H detected corresponding to the content displayed on the display unit 20, as in the content evaluation data generation system 3 of the third embodiment. And the evaluation result of evaluating the effectiveness of the heart rate are recorded in the response data. Thereby, also in the content evaluation data generation system 4, as in the content evaluation data generation system 3 of the third embodiment, the response data for evaluating the response of the subject H is corrected and more effective response data is acquired. can do. Thus, for example, when the provider of the video content aggregates the audience rating of the video content, response data for an effective period in which it is considered that the video content is actually viewed in each subject's heart rate change Can only be counted. More specifically, for example, during the period when the moving image content is stimulating content, the heart rate of the index data is high, but there is no change in the heart rate of the subject associated with the response data. When the rate of change is small, it can be determined that the subject is only playing the video content and is not viewing it. When making this determination, it should be determined from the period in which the heart rate of the index data is the highest value in the target period, taking into account the deviation of the subject's heart rate change within a predetermined period. Is desirable. Then, by not including the evaluation of this period in the audience rating tabulation, a more accurate audience rating tabulation result can be obtained.

  Note that, in the content evaluation data generation system 4, as in the content evaluation data generation system 2 of the second embodiment, moving image data, time data, and index data acquired from the content server 100 via the Internet 200 are used. A case has been described in which the content data is temporarily stored in the content storage unit 42 and moving image data, time data, or index data is output from the content storage unit 42 to the display unit 20, the heart rate detection unit 30, and the data recording unit 41. However, as with the content evaluation data generation system 2 of the second embodiment, the method for outputting the respective data to the display unit 20, the heartbeat detection unit 30, and the data recording unit 41 is limited to the above-described example. is not. For example, as in the content evaluation data generation system 2 of the second embodiment, the display unit 20 may directly acquire moving image data from the content server 100 via the Internet 200.

  In the content evaluation data generation system 4, the time data associated with the heart rate of the subject H and the index data are recorded in the data recording unit 41, and the time when the evaluation unit 50 is recorded in the data recording unit 41. The case where the effectiveness of the heart rate of the subject H is evaluated based on the data and the index data has been described. However, the method by which the evaluation unit 50 acquires each data for evaluating the effectiveness of the heart rate of the subject H is not limited to the above-described example. For example, the heart rate detection unit 30 directly outputs time data associated with the heart rate of the subject H to the evaluation unit 50, and the evaluation unit 50 directly acquires index data from the content server 100 via the Internet 200, The structure which evaluates the effectiveness of the heart rate of H may be sufficient. Further, for example, the evaluation unit 50 may output response data directly to the content server 100 or the like via the Internet 200.

<Fifth Embodiment>
Next, a content evaluation data generation system according to a fifth embodiment will be described. FIG. 6 is a block diagram illustrating an example of a schematic configuration of a content evaluation data generation system according to the fifth embodiment. The content evaluation data generation system 5 illustrated in FIG. 6 is a system including a photographing unit 10, a display unit 20, a heartbeat detection unit 30, a recording unit 40, and a subject state detection unit 60. In the content evaluation data generation system 5 of the fifth embodiment, the recording unit 40 includes a data recording unit 41 and the content evaluation data generation system 2 of the second embodiment shown in FIG. The content storage unit 42 and the content server 100 are configured such that the content storage unit 42 and the content server 100 are connected via the Internet 200. FIG. 6 also shows the subject H in the content evaluation data generation system 5, but the subject H is not an element constituting the content evaluation data generation system 5 of the fifth embodiment.

  The content evaluation data generation system 5 of the fifth embodiment has a configuration in which a subject state detection unit 60 is added to the content evaluation data generation system 2 of the second embodiment shown in FIG. Other components are the same as those of the content evaluation data generation system 2 of the second embodiment. Accordingly, in the following description, the same components as those in the content evaluation data generation system 2 of the second embodiment are denoted by the same reference numerals, detailed description thereof is omitted, and different components and operations will be described. .

  The subject state detection unit 60 performs image processing on the RAW data output from the imaging unit 10 to detect the orientation of the face of the subject H included in the moving image. Then, the subject state detection unit 60 outputs information on the detected face orientation of the subject H to the heartbeat detection unit 30. Note that the method of detecting the orientation of the face of the subject H by the subject state detection unit 60 is performed using a known face detection technique that is generally used in, for example, a digital still camera or a digital video camera. Since it can do, detailed description is abbreviate | omitted.

  The heartbeat detection unit 30 changes the region of the RAW data data to be analyzed when performing the heartbeat detection process according to the information on the face orientation of the subject H input from the subject state detection unit 60. More specifically, the heartbeat detection unit 30 determines the orientation of the face of the subject H input from the subject state detection unit 60 when setting the analysis region ROI in step S4 of the flowchart of the heartbeat detection process shown in FIG. Set based on the information. For example, when the information on the orientation of the face of the subject H input from the subject state detection unit 60 indicates that the face of the subject H is facing the front with respect to the imaging unit 10, “ A portion including “Forehead” is set as an analysis region ROI. Further, for example, the information on the face orientation of the subject H input from the subject state detection unit 60 is in the lateral direction with respect to the photographing unit 10, that is, the face of the subject H photographed by the photographing unit 10 is the side surface. In this case, the part including “Komekami” in the face area of the subject H is set as the analysis area ROI.

  With such a configuration, the content evaluation data generation system 5 calculates the heart rate of the subject H in consideration of the orientation of the face when the subject H is viewing the content displayed on the display unit 20. Thereby, the content evaluation data generation system 5 can detect the heart rate information of the subject H with higher accuracy, that is, calculate the heart rate. That is, in the content evaluation data generation system 5, if the RAW data captured by the imaging unit 10 includes the face area of the subject H, the subject can accurately capture the image of the eyeball of the subject H in the conventional technique. It is not necessary for H to be substantially facing the front, and the heart rate of the subject H can be detected with high accuracy even when the subject H is facing in any direction.

  Note that the content evaluation data generation system 5 outputs information about the face orientation of the subject H detected by the subject state detection unit 60 to the heartbeat detection unit 30 and the heartbeat detection unit 30 sets the analysis region ROI. did. However, the method for setting the analysis region ROI is not limited to the above-described example. For example, the subject state detection unit 60 may determine the analysis region ROI based on the detected face orientation of the subject H, and set the determined analysis region ROI in the heartbeat detection unit 30. In this case, the heartbeat detection unit 30 performs the heartbeat detection process with the set analysis region ROI.

<Sixth Embodiment>
Next, a content evaluation data generation system according to the sixth embodiment will be described. FIG. 7 is a block diagram illustrating an example of a schematic configuration of a content evaluation data generation system according to the sixth embodiment. The content evaluation data generation system 6 illustrated in FIG. 7 is a system including an imaging unit 10, a display unit 20, a heartbeat detection unit 30, a recording unit 40, and an illumination state acquisition unit 70. Further, in the content evaluation data generation system 6 of the sixth embodiment, as in the content evaluation data generation system 2 of the second embodiment shown in FIG. The content storage unit 42 and the content server 100 are configured such that the content storage unit 42 and the content server 100 are connected via the Internet 200. FIG. 7 also shows the subject H in the content evaluation data generation system 6, but the subject H is not an element constituting the content evaluation data generation system 6 of the sixth embodiment.

  The content evaluation data generation system 6 of the sixth embodiment has a configuration in which an illumination state acquisition unit 70 is added to the content evaluation data generation system 2 of the second embodiment shown in FIG. Other components are the same as those of the content evaluation data generation system 2 of the second embodiment. Accordingly, in the following description, the same components as those in the content evaluation data generation system 2 of the second embodiment are denoted by the same reference numerals, detailed description thereof is omitted, and different components and operations will be described. .

  The illumination state acquisition unit 70 detects the brightness of the environment where the subject H is viewing the content, and the brightness and color of the content displayed on the display unit 20. Then, the illumination state acquisition unit 70 outputs the detected information on the brightness of the environment and the information on the brightness and color of the content to the heartbeat detection unit 30. More specifically, the illumination state acquisition unit 70 sequentially detects the brightness of the environment where the subject H views the content in accordance with control by a control unit (not shown). In addition, the illumination state acquisition unit 70 acquires content data to be displayed on the display unit 20 from the content server 100 via the Internet 200 in accordance with control by a control unit (not shown), and the current environment (brightness). When the content is viewed, the brightness and color of the content displayed on the display unit 20 reflected on the face of the subject H, or changes in brightness and color are sequentially detected. Then, the illumination state acquisition unit 70 sequentially outputs the detected content brightness and color information and environmental brightness information to the heartbeat detection unit 30.

  The heartbeat detection unit 30 corrects the RAW data of each frame input from the imaging unit 10 according to the brightness and color information of the content input from the illumination state acquisition unit 70 and the brightness information of the environment. . More specifically, the heartbeat detection unit 30 sequentially corrects the RAW data of each frame so as to suppress (reduce) the brightness and color of the content. Thereby, for example, when the surrounding illumination environment where the content is viewed is dark such as at night, the illumination that hits the face of the subject H is greatly influenced by the change in the brightness and color of the content displayed on the display unit 20. The component of each pixel in the RAW data that hinders heartbeat detection processing can be suppressed in advance. Then, the heartbeat detection unit 30 performs a heartbeat detection process based on the corrected RAW data.

  With such a configuration, the content evaluation data generation system 6 suppresses the brightness and color of the content reflected on the face when the subject H is viewing the content displayed on the display unit 20. The heart rate of subject H is calculated. Thereby, the content evaluation data generation system 6 can detect the heart rate information of the subject H with higher accuracy, that is, calculate the heart rate.

  In the content evaluation data generation system 6, the case where the illumination state acquisition unit 70 detects the brightness and color of content from the content data displayed on the display unit 20, that is, the same data has been described. However, the data that the illumination state acquisition unit 70 detects the brightness and color of the content is not limited to the same data as the content data displayed on the display unit 20, and the brightness of the content that displays the content on the display unit 20. Any data may be used as long as it can recognize the sheath color. For example, data representing the brightness and color of content prepared separately from the content data displayed on the display unit 20 may be used.

  In the content evaluation data generation system 6, the case has been described in which the illumination state acquisition unit 70 directly acquires content data to be displayed on the display unit 20 from the content server 100 via the Internet 200. However, the method by which the illumination state acquisition unit 70 acquires content data is not limited to the above-described example. For example, the illumination state acquisition unit 70 may read the content data temporarily stored in the content storage unit 42.

  As described above, according to the embodiment for carrying out the present invention, the heartbeat detection unit 30 displays content on the display unit 20 based on the RAW data for each frame input from the imaging unit 10. The biological information (heart rate information: heart rate) of the subject H synchronized with the timing is detected. The detected biological information is recorded in response data in association with the content. Thereby, in the form for implementing this invention, the response data effective in evaluating the reaction of the test subject H who is viewing the content can be acquired more accurately without making the test subject H aware of it. . Then, by evaluating the change in the biological information (heart rate information: heart rate) of the subject H associated with the response data, the response of the subject H can be more accurately evaluated. Moreover, in the form for implementing this invention, when detecting biometric information (heart rate information: heart rate), there is no restriction regarding the orientation of the face of the subject H as in the prior art, and the content evaluation data generation system Can be used in a wide range of practical use.

  In the present embodiment, data of each content (in the embodiment, moving image data in the moving image content and question data in the question content) and accompanying data (in the embodiment, time data in the moving image content, In the above description, the question number data and the index data in the question content are different data. However, the format of each data is not limited to the mode for carrying out the present invention, and may be a format including data accompanying the content data.

  Moreover, in this embodiment, about the case where the imaging | photography part 10, the display part 20, the heart rate detection part 30, and all or one part of the recording part 40 are arrange | positioned in the position where the test subject H is viewing the content. explained. However, the arrangement of each component in the content evaluation data generation system is not limited to the form for carrying out the present invention. For example, only the photographing unit 10, the display unit 20, and the heart rate detection unit 30 are arranged at a position where the subject H is viewing content, and all of the recording units 40 are arranged at different positions and connected via the Internet 200. It may be configured. In addition, for example, a configuration in which all the components other than the imaging unit 10 and the display unit 20 are arranged at other positions connected via the Internet 200 may be employed.

  Moreover, in this embodiment, the structure provided with any one component of the evaluation part 50, the test subject state detection part 60, or the illumination state acquisition part 70 in the content evaluation data generation system was demonstrated. The components provided in the content evaluation data generation system are not limited to the mode for carrying out the present invention. For example, the content evaluation data generation system may be configured to include any one component or all the components of the evaluation unit 50, the subject state detection unit 60, and the illumination state acquisition unit 70.

  Moreover, in this embodiment, although demonstrated that the heart-rate detection part 30 was comprised with the hardware, the means to implement | achieve the function of the heart-rate detection part 30 is limited to the form for implementing this invention. It is not a thing. For example, the function of the heartbeat detecting unit 30 can be realized by software by a computer program or the like. In this case, it is conceivable that the heartbeat detection unit 30 is realized by hardware such as a CPU and a memory of a computer.

  In that case, a program for realizing processing by the heartbeat detection unit 30 or the content evaluation data generation system is recorded on a computer-readable recording medium (recording device), and the program recorded on the recording medium is recorded on the computer. The above-described various processes related to the heartbeat detection unit 30 and the content evaluation data generation system may be performed by being read and executed by the system. Here, the “computer system” may include an OS and hardware such as peripheral devices. Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

  Further, the “computer-readable recording medium” refers to a volatile memory (for example, DRAM (Dynamic) in a computer system serving as a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Random Access Memory)) that holds a program for a certain period of time is also included. The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  The embodiment of the present invention has been described above with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes various modifications within the scope of the present invention. It is.

1, 2, 3, 4, 5, 6 ... content evaluation data generation system 10 ... photographing unit 20 ... display unit 30 ... heart rate detection unit (detection unit)
40: Recording unit 41: Data recording unit (recording unit)
42 ... Content storage unit (recording unit)
50: Evaluation unit 60: Subject state detection unit (state detection unit)
70: Illumination state acquisition unit (state acquisition unit)
100: Content server (recording unit)
200 ... Internet

Claims (22)

A display for displaying content;
An imaging unit that images a subject who is browsing the content displayed on the display unit, and outputs a captured image of the imaged subject.
A detection unit for detecting a heartbeat of the subject based on the captured image;
A recording unit that records the heartbeat data detected by the detection unit and the content that the subject was browsing when the heartbeat was detected, or data indicating the content in association with each other;
A state acquisition unit that acquires a display state of the display unit when the subject is browsing the content;
Equipped with a,
The detector is
Correcting the captured image based on the display change of the display acquired by the state acquisition unit, and detecting the heartbeat of the subject based on the corrected captured image;
A content evaluation data generation system characterized by that. The detector is
The imaging so as to suppress the change in display of the display unit appearing in the image, content evaluation data generation system according to claim 1, characterized in that to correct the captured image. The detection unit evaluates the effectiveness of the subject's heartbeat detected by the detection unit based on the content viewed by the subject when the heartbeat of the subject is detected or data indicating the content. Evaluation department,
Further comprising
Content evaluation data generation system according to claim 1 or claim 2, characterized in that. The recording unit is
The evaluation unit records information indicating the result of evaluating the effectiveness of the heartbeat of the subject in association with the content that the subject was browsing, or data indicating the content,
The content evaluation data generation system according to claim 3 . A display for displaying content;
An imaging unit that images a subject who is browsing the content displayed on the display unit, and outputs a captured image of the imaged subject.
A detection unit for detecting a heartbeat of the subject based on the captured image;
A recording unit that records the heartbeat data detected by the detection unit and the content that the subject was browsing when the heartbeat was detected, or data indicating the content in association with each other;
The detection unit evaluates the effectiveness of the subject's heartbeat detected by the detection unit based on the content viewed by the subject when the heartbeat of the subject is detected or data indicating the content. An evaluation unit;
A content evaluation data generation system comprising: The recording unit is
The evaluation unit records information indicating the result of evaluating the effectiveness of the heartbeat of the subject in association with the content that the subject was browsing, or data indicating the content,
The content evaluation data generation system according to claim 5. A state acquisition unit for acquiring a display state of the display unit when the subject is browsing the content;
Further comprising
The detector is
Correcting the captured image based on the display change of the display acquired by the state acquisition unit, and detecting the heartbeat of the subject based on the corrected captured image;
The content evaluation data generation system according to any one of claims 5 to 6. The detector is
The photographed image is corrected so as to suppress a change in display on the display unit that appears in the photographed image.
The content evaluation data generation system according to claim 7. The detector is
Based on the captured image, a region for detecting the heartbeat of the subject is limited, and the heartbeat of the subject is detected as the heartbeat data from the captured image of the limited region.
9. The content evaluation data generation system according to claim 1 , wherein the content evaluation data generation system is a content evaluation data generation system. A state detector that detects the state of the subject based on the captured image;
Further comprising
The detector is
Based on the state of the subject detected by the state detection unit, the region for detecting the heartbeat is limited.
The content evaluation data generation system according to claim 9 . A display step of displaying content on the display unit;
A photographing step of causing the photographing unit to photograph a subject who is browsing the content displayed on the display unit, and outputting a photographed image of the photographed subject.
A detection step of causing the detection unit to detect the heartbeat of the subject based on the captured image;
A recording step of causing the storage unit to record the heartbeat data detected by the detection unit and the content that the subject was browsing when the heartbeat was detected, or the data indicating the content in association with each other;
A state acquisition step of causing the state acquisition unit to acquire a display state of the display unit when the subject is browsing the content;
Only including,
The detecting step includes
Correcting the captured image based on the display change of the display unit acquired by the state acquisition unit in the state acquisition step, and detecting the heartbeat of the subject based on the corrected captured image;
A content evaluation data generation method characterized by the above. The detecting step includes
The photographed image is corrected so as to suppress a change in display on the display unit that appears in the photographed image.
The content evaluation data generation method according to claim 11, wherein: A display step of displaying content on the display unit;
A photographing step of causing the photographing unit to photograph a subject who is browsing the content displayed on the display unit, and outputting a photographed image of the photographed subject.
A detection step of causing the detection unit to detect the heartbeat of the subject based on the captured image;
A recording step of causing the storage unit to record the heartbeat data detected by the detection unit and the content that the subject was browsing when the heartbeat was detected, or the data indicating the content in association with each other;
Based on the content viewed by the subject when the detection unit detects the heartbeat of the subject, or the data indicating the content, the evaluation unit evaluates the effectiveness of the subject's heartbeat detected by the detection unit An evaluation step that evaluates
The content evaluation data generation method characterized by including. The recording step includes
In the evaluation step, the evaluation unit records information indicating a result of evaluating the effectiveness of the heartbeat of the subject in association with the content that the subject was browsing, or data indicating the content,
The content evaluation data generation method according to claim 13. The detecting step includes
Based on the captured image, a region for detecting the heartbeat of the subject is limited, and the heartbeat of the subject is detected as the heartbeat data from the captured image of the limited region.
15. The content evaluation data generating method according to claim 11, wherein the content evaluation data is generated. A state detecting step of detecting a state of the subject based on the captured image in a state detecting unit;
Further including
The detecting step includes
Limiting the region for detecting the heartbeat based on the state of the subject detected in the state detection step,
The content evaluation data generation method according to claim 15, wherein: A display step of displaying content on the display unit;
A photographing step of causing the photographing unit to photograph a subject who is browsing the content displayed on the display unit, and outputting a photographed image of the photographed subject.
A detection step of causing the detection unit to detect the heartbeat of the subject based on the captured image;
A recording step of causing the storage unit to record the heartbeat data detected by the detection unit and the content that the subject was browsing when the heartbeat was detected, or the data indicating the content in association with each other;
A state acquisition step of causing the state acquisition unit to acquire a display state of the display unit when the subject is browsing the content;
Including
The detecting step includes
Processing for correcting the captured image based on the display change of the display unit acquired by the state acquisition unit in the state acquisition step, and detecting the heartbeat of the subject based on the corrected captured image;
A program that causes a computer to execute. The detecting step includes
A process of correcting the captured image so as to suppress a change in display of the display unit that has appeared in the captured image;
The program according to claim 17, wherein the program is executed by a computer. A display step of displaying content on the display unit;
A photographing step of causing the photographing unit to photograph a subject who is browsing the content displayed on the display unit, and outputting a photographed image of the photographed subject.
A detection step of causing the detection unit to detect the heartbeat of the subject based on the captured image;
A recording step of causing the storage unit to record the heartbeat data detected by the detection unit and the content that the subject was browsing when the heartbeat was detected, or the data indicating the content in association with each other;
Based on the content viewed by the subject when the detection unit detects the heartbeat of the subject, or the data indicating the content, the evaluation unit evaluates the effectiveness of the subject's heartbeat detected by the detection unit An evaluation step that evaluates
A program that causes a computer to execute. The recording step includes
Processing in which the evaluation unit in the evaluation step records information indicating the result of evaluating the effectiveness of the subject's heartbeat in association with the content that the subject was browsing, or data indicating the content;
The program according to claim 19, wherein the program is executed by a computer. The detecting step includes
A process of limiting a region for detecting the heartbeat of the subject based on the captured image, and detecting the heartbeat of the subject as the heartbeat data from the captured image of the limited region;
21. The program according to claim 17, wherein the program is executed by a computer. A state detecting step of detecting a state of the subject based on the captured image in a state detecting unit;
Further including
The detecting step includes
A process for limiting a region for detecting the heartbeat based on the state of the subject detected in the state detection step;
The program according to claim 21, wherein the program is executed by a computer.

Images