JP6856959B1 - Information processing equipment, systems, methods and programs - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique capable of calculating a degree of satisfaction of a viewer who views a video content. SOLUTION: A server 10 of a satisfaction calculation system 1 has, as its functions, an opening degree acquisition module 1034 that acquires and analyzes image data taken by a camera and acquires an eye opening degree in the eyes of a viewer. Satisfaction that calculates the degree of satisfaction with the video content of the viewer by analyzing the image data and acquiring the number of blinks in the eyes of the viewer with the blink number acquisition module 1035 and the degree of opening of the eyes and the number of blinks as parameters. It includes a calculation module 1038. [Selection diagram] Fig. 4

Description

The present disclosure relates to information processing devices, systems, methods and programs.

In recent years, in order to reliably recover the cost of producing video content, or for marketing purposes to analyze the taste of video content, we have provided an opportunity to view video content and obtained evaluation information for the video content. The method of doing this is being studied. Since the video content itself can be easily edited by digitization, it is considered to analyze the evaluation of the video content by using a marketing method such as an A / B test.

There are several examples of evaluation methods for videos on the Internet. Patent Document 1 discloses a social platform for evaluating moving images by so-called throwing money for moving images in a moving image sharing service on the Internet. In the social platform described in Patent Document 1, when an item image for making money is displayed, a payment means for settlement of the amount of money corresponding to the item image is provided, and a money throwing function is implemented.

Japanese Unexamined Patent Publication No. 2012-121998

However, in order to obtain an evaluation for unpublished video content, for example, video content such as a movie, there is currently only a method of taking a questionnaire at a place such as a preview, and for such video content For example, a technique capable of acquiring information indicating a viewer's reaction for each scene has been desired.

Therefore, in the present disclosure, a technique capable of calculating the satisfaction level of a viewer who views a video content will be described.

According to one embodiment of the present disclosure, an information processing device including a control unit is provided. The control unit detects the viewer's eyes from the image data of the viewer viewing the video content and acquires the viewer's eye opening degree, and the viewer's eye from the image data. It is provided with a blinking number acquisition unit that detects the number of blinks of the viewer and acquires the number of blinks of the viewer, and a calculation unit that calculates the satisfaction level of the viewer with respect to the video content by using the degree of opening and the number of blinks as parameters.

According to one embodiment of the present disclosure, a system including a camera and an information processing device is provided. The camera captures a viewer viewing the video content, generates image data and causes the information processing device to acquire the image data, the information processing device includes a control unit, and the control unit captures the viewer viewing the video content. From the image data, the viewer's eyes are detected and the viewer's eye opening degree is acquired. The opening degree acquisition unit and the image data are used to detect the viewer's eyes and acquire the number of blinks of the viewer. A unit for acquiring the number of blinks and a calculation unit for calculating the degree of satisfaction of the viewer with the video content by using the degree of opening and the number of blinks as parameters are provided.

According to one embodiment, a method for being performed by a computer comprising a processor is provided. The method is a step in which the processor detects the viewer's eyes from the image data of the viewer viewing the video content and acquires the degree of opening of the viewer's eyes, and the viewer's eyes are obtained from the image data. A step of detecting and acquiring the number of blinks of the viewer and a step of calculating the satisfaction level of the viewer with respect to the video content are executed by using the degree of opening and the number of blinks as parameters.

Further, according to one embodiment, a program for causing a computer having a processor to execute the program is provided. The program tells the processor a step of detecting the viewer's eyes from the image data of the viewer viewing the video content and acquiring the degree of opening of the viewer's eyes, and the viewer's eyes from the image data. A step of detecting and acquiring the number of blinks of the viewer and a step of calculating the satisfaction level of the viewer with respect to the video content are executed by using the degree of opening and the number of blinks as parameters.

According to the present disclosure, the degree of opening of the viewer's eyes and the number of blinks are acquired from the image data of the viewer viewing the video content, and the satisfaction level of the viewer with the video content is calculated based on these. To do. Therefore, it is possible to acquire information indicating the reaction of the viewer to the video content, for example, for each scene. This makes it possible to analyze the evaluation of the video content.

It is a figure which shows the whole structure of the satisfaction degree calculation system 1. It is a bird's-eye view which shows the example of the appearance of the facility where the satisfaction degree calculation system 1 is installed. It is a perspective view which shows the example of the appearance of the voice output device 30. It is a block diagram which shows the functional configuration of the server 10 which comprises the satisfaction calculation system 1 of Embodiment 1. FIG. It is a schematic diagram which shows the specific method of calculating the opening degree of eyes in the opening degree acquisition module 1034 of FIG. It is a figure which shows the functional configuration of the audio output device 30 which comprises the satisfaction calculation system 1 of Embodiment 1. FIG. It is a figure which shows the data structure of the video content database 181 and the viewer database 184 stored in a server 10. It is a flowchart which shows an example of the flow which performs the video content output processing by the satisfaction calculation system 1 of Embodiment 1. FIG. It is a flowchart which shows an example of the flow which performs the satisfaction calculation processing by the satisfaction calculation system 1 of Embodiment 1. FIG. It is a figure which shows the screen example of the server 10 which displayed the transition of each parameter in the satisfaction degree calculation process.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following description, the same parts are designated by the same reference numerals. Their names and functions are the same. Therefore, the detailed description of them will not be repeated.

<Overview>
Hereinafter, the calculation process of the satisfaction of the viewer who has viewed the video content in the satisfaction calculation system will be described. The satisfaction calculation system provides the viewer with an opportunity to display (screen) a movie-like video content and watch the video content, and at that time, the viewer is photographed and watched. It is a system that acquires the state of a person with a sensor, analyzes the viewer from the captured image data and the detection signal of the sensor, and calculates the degree of satisfaction.

Here, the degree of satisfaction is a numerical value of the degree of satisfaction with the video content, which is estimated from the biological reaction of the viewer who has viewed the video content. In general, when a person watching something (not limited to video content) concentrates on the subject, the eyes (eyelids) open wider, the number of blinks decreases, and the movement of the face. It is empirically known that (number of times and magnitude of movement) decreases and pulse rate and body temperature increase. Therefore, when the viewer who is viewing the video content shows the above reaction, it is considered that the viewer is concentrating on viewing the video content.

Also, if the viewer who is watching the video content finds the video content interesting and fun, he / she may be more focused, and if he / she finds the video content boring or boring. It is thought that concentration becomes distracted and decreases. From the above, it is considered that there is a causal relationship between the viewer's eye opening, the number of blinks, the movement of the face, the pulse rate and the body temperature, and the satisfaction level of the video content viewed.

Therefore, in the satisfaction calculation system of the present disclosure, the state in which the viewer is viewing the video content is photographed with a camera, the face of the viewer is identified from the photographed image data, and the eyes of each viewer are seen. Detect. From the movement of the viewer's eyes in the image data, the degree of eye opening and the number of blinks are acquired. In addition, the movement of the viewer's face is acquired by the detection signal of the sensor that detects the movement of the viewer's face and the biological information, and the biological information of the viewer is acquired. Based on the acquired degree of eye opening, number of blinks, face movement, and biological information, the degree of satisfaction indicating the degree of satisfaction with the video content is calculated. With such a configuration, it is possible to surely acquire the satisfaction level of the viewer's video content based on the viewer's biological reaction.

Further, in the satisfaction calculation system of the present disclosure, when viewing video content, an audio output device such as a headphone or an earphone is attached, and the audio is heard from the audio output device. The above-mentioned sensor is arranged in the audio output device, and the viewer can detect the movement of the viewer's face and the biological information of the viewer by wearing the audio output device. With this configuration, it is possible to reliably acquire the movement of the viewer's face and the biological information of the viewer.

Further, in the satisfaction calculation system of the present disclosure, each viewer who is viewing the video content is identified, and the satisfaction is calculated for each viewer. This makes it possible to acquire satisfaction for each viewer attribute and analyze the evaluation of the video content.

<First Embodiment>
Hereinafter, the satisfaction calculation system 1 will be described.

<1 Overall configuration of satisfaction calculation system 1>
FIG. 1 is a diagram showing the overall configuration of the satisfaction calculation system 1. The satisfaction calculation system is a system for displaying (screening) video content such as a movie, providing the viewer with an opportunity to view the content, analyzing the viewer at that time, and calculating the satisfaction. As shown in FIG. 1, the satisfaction calculation system 1 shows a server 10, a video output device 20, and a plurality of audio output devices (in FIG. 1, the audio output device 30A and the audio output device 30B are shown below. Collectively, it may be referred to as "audio output device 30") and the camera 40. The server 10, the video output device 20, the audio output device 30, and the camera 40 are connected to each other so as to be able to communicate with each other via the network 80. The network 80 is composed of a wired or wireless network.

The server 10 is a device that manages video content to be viewed by the viewer, image data obtained by capturing the viewer, a viewer's detection signal by a sensor, and information on satisfaction with the video content for each viewer. The server 10 transmits the video data of the video content to the video output device 20, and transmits the audio data to the audio output device 30. In addition, the server 10 acquires the image data of the viewer and the detection signal of the sensor, and calculates the satisfaction level of the viewer.

The server 10 is communicably connected to the video output device 20, the audio output device 30, and the camera 40 via the network 80. The server 10 supports wireless base stations 81 that support communication standards such as 4G, 5G, and LTE (Long Term Evolution), and wireless LAN (Local Area Network) standards such as IEEE (Institute of Electrical and Electronics Engineers) 802.11. It is connected to the network 80 by communicating with a communication device such as the wireless LAN router 82. As another configuration of the network 80, each device may be connected wirelessly or by wire, and when connecting wirelessly, the communication protocols include, for example, Z-Wave (registered trademark), ZigBee (registered trademark), and Bluetooth (registered). A trademark) or the like may be used, and when connecting by wire, a USB (Universal Serial Bus) cable or the like may be used for connection.

The server 10 is a computer connected to the network 80. The server 10 includes a communication IF (Interface) 12, an input / output IF 13, a memory 15, a storage 16, and a processor 19.

The communication IF 12 is an interface for inputting / outputting signals because the server 10 communicates with an external device. The input / output IF 13 is an interface with an input device (for example, a pointing device such as a mouse, a keyboard) for receiving an input operation from a user, and an output device (display, a speaker, etc.) for presenting information to the user. Functions as. The memory 15 is for temporarily storing a program, data processed by the program or the like, and is a volatile memory such as a DRAM (Dynamic Random Access Memory). The storage 16 is a storage device for storing data, and is, for example, a flash memory, an HDD (Hard Disc Drive), or an SSD (Solid State Drive). The processor 19 is hardware for executing an instruction set described in a program, and is composed of an arithmetic unit, registers, peripheral circuits, and the like.

The video output device 20 is a device that receives video data of video content and displays (screens) it as a video according to an instruction from the server 10. For example, an LED panel in which LED elements are arranged on the edge of a frame or an organic device. It is composed of an EL display or a liquid crystal display. The video output device 20 is provided in a screening facility such as a movie theater. The video output device 20 may be configured such that a plurality of LED panels are arranged in parallel to form a large-scale vision of about several hundred inches so that a large number of viewers can view the video at a time. Further, the video output device 20 is not limited to one, and may be configured by a plurality of video output devices 20, so that a facility provided with the plurality of video output devices 20 can be remotely controlled by one server 10, for example. It may be configured.

The audio output device 30 is a device that outputs audio data of video content as audio in response to an instruction from the server 10, and the audio output from the audio output device 30 synchronizes with the video displayed on the video output device 20. It is configured as follows. In addition, when a plurality of video output devices 20 are provided as described above and different video contents are screened, the sound corresponding to the screened video content is configured to be output, and the plurality of sounds are output. It may be configured to be selectable from the output channel.

The audio output device 30 is mounted so as to cover the ears of a viewer viewing the video content, and is realized by wireless headphones or the like that receive the audio of the video content by wireless communication and output it as audio under the control of the server 10. .. In addition, the audio output device 30 is not limited to, for example, headphones connected by wire or those that cover the ears of the viewer, and may be earphones connected wirelessly or by wire.

As shown as the audio output device 30B in FIG. 1, the audio output device 30 includes a communication IF 32, a sensor unit 33, a speaker 34, a memory 35, a storage unit 36, and a processor 39.

The communication IF 32 is an interface for inputting / outputting signals because the voice output device 30 communicates with an external device. The sensor unit 33 is a device that acquires the facial movement and biological information of the viewer who is viewing the video content. For example, a gyro sensor that detects the facial movement of the viewer as an angular speed, and a pulse and body temperature are detected. It is composed of an electrocardiographic sensor such as a temperature sensor and an optical sensor that detect it. The sensor unit 33 may be configured by a MEMS (Micro Electro Mechanical Systems) in which a sensor and other parts are integrated. That is, the sensor unit 33 has functions such as vibration detection, a thermometer, an electroencephalograph, and a heart rate monitor.

The speaker 34 is an output device for converting a voice signal into voice and outputting information as voice to a user. The memory 35 is for temporarily storing a program, data processed by the program or the like, and is a volatile memory such as a DRAM (Dynamic Random Access Memory). The storage unit 36 is a storage device for storing data, and is, for example, a flash memory, an HDD (Hard Disc Drive), or an SSD (Solid State Drive). The processor 39 is hardware for executing an instruction set described in a program, and is composed of an arithmetic unit, registers, peripheral circuits, and the like.

The camera 40 is a device that captures a viewer who is viewing video content according to an instruction from the server 10. For example, an image pickup device such as a CCD (Charge Coupled Device) image sensor and the captured image are captured from a digital signal. It is composed of a digital camera or the like provided with an A / D conversion device for converting the image data. The camera 40 may be configured to change the shooting direction under the control of the server 10, or may be configured to appropriately move the shooting direction in order to enable shooting in a wide range.

The camera 40 shoots a viewer who is viewing the video content as a moving image, or shoots the viewer as a still image at predetermined time intervals, and converts the moving image or the still image into image data. Further, the camera 40 transmits the captured image data to the server 10. The number of cameras 40 is not limited to one, and a plurality of cameras 40 may be provided in one facility. Further, the camera 40 is not limited to a single device, and may be configured by, for example, a camera function provided in a mobile terminal such as a smartphone or tablet compatible with a mobile communication system.

<1.1 Configuration of video output device 20 and audio output device 30>
FIG. 2 is a bird's-eye view showing an example of the appearance of the facility in which the satisfaction calculation system 1 is installed. The satisfaction calculation system 1 shown in FIG. 2 shows an example in which the satisfaction calculation system 1 is installed in a space for showing video contents, such as a movie theater. Further, since the satisfaction calculation system 1 is configured so that the viewer wears an audio output device 30 such as a headphone to acquire audio, it is possible to eat, drink, and perform other work other than viewing the video content. It is a facility that constitutes a multipurpose space with a so-called cafe space. Further, the satisfaction calculation system 1 may be configured at, for example, the viewer's home. For example, the server 10 is composed of a cloud server or the like, and the server 10 outputs an image to a video output device 20 composed of a TV, a display, or the like used by the viewer by a service such as on-demand, and the headphones used by the viewer. The audio output device 30 configured by the above may output the audio, the viewer may be photographed by the camera 40 configured by the smartphone or the like used by the viewer, and the image data may be transmitted to the server 10.

The satisfaction calculation system 1 shown in FIG. 2 is provided on the floor F in which the screening space for screening the video content and the cafe space are provided, and the video is displayed at a predetermined location (upper right direction shown in FIG. 2) on the floor F. An output device 20 and a camera 40 are installed. On the front side where the video output device 20 screens the video information of the video content, a seat is provided for the viewer W to view the video content. The viewer W wears the audio output device 30 when viewing the video content as described above.

At this facility, membership registration is required for viewer W to enter the room, attribute information such as viewer W's name, gender, and age is managed, and the movement of each viewer is grasped by shooting with the camera 40. It is supposed to be done. The audio output device 30 is lent by the manager of the facility. As a result, the attribute information of the viewer W registered as a member, the information of the viewer W acquired by the sensor unit 33, and the image data of the viewer W taken by the camera 40 can be associated with each other. , It is possible to grasp the degree of satisfaction for each attribute of the viewer W.

Further, with such a configuration, the viewer W can concentrate on viewing the video content without worrying about the surrounding situation, and can enjoy the video content like a user of the cafe space. Those who are not watching can eat, drink, and perform other actions without worrying about the audio of the video content.

Further, since the video output device 20 is composed of an LED panel or the like, the viewer W side is irradiated with light having a constant brightness. Therefore, the camera 40 can shoot the viewer W at a constant resolution by the light.

As shown in FIG. 2, the facility in which the satisfaction calculation system 1 is installed is not limited to a closed space such as a movie theater, but is a space of a commercial facility such as an eat-in corner of a convenience store or an event space. You may. Further, the satisfaction calculation system 1 may be installed in a space where a certain level of quietness is ensured, such as in a restaurant, a hotel lounge, or a shared space of an apartment house such as an apartment. Further, the satisfaction calculation system 1 may be installed in a game facility such as a pachinko or pachislot store, or a facility having a certain amount of noise such as a game center or an amusement facility.

FIG. 3 is a perspective view showing an example of the appearance of the audio output device 30. As shown in FIG. 3, the audio output device 30 is composed of headphones worn so that both ears are sandwiched from the head in order for the viewer viewing the video content to hear the audio, and the pair of ears accommodating each covering both ears. It is composed of a portion 37 and an arm portion 38 that supports the pair of ear accommodating portions 37 and attaches the ear accommodating portion 37 to the viewer's ear by pressing the viewer's head with an appropriate pressure. The audio output device 30 is not limited to such a configuration, and may be configured such that the ear accommodating portion 37 is mounted so as to be locked to the viewer's earlobe without providing the arm portion 38. It may be a canal type earphone that is inserted into the ear hole of the viewer.

As shown in FIG. 3, a sensor unit 33 is provided at a position inside the pair of ear accommodating units 37 in contact with the skin around the viewer's ears. According to the biological information acquired by the sensor unit 33, the position in the ear accommodating unit 37 is not limited to a specific position, and a plurality of positions may be provided in the ear accommodating unit 37. Further, depending on the type of biological information acquired by the sensor unit 33, it may be arranged so as to be in direct contact with the skin of the viewer, or may be arranged so as to be separated from the skin of the viewer. Since the audio output device 30 has such a configuration, it is desirable to use the audio output device 30 without attaching the ear pads, but it does not prevent the ear pads from being attached.

<1.2 Functional configuration of server 10>
FIG. 4 is a diagram showing a functional configuration of the server 10 constituting the satisfaction calculation system 1 of the first embodiment. As shown in FIG. 4, the server 10 functions as a communication unit 101, a storage unit 102, and a control unit 103.

The communication unit 101 performs a process for the server 10 to communicate with an external device.

The storage unit 102 stores data and programs used by the server 10. The storage unit 102 stores the video content database 181, the sensor detection signal database 182, the camera-captured image database 183, the viewer database 184, and the like.

The video content database 181 is a database for holding information on the video content and its screening time provided by the satisfaction calculation system 1, that is, the screening schedule of the video output device 20. Details will be described later.

The sensor detection signal database 182 is a database for holding the detection signals from the sensor unit 33 of the voice output device 30 in chronological order, which is worn by the viewer who uses the satisfaction calculation system 1. Since the detection signal of the sensor unit 33 is transmitted from the voice output device 30 to the server 10 with the date and time information added, the detection signal is stored.

The camera-captured image database 183 is a database for holding the image data of the camera 40 in chronological order, which captures the viewers who use the satisfaction calculation system 1. Since the image data of the camera 40 is transmitted from the camera 40 to the server 10 with date and time information added, the detection signal is stored.

The viewer database 184 is a database for holding information on viewers who use the satisfaction calculation system 1 and information on the satisfaction of the video content viewed for each viewer. Details will be described later. The viewer database 184 may store member information (including attribute information) registered in the facility.

The control unit 103 performs processing according to a program by the processor of the server 10, and as various modules, the reception control module 1031, the transmission control module 1032, the video content output control module 1033, the opening degree acquisition module (opening degree acquisition unit) 1034, The functions shown in the blink count acquisition module (blink frequency acquisition unit) 1035, face movement acquisition module (movement acquisition unit) 1036, biometric information acquisition module (biological information acquisition unit) 1037, and satisfaction calculation module (calculation unit) 1038 are exhibited. To do.

The reception control module 1031 controls a process in which the server 10 receives a signal from an external device according to a communication protocol.

The transmission control module 1032 controls a process in which the server 10 transmits a signal to an external device according to a communication protocol.

The video content output control module 1033 controls a process of transmitting the video data of the video content to the video output device 20 and transmitting the audio data to the audio output device 30 in order to allow the viewer to view the video content. The video content output control module 1033 refers to the video content database 181 and controls the video content to be screened while transmitting the image data and the audio data of the video content based on the schedule information indicating the date and time when the video content is screened. It transmits signals and also transmits information such as the data capacity, data format, and resolution of video content.

At this time, the video content output control module 1033 may collectively transmit and screen the image data and the audio data to the video output device 20 and the audio output device 30, and sequentially transmit and screen the image data and the audio data in a streaming format. May be good. The video content output control module 1033 may transmit credit information such as the screening time of the video content and the name or name of the creator.

The opening degree acquisition module 1034 controls a process of acquiring image data captured by the camera 40, analyzing the image data, recognizing and detecting the viewer's eyes, and acquiring the degree of opening of the viewer's eyes. The opening degree acquisition module 1034 uses an image recognition technique such as machine learning, which is a known technique, as a method for identifying the eyes of the viewer.

For example, the degree of opening acquisition module 1034 analyzes the image data to detect the eyes of the viewer, calculates the area of the portion of the image data in which the eyes of the viewer are opened, and acquires this area as the degree of opening of the eyes. .. Further, the opening degree acquisition module 1034 analyzes the image data to detect the viewer's eyes, and the contact point between the upper eyelid and the lower eyelid (the left and right end points of the eyes) in the portion of the image data where the viewer's eyes are opened. ) And the upper end point of the upper eyelid, or the contact point between the iris (so-called black eye) and the upper eyelid, and the lower end point of the lower eyelid, or the position information of the contact point between the iris and the lower eyelid. Based on the information, it is acquired as the degree of eye opening of the viewer.

FIG. 5 is a schematic diagram showing a specific method for calculating the degree of opening of the eyes in the degree of opening degree acquisition module 1034 of FIG. The method of calculating the degree of eye opening will be described with reference to FIG. FIG. 5 shows an eye E schematically showing the viewer's eyes, and the eye E shows an upper eyelid L1, a lower eyelid L2, and an iris L3. Here, FIG. 5A is an example in which the iris L3 is in contact with the upper eyelid L1 and the lower eyelid L2, and FIG. 5B is an example in which the iris L3 is in contact with the upper eyelid L1 and the lower eyelid L2. This is an example when there is no such thing.

As shown in FIG. 5A, when the iris L3 is in contact with the upper eyelid L1 and the lower eyelid L2, the contact points P1 and P2 between the upper eyelid L1 and the lower eyelid L2 and the iris are analyzed by analyzing the image data. It is possible to recognize the positions of the contacts P3 and P4 between L3 and the upper eyelid L1 and the contacts P5 and P6 between the iris L3 and the lower eyelid L2. Therefore, the coordinates of the contacts P1 to P6 in the image data of the viewer's eyes E are calculated, and the degree of opening of the viewer's eyes is acquired based on the position information of the coordinates.

Further, as shown in FIG. 5B, when the iris L3 is not in contact with the upper eyelid L1 and the lower eyelid L2, the contact points P1 and P2 between the upper eyelid L1 and the lower eyelid L2 are obtained by analyzing the image data. It is possible to recognize the positions of the upper end point P7 of the upper eyelid L1 and the lower end point P8 of the lower eyelid L2. Therefore, the coordinates of the contacts P1, P2 and the end points P7 and P8 in the image data of the viewer's eyes E are calculated, and the degree of opening of the viewer's eyes is acquired based on the position information of the coordinates. For example, when the iris L3 is in contact with the upper eyelid L1 but not the lower eyelid L2, the contact points P3 and P4 are recognized for the upper eyelid L1 by the method shown in FIG. 5 (a). For the lower eyelid L2, the end point P8 may be recognized by the method shown in FIG. 5 (b), and vice versa.

The blink count acquisition module 1035 controls a process of acquiring image data taken by the camera 40, analyzing the image data, recognizing and detecting the viewer's eyes, and acquiring the number of blinks in the viewer's eyes. .. Similar to the opening degree acquisition module 1034, the blink count acquisition module 1035 uses an image recognition technique such as machine learning, which is a known technique, as a method for identifying the eyes of the viewer. In addition, in order to acquire the blink of a person's eyes as image data, a frame rate of 50 fps (frame per second) or more is generally required. Therefore, the camera 40 shoots at a frame rate of 50 fps or more. It is said.

For example, the blink count acquisition module 1035 analyzes the image data to detect the viewer's eyes and acquires the number of blinks of the viewer in a predetermined time (for example, 1 minute). Further, in the blink count acquisition module 1035, when the viewer's eyes are closed and the viewer's eyes are opened within a predetermined time (for example, 1 second) from that state, the viewer Recognize that you have blinked and count as the number of blinks. In such a configuration, if the viewer's eyes are closed and the eyes are not opened for a certain period of time, it cannot be said that the viewer is watching the video content during that period, and it is considered that the viewer is in a sleeping state, for example. Therefore, it is set to distinguish it from the sleeping state.

The face movement acquisition module 1036 acquires a detection signal detected and output by the sensor unit 33 from the audio output device 30 worn when the viewer is viewing the video content, and analyzes the detection signal. Controls the process of acquiring the movement of the viewer's face. The face movement acquisition module 1036 acquires the movement of the viewer's face from the angular velocity detected by the gyro sensor constituting the sensor unit 33 and the time of the movement.

For example, in the face movement acquisition module 1036, the viewer tilts the face back and forth and left and right, or the viewer moves the direction of the face based on the angular velocity detected by the gyro sensor constituting the sensor unit 33 and the time of the movement. Acquires the movement (movement rotated in the left-right direction). Here, the movement in which the viewer tilts his / her face may be a movement by moving the neck or a movement of the entire upper body by moving the trunk portion. Further, the specific example of the movement of the viewer's face to be acquired may be the number of times the viewer moves the face beyond the predetermined movement range within a predetermined time (for example, 10 minutes), and the angle or distance of movement of the face may be obtained. But it may be.

The biological information acquisition module 1037 acquires a detection signal detected and output by the sensor unit 33 from the audio output device 30 worn when the viewer is viewing the video content, and analyzes the detection signal. Controls the process of acquiring the viewer's biometric information. The biological information acquisition module 1037 acquires the viewer's pulse, blood pressure, and body temperature detected by the electrocardiographic sensor constituting the sensor unit 33.

The satisfaction calculation module 1038 includes the degree of opening of the eyes acquired by the degree of opening acquisition module 1034, the number of blinks acquired by the number of blinks acquisition module 1035, the movement of the face acquired by the face movement acquisition module 1036, and the biometric information acquisition module 1037. Controls the process of calculating the viewer's satisfaction with the video content using the biometric information acquired by The satisfaction calculation module 1038 may calculate the satisfaction only from the degree of eye opening and the number of blinks as parameters, may add facial movement as a parameter, or may add biometric information as a parameter. .. Satisfaction is the degree of satisfaction with the video content, which is estimated from the biological reaction of the viewer who viewed the video content as described above, and is quantified in chronological order at predetermined time (for example, 1 minute). Yes, it is calculated as the satisfaction level of the viewer for each viewer, or the satisfaction level of a plurality of viewers is calculated as a total value or an average value and used as an evaluation index of the video content. Further, this satisfaction level may be acquired as a time-series transition at predetermined time intervals.

For example, by grasping the transition of individual satisfaction of viewers, it is possible to group viewers who have similar transitions of satisfaction with a certain video content, or to analyze the attributes of viewers for each group. It may be used as an index value for marketing video content. In addition, by recommending viewers with similar changes in satisfaction, or by assuming that the video content viewed by other viewers with similar changes in satisfaction is also highly satisfying for the viewer, the video content can be selected. You may recommend it.

Here, since the size of the eyes, the number of blinks, the movement of the face, and the biological information depend on the biological activity of the viewer, it is considered that there are large individual differences. Therefore, the satisfaction calculation module 1038 sets, for example, the eyes, face, and physical condition of the viewer first acquired as initial values (reference values), and calculates the satisfaction from the subsequent changes in the respective values or the rate of change. Satisfaction may be calculated from the change in each value within a predetermined time or the rate of change, and the difference from the eyes, face, and physical condition (for example, average value) of a general viewer is satisfied. The degree may be calculated.

For example, the satisfaction calculation module 1038 performs predetermined calculations with the parameters of the degree of eye opening, the number of blinks, the movement of the face, and the value of biometric information quantified by the above example, and after the calculation, each of them performs a predetermined calculation. Satisfaction is calculated by adding the values of. Specifically, the function J for calculating the degree of opening of the eyes is a function of the acquired degree of opening value j. The function J is, for example,
J = αj + β
It may be a linear function such as
J = αj ^ n + βj ^ (n-1) + ... + γ
Higher-order functions such as
J = α ^ (j + n) + β
It may be an exponential function such as. Similarly, let the function K for calculating the number of blinks be a function of the value k of the acquired number of blinks. Let the function L for calculating the movement of the face be a function of the acquired value l of the movement of the face. Let the function M for calculating the biological information be the value m of the acquired biological information. Here, each function may use each value as it is. Satisfaction P is calculated by adding each function as shown in the following formula.
P = J + K + L + M
It is calculated as.

Further, for example, the satisfaction calculation module 1038 multiplies the degree of eye opening, the number of blinks, the movement of the face, and the function of biometric information by different predetermined coefficients as weights according to the above example, and after multiplication. Satisfaction is calculated by adding each value of. In the case of the above example, the satisfaction level P is the following formula P = aJ + bK + cL + dM.
Is calculated by. Here, a, b, c, and d are predetermined coefficients, respectively.

In addition, as described above, when the viewers of the video content are intensively watching, the eye opening is widened, the number of blinks is reduced, the movement of the face is reduced, the pulse rate and the body temperature are reduced. Is empirically known to increase. Therefore, the degree of satisfaction may be calculated by using the coefficients a and d as positive coefficient and the coefficients b and c as negative coefficients. Further, it is considered that the value and the rate of change of each of the above values differ depending on the genre of the video content being viewed. Therefore, the weighting of the coefficients a, b, c, and d may be changed for each genre of the video content. Further, it is considered that a time lag may occur until the viewer reacts to each of the above values depending on the genre of the video content being viewed and the emotion of the viewer for each scene. Therefore, each value may be acquired by correcting the time lag.

Further, for example, the satisfaction calculation module 1038 adds the functions of the degree of opening of the eyes and the number of blinks according to the above example to obtain the degree of concentration, and obtains the degree of satisfaction by performing a predetermined calculation on the degree of concentration. It may be calculated. In the case of the above example, the concentration Q is calculated by the following formula Q = aJ + bK.
The satisfaction level P is calculated by the following formula P = αQ + βR.
Is calculated by. Here, R may be a function or value obtained by performing a predetermined operation on the function of facial movement and biometric information, or may be a function or value calculated using other values as parameters.

<1.3 Functional configuration of audio output device 30>
FIG. 6 is a diagram showing a functional configuration of the audio output device 30 constituting the satisfaction calculation system 1 of the first embodiment. As shown in FIG. 6, the voice output device 30 exhibits functions as a communication unit 301, a detection unit 302, a storage unit 303, and a control unit 304.

The communication unit 301 performs a process for the voice output device 30 to communicate with an external device.

The detection unit 302 performs a process for the voice output device 30 to acquire the movement of the viewer's face and biological information. It is composed of the above-mentioned gyro sensor and electrocardiographic sensor.

The storage unit 303 stores data and programs used by the voice output device 30. The storage unit 303 stores the gyro sensor detection signal database 381, the biosensor detection signal database 382, and the like.

The gyro sensor detection signal database 381 is a database for holding the detection signals of the gyro sensors constituting the detection unit 302 in time series.

The biosensor detection signal database 382 is a database for holding the detection signals of the electrocardiographic sensors constituting the detection unit 302 in time series.

The control unit 304 performs processing according to a program by the processor of the voice output device 30, and as various modules, the reception control module 3031, the transmission control module 3032, the voice output module 3033, the gyro sensor control module 3034, and the biological sensor control module 3035. Demonstrate the function shown.

The reception control module 3031 controls a process in which the voice output device 30 receives a signal from an external device according to a communication protocol.

The transmission control module 3032 controls a process in which the voice output device 30 transmits a signal to an external device according to a communication protocol.

The audio output module 3033 controls a process of outputting audio data transmitted from the server 10 as audio of video content from the speaker 34.

The gyro sensor control module 3034 controls the operation processing of the gyro sensor constituting the detection unit 302. Further, the gyro sensor control module 3034 transmits a detection signal to the server 10.

The biosensor control module 3035 controls the operation processing of the electrocardiographic sensor constituting the detection unit 302. Further, the biosensor control module 3035 transmits a detection signal to the server 10.

<2 data structure>
FIG. 7 is a diagram showing the data structures of the video content database 181 and the viewer database 184 stored in the server 10.

As shown in FIG. 7, each of the records in the video content database 181 includes an item "facility ID", an item "facility name", an item "content screening information", and the like.

The item "facility ID" is information for identifying each facility in which the satisfaction calculation system 1 is installed. When a plurality of video output devices 20 are installed in one facility and different video contents are screened, the item "facility ID" having a different value is assigned and managed separately.

The item "facility name" is information indicating the name of the facility in which the satisfaction calculation system 1 is installed.

The item "content screening information" is information on the screening schedule of video content such as a movie to be screened at a facility using the satisfaction calculation system 1, and specifically, the item "start date and time" and the item "end time". , And the item "content name" and the like. As shown in FIG. 7, one or a plurality of items "content screening information" are stored in chronological order for one item "facility ID".

The item "start date and time" is information indicating the start date and start time of the video content to be screened at the facility.

The item "end time" is information indicating the end time of the video content to be screened at the facility.

The item "content name" is information indicating the name of the video content to be screened at the facility.

The server 10 updates the video content database 181 in response to receiving input of video content screening information from the facility manager.

Each of the records in the viewer database 184 includes a "viewer ID", an item "viewer name", an item "viewing content", an item "satisfaction information", and the like.

The item "viewer ID" is information that identifies each viewer who has viewed the video content screened at the facility that uses the satisfaction calculation system 1.

The item "viewer name" is information indicating the name of the viewer who has viewed the video content screened at the facility using the satisfaction calculation system 1. If the viewer's name information cannot be obtained at the facility, the item "viewer ID" shown in FIG. 7 does not have to be stored as in the record shown by "# 0202".

The item "viewing content" is information indicating the name of the video content viewed by the viewer, and corresponds to the item "content name" of the video content database 281.

The item "satisfaction information" is information on the satisfaction of the viewer who has viewed the video content screened at the facility using the satisfaction calculation system 1, and specifically, the item "elapsed time" and the item "elapsed time". Includes "satisfaction" and the like. As shown in FIG. 7, one or a plurality of items "satisfaction information" are stored in a time series for each predetermined time (for example, one minute) for one item "viewer ID".

The item "elapsed time" is information indicating the elapsed time for each predetermined time from the start of the video content.

The item "satisfaction" is information indicating the satisfaction of the viewer who has viewed the video content at the time of the item "elapsed time", and is a value of the satisfaction calculated by the satisfaction calculation module 1038.

The satisfaction calculation module 1038 of the server 10 updates the viewer database 184 every time the satisfaction of the viewer who has viewed the video content is calculated.

<3 operation>
Hereinafter, each process of satisfaction calculation by the satisfaction calculation system 1 in the first embodiment will be described with reference to FIGS. 8 and 9.

FIG. 8 is a flowchart showing an example of a flow of performing video content output processing by the satisfaction calculation system 1 of the first embodiment.

In step S111, the video content output control module 1033 of the server 10 refers to the video content database 181 and manages the date and time when the video content is screened. The video content output control module 1033 proceeds to the process of step S112 when it is time to transmit the video content in order to screen it.

In step S112, the video content output control module 1033 of the server 10 transmits the video data of the video content to the video output device 20 and the audio data to the audio output device 30 in order to allow the viewer to view the video content.

In step S122, the video output device 20 receives the video data of the video content transmitted by the server 10. The audio output device 30 receives audio data of video content transmitted by the server 10.

In step S123, the video output device 20 screens the video data of the received video content as a video. The audio output device 30 outputs the audio data of the received video content as audio.

As described above, the satisfaction calculation system 1 transmits the video data of the video content to the video output device 20 and the audio data to the audio output device 30 at a predetermined time. The video output device 20 screens video data as video, and the audio output device 30 outputs audio data as audio. As a result, the video content is screened, and the viewer wears the audio output device 30 for viewing.

FIG. 9 is a flowchart showing an example of a flow of performing satisfaction calculation processing by the satisfaction calculation system 1 of the first embodiment.

In step S211 the gyro sensor control module 3034 of the voice output device 30 controls the operation of the gyro sensors constituting the detection unit 302, and transmits the detection signal of the gyro sensor to the server 10 via the communication unit 301. Further, the biosensor control module 3035 controls the operation of the biosensors constituting the detection unit 302, and transmits the detection signal of the biosensor to the server 10 via the communication unit 301.

In step S231, the camera 40 transmits the captured image data to the server 10.

In step S221, the server 10 receives the detection signal transmitted from the voice output device 30 via the communication unit 101. Further, the server 10 receives the image data transmitted from the camera 40 via the communication unit 101. The server 10 stores the received detection signal in the sensor detection signal database 182 and the received image data in the camera-captured image database 183.

In step S222, the open degree acquisition module 1034 of the server 10 acquires the image data stored in the camera-captured image database 183. The opening degree acquisition module 1034 analyzes the image data, recognizes and detects the eyes of the viewer, and acquires the degree of opening of the eyes in the eyes of the viewer.

In step S223, the blink count acquisition module 1035 of the server 10 acquires the image data stored in the camera-captured image database 183. The blink count acquisition module 1035 analyzes the image data, recognizes and detects the viewer's eyes, and acquires the number of blinks of the viewer.

In step S224, the face movement acquisition module 1036 of the server 10 acquires the detection signal stored in the sensor detection signal database 182. The face movement acquisition module 1036 analyzes the detection signal of the gyro sensor and acquires the movement of the viewer's face from the angular velocity of the detection signal and the time of the movement.

In step S225, the biometric information acquisition module 1037 of the server 10 acquires the detection signal stored in the sensor detection signal database 182. The biological information acquisition module 1037 analyzes the detection signal of the electrocardiographic sensor and acquires the biological information of the viewer (the pulse and body temperature of the viewer).

In step S226, the satisfaction calculation module 1038 of the server 10 is based on the degree of eye opening acquired in step S222, the number of blinks acquired in step S223, the facial movement acquired in step S224, and the biometric information acquired in step S225. Based on this, the degree of satisfaction of the viewer with the video content is calculated.

In step S227, the satisfaction calculation module 1038 of the server 10 stores the calculated satisfaction in the viewer database 184 and updates the database.

As described above, the server 10 of the satisfaction calculation system 1 acquires the degree of eye opening and the number of blinks from the image data of the viewer, and the face movement and biological information from the detection signal detected by the sensor. To get. From this information, the degree of satisfaction with the video content is calculated. This makes it possible to analyze the evaluation of the video content.

<4 screen example>
Hereinafter, a screen example in which the transition of each parameter by the satisfaction calculation system 1 is displayed will be described with reference to FIG.

FIG. 10 is a diagram showing a screen example of the server 10 displaying the transition of each parameter in the satisfaction calculation process. The screen example of FIG. 10 shows a screen example in which the degree of eye opening, the number of blinks, the movement of the face, and the time-series transition of biological information (pulse) are displayed on the input / output IF13 (display) of the server 10. Corresponds to the calculation result in step S226 of FIG.

As shown in FIG. 10, on the input / output IF13 (display) of the server 10, the graph 1031a showing the degree of opening of the viewer's eyes, the number of blinks, the movement of the face, and the time-series transition of biological information (pulse). Is displayed as an example. In this graph 1031a, the degree of eye opening, the number of blinks, the movement of the face, and the pulse at the start of the video content are set to 100, and the rate of change of each parameter with the subsequent passage is calculated and graphed. The screening time of the video content is 2 hours.

Referring to the graph 1031a of FIG. 10, the degree of eye opening and the pulse continue to decrease after the start of the video content, reverse to an increase at 30 minutes after the start, and decrease after peaking at 90 minutes after the start. Then, at the end of the video content, the value is around 100. In addition, the number of blinks and the movement of the face continue to increase after the start of the video content, peak at 30 minutes after the start and reverse to decrease, and then reverse to increase at 90 minutes after the start of the video content. At the end, the value is around 100.

As mentioned above, when the people who are watching the video content are watching intensively, the eye opening tends to increase, the number of blinks decreases, the movement of the face decreases, and the pulse rate tends to increase. Therefore, in the case of the example shown in FIG. 10, it can be seen that the time point 30 minutes after the start is the least concentrated state and the time point 90 minutes after the start is the most concentrated state. Therefore, the coefficient for calculating the degree of satisfaction is set so that the degree of eye opening and the degree of satisfaction are high when the pulse is large (positive coefficient), and when the number of blinks and the movement of the face are small. Set so that the satisfaction level is high (negative coefficient).

<Summary>
As described above, according to the present embodiment, the image data of the viewer is analyzed, and the degree of eye opening and the number of blinks are acquired. Face movement and biological information are acquired from the detection signal detected by the sensor. Based on this information, the degree of satisfaction with the video content is calculated. Therefore, it is possible to acquire information indicating the reaction of the viewer to the video content. This makes it possible to analyze the evaluation of the video content.

In addition, the degree of eye opening, the number of blinks, the movement of the face, and the biological information are acquired at predetermined time intervals, and the satisfaction level is calculated at predetermined time intervals. As a result, it is possible to acquire information indicating the reaction of the viewer in detail for each time of the video content. This makes it possible to analyze the evaluation of video content for each scene, and for example, it becomes possible to easily extract important scenes used for promotion and the like.

Although the embodiments related to the disclosure have been described above, these can be implemented in various other embodiments, and can be implemented by making various omissions, substitutions, and changes. These embodiments and modifications, as well as those omitted, replaced or modified, are included in the technical scope of the claims and the equivalent scope thereof.

<Additional notes>
The matters described in each of the above embodiments will be added below.

(Appendix 1) An information processing device including a control unit, in which the control unit detects the viewer's eyes from image data obtained by capturing a viewer viewing the content and acquires the degree of opening of the viewer's eyes. The degree of opening acquisition unit, the number of blinks acquisition unit that detects the eyes of the viewer from the image data and acquires the number of blinks of the viewer, and the degree of opening and the number of blinks as parameters for the video content of the viewer. An information processing device including a calculation unit for calculating satisfaction.

(Appendix 2) The control unit further includes a motion acquisition unit that acquires the movement of the viewer's face from image data or from a detection signal output from a sensor provided in the audio output device worn by the viewer. The information processing device according to (Appendix 1), wherein the calculation unit calculates the degree of satisfaction of the viewer with the video content using the movement of the face as a parameter.

(Appendix 3) The motion acquisition unit acquires the motion of the viewer's face from the detection signal output from the sensor provided in the audio output device worn by the viewer on his / her ear, according to (Appendix 2). Information processing equipment.

(Appendix 4) The motion acquisition unit acquires a motion in which the viewer's face is tilted or a motion in which the viewer's face is moved from the detection signal output from the gyro sensor provided in the voice output device. , (Appendix 2) or (Appendix 3).

(Appendix 5) The control unit further includes a biometric information acquisition unit that acquires the biometric information of the viewer from the detection signal output from the sensor provided in the audio output device worn by the viewer, and the calculation unit includes a biometric information acquisition unit. The information processing device according to any one of (Appendix 1) to (Appendix 4), which calculates a viewer's satisfaction level with video content using biological information as a parameter.

(Appendix 6) The biometric information acquisition unit acquires the viewer's pulse or blood pressure information as biometric information from the detection signal output from the pulse sensor provided in the audio output device worn by the viewer (Appendix 5). ).

(Appendix 7) The biometric information acquisition unit acquires the viewer's body temperature information as biometric information from the detection signal output from the temperature sensor provided in the audio output device worn by the viewer (Appendix 5) or. The information processing apparatus according to (Appendix 6).

(Appendix 8) The calculation unit determines whether the difference between each value of the parameter and the reference value at a predetermined time, the rate of change from the reference value, the value of change within a predetermined time, or the rate of change within a predetermined time. The information processing apparatus according to any one of (Appendix 1) to (Appendix 7), which calculates satisfaction based on one or more of them.

(Appendix 9) The calculation unit performs an operation by a predetermined function for each value of the parameter, and adds each value after the operation to calculate the satisfaction level, from (Appendix 1) to (Appendix 8). The information processing device according to any one.

(Appendix 10) The calculation unit calculates satisfaction by multiplying each value after the parameter or calculation by a predetermined coefficient and adding each value after the multiplication. Information processing equipment.

(Appendix 11) The calculation unit multiplies each predetermined value after the parameter or calculation by a coefficient of a positive value, and the coefficient of a negative value with respect to each predetermined value after the parameter or calculation. The information processing apparatus according to (Appendix 10), wherein is multiplied by and the respective values after multiplication are added.

(Appendix 12) The information processing device according to any one of (Appendix 1) to (Appendix 11), wherein the calculation unit identifies one or a plurality of viewers, and calculates satisfaction for each viewer.

(Appendix 13) The opening degree acquisition unit calculates the area of the portion where the viewer's eyes are opened and acquires it as the viewer's eye opening degree, according to any one of (Appendix 1) to (Appendix 12). Information processing device.

(Appendix 14) The degree of opening acquisition unit is the contact point between the upper eyelid and the lower eyelid and the upper end point of the upper eyelid, or the contact point between the iris and the upper eyelid, and the lower eyelid Any of (Appendix 1) to (Appendix 12) that calculates the position information of the lower end point or the contact point between the iris and the lower eyelid and acquires it as the degree of opening of the viewer's eyes based on the position information. Information processing device described in the eyelids.

(Appendix 15) The information processing device according to any one of (Appendix 1) to (Appendix 14), wherein the blink count acquisition unit acquires the number of blinks of the viewer in a predetermined time of image data obtained by photographing the viewer.

(Appendix 16) When the viewer's eyes are opened within a predetermined time from the state in which the viewer's eyes are closed, the blink count acquisition unit recognizes the viewer's blink and counts the number of times. , (Appendix 1) to (Appendix 15).

(Appendix 17) A system including a camera and an information processing device. The camera photographs a viewer who views video content, generates image data, and causes the information processing device to acquire the image data. , The control unit includes an opening degree acquisition unit that detects the viewer's eyes from image data obtained by capturing a viewer viewing the video content and acquires the opening degree of the viewer's eyes, and image data. From, a blinking count acquisition unit that detects the viewer's eyes and acquires the number of blinks of the viewer, and a calculation unit that calculates the satisfaction level of the viewer with the video content using the degree of opening and the number of blinks as parameters. , A system.

(Appendix 18) A method for being executed by a computer equipped with a processor, wherein the processor detects the eyes of the viewer from the image data obtained by capturing the viewer who views the video content, and the viewer's eyes are detected. The step of acquiring the degree of eye opening, the step of detecting the viewer's eyes from the image data and acquiring the number of times the viewer blinks, and the degree of opening and the number of times of blinking are used as parameters for the viewer's video content. Steps to calculate satisfaction with, and how to perform.

(Appendix 19) A program for causing a computer equipped with a processor to execute the program. The program detects the viewer's eyes from image data obtained by capturing a viewer who views video content, and the program detects the viewer's eyes. Satisfaction with the video content of the viewer with the steps of acquiring the degree of opening and the step of detecting the viewer's eyes from the image data and acquiring the number of blinks of the viewer, the degree of opening, and the number of blinks as parameters. A program that executes the steps to calculate the degree.

10 server, 20 video output device, 30 audio output device, 40 camera, 80 network, 12 communication IF, 13 input / output IF, 15 memory, 16 storage, 19 processor, 33 sensor unit, 101 communication unit, 102 storage unit, 181 Video content database, 182 sensor detection signal database, 183 camera shot image database, 184 viewer database, 103 control unit, 1033 video content output control module, 1034 opening degree acquisition module (opening degree acquisition unit), 1035 blink count acquisition module ( Blink count acquisition unit), 1036 Face movement acquisition module (movement acquisition unit), 1037 Biometric information acquisition module (biological information acquisition unit), 1038 Satisfaction calculation module (calculation unit)

Claims (15)

An information processing device including a control unit and a storage unit.
The control unit
An opening degree acquisition unit that identifies and detects the eyes of the viewer from time-series image data obtained by capturing one or more viewers who view the video content, and acquires the opening degree of the viewer's eyes.
A blink count acquisition unit that identifies and detects the viewer's eyes from the image data and acquires the number of blinks of the viewer.
A motion acquisition unit that acquires the motion of the viewer's face from the image data or from a time-series detection signal output from a sensor provided in an audio output device worn by the viewer on his / her ear.
A biometric information acquisition unit that acquires biometric information of the viewer from the detection signal,
The opening degree, the number of times before Symbol blinking, the viewer face of the movement, and the biometric information of the viewer as a parameter, and a calculation unit for calculating a satisfaction time series with respect to the video contents of the viewer, With
The audio output device is managed by the manager of the facility for viewing the video content.
Correspondence between the information for grasping the movement of the viewer captured in the image data and identifying the viewer, the detection signal output from the sensor , and the viewer captured in the image data. Attach and attach
The storage unit is associated with the information that identifies the viewer, and for each viewer, the attribute information of the viewer stored in advance, the detection signal output from the sensor , and the viewer's said An information processing device that stores satisfaction. The motion acquisition unit acquires a motion in which the viewer's face is tilted or a motion in which the viewer's face is moved from a detection signal output from a gyro sensor provided in the voice output device. , The information processing apparatus according to claim 1. The biometric information acquiring unit, from the detection signal outputted from the pulse sensor of the viewer is provided to the audio output device to be mounted, to obtain a pulse or blood pressure information of the viewer as the biometric information, according to claim 1 Alternatively, the information processing device according to claim 2. The biometric information acquisition unit obtains information on the body temperature of the viewer as the biometric information from a detection signal output from a temperature sensor provided in the audio output device worn by the viewer, according to claim 1. Item 3. The information processing apparatus according to any one of items 3. The calculation unit is any one of a difference between each value of the parameter and a reference value at a predetermined time, a rate of change from the reference value, a value of change within a predetermined time, and a rate of change within a predetermined time. The information processing apparatus according to any one of claims 1 to 4 , which calculates the satisfaction level based on one or more. The calculation unit calculates the satisfaction level by performing an operation by a predetermined function on each value of the parameter and adding each value after the operation to calculate the satisfaction level. Any one of claims 1 to 5. The information processing device described in the section. The information according to claim 6 , wherein the calculation unit multiplies each of the parameters or each value after calculation by a predetermined coefficient, and adds each value after multiplication to calculate the satisfaction level. Processing equipment. The calculation unit
Multiply each of some of the parameters or each predetermined value after the calculation by a coefficient of positive value.
Multiply a coefficient of a negative value by multiplying each predetermined value of some of the parameters or after calculation, which is different from each predetermined value of some of the parameters or after calculation, and after multiplication. The information processing apparatus according to claim 7 , wherein the respective values are added. The information according to any one of claims 1 to 8 , wherein the opening degree acquisition unit calculates the area of the portion where the viewer's eyes are opened and acquires it as the viewer's eye opening degree. Processing equipment. The degree of opening acquisition unit is the contact point between the upper eyelid and the lower eyelid and the upper end point of the upper eyelid, or the contact point between the iris and the upper eyelid, and the lower part of the lower eyelid in the portion where the viewer's eyes are opened. Any one of claims 1 to 8 that calculates the position information of the end point or the contact point between the iris and the lower eyelid and acquires it as the degree of eye opening of the viewer based on the position information. The information processing device described in the section. The information processing device according to any one of claims 1 to 10 , wherein the blink number acquisition unit acquires the number of blinks of the viewer in a predetermined time of image data obtained by photographing the viewer. When the viewer's eyes are opened within a predetermined time from the state in which the viewer's eyes are closed, the blink count acquisition unit recognizes the viewer's blinks and counts the number of times. , The information processing apparatus according to any one of claims 1 to 11. A system equipped with a camera and an information processing device.
The camera captures a viewer viewing the video content, generates image data, and causes the information processing device to acquire the image data.
The information processing device includes a control unit and a storage unit.
The control unit
An opening degree acquisition unit that identifies and detects the eyes of the viewer from time-series image data obtained by capturing one or more viewers who view the video content, and acquires the opening degree of the viewer's eyes.
A blink count acquisition unit that identifies and detects the viewer's eyes from the image data and acquires the number of blinks of the viewer.
A motion acquisition unit that acquires the motion of the viewer's face from the image data or from a time-series detection signal output from a sensor provided in an audio output device worn by the viewer on his / her ear.
A biometric information acquisition unit that acquires biometric information of the viewer from the detection signal,
The opening degree, the number of times before Symbol blinking, the viewer face of the movement, and the biometric information of the viewer as a parameter, and a calculation unit for calculating a satisfaction time series with respect to the video contents of the viewer, With
The audio output device is managed by the manager of the facility for viewing the video content.
Correspondence between the information for grasping the movement of the viewer captured in the image data and identifying the viewer, the detection signal output from the sensor , and the viewer captured in the image data. Attach and attach
The storage unit is associated with the information that identifies the viewer, and for each viewer, the attribute information of the viewer stored in advance, the detection signal output from the sensor , and the viewer's said A system that remembers satisfaction. A method for being executed by a computer with a processor and memory.
In the method, the processor
A step of identifying and detecting the eyes of the viewer from time-series image data obtained by shooting one or more viewers who view the video content, and acquiring the degree of opening of the viewer's eyes.
A step of identifying and detecting the eyes of the viewer from the image data and acquiring the number of blinks of the viewer.
A step of acquiring the movement of the viewer's face from the image data or from a time-series detection signal output from a sensor provided in an audio output device worn by the viewer in his / her ear.
The step of acquiring the biological information of the viewer from the detection signal,
The opening degree, the number of times before Symbol blinking, the viewer face of the movement, and the biometric information of the viewer as a parameter, calculating a satisfaction time series with respect to the video contents of the viewer, the It ’s a way to do it,
The audio output device is managed by the manager of the facility for viewing the video content.
Correspondence between the information for grasping the movement of the viewer captured in the image data and identifying the viewer, the detection signal output from the sensor , and the viewer captured in the image data. Attach and attach
The memory is associated with information that identifies the viewer, and for each viewer, the attribute information of the viewer stored in advance, the detection signal output from the sensor , and the satisfaction of the viewer. How to remember the degree. A program to be executed by a computer equipped with a processor and memory.
The program is delivered to the processor.
A step of identifying and detecting the eyes of the viewer from time-series image data obtained by shooting one or more viewers who view the video content, and acquiring the degree of opening of the viewer's eyes.
A step of identifying and detecting the eyes of the viewer from the image data and acquiring the number of blinks of the viewer.
A step of acquiring the movement of the viewer's face from the image data or from a time-series detection signal output from a sensor provided in an audio output device worn by the viewer in his / her ear.
The step of acquiring the biological information of the viewer from the detection signal,
The opening degree, the number of times before Symbol blinking, the viewer face of the movement, and the biometric information of the viewer as a parameter, calculating a satisfaction time series with respect to the video contents of the viewer, the A program to be executed
The audio output device is managed by the manager of the facility for viewing the video content.
Correspondence between the information for identifying the viewer by grasping the movement of the viewer captured in the image data, the detection signal output from the sensor , and the viewer captured in the image data. Let me do it
In the memory, the attribute information of the viewer, the detection signal output from the sensor , and the satisfaction of the viewer, which are associated with the information for identifying the viewer and are stored in advance for each viewer. A program that remembers the degree.

Images