JP2005210155A - Mobile viewing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a mobile viewing apparatus whereby an open or real impression of a viewer with respect to a program can easily be obtained without the need for complicated labor and time. <P>SOLUTION: The mobile viewing apparatus for receiving a television signal and outputting video and audio includes: a program identification means for identifying a program during viewing; a state monitor means for monitoring a state of the viewer; and a ranking determining means for ranking the states of the viewer on the basis of a monitoring result for a prescribed time and obtaining the state ranking of the viewer with respect to the program. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a portable viewing device such as a mobile phone with a television function, and more particularly to a device having a function of examining a viewer's reaction.

The TV audience rating survey is conducted by reporting the station currently being viewed from a TV receiver equipped with a dedicated device to the research company.
In some cases, a questionnaire survey on a specific TV program is individually performed on the viewer, and the results are aggregated to check the reputation of the program.
Mobile phones equipped with a function as a television receiver are provided. Currently, only analog broadcast receiving devices are provided, but in the near future, mobile phone devices equipped with a digital broadcast receiving function are also expected to be provided.

A market research system has been proposed in which a questionnaire for market research is distributed from a telecommunications carrier's server to a member's terminal device via the Internet, and responses are received and analyzed, and points are returned to the members who answered ( (See Patent Document 1).
An electrocardiogram sensor that uses a handle cover as an electrode, detects an electrocardiogram based on a potential difference when both hands touch the handle, and transmits the signal from a transmission unit provided on the handle is known (Non-Patent Document 1). ,reference). In this technique, a continuous change in heart rate is obtained from an electrocardiogram, and the degree of sleepiness of the driver while driving a car is estimated.
It is suggested to the AIST patent introduction that sensors for detecting biological signals such as blood flow are embedded in daily life goods and utensils such as tea cups and telephones, and the health status is monitored in a timely manner. (Htto: //www.aist.go.jp/aist_j/research/patent/2002/05_1/).
A technique for estimating a psychological state by analyzing speech is known. For example, as an application, Alpha Psychoanalysis software “Trust Pro” (http://pr.fujitsu.com/jp/news/ 2001/02 / 15-3.html) and “Truster” (http: //www/syatyu.com/shop/143/top.htm) of Trust Tech are known.

Japanese Patent Application Laid-Open No. 2002-140488. PIONEER R & D Vol.13 No.2

In the audience rating survey, it is known whether or not a certain program has been viewed, but it is not known how the viewer felt the certain program. Further, since the survey is performed using a dedicated device, the parameter of the survey is limited, and as a result, there is a problem in the accuracy of the audience rating.
In a questionnaire survey, it is necessary to extract viewers and request responses, which is troublesome. In addition, viewers often ponder at the time of answering, and as a result, not the raw impression but the ideal impression for the audience, in other words, the prepared impression may be answered. There is also a problem with accuracy.
The first object of the present invention is to make it possible to easily obtain a viewer's live impression or real impression regarding a program without troublesome work. Further, a second object is to enable the obtained result to be transmitted to, for example, a broadcaster or a program provider.

The first object of the present invention is achieved by any of the following configurations [1] to [7], and the second object is achieved by the following configurations [5] and [7].
[1] Configuration 1:
A portable viewing device that receives a television signal and outputs video and audio,
Program identification means for identifying the program being viewed;
Status monitoring means for monitoring the status of the viewer;
Rank determining means for ranking the state based on the monitoring result for a predetermined time and setting the state rank of the viewer for the program;
A portable viewing device comprising:
In the case of digital broadcasting, the program can be identified based on PSI (Program Specific Information) in the header information of the transport stream (TS). In the case of analog broadcasting, a program guide is held on the device side and can be obtained from the current time and the channel selection state.
As means for monitoring the state, for example, sensors described in the following configurations 2, 3, and 6 can be cited.
The order of the state is, for example, the degree of watching the program (gaze degree), the degree of feeling (anger, joy, sadness, etc.) felt about the program, and the like.

[2] Configuration 2:
In configuration 1,
The state monitoring means is a camera provided so as to be able to photograph the front of the screen, and a visual discrimination means for determining whether or not the viewer is looking at the screen based on a captured image of the camera,
The rank determining means ranks based on a ratio of a time during which the viewer is watching the screen in the predetermined time.
A portable viewing device.
[3] Configuration 3:
In configuration 1,
The state monitoring means is a biological sensor that detects the biological state of the viewer,
It further includes representative value holding means for periodically obtaining the detected value of the biometric sensor when using the portable viewing device and obtaining and holding the representative value.
The rank determining means ranks the viewer based on a deviation from the representative value by comparing the biological state of the viewer with the representative value.
A portable viewing device.
[4] Configuration 4:
In configuration 3,
The biological sensor includes a pressure sensor that detects a force gripping the casing of the portable viewing device, a sweating sensor that detects a sweating amount of a hand gripping the casing of the portable viewing device, and a casing of the portable viewing device. At least one sensor selected from a body temperature sensor that detects the body temperature of the hand holding the
A portable viewing device.
[5] Configuration 5:
In any one of configurations 1 to 4,
Mobile phone function,
A rank transmitting means for transmitting the state rank determined by the rank determining means to a predetermined destination in association with the state type and the program;
The portable viewing device further comprising:

[6] Configuration 6:
In configuration 1,
It further has a mobile phone function,
The state monitoring means is a microphone of a mobile phone,
The rank determining means ranks predetermined emotions based on analysis results of viewers' voices input from the microphone.
A portable viewing device.
[7] Configuration 7:
In configuration 6,
A rank transmission unit that transmits the state rank determined by the rank determination unit to a predetermined destination in association with the type of the state and the program;
A portable viewing device.

The portable television viewing device of Configuration 1 ranks a state based on a monitoring result of a predetermined time, a program identifying unit that identifies a program being viewed, a state monitoring unit that monitors the state of the viewer, and the program Therefore, it is possible to easily obtain knowledge about the viewer's state resulting from viewing the program without any troublesome effort.
In the portable television viewing device of configuration 2, the state monitoring unit of configuration 1 determines whether the viewer is watching the screen based on the camera provided so that the front of the screen can be photographed and the captured image of the camera And the ranking determining means ranks based on the proportion of the time during which the viewer is watching the screen in the predetermined time, so that the time when the viewer is actually watching the program is shown. You can get a percentage.
The portable television viewing device of configuration 3 is a biological sensor in which the state monitoring unit of configuration 1 detects a viewer's biological state, and periodically acquires a detection value of the biological sensor when the portable viewing device is used. A representative value holding means for obtaining and holding a representative value, and the rank determining means compares the viewer's biological state with the representative value and ranks based on the deviation from the representative value. Knowledge about the viewer's biological reaction resulting from viewing can be obtained with high accuracy.
The portable television viewing device of configuration 4 can give specific examples of a pressure sensor, a sweat sensor, and a body temperature sensor as the biological sensor of configuration 3.
The portable television viewing device of configuration 5 has a predetermined function in any one of configurations 1 to 4 in which the function as a mobile phone and the status ranking determined by the ranking determination means are associated with the status type and the program. Therefore, the obtained result can be transmitted to, for example, a broadcaster or a program provider.
The portable TV viewing device of configuration 6 further comprises a mobile phone function in configuration 1, uses a microphone of the mobile phone as state monitoring means, and the order determination means analyzes the voice of the viewer input from the microphone. Since ranking of predetermined emotions is performed based on the result, knowledge about viewer emotions resulting from program viewing can be obtained using a microphone originally provided in the mobile phone.
The portable television viewing device of configuration 7 further includes rank transmission means for transmitting the state rank determined by the rank determination means in association with the state type and the program to a predetermined destination in configuration 6. The obtained result (knowledge about viewers' emotions resulting from viewing the program) can be transmitted to, for example, a broadcaster or a program provider.
Broadcasters can create programs that viewers want to watch based on the raw data collected from viewers, and as a result, an increase in notification revenue can be expected. In addition, companies that provide programs can be expected to improve brand value.

Hereinafter, the present invention will be specifically described with reference to the drawings.
FIG. 1 is a block diagram illustrating an apparatus embodying the present invention as a mobile phone having a television function for receiving digital broadcasts and reproducing video and audio.

Mobile phone:
The components of the cellular phone are an antenna 59, a telephone transmission / reception circuit 21, a channel codec 22, an audio codec 23, a microphone 51, a speaker 52, a camera 53, an input device 54, a display device 55, and the control device 10. Although the camera 53 is not exactly a component of the mobile phone, it is now included in the mobile phone as it is now standardly installed in the mobile phone. The microphone 51 and the camera 53 are also used as devices for detecting the viewer's state (program watching state, emotion) during program viewing.

The telephone radio wave received by the antenna 59 is demodulated by the telephone transmission / reception circuit 21, decomposed by the channel codec 22, a necessary channel audio signal is taken out, converted into an analog signal by the audio codec 23, and the speaker. 52 is output as sound.
The voice input from the microphone 51 is digitized by the voice codec 23, converted into a TDMA (or CDMA) signal by the channel codec 22 together with the control signal from the control device 10, and modulated by the telephone transmission / reception circuit 21. And transmitted from the antenna 59.

When an incoming call is detected, the control device 10 outputs a notification sound from the speaker 52. Note that a vibrator (not shown) may be vibrated (in the manner mode).
The input device 54 is used for various inputs necessary for a cellular phone, such as input of a telephone number of a calling party, mode setting or switching (switching between a telephone mode and a TV viewing mode, etc.), mail input, and the like.
The display device 55 is used for various displays necessary as a cellular phone, such as display of the telephone number of the incoming / calling party, mode display, mail display, and the like.

TV viewing device:
The components of the television viewing device are an antenna 59, a television receiving circuit 31, a descrambler 32, a demultiplexer 33, a decoder (eg, MPEG decoder) 34, an audio processing circuit 35, a speaker 52, a video processing circuit 36, and a display device 55. , Input device 54, and control device 10. In addition to the microphone 51 and the camera 53 described above, a pressure sensor / sweat sensor / body temperature sensor 57 is provided as a member for detecting a biological reaction that is one of the states of the viewer when watching the program. .

  The TV radio wave received by the antenna is selected, demodulated, and error-corrected by the TV receiving circuit 31, descrambled by the descrambler 32, and a transport stream (TS) is taken out. A packet of the target program is extracted from the transport stream (TS) by the demultiplexer 33, and identification information for specifying the target program is extracted. Furthermore, service information (SI) is acquired and the clock is reproduced. The extracted packets are decoded into audio data and video data by the decoder 34, converted into analog audio signals by the audio processing circuit 35, output as audio from the speaker 52, and displayed by the video processing circuit 36. It is converted into an analog video signal in a scanning format suitable for the device 55 and output from the display device 55 as an image.

Viewer response acquisition procedure:
FIG. 2 is a flowchart showing a procedure for acquiring the reaction of the program viewer.
When the update timing comes (YES in S01), the detection signal of the sensor (pressure sensor / sweat sensor / body temperature sensor) at the update timing is fetched and stored in a predetermined area for storing detected values in a memory (not shown). . Further, an average value (or other representative values such as a mode value and a median value) with a past detection value is obtained, and the detected living body state (the pressure of the hand gripping the casing of the mobile phone / portability) As a representative value of the sweating amount of the hand holding the telephone casing / the body temperature of the hand holding the mobile telephone casing), the representative value is stored in a predetermined area for storing a representative value in a memory (not shown) (S03). The update timing may be common among the pressure sensor, the sweat sensor, and the body temperature sensor, or may be different for each sensor. For example, it is assumed that it is set appropriately every 10 seconds, every 1 minute, every 5 minutes, and the like. In addition, the past detection value can be the latest past predetermined period (eg, about half a day to 10 days) or a predetermined number of detection values. Here, since the past detected value of the viewer for a relatively short period is incorporated into the representative value calculation, a value suitable for the viewer can be used for comparison with the monitoring result (see S17). . Further, even when fever is caused by a disease or the like, since the fever is incorporated into the representative value itself, the influence of the fever can be removed and a highly accurate comparison can be performed.

When the program is started (YES in S11), state monitoring is started (S13).
That is, the detection signals of the pressure sensor / sweat sensor / body temperature sensor are continuously captured, and each detection value is stored in a predetermined area of the memory. “Continuous” means that it is sufficient to periodically capture at relatively short intervals during a program, and it is not always necessary to continuously capture. For example, it may be taken in every 5 seconds. Of course, you may capture continuously.
Further, based on the image taken by the camera 53, it is determined whether or not the viewer is looking at the screen, and the viewing time and the non-viewing time are accumulated and stored in a predetermined area of the memory. To do. A method for determining whether or not the viewer is viewing the screen will be described later.
Further, the voice input from the microphone 51 is analyzed, and the strength of the viewer's emotion (surprise, joy, sadness, anger, etc.) is stored for each type in a predetermined area of the memory. Here, for example, points of the emotion may be added according to the strength of the emotion, and the point may be stored. As a method for estimating the emotional strength of the viewer for each emotion type based on voice analysis, the above-described known method can be used. Note that whether the sound is generated by the viewer or ambient noise can be separated by storing the voice of the viewer during a call and comparing the voice prints.

The state monitoring (S13) is performed from the start to the end of the program. For example, in the case of digital broadcasting, the start and end timing of a program can be acquired from PSI (Program Specific Information) in header information of a transport stream (TS).
In this example, the monitoring is performed from the start to the end of the program. Instead, for example, the program is divided to some extent, for example, 10 minutes at the beginning of the program, 10 minutes in the middle, 10 minutes immediately before the end, etc. May be. Further, monitoring may be performed except for the commercial section, and the commercial section may be monitored separately from the program.

When the program ends (YES in S15), the rank (rank) of each state (pressure / sweat / body temperature / actual viewing / emotion) is determined (S17).
That is, the pressure, sweating, and body temperature are ranked according to the deviation from the representative value held in the memory (eg, deviation value of normal distribution). It can be estimated that the greater the deviation (= the higher the rank), the greater the degree of excitement of the viewer.
Also, ranking is performed by obtaining a ratio of the time that the viewer is actually watching the screen to the program time (the time that the viewer is watching + the time that the viewer is not watching). It can be estimated that the higher the ratio of the actual viewing time (= the higher the rank), the more enthusiastically watching the program.
In addition, ranking for each emotion type is performed based on the points for each emotion type obtained based on the voice of the microphone input. For example, if the point of surprise is large, it can be estimated that the viewer has enjoyed a big surprise by watching the program. Further, if the program is a tragedy and the sadness point is large, it can be estimated that the viewer is enthusiastic about the program. The same applies to other emotions.

The rank thus determined is transmitted to a predetermined destination (S19). For example, it is transmitted to a broadcaster, a program provider (commercial provider), and the like. For this transmission, the data transmission function of the mobile phone is used. In addition, instead of the order of each state information (pressure, sweating, body temperature, gaze time, emotion) or together with the order of each state information, the overall order calculated from the order of each state information is transmitted. May be.
In addition, since the time ratio actually observed does not depend on the difference between viewers (difference in comparison data) unlike the body temperature or the like, the raw data may be transmitted as it is.
On the broadcaster or program provider side, the received data can be aggregated to evaluate the reputation of the program.

How to determine whether viewers are looking at the screen:
FIG. 3 is a flowchart showing a procedure for determining whether or not a person is looking at the mobile phone of FIG. 1, and FIG. 4 is a flowchart showing an example of the process of step S111 of FIG.

First, in response to an instruction from the display state control or the recording / accelerated playback control, a photographed image of the camera 53 is captured (S101).
Next, a face area is extracted from the captured image area (S103). A known pattern matching can be used as a method for extracting a face region from an image region.
Next, left and right eye areas are extracted from the extracted face area (S105). As a method for extracting the eye region from the face region, a known pattern matching can be used.
Next, a black eye (pupil) region is extracted from each eye region (S107). As a method for extracting the black eye region from the eye region, known pattern matching can be used.
In step S109, the orientation of the face is determined based on the face area and the eye area. Various methods are known as a method for determining the orientation of the face. For example, a method of recognizing the face orientation using affine parameters can be used.

  In step S111, based on the orientation of the face, the position of the face in the image area, and the position of the black eye in the eye area, it is determined whether or not a person is looking at the screen.

An example of the processing in step S111 will be described with reference to FIGS.
If the face is not facing the display panel (NO in S161), the position of the face area is checked (S163).
If the face area is on the left side in the image area and the black eye area is in the right position in the eye area (YES in S163 and YES in S165), this is the case illustrated in the left column of FIG. It is likely that you are looking at the display panel. Therefore, the value of P is increased by 1 (S167).
If the face area is on the left side in the image area and the black eye area is not in the right position in the eye area (YES in S163 and NO in S165), this is the case illustrated in the left column of FIG. Most likely you have not seen the panel. Therefore, the value of P is decreased by 1 (S169).

When the face area is not on the left side in the image area and the black eye area is in the left position in the eye area (NO in S163 and YES in S175), this is the case illustrated in the right column of FIG. It is likely that you are looking at the display panel. Therefore, the value of P is increased by 1 (S177).
If the face area is not to the left in the image area and the black eye area is not in the left position in the eye area (NO in S163 and NO in S175), this is the case illustrated in the right column of FIG. Most likely you have not seen the panel. Therefore, the value of P is decreased by 1 (S179).

If the face is facing the display panel (YES in S161), the position of the black eye area in the eye area is checked (S185).
If the face is facing the display panel, the black eye area is positioned at the approximate center of the eye area regardless of the position of the face area in the image area (YES in S185). As illustrated in FIG. 7, there is a high possibility that a person is looking at the display panel. Therefore, the value of P is increased by 1 (S187).
On the other hand, if the face is facing the display panel and the black eye area is not at the approximate center position in the eye area (NO in S185), regardless of the position of the face area in the image area, As illustrated in FIG. 8, there is a high possibility that a person is not looking at the display panel. Therefore, the value of P is decreased by 1 (S189).

  Next, the value of P is checked (S191). If P is positive (YES in S191), it is determined that a person is looking at the display panel (S193). If P is not positive (NO in S191). It is determined that the person is not looking at the display panel (S195).

  Note that the above processing is a simplified example of the discrimination method, and a more accurate discrimination method may be used. For example, the position of the face in the image area and the position of the black eye in the eye area are not set as two positions, right and left, but are set as right / center / left, further up / down, etc. You may make it perform.

  Further, when the camera installation position is deviated from the position of the display panel, that is, in the case illustrated in FIG. 9, the line of sight is shifted between when the person looks at the camera and when the person looks at the display panel. For this reason, for example, when the display panel is viewed with the face facing the display panel, the position of the black eye in the eye area shifts to the left (or right) instead of the center in the image captured by the camera. Therefore, it is necessary to incorporate such a deviation in the above determination.

  Further, in a portable display device, viewing is often performed with the face close to the display panel. In other words, when the face is away from the display panel, there is a high possibility that the display panel is not seen as illustrated in FIG. This may be incorporated into the above determination. That is, the above determination may be corrected according to the ratio of the face area in the image area captured by the camera.

The block diagram which illustrates the apparatus which actualized this invention as a mobile telephone provided with the television function which receives digital broadcasting and reproduces | regenerates an image | video and an audio | voice. The flowchart which shows the procedure which acquires a program viewer's reaction. The flowchart which shows the procedure which discriminate | determines whether the person is watching in the mobile telephone of FIG. The flowchart which shows an example of the state discrimination | determination process (S111) of FIG. Explanatory drawing which illustrates the case where there is a high possibility of viewing the screen when the orientation of the display panel does not match the orientation of the face. Explanatory drawing which illustrates the case where there is a high possibility that the screen is not viewed when the orientation of the display panel does not match the orientation of the face. Explanatory drawing illustrating the case where there is a high possibility of looking at the screen when the orientation of the display panel matches the orientation of the face. Explanatory drawing illustrating the case where there is a high possibility that the screen is not viewed when the orientation of the display panel matches the orientation of the face. Explanatory drawing which illustrates the case where it is necessary to consider in relation to the mounting position of the camera when determining whether or not the screen is being viewed. Explanatory drawing which illustrates the case where it is necessary to consider in relation to the distance between a camera and a person when determining whether or not the screen is being viewed.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Control apparatus 21 Telephone transmission / reception circuit 22 Channel codec 23 Audio codec 31 Television receiving circuit 32 Descrambler 33 Demultiplexer 34 Decoder 35 Audio processing circuit 36 Video processing circuit 51 Microphone 52 Speaker 53 Camera 54 Input device 55 Display device 57 Pressure sensor / Sweating sensor / Body temperature sensor 59 Antenna

Claims (7)

A portable viewing device that receives a television signal and outputs video and audio,
Program identification means for identifying the program being viewed;
Status monitoring means for monitoring the status of the viewer;
Rank determining means for ranking the state based on the monitoring result for a predetermined time and setting the state rank of the viewer for the program;
A portable viewing device comprising: In claim 1,
The state monitoring means is a camera provided so as to be able to photograph the front of the screen, and a visual discrimination means for determining whether or not the viewer is looking at the screen based on a captured image of the camera,
The rank determining means ranks based on a ratio of a time during which the viewer is watching the screen in the predetermined time.
A portable viewing device. In claim 1,
The state monitoring means is a biological sensor that detects the biological state of the viewer,
It further includes representative value holding means for periodically obtaining the detected value of the biometric sensor when using the portable viewing device and obtaining and holding the representative value.
The rank determining means ranks the viewer based on a deviation from the representative value by comparing the biological state of the viewer with the representative value.
A portable viewing device. In claim 3,
The biological sensor includes a pressure sensor that detects a force gripping the casing of the portable viewing device, a sweating sensor that detects a sweating amount of a hand gripping the casing of the portable viewing device, and a casing of the portable viewing device. At least one sensor selected from a body temperature sensor that detects the body temperature of the hand holding the
A portable viewing device. In any one of Claims 1-4,
Mobile phone function,
A rank transmitting means for transmitting the state rank determined by the rank determining means to a predetermined destination in association with the state type and the program;
The portable viewing device further comprising: In claim 1,
It further has a mobile phone function,
The state monitoring means is a microphone of a mobile phone,
The rank determining means ranks predetermined emotions based on analysis results of viewers' voices input from the microphone.
A portable viewing device. In claim 6,
A rank transmission unit that transmits the state rank determined by the rank determination unit to a predetermined destination in association with the type of the state and the program;
A portable viewing device.

Images