JP2011259232A - Display device, control method therefor, program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To carry out operation required by a random user according to a facial expression of the user when a plurality of users view a content.SOLUTION: A display device 10 includes: an image acquisition unit 14 which acquires an image of the user obtained by photographing the user existing within a predetermined range by a camera 103; a detection signal reception unit 13 which receives, from a human-sensing sensor 102 for detecting whether other user exists outside the predetermined range, a detection signal indicating the presence of the other user; a facial expression detection unit 23 which detects a facial expression of a random user from the image acquired by the image acquisition unit 14; and a content processing unit 22 which, while video is displayed, acquires setting information corresponding to the facial expression of the random user detected by the facial expression detection unit 23 from correspondence information between facial expression information indicating a plurality of different facial expressions and setting information for carrying out a predetermined operation, and carries out the predetermined operation on the basis of the acquired setting information.

Description

  The present invention relates to a display device, a control method therefor, a program, and a recording medium. Specifically, when there are a plurality of users around the device, the display device performs a predetermined operation according to the user's facial expression, and The present invention relates to a control method, a program, and a recording medium.

  2. Description of the Related Art There is known a technique for photographing a user's face in a display device such as a television and performing various processes based on the user's state evaluated from the photographed face image.

  For example, Patent Document 1 describes that facial image information is extracted from a user's facial image captured by a camera, and the degree of attention and facial expression of the user when a predetermined content is reproduced are evaluated and evaluated. Yes.

  Japanese Patent Application Laid-Open No. 2004-228561 detects the number of blinks from a user's face image photographed by a CCD (Charge Coupled Devices) camera, estimates the user's arousal level, and controls the hue of the display image according to the arousal level. It is described to do.

JP 2009-267445 A (published on November 12, 2009) JP 2007-3618 A (published January 11, 2007)

  By the way, when a plurality of users are viewing content such as a TV program, it is not sufficient to perform an operation according to the photographed facial expression of any user. For example, it is more entertaining than when viewing alone. For example, when there are a plurality of users, an operation different from when one user is required may be required.

  On the other hand, in Patent Documents 1 and 2, the content that matches the user's preference is guided based on the photographed user's face image or the hue of the display image is controlled. The operation according to whether it is is not performed.

  In general, many cameras used for display devices such as televisions have a viewing angle of less than 90 °. Therefore, when a plurality of users are viewing, a user other than a user who is located within the range of the viewing angle of the camera cannot take a picture. Therefore, it is impossible to determine whether the user is alone or plural with the camera alone, and the operation cannot be controlled accordingly. In addition, a wide-angle camera with a wider viewing angle is expensive and expensive.

  The present invention has been made in view of the above problems, and an object of the present invention is to execute an operation requested by a user according to an arbitrary user's facial expression when there are a plurality of users viewing the content. It is to provide a display device.

  In order to solve the above-described problems, a display device according to the present invention is a display device that includes a display unit that acquires content and displays video included in the acquired content. From an image acquisition unit that acquires an image of the user captured by the imaging unit that captures the image, and a detection unit that detects whether there is another user around the device and outside the predetermined range A receiving unit that receives a detection signal indicating the presence of the other user, a detecting unit that detects a facial expression of an arbitrary user from the image acquired by the image acquiring unit, and a plurality of displays during the display of the video The correspondence corresponding to the facial expression of any user detected by the detection means from the correspondence information of the recording unit that records in advance the correspondence information between the facial expression information indicating different facial expressions and the setting information for executing a predetermined operation Setting And an execution means for executing a predetermined operation based on the acquired setting information, the image acquisition unit determines whether the user is one or more from the acquired image, The execution means executes the predetermined operation when the receiving unit receives the detection signal and / or when the image acquisition unit determines that there are a plurality of users. It is a feature.

  In order to solve the above-described problem, the control method according to the present invention captures a display unit that acquires content, displays a video included in the acquired content, and a capturing unit that captures a user within a predetermined range. The other user is present from an image obtaining unit that obtains the image of the user and a detection unit that detects whether there is another user around the device and outside the predetermined range. A determination method for determining whether the user is one or more from the image acquired by the image acquisition unit, comprising: a receiving unit that receives a detection signal indicating that A facial expression detection step for detecting a facial expression of an arbitrary user from the image after the determination step, and facial expression information indicating a plurality of different facial expressions during the display of the video after the facial expression detection step; A setting information acquisition step for acquiring the setting information corresponding to the detected facial expression of an arbitrary user from the correspondence information of the recording unit that previously records the correspondence information with the setting information for executing a predetermined operation; and the setting An operation execution step for executing a predetermined operation based on the setting information acquired in the information acquisition step. In the operation execution step, when the reception unit receives the detection signal, and in the determination step, the user The predetermined operation is performed when at least one of the determinations is made.

  According to the above configuration and method, in the display device, when the detection signal is received from the detection unit and when it is determined that there are a plurality of users photographed by the photographing unit, the content is displayed at least one of the times. It is determined that there are a plurality of viewing users.

  Further, in the display device, when the user's facial expression is detected from the image captured by the photographing unit and it is determined that there are a plurality of users viewing the content, the setting information is obtained from the correspondence information according to the detected facial expression. Acquire and execute a predetermined operation based on the acquired setting information.

  For example, when a plurality of users are viewing content, when there are a plurality of users, a different operation may be required when there are a plurality of users.

  Therefore, according to the present invention, when there are a plurality of users viewing the content, it is possible to execute an operation requested by the user according to the facial expression of an arbitrary user.

  In the display device according to the present invention, it is preferable that the execution unit creates a composite video by superimposing the image acquired by the image acquisition unit on the video, and displays the generated composite video on the display unit. .

  According to the above configuration, when displaying exciting content such as a sports program, for example, the user's face (for example, a facial expression at the time of excitement) photographed by the photographing unit is superimposed on the video displayed on the display unit and displayed on the child screen indicate. Thereby, the content can be interesting.

  In the display device according to the present invention, the execution means creates display information for prompting the user to instruct whether or not to display the past video displayed by the display unit again. It is preferable to display information on the display unit.

  For example, even when a user who is not a detection target misses a scene that the user who is a facial expression detection target feels fun, display information that prompts an instruction whether or not to display the scene again is displayed. It is possible to share enjoyment among a plurality of users.

  In the display device according to the present invention, the content includes a sound and includes a sound output unit that outputs the sound, and the execution unit detects a cheer sound from the sound and adjusts a volume of the detected cheer sound. Then, it is preferable to output to the audio output unit.

  According to the above configuration, for example, when a cheering sound is detected from the sound included in the content such as a sports program, an effect of increasing the volume of the detected cheering sound is provided. Thereby, the entertainment property of content can be improved.

  In the display device according to the present invention, the recording unit records a sound effect in association with the correspondence information, and the execution unit acquires the sound effect from the recording unit, and the acquired sound effect is stored in the recording unit. It is preferable to output to the audio output unit.

  According to said structure, in content, such as a variety program, for example, by adding a sound effect and displaying a content, it can be interesting by content and a pleasure can be shared among several users.

  The display device according to the present invention further includes genre information acquisition means for acquiring genre information indicating the genre of the content, the recording unit records the genre information in association with the correspondence information, and the execution means includes Preferably, the setting information corresponding to the genre information acquired by the genre information acquisition means is acquired from the correspondence information of the recording unit, and the predetermined operation is executed based on the acquired setting information.

  Thereby, the effect suitable for the genre of content can be given.

  In the display device according to the present invention, it is preferable that the detection unit divides the periphery of the device itself into a plurality of areas and detects any of the plurality of areas at predetermined time intervals.

  According to said structure, the number of the users who exist around the own apparatus can be grasped | ascertained.

  In the display device according to the present invention, it is preferable that the region detected by the detection unit does not include the predetermined range.

  According to the above configuration, the detection unit misidentifies a user who is within the range of the viewing angle of the imaging unit as another user without overlapping the area detected by the detection unit and the area captured by the imaging unit. There is nothing.

  The display device may be realized by a computer. In this case, a program for causing a computer to operate as each of the above-described means and a computer-readable recording medium recording the program also fall within the scope of the present invention.

  The present invention is a display device that includes a display unit that acquires content and displays a video included in the acquired content, and the image of the user captured by a capturing unit that captures a user within a predetermined range Detection indicating that the other user exists from an image acquisition unit that acquires the image and a detection unit that detects whether or not the other user exists outside the predetermined range. A receiving unit that receives a signal; a detecting unit that detects a facial expression of an arbitrary user from the image acquired by the image acquiring unit; facial expression information indicating a plurality of different facial expressions during display of the video; and a predetermined operation The setting information corresponding to the facial expression of an arbitrary user detected by the detecting means is acquired from the correspondence information of the recording unit that previously records the correspondence information with the setting information for executing the setting information, and based on the acquired setting information The Execution means for executing a predetermined operation, and the image acquisition unit determines whether the user is one person or a plurality of users from the acquired image, and the reception means is configured to detect the detection signal. When there is a plurality of users who are viewing content in order to execute the predetermined operation at least one of when the image acquisition unit determines that there are a plurality of users. The operation desired by the user can be executed in accordance with the facial expression of an arbitrary user.

It is a block diagram which shows the principal part structure of the display apparatus which concerns on one Embodiment of this invention. It is a figure which shows the detection angle of the viewing angle of a camera mounted in the display apparatus shown in FIG. 1, and a human sensitive sensor. It is a figure which shows a mode when the detection area of the human sensitive sensor mounted in the display apparatus shown in FIG. 1 is divided | segmented into several area | regions. 4 is a table showing correspondence information used for content processing in the display device shown in FIG. 1. It is a sequence diagram which shows an example of the flow of the content process in the display apparatus shown in FIG. It is a sequence diagram which shows another example of the flow of the content process in the display apparatus shown in FIG.

  An embodiment of a display device according to the present invention will be described below with reference to FIGS. In the following description, various preferred limitations for implementing the present invention are given, but the technical scope of the present invention is not limited to the description of the following embodiments and drawings.

[1. Configuration of display device]
FIG. 1 is a block diagram showing a configuration of a display device 10 according to an embodiment of the present invention. As shown in FIG. 1, the display device 10 includes a tuner 11, a remote control light receiving unit 12, a detection signal receiving unit 13 (receiving unit), an image acquiring unit 14, a video output unit 15, an audio output unit 16, a recording unit 17, and a display. Unit 18, speaker 19, and control unit 20. The control unit 20 includes a genre information acquisition unit 21 (genre information acquisition unit), a content processing unit 22 (execution unit), and a facial expression detection unit 23 (detection unit). The content processing unit 22 includes a video processing unit 24 and audio. A processing unit 25 is provided. Further, the display device 10 is connected to a human sensor 102 (detection unit) and a camera 103 (imaging unit).

  The display device 10 determines the user's facial expression while viewing the content by detecting the user's facial expression captured by the camera 103, and views the content depending on whether a detection signal is received from the human sensor 102. It is determined whether there are one or more users. Here, “plurality” means that there are two or more people.

  The camera 103 photographs a user who exists within a predetermined range in order to detect the facial expression of an arbitrary user who is viewing the content. In this specification, the “predetermined range” refers to the range of the viewing angle of the camera 103, and the numerical range may vary depending on the type or performance of the camera 103. This predetermined range may be, for example, around the front surface of the display unit 18.

  A specific example of the camera 103 is a web camera, for example. The web camera refers to a camera having an interface such as USB (Universal Serial Bus) or IEEE1394, or a camera having a network interface, for example. The camera 103 is not particularly limited as long as video data is output to the display device 10, and other cameras such as a camera or a digital video camera built in a mobile phone having a video output interface may be used. Can be used.

  The image captured by the camera 103 may be a still image or a moving image. The captured images can be stored in real time or at regular intervals. The image storage destination may be, for example, the camera 103 provided with a memory, or the recording unit 17 of the display device 103.

  The human sensor 102 detects whether or not another user exists around the display device 10 and outside a predetermined range (that is, the viewing angle range of the camera 103). Further, the periphery of the display device 10 is a range in which the user can see the video displayed on the display unit 18.

  Hereinafter, for convenience of explanation, a user who is photographed by the camera 103 is referred to as a “main user”, and another user who is not photographed by the camera 103 is referred to as a “sub user”. Further, when referring to either the main user or the sub-user, it is simply referred to as “user”.

  Specific examples of the human sensor 102 include a pyroelectric infrared sensor that detects a change in a heat source in a detection area. In this case, when an object having a temperature difference from the surrounding temperature moves within the detection area, the temperature change is detected, and the detected change is transmitted to the detection signal receiving unit 13 as a detection signal.

  That is, the human sensor 102 detects a human movement within the detection area (that is, around the device itself and outside the predetermined range), and the detection signal receiving unit 13 receives the detection signal. Thus, the display device 10 determines that there is a sub user.

  Specific examples of the human sensor 102 are not limited to pyroelectric infrared sensors. For example, it is possible to detect a change in pressure of a camera capable of shooting at 360 °, a camera installed on a ceiling, or a floor. Sensor. When the human sensor 102 detects a change in the detection area, it sends it to the detection signal receiving unit 13 as a detection signal.

  In addition, it is preferable that the human sensor 102 divides the periphery of the display device 10 into a plurality of regions and detects any of the plurality of regions every predetermined time. Thereby, the number of sub-users existing in the detection area can be grasped by detecting the presence or absence of a change in each divided area.

  At this time, it is more preferable that the detection area of the human sensor 102 does not include an area captured by the camera 103. As a result, the human sensor 102 may mistakenly recognize the main user who is within the shooting range of the camera 103 as a sub user without overlapping the area detected by the human sensor 102 and the area captured by the camera 103. Absent.

  In the present embodiment, the human sensor 102 and the camera 103 are externally attached. However, the present invention is not limited to this configuration, and any one or both of them may be built in the display device 10. Good.

  The tuner 11 receives a broadcast signal including content via the antenna 100. The antenna 100 may be various antennas such as an analog broadcast receiving antenna, a digital broadcast receiving antenna, or an IP broadcast receiving antenna. The broadcast signal received by the tuner 11 is converted into a video signal and an audio signal through various processes such as a selector, separation and decoding, and the video signal is sent to the video processing unit 24 of the content processing unit 22, and the audio signal is It is sent to the audio processing unit 25 of the content processing unit 22.

  In the present embodiment, broadcast content is acquired. However, the present invention is not limited to this. For example, content acquired from an external device connected to the display device 10 such as a DVD, BD, or HDD may be used. It may also be content acquired via the Internet.

  The remote control light receiving unit 12 receives a remote control signal transmitted from the remote control 101 by a user operation using light such as infrared rays. The remote control signal includes various user instruction information for specifying the operation of the display device 10 such as an instruction for adjusting the volume output from the speaker 19 or a channel switching instruction for a television. The instruction information can also be received by the user operating an input mechanism such as a button provided on the main body of the display device 10, for example. The remote control light receiving unit 12 notifies the control unit 20 of an instruction from the user included in the remote control signal.

  The detection signal receiving unit 13 receives the detection signal transmitted from the human sensor 102 and notifies the content processing unit 22 that a sub-user exists in the detection area of the human sensor 102. That is, it is notified that a plurality of users are viewing the content. The interface between the detection signal receiving unit 13 and the human sensor 102 is, for example, a USB or the like, and may be detected directly by an IC having an A / D converter, or the line is H / L with a certain threshold value. It may be changed to

  The image acquisition unit 14 acquires an image of the main user taken by the camera 103. The interface between the image acquisition unit 14 and the camera 103 may be, for example, USB or IEEE 1394, or may be a network interface.

  Incidentally, there may be a plurality of users within the range of the viewing angle of the camera 103. Therefore, the image acquisition unit 14 determines from the acquired image whether the number of photographed users is one or more, and notifies the content processing unit 22 of the determination result.

  The control unit 20 comprehensively controls various configurations of the display device 10. The function of the control unit 20 is realized by a CPU (Central Processing Unit) executing a program recorded in a recording device such as a RAM (Random Access Memory) or a flash memory. The genre information acquisition unit 21, the content processing unit 22, and the facial expression detection unit 23 included in the control unit 20 will be described below.

  The genre information acquisition unit 21 acquires genre information indicating the genre of content included in the broadcast signal received by the tuner 11. The genre is classified into, for example, “sports”, “movie”, “news”, or “drama”. Further, these genre information may be recorded in advance in association with the correspondence information in the recording unit 17. The genre information acquisition unit 21 notifies the content processing unit 22 of the acquired genre information in response to a request from the content processing unit 22.

  The content processing unit 22 performs predetermined processing such as image quality adjustment on the video signal of the content acquired via the tuner 11 and supplies the processed video signal to the video output unit 15. In addition, the audio signal of the content is subjected to predetermined processing such as sound quality adjustment and supplied to the audio output unit 16. In this embodiment, the video signal processing is performed by the video processing unit 24, and the audio signal processing is performed by the audio processing unit 25.

  In addition, when the content processing unit 22 receives the above-described notification from the detection signal receiving unit 13 during the display of the video or the notification of the determination result that there are a plurality of users photographed from the image acquisition unit 14, Setting information corresponding to the facial expression detected by the facial expression detection unit 23 described later is acquired from the correspondence information in the recording unit 17, and a predetermined operation is performed based on the acquired setting information.

  In other words, the content processing unit 22 has a single user photographed by the camera 103, and when there is a detection signal from the human sensor 102, there are a plurality of users photographed by the camera 103 and the human sensor. When there is no detection signal from 102, or when there are a plurality of users photographed by the camera 103 and there are detection signals from the human sensor 102, the predetermined operation is executed.

  In the present embodiment, when the content processing unit 22 receives a notification from the detection signal receiving unit 13 or the image acquisition unit 14, the content processing unit 22 adds a special effect corresponding to the facial expression detected by the facial expression detection unit 23 to the acquired content. Display information for presenting the recommended functions of the apparatus 10 to the user is created and supplied to the video output unit 15 or the audio output unit 16.

  As the special effect, for example, the main user image acquired by the image acquisition unit 14 is superimposed on the video included in the content to create and display a composite video, or a cheer detected by detecting a cheer sound from the sound. Examples include effects such as adjusting the sound volume and providing the sound output unit 16 with sound effects recorded in advance in the recording unit 17.

  In the display of the recommended function, for example, a screen for asking the user whether or not to execute a function of replaying a video reproduced in the past (also referred to as a time shift reproduction function) is created and provided to the video output unit 15. In the present specification, special effects and recommended function displays may be collectively referred to as “additional effects”.

  Further, the setting information when the content processing unit 22 executes a predetermined operation may be associated with the genre information of the content. In this case, the content processing unit 22 acquires setting information corresponding to the genre information of the content acquired by the genre information acquisition unit 21 from the correspondence information of the recording unit 17, and performs a predetermined operation based on the acquired setting information. Details of the predetermined operation by the content processing unit 22 will be described later.

  The facial expression detection unit 23 detects the facial expression of any user among the users photographed by the camera 103. For example, the main user's face is recognized as a face trace from the image acquired by the image acquisition unit 14, and a facial expression is detected from the recognized face trace.

  Here, when there is only one user photographed by the camera 103, the photographed user is an arbitrary user as it is, and when there are a plurality of users, for example, a face recognized by using a face recognition technique. The user having the largest area is defined as an arbitrary user, but an arbitrary user may be set according to other rules such as setting the priority order of each user in advance through a menu.

  In response to an acquisition request from the content processing unit 22, the facial expression detection unit 23 notifies the content processing unit 22 of the detected facial expression as facial expression data (expression information). The “expression data” here is indicated as an expression representing an emotion such as “excitement” or “smile”.

  The recording unit 17 records various data and programs. As the recording unit 17, for example, a ROM (Read Only Memory) that is a read-only semiconductor memory that records fixed data such as a program necessary when the control unit 20 controls the operation, data used for calculation, and There is a RAM as a so-called working memory that temporarily records data of calculation results.

  In the present embodiment, the recording unit 17 records correspondence information between facial expression information indicating a plurality of different facial expressions and setting information for the content processing unit 22 to execute a predetermined operation. Here, the “expression information” is information indicating an expression representing an emotion such as “excitement” or “smile” as described above, and the setting information is, for example, a sub-screen of a user image captured by the camera 103 This is information indicating an additional effect such as “display” or “add sound effect”.

  In the present embodiment, the correspondence information is recorded in the recording unit 17, but can be recorded in, for example, an external recording device (not shown), and the content processing unit 22 executes a predetermined operation. You may acquire from this recording apparatus every time.

  The recording unit 17 may record sound effects when the content processing unit 22 performs a predetermined operation. This sound effect may also be acquired from an external recording device in the same manner as the correspondence information.

  The video output unit 15 outputs the video signal processed by the video processing unit 24 and the synthesized video or display information created by the content processing unit 22 to the display unit 18.

  The display unit 18 includes a display element such as an LCD (Liquid Crystal Display), an EL (Electroluminescence) display, and a PDP (Plasma Display Panel), and a driver circuit that drives the display element based on the received video signal. . The video includes characters as well as pictures and graphics, and also includes still images and moving images.

  The audio output unit 16 outputs the audio signal processed by the audio processing unit 25 and the sound effect acquired by the content processing unit 22 from the recording unit 17 from the speaker 19.

  If the user exists outside the viewing angle of the original camera 103 and the presence sensor 102 detects the presence of the user and photographing of the user's facial expression or the like is required for application specifications, A drive unit for changing the orientation of the camera 103 so as to enter the corner may be provided.

[2. (Relationship between shooting range of camera and detection area of motion sensor)
Here, the relationship between the imaging range of the camera 103 provided in the display device 10 and the detection area of the human sensor 102 will be described with reference to FIGS. 2 is a diagram showing the viewing angle of the camera 103 mounted on the display device 10 and the detection area of the human sensor 102. FIG. 3 shows a plurality of detection areas of the human sensor 102 mounted on the display device 10. It is a figure which shows a mode when it divides | segments into this area | region.

  In the following embodiment, description will be made on the assumption that there is one user within the shooting range of the camera 103, but there may be a plurality of users within the shooting range. That is, when the image acquisition unit 14 determines that there are a plurality of photographed users, the content processing unit 22 determines that there are a plurality of users viewing the content by receiving notification of the determination result, What is necessary is just to perform a predetermined | prescribed operation | movement.

  As described above, the display device 10 detects the user's facial expression captured by the camera 103 to determine the user's facial expression when viewing the content, and receives a detection signal from the human sensor 102. It is determined that there are a plurality of users viewing the content. In this way, by combining the camera 103 and the human sensor 102, it is possible to determine whether the user is single or plural, and to perform an operation requested by the user according to an arbitrary user's facial expression. Can do.

  In FIG. 2, “Φ” represents the viewing angle of the camera 103, and “θ” represents the detection area of the human sensor 102. Here, the main user (hereinafter referred to as “user A”) exists within the viewing angle of the camera 103, and the two sub users are outside the viewing angle of the camera 103 and within the detection area of the human sensor 102. There are users (hereinafter referred to as “user B” and “user C”). That is, the camera 103 captures the user A, and the user B and the user C are detected by the motion sensor 102.

  In general, since the human sensor 102 detects movement in a detection area, if the shooting range (viewing angle) of the camera 103 is included in the detection area, the user is only in the shooting range of the camera 103. Even if it exists, the human sensor 102 notifies the detection signal. Therefore, it cannot be determined whether there is one user or a plurality of users viewing the content.

  Therefore, it is preferable to divide the detection area of the human sensor 102 into a plurality of regions (hereinafter, the divided regions are referred to as “sub-areas”) and detect them. At this time, it is not necessary to include the shooting range of the camera 103 in the sub area.

  That is, the human sensor 102 may detect the movement of the sub user so as not to include the area surrounded by the triangular dotted line indicating the shooting range of the camera 103 in FIG. Thus, the main user who is in the shooting range of the camera 103 is not mistaken as a sub user without overlapping the area detected by the human sensor 102 and the area shot by the camera 103.

  In addition, as shown in FIG. 3B, the detection area is divided into a plurality of sub-areas, and the detection information for each sub-area (af) is acquired, so that the number of users viewing the content is obtained. Can be grasped. The detection information acquired at this time is shown in FIG.

  FIG. 3C shows that a user is present in the sub-area while the line is down. That is, the period in which the user exists in the subarea c is the longest, and during this period, the user exists in the subarea a and the subarea b in different periods. From this, for example, it can be determined that there are two users in the detection area, and one user has moved from the sub-area a to the sub-area b within the detection period.

  In any case, it is possible to determine whether the number of users who are viewing the content is one or plural by independently detecting the users existing in the sub-area.

  In addition to this, for example, the detection area is divided into a plurality of sub-areas, and one of the sub-areas is detected every predetermined time. It is also possible. In this case, the predetermined time may be any time and is not particularly limited. Thereby, the number of sub-users existing in the detection area can be grasped.

[3. Content processing)
Next, an example of the flow of content processing in the display device 10 will be described below. In the content processing shown below, correspondence information in which content genre information, main user facial expression information, sub-user detection information, and setting information are associated as shown in the table of FIG. 4 is used. FIG. 4 is a table showing correspondence information used for content processing in the display device 10.

  As shown in the table of FIG. 4, in this embodiment, the content genre is “sports”, the facial expression of the user A indicates “excitement”, and the detection signal for detecting the movement of the user B or the user C Is received (indicated as “present”), an additional effect is provided such that the image of the user A taken by the camera 103 is displayed on the sub-screen or the volume of the cheering sound is increased.

  Further, when the content genre is “variety”, the expression of the user A indicates “smile”, and the detection signal for detecting the movement of the user B or the user C is received, the time-shift playback function is executed. A screen that prompts the user to specify whether or not to add an applause sound is added.

  Here, the flow of processing when the user is watching a sport and the excited state is detected from the facial expression of the user A will be described based on FIG. FIG. 5 is a sequence diagram illustrating an example of the flow of content processing in the display device 10.

  First, the tuner 11 receives a broadcast signal including content via the antenna 100 (step S1), and after various processing, sends the video signal to the video processing unit 24 of the content processing unit 22 and processes the audio signal as content processing. The data is sent to the voice processing unit 25 of the unit 22. At this time, the genre information acquisition unit 21 acquires the genre information of the content (step S2). Here, the genre information acquisition unit 21 acquires genre information indicating the genre of “sports”.

  At the same time as steps S1 and S2, the image acquisition unit 14 acquires an image of the user taken from the camera 103, and the facial expression detection unit 23 detects the face of the user A from the image acquired by the image acquisition unit 14. It is recognized as a trace (step S3), and an expression is detected from the recognized face trace (step S4).

  On the other hand, when the detection signal receiving unit 13 receives the detection signal from the human sensor 102, the detection signal receiving unit 13 determines that the user B or the user C exists outside the shooting range of the camera 103, and the subordinate to the content processing unit 22 of the control unit 20. The user detection data is notified (step S5). Upon receiving this notification, the content processing unit 22 requests the genre information acquisition unit 21 to acquire genre information, and requests the facial expression detection unit 23 to acquire facial expression information (facial expression data).

  Then, the content processing unit 22 determines an operation based on the acquired genre information, facial expression data, and detection data. Here, as shown in the table of FIG. 4, the image of the user A captured by the camera 103 is displayed on the sub-screen from the information that the sport, the excited state, and the presence of the user B or the user C are “present”, and the voice cheers Determine the action of increasing the volume of the sound.

  In order to execute the determined operation, the content processing unit 22 requests the image acquisition unit 14 to acquire the image of the user A, and after acquiring the image, stores the acquired image in the recording unit 17 (step S7). .

  Thereafter, the content processing unit 22 creates a composite video by superimposing the acquired image of the user A on the video of the content, supplies the composite video to the video output unit 15, and detects and detects a cheering sound from the audio signal of the content It adjusts so that the volume of a cheering sound may be raised, and it supplies to the audio | voice output part 16 (step S8). Thereby, the content can be interesting and the enjoyment can be shared among a plurality of users.

  Next, the flow of processing when the user is viewing variety and detects a smile from the facial expression of user A will be described with reference to FIG. FIG. 6 is a sequence diagram illustrating another example of the flow of content processing in the display device 10.

  First, the tuner 11 receives a broadcast signal including content via the antenna 100 (step S10), and after various processing, sends the video signal to the video processing unit 24 of the content processing unit 22 and processes the audio signal as content processing. The data is sent to the voice processing unit 25 of the unit 22. At this time, the genre information acquisition unit 21 acquires the genre information of the content (step S11). Here, the genre information acquisition unit 21 acquires genre information indicating the genre of “variety”.

  Further, unlike the above-described example, the acquired content is recorded for time shift reproduction (step S12). The recording destination may be the recording unit 17 or an external recording device such as a DVD, HDD, or BD.

  At the same time as steps S <b> 10 to S <b> 12, the image acquisition unit 14 acquires an image obtained by capturing the user A from the camera 103, and the facial expression detection unit 23 traces the face of the user A from the image acquired by the image acquisition unit 14. (Step S13), and an expression is detected from the recognized face trace (step S14).

  On the other hand, when the detection signal receiving unit 13 receives the detection signal from the human sensor 102, the detection signal receiving unit 13 determines that the user B or the user C exists outside the shooting range of the camera 103, and the subordinate to the content processing unit 22 of the control unit 20. The user detection data is notified (step S15). Upon receiving this notification, the content processing unit 22 requests the genre information acquisition unit 21 to acquire genre information, and requests the facial expression detection unit 23 to acquire facial expression information (facial expression data).

  Then, the content processing unit 22 determines an operation based on the acquired genre information, facial expression data, and detection data. Here, as shown in the table of FIG. 4, a screen that prompts an instruction as to whether or not to execute the time-shift playback function is displayed based on the information that the variety, the smile, and the presence of the user B or the user C are “present”. Determine the behavior.

  The content processing unit 22 creates a screen asking whether to replay the content scene when the “smile” facial expression data is detected (whether to execute the time shift playback function), and provides the screen to the video output unit 15. . Then, the video output unit 15 displays the provided screen on the display unit 18 (step S17). Here, a screen with the message “interesting scene start” is displayed. The display time is not particularly limited, and may be about 1 minute, for example.

  Thereafter, when the user presses an “Enter” key provided on the remote controller 101, for example, an instruction to execute time-shifted playback is transmitted from the remote controller 101. When the instruction is received, the video from several minutes before is played back. (Step S18). Thus, even when the user B or the user C misses the scene that the user A feels fun, the screen asking whether to execute the recommended function of time shift playback is displayed and executed. Thus, the enjoyment can be shared among a plurality of users.

  As described above, according to the display device 10, it is possible to provide entertainment by adding an additional effect to the content, and a plurality of users can share the enjoyment.

(Program and recording medium)
Finally, each unit included in the display device 10 may be configured by hardware logic. Or you may implement | achieve by software using CPU as follows.

  That is, the display device 10 includes a CPU that executes instructions of a program that realizes each function, a ROM that stores the program, a RAM that expands the program into an executable format, and a memory that stores the program and various data. Storage device (recording medium). With this configuration, the object of the present invention can also be achieved by a predetermined recording medium.

  This recording medium only needs to record the program code (execution format program, intermediate code program, source program) of the program of the display device 10 which is software for realizing the above-described functions so as to be readable by a computer. This recording medium is supplied to the display device 10. Thereby, the display device 10 (or CPU or MPU) as a computer may read and execute the program code recorded on the supplied recording medium.

  The recording medium that supplies the program code to the display device 10 is not limited to a specific structure or type. That is, the recording medium includes, for example, a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. System, a card system such as an IC card (including a memory card) / optical card, or a semiconductor memory system such as a mask ROM / EPROM / EEPROM / flash ROM.

  Further, even if the display device 10 is configured to be connectable to a communication network, the object of the present invention can be achieved. In this case, the program code is supplied to the display device 10 via the communication network. The communication network is not limited to a specific type or form as long as it can supply the program code to the display device 10. For example, it may be the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication network, and the like.

  The transmission medium constituting the communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type. For example, even with wired lines such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, infrared rays such as IrDA and remote control, Bluetooth (registered trademark), 802.11 wireless, HDR, mobile phone It can also be used by radio such as a telephone network, a satellite line, and a terrestrial digital network.

  The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope shown in the claims, and the present invention is also applied to an embodiment obtained by appropriately combining technical means disclosed in the embodiment. It is included in the technical scope of the invention.

  The present invention can be suitably used as a display device including a television receiver such as a DVD built-in television, a BD built-in television, or a PC built-in television.

DESCRIPTION OF SYMBOLS 10 Display apparatus 11 Tuner 12 Remote control light-receiving part 13 Detection signal receiving part (receiving part)
DESCRIPTION OF SYMBOLS 14 Image acquisition part 15 Video | video output part 16 Audio | voice output part 17 Recording part 18 Display part 19 Speaker 20 Control part 21 Genre information acquisition part (genre information acquisition means)
22 Content processing unit (execution means)
23 Facial expression detection unit (detection means)
24 video processing unit 25 audio processing unit

Claims (11)

A display device including a display unit that acquires content and displays video included in the acquired content,
An image acquisition unit that acquires an image of the user captured by an imaging unit that captures a user existing within a predetermined range;
A receiving unit that receives a detection signal indicating that the other user exists, from a detection unit that detects whether or not the other user exists outside the predetermined range, around the device itself;
Detecting means for detecting a facial expression of an arbitrary user from the image acquired by the image acquisition unit;
An arbitrary detected by the detecting means from the correspondence information of the recording unit that records in advance correspondence information between facial expression information indicating a plurality of different facial expressions and setting information for executing a predetermined operation during display of the video An execution means for acquiring the setting information corresponding to the facial expression of the user and executing a predetermined operation based on the acquired setting information,
The image acquisition unit determines whether the user is one or more from the acquired image,
The execution means executes the predetermined operation when the receiving unit receives the detection signal and / or when the image acquisition unit determines that there are a plurality of users. Characteristic display device.   The display device according to claim 1, wherein the execution unit creates a composite video by superimposing the image acquired by the image acquisition unit on the video, and displays the generated composite video on the display unit. .   The execution means creates display information for prompting the user whether to display the past video displayed by the display unit again, and displays the created display information on the display unit. The display device according to claim 1, wherein the display device is a display device. The content includes audio, and includes an audio output unit that outputs the audio.
The display according to any one of claims 1 to 3, wherein the execution means detects a cheer sound from the sound, adjusts a volume of the detected cheer sound, and outputs the sound to the sound output unit. apparatus. The recording unit records sound effects in association with the correspondence information,
The display device according to claim 4, wherein the execution unit acquires the sound effect from the recording unit and outputs the acquired sound effect to the sound output unit. Genre information acquisition means for acquiring genre information indicating the genre of the content,
The recording unit records the genre information in association with the correspondence information,
The execution means acquires the setting information corresponding to the genre information acquired by the genre information acquisition means from the correspondence information of the recording unit, and executes the predetermined operation based on the acquired setting information. The display device according to any one of claims 1 to 5.   The said detection part divides the circumference | surroundings of the said own apparatus into a some area | region, and detects either of the said some area | region for every predetermined time, The one of Claim 1 characterized by the above-mentioned. Display device.   The display device according to claim 7, wherein a region detected by the detection unit does not include the predetermined range. A display unit for acquiring content and displaying a video included in the acquired content;
An image acquisition unit that acquires an image of the user captured by an imaging unit that captures a user existing within a predetermined range;
A receiving unit that receives a detection signal indicating the presence of the other user from a detection unit that detects whether or not the other user exists outside the predetermined range, around the device itself; A display device control method comprising:
A determination step of determining whether the user is one or more from the image acquired by the image acquisition unit;
After the determination step, a facial expression detection step for detecting an arbitrary user facial expression from the image,
After the facial expression detection step, during the display of the video, from the correspondence information of the recording unit that pre-records correspondence information between facial expression information indicating a plurality of different facial expressions and setting information for executing a predetermined operation, A setting information acquisition step for acquiring the setting information corresponding to the detected facial expression of any user;
An operation execution step of executing a predetermined operation based on the setting information acquired in the setting information acquisition step,
In the operation execution step, the predetermined operation is executed at least one of when the receiving unit receives the detection signal and when the determination unit determines that there are a plurality of users. Control method characterized.   The program for operating a computer as each means of the display apparatus of any one of Claim 1 to 8. The computer-readable recording medium which described the program of Claim 10.

Images