JP2007249595A - Display, projector, display system, display method, display program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a projector allowing hands-free operation control. <P>SOLUTION: This projector is provided with a moving image input part 32 for inputting a face image, a face image recognition part 44 for recognizing a face action based on the face image, a command determination part 48 for determining a command for instructing reproduction of a content, based on the face action, and an output part 38 for display-outputting the content reproduced based on the command, the face image is input from the animation input part 32, the face image is tracked time-serially to recognize the face action, the command for instructing the reproduction of the content is determined based on the face action, the content reproduced based on the command is display-output by the output part 38, and the command for instructing a reproduction operation for the content is thereby determined using the series of face actions. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a display device, a projector, a display system, a display method, a display program, and a recording medium that instruct a content reproduction operation using an image.

  Conventionally, a system for recognizing an operation instruction by an operator by analyzing an image and / or sound has been used.

  Patent Document 1 discloses a digital content display method that monitors a user, determines interaction information from an operation, and changes the content presentation method. In the digital content display method, it is indicated that the interaction information is specified from the user operation and the display of the content is changed.

  Patent Document 2 discloses a method and apparatus for predicting an event using visual cues such as eyes, face poses, body postures, hand gestures, facial expressions, and the like.

  Patent Document 3 discloses an information processing apparatus that executes an instruction command according to an operator's action by analyzing an image captured by the operator. In the information processing apparatus, an arbitrary action and an instruction command are registered in association with each operator so that a desired process can be executed with an action that suits the operator's preference.

  Patent Document 4 discloses a human face motion detection method capable of reducing the burden of input and the burden on the user in a conventional device. In the human face motion detection method described above, an image including a face image is input, the attention direction of the person is extracted from the image, a change in a part inside the face is detected, and whether or not the detected change is an operation command. Is judged.

Patent Document 5 discloses an operation instruction device that identifies the content of a user's gesture based on a photographed image, extracts a plurality of command candidates, and outputs a plurality of command candidates from voice input.
JP 2005-108210 A (released on April 21, 2005) Special table 2004-515982 gazette (published May 27, 2004) Japanese Patent Laying-Open No. 2005-92419 (released on April 7, 2005) JP 9-305743 A (published on November 28, 1997) JP 2002-182680 A (released on June 26, 2002)

  However, in the system as described above, for example, when giving a presentation, it is difficult to specify a command from an image of a lecturer, and when an easy operation is associated with a command, The problem of increased misrecognition occurs.

  The present invention has been made in view of the above problems, and its purpose is to provide a display device, a projector, a display system, a display method, and a display device that can more reliably specify a command by an easy operation using a face. The object is to provide a display program and a recording medium on which the program is recorded.

  In order to solve the above problem, the display device of the present invention includes an image input unit that inputs a face image, a face image recognition unit that recognizes a face motion from the face image, and a command that instructs content reproduction. Command determining means for determining based on a face motion and an output unit for displaying and outputting content reproduced based on the command are provided.

  According to the above configuration, the face image is input from the image input unit, the face motion is recognized in time series, the face motion is recognized, the command for instructing the content reproduction from the face motion is determined, and the command The output unit displays and outputs the content reproduced based on the above. As a result, it is possible to determine a command for instructing a playback operation for displaying content using a series of face motions.

  By using the above configuration, by performing a face motion toward the image input unit of the display device, for example, operation control can be performed in a hands-free state without using a physical input device such as a remote controller. A display device that can be realized can be realized.

  In the display device of the present invention, in the above configuration, the display device further includes a command storage unit that stores the face motion in association with the command, and the command determination unit refers to the command storage unit to perform the face motion. It is characterized in that the corresponding command is determined.

  According to said structure, the table which matched the face motion and the command was memorize | stored in the command memory | storage part, and a command determination means determines a command with reference to a table. As a result, the face motion and the command can be set independently. For example, a command corresponding to the display device or the content to be displayed can be set, and the face motion can be set independently for each user who instructs display output. it can.

  In the display device of the present invention, when the first face motion is input, the command determining means enters a command input state for accepting a command input, and the second face motion recognized after the command input state is reached. It is characterized by determining a command based on the above.

  According to said structure, it can be in the state which receives the input of a command by inputting a 1st face action, and can reduce the misrecognition of a command by inputting a 2nd face action after that. For this reason, for example, a combination of simple movements is set by setting a characteristic movement that does not frequently perform the same movement as the first facial movement and setting a simple movement as the second facial movement. Therefore, it is possible to reduce erroneous recognition and malfunction caused by performing the same operation accidentally.

  In the display device of the present invention, the first face motion is a motion in which the face is stationary in a predetermined direction, and the second face motion is a motion in which at least a part of the face moves. Yes.

  According to the above configuration, an error in command input can be obtained by setting the command type by moving a part of the face after moving the face to a state where the face is allowed to stand still in a predetermined direction. Recognition can be reduced.

  A specific example of the predetermined direction is, for example, the direction of the display device.

  The display device of the present invention further includes an input state notifying unit for notifying a user that the command input state has been entered, and the command determining unit notifies the input state notifying unit when the command input state is entered. It is characterized by letting.

  According to the above configuration, the user is notified that the command input state has been entered using the input state notifying means, so the user confirms that the command input state has been entered, and then inputs the second facial motion. be able to. Thus, there is an effect that the user can surely input the second command after grasping and confirming the command input state.

  In the display device of the present invention, the face image recognition means recognizes whether the identification information registered for identifying a person matches the feature information extracted from the face image, and the command determination means A command is determined based on a face motion recognized from the face image from which the feature information matching the identification information is extracted.

  According to the above configuration, since the command is determined after identifying the face image based on the identification information, it is possible to identify the user who has issued the instruction and accept only the instruction from the identified user. it can. As a result, even when face images of a plurality of persons exist in the image data, it is possible to identify the user and determine the command.

  The display device of the present invention further includes a voice input unit that inputs voice and voice recognition means that recognizes the inputted voice, and the command determination means is based on the face motion and the recognized voice. The command is determined.

  According to the above configuration, in addition to the recognition by the face image, the command is determined based on the voice recognition result by the voice recognition means. Therefore, even if the command recognition by the face image fails, the command is input by the voice command input. Can be corrected.

  A display unit, a content storage unit that stores the content to be played back, and a playback unit that plays the content from the content storage unit based on a command from the display device and inputs the content to the output unit of the display device By providing, a projector can be configured.

  According to said structure, the above-mentioned effect is applicable to a projector.

  A display system can be configured by including the display device described above and a playback device that plays back the content based on a command from the display device.

  According to said structure, the display system which has an effect similar to the above-mentioned thing can be constructed | assembled easily.

  In order to solve the above problem, the display method of the present invention includes an image input unit that inputs a face image, and an output unit that displays and outputs the content reproduced based on a command for instructing the reproduction of the content. A display method in a display device, comprising: a face image recognizing unit recognizing a face motion from the face image; and a command determining unit determining the command based on the face motion. It is said.

  According to the above configuration, it is possible to input a face image, recognize a face motion from the input face image, and determine a command for instructing a content reproduction operation based on the face motion. As a result, it is possible to determine a command for instructing a playback operation for displaying content using a series of face motions.

  By using the above method, it is possible to control the operation in a hands-free state without using a physical input device such as a remote controller by performing a face motion toward the image input means of the display device. A display method that can be realized.

  Note that the above display method can be executed on a computer under the control of the computer. Furthermore, by storing the display program in a computer-readable recording medium, the display program can be executed on any computer.

  As described above, the display device according to the present invention inputs a face image from the image input unit, recognizes the face motion by tracking the face image in time series, and instructs the playback of content from the face motion. The command is determined, and the output unit displays and outputs the content reproduced based on the command. As a result, a command for instructing a content reproduction operation can be determined using a series of face movements.

  An embodiment of the present invention will be described below with reference to FIGS. In the present embodiment, a projector having a function of each unit described below will be described as an example of a display device, but is not limited thereto. You may make it function as a display system by connecting the display apparatus provided with the function of each part demonstrated below to an external content reproduction apparatus.

  FIG. 1 is a block diagram illustrating a schematic configuration of each unit of the projector 10 according to the present embodiment. The projector (display device) 10 includes a reproduction unit (reproduction unit) 20, an input / output unit 30, a processing unit 40, and a storage unit 50. The input / output unit 30 includes a moving image input unit (image input unit) 32, an audio input unit (audio input unit) 34, a notification unit (input notification unit) 36, and an output unit (video output unit) 38. . The processing unit 40 includes a time-series image recording unit 42, a face image recognition unit (face image recognition unit) 44, a voice recognition unit (voice recognition unit) 46, and a command determination unit (command determination unit) 48. . The storage unit 50 includes a time-series image storage unit 52, an identification image storage unit 54, a command storage unit (command recording unit) 56, and a content storage unit (content storage unit) 58.

  First, the reproducing unit 20 will be described. The playback unit 20 reads content from the content storage unit 58 based on an instruction from the command determination unit 48 of the processing unit 40 and outputs the content to the output unit 38. The content to be output may be a slide show that sequentially displays image data, or may be presentation data in which an action in the slide proceeds based on an operation. Any content can be used as long as the content can be operated based on an instruction from the command determination unit 48.

  Next, each part of the input / output unit 30 will be described. The moving image input unit 32 is a video input device for inputting video as image data from the outside. Specifically, the input / output unit 30 may be a camera using, for example, a CCD (Charge Coupled Devices) or a CMOS (Complementary Metal Oxide Semiconductor), or an external AV (Audio Visual) device or an external camera. It may be an input terminal for inputting video from. Any input device may be used as long as it can input a user's video instructing the reproduction operation of the projector 10.

  The voice input unit 34 is a voice input device for inputting voice as voice data from the outside. Specifically, the voice input unit 34 is a voice input device using a dynamic microphone, a condenser microphone, or the like. The voice input unit 34 is mainly used for inputting a content reproduction instruction from the user by voice.

  The notification unit 36 is a display unit for notifying the user that the command input state is entered when the command determination unit 48 of the processing unit 40 is in the command input state. Specifically, the notification unit 36 notifies the user by sound using a light emitting means for notifying the user by a light beam or blinking using a light, an LED (light emitting diode), or the like, or using a dial tone or a voice synthesis output. An output device using calling means or the like. The notification unit 36 may be any device as long as it can notify the user that it is in the command input state.

  The output unit 38 is a display output device for displaying the content output from the playback unit 20 to the outside. In the present embodiment, it is assumed that a projection unit that projects an image on a screen using an optical lens is used. The output unit 36 may be any output device as long as the content can be output in response to the command.

  Next, each part of the processing unit 40 will be described. The time-series image recording unit 42 records the video input from the moving image input unit 32 in the time-series image storage unit 52 as a continuous still image or a moving image input as a stream. In the present embodiment, still images taken at 100 ms intervals are recorded side by side in time series, but the present invention is not limited to this.

  The face image recognition unit 44 specifies a face image area from the current input image input from the moving image input unit 32 and inputs the region to the command determination unit 48. Note that the image recognition unit 44 may track the face image of a specific person with reference to a past image from the time-series image storage unit 52 in order to identify the face image, or may identify the identification image storage unit 54. The area of the face image of the specific person may be specified based on the identification image registered in the above. Based on past images and identification images, the face image recognition unit 44 can more easily and reliably specify a face image.

  The voice recognition unit 46 recognizes the voice input from the voice input unit 34 and inputs a voice signal to the command determination unit 48. Note that the voice recognition unit 46 may perform a process of removing noise from the input voice, separating a plurality of voices, and selecting only voices transmitted at a specific frequency.

  The command determination unit 48 determines a command from the face image from the face image recognition unit 44 and / or the voice signal from the voice recognition unit 46. Specifically, a face motion is identified based on a plurality of input face images, and a corresponding command is selected. Alternatively, the input voice signal is identified and a corresponding command is selected. Details of a method for identifying a face motion from a plurality of face images will be described later.

  Next, each part of the storage unit 50 will be described. In addition, each memory | storage part in the memory | storage part 50 may classify | categorize and memorize | store the inside of the same memory | storage device, and may be comprised using another memory element according to a use. Further, it may be an external storage medium such as a flash memory or an optical disk, or a recording device such as a semiconductor memory, a magnetic storage medium, or an optical storage medium.

  The time series image storage unit 53 is a storage unit for recording the time series image input from the time series image recording unit 42. In the present embodiment, the time-series image storage unit 53 stores images that are continuous in time series as still images. The storage format may be stored in a format obtained by taking a difference from a past image, or may be stored in a streaming format or the like. In addition, the face area may be detected for each still image, and only the face area image may be recorded, or the face area image may be analyzed, and features such as eyes and mouth open / closed states, which will be described later as command elements. May be extracted and recorded.

  The identification image storage unit 54 is a storage device for preliminarily registering identification information for identifying the user with reference to the face image recognition unit 54. The identification information may be recorded as face image or audio information, or a feature amount extracted from the face image and audio information may be recorded as identification information.

  The command storage unit 56 is a storage device for storing a table in which face motions are associated with operation commands. The association between the face motion and the operation command may be performed for each user, or a common standard command may be set. Further, the operation command may be set as appropriate in accordance with the operation performed by the playback unit 20.

  The content storage unit 58 is a storage device for accumulating content data to be played back by the playback unit 20. The content storage unit 58 may be a drive device for reading content data from various recording media or may download data via a network. The content storage unit 58 may be any storage device as long as content data can be provided in response to a request from the playback unit 20.

  In the above-described example, the projector 10 includes all the functional units. However, the configuration is not limited thereto. FIG. 2 is a block diagram showing a schematic configuration of each part of the projector 10 according to another embodiment of the present invention. Blocks having the same functions as those in the embodiment of FIG. 1 are denoted by the same reference numerals and description thereof is omitted.

  The projector 10 includes an input / output unit 30, a processing unit 40, and a storage unit 50. The PC 60 includes a playback unit 20 and a content storage unit 58. Compared with the embodiment of FIG. 1, the present embodiment is different in that the playback unit 20 and the content storage unit 58 are provided in the PC 60, and the other configurations are the same.

  The projector 10 transmits a command for instructing the operation to the reproduction unit 20 of the PC 60, and the PC 60 takes out the content corresponding to the command from the content storage unit 58 and reproduces it, and outputs it to the output unit 38 in the projector 10. The playback unit 20 may be presentation software executed on the PC 60, and the content storage unit 58 may be a storage medium reading drive provided in the PC 60.

  Next, how a presentation is actually performed using the projector 10 of the present embodiment will be described with reference to FIG. FIG. 3 is a schematic diagram showing a state when a presentation is performed using the projector 10 and the PC 60 shown in FIG.

  The projector 10 projects the content on the screen 80 while inputting the face area 72 of the speaker 70 using the camera of the moving image input unit 32. The speaker 70 performs a face motion toward the camera of the moving image input unit 32 of the projector 10 while confirming the content projected on the screen 80. The projector 10 takes out a plurality of face image areas in time series from the face area 72 of the speaker 70 photographed by the moving image input unit 32 and recognizes the face motion.

  Thereafter, the projector 10 transmits a command corresponding to the face motion to the PC 60 as a trigger signal. When receiving the trigger signal, the PC 60 reproduces the corresponding content and transmits it to the projector 10 as display data. The projector 10 receives the display data and projects content from the output unit 38 in the projector 10 onto the screen 80.

  As described above, when the speaker 70 performs a face motion toward the projector 10, a command for controlling the content to be reproduced on the PC 60 is transmitted as a trigger signal, and the display data is received by the projector 10 and projected onto the screen 80. By doing so, it is not necessary to prepare an operator for performing the content reproduction operation, and it is possible to control the content and advance the presentation without requiring the speaker 70 to hold a remote controller or the like.

  Next, the flow of facial motion recognition processing will be described with reference to FIG. FIG. 4 is a flowchart showing the flow of facial motion recognition processing. When the recognition process is started, first, various parameters are reset.

  In S101, the face image recognition unit 48 resets the counter and sequence history. In S <b> 102, the face image recognition unit 48 starts a timer and executes face detection processing every 100 ms.

  Note that the sequence history is a series of face movements connected in a time-series manner. Specific examples include face movement, face tilting, mouth opening / closing, and eyelid opening / closing. Face motions such as a motion to move, a motion to whisper and a motion to shake the face. In the sequence history, even the same operation may be regarded as a different sequence depending on the length of time elapsed in each operation.

  In S103, the movement of the face area is tracked by comparing the previous image data and the current image data. When the movement of the face area is not more than a certain value from the previous detection position (YES in S103), the process proceeds to S104. If the movement of the face area exceeds a certain value from the previous detection position (NO in S103), the process returns to S101 to reset the counter.

  In S104, the face image recognition unit 48 increments the counter, and in S105, the elapsed time is determined from the value of the counter. Then, the face image recognition unit 48 determines based on the elapsed time whether or not the user has performed an operation (command input operation) for keeping the face stationary for a certain period of time. If the counter is 20 or more (YES in S105), the process proceeds to S106. If the counter is less than 20 (NO in S105), it is determined that the face still time has not exceeded 2 seconds, and the process returns to S102 to resume counting.

  In the above-described example, the face motion that stops the face toward the camera for a certain period of time has been described as the command input operation. However, the present invention is not limited to this. It may be a face action that closes the eyelid for a certain time toward the camera, or a face action that whips twice. However, when the operation is to make the face stand still toward the camera, a command can be easily input with fewer face motions, and a command input operation that is less likely to cause erroneous recognition can be achieved.

  In S106, the face image recognition unit 44 measures the opening degree of the mouth and determines the open / closed state. In step S107, the stillness of the face and the facial actions of opening and closing the mouth are updated as a sequence history as a sequence history. In this case, the face still and the opening and closing of the mouth are recorded in the sequence history as continuous face movements, but the present invention is not limited to this. Any combination of continuous motions using the face may be used. Details of the correspondence table of face motion and command operation will be described later.

  In S108, the command recognition unit 48 determines whether the sequence history matches the specified one. That is, it is determined whether a series of face movements is associated with a command. If the sequence history is as specified (YES in S108), the process proceeds to S109. If the sequence history is not as specified (NO in S108), the process returns to S102 and repeats the process of incrementing the count again.

  In S109, the command recognition unit 48 transmits a command corresponding to a series of face movements to the reproduction unit 20 as a trigger signal. Thereafter, the process returns to S101, the counter and the sequence history are reset again, and the above process is repeated until the display of the content by the projector 10 is completed.

  Next, an example of an algorithm for determining whether the face motion is as specified will be described with reference to FIGS. 5 and 6. FIG. 5 is a schematic diagram for illustrating the characteristics of a human face. FIG. 6 is a graph showing pixel values calculated by the algorithm used in this embodiment.

  A large circle in FIG. 5 schematically shows a human face. Two small circles within the large circle schematically represent the human eye. The arc under the two small circles schematically represents the human mouth.

  Here, the rectangle in FIG. 5 is a frame line indicating the face area. The face area is specified by a square, and the vertical and horizontal widths are indicated by D. In this case, the opening / closing of the mouth is detected on the assumption that the human mouth exists within a distance of D / 3 from the lower side in the rectangle specified as the face region. Here, D is 179 pixels. Note that various algorithms conventionally used can be applied to the identification of the face area. Here, description will be made assuming that a square face area including both eyes and lips is specified.

  FIG. 6 shows values calculated from pixel values in a rectangle indicating the detected face area. Specifically, for the pixel value P on the straight line L obtained by drawing a vertical straight line from the lower side in the region to the position of 1/3 within the rectangle of the face region, the following formula (1 ) Is applied.

P = R / (G + B) (1)
The vertical axis indicates the pixel value P, and the horizontal axis indicates the rectangular Y coordinate indicating the face area. Here, R, G, and B are values indicating red, green, and blue intensities in each pixel. In the present embodiment, description will be made assuming that each value of RGB takes a range from 0 to 255. Here, 0 indicates a pixel value on the dark side, and 255 indicates a pixel value on the bright side.

  In FIG. 6, a thick straight line M drawn horizontally between pixel values P of 0.5 to 1 is a value calculated from the following equations (2) and (1).

M = (ΣP (V ₁ ) + ΣP (V ₂ )) / (V ₁ + V ₂ ) × 1.3 (2)
In other words, from both ends of the straight line L, an average of 1.3% is obtained with respect to the pixel values P in the range of approximately 3% of the areas V ₁ (Y = 0 to 5) and V ₂ (Y = 174 to 179), respectively. It is doubled. When an actual numerical value is applied to the above equation (2), the value of M is approximately 0.89.

  In the above formula, the minimum value and the maximum value of Y satisfying P> M are searched from both ends of the Y coordinate. When Y = 61, P = M. When Y = 122, P = M. On the straight line M, when the point of Y = 61 is a and the point of Y = 122 is b, it is N between a and b. When N> 179/2 is satisfied, it is determined that the person shown in the face image is open.

  Each numerical value in the above algorithm is an example and may be different. Also, the calculation formulas and ratios can be realized by other things. For example, a facial organ contour extraction technique may be used as the algorithm.

  Next, the correspondence between the command instructed to the reproduction unit 20 and the face motion will be described with reference to FIG. (A) and (b) of FIG. 7 are tables showing examples of correspondence between commands and face motions, respectively.

  In FIG. 7A, as “command 1”, the “progress presentation” command is associated with a face motion that opens and closes the mouth at intervals of less than 0.5 seconds from the state where the mouth is closed. . Also, as “Command 2”, the “Proceed Presentation” command is opened for a period of 0.5 seconds or more from the closed state, closed again, and less than 0.5 seconds. This is associated with a face action in which the mouth is closed after the mouth is closed.

  In addition, as “command 3”, a “sound / moving image stop” command is associated with a face action in which the mouth is opened after the mouth is closed for a period of 0.5 seconds or more and then the mouth is closed. In addition, as a “command 4”, a “sound / video playback” command is used. After the mouth is closed, the mouth is opened for a period of 0.5 seconds or more, and then the mouth is closed again for another 0.5 seconds. After opening the mouth only for the above-mentioned period, it is associated with a face action that closes the mouth again.

  As described above, when the mouth open time is distinguished from those longer than 0.5 seconds and those shorter than 0.5 seconds, four commands can be defined by opening and closing the mouth up to two times. In the above example, short actions are assigned to frequently used commands. Also, if the mouth open time is mixed, the command is not treated as an error.

  In FIG. 7B, as “command 1”, the “next presentation” command is changed to a series of face motions of “face front → face facing → face facing front → face facing right → face facing”. Corresponds. In addition, as “command 2”, the “progress presentation” command is associated with a series of face actions “face front → face left → face front → face left → face front”.

  Further, as “command 3”, the “sound / moving video stop” command is associated with a series of face motions of “face in front → face down → face in front”. Further, as “command 4”, the “sound / moving image reproduction” command is associated with a face action of “face front → face downward → face front → face downward → face front”.

  As described above, the projector 10 according to the present embodiment uses the moving image input unit 32 for inputting a face image, the face image recognition unit 44 for recognizing the face motion from the face image, and the command for instructing the content reproduction as the face motion. A command determination unit 46 for determining the content based on the command, and an output unit 38 for displaying and outputting the content reproduced based on the command. By searching for a region including an image, tracking a plurality of face images in time series, identifying a speaker 70 who instructs a command, and determining a command for instructing a playback device from the tracked face image A command for instructing a content reproduction operation can be determined using a series of face movements.

  By using the above configuration, by performing a face motion toward the moving image input unit 32 of the projector 10, it is possible to control the operation in a hands-free state without using a physical input device such as a remote controller. A projector can be realized.

  Further, the projector 10 according to the present embodiment further includes a command recording unit 56 that records the face motion in association with the command, and the command determination unit 48 refers to the command recording unit 56 to determine the corresponding command. The recording unit 48 stores a table in which the face motion and the command are associated with each other, and the command determining unit 48 can uniquely set the face motion and the command by determining the command with reference to the table. Therefore, a command corresponding to the projector 10 and the content to be displayed can be set, and the face motion can be uniquely set for each speaker 70 who instructs display output.

  Further, in the projector 10 of the present embodiment, the command determination unit 48 enters a command input state for accepting command input based on the first face motion, and the second face motion is input after entering the command input state. Therefore, the command can be received by inputting the first facial motion, and then the second facial motion is input to reduce erroneous recognition of the command. it can. For this reason, for example, a combination of simple movements is set by setting a characteristic movement that does not frequently perform the same movement as the first facial movement and setting a simple movement as the second facial movement. Therefore, it is possible to reduce erroneous recognition and malfunction caused by performing the same operation accidentally.

  Further, in the projector 10 according to the present embodiment, the first face motion is a motion that stops the face in a predetermined direction, and the second face motion is a motion that moves at least a part of the face. It is possible to reduce misrecognition of command input by setting the type of command by moving a part of the face after moving the part of the face by allowing the face to stand still in a predetermined direction. . The predetermined direction includes the direction of the camera provided in the projector 10.

  The projector 10 according to the present embodiment further includes a notification unit 36 that notifies the speaker 70 that the command input state has been entered, and the command determination unit 48 notifies the notification unit 36 when the command input state has been entered. Since the notification is made, the speaker 70 is notified that the command input state has been entered using the input state notification means, the speaker 70 confirms that it is in the command input state, and then inputs the second facial motion. be able to. Thus, there is an effect that the lecturer 70 can grasp the command input state and confirm the input, and then reliably input the second command.

  In the projector 10 according to the present embodiment, the face image recognition unit 44 determines whether the identification information registered in the identification image 54 for identifying a person matches the feature information extracted from the face image. The command determination unit 48 determines the command based on the face motion recognized from the face image from which the extracted information that matches the identification information is extracted, and thus determines the command after identifying the face image based on the identification information. Therefore, it is possible to identify the speaker 70 who has given the instruction and accept only the instruction from the identified speaker 70. As a result, even when face images of a plurality of persons exist in the image data, it is possible to identify the speaker 70 and determine the command.

  The projector 10 according to the present embodiment further includes a voice input unit 34 for inputting voice and a voice recognition unit 46 for recognizing the input voice. The command recognition unit 48 is recognized in addition to the face motion. Since the command is determined based on the voice, the command is determined based on the voice recognition result by the voice recognition means in addition to the recognition by the face image, so even if the command recognition by the face image fails, the voice command Commands can be corrected by input.

  Finally, each block of the projector 10, in particular, the face image recognition unit 44, the voice recognition unit 46, and the command determination unit 48 of the processing unit 40 may be configured by hardware logic, and the CPU is configured as follows. And may be realized by software.

  That is, the processing unit 40 includes a CPU (central processing unit) that executes instructions of a control program that realizes each function, a ROM (read only memory) that stores the program, a RAM (random access memory) that expands the program, A storage device (recording medium) such as a memory for storing the program and various data is provided. An object of the present invention is to provide a recording medium on which a program code (execution format program, intermediate code program, source program) of a control program of the playback apparatus 1 that is software that realizes the above-described functions is recorded so as to be readable by a computer. This can also be achieved by supplying the projector 10 and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

  Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  Further, the projector 10 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Also, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  As described above, the projector 10 according to the present invention can determine a command for instructing a reproduction operation for displaying content using a series of face motions, and thus can be applied to any display device such as a presentation system.

FIG. 2 is a functional block diagram illustrating a schematic configuration of each unit of the projector according to the embodiment of the invention. It is the functional block diagram which showed schematic structure of each part of the display system which is another embodiment of this invention. It is the schematic which showed a mode that a presentation was performed using said display system. It is a flowchart which shows the flow of the recognition process of a face motion using said display system. In said display system, it is a schematic diagram for showing the feature of the human face used for calculation. It is a graph which shows the pixel value calculated by the algorithm used with said display system. (A) And (b) is a table | surface which shows an example of a response | compatibility of the command and face motion which are respectively used with said display system.

Explanation of symbols

10 Projector (display device)
20 Reproduction unit (reproduction means)
30 Input / output unit 32 Video input unit (image input device)
34 Voice input unit (voice input device)
36 Notification section (input notification means)
38 Output unit (video output means)
40 processing unit 42 time-series image recording unit 44 face image recognition unit (face image recognition means)
46 Voice recognition unit (voice recognition means)
48 Command decision part (command decision means)
50 Storage Unit 52 Time Series Image Storage Unit 54 Identification Image Storage Unit 56 Command Storage Unit (Command Storage Device)
58 Content storage unit (content storage device)

Claims (12)

An image input unit for inputting a face image;
Facial image recognition means for recognizing facial motion from the facial image;
Command determining means for determining a command for instructing reproduction of content based on the face motion;
A display device comprising: an output unit that displays and outputs content reproduced based on the command. A command storage unit that stores the face motion in association with the command;
The display device according to claim 1, wherein the command determination unit determines the command corresponding to the face motion with reference to the command storage unit.   When the first face motion is input, the command determination means enters a command input state for accepting a command input, and determines a command based on the second face motion recognized after the command input state. The display device according to claim 2, wherein the display device is performed.   The first face motion is a motion in which the face is stationary in a predetermined direction, and the second face motion is a motion in which at least a part of the face moves. Display device. It further comprises an input state notification means for notifying the user that the command input state has been entered,
The display device according to claim 3, wherein the command determination unit causes the input state notification unit to notify when the command input state is entered. The face image recognition means recognizes whether the identification information registered for identifying a person matches the feature information extracted from the face image;
The display device according to claim 1, wherein the command determination unit determines a command based on a facial motion recognized from the facial image from which the feature information that matches the identification information is extracted. A voice input unit for inputting voice;
Voice recognition means for recognizing the input voice,
The display device according to claim 1, wherein the command determination unit determines the command based on the face motion and the recognized voice. A display device according to claim 1;
A content storage unit for storing the content to be reproduced;
A projector comprising: a reproduction unit that reproduces content from the content storage unit based on a command from the display device and inputs the content to an output unit of the display device.   8. A display system comprising: the display device according to claim 1; and a playback device that plays back the content based on a command from the display device. A display method in a display device, comprising: an image input unit for inputting a face image; and an output unit for displaying and outputting content reproduced based on a command for instructing reproduction of the content,
A step of recognizing a face motion from the face image;
A command determining means comprising: determining the command based on the face motion;   A program for operating the display device according to any one of claims 1 to 7, wherein the program causes a computer to function as each of the means.   A computer-readable recording medium on which the display program according to claim 11 is recorded.

Images