WO2007029393A1

WO2007029393A1 - Multimedia reproducing apparatus, menu operation accepting method, and computer program

Info

Publication number: WO2007029393A1
Application number: PCT/JP2006/312029
Authority: WO
Inventors: Katsu Saito
Original assignee: Sony Computer Entertainment Inc.
Priority date: 2005-09-05
Filing date: 2006-06-15
Publication date: 2007-03-15
Also published as: JP2007072564A

Abstract

A multimedia reproducing apparatus for reproducing a content designated by a menu operation. The multimedia reproducing apparatus comprises a moving image entering means that enters a moving image of the user; a menu image generating means that generates a menu image for accepting an operation related to a content reproduction; a superimposed image generating means that generates a superimposed moving image obtained by superimposing a mirror moving image of the entered moving image on the menu image; and a movement direction determining means that determines the direction of a user's movement in a movement recognition area established in the superimposed moving image. The menu image generating means accepts, as an operational instruction of the menu image, the determined direction of the user's movement.

Description

Specification

Multimedia playback device, menu operation reception method, and computer program

Technical field

[0001] The present invention relates to an information processing system that recognizes a user's action from an image taken by a video camera and receives a menu operation.

Background art

[0002] Patent Document 1 includes a media icon array including a plurality of media icons indicating types of media that can be played back, and a content icon array 72 including a plurality of content icons indicating content items that can be played back. In addition, it is disclosed that the convenience of the user in multimedia playback is improved by displaying the display screen so as to cross the center of the display screen.

[0003] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2004-356774

Disclosure of the invention

Problems to be solved by the invention

[0004] The input interface disclosed in Patent Document 1 is capable of selecting, playing, and executing desired multimedia content according to an up / down direction and a left / right direction, so that it is easy to understand for everyone. have. For this reason, application in the field of multimedia playback systems with a wide range of users is expected.

[0005] On the other hand, visual and intuitive operability is also required for applications in the field of multimedia playback systems.

[0006] An object of the present invention is to further improve the user operability in multimedia playback.

Means for solving the problem

[0007] A multimedia playback device according to a first aspect of the present invention for solving the above-described problem is a multimedia playback device that plays back content specified by a menu operation. And

Moving image input means for inputting a user's moving image;

A menu image generating means for generating a menu image for accepting an operation relating to content reproduction;

A superimposed image generating means for generating a superimposed moving image of the mirrored moving image of the input moving image and the menu image;

Motion direction determining means for determining the direction of the user's motion in the motion recognition area set in the superimposed moving image,

The menu image generation means receives the determined user movement direction as an operation instruction for the menu image.

[0008] Since the user can perform the menu operation by his / her own operation included in the superimposed moving image, it is possible to obtain visual and intuitive operability. Therefore, the user operability in multimedia playback is further enhanced.

[0009] A menu operation accepting method according to a second aspect of the present invention for solving the above-mentioned problem is a menu operation accepting method in a multimedia reproducing apparatus for reproducing content specified by a menu operation,

An imaging step for capturing a moving image of the user;

A menu image generation step for generating a menu image for accepting an operation relating to content reproduction;

Generating a superimposed moving image of the mirror moving image of the user's moving image and the menu image;

A motion direction determination step of determining a direction of the user's motion in the motion recognition area set in the superimposed moving image;

And an operation instruction receiving step of receiving the determined operation direction of the user as an operation instruction of the menu image.

[0010] A computer program according to a third aspect of the present invention for solving the above-described problem is a computer program that causes an information processing device to function as a multimedia playback device that plays back content specified by a menu operation. Moving image input means for inputting a user's moving image;

In the motion recognition area set in the superimposed moving image, the information processing apparatus further functions as a motion direction determination unit that determines the direction of the user's motion,

Brief Description of Drawings

FIG. 1 is a diagram showing a basic configuration of a multimedia playback system.

FIG. 2 is a diagram for explaining an outline of a menu configuration in the present embodiment.

FIG. 3 is a block diagram showing a functional configuration of the multimedia playback device.

FIG. 4 is a diagram showing recognition areas set on the screen.

FIG. 5 is a diagram for explaining a recognition area and a peripheral area.

FIG. 6 is a diagram showing a user operation in the recognition area.

FIG. 7 is a diagram showing an image of a recognition area at time (t) and an image of each peripheral area at time (t + 1).

FIG. 8 is a diagram for explaining another example of motion recognition.

FIG. 9 is a flowchart showing motion recognition processing.

FIG. 10 is a flowchart illustrating menu operation acceptance processing.

FIG. 11 is a diagram for explaining the positions of menus and recognition areas.

Explanation of symbols

[0012] 10 ... Multimedia playback system

20 ··· Multimedia playback device

30 ... Television receiver

32 ... TV line

34… Internet access 36 ... Optical disc

38 ... Memory card

40 ... Operation part

45 ... Video camera

47 ... Cable

50 ... Display screen

70… Media Icon IJI IJ

72… Content icon array

80

101 ... Video image input section

102: Image inversion part

103 ... Main control unit

103a: Face position recognition unit

103b ... Recognition area setting section

103c: Motion determination unit

104 ... Menu image generator

105 ... Superimposed image generator

106 ... Content processing section

107: Display control unit

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing a basic configuration of a multimedia playback system in an embodiment of the present invention. Here, the multimedia playback system is a composite device capable of playing back or executing various contents via various media. Examples of content include broadcast content, web content, games, movies, music, still images, and moving images.

As shown in the figure, the multimedia playback system 10 is configured around a multimedia playback device 20. The multimedia player 20 is essentially a CPU, memory The following functions can be realized by performing processing according to various computer programs.

[0016] For example, a television receiver 30 and an operation unit 40 are connected to the multimedia playback device 20. The television receiver 30 includes a display screen 50 and has a function of displaying or outputting video and audio output from the multimedia playback device 20. The operation unit 40 is a device that acquires various instructions related to content reproduction when operated by a user. The operation unit 40 may function as a game controller, or may be realized as a remote controller used for controlling various functions of the multimedia playback device 20. However, in the present embodiment, the operation unit 40 is not necessarily provided. That is, the user can give various instructions regarding content reproduction without using the operation unit 40.

The multimedia playback device 20 can input electronic information such as content through various media.

[0018] For example, data such as video and audio of a program is input to the multimedia playback device 20 via the television line 32 as broadcast content. The broadcast content may be recorded by the multimedia playback device 20, and may be output to the television receiver 30 through a decoding process by the multimedia playback device 20.

In addition, data on the web is input to the multimedia playback device 20 via the Internet line 34 as web content. This web content may be stored in the multimedia playback device 20, or may be used by various functions such as a game function of the multimedia playback device 20.

[0020] The multimedia playback device 20 has a function of reading data stored on an optical disk 36 such as a DVD or CD, and a function of reading and writing data stored on a removable memory card. The multimedia playback device 20 may read data such as a movie or a music album from the optical disc 36, or may read data such as a shot image or a shot image from the memory card 38.

In the present embodiment, a video camera 45 is connected to the multimedia playback device 20 via a cable 47 such as a USB. The video camera 45 is preferably a television receiver. The user is placed on the device 30 so that the user, particularly the upper body, can be photographed at the position facing the television receiver 30.

[0022] The multimedia playback device 20 captures a moving image captured by the video camera 45, performs mirror processing (left-right reversal processing), and displays it on the display screen 50 of the television receiver 30 in real time. . That is, the user's operation displayed on the television receiver 30 is in the same state as when the user is looking at the mirror.

[0023] Further, the multimedia playback device 20 creates a superimposed image by superimposing the mirror motion image captured by the video camera 45 and the computer image generated by the multimedia playback device 20, particularly the menu image. Displayed on the television receiver 30. As a result, the user can perform the menu operation while performing the operation while viewing the appearance and the menu image displayed on the display screen 50 of the television receiver 30.

Here, an outline of the menu format used in the present embodiment will be described with reference to FIG. Since this menu configuration is described in detail in the above-mentioned Patent Document 1 (Japanese Patent Laid-Open No. 2004-356774), refer to Patent Document 1 for details.

However, the present invention can be applied not only to the menu format described below, but also to various types of menu operations.

[0026] As shown in the figure, on the display screen 50, a media icon array 70 in which a plurality of media icons are arranged in the horizontal direction intersects with a content icon array 72 in which a plurality of content icons are arranged in the vertical direction. A two-dimensional array is displayed.

[0027] The media icon array 70 includes a photo icon 52, a music icon 54, a video icon 56, a broadcast icon 58, a disc icon 60, and a game icon 62 as marks indicating the types of media that can be played back by the multimedia playback device 20. Is included.

[0028] The content icon array 72 includes icons such as thumbnails of a plurality of contents that the multimedia playback device 20 can actually execute.

A media icon located in a region where the media icon array 70 and the content icon array 72 intersect is enlarged and displayed in a color different from that of other media icons as illustrated as a moving image icon 56. The position of the intersection area 76 is deferred while being located at the approximate center of the display screen 50. And as will be described later, In response to the instruction, the entire array of media icons moves in the horizontal direction, and the color and icon size of the media icon located in the intersection area 76 change.

[0030] Therefore, the user can select a medium only by instructing left and right directions. In addition, since the media icon located in the intersection area 76 is displayed in a display mode different from other media icons, the user can easily select the media.

[0031] When the media icon is moving in the left-right direction and no media icon is positioned in the intersection area 76, the content icon array 72 is not displayed, and any media icon is fixed in the intersection area 76. When this is done, the content icon array 72 is expanded and displayed so as to be drawn up and down from the intersection area 76.

[0032] The user can recognize the content belonging to the media by simply scrolling the media icon to the left and right with the intersection area 76 as the target point, and recognizes and recognizes the content from the expanded display. You can also

[0033] The attention area 75 located below the intersection area 76 is an area in which an icon of content to be operated by the user is displayed. The attention area 75 constitutes a part of the content icon array 72, and the content icons located in the attention area 75 are enlarged and displayed as the attention thumbnail 64. The video thumbnail 66, which is another content icon, is displayed in a size smaller than the target thumbnail 64. In the vicinity of the target thumbnail 64, for example, on the right side, the title and the recording date are displayed as attributes relating to the video of the target thumbnail 64.

[0034] While the user gives an instruction in the vertical direction, each content icon moves in the vertical direction in accordance with the instruction. The movement of the content icons in the vertical direction is realized by scrolling the entire area of the content icon array 72 in the vertical direction.

[0035] The multimedia playback device 20 displays all the content icons in the same size while quickly moving them up and down, and the vertical movement is stopped and any content icon is positioned in the attention area 75. When confirmed, the size of the notice thumbnail 64 that is the content icon is enlarged. At this time, elements such as lightness, saturation, and hue of the target thumbnail 64 may be further changed, or may be blinked.

In this state, an instruction to determine the reproduction of the content indicated as the target thumbnail 64 will be described later. The multimedia playback device 20 hides the image data on the menu screen and starts playback / execution of the selected content.

In FIG. 2, a photo icon 52 is an icon to be selected when an image read into the multimedia playback device 20 is played back. The images to be played back are mainly still images and moving images taken with a digital camera, and are read from, for example, the memory card 38. The content icon displayed when the photo icon 52 is positioned in the intersection area 76 is a thumbnail of a still image or a moving image. As attributes, shooting information such as the shooting date and the size of the camera is displayed.

[0038] The music icon 54 is an icon to be selected when music data read into the multimedia playback device 20 is played back. The music data to be reproduced is generated, for example, by encoding data read from the optical disc 36 in a specific compression format. The content icon displayed when the music icon 54 is located in the intersection area 76 may be displayed in units of music or may be displayed in units of albums. As attributes, information such as the titles of music and albums and playback times is displayed. The music data may include video data such as music promotion videos. In that case, the thumbnail of the video may be used as a content icon.

[0039] The moving image icon 56 is an icon to be selected when the recorded video stored in the multimedia playback device 20 is played back. The recorded video to be played back is a moving image obtained by recording a television program input from the television circuit 32 or a moving image taken by a digital video camera, and is encoded in a specific compression format. In this case, a thumbnail obtained by reducing the frame included in the moving image is used as the content icon. Information such as the recording date and playback time is displayed as attributes.

[0040] The broadcast icon 58 is an icon to be selected when a television program input from the television line 32 is viewed. The content icon developed for viewing may be a program unit, a broadcast station or a broadcast channel unit. As attributes, information such as broadcast station name, broadcast channel name, program title, and broadcast time acquired based on EPG (electronic program guide) data is displayed.

[0041] The disc icon 60 is selected when viewing video or music stored on the optical disc 36. Icon to be selected. When a DVD is recognized as the optical disc 36, “DVD” is displayed below the media icon. When a CD is recognized, “CD” is displayed below the media icon. When a moving image is stored on a DVD or CD, the thumbnail of the moving image is used as a content icon. When music is stored on a DVD or CD, a mark indicating music or a mark indicating a music genre is used as a content icon. Information such as video and music titles and playback time is displayed as attributes.

[0042] The game icon 62 is an icon to be selected when the game program is executed. The execution target is a game program stored in the multimedia playback device 20 or a game program read from the optical disc 36. If the game program contains a game icon as data, the icon is displayed as a content icon. Information such as the title of the game program is displayed as the attribute.

[0043] As described above, the multimedia playback device 20 displays the menu screen including the media arrangement and the content arrangement on the display screen 50. In the media icon array 70 and the content icon array 72 that are actually displayed, icons of different contents of different media are displayed one after another by scrolling up, down, left, and right in one row or one column. Therefore, the user can feel as if many contents are arranged in a matrix over the entire screen.

[0044] The user can select the content as if the user wants to focus on the desired content while freely moving the matrix in which a large number of content is arranged. However, since the actual number of icons displayed on the screen is small, it does not give a complicated impression to the user. The user can select content one after another by simply pointing up, down, left, and right by the operations described later, and can perform so-called zabbing operations with simple and comfortable operations.

Next, the functional configuration of the multimedia playback device 20 according to the present embodiment will be described with reference to the block diagram shown in FIG.

[0046] As shown in the figure, the multimedia playback device 20 includes a video image input unit 101 that inputs a moving image from the video camera 45, and a mirrored moving image that performs left-right reversal of the input moving image. An image inversion unit 102 that generates an image, a main control unit 103 that controls various processes in the multimedia playback device 20, and a menu image generation unit that generates a menu image to be displayed on the display screen 50 of the television receiver 30. 104, a superimposed image generation unit 105 that generates a superimposed image of the mirrored moving image and the menu image, a content processing unit 106 that performs reproduction / execution processing of the content specified by the menu operation, and a superimposed image or content A display control unit 107 that performs display control and a content DB 108 that stores content are provided.

[0047] Further, the main control unit 103 includes a face position recognition unit 103a, a recognition region setting unit 103b for setting a recognition region to be described later, and an operation determination unit 103c for determining a user operation included in the moving image. ing.

[0048] The face position recognition unit 103a determines whether or not the moving image captured by the video camera 45 and mirror-processed includes the user's face. Performs the process of recognizing the position. The face position recognition unit 103a only needs to be able to determine in which position of the moving image the face shape is included, and does not need to recognize each user. In other words, it is sufficient to detect that the user's face is reflected in a certain part of the image. For pattern recognition, conventional pattern recognition techniques such as boosting and support vector machines can be used.

[0049] Here, the recognition area set by the recognition area setting unit 103b will be described. As shown in FIG. 4, the recognition area is an area 80 set on the screen 50 by the multimedia playback device 20, and is set for recognizing a user's action, particularly a hand action, in the area. A plurality of recognition areas 80 can be set in various sizes according to the purpose, and are not displayed here. As will be described later, in this embodiment, the setting is made in accordance with the menu image.

[0050] It should be noted that in regions other than the recognition region 80, the motion is not recognized. Thereby, it is possible to reduce the load of motion recognition processing in the multimedia playback device 20.

[0051] Next, a method for recognizing the user's action in the recognition area will be described. As shown in FIG. 5, for one recognition area 80 (dashed rectangular area), an area having the same shape and slightly shifted around the area is set (solid rectangular area). This is called the peripheral area It shall be. Here, eight peripheral areas are set in the up / down / left / right and diagonal directions. However, it is not limited to this. For example, you may have four surrounding areas, top, bottom, left, and right, or even more powerful areas.

[0052] Note that when there are eight peripheral areas, eight directions can be detected as user actions, and when there are four peripheral areas, four directions are detected as user actions. That power S. That is, the present embodiment recognizes the user's motion as a motion in a certain direction.

Therefore, in the present embodiment, an image of the recognition area 80 at a certain time (t) ((t) X in FIG. 5) and each peripheral area at a time (t + 1) after a predetermined time has elapsed. (T + 1) A to (t + 1) H) in Fig. 5 and the recognition area image at time (t + 1). Then, an image of the most similar peripheral area or recognition area is determined, and it is recognized that the user has moved in the direction of the peripheral area.

For example, as shown in FIG. 6, it is assumed that the user's hand moves in the upper right direction between time (t) and time (t + 1) in the recognition area.

In this case, an image of the recognition area at time (t) and an image of each peripheral area at time (t + 1) are as shown in FIG. This figure corresponds to FIG. 5, and the image of the recognition area at time (t) ((t) X)) and the image of each peripheral area at time (t + 1) ((t + 1) A ~

(t + l) H).

In this example, the (t + 1) C image is most similar to the image of the recognition area at time (t). Therefore, the multimedia playback device 20 can recognize that the user's operation has been performed in the upper right direction. This recognition processing is performed in the operation determination unit 103c of the main control unit 103.

By repeating such processing, the main control unit 103 can continuously recognize user actions in the recognition area. It is also possible to detect the moving speed. Furthermore, by recording an action history, it is possible to distinguish between a one-way action and an action whose direction is frequently reversed (for example, a state of waving).

Here, in the case of continuous recognition, whether or not to update the position or contents of the image in the recognition area at the reference time (t) at any time is arbitrary, and the purpose of motion recognition It can be determined based on the nature of the operation.

[0059] It should be noted that a method for determining similar peripheral region images (including recognition region images at time (t + 1)) includes, for example, each pixel in the recognition region at time (t) and the peripheral region for each peripheral region. The absolute values of the differences in RGB values from the pixels at the corresponding positions are integrated, and the peripheral area with the smallest value can be set as a similar peripheral area. Of course, the present invention is not limited to this, and various conventionally proposed techniques can be used.

[0060] If the image of the recognition area does not change, that is, if the image of the recognition area at time (t + 1) is most similar to the image of the recognition area at time ω, the user's action Is treated as not detected. When a plurality of recognition areas are set, a user action is detected for each recognition area.

[0061] As another case, a case as shown in FIG. 8 may occur. That is, for example, as shown in FIG. 8 (a), the user's hand that was outside the recognition area at time (t) becomes the recognition area at time (t + 1) as shown in 8 (b). It is the case of entering the left end of

[0062] As described above, the motion recognition of the present embodiment is essentially for recognizing the moving direction based on the similarity of the position of the user's hand. As shown, since the user's hand is not included in the recognition area at time t), the similarity of the surrounding area in the upper, lower, and left directions that include the user's hand at time (t + 1) is shown. Lower. As a result, the similarity of the surrounding area on the right side that does not include the user's hand increases, and the movement in the right direction is recognized. Since the user's hand moves in the right direction, this recognition is consistent with the case where the user's hand is included in the recognition area at time ( _t ), and is considered to be appropriate as a result. It is done.

FIG. 9 is a flowchart showing the above operation recognition process. First, the recognition area setting unit 103b of the main control unit 103 sets one or more recognition areas according to a predetermined rule (S

101). The predetermined rule will be described later.

Then, the image data of the recognition area at time (t) is acquired (S102), and then the image data of each peripheral area at time (t + 1) is acquired (S103). In addition, the time (t

+ Get the image data of the recognition area in 1). [0065] For each peripheral area image (including the recognition area image at time (t + 1)), the similarity with the recognition area image at time (t) is calculated (S104), and similar peripheral areas are calculated. The image data of is determined (S105).

[0066] According to the determined peripheral area, the user's direction of motion is determined (S106). This direction of movement is at that moment. Although this movement direction may be adopted as it is, it is recorded here as a history (S107), and the user's movement is determined based on the tendency of the movement direction (S108). This is to prevent minor movements from being determined as user actions.

[0067] The user's motion is determined according to a predetermined motion determination rule. Here, the operation determination rule can be determined, for example, by referring to an operation history as having the following operation. In other words, if movement in the same direction continues for a predetermined time, it is determined that movement has occurred in that direction.If movement in which the direction is frequently reversed in the same recognition area continues, It is determined that there has been a motion of shaking hands.If there is a sudden movement from the same direction to the opposite direction, it is determined that there has been a reversal operation. It can be determined that there was no error.

The main control unit 103 repeats the above processing until the region recognition ends (S109).

[0069] It should be noted that in determining the motion, the next moving direction may be predicted with reference to the history, and the similarity weight in the direction may be increased. By doing so, it is possible to reduce the influence of the motion fluctuation of the user on the motion determination. Also, check the operation speed and determine the speed of the operation.

[0070] Next, a case where these motion recognition processes are applied to the menu operation will be described. That is, in this embodiment, the menu screen as shown in FIG. 2 is displayed on the screen 50 of the television receiver 30, and the operation of the menu is recognized by recognizing the operation of the mirror moving image of the user displayed in a superimposed manner. Is accepted from the user. Specifically, the up / down / left / right instructions regarding the menu operation are received by the user's up / down / left / right movements. FIG. 10 is a flowchart for explaining the processing of the multimedia playback device 20 at this time.

[0072] The menu operation in the present embodiment starts when the moving image captured by the video camera 45 and mirror-processed includes the user's face (S201). In other words, recognizing the user's face is used as a switch. As a result, if the user is in front of the television receiver 30, the menu processing is not performed, and the reception of inadvertent menu operations due to movement of the background or the like is prevented. I have to.

When the user's face is recognized in the moving image (S201: Y), the position of the face is determined, and the menu position is set based on the position (S202).

Here, the position of the menu, specifically, the media icon array 70 in the horizontal direction and the content icon array 72 in the vertical direction are set to positions as shown in FIG. That is, based on the determined position 90 of the user's face, the horizontal media icon array 70 is positioned below the face, and the vertical content icon array 72 is positioned on the right side of the face toward the screen. Set

As a result, the user can easily perform menu operations by the action of the right hand. Of course, the content icon array 72 may be positioned on the left side of the face when the user performs a menu operation by the action of the left hand. In addition, the horizontal media icon array 70 can be operated by left and right hand movements.

[0075] Then, the recognition area 80 is set in correspondence with the area of the media icon column 70 in the horizontal direction and the content icon column 72 in the vertical direction (S203).

Here, a plurality of recognition areas are set for each icon array. This is because the operation can be recognized accurately even when a new user's hand enters the recognition area, as explained in FIG. Here, it is only necessary to recognize the up / down / left / right movement, so it is not necessary to associate each icon with the recognition area. However, a recognition area 80 corresponding to the attention area 75 may be set so that a specific action for the attention area 75, for example, a decision operation for a reproduction instruction, can be recognized.

In the present embodiment, for each recognition area 80, eight peripheral areas that are diagonally up, down, left, and right are set. However, when the operation in the oblique direction is recognized, the multimedia playback device 20 regards this as an instruction in the vertical direction. Ie, content Priority is given to scrolling the icon array 72. As a result, it is possible to prevent an erroneous operation in which the user suddenly changes media, that is, scrolls the media icon array 70 in the horizontal direction while scrolling a plurality of contents. It should be noted that it is possible to recognize only the movements in the up / down / left / right directions by setting the surrounding area to four in the vertical and horizontal directions.

[0078] Then, the user's action is determined in each recognition area 80, and it is determined whether or not a left / right instruction, an up / down instruction, and a determination instruction are accepted in the left or right area (S204). For example, the decision instruction S can be assigned to the decision instruction when the user shakes his / her hand on the recognition area 80. That is, when a hand is waved on the recognition area 80, it is determined that the content is indicated as the content icon corresponding to the recognition area 80. Instead of waving, you can assign the decision to move your hand to the right or left.

[0079] Also, for example, when an operation of rapidly reversing the direction is recognized in any of the recognition areas 80, it may be handled as an instruction to stop the menu operation.

As a result, when a left / right instruction is received, a process of moving the arrangement of media icons to the left / right is performed (S205).

If an up / down instruction is accepted, a process of moving the content icon array up / down is performed (S206).

If a determination instruction is accepted, the content indicated as the attention thumbnail 64 in the attention area 75 is reproduced and executed (S207). After that, if there is an instruction to end (S208: Y), the menu processing is ended. If there is no instruction to end (S209: Y), the face position is recognized again (S201). The above process is repeated. Here, the operation itself for instructing the end is included in the menu screen according to the present invention.

[0083] As described above, according to the present embodiment, the user can perform the menu operation by his / her own operation displayed on the television receiver 30, so that visual and intuitive operability is achieved. Can be obtained. Therefore, the user's operability in multimedia playback is further enhanced.

[0084] In the above embodiment, the menu format is a combination of a vertical icon arrangement and a horizontal icon arrangement. For example, a circular shape is displayed and a rotation method is displayed. It is also possible to accept an instruction imitating a so-called jog dial by recognizing the movement in the direction.

You can also display multiple menu sets on the screen so that multiple people can perform different menu operations at the same time.

Claims

The scope of the claims

[1] A multimedia playback device that plays back content specified by menu operations.

Moving image input means for inputting a user's moving image;

The multimedia playback apparatus according to claim 1, wherein the menu image generation means receives the determined user movement direction as an operation instruction for the menu image.

[2] The multimedia playback device according to claim 1,

The menu image generation means includes

Multimedia playback characterized by generating a menu image in a two-dimensional array of media icons that indicate the types of media that can be played and content icons that indicate the content to be played on each media apparatus.

[3] The multimedia playback device according to claim 2,

The media icon and the content icon can be selected by up / down / left / right instructions,

The multimedia playback device according to claim 1, wherein the motion direction determination means determines the motion direction of the user in at least the vertical and horizontal directions.

[4] The multimedia playback device according to claim 1,

The movement direction determination means determines the movement direction of the user by comparing an image of the movement recognition area at a certain time with a plurality of images around the movement recognition area after a predetermined time. A multimedia playback device.

[5] The multimedia playback device according to claim 4,

The movement direction determination means includes The multimedia playback device characterized in that images in at least four regions shifted in the vertical and horizontal directions from the motion recognition region are used as the plurality of images around the motion recognition region.

[6] The multimedia playback device according to claim 5,

The movement direction determination means includes

The multimedia playback device, wherein among the images in the four regions, the direction of the region corresponding to the image most similar to the image in the motion recognition region is determined as the user motion direction.

[7] The multimedia playback device according to claim 5,

The movement direction determination means includes

A multimedia playback device, wherein a history of the determined user's motion direction is recorded, the user's motion direction is determined based on the history, and output as an operation instruction to the menu image generating means.

[8] The multimedia playback device according to claim 1,

A face determination means for determining whether or not a user's face is present in the superimposed moving image;

The multimedia playback device according to claim 1, wherein the motion direction determination means determines the motion direction of the user when it is determined that a user's face exists.

[9] The multimedia playback device according to claim 2,

Face position detecting means for detecting the position of the user's face in the superimposed moving image;

The menu image generation means includes

A multimedia reproducing apparatus, wherein the media icon group and the content icon group are arranged based on the detected face position.

[10] The multimedia playback device according to claim 1,

The multimedia playback device characterized in that the motion recognition area is set at a position corresponding to the menu image.

[11] In a multimedia playback device that plays back content specified by menu operations. Menu operation reception method,

An imaging step for capturing a moving image of the user;

An operation instruction receiving step of receiving the determined user operation direction as an operation instruction for the menu image.

A computer program that causes an information processing device to function as a multimedia playback device that plays back content specified by a menu operation,

Moving image input means for inputting a user's moving image;

The computer program according to claim 1, wherein the menu image generation unit receives the determined user movement direction as an operation instruction for the menu image.