JP2014026355A - Image display device and image display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display device having a function for determining an operation area by recognizing a hand of user that enables the user to perceive a range of the operation area to perform smooth operations.SOLUTION: Operation area means monitors a relative position between a projection position to which a user's hand is projected and an operation area, and when the projection position of the user's hand deviates from the operation area, uses its deviation direction and a relative distance to present an image of the user operation situation indicating the situation of the user's hand deviating from the operation area.

Description

  The present invention relates to an image display apparatus and an image display method for recognizing a user's gesture operation and displaying a response to the operation.

  Conventionally, for video devices such as TVs and recorders, or information processing devices such as PCs, the user inputs data and instructions using a pointing device such as a keyboard and mouse, or operates and displays channels using a remote control. It was common to do.

  However, in recent years, with the improvement of image recognition technology, a method for operating a device by recognizing a user's gesture, judging a user's operation, particularly in the field of game consoles and operation guide devices has been proposed. .

  For example, Patent Document 1 discloses an image recognition device that recognizes the shape and movement of a finger and determines an operation. Further, in Patent Document 2, an operation area for the user to recognize the user's hand is set, and when the position of the user's hand approaches the periphery of the operation area, the position of the user's hand is A video display device that moves an operation area following a moving direction and a display control method thereof are disclosed.

Japanese Patent No. 4318056 JP 2011-81506 A

  In Patent Document 1 and Patent Document 2, there is a mechanism for providing a function of recognizing a hand movement pattern in a predetermined space (hereinafter referred to as an operation area) set around a user's hand as a gesture operation. It is disclosed.

  However, in the mechanisms disclosed in these patent documents, when the user performs an operation on the screen with a hand gesture, the position operated by the user and the operation target screen displayed on, for example, a projector or a television are displayed. When the position is far away to some extent, there is a problem that it is difficult for the user to understand the correspondence of the position between the operation area around the user's hand and the operation target screen.

  Further, Patent Document 1 and Patent Document 2 describe that a pointer is displayed on the screen corresponding to the position of the hand in the operation area. However, when the user operates, when the user's hand moves outside the operation area, the user intuitively determines in which direction and how far the hand is away from the operation area. There is a problem that it cannot be grasped. In particular, an operation by gesture is an operation by hand movement in the air, and there is no physical feedback to the user by touch or the like. Therefore, said subject becomes remarkable as a problem which prevents a user's smooth operation.

  The present invention has been made in order to solve the above-described problem, and the object thereof is when the user's hand is out of the operation area when the user operates the video display device with a hand gesture. The video display device presents the user so that the user can understand the direction and how far the hand is away from the operation area, and the user can smoothly grasp the operation status intuitively. It is to be able to operate.

  In order to achieve the above object, for example, the configuration described in the claims is adopted.

  According to the present invention, by recognizing the user's hand and presenting the feedback display when the hand is removed from the operation area, the user can be made aware of the range of the operation area in which the operation is effective. Smooth operation is possible.

It is a block diagram of the video display apparatus of 1st embodiment of this invention. It is a figure explaining the external appearance which a user operates with respect to the video display apparatus 100 which concerns on 1st embodiment of this invention. It is a figure explaining an example of the presentation method of the feedback display by the video display apparatus 100 which concerns on 1st embodiment of this invention. It is a figure explaining the 2nd example of the presentation method of the feedback display by the video display apparatus 100 which concerns on 1st embodiment of this invention. It is a figure explaining the 3rd and 4th example of the presentation method of the feedback display by the video display apparatus 100 which concerns on 1st embodiment of this invention. It is a flowchart which shows the procedure of the setting process of the feedback display by the video display apparatus 100 which concerns on 1st embodiment of this invention. It is a figure explaining the presentation method of the feedback display when the operation area | region by the video display apparatus 100 which concerns on 1st embodiment of this invention is three-dimensional. It is a block diagram of the video display apparatus of 2nd embodiment of this invention. It is a figure explaining the detection method of gesture operation in the outer side of the operation area | region by the video display apparatus 100 which concerns on 2nd embodiment of this invention. It is a flowchart which shows the procedure of the detection process of gesture operation on the outer side of the operation area | region by the video display apparatus 100 which concerns on 2nd embodiment of this invention. It is a figure explaining an example of the presentation method of the operation area | region and feedback display by the video display apparatus 100 which concerns on 3rd embodiment of this invention. It is a flowchart which shows the procedure of the setting process of the operation area | region and feedback display by the video display apparatus 100 which concerns on 3rd embodiment of this invention. It is a figure explaining the 2nd example presentation method of the operation area | region and feedback display by the video display apparatus 100 which concerns on 3rd embodiment of this invention.

  Embodiments according to the present invention will be described below with reference to FIGS.

  A first embodiment according to the present invention will be described below with reference to FIGS.

  In the video display device according to the present embodiment, when the user's hand is in the operation area where the operation can be performed and the state is changed from the operation area, the direction in which the hand moves with respect to the operation area is determined. The present invention provides a function of presenting the feedback display for telling the user whether the user is away by a distance.

  FIG. 1 is a configuration diagram of a video display device according to a first embodiment of the present invention.

  The video display apparatus 100 according to the present embodiment is a television or a projector, for example, and includes an input unit 110, a system control unit 120, a video signal processing unit 130, and a display unit 140, as shown in FIG.

  The input unit 110 is a device that acquires data including position information of a user's hand, and is, for example, an infrared camera, a stereo camera, or an RGB camera having a TOF (Time of flight) sensor function. The camera used for the input unit 110 is not particularly limited, and a function of recognizing an image for the user and finally obtaining a captured image for converting the obtained image into sensing data in order to detect the position of the hand. If you have. In addition, the input unit 110 may be a pointing device that acquires sensing data that can detect the position of the user's hand, such as an aerial mouse or an aerial controller that the user performs with a hand instead of a camera.

  The system control unit 120 is a part that detects the position of the user's hand and performs data processing for setting the operation region at an appropriate position. The system control unit 120 may be realized by the CPU executing a software module stored on the memory, or may be realized by a dedicated hardware circuit.

  The system control unit 120 includes parts that realize the functions of the hand position detection unit 121, the position determination processing unit 122, the operation region setting unit 123, and the feedback display setting unit 125.

  The hand position detection unit 121 is a part that detects the hand position from the sensing data acquired by the input unit 110. The position determination processing unit 122 is a part that determines the positional relationship between the position of the hand and the operation area. The operation area setting unit 123 is a part for setting an operation area. The feedback display setting unit 125 is a part for setting the feedback display.

  The video signal processing unit 130 is a part that receives instructions and data from the system control unit 120 and creates a video signal to be displayed on the display unit 140.

  The display unit 140 is a part that displays a video signal to the user based on the video signal. For example, the display unit 140 can display a still image or a moving image such as an LCD (Liquid Crystal Display), a PDP (Plasma Display Panel), or a projector. It is a display device.

  Next, the feedback display setting process by the video display apparatus 100 according to the first embodiment of the present invention will be described with reference to FIG.

  FIG. 2 is a diagram illustrating an overview when the user operates the video display device 100 according to the first embodiment of the present invention.

  The display unit 140 of the video display device 100 according to the first embodiment of the present invention is, for example, a television display 10 or a projection surface 17 of a projector.

As shown in FIG. 2A, the operation image of the video display device 100 according to the first embodiment of the present invention is an operation screen 18 displayed on the display unit 140 that is a screen to be operated by the user 30. In this embodiment, the video display device 100 performs an operation by a gesture in the virtual operation area 20 while looking at the image, recognizes an image of the operation, and performs a display corresponding to the operation based on the recognition result. It becomes to detect the position of the hand. The input unit 110 of the video display device 100 according to the present embodiment is, for example, an infrared camera 12 and has a distance measuring function using infrared rays so that the distance from the screen to the hand can be measured. The hand 32 protrudes and can be said to be a part that is easily recognized by the infrared camera 12 in the body of the user 30.

  FIG. 2B shows a relationship among the infrared camera 12, the imaging area surface 22, the operation area 20, and the operation screen 18 in the present embodiment.

  The imaging area plane 22 is a plane that represents a range that the infrared camera 12 can capture at the position of the user's hand. The operation screen 18 is an area for displaying moving image content to be presented to the user, an operation menu, the feedback display, and the like. On the operation screen 18 of the video display apparatus 100 according to the present embodiment, a still image 19 and a pointer 26 indicating the position of the user's hand in the operation area 20 set by the feedback display setting unit 125 are displayed. Here, the size of the still image 19 is the same as the size of the operation screen 18, and the entire still image 19 can be seen on the operation screen 18. Note that what is displayed on the operation screen 18 may be a moving image, animation, CG (Computer Graphics) video, or the like in addition to a still image. The pointer 26 is indicated by an arrow icon in FIG. 2B, but is indicated by a hand-shaped icon, a basic figure icon such as a circle or a square, or an icon of an anime character. May be.

  When the video display apparatus 100 recognizes the image and detects the user's hand 32, the operation area 20 corresponding to the operation screen 18 is then determined according to the position where the hand is recognized. The operation area 20 is a virtual area for causing the user 30 to perform an operation, and is created in the imaging area surface 22 of the infrared camera 12 at the position where the user 30 stands. The operation area 20 is an enlargement / reduction of the operation screen 18 on the imaging area surface 22 in front of the user. When the operation area 20 is first determined, the hand 32 of the user 30 is Try to be in the center. The operation area 20 is determined as an area in which the extension line in the direction in which the hand is extended indicates the screen according to the distance from the position of the hand 32 to the operation screen 18. The user 30 can give an instruction to the video display device 100 with an image to be used.

  As shown in FIG. 2B, when the operation area 20 is determined, a pointer 26 is displayed to indicate to the user 30 that an instruction to the video display device 100 is possible. It is also possible to present that the operation can be performed from an invalid state by changing the brightness of the image on the operation screen 18 without displaying the image 26.

  In addition, as shown in FIG. 2C, in the video display device of the present invention, the operation area for performing the operation is not a two-dimensional surface but a three-dimensional operation area such as the operation area 20a. May be.

  Here, the user gives an instruction to the video display device 100 by performing a predetermined gesture operation such as hand rotation, hand shaking, and pushing in the operation area 20 or the operation area 20a, and the operation screen 18 displays the instruction. The execution result and the feedback display are displayed. Next, the operation region setting process and the feedback display setting method by the video display device 100 according to the first embodiment of the present invention will be described with reference to FIGS.

  FIG. 3 shows the correspondence between the position of the user's hand 32 recognized by the video display device 100 and the feedback display. When the position of the user's hand deviates from the operation area 20, the hand with respect to the operation area 20 is shown. It is a figure explaining the said feedback display displayed on the operation screen 18 according to a position, a direction, and distance.

  In the video display device 100 of the present embodiment, when the user puts the hand 32 on the video display device 100 and the operation area 20 is first set, the pointer 26 is operated as shown in FIG. It is displayed in the center of the screen 18. As shown in FIG. 3B, when the user moves the hand 32 in the right direction of the operation area 20, the pointer 26 is moved in the right direction of the operation screen 18 accordingly and displayed.

  Here, as shown in FIG. 3C, when the user moves the hand 32 in the lower right direction and moves it outside the operation area 20 from the state of FIG. The still image 19 and the pointer 26 are displayed while being shifted in the lower right direction with respect to the operation screen 18. At this time, since the pointer 26 is outside the operation screen 18, it is not displayed on the operation screen 18. Also, the upper left frame of the still image 19 can be seen.

  Similarly, as shown in FIG. 3D, when the user moves the hand 32 in the upper right direction and moves it outside the operation area 20 from the state of FIG. The image 19 and the pointer 26 are displayed while being shifted in the upper right direction with respect to the operation screen 18. At this time, since the pointer 26 is outside the operation screen 18, the operation is not displayed on the surface 18. Also, the lower left frame of the still image 19 can be seen.

  3C and 3D, when the user's hand 32 is removed from the operation area 20, the position of the still image 19 moves following the hand. That is, the position of the virtual surface 200 having the same size as the operation area 20 corresponds to the position of the operation screen 18. That is, when the user's hand 32 comes out of the operation area 20, the shift direction and the shift distance of the still image 19 with respect to the operation screen 18 correspond to the shift direction and the distance of the virtual surface 200 with respect to the operation area 20.

  Here, when the user's hand 32 deviates from the operation area 20, the direction in which the still image 19 viewed by the user deviates may be opposite to the direction in which the user's hand 32 moves. That is, as shown in FIG. 4, when the user's hand 32 is removed from the operation area 20, the position of the still image 19 corresponds to the position of the operation area 20, and the position of the operation screen 18 is the position of the virtual plane 200. You may make it correspond.

  As shown in FIG. 4A, when the user moves the hand 32 in the lower right direction and moves it outside the operation area 20 from the state of FIG. 3B, the video display device 100 displays the still image 19. Are displayed with a shift in the upper left direction with respect to the operation screen 18. At this time, the pointer 26 is displayed inside the operation screen 18 so that the lower right frame of the still image 19 can be seen.

  Similarly, as shown in FIG. 4B, when the user moves the hand 32 in the upper right direction and moves it outside the operation area 20 from the state of FIG. The still image 19 is displayed while being shifted in the lower left direction with respect to the operation screen 18. At this time, the pointer 26 is displayed inside the operation screen 18, the display of the still image 19 is shifted, and the upper right frame can be seen.

  Although not shown in FIGS. 3C, 3D, 4A, and 4B, the operation area 20 and the virtual plane 200 are in the imaging area plane 22.

  In the above description, when the user's hand 32 moves outside the operation area 20, the pointer 26 is not displayed in the operation screen 18 to indicate that the operation is invalid to the user. May fall outside the operation area 20, the user may be presented with an invalid operation, for example, by reducing the overall brightness of the operation screen 18.

  As described above, the video display device 100 according to the present embodiment has the user's hand 32 when the user's hand 32 comes out from the inside of the operation area 20 and the user's hand 32 is inside the imaging area 22. Is notified to the user by feedback display on the operation screen 18 in accordance with the position, direction, and distance away from the operation area 20.

  Next, another example of feedback display when the user's hand 32 is removed from the operation area 20 will be described with reference to FIG.

  FIGS. 5A and 5B show an example of another feedback display on the operation screen 18 when the user's hand 32 is removed from the operation area 20.

  As shown in FIG. 5A, when the user moves the hand 32 in the lower right direction from the state shown in FIG. 3B and moves it outside the operation area 20, the pointer 26 is moved to the operation screen 18. The feedback icon 27 that blinks at the lower right end of the screen is displayed according to the direction and the distance that the user's hand 32 is removed from the operation area 20.

  Similarly, as shown in FIG. 5B, when the user moves the hand 32 in the upper right direction from the state shown in FIG. 3B and moves it outside the operation area 20, the operation area 20 is displayed. A flashing feedback icon 27 is displayed at the upper right end of the operation screen 18 in accordance with the direction and distance of the user's hand 32 being removed. Here, in order to allow the user to understand how far the user's hand 32 is away from the operation area 20, the blinking interval of the feedback icon 27 and the size of the icon are set to the distance away from the operation area 20. Change according to. FIG. 5C shows an example of the correspondence relationship between the blinking interval of the feedback icon 27 and the size of the icon, and the distance between the user's hand 32 and the operation area 20.

  FIGS. 5D and 5E show another example of feedback display on the operation screen 18 when the user's hand 32 is removed from the operation area 20.

  As shown in FIG. 5D, when the user moves the hand 32 in the lower right direction from the state shown in FIG. 3B and moves it outside the operation area 20, the pointer 26 is moved to the operation screen 18. Disappears at the lower right edge of the screen, and according to the direction and distance that the user's hand 32 is disengaged from the operation area 20, a portion of the screen appears as if it is about the lower right of the still image 19 displayed on the operation screen 18. A display effect 28 that is pulled in the direction and distorted is displayed.

  Similarly, as shown in FIG. 5 (e), when the user moves the hand 32 in the upper right direction from the state shown in FIG. 3 (b) and moves it outside the operation area 20, the operation screen 18 is displayed. A display effect 28 is displayed around the upper right side of the displayed still image 19 as if a part of the screen is pulled and distorted in the upper right direction. Note that the magnitude and direction of distortion of the display effect 28 correspond to the direction and distance in which the hand 32 is removed from the operation area 20.

  Next, the procedure of the feedback display setting process of the video display apparatus 100 will be described with reference to FIG.

  FIG. 6 is a flowchart showing a procedure of feedback display setting processing by the video display apparatus 100 according to the first embodiment of the present invention.

  The feedback display setting process on the operation screen 18 is a process performed by the system control unit 120 shown in FIG.

  First, the system control unit 120 analyzes data from the input unit 110 and performs recognition processing of the hand 32 of the user 30.

  When the hand 32 is recognized (S600), the operation area 20 is set according to the position of the hand 32 (S601). At this time, a pointer 26 is displayed on the operation screen 18.

  Next, the change in the position of the hand 32 is recognized, and the position of the pointer 26 on the operation screen 18 is moved according to the position of the hand 32 (S602).

  Further, it is determined again whether the hand 32 can be recognized (S603). When the hand 32 cannot be recognized, the operation area 20 is deleted (S606). At this time, the pointer 26 on the operation screen 18 is also deleted, and the process returns to S600.

  When the hand 32 can be recognized and the position of the hand 32 comes out of the operation area 20 from the inside of the operation area 20 (S604: Yes), the relative position between the hand 32 and the operation area 20 is calculated. (S607) Using the relative position between the hand 32 and the operation area 20, the feedback display setting unit displays the operation screen 18 with respect to FIGS. 3C, 3D, 4A, and 4. As shown in FIGS. 5B, 5A, 5B, 5D, and 5E, the feedback display expressing that the pointer 26 is out of the operation area 20 is displayed. Setting is made (S608). Further, it is determined whether the distance of the hand 32 to the operation area 20 is greater than a predetermined threshold (S609). When the distance of the hand 32 to the operation area 20 is larger than a certain threshold (S609: Yes), the feedback display in which the hand 32 is out of the operation area is terminated (S610), and the operation area 20 is deleted (S606). . At this time, the pointer 26 on the operation screen 18 is also deleted, and the process returns to S600. At this time, for example, the operation screen 18 is displayed as shown in FIGS. 3C, 3D, 4A, 4B, 5A, 5B, and 5D. 5 (e) may be immediately returned to the original state (FIG. 3 (a) without the pointer 26) from the state of presenting the feedback display. In order to convey the direction to the user in an easy-to-understand manner, the display may be changed little by little by animation or the like to return to the original state of the operation screen 18.

  On the other hand, when the distance of the hand 32 to the operation area 20 is smaller than the predetermined threshold (S609: No), the process returns to S602. When the hand 32 can be recognized, the position of the hand 32 is inside the operation area 20 (S604: No), and the user 32 instructs to end the operation by a predetermined operation (S605), the process ends. Otherwise, the process returns to S602.

  Note that the above-described predetermined threshold is preferably the range of the imaging region. In other words, if the hand moves outside the imaging area, the process is terminated without being processed.

  As described above, in the third embodiment of the present invention, when the hand 32 moves from the inside of the operation area 20 to the outside of the operation area 20, the position and direction of the hand 32 with respect to the operation area 20 are described. The function of presenting the distance to the user by the feedback display is provided.

  As described above, by performing the feedback display, which means that the user's hand is out of the operation area, the user can grasp the position of his / her hand with respect to the operation area where the operation is effective, and the operation is smoothly performed. be able to.

  Next, a case where the operation area is a three-dimensional space will be described with reference to FIG.

  As shown in FIG. 7A, when the position of the user's hand 32 is inside the operation area 20a, the pointer 26 is displayed on the operation screen 18 of the video display device 100. From this state, as shown in FIG. 7B, when the user moves his / her hand in the depth direction and exits the operation area 20a, the still image 19 is as far away in the depth direction on the operation screen 18. Is displayed. Further, as shown in FIG. 7C, when the user moves the hand 32 in the direction toward the user and exits the operation area 20a, the still image 19 approaches the front on the operation screen 18. Is displayed. At this time, in order to express that the operation is invalid (the operation of the hand 32 by the user is not reflected), for example, the operation screen 18 is displayed darkly.

  As described above, the video display device 100 of the video display device according to the present invention performs feedback display indicating the state on the screen when the user's hand is out of the predetermined space area where the operation is effective. The user can operate smoothly.

  Hereinafter, a second embodiment according to the present invention will be described with reference to FIGS.

  In the display control method of the video display device 100 according to the first embodiment, when the hand changes from the state in the operation region to the state in which the hand is out of the operation region, in which direction the hand is in the operation region. It provided a function to show how far away. In the present embodiment, in addition to the display control method of the first embodiment, a function of detecting a gesture operation using a state where the hand is separated from the operation area is provided.

  FIG. 8 is a configuration diagram of a video display apparatus according to the second embodiment of the present invention.

  Also in the video display device 100 of the present embodiment, as shown in FIG. 8, the input unit 110, the system control unit 120, the hand position detection unit 121, the position determination processing unit 122, the operation area setting unit 123, and the feedback display setting unit. 125, the video signal processing unit 130, and the display unit 140 are the same as in the first embodiment, except that the system control unit 120 includes the gesture detection unit 126.

  The gesture detection unit 126 is a part that detects whether or not a so-called gesture, which is a predetermined movement pattern of the user's hand 32, has been performed within the imaging range of the camera 12.

  FIG. 9 is a diagram for explaining a method of detecting a gesture operation outside the operation area by the video display device 100 according to the second embodiment of the present invention.

  As shown in FIG. 5A and FIG. 5B, when the user's hand 32 comes out of the operation area 20 from the state in the operation area 20, the user's hand 32 moves to the operation area. A flashing feedback icon 27 is displayed at the lower right end of the screen according to the position, direction, and distance away from 20. At this time, when the user's hand 32 is outside the operation area 20 and inside the imaging area 22, the gesture detection unit 126 has a predetermined movement pattern in which the movement of the user's hand 32 corresponds to an instruction to the video display device 100. It is determined whether or not. Here, the predetermined movement pattern is, for example, a movement of shaking hands up and down as shown in FIG. 9A, a movement of the hand 32 going around the outside of the operation area 20 shown in FIG. It is a movement pattern such as a movement of pushing the hand or a movement of the user rotating the hand. When the predetermined movement pattern is detected, the video display 100 interprets it as an instruction and executes a predetermined operation. In the video display device 100 according to the second embodiment of the present invention, the pop-up message 29 is displayed on the screen 18 when the user's hand 32 is out of the operation area 20 and the predetermined hand movement pattern is detected. Is displayed. As an operation executed by the video display device 100, in addition to displaying the pop-up message 29, for example, a page turning operation of a presentation, an operation of turning off the display, or the like may be used.

  As described above, by recognizing that the hand 32 has deviated from the operation area 20 and then recognizing a predetermined movement pattern in that state, various operations can be performed on the video display device 100.

  FIG. 10 is a flowchart showing a procedure of feedback display setting processing and gesture operation detection processing outside the operation region by the video display apparatus 100 according to the second embodiment of the present invention.

  First, the system control unit 120 analyzes data from the input unit 110 and performs recognition processing of the hand 32 of the user 30.

  When the hand 32 is recognized (S800), the operation area 20 is set according to the position of the hand 32 (S801). At this time, a pointer 26 is displayed on the operation screen 18.

  Next, the change in the position of the hand 32 is recognized, and the position of the pointer 26 on the operation screen 18 is moved according to the position of the hand 32 (S802).

  Further, it is determined again whether the hand 32 can be recognized (S803). If the hand 32 cannot be recognized, the operation area 20 is deleted (S808). At this time, the pointer 26 on the operation screen 18 is also deleted, and the process returns to S800.

  When the hand 32 can be recognized and the position of the hand 32 comes out of the operation area 20 from the inside of the operation area 20 (S804: Yes), the relative position between the hand 32 and the operation area 20 is calculated. (S808) Using the relative position between the hand 32 and the operation area 20, the feedback display setting unit sets the feedback display expressing that the pointer 26 is out of the operation area 20 on the operation screen 18. (S808). Further, it is determined whether the distance of the hand 32 to the operation area 20 is greater than a predetermined threshold (S809). When the distance of the hand 32 to the operation area 20 is larger than the predetermined threshold (S809: Yes), the feedback display that the hand 32 is out of the operation area is terminated (S812), and the operation area 20 is deleted (S806). ). At this time, the pointer 26 on the operation screen 18 is also deleted, and the process returns to S800. On the other hand, when the distance of the hand 32 to the operation area 20 is smaller than a predetermined threshold (S809: No), it is determined which predetermined movement pattern of hand gesture or hand rotation is performed by the user (S810), At this time, for example, a predetermined operation corresponding to the predetermined movement pattern such as presenting a pop-up message on the screen as shown in FIG. 9 is executed (S811). Then, the process returns to S802. When the hand 32 can be recognized, the position of the hand 32 is inside the operation area 20 (S804: No), and the user 32 instructs the end of the operation by a predetermined operation (S805), the process is terminated, Otherwise, the process returns to S803.

  As described above, according to the display control method of the second embodiment of the present invention, when a hand is removed from the operation area, the situation is presented to the user, and the situation is further utilized to obtain a predetermined hand. Motion patterns can be detected as operations, and more various input operations of the video display device can be performed.

  A third embodiment according to the present invention will be described below with reference to FIGS.

  In the display control method of the video display device 100 according to the first embodiment, when the hand changes from the state in the operation region to the state in which the hand is out of the operation region, in which direction the hand is in the operation region. It provided a function to show how far away. In this embodiment, in addition to the display control method of the first embodiment, when a second operation area is provided and the hand changes from the state in the operation area to the state outside the first operation area. In addition, the interface is shown to the user in which direction and how far the hand is away from the operation area, and when the hand is removed from the second operation area, the interface follows the operation area according to the movement of the hand. Is to provide.

  First, the configuration of the third video display device 100 of the present invention will be described.

  The configuration of the third video display device 100 of the present invention is the same as that of the video display device of the first embodiment shown in FIG. 1, and includes an input unit 110, a system control unit 120, a hand position detection unit 121, and a position. It includes a determination processing unit 122, an operation area setting unit 123, a feedback display setting unit 125, a video signal processing unit 130, and a display unit 140. The video display apparatus 100 according to the present embodiment sets two operation areas using the operation area setting unit 123 when the user's hand 32 is detected.

  Next, the operation area of the video display device 100 according to the third embodiment of the present invention will be described with reference to FIG.

  The first operation area 20 in FIG. 11 corresponds to the position where the user's hand 32 is recognized in the imaging area surface 22 of the infrared camera 12, similarly to the operation areas of the first embodiment to the second embodiment. This is a virtual area in which the operation of the user 30 is determined, and is an area corresponding to the operation screen 18. If the user's hand 32 is detected in the first operation area 20, the pointer 26 is displayed on the operation screen 18.

  Similar to the first operation region 20, the second operation region 1501 is a virtual region created in the imaging region 22 of the infrared camera 12, and the size of the second operation region 1501 is the first operation region 1501. The position of the second operation area 1501 is set at the same center position as that of the first operation area 20, for example.

  When the user's hand 32 is first recognized, the video display device 100 according to the third embodiment of the present invention has a first operation area around the user's hand 32 as shown in FIG. 20 and the second operation area 1501 are set. At this time, the center of the first operation area 20 is the user's hand 32, and the pointer 26 is displayed at the center of the operation screen 18. As shown in FIG. 11B, when the user's hand 32 goes out of the first operation area 20 and is inside the second operation area 1501, the video display device 100 displays a pointer. Is displayed in the lower right direction, and the still image 19 displayed on the operation screen 18 is shifted and displayed in the lower right direction according to the position, direction, and distance of the hand 32 with respect to the first operation area 20. At this time, the displacement direction of the still image 19 with respect to the operation screen 18 when the user's hand 32 comes out of the first operation region 20 corresponds to the displacement direction of the virtual surface 200 with respect to the first operation region 20. Further, the shift width of the still image 19 corresponds to the distance that the hand 32 is removed from the first operation area 20.

  Here, as shown in FIG. 11C, when the user further moves the hand 32 in the lower right direction and is recognized as exceeding the lower right side of the second operation area 1501, the video display device 100 The first operation area 20 and the second operation area 1501 are moved in the lower right direction following the position, direction, and distance of the hand 32 with respect to the second operation area 1501.

  Here, a graphic 1502 representing an area range in which the operation area can be moved and a graphic 1503 representing the position of the current operation area with respect to the movable area range are displayed on the operation screen 18. In the present embodiment, the graphic 1502 corresponds to, for example, the imaging area surface 22, and the graphic 1503 corresponds to the second operation area 1501. As shown in FIG. 11C, when the user's hand 32 goes outside the second operation area 1501, the first operation area 20 and the second operation area 1501 are moved by the movement of the hand. However, the display position of the graphic 1503 in the graphic 1502 changes accordingly.

  Since the first operation area 20 always moves simultaneously with the second operation area 1501, the display of the graphic 1502 and the graphic 1503 also represents the range in which the first operation area 20 can move. That is, by presenting the feedback display using the graphic 1502 and the graphic 1503, the user can understand the range in which the operation area around the hand 32 can move.

  By providing the second operation area 1501 in this way, the user can confirm that his / her hand 32 has been removed from the operation area by the feedback display, and further move the operation area to an arbitrary position. be able to.

  Next, a procedure for setting an operation region and a feedback display by the video display device 100 according to the third embodiment of the present invention will be described with reference to FIG.

  FIG. 12 is a flowchart showing a procedure for setting an operation area and a feedback display by the video display apparatus 100 according to the third embodiment of the present invention.

  The setting process of the first operation area 20 and the second operation area 1501 is a process performed by the system control unit 120 shown in FIG.

  First, the system control unit 120 analyzes data from the input unit 110 and performs recognition processing of the hand 32 of the user 30.

  When the hand 32 is recognized (S700), the operation area 20 is set according to the position of the hand 32 (S701). At this time, a pointer 26 is displayed on the operation screen 18.

  Next, the change in the position of the hand 32 is recognized, and the position of the pointer 26 on the operation screen 18 is moved according to the position of the hand 32 (S702).

  Furthermore, it is determined again whether the hand 32 can be recognized (S703). If the hand 32 cannot be recognized, the operation area 20 is deleted (S706). At this time, the pointer 26 on the operation screen 18 is also deleted, and the process returns to S700.

  When the hand 32 can be recognized and the position of the hand 32 comes out of the first operation area 20 from the inside of the first operation area 20 (S704: Yes), the hand 32 and the first operation The relative position with respect to the area 20 is calculated (S707), and the feedback display as shown in FIG. 10B is set using the relative position between the hand 32 and the first operation area 20 (S708). And displayed on the operation screen 18. Further, it is determined whether the hand 32 has been recognized and the position of the hand 32 has moved outside the second operation area 1501 (S709). At that time, the second operation area 1501 is determined according to the movement of the hand 32. And the 1st operation area | region 20 is moved. That is, the second operation area 1501 and the first operation area 20 are moved following the movement of the hand 32 (S710). Next, as shown in FIG. 11C, the feedback display representing the position of the movable range of the first operation area is updated (S711), and the process returns to S702. When the hand 32 can be recognized and the position of the hand 32 does not move outside the operation area 20 (S704: No), when the user 32 instructs the end of the operation by a predetermined operation (S705), the process is ended. .

  When the user's hand 32 is removed from the first operation area 20 and is still inside the second operation area 1501, the two operation areas are moved following the movement of the hand, and the user's hand 32 When the hand 32 is removed from the second operation area 1501, the hand is removed from the operation area on the operation screen 18 according to the direction or distance of the hand 32 being removed from the second operation area 1501. You may make it set the said feedback display which shows.

  That is, the video display device 100 according to the third embodiment of the present invention has a hand in the first operation area 20 as shown in FIG. 13A at the timing of detecting the position of a certain hand. At the timing of detecting the position of the next hand, as shown in FIG. 13B, the hand 32 comes out of the first operation area 20 and is still inside the second operation area 1501. In some cases, the first operation region 20 and the second operation region 1501 are moved following the position, direction, and distance of the hand 32 with respect to the first operation region 20. Then, as shown in FIG. 13C, the hand 32 comes out of the second operation area 1501 at the timing of detecting the position of the next hand from the state shown in FIG. That is, when the user's hand 32 is moved faster than when the user's hand 32 is in the state shown in FIG. 13B), the second operation area 1501 depends on the direction and distance the hand 32 is removed. Then, a feedback screen indicating that the hand 32 is out of the operation area is presented on the operation screen 18. With this mechanism, the user can move his / her hand and move the operation area following the movement of the hand when it is removed from the operation area. By the display, it can be recognized that the hand is out of the operation area.

  As described above, according to the display control method of the video display device of the present invention, when the user's hand is out of the set virtual operation area during the operation, the situation can be easily understood on the screen by the feedback display. In addition, since the operation area once set is dynamically changed according to the movement of the hand by the user, it is not necessary to determine the timing of calibration halfway, and the operability can be improved.

  As described above in the first embodiment to the third embodiment, when the user's hand is out of the set virtual operation area during the operation, the situation can be understood on the screen by the feedback display. Since it is easily communicated, the user can perform the operation smoothly.

DESCRIPTION OF SYMBOLS 100 ... Video display apparatus, 110 ... Input part, 130 ... Video signal processing part, 140 ... Display part, 121 ... Hand position detection part, 122 ... Position determination processing part, 123 ... Operation area setting part, 124 ... Hand-movement operation detection Part, 10 ... display, 12 ... infrared camera, 14 ... LED, 16 ... lens, 18 ... operation screen, 19 ... still image, 20 ... operation area, 22 ... imaging area surface, 26 ... pointer, 30 ... user, 32 ... hand

Claims (8)

A video display device that captures an image of a user, performs image recognition on the image data of the image, and presents a status of the user's operation based on a result of the image recognition,
A display for displaying video to be presented to the user;
Based on the result of the image recognition, an operation area that can be operated by the user is set around the position of the user's hand on the imaging area surface,
A control unit that detects the position of the user's hand in the imaging region and displays the operation status on the display unit when the position of the user's hand is outside the range of the operation region;
A video display device comprising:   When the position of the user's hand is outside the range of the operation area, the control unit causes the display unit to display a distance and direction from the operation area to the position of the user's hand. The video display device according to claim 1, wherein   The control unit detects the predetermined movement as a predetermined operation when the position of the user's hand is outside the range of the operation area and the user's hand has made a predetermined movement. The video display device according to claim 1, wherein the display unit performs display for a predetermined operation.   The control unit sets a second operation region different from the operation region, and when the position of the user's hand moves outside the range of the second operation region, the control region and the second operation region The operation area is moved in the direction in which the position of the user's hand moves to reset each operation area, and the position of the reset second operation area in the imaging area plane is displayed on the display unit. The video display device according to claim 1. A video display method for capturing an image of a user, performing image recognition on image data of the image, and presenting a status of the user's operation based on a result of image recognition,
Based on the result of the image recognition, an operation area that can be operated by the user is set around the position of the user's hand on the imaging area surface,
Detecting the position of the user's hand in the imaging area;
An image display method, comprising: displaying an operation status when the position of the user's hand is outside the range of the operation area. 6. The display according to claim 5, wherein when the position of the user's hand is outside the range of the operation area, the distance and direction from the operation area to the position of the user's hand are displayed. Video display method. When the position of the user's hand is outside the range of the operation area and the user's hand has made a predetermined movement, the predetermined movement is detected as a predetermined operation,
6. The video display method according to claim 5, wherein display is performed for a predetermined operation. Set a second operation area different from the operation area,
When the position of the user's hand moves outside the range of the second operation area, each operation is performed by moving the operation area and the second operation area by the distance in the direction in which the position of the user's hand has moved. Reset the area,
6. The video display method according to claim 5, wherein the position of the reset second operation area in the imaging area plane is displayed.

Images