JP2010154405A - Video image reproducing device, control signal generating device, and method of generating control signal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video reproducing device and control signal generating device that recognize more manual operations, as compared to a conventional device, and a method of generating control signal. <P>SOLUTION: The video image reproducing device 1 includes: a depth image acquiring portion 20, that acquires the depth image comprised of a plurality of detection regions, by receiving information on the distance to a user 4 detected by a distance sensor 3 and by dividing the information of the distance as a plurality of detection regions based on the distance; a both hands detection portion 21 for detecting detection regions as hands that are most approximated to a plurality of detection regions of the depth image; a movement recognizing portion 22 for recognizing hands movement detected by both hands detection portion 21 and for generating control signals, based on the hands movement; and a reproduction control portion 23 for reproducing and controlling a video image content 25a, based on the control signal. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a video reproduction device, a control signal generation device, and a control signal generation method.

  As a conventional technique, there is a video reproduction apparatus that operates based on the shape of a user's hand photographed by a camera (see, for example, Patent Document 1).

  This video playback device includes a camera for photographing a user, a hand shape database for storing a plurality of preset hand shapes, and a hand shape database for user hand shapes in a video photographed by the camera. A hand shape recognition unit for recognizing based on the stored hand shape; and a hand motion analysis unit for analyzing a user's hand motion recognized by the hand shape recognition unit. Since an operation signal is transmitted to the software to be operated based on the analysis result of the operation, the user can operate the software by hand operation.

However, according to the conventional video reproduction device, the hand shape recognition unit recognizes the user's hand only when the hand shape stored in the hand shape database is detected in advance, so that the hand movement is limited. There's a problem. In particular, since the video captured by the camera is analyzed as a two-dimensional image, it cannot cope with the three-dimensional operation of the user.
JP 2008-146243 A

  Therefore, an object of the present invention is to provide a video reproduction device, a control signal generation device, and a control signal generation method that can recognize more manual movements than in the past.

(1) In order to achieve the above object, the present invention classifies the detection surface into a plurality of detection regions based on the distance information indicating the distance between each element provided on the detection surface of the distance sensor and the user. A video reproduction device for recognizing the movement of the detection area closest to the distance sensor from the plurality of detection areas and receiving a control signal generated based on the movement of the detection area, based on the control signal A video playback device characterized by having a playback control unit that controls playback of video information.

  According to the configuration described above, a part of the user's body is detected as the detection area, the three-dimensional motion of the detection area with respect to the distance sensor is recognized, and is reproduced by the control signal generated based on the movement of the detection area. Since the control unit is controlled, it is possible to control the reproduction of the video information in the reproduction control unit with more operation content than in the conventional case where the two-dimensional operation is recognized.

(2) In order to achieve the above object, the present invention receives distance information indicating the distance between each element of the distance sensor and the user, classifies the distance information as a plurality of detection regions based on the distance, A depth image acquisition unit that acquires a depth image constituted by a plurality of detection regions; an indication region detection unit that detects, as an indication region, a detection region that is closest to the distance sensor from the plurality of detection regions of the depth image; There is provided a control signal generation device including a motion recognition unit that recognizes the movement of the pointing region detected by the pointing region detection unit and generates a control signal based on the movement of the pointing region.

  According to the configuration described above, the depth image acquisition unit and the instruction region detection unit detect the user's instruction region, and the motion recognition unit indicates the distance of the instruction region with respect to the distance sensor and an instruction on a plane parallel to the distance sensor. By recognizing the movement of the area, the movement of the three-dimensional instruction area can be recognized, so that more hand movements can be recognized as compared with the conventional case where the hand shape is recognized on the image captured by the camera.

(3) In order to achieve the above object, the present invention receives distance information indicating the distance between each element of the distance sensor and the user, classifies the distance information as a plurality of detection areas based on the distance, A depth image acquisition step of acquiring a depth image constituted by a plurality of detection regions; an indication region detection step of detecting a detection region closest to the distance sensor from the plurality of detection regions of the depth image as an indication region; And a motion recognition step of recognizing the detected movement of the pointing area and generating a control signal based on the movement of the pointing area.

  According to the configuration described above, (2) a control signal generation method having the same effect can be achieved.

  According to the present invention, it is possible to recognize more manual movements than in the past.

  Hereinafter, embodiments of a video reproduction device, a control signal generation device, and a control signal generation method of the present invention will be described in detail with reference to the drawings.

[Embodiment]
(Configuration of video playback device)
FIG. 1 is a schematic diagram showing a configuration of a video playback apparatus according to an embodiment of the present invention.

  The video playback device 1 can play back video content received by a tuner provided in the main body 2 and video content stored in a storage unit such as an HDD (Hard Disk Drive) provided in the main body 2. Yes, as a result of the reproduction, the video is displayed on the display unit 24 having a display screen such as an LCD (Liquid Crystal Display).

  In addition, the video playback device 1 includes a distance sensor 3 that acquires distance information of an object including the user 4 facing the video playback device 1, and operates based on the distance information acquired by the distance sensor 3. The distance sensor 3 includes, for example, an infrared camera including an infrared laser and a plurality of light receiving elements, and obtains distance information by photographing the reflection of infrared rays emitted by the infrared laser with the infrared camera.

  The video reproduction apparatus 1 also includes a remote controller 2A composed of a plurality of operation switches, and a remote control receiver (not shown) that receives an operation signal transmitted to the main body 2 based on the operation of the operation switches from the remote controller 2A. Operates based on the signal.

  FIG. 2 is a block diagram showing the configuration of the video playback apparatus according to the embodiment of the present invention.

  The main body 2 of the video reproduction device 1 uses the depth image acquisition unit 20 that acquires the distance information acquired by the distance sensor 3 as a depth image, and both hands of the user 4 as an instruction area from the depth image acquired by the depth image acquisition unit 20. A both-hand detection unit 21 as an instruction area detection unit to be detected, a movement recognition unit 22 that recognizes the movement of both hands detected by the two-hand detection unit 21 and outputs a control signal based on the movement to the reproduction control unit 23, and will be described later A playback control unit 23 that reads the video content 25a, decodes it, and plays it back, a display unit 24 that displays the video content played back by the playback control unit 23, a storage unit 25 that stores the video content 25a, and a remote control reception unit And a remote control information acquisition unit 26 for receiving an operation signal from the remote control 2A and transmitting a control signal to the reproduction control unit 23.

(Operation)
Hereinafter, the operation of the video reproduction apparatus according to the embodiment of the present invention will be described with reference to the drawings.

  When the user 4 faces the detection surface 30 of the distance sensor 3, the distance sensor 3 detects the user 4 as an object to be detected and acquires the distance information. Below, the distance information acquisition operation of the distance sensor 3 will be described.

  FIGS. 3A to 3F are schematic diagrams illustrating distance information acquisition operation of the distance sensor according to the embodiment of the present invention.

  The distance sensor 3 has a detection surface 30 at z = 0 on the xy plane, and when the detection objects 3A and 3B are arranged at the positions shown in FIGS. The distance information as shown in d) is acquired. 3A is a perspective view, FIG. 3B is a plan view from the positive direction of the y-axis, and FIG. 3C is a side view from the negative direction of the x-axis.

  The distance information shown in FIG. 3D is obtained by dividing the detection surface 30 of the distance sensor 3 into, for example, 25 detection surfaces and describing the distances to the objects to be detected facing the divided detection surfaces. And the unit is cm. Further, when there is no object to be detected or when the object to be detected is farther from the reference surface 3C at the position of the detection surface 30 to 40 cm, for example, the distance is represented as 0 cm. Note that the number of divisions of the detection surface is 25 for simplification of description, but the detection surface may be divided by, for example, a resolution of VGA (Video Graphics Array) composed of 640 × 480 dots. The upper limit of the number of divisions is the number of light receiving elements of the distance sensor 3.

  Next, when the depth information acquisition unit 20 acquires the distance information illustrated in FIG. 3D from the distance sensor 3, the depth image acquisition unit 20 groups the distances of the divided detection surfaces by close values, and the process illustrated in FIG. As shown, the detection areas 30A and 30B are recognized.

  Next, as shown in FIG. 3F, the depth image acquisition unit 20 converts the acquired distance value into a distance from the reference plane 3C.

  4A to 4C are schematic diagrams illustrating examples of depth images acquired by the depth image acquisition unit according to the embodiment of the present invention.

  When the depth image acquisition unit 20 acquires the distance information from the distance sensor 3, the depth image acquisition unit 20 converts it into a depth image as shown in FIG. It recognizes as an area | region and converts into the depth image 31B shown in FIG.4 (b). In addition, the same hatching area | region of FIG.4 (b) shows the same detection area | region. Here, the depth image 31 </ b> A is converted based on distance information acquired at VGA resolution, for example.

  Next, as shown in the depth image 31 </ b> C of FIG. 4C, the both-hand detection unit 21 detects, for example, the closest detection area as the right hand 310 and the left hand 311. The right hand 310 and the left hand 311 are distinguished by whether the center 310a and the left hand center 311a of the right hand 310 are located on the right side or the left side of the center 312. The center 312 is calculated by the method described below. Is done.

Each detection surface divided by the depth image 31B is represented by (x, y), and the depth converted to the distance from the reference surface is D (x, y). The coordinates of the center 312 (x _Center , y _Center) ) Is expressed by the following equations 1 and 2.

  Next, the motion recognition unit 22 transmits a control signal to the reproduction control unit 23 based on the movements of the right hand center 310a and the left hand center 311a detected by the both-hand detection unit 21. Below, the specific example of the motion recognition of the motion recognition part 22 is demonstrated.

  FIGS. 5A to 5C are schematic diagrams illustrating an example of motion recognition of the motion recognition unit according to the embodiment of the present invention.

  Normally, the video playback apparatus 1 displays the content video 24A on the display unit 24 as shown in FIG. 5A. However, the motion recognition unit 22 uses the left hand 40A of the user 4 from the other part of the user 4. If it recognizes that it protruded before, a control signal will be transmitted to the reproduction | regeneration control part 23 so that operation display 24a and 24b may be displayed on the content image | video 24A, as shown in FIG.5 (b).

  When it is recognized that the left hand 40A is pulled back again, the operation displays 24a and 24b are stopped as shown in FIG. The operation display 24a is a display indicating the playback position on the timeline indicating the playback time of the video content being played back, and the operation display 24b is a display indicating a menu including operation symbols such as playback, fast forward, and rewind. It is. The number and type of operation displays to be displayed are set in advance by the user 4 on a setting screen (not shown).

  FIG. 6 is a schematic diagram illustrating another example of motion recognition of the motion recognition unit according to the embodiment of the present invention.

  When the movement recognizing unit 22 recognizes that the left hand 40A has moved to the right after displaying the operation display 24a as shown in FIG. 5B, the movement recognition unit 22 displays the reproduction position indicated on the timeline 241 of the operation display 24a. A control signal is transmitted to the reproduction control unit 23 so as to move in proportion to the moving distance of the left hand 40A. Further, when the left hand 40A is moved in the left direction, the reproduction position 240 is moved to the left side of the timeline 241.

  FIGS. 7A and 7B are schematic diagrams illustrating another example of motion recognition of the motion recognition unit according to the embodiment of the present invention.

  The motion recognition unit 22 recognizes that the left hand 40A and the right hand 40B are projected as shown in FIG. 6A after displaying the operation display 24a as shown in FIG. As shown, when it is recognized that both hands are expanded outward, a control signal is transmitted to the reproduction control unit 23 so as to expand the display scale of the timeline 241 of the operation display 24a in proportion to the movement distance. When it is recognized that both hands are narrowed inward, a control signal is transmitted to the reproduction control unit 23 so as to reduce the display scale of the timeline 241 of the operation display 24a in proportion to the movement distance.

  8A and 8B are schematic diagrams illustrating another example of motion recognition by the motion recognition unit according to the embodiment of the present invention.

  When the motion recognition unit 22 recognizes that the right hand 40B of the user 4 has protruded before the other part of the user 4, the motion recognition unit 22 displays a playback speed display 24c and an operation display 24a as shown in FIG. When it is recognized that the right hand 40B is moved so as to rotate on the same plane as shown in FIG. 8B, the playback speed is changed to normal playback, 2 × playback, 4 × playback,. A control signal is transmitted to the reproduction control unit 23, and the display scale of the operation display 24a is changed in proportion to the reproduction speed.

  FIGS. 9A and 9B are schematic diagrams illustrating another example of motion recognition of the motion recognition unit according to the embodiment of the present invention.

  When the motion recognition unit 22 recognizes that the right hand 40B of the user 4 protrudes ahead of the other part of the user 4 with the palm facing upward, a volume display 24d is displayed as shown in FIG. Further, when it is recognized that the right hand 40B is turned down as shown in FIG. 9B, that is, when it is recognized that the distance of the detection region is reduced and the area is inhabited, reproduction is performed so that the volume is changed to decrease by one scale. A control signal is transmitted to the control unit 23. Also, when it is recognized that the right hand 40B has changed upward from the state in which the right hand 40B is faced down as opposed to the operation shown in FIG. 9B, a control signal is transmitted to the reproduction control unit 23 so as to change the volume to be increased by one scale. To do.

  FIGS. 10A and 10B are schematic diagrams illustrating another example of motion recognition of the motion recognition unit according to the embodiment of the present invention.

  When the motion recognizing unit 22 recognizes that the left hand 40A of the user 4 protrudes ahead of the other part of the user 4 with the palm facing upward, the channel display 24e is displayed as shown in FIG. When it is recognized that the left hand 40A is turned down as shown in FIG. 10B, a control signal is transmitted to the reproduction control unit 23 so that the channels are changed in the order of the set channels. In contrast to the operation shown in FIG. 10B, when recognizing that the left hand 40 </ b> A has changed upward from the face down state, a control signal is transmitted to the reproduction control unit 23 so as to change the channel order in the reverse order.

  FIG. 11 is a schematic diagram illustrating another example of motion recognition of the motion recognition unit according to the embodiment of the present invention.

  When the motion recognizing unit 22 recognizes that the right hand 40B of the user 4 protrudes before the other part of the user 4 and is set to display the operation display 24b, the operation display as shown in FIG. 24b and a cursor 24f are displayed. When recognizing that the right hand 40B is moved, a control signal is transmitted to the reproduction control unit 23 so that the cursor 24f moves in proportion to the movement amount of the right hand 40B. In addition, the playback control unit executes the fast-forward operation when the cursor 24f is overlaid on a predetermined time, for example, one second, on an icon indicating the operation content displayed on the operation display 24b, for example, an icon indicating fast-forward. 23 transmits a control signal.

(Effect of embodiment)
According to the above-described embodiment, the video reproduction apparatus 1 detects the hand of the user 4 by the distance sensor 3, the depth image acquisition unit 20, and the both-hand detection unit 21, and the motion recognition unit 22 detects the distance of the hand with respect to the distance sensor 3. And, by recognizing the movement of the hand on the plane parallel to the distance sensor 3, it is possible to recognize the three-dimensional hand movement, so compared to the conventional method of recognizing the hand shape on the image taken by the camera. Many hand movements can be recognized.

  The relationship between the manual operation described above and the operation content of the reproduction control unit 23 is an example, and the combination may be changed, or the reproduction control unit 23 may recognize another manual operation. Further, the user 4 may freely set the relationship between the manual operation and the operation content of the reproduction control unit 23.

It is the schematic which shows the structure of the video reproduction apparatus which concerns on embodiment of this invention. It is a block diagram which shows the structure of the video reproduction apparatus concerning embodiment of this invention. (A)-(f) is the schematic which shows the acquisition operation | movement of the distance information of the distance sensor which concerns on embodiment of this invention. (A)-(c) is the schematic which shows the example of the depth image which the depth image acquisition part which concerns on embodiment of this invention acquires. (A)-(c) is the schematic which shows an example of the motion recognition of the motion recognition part which concerns on embodiment of this invention. It is the schematic which shows the other example of the motion recognition of the motion recognition part which concerns on embodiment of this invention. (A) And (b) is the schematic which shows the other example of the motion recognition of the motion recognition part which concerns on embodiment of this invention. (A) And (b) is the schematic which shows the other example of the motion recognition of the motion recognition part which concerns on embodiment of this invention. (A) And (b) is the schematic which shows the other example of the motion recognition of the motion recognition part which concerns on embodiment of this invention. (A) And (b) is the schematic which shows the other example of the motion recognition of the motion recognition part which concerns on embodiment of this invention. It is the schematic which shows the other example of the motion recognition of the motion recognition part which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Video reproduction apparatus, 2 ... Main body, 2A ... Remote control, 3 ... Distance sensor, 3A, 3B ... Object to be detected, 3C ... Reference plane, 4 ... User, 20 ... Depth image acquisition part, 21 ... Both-hands detection part, 22 ... Recognition unit, 23 ... Playback control unit, 24 ... Display unit, 24A ... Content video, 24a ... Operation display, 24b ... Operation display, 24c ... Playback speed display, 24d ... Volume display, 24e ... Channel display, 24f ... Cursor 25 ... Storage unit, 25a ... Video content, 26 ... Remote control information acquisition unit, 30 ... Detection surface, 30A ... Detection area, 31A ... Depth image, 31B ... Depth image, 31C ... Depth image, 40A ... Left hand, 40B ... Right hand , 240 ... playback position, 241 ... time line, 310 ... right hand, 310a ... center, 311 ... left hand, 311a ... center, 312 ... center

(1) In order to achieve the above object, the present invention classifies the detection surface into a plurality of detection regions based on the distance information indicating the distance between each element provided on the detection surface of the distance sensor and the user. recognizes the movement of the detection region from the plurality of detection areas closest to the distance sensor, which receives the generated control signals based on the movement of the detection region, reproducing video information on the basis of the control signal Provided is a video reproduction apparatus characterized by having a reproduction control unit for controlling.

Motion recognition unit 22, after displaying the operation display 24a as shown in FIG. 5 (b), recognizes that the left hand 40A and right hand 40B as shown in FIG. 7 (a) protruding, in FIG. 7 (b) As shown, when it is recognized that both hands are expanded outward, a control signal is transmitted to the reproduction control unit 23 so as to expand the display scale of the timeline 241 of the operation display 24a in proportion to the movement distance. When it is recognized that both hands are narrowed inward, a control signal is transmitted to the reproduction control unit 23 so as to reduce the display scale of the timeline 241 of the operation display 24a in proportion to the movement distance.

When the motion recognition unit 22 recognizes that the right hand 40B of the user 4 protrudes ahead of the other part of the user 4 with the palm facing upward, a volume display 24d is displayed as shown in FIG. Further, when it is recognized that the right hand 40B is turned down as shown in FIG. 9B, that is, when it is recognized that the distance of the detection region is reduced and the area is reduced , reproduction control is performed so that the volume is changed to be decreased by one scale. A control signal is transmitted to the unit 23. Also, when it is recognized that the right hand 40B has changed upward from the state in which the right hand 40B is faced down as opposed to the operation shown in FIG. 9B, a control signal is transmitted to the reproduction control unit 23 so as to change the volume to be increased by one scale. To do.

Claims (11)

The distance information indicating the distance between each element provided on the detection surface of the distance sensor and the user is divided into a plurality of detection regions based on the distance, and the closest to the distance sensor from the plurality of detection regions A video reproduction device for recognizing the movement of the detected area and receiving a control signal generated based on the movement of the detection area,
A video playback apparatus comprising a playback control unit that controls playback of video information based on the control signal.   The video playback apparatus according to claim 1, wherein the playback control unit generates a control signal for controlling a playback position of the video information based on the received control signal.   The video reproduction apparatus according to claim 1, wherein the reproduction control unit changes a display range for displaying a reproduction position of the video information based on the received control signal.   The video playback apparatus according to claim 1, wherein the playback control unit controls the playback speed of the video information based on the received control signal.   The video playback apparatus according to claim 1, wherein the playback control unit controls a playback volume of the video information based on the received control signal.   The video playback apparatus according to claim 1, wherein the playback control unit controls a playback channel of the video information based on the received control signal.   The video playback apparatus according to claim 1, wherein the playback control unit displays an operation content selection cursor on the video information to be played back, and moves the cursor based on the received control signal. Distance information indicating the distance between each element of the distance sensor and the user is received, the distance information is divided into a plurality of detection areas based on the distance, and a depth image configured by the plurality of detection areas is acquired. A depth image acquisition unit;
An indication area detector that detects, as an indication area, a detection area closest to the distance sensor from the plurality of detection areas of the depth image;
A control signal generation device comprising: a motion recognition unit that recognizes the movement of the pointing region detected by the pointing region detection unit and generates a control signal based on the movement of the pointing region.   9. The control signal generation device according to claim 8, wherein the instruction area detection unit calculates a center of gravity of the depth from the depth image, and identifies the instruction area into a plurality of types based on the center of gravity of the depth.   The control signal generation device according to claim 8, wherein the motion recognition unit recognizes a motion in a depth direction and a planar direction of the indication area. Distance information indicating the distance between each element of the distance sensor and the user is received, the distance information is divided into a plurality of detection areas based on the distance, and a depth image configured by the plurality of detection areas is acquired. Depth image acquisition step;
An instruction area detecting step for detecting, as an instruction area, a detection area closest to the distance sensor from the plurality of detection areas of the depth image;
And a motion recognition step of recognizing the detected movement of the pointing area and generating a control signal based on the movement of the pointing area.

Images