JP5358548B2 - Gesture recognition device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enhance recognition precision by reducing wrong recognition due to a position shift of an end point. <P>SOLUTION: Image data on a user imaged by a camera 4 is input, and an initial position of a finger is detected first from the input image data to set an end/start point input determination area of a first size E0 including the initial position in the image data. In a process of detecting a track of position coordinates of the finger from when position coordinates of the finger detected from the image data exit from the end/start point input determination area until when the position coordinates return into the end/start point input determination area, it is determined under the control of an end/start point input determination area enlargement/reduction part 32 whether the position coordinates of the finger are present within the end/start point input determination area, so that the end/start point input determination area is enlarged to a second size E1 larger than the first size E0 when it is determined that the position coordinates are present, or reduced to the first size E0 when it is determined that the position coordinates are not present. <P>COPYRIGHT: (C)2012,JPO&amp;INPIT

Description

  The present invention relates to a gesture recognition device for recognizing a gesture caused by movement of a finger or an arm, which is used for inputting channel information, control information, and the like from a remote place, for example, in a television receiver or a recording / playback apparatus.

  With the advent of network-compatible television receivers equipped with an external interface (for example, USB) that can be connected to a camera, services using networks and cameras, such as television phones, are becoming widespread. In addition, with the advent of digital signage that incorporates technology that recognizes the age and gender of people on the spot using image processing technology using a camera, a service that displays advertisements tailored to the age and gender of the people on the spot It is becoming popular.

  By the way, when using these services, the user needs to perform a menu selection operation or the like. As an input operation method, a gesture input method using a user's movement has been proposed. In this gesture input method, for example, the movement of a user's finger is imaged using a camera, the movement locus of a specific body part of the user is recognized from the captured image data by pattern recognition processing, and the recognition result is also obtained. In addition, menus and the like are pointed (see, for example, Patent Document 1).

Japanese Patent No. 4053903

However, the conventionally proposed gesture recognition processing technique has the following problems to be solved.
In other words, in the conventional technique, when the user moves the finger position to the center of the imaging screen of the camera as shown in FIG. 22B from the state before the input start shown in FIG. The starting point input determination area E is created based on the initial position Ps of the finger at that time. In this state, when the user draws a figure with a single stroke, as shown in FIG. 22 (c), the input determination flag is turned on when the position of the finger goes out of the start / end input determination area E, and thereafter FIG. As shown in d), the input determination flag is turned OFF when the finger position Pe returns to the end point input determination area E. And the figure represented by the locus | trajectory of the finger | toe obtained in the period when the said input determination flag is set to ON is recognized as a gesture.

  However, when the position of the finger returns to the start point input determination area E, the position slightly changes even if the user intends to stop the finger. Therefore, if the stop position of the finger, that is, the end point Pe is on the region boundary as shown in FIGS. 23A and 23B, the end point Pe position passes through the region boundary line many times, so that FIG. As shown in c), a phenomenon such as so-called chattering occurs, and as a result, erroneous recognition may occur.

Moreover, even if the user draws a one-stroke drawing figure, this figure may not be recognized as a gesture. There are the following two examples where this problem occurs.
(1) Even when the user intends to draw a circle, the trajectory of the finger does not pass through the end point input determination area E as shown in FIGS.
(2) Even if it is intended to draw a circle in the same manner, as shown in FIG. 24C, the position of the finger does not reach the starting point input determination area E and stops in front.

The present invention has been made paying attention to the above circumstances, and a first object of the invention is to provide a gesture recognition device in which the recognition accuracy is improved by reducing the erroneous recognition due to the positional deviation of the end point.
A second object is to provide a gesture recognition device that can reliably recognize a graphic even if the drawing position of the graphic by the user moves.

  A third object is to provide a gesture recognition device capable of suppressing the movement range of a graphic drawn by a user to a narrow range.

  In order to achieve the first object, a first aspect of the present invention includes an imaging device that captures a motion of drawing a figure in a space by a gesture and outputs the image data, and image data output from the imaging device In the gesture recognition device used in the system including the gesture recognition device for recognizing the graphic drawn by the gesture based on the image data, the image data output from the imaging device is captured, and the captured image data First, the initial position of the drawing point is detected, and the first size end point input determination area including the initial position is set in the image data. While the drawing point detected from the image data is outside the start point input determination area and then returns to the start point input determination area, the drawing point is being detected. Is determined to exist within the end point input determination area, and if it is determined to exist, the size of the end point input determination area is expanded to a second size larger than the first size, and exists When it is determined not to do so, the size of the start point input determination area is reduced to the first size.

  Therefore, according to the first aspect of the present invention, when the drawing point returns into the end point input determination region, the size of the end point input determination region is enlarged to the second size. That is, the position of the finger stopped on the boundary line of the first size exists inside the second size. For this reason, even if the end point of the drawing point is slightly misaligned due to the shaking or shaking of the user's finger, it is possible to stably keep the end point position within the start / end point input determination area. Even if such a phenomenon occurs, the occurrence of erroneous recognition can be reduced.

  In order to achieve the second object, a second aspect of the present invention provides an input for detecting passage of a locus of the drawing point when at least the drawing point is outside the start point input determination region. A means for setting a determination boundary line in the image data is further provided. Then, when recognizing the gesture, a first point for determining whether or not the locus of the drawing point does not pass the set input determination boundary line and the drawing point has returned to the start point input determination region. A determination process; a second determination process for determining whether the locus of the drawing point has passed the input determination boundary line a plurality of times; and the locus of the drawing point has passed the input determination boundary line once; When the determination result is “OK” in any of the third determination processes for determining whether or not the drawing point has returned to the end point input determination area, the locus of the drawing point is drawn by the gesture. It is designed to be recognized as a figure.

  Therefore, according to the second aspect of the present invention, the end point of the drawing point does not stop when the end point of the drawing point returns to the end point input determination area, and the trace of the drawing point does not stop when the user intends to draw a circle. Even if it doesn't pass through, or even if the position of the drawing point does not reach the end point input determination area and stops in front, it is possible to recognize the drawn figure as a recognition target gesture Become.

  In order to achieve the third object, the third aspect of the present invention further comprises correction means for correcting the position of the start point input determination area. Then, when the locus of the drawing point satisfies the determination condition defined by the second determination process, the position of the start point input determination area is moved to a position including the end point of the drawing point by the correction unit. Let Further, when the locus of the drawing point satisfies the determination condition defined by the first or third determination process, the position of the start point input determination area is moved closer to the initial position of the drawing point. It is made to let you.

  If it does in this way, whenever a user draws one figure, the position of the starting point input determination area will move to a position including the end point of the figure. For this reason, when drawing a plurality of figures continuously, it becomes possible to recognize the next figure more reliably. In the process of drawing a plurality of figures continuously, if the end point position of the figure returns to the initial position of the start point input determination area, the position of the start point input determination area is close to the initial position. Modified to return to For this reason, it becomes possible to restrict the movement range of the figure drawn by the user to a narrow range.

  In order to achieve the second object, the fourth aspect of the present invention further includes expansion means for expanding the start point input determination region and the input determination boundary line. Then, it is determined by the extension means whether or not the drawing point has been continuously stopped for a preset time or longer, the stop of the drawing point is detected by this determination processing, and the stop position is determined in the start point input determination area. When it is outside, at least one of the end point input determination area and the input determination boundary line is brought close to the stopped drawing point.

  In this way, when the end point position of the drawing point is stopped at a position outside the end point input determination area for a certain time or longer, the end point input determination area is expanded so as to approach the stopped drawing point, or The position of the input determination boundary line is moved so as to approach the drawing point that is stopped. For this reason, even if the trace of the figure drawn by the user does not pass through the end point input determination area, or the end point of the figure does not reach the end point input determination area, it stops at this point. Can be recognized as a gesture.

That is, according to the first aspect of the present invention, it is possible to provide a gesture recognition device that reduces recognition errors due to misalignment of the end point and increases recognition accuracy.
Further, according to the second aspect of the present invention, it is possible to provide a gesture recognition device capable of reliably recognizing a graphic even if the drawing position of the graphic by the user moves.
Furthermore, according to the 3rd viewpoint of this invention, the gesture recognition apparatus which can suppress the movement range of the figure which a user draws to a narrow range can be provided.

1 is a schematic configuration diagram of a system including a gesture recognition device according to an embodiment of the present invention. The block diagram which shows the function structure of the television receiver shown as a gesture recognition apparatus in FIG. The flowchart which shows the whole process sequence and process content by the television receiver shown in FIG. The flowchart which shows the process sequence and process content of the starting point input determination area expansion / contraction process among the whole process procedures shown in FIG. The flowchart which shows the process sequence and process content of an input determination boundary line creation deletion process among the whole process procedures shown in FIG. The flowchart which shows the process sequence and process content of the starting point input determination area | region / boundary line expansion process among the whole process procedures shown in FIG. The flowchart which shows the process sequence and process content of a gesture input process among the whole process procedures shown in FIG. The figure for demonstrating the specific example of the starting point input determination area expansion / contraction process shown in FIG. The figure which shows an example of the input determination horizontal boundary line creation area used for description of the input determination boundary line creation deletion process shown in FIG. FIG. 6 is a diagram illustrating an example of an input determination vertical boundary line creation region used for explaining the input determination boundary line creation deletion process illustrated in FIG. 5. The figure which shows an example of the input determination boundary line used for description of the input determination boundary line creation deletion process shown in FIG. The figure which shows an example of the expansion process of a starting point input determination area | region among the starting point input determination area | region and boundary line expansion processing shown in FIG. The figure which shows an example of the approach process of an input determination boundary line among the starting point input determination area | region / boundary line expansion processes shown in FIG. The figure which shows an example of the input determination process using the input determination vertical boundary line among the gesture input processes shown in FIG. The figure which shows an example of the input determination process using the input determination horizontal boundary line among the gesture input processes shown in FIG. The figure which shows an example of the input determination process which used the input determination boundary line and the start point input determination area | region among the gesture input processes shown in FIG. The figure which shows an example of the position correction process of the starting point input determination area | region among the gesture input processes shown in FIG. The figure which shows the other example of the position correction process of the starting point input determination area | region among the gesture input processes shown in FIG. The figure which shows the 1st example of the use of this invention. The figure which shows the 2nd example of the use of this invention. The figure which shows the 3rd example of the use of this invention. The figure which shows the basic gesture recognition procedure which concerns on this invention. The figure for demonstrating the subject of the conventional recognition means. The figure for demonstrating the subject of the conventional recognition means.

Embodiments according to the present invention will be described below with reference to the drawings.
[Constitution]
FIG. 1 is a schematic configuration diagram of an information input system using gesture recognition according to an embodiment of the present invention. In this system, a camera 4 is mounted on a television receiver 2. The camera 4 images a gesture using the finger movement of the user 1 and outputs the captured image data to the television receiver 2 via a USB cable. Note that a light emitting marker 6 using, for example, an LED (Light Emitting Diode) is attached to the fingertip of the user 1 in order to make it easier to recognize the movement of the fingertip.

  The television receiver 2 has a function as a gesture recognition device, and is configured as follows. FIG. 2 is a block diagram showing the configuration of the television receiver 2 together with the configuration of the camera 4.

  The camera 4 includes an imaging processing unit 41 and an image transmission unit 42. The imaging processing unit 41 performs a process of imaging a gesture of the user 1 every time a trigger signal is generated from a real-time event issuing unit 50 described later. The image transmission unit 42 performs a process of storing the image data obtained by the imaging process in the image information storage unit 21 in the storage unit 20 described later.

  The television receiver 2 includes a storage unit 20, a gesture recognition unit 30, a display image control unit 40, a real-time event issuing unit 50, and a finger position detection unit 60 as functions necessary for performing gesture recognition processing. I have.

  The storage unit 20 uses, for example, a hard disk or a NAND flash memory as a storage medium. As a storage function necessary for carrying out the present invention, an image information storage unit 21, a gesture command conversion table unit 22, and a display An image data storage unit 23, a graphic pattern storage unit 24, and a finger position information storage unit 25 are provided.

  The image information storage unit 21 is used for storing the image data output from the image transmission unit 42 of the camera 4. The gesture command conversion table unit 22 stores in advance information indicating the type of a graphic pattern meaning the input command and its starting point in association with a plurality of input commands to be recognized. The display image data storage unit 23 stores various display image data necessary for displaying an image corresponding to the display processing content meaning the input command. The graphic pattern storage unit 24 stores basic patterns of a plurality of graphic shapes to be recognized. The finger position information accumulating unit 25 is used for accumulating a position coordinate set representing a locus of the finger of the user 1 detected by a finger position detecting unit 60 described later.

The real-time event issuing unit 50 uses a timer, for example, to generate a trigger signal for operating the camera 4 and the like at a predetermined cycle.
Each time new image data is stored in the image information storage unit 21 at the trigger signal issuance period determined by the real-time event issuing unit 50, the finger position detecting unit 60 uses the image data to detect the brightness of the luminescent marker 6. A point image is detected as information representing the position of the finger, and the position coordinates of the bright spot image in the image data are stored in the finger position information storage unit 25 in the storage unit 20.

  The gesture recognition unit 30 includes a gesture input initial setting unit 31, a start / end point input determination area enlargement / reduction unit 32, and an input determination boundary line creation / deletion unit 33 as processing functions necessary for carrying out an embodiment of the present invention. And a starting point input determination area / boundary line extension unit 34 and a gesture recognition unit 35.

  The gesture input initial setting unit 31 is activated when the user 1 performs an input operation start operation using a gesture in a standby state. Then, the position of the finger of the user 1 detected by the finger position detection unit 60 is recognized as an initial position, and a process for setting an end point input determination region centered on the initial position in the image data is performed. The end point input determination area is for detecting the start point of a one-stroke drawing drawn by a gesture. When this end point is detected, it is determined that the input of one figure is completed.

  The end point input determination area enlargement / reduction unit 32 is in a state where the finger position detection unit 60 detects the finger position coordinates with the end point input determination area set by the gesture input initial setting unit 31. It is determined whether or not the detected finger position coordinates exist within the end point input determination area. When the finger position coordinates are outside the start point input determination region, the size of the start point input determination region is maintained at the first size initially set by the gesture input initial setting unit 31, and When the position coordinates enter the inside of the end point input determination area, a process for enlarging the size of the end point input determination area to a second size larger than the first size is performed.

  The input determination boundary line creation / deletion unit 33 performs a process of setting the input determination boundary line in the image data according to the detection position of the finger in the image data during the period in which the user 1 draws a one-stroke drawing by the gesture. Do. The input determination boundary line is composed of a horizontal boundary line and a vertical boundary line, and determines whether or not a finger drawing trajectory has passed in the image data. The input determination boundary line cooperates with the start point input determination area to detect the gesture. This is used to determine whether or not the input operation of one figure has been completed.

  The starting point input determination area / boundary line extension unit 34 determines whether or not the finger position coordinates detected by the finger position detection unit 60 have been continuously stopped for a predetermined time period or longer. It is determined whether or not the position coordinates are included in the starting point input determination area. Then, when it is determined that the detected position coordinates of the finger are continuously stopped for a certain time or more and are included in the start point input determination region, the end point of the figure drawn by the user 1 with the gesture is always Judging that the point input determination area has not been reached and gradually expanding the start point input determination area to bring it closer to the position coordinates of the stopped finger, or gradually moving the input determination boundary line A process of approaching the position coordinates of the stopped finger is performed.

  The gesture recognizing unit 35 is for recognizing a figure drawn by the user 1 based on the gesture based on the information representing the finger trajectory stored in the finger position information accumulating unit 25. The gesture recognizing unit 35 functions as a gesture input determining unit 351. A starting point input determination region position correction unit 352 and a drawing direction / shape recognition unit 353.

  The gesture input determination unit 351 uses the start point input determination area and the input determination boundary line to input one figure from the position coordinate set representing the finger trajectory stored in the finger position information storage unit 25. A process for determining whether or not the operation is completed is performed.

  The starting point input determination area position correction unit 352 inputs the starting point according to the gesture input determination result by the gesture input determination unit 351 so that the user 1 can continuously input a gesture within the smallest possible drawing range. A process of adaptively changing the position of the determination area is performed.

  The drawing direction / shape recognition unit 353 reads the position coordinate set of the finger from the finger position information storage unit 25 when the gesture input determination unit 351 determines that the input operation of one figure is completed, The shape and direction of the figure are recognized from the coordinate set. Then, the data representing the shape and direction of the graphic is transferred to the gesture command conversion table unit 22 and converted into a command representing the content of the gesture.

  The display image control unit 40 updates the display image data based on the command representing the content of the gesture converted by the gesture command conversion table unit 22. The updated display image data is output to a display (not shown) and displayed.

  Note that the processing functions of the gesture recognition unit 30, the display image control unit 40, the real-time event issuing unit 50, and the finger position detection unit 60 perform central processing of application programs stored in a program memory (not shown) in the storage unit 20. This is realized by causing a unit (Central Processing Unit: CPU) to execute.

[Operation]
Next, the operation of recognizing input information using gestures by the television receiver 2 configured as described above will be described.
FIG. 3 is a flowchart showing the overall processing procedure and processing contents. Here, a case will be described as an example in which electronic program guide (EPG) information is displayed on the display of the television receiver 2 and the user 1 performs a program selection operation by a gesture with respect to the EPG information.

(1) Gesture Input Initial Setting Process Under the control of the gesture input initial setting unit 31, the television receiver 2 first monitors whether or not the user has performed an operation for requesting the start of gesture input in step S10. The gesture input start requesting operation may be performed by an input device capable of interacting with the television receiver 2 such as a remote controller or by a user gesture. Here, the latter is used.

  It is assumed that the user shakes his / her hand toward the camera 4. Then, the gesture input initial setting unit 31 extracts a hand region from the image data obtained by the camera 4 by color matching processing, and the movement locus of the extracted hand region repeatedly shakes the hand a plurality of times left and right. If the pattern corresponds, the gesture at this time is recognized as a gesture input start request.

  When the gesture input start request is recognized, the gesture input initial setting unit 31 subsequently determines that the position of the finger at the start request operation is a position where the gesture input can be started (the screen center position indicated by a cross in FIG. 22A). It is determined whether or not it is in the peripheral area. As a result of this determination, if the finger position does not exist at a position where gesture input can be started, a guidance message that prompts the user to change the finger position is displayed on the display of the television receiver 2. Then, when the position of the user's finger moves to a position where gesture input can be started, as shown in FIG. 22 (b), an end point input determination centered on the screen center position and having the diameter set to the first size is performed. Region E is generated.

(2) Enlargement / reduction processing of end point input determination area When the setting of the end point input determination area is completed and the gesture input mode is set, the real-time event issuing unit 50 is activated in step S11, and the timer time is preset. A trigger signal is generated every time, for example, 33 msec. That is, when the gesture input mode is entered, a trigger signal is generated with a period of 33 msec.

  When the trigger signal is generated, the camera 4 performs an imaging process for the user (step S12), and image data obtained by the imaging process is stored in the image information storage unit 21 (step S13). When new image data is stored in the image information storage unit 21, the finger position detection unit 60 is activated in step S14, and the finger position detection unit 60 causes the bright spot image of the light emitting marker 6 from the image data. A process of detecting is performed. And the position coordinate in the image data of this detected bright spot image is memorize | stored in the finger | toe position information storage part 25 as information showing the position of a user's finger | toe (step S15). The above processing is repeated at the 33 msec cycle. As a result, a set of finger position coordinates is stored in the finger position information accumulating unit 25 as data indicating a figure drawn by the user.

  Now, every time a new finger position coordinate is stored in the finger position information accumulating unit 25, the start point input determination region enlargement / reduction unit 32 is activated in step S16, and the control of the start point input determination region enlargement / reduction unit 32 is performed. The following processing is performed under FIG. 4 is a flowchart showing the processing procedure and processing contents.

  That is, first, in step S30, it is determined whether or not the detected finger position coordinates are present in the end point input determination region. As a result of this determination, for example, the position coordinates of the finger are present in the initially set end point input determination area E0 as indicated by Ps in FIG. 8A, and the end point input determination area is not enlarged. In this case, in step S31, the size of the starting point input determination area is enlarged to the second size E1 larger than the first size E0.

  Subsequently, it is assumed that the position coordinates of the finger are outside the end point input determination area E1 as shown in FIG. Then, this state is detected in step S32, and the size of the start point input determination area is reduced to the first size E0 initially set by the gesture input initial setting unit 31 in step S33.

  Further, after the locus L of the figure is drawn, when the finger position coordinates return within the starting point input determination area E0, for example, as shown in FIG. 8C, this state is detected by step S30 and the process proceeds to step S31. Transition. In step S31, the size of the starting point input determination area is enlarged to the second size E1.

  Therefore, even if the position of the user's finger returns to the end point input determination area E0 and then a position shift occurs near the boundary of the area E0 due to shaking or the like, the position coordinates of the finger at this time, that is, the position of the end point Can be stably kept within the starting point input determination area E1. For this reason, even if a phenomenon such as so-called chattering occurs on the boundary of the start point input determination region, it is possible to prevent erroneous recognition from occurring in the drawing end determination (input determination) described later.

(3) Input Determination Boundary Line Creation / Deletion Processing Each time a new finger position coordinate is stored in the finger position information storage unit 25, the input determination boundary line creation / deletion unit 33 is activated in step S17. The following processing is performed under the control of the input determination boundary line creation / deletion unit 33. FIG. 5 is a flowchart showing the processing procedure and processing contents.

  That is, first, in step S40, it is determined whether or not the detected finger position coordinates are present in the horizontal boundary line creation region. As a result of this determination, if the finger position coordinates exist in one of the horizontal boundary line creation areas H1 and H2 shown in FIG. 9, for example, the input determination horizontal boundary line H0 passing through the center point of the image data in step S41. Is generated. Next, in step S42, it is determined whether or not the detected position coordinate of the finger exists in the vertical boundary line creation region. As a result of this determination, if the finger position coordinates exist in one of the vertical boundary line creation areas V1 and V2 as shown in FIG. 10, for example, the input determination vertical boundary line that passes through the center point of the image data in step S43. V0 is generated.

FIG. 11 shows an example in which both the input determination horizontal boundary line H0 and the input determination vertical boundary line V0 are generated. These input determination horizontal boundary line H0 and input determination vertical boundary line V0 are used for determining the passage of the finger trajectory in a gesture input process described later.
The generated input determination horizontal boundary line H0 and input determination vertical line V0 are deleted when the position coordinates of the finger go outside the boundary line creation region, that is, when the input point determination region E0 is entered. .

(4) Expansion of end point input determination area and input determination boundary line movement process Every time a new finger position coordinate is stored in the finger position information storage unit 25, in step S18, the start point input determination area / boundary line extension is performed. The unit 34 is activated, and the following processing is performed under the control of the starting point input determination region / boundary line extension unit 34. FIG. 6 is a flowchart showing the processing procedure and processing contents.

  That is, in step S50, whether or not the detected finger position coordinates are continuously stopped for a predetermined time or more and the detected finger position coordinates are included in the start point input determination area E0. It is determined whether or not. If it is determined that the detected finger position coordinates are continuously stopped for a certain time (for example, 2 seconds) and are included in the starting point input determination area E0, the figure drawn by the user 1 by the gesture Is determined that the end point has not reached the end point input determination area E0, and the process proceeds to step S51 where the end point input determination area is gradually expanded to approach the position coordinates of the stopped finger, Alternatively, the input determination boundary line is gradually moved so as to approach the stopped finger position coordinates.

  For example, when the end point input determination area is expanded, as shown by E0 to En in FIG. 12, a process of increasing the diameter by a certain size at a period of 33 msec is performed. For this reason, when the state where the finger is stopped continues for a while, the stop position coordinate of the finger enters the end point input determination region, and thus it is possible to determine that the input of one figure is completed.

  Further, when the input determination boundary line is expanded, the input determination horizontal boundary line H0 and the input determination vertical boundary line V0 are shown in increments of 33 msec as shown by H0 to Hm and V0 to Vm in FIG. It can be brought close to the finger stop position coordinates. As a result, the probability that the locus of the finger can be detected by the input determination horizontal boundary line H0 and the input determination vertical boundary line V0 is increased.

  When the position of the stopped finger starts to move again, the size En of the start point input determination area immediately before that and the positions of the input determination horizontal boundary line Hm and the input determination vertical boundary line Vm are retained. . If the movement of the finger stops for a certain time or more, the size of the start point input determination area is further expanded from the above En, and the positions of the input determination horizontal boundary line Hm and the input determination vertical boundary line Vm From Hm and Vm, the coordinates of the stop position of the finger can be made closer.

(5) Gesture Input Processing In step S19, the gesture recognition unit 35 is activated, and under the control of the gesture recognition unit 35, gesture input determination processing and recognition processing are performed as follows. FIG. 7 is a flowchart showing the processing procedure and processing contents.
That is, first, at step S60, it is determined whether or not the finger stop position coordinates have entered the initially set end point input determination region E1 or the end point input determination region expanded at step S18. Next, in step S61, it is determined whether or not the finger trajectory stored in the finger position information storage unit 25 has passed the input determination horizontal boundary line H0 or the input determination vertical boundary line V0 twice. Further, in step S62, the finger trajectory stored in the finger position information accumulating unit 25 passes the input determination horizontal boundary line H0 or the input determination vertical boundary line V0 once, and the stop position of the finger is expanded. It is determined whether or not the end point input determination area has been entered.

  When the determination condition is satisfied in any one of the above steps S60, S61, and S62, the process proceeds to step S63, and the drawing direction / shape recognition unit 353 performs a gesture recognition process. In this recognition process, a set of finger position coordinates stored in the round finger position information storage unit 25 is read, and based on the finger trajectory represented by the set of finger position coordinates, the user performs a gesture until an input is determined. This is done by recognizing the direction and shape of the drawn figure.

  For example, as shown in FIGS. 14A and 14B, when the finger trajectory L crosses the input determination vertical boundary line V0 at two locations C1 and C2, the process proceeds from step S61 to step S63. The direction and shape of the drawing figure at this time are recognized. Similarly, as shown in FIGS. 15A and 15B, when the finger trajectory L crosses the input determination horizontal boundary line V0 at two points C1 and C2, the process proceeds from step S61 to step S63. Thus, the direction and shape of the drawing figure at this time are recognized.

  Further, as shown in FIG. 16, although the finger trajectory L crosses only one place on the input determination horizontal boundary line V0, the stop position coordinate of the finger is in the end point input determination area En after the expansion process. In this case, the process proceeds from step S62 to step S63, and the direction and shape of the drawing figure at this time are recognized.

  The recognized direction and shape of the figure are converted into input commands by the gesture command conversion table unit 22, and the converted input commands are sent to the display image control unit 40.

  When the gesture recognition process is completed, the start point input determination region correction unit 352 is activated in step S64, and the position of the start point input determination region is corrected under the control of the start point input determination region correction unit 352. Processing is performed. This correction process is performed so that the user 1 can continuously perform gesture input within a drawing range as small as possible, and is performed as follows, for example.

That is, when the determination condition of step S61 is satisfied in the gesture recognition process, the start point input determination area E0 is changed to an area EM including the finger stop position coordinates as shown in FIGS. 17 (a) and 17 (b), for example. Moved. On the other hand, when the determination conditions of step S60 and step S62 are satisfied in the gesture recognition process, the position of the starting point input determination area EM is, for example, as shown by Ek in FIGS. 18 (a) and 18 (b). Correction is made so as to approach the initially set end point input determination area E0. However, at this time, the stop point coordinates of the finger are included in the starting point input determination area Ek after the movement. If the finger stop position coordinates have returned to the end point input determination area E0 initially set by the gesture input initial setting process (step S10), the position of the end point input determination area E0 is changed. do not do.
By doing in this way, it becomes possible to suppress the movement range of the figure drawn by the user within a compact range.

(6) Display Image Update Processing In step S20, the display image control unit 40 is activated, and display image update processing is performed as follows under the control of the display image control unit 40.
That is, the displayed EPG display image is updated according to the input command output from the gesture command conversion table unit 22. For example, in the state where the EPG display image G0 is displayed as shown in FIG. 19, as shown in FIG. 19A, a figure having a drawing direction “up” and a shape “circle” is input by a gesture. In this case, the cursor position (shaded display) on the EPG display image is shifted upward by one step as indicated by G1. Similarly, when a “circle” figure is input in the “right”, “left”, and “down” directions by the gestures as shown in FIGS. The cursor position (shaded display) on the display image is shifted by one step in the "right", "left", and "down" directions as indicated by G4, G3, and G2.

  In addition, in the state where the EPG display image G0 is displayed as shown in FIG. 20, as shown in FIG. 20A, a figure having a drawing direction “up” and a shape “line” is input by a gesture. In such a case, the EPG display image is updated to "one day ago" as indicated by G5. Similarly, when a graphic having a drawing direction “down” and a shape “line” is input by a gesture, the EPG display image is updated to “one day later” as indicated by G6.

  Further, in the state where the EPG display image G0 is displayed as shown in FIG. 21, a figure having a drawing direction of “up” and a shape of “triangle” is inputted by a gesture as shown in FIG. In such a case, the EPG display image is updated to "one week ago" as indicated by G7. Similarly, when a figure having a drawing direction of “down” and a shape of “triangle” is input by a gesture, the EPG display image is updated to “one week later” as indicated by G8.

  As described above in detail, in this embodiment, the user's image data captured by the camera 4 is captured, the initial position of the finger is first detected from the captured image data, and the first size E0 including this initial position is detected. Is set in the image data. In the process of detecting the trajectory of the finger position coordinate from when the finger position coordinate detected from the image data goes out of the start point input determination area to when it returns to the end point input determination area, Under the control of the point input determination area enlargement / reduction unit 32, it is determined whether or not the detected finger position coordinates exist within the start point input determination area. The size of the determination area is enlarged to a second size E1 larger than the first size E0, and when it is determined that it does not exist, the size of the start point input determination area is reduced to the first size E0. ing.

  Accordingly, when the position coordinates of the finger return to the end point input determination area, the size of the end point input determination area is enlarged to the second size E1. In other words, the position of the finger stopped on the boundary line of the first size E0 exists inside the second size E1. For this reason, even if the end point of the finger position coordinate is slightly misaligned due to the shaking or shaking of the user's finger, it is possible to stably keep the end point position within the start point input determination area. Therefore, even if a phenomenon such as so-called chattering occurs, the occurrence of erroneous recognition can be reduced.

In this embodiment, when the position coordinates of the finger are outside the start point input determination area, the passage of the locus of the position coordinates of the finger is detected under the control of the input determination boundary line creation / deletion unit 33. An input determination horizontal boundary line H0 and an input determination vertical boundary line V0 are set in the image data. And when judging gesture input,
(1) The trajectory of the finger position coordinates does not pass through the set input determination horizontal boundary line H0 and the input determination vertical boundary line V0, and the finger position coordinates return to the end point input determination area. Processing to determine whether or not.
(2) A process for determining whether or not the locus of the position coordinates of the finger has passed the input determination horizontal boundary line H0 or the input determination vertical boundary line V0 twice.
(3) Whether or not the trajectory of the finger position coordinate has passed the input determination horizontal boundary line H0 or the input determination vertical boundary line V0 once, and the finger position coordinate has returned to the start point input determination area. Processing to determine.
When the determination result is “OK” in any of these determination processes, the locus of the position coordinates of the finger is recognized as a figure drawn by the gesture.

  Therefore, even if the user intends to draw a circle, if the trajectory of the position coordinate of the finger does not pass the end point input determination area, or the finger position coordinate does not reach the end point input determination area and stops in front. Even if it is closed, the figure drawn at this time can be recognized as a gesture to be recognized.

  Further, in this embodiment, when the trajectory of the finger position coordinates satisfies the determination condition defined by the determination process (2) under the control of the start point input determination region position correction unit 352, the end point input is performed. The position of the determination area is moved to a position including the end point of the position coordinates of the finger. In addition, when the locus of the finger position coordinates satisfies the determination condition defined by the determination process of (1) or (3), the position of the start point input determination area is set to the initial position coordinates of the finger. The position is returned to a position close to the position.

  Therefore, every time the user draws one figure, the position of the start point input determination area moves to a position including the end point of the figure. For this reason, when drawing a plurality of figures continuously, it becomes possible to recognize the next figure more reliably. In the process of drawing a plurality of figures continuously, if the end point position of the figure returns to the initial position of the start point input determination area, the position of the start point input determination area is close to the initial position. Modified to return to For this reason, it is possible to restrict the movement range of the figure drawn by the user to a compact range.

  Furthermore, in this embodiment, it is determined whether or not the finger position coordinates are continuously stopped for a preset time (for example, 2 seconds) under the control of the start point input determination region / boundary line extension unit 24. When the finger position coordinate is stopped for the set time or longer and the stop position is outside the start point input determination region, the stop point input position and the input determination boundary line are set to the position of the stopped finger. The coordinates are gradually approached.

  Therefore, when the end point of the finger position coordinate does not reach the start point input determination area and stops for a certain period of time, the size of the start point input determination area gradually expands to approach the stop finger position coordinate. Is done. Further, the position of the input determination boundary line gradually moves so as to approach the position coordinates of the stopped finger. For this reason, even if the trace of the figure drawn by the user does not pass through the end point input determination area, or the end point of the figure does not reach the end point input determination area, it stops at this point. Can be recognized as a gesture input.

  The present invention is not limited to the above embodiment. For example, in the embodiment, when the finger position coordinates are stopped for a certain time (for example, 2 seconds) or more outside the start point input determination area, the end point input determination area is gradually brought closer to the stop position of the finger position coordinates. I made it. However, if the distance between the stop position of the finger position coordinates and the center position of the start point input determination area is calculated before the start point input determination area is brought closer, this separation distance is within a predetermined threshold value. If the separation distance exceeds the threshold value, it is determined that the figure drawn by the user is incorrect, and the start point input determination area is brought closer to the stop position of the finger position coordinates. It may not be possible.

  In the above embodiment, when the position coordinates of the finger go out of the start point input determination area and return to the end point input determination area again, the size of the start point input determination area is increased to a large size. However, the present invention is not limited thereto, and the start point input determination area is moved so that the center position of the start point input determination area matches or approaches the position of the position coordinate of the finger returned to the start point input determination area. Also good.

Furthermore, in the above-described embodiment, the case where the EPG information displayed on the television receiver 2 is operated by the gesture has been described as an example. However, the display screen of the personal computer or the display screen by the video projector is operated by the gesture. Also good.
In addition, the type and configuration of the gesture recognition device, the processing procedure and processing content, the shape of the figure, and the like can be variously modified and implemented without departing from the gist of the present invention.

  In short, the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine suitably the component covering different embodiment.

  DESCRIPTION OF SYMBOLS 1 ... User, 2 ... Television receiver, 3 ... Communication line, 4 ... Camera, 6 ... Light emission marker, 20 ... Memory | storage unit, 21 ... Image signal storage part, 22 ... Gesture command conversion table part, 23 ... Display image data Accumulation unit, 24 ... Graphic pattern storage unit, 25 ... Finger position information accumulation unit, 30 ... Gesture recognition unit, 31 ... Gesture input initial setting unit, 32 ... End point input determination area enlargement / reduction part, 33 ... Input determination boundary line creation Deletion unit 34... Start point input determination region / boundary line extension unit 35... Gesture recognition unit 351... Gesture input determination unit 352... Start point input determination region position correction unit 353. Display image control unit 41 Image capturing unit 42 Image transmission unit 50 Real-time event issuing unit 60 Finger position detection unit

Claims (4)

An imaging device that captures a motion of drawing a figure in a space by a gesture and outputs the image data; and a gesture recognition device that recognizes the graphic drawn by the gesture based on the image data output from the imaging device. The gesture recognition device used in a system comprising:
Means for capturing image data output from the imaging device;
Means for detecting an initial position of a drawing point from the captured image data, and setting an end point input determination region of a first size including the detected initial position of the drawing point in the image data;
In a state where the end point input determination area is set, a drawing point trajectory from when the drawing point detected from the image data comes out of the end point input determination area to the inside of the end point input determination area Gesture recognition means for detecting and recognizing the detected locus as a figure drawn by the gesture;
During the detection period of the locus of the drawing point, it is determined whether or not the drawing point exists in the start point input determination region. If it is determined that the drawing point exists, the size of the start point input determination region is set to the first point. An enlargement / reduction control means for enlarging to a second size larger than 1 and reducing the size of the start point input determination area to the first size when it is determined that it does not exist. Gesture recognition device. Means for setting, in the image data, an input determination boundary line for detecting passage of a locus of the drawing point when at least the drawing point exists outside the start point input determination region;
The gesture recognition means includes:
First determination means for determining whether a locus of the drawing point does not pass through the set input determination boundary line and whether the drawing point has returned to the start point input determination region;
Second determination means for determining whether or not the locus of the drawing point has passed the input determination boundary line a plurality of times;
Third determination means for determining whether the locus of the drawing point has passed the input determination boundary line once and whether the drawing point has returned to the start point input determination region;
Means for recognizing the locus as a figure drawn by a gesture when the locus of the drawing point satisfies a determination condition defined by any of the first, second, and third determination means. The gesture recognition device according to claim 1, wherein: A correction means for correcting the position of the start point input determination area;
The correction means includes
Means for moving the position of the start point input determination region to a position including the end point of the drawing point when the locus of the drawing point satisfies the determination condition defined by the second determination unit;
Means for moving the position of the start point input determination area so as to approach the initial position of the drawing point when the locus of the drawing point satisfies the determination condition defined by the first or third determination means; The gesture recognition apparatus according to claim 2, further comprising: An expansion means for extending the start point input determination area and the input determination boundary line;
The expansion means includes
Means for determining whether or not the drawing point has been continuously stopped for a preset time;
Means for bringing at least one of the start point input determination area and the input determination boundary line closer to the stopped drawing point when the stop of the drawing point is detected and the stop position is outside the start point input determination area The gesture recognition device according to claim 2, further comprising:

Images