JP2013142980A - Gesture recognition device and program therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a gesture recognition device capable of recognizing many different figures input with gestures using a small number of calculations.SOLUTION: The gesture recognition device creates an image representing a drawing track based on a set of positional coordinates of drawing points from the start to the end of the gesture stored in a finger position accumulation section 25, and takes out a minimum rectangular area representing the drawing track of the image using a frame pattern. After normalizing the size of the rectangular area image of the minimum area which includes the taken out drawing track, the figure represented by the drawing track is recognized using a recognition algorithm applying "SVM using DtB" based on the rectangular area image. The recognition processing of the figure is carried out as described below; i.e., an angle range of each of the upper side, the lower side, the right side and the left side is selected in order with respect to the center point of the drawing track, and distance information (information on the center point DtB) from the center point of the drawing track to the drawing track is calculated and input in a SVM model. The recognition processing of the figure is carried out by repeating the above recognition processing the figure represented by the drawing track.

Description

  The present invention relates to a gesture recognition device for recognizing a gesture based on the movement or shape of a finger or arm and a program thereof used for inputting channel information, control information, etc. from a remote location, for example, to a television receiver or a recording / playback device. About.

  Electronic devices equipped with cameras and infrared distance sensors that can be attached to television receivers can be obtained easily and inexpensively, and menu items for television receivers can be accessed by hand or finger gestures without using a remote controller. A method of selecting has been proposed. In this gesture input method, for example, the movement of the user's finger is imaged using a camera, and the movement locus of the finger is detected as a graphic from the captured image data, and the detected graphic is recognized. Yes.

  There are various methods for recognizing gestures, and one stroke writing gesture input method has been proposed as one of them. This one-stroke gesture input method inputs a gesture previously associated with a menu item to be input by a one-stroke operation, and has an advantage that a gesture can be input by an intuitive and natural movement (for example, non-patent literature). 1).

Ryosuke Aoki, Yutaka Karatsu, Masayuki Ihara, Atshiko Maeda, Tomoki Watanabe, Satoshi Kobayashi, Shingo Kagami: “Single-stroke gesture interface with a camera suitable for menu selection on a large display”, Human Interface Society Research Report 2010, VOL.12 , NO.9, 35-42.

  The single-stroke gesture input method identifies single-stroke straight lines and circles drawn by a human hand or finger, but there are actually a large number of items in the operation menu of the television receiver. It is necessary to be able to identify the figure. “Hidden Markov Machine” is generally used as a gesture recognition algorithm. However, this algorithm requires a large amount of calculation and is not suitable for menu operation of a television receiver that requires real-time performance.

  The present invention has been made paying attention to the above circumstances, and the object of the present invention is to make it possible to recognize various types of figures input by gestures in a short time with a simple algorithm, thereby improving real-time characteristics. The object is to provide a gesture recognition apparatus and a program thereof.

In order to achieve the above object, one aspect of the present invention is based on an imaging device that captures a motion of drawing a figure in a space by a gesture and outputs the image data, and image data output from the imaging device. In the gesture recognition device used in a system comprising a gesture recognition device having a function of recognizing a figure drawn by the gesture,
Means for capturing image data output from the imaging device at a predetermined cycle, and each time the image data is captured, position information of the drawing point by the gesture is detected from the image data, and the position of the detected drawing point is detected. Means for storing information in the storage means, means for creating information representing the drawing trajectory by the gesture based on a set of drawing point position information stored in the storage means, and figure recognition means. Then, the graphic recognition means obtains the center point or centroid point of the drawing locus based on the information representing the created drawing locus, and from the obtained center point or centroid point to the drawing locus. The distance is obtained at a predetermined angular interval, the shape of the drawing trajectory is identified based on the obtained set of distances, and the figure represented by the drawing trajectory based on the identified shape of the drawing trajectory The process which recognizes is performed.

  One aspect of the present invention is that the graphic recognizing means sets a plurality of angle ranges with respect to the drawing trajectory direction around the obtained center point or center of gravity, and each of the set plurality of angle ranges. The distance from the center point or the barycentric point to the drawing trajectory is obtained at a predetermined angular interval, the partial shape of the drawing trajectory is identified based on the obtained set of distances, and the identified drawing The process of inferring the figure represented by the drawing trajectory based on the partial shape of the trajectory is repeatedly executed by sequentially selecting the plurality of angle ranges until the figure represented by the drawing trajectory is recognized. Also features.

  Therefore, according to one aspect of the present invention, when recognizing a gesture figure from the locus of the drawing point, the shape of the drawing locus is identified from the distance from the center point or the center of gravity of the drawing locus to the drawing locus. A gesture figure is recognized based on the identified shape. For this reason, when “Hidden Markov Machine” is used as the gesture recognition algorithm or when the conventional “SVM using DtB” is used as it is, drawing is performed from the four sides of the rectangular frame used for normalizing the drawing trajectory. Compared to the case where the distance to the locus (4-side DtB information) is used, it is possible to recognize the gesture figure with a small amount of calculation processing, thereby shortening the time required for the figure recognition process.

  In addition, when recognizing a drawing figure from the locus of drawing points, a plurality of angle ranges are set around the drawing locus, and the partial shape of the drawing locus is identified based on the set of distances obtained and identified. Based on the result, the process of estimating the graphic represented by the drawing trajectory is repeatedly executed by selecting the plurality of angle ranges in order until the graphic represented by the drawing trajectory is recognized. That is, for each angle range set around the drawing trajectory, the drawing figure recognition processing is performed while narrowing down in stages. For this reason, it is possible to reduce the time required for the graphic recognition processing compared to the case where the distance from the center point or the center of gravity of the drawing trajectory to the drawing trajectory is always obtained for all angle ranges and the recognition processing of the drawing graphic is performed. It becomes.

  In other words, according to the present invention, it is possible to recognize various types of figures input by gestures in a short time with a simple algorithm, thereby providing a gesture recognition device that improves real-time characteristics.

1 is a schematic configuration diagram of a system including a gesture recognition device according to an embodiment of the present invention. The block diagram which shows the function structure of the information processing apparatus shown as a gesture recognition apparatus in FIG. The flowchart which shows the whole process sequence and process content by the information processing apparatus shown in FIG. The flowchart which shows the process sequence and processing content of the ON / OFF switching process of the 1 stroke writing gesture recognition function in the flowchart shown in FIG. The figure which shows the user's operation example for gesture function ON. The figure which shows the user's operation example for gesture function OFF. The flowchart which shows the process sequence and process content of the one-stroke writing gesture recognition process in the flowchart shown in FIG. The flowchart which shows the process sequence and process content of the one-stroke drawing identification process among the 1 stroke writing gesture recognition process procedures shown in FIG. The figure for demonstrating an example of the one-stroke drawing identification process shown in FIG. The figure for demonstrating an example of the one-stroke drawing identification process shown in FIG. The figure which shows an example of the one-stroke drawing figure recognized by the one-stroke writing figure identification process shown in FIG. The figure which shows step 1 of the 1st specific example of the one-stroke drawing figure identification process shown in FIG. The figure which shows step 2 of the 1st specific example of the one-stroke drawing figure identification process shown in FIG. The figure which shows step 1 of the 2nd specific example of the one-stroke drawing identification process shown in FIG. The figure which shows step 2 of the 2nd specific example of the one-stroke drawing identification process shown in FIG. The figure which shows Step 3 of the 2nd specific example of the one-stroke drawing identification process shown in FIG.

Hereinafter, an embodiment according to the present invention will be described with reference to the drawings.
[Constitution]
FIG. 1 is a schematic configuration diagram of an operation information input system using a gesture recognition device according to an embodiment of the present invention. In this system, a camera 4 is mounted on a television receiver 2 and an information processing device 3 is connected to the television receiver 2. The camera 4 images a gesture using the finger movement of the user 1 and outputs the captured image data to the television receiver 2 via a USB cable. Note that a light emitting marker 6 using, for example, an LED (Light Emitting Diode) is attached to the fingertip of the user 1 in order to make it easier to recognize the movement of the fingertip.

  The information processing device 3 has a function as a gesture recognition device and is configured as follows. FIG. 2 is a block diagram illustrating a functional configuration of the information processing apparatus 3. The information processing apparatus 3 includes, as functions necessary for performing gesture recognition processing, a finger position detection unit 10, a database 20, a one-stroke writing gesture recognition processing unit 30, a real-time processing generation unit 40, and a display screen processing unit. 50.

  The database 20 uses, for example, a hard disk or a NAND flash memory as a storage medium. As a storage area necessary for carrying out the present invention, a camera image storage unit 21, a one-stroke drawing storage unit 22, an operation content The storage unit 23 includes a display image / video storage unit 24, a finger position storage unit 25, and an SVM learning data storage unit 26.

  The camera image storage unit 21 is used to store image data acquired from the camera 4 by the finger position detection unit 10. The finger position accumulating unit 25 is used for associating and storing the finger pointing position detected from the image data accumulated in the camera image accumulating unit 21 and the time when the image data was captured. The one-stroke figure storage unit 22 stores in advance a one-stroke figure model used for recognizing a one-stroke figure. The operation content storage unit 23 stores in advance information representing the operation content corresponding to the recognized gesture type and direction. The display image / video storage unit 24 stores in advance image and video data to be displayed on the display screen in association with information representing the operation content stored in the operation content database 23. The SVM learning data storage unit 26 stores distance information obtained for each side of the cutout frame in the SVM learning mode and an SVM model created for each side based on the distance information.

The real-time processing generation unit 40 has a function of issuing an event signal, for example, every 33 msec using a timer.
The captured image acquisition unit 10 includes a camera image acquisition unit 11, acquires image data from the camera 4 each time an event signal is issued by the real-time generation unit, and stores the image data in the camera image storage unit 21. It has a function to make it.

The one-stroke writing gesture recognition processing unit 30 includes a one-stroke writing gesture recognition function ON / OFF switching unit 310 and a one-stroke writing gesture recognition processing unit 320.
Among these, the one-stroke writing gesture recognition function ON / OFF switching unit 310 includes a finger position detection unit 311 and a one-stroke writing gesture recognition function ON / OFF determination / switching unit 312. The finger position detection unit 311 detects the position coordinate of the drawing point of the marker 5, that is, the pointing position of the user's finger from the image data stored in the camera image storage unit 21, and uses the detected position coordinate of the drawing point. Stored in the finger position accumulating unit 25.

The one-stroke gesture recognition function ON / OFF determination / switching unit 312 has the following processing functions.
(1) With the function of the one-stroke writing gesture recognition processing unit 320 set to OFF, a gesture start point determination region (gesture start region) is set in the imaging range of the camera 4, that is, the gesture recognition region. Then, it is determined whether or not the drawing point of the light emitting marker 5 detected by the finger position detection unit 311 exists in the gesture start region. If it is determined that the drawing point exists, A function of setting the function to ON and setting a detection zone for detecting the end point of the gesture within the recognition area of the gesture.

  (2) In a state where the function of the gesture recognition processing unit 320 is set to ON, it is determined whether or not the drawing point of the light emitting marker 5 detected by the finger position detection unit 311 has moved out of the gesture recognition region. And a function of transitioning the function of the one-stroke writing gesture recognition processing unit from ON to OFF when it is detected that the object has moved out of the gesture recognition region.

Next, the one-stroke writing gesture recognition processing unit 320 includes a one-stroke writing gesture input determination unit 321, a one-stroke drawing graphic direction identification unit 322, and a one-stroke drawing graphic identification unit 323.
The one-stroke gesture input determination unit 321 is a position when it is detected that a drawing point detected from the image data has entered the gesture start area in a state where the function of the one-stroke gesture recognition processing unit 320 is ON. The coordinates are detected as the starting point of the gesture. Further, when the drawing point comes out of the gesture start area and returns to the detection zone, the position coordinate of the drawing point at this time is detected as the end point of the gesture.

  When the end point of the gesture is detected, the one-stroke figure direction identification unit 322 is a position coordinate that represents a trajectory until the drawing point of the gesture returns from the gesture start area to the detection zone. Based on the set, the barycentric position coordinates of the drawing trajectory are obtained, and the drawing direction with respect to the drawing starting point is identified based on the relationship between the barycentric position coordinates of the drawing trajectory and the drawing start point of the gesture.

  The one-stroke figure identification unit 323 recognizes a figure drawn by the gesture using a recognition algorithm unique to the present invention to which “Support Vector Machine using Distance to Boarders” (hereinafter referred to as SVM using DtB) is applied. Functions necessary for this graphic recognition processing include a graphic creation function 3231, a graphic region extraction function 3232, an extracted image normalization function 3233, a center point DtB information extraction function 3234, a mode identification function 3235, and learning data. An accumulation function 3236, an SVM model creation function 3237, and a figure recognition function 3238 are provided.

SVM using DtB itself is a known technique and is described in the following documents.
Lafuente-Arroyo, S .; Gil-Jimenez, P .; Maldonado-Bascon, R .; Lopez-Ferreras, F .; Maldonado-Bascon, S .; “traffic sign shape classification evaluation I: SVM using distance to borders”

  The figure creation function 3231 displays an image representing a drawing trajectory based on a set of drawing point position coordinates from the start to the end of a gesture stored in the finger position storage unit 25, that is, a figure drawn by a finger. The process which produces the image showing is performed.

  The graphic area extraction function 3232 uses a rectangular frame pattern composed of four sides to cut out a rectangular area having the minimum area including the drawing trajectory from the image representing the drawing trajectory created by the graphic creation function 3231. I do.

  The extracted image normalization function 3233 normalizes the image size of the rectangular area having the minimum area including the drawing locus cut by the graphic area extraction function 3232 according to a predetermined size.

  The center point DtB information extraction function 3234 is a representative point such as the center point or the center of gravity of the drawing locus (hereinafter referred to as the center point as an example) for the image of the rectangular area including the drawing locus normalized by the extracted image normalization function 3233. The distance information (center point DtB information) from the determined center point to the drawing figure is extracted.

  The mode identification function 3235 determines whether the processing mode of the one-stroke drawing graphic identification unit 323 is set to the learning mode or the recognition mode.

  The learning data storage function 3236 obtains distance information (center point DtB information) from the center point of the drawing locus to the drawing locus extracted by the center point DtB information extraction function 3234 in a state where the learning mode is set. Then, a process of storing the data as learning data in the SVM learning data storage unit 26 is performed.

  The SVM model creation function 3237 creates an SVM model for each predetermined angle range from the center point of the drawing trajectory based on the learning data stored in the SVM learning data storage unit 26. For example, an SVM model is created for an angle range (90 degrees) corresponding to each direction of the upper side, the lower side, the left side, and the right side of the square rectangular frame from the center point of the drawing locus. Then, a process of storing the created SVM model in the SVM learning data storage unit 26 is performed.

The figure recognition function 3238 has a function of executing the following processing in a state where the recognition mode is set.
(1) From the center point of the drawing trajectory, the upper side, the lower side, the left side, and the right side are selected in order (each 90 degree angle range) and extracted by the center point DtB information extraction function 3234 for each selected direction. The distance information from the center point of the drawing locus to the drawing locus (center point DtB information) is input to the SVM model in the corresponding direction stored in the SVM learning data storage unit 26, thereby A process of identifying the shape of a part included in the selected direction.

  (2) Every time the partial shape of the drawing trajectory in the one selected direction from the center point of the drawing figure is identified by the identification processing of (1), the entire drawing trajectory is based on the partial shape. Identify the shape, that is, the shape of the one-stroke drawing drawn by the gesture. A process of recognizing the type of the one-stroke figure by comparing the shape of the identified figure with the one-stroke figure model stored in the one-stroke figure storage unit 22.

  The display screen processing unit 50 includes a display screen processing unit 51. The display screen processing unit 51 responds from the operation content storage unit 23 based on the drawing direction of the gesture graphic and the type of graphic identified by the single stroke graphic direction identification unit 322 and the single stroke graphic identification unit 323, respectively. Reads information indicating the operation content. Then, based on the information representing the read operation content, the corresponding image or video display data is read from the display image / video storage unit 24, and the display of the television receiver 2 is displayed based on the display data. Update the screen.

  Note that each processing function of the captured image acquisition unit 10, the one-stroke gesture recognition processing unit 30, the real-time processing generation unit 40, and the display screen processing unit 50 has an application program stored in a program memory (not shown) as a central processing unit ( This is realized by causing a central processing unit (CPU) to execute.

[Operation]
Next, an operation of recognizing input information using a gesture by the information processing apparatus 3 configured as described above will be described.
FIG. 3 is a flowchart showing the overall processing procedure and processing contents. Here, a case will be described as an example in which electronic program guide (EPG) information is displayed on the display of the television receiver 2 and the user 1 performs a program selection operation by a gesture with respect to the EPG information.

(1) Real-time event generation processing When the gesture input mode is set, the real-time processing generation unit 40 is started in step S10, and an event is performed in step S11 every time the timer time T reaches a preset time, for example, 33 msec. A signal is generated. That is, in the gesture input mode, an event signal is generated with a period of 33 msec. Note that the timer time T is reset in step S12 each time an event signal is generated.

(2) Camera Image Acquisition and Finger Position Detection Processing When an event signal is generated from the real-time processing generation unit 40, first, the captured image acquisition unit 10 is activated in step S20, and control of the camera image acquisition unit 11 is performed. Below, the image data of the user 1 captured by the camera 4 is captured and stored in the camera image storage unit 21.

  When new image data is stored in the camera image storage unit 21, processing for detecting a bright spot image of the light emitting marker 5 from the image data is performed in step S311 shown in FIG. Then, the coordinates indicating the center of gravity position in the image data of the detected bright spot image are stored in the finger position accumulation unit 25 as a drawing point representing the position of the finger of the user 1. The above processing is repeated at the above 33 msec cycle. As a result, the finger position storage unit 25 stores a set of drawing point position coordinates representing the position of the finger of the user 1 and data indicating a figure drawn by the user 1 with the gesture. Is remembered as At this time, it is preferable that the aperture value (F value) of the camera 4 is set large to limit the amount of received light so that only the light emitted from the light emitting marker 5 is easily detected.

(3) ON / OFF switching process of single-stroke gesture recognition function Each time a new finger position coordinate is stored in the finger position storage unit 25, a single-stroke gesture of the single-stroke gesture recognition processing unit 30 in step S31 of step S30. The recognition function ON / OFF switching unit 310 is activated, and the following processing is executed under the control of the one-stroke writing gesture recognition function ON / OFF switching unit 310. Step S312 in FIG. 4 is a flowchart showing the processing procedure and processing contents.

(3-1) When the one-stroke writing gesture recognition function is turned ON First, it is determined in step S3121 whether or not the function of the one-stroke writing gesture recognition processing unit 320 is ON. Then, since it is not turned on yet, the process proceeds to step S3125, and a gesture start point determination region (gesture start region) is set in the imaging range of the camera 4, that is, the gesture recognition region. For example, as shown in FIG. 5A, a gesture start area E1 is set at the center of the gesture recognition area C. Then, it is determined whether or not the position coordinates of the drawing point of the light emitting marker 5 detected by the finger position detection process (step S311) are present in the gesture start area E1.

  In this state, the user 1 moves the position of the finger, and when the position coordinate P0 of the drawing point enters the gesture start area E0 as indicated by the broken line L1 as shown in FIG. The function of the one-stroke writing gesture recognition unit 320 is set to ON. At the same time, in step S3127, a detection zone E2 for detecting the end point of the gesture is set in the gesture recognition area C as shown in FIG. 5B.

  At this time, the size of the detection zone E2 is set to be larger than the gesture start area E1. In this way, even if the position of the user's finger by the user 1 returns to the detection zone E2 and then a position shift occurs near the boundary of the detection zone E2 due to shaking or the like, the position coordinates of the finger at this time, That is, the position of the end point can be stably kept in the detection zone E2 as the end point determination area. For this reason, even if a phenomenon such as so-called chattering occurs on the boundary of the detection zone E2, it is possible to prevent erroneous recognition from occurring in the drawing end determination (input determination) described later.

(3-2) When the one-stroke writing gesture recognition function is turned off With the one-stroke writing gesture recognition function turned on, the user 1 changes the position of the finger so as to temporarily stop the operation of the television receiver 2 by the gesture. For example, as shown by L2 in FIG. 6A, it is assumed that the camera 4 is moved outside the imaging range of the camera 4, that is, outside the gesture recognition area C. Then, the one-stroke writing gesture recognition function ON / OFF switching unit 310 determines whether or not the function of the one-stroke writing gesture recognition processing unit 320 is ON in step S3121. Since the function is ON here, the process proceeds to step S3122. To do. Then, it is determined whether or not the position coordinate of the drawing point of the light emitting marker 5 detected by the finger position detection process (step S311) has gone out of the gesture recognition area C.

  As a result of this determination, as shown by L2 in FIG. 6A, when the position coordinate of the drawing point of the light emitting marker 5 goes out of the gesture recognition area C, the one-stroke writing gesture recognition processing unit 320 performs step S3123. Transition the function to OFF. Thus, the user 1 can return the one-stroke gesture recognition function to OFF only by moving the finger position out of the imaging range of the camera 4.

  When the one-stroke gesture recognition function returns to OFF, when the next event signal is generated (33 msec), a gesture start area E1 is set in the gesture recognition area C as shown in FIG. 6B in step S3125. . That is, when the one-stroke gesture recognition function is turned off, a smaller gesture start area E1 is reset instead of the erased detection zone E2.

(4) Single-stroke gesture recognition processing With the single-stroke gesture recognition function turned on, the single-stroke gesture recognition processing unit 320 executes gesture recognition processing as follows. FIG. 7 is a flowchart showing the overall processing procedure and processing contents.

  That is, first, in step S321, the one-stroke writing gesture input determining unit 321 is activated, and under the control of the one-stroke writing gesture input determining unit 321, the drawing point by the light emitting marker 5 attached to the finger of the user 1 is the detection zone E2. It is determined in step S3211 whether or not it has entered. If the drawing point is not within the detection zone E2, a gesture flag indicating whether or not a gesture is being input is set to “TRUE” in step S3214, and each time an event occurs thereafter, the drawing point is set. Repeats the process of monitoring whether or not enters the detection zone E2.

(4-1) One-stroke drawing direction identification processing Now, it is assumed that a drawing point by the light emitting marker 5 attached to the finger of the user 1 in this state is within the detection zone E2. Then, after confirming that the gesture flag is “TRUE” in step S3212, first, in step S322, a one-stroke drawing direction identification process is executed.

  This one-stroke drawing direction identification process is performed as follows. That is, first, a set of finger position coordinates stored in the finger position accumulation unit 25 is read, and the barycentric position of the image representing the drawing locus based on the image representing the drawing locus of the finger represented by the set of finger position coordinates. Calculate the coordinates. Then, based on the relationship between the calculated gravity center position coordinates of the drawing trajectory image and the coordinates at the drawing start point, the drawing direction of the drawing trajectory with respect to the drawing start point is identified as up, down, left and right. .

(4-2) One-stroke Drawing Graphic Identification Processing Next, one-stroke writing graphic identification processing is executed as follows in step S323. FIG. 8 is a flowchart showing the processing procedure and processing contents.
That is, first, in step S3231, a set of drawing point position coordinates from the start to the end of the gesture stored in the finger position accumulating unit 25 is read, and this read drawing point position coordinate set is also stored. Then, an image representing the drawing trajectory, that is, an image representing the figure drawn by the light emitting marker 5 attached to the finger is created.

  In step S3232, a rectangular area having the minimum area including the drawing locus is cut out from the image data including the drawing locus created by the graphic creation function 3231 using a rectangular frame pattern including four sides. For example, if the drawing trajectory is “circle” or “ellipse” close to it, as shown in FIG. 9A, a rectangular area having the minimum area including the drawing trajectory V, that is, the drawing from the image data C including the drawing trajectory, is drawn. A rectangular area whose four sides are in contact with the locus V is cut out as shown in FIG.

  In step S 3233, the image size of the rectangular area having the minimum area including the drawing locus V cut out by the graphic area extracting function 3232 is normalized according to a predetermined size. For example, as shown in FIG. 9 (c), the cut drawing trace V and the image size of the frame pattern W used for cutting the drawing trace V are set to “true circle” and “square” each having 100 pixels × 100 pixels. Normalize to

  Next, in step S3234, the center point of the normalized image of the drawing locus V ′ is first obtained from the coordinate value of the drawing locus V ′. Instead of the center point, the center-of-gravity position coordinates previously calculated in the (4-1) one-stroke figure direction identification process may be used. Next, distance information (center point DtB information) from the obtained center point to the drawing locus is calculated for each predetermined angle based on the coordinate value of the drawing locus. For example, when the normalized drawing trajectory V ′ is “circle”, as shown in FIG. 10, the distance d from the center of the circle to the drawing trajectory V ′ is calculated at a constant angle.

  Subsequently, it is determined in step S3235 whether the processing mode of the one-stroke drawing graphic identification unit 323 is set to “learning mode” or “recognition mode”. If the “learning mode” is set as a result of this determination, distance information (center point DtB information) from the center point of the image including the drawing locus to the drawing locus extracted in step S3234 in step S3237, The data is stored in the SVM learning data storage unit 26 as learning data. FIG. 10 shows an example of distance information (center point DtB information) from the center to the drawing locus when the drawing locus is “circle”. In step S3238, the learning data stored in the SVM learning data storage unit 26 is used for the SVM model in turn for each direction (90 degree angle range) from the center point of the drawing trajectory toward the upper side, the lower side, the left side, and the right side. And the created SVM model is stored in the SVM learning data storage unit 26.

  In other words, in the “learning mode”, all or representative one-stroke figures used for the operation of the television receiver 2 are actually drawn by gestures, and the drawing trajectory centered on the drawing trajectory is cut out by the frame pattern. The distance information from the point to the drawing trajectory (center point DtB information) is extracted, and based on this distance information (center point DtB information), each direction from the center point of the drawing trajectory to the upper, lower, left, and right sides An SVM model corresponding to (angle range of 90 degrees) is created and stored in the SVM learning data storage unit 26.

  On the other hand, when the processing mode of the one-stroke figure identifying unit 323 is set to “recognition mode”, the figure recognition process is performed as follows. That is, first, for the image including the drawing trajectory normalized in step S3233, one of the directions from the center point to the upper side, the lower side, the left side, and the right side (angle range of 90 degrees) is selected. Next, in step S3234, distance information (center point DtB information) from the center point of the drawing locus to the drawing locus in the selected angle range is calculated at a constant angular interval. Subsequently, in step S3236, the set of the calculated distance information (center point DtB information) is input to the SVM model of the corresponding angle range stored in the SVM learning data storage unit 26, and thereby the drawing trajectory. The shape of the part included in the selected angle range is identified. Then, based on this identification result, the shape of the entire drawing trajectory, that is, the figure drawn by the gesture is identified, and if a figure having the same shape as this figure exists in the pre-stored one-stroke figure to be recognized, This one-stroke figure is taken as the recognition result.

  For example, assume that eight types of graphic pattern information shown in (1) to (8) of FIG. In this case, an angle range (90 degrees) in the upper side direction from the center point of the image including the drawing locus is first selected, and distance information (center point DtB information) from the center point of the drawing locus to the drawing locus in the angle range in the upper side direction. ) At regular angular intervals. Then, distance information (center point DtB information) in the calculated angle range in the upper side direction is input to the corresponding SVM model. As a result, it is possible to recognize one-stroke figures other than (2) and (4) in FIG.

  On the other hand, it is assumed that the one-stroke drawing figure cannot be recognized only by the distance information (center point DtB information) in the angle range in the upper side direction like the one-stroke drawing figures shown in FIGS. 11 (2) and (4). In this case, the angle range (90 degrees) in the lower side direction is then selected. Then, distance information (center point DtB information) from the center point of the drawing locus to the drawing locus in the selected angle range in the lower side direction is calculated at a constant angular interval. Then, the calculated distance information (center point DtB information) in the angle range in the lower side direction is input to the corresponding SVM model to identify the shape of the drawing trajectory, that is, the one-stroke figure drawn by the gesture. If, as a result of this identification, a one-stroke drawing figure whose shape matches the figure represented by the drawing locus is present in the one-stroke drawing figure to be recognized that has been stored in advance, this one-stroke drawing figure is taken as the recognition result. Thus, it is also possible to recognize a one-stroke drawing figure as shown in (2) and (4) of FIG. 11 that could not be recognized from the upper side of the rectangular area image.

  Also, if you cannot recognize a single stroke figure by referring to the distance information (center point DtB information) in the angle range in the lower side direction, select the angle range in the right side direction, and then select the angle range in the right side direction. Distance information (center point DtB information) from the center point of the drawing locus to the drawing locus is calculated at a constant angular interval. Then, the calculated distance information (center point DtB information) in the angle range in the right side direction is input to the corresponding SVM model, and recognition of the figure represented by the drawing locus is attempted. Furthermore, if a stroke is not recognized by referring to the distance information (center point DtB information) in the right-side angle range, the left-side angle range is selected last, and drawing is performed in this left-side angle range. The distance information (center point DtB information) from the center point of the locus to the drawing locus is calculated at a constant angular interval. Then, the calculated distance information (center point DtB information) in the angle range in the left-side direction is input to the corresponding SVM model to attempt to recognize the figure represented by the drawing locus.

  That is, until the one-stroke figure corresponding to the figure represented by the drawing trajectory can be recognized, the angle range in each direction of the upper side, the lower side, the right side, and the left side is sequentially selected from the center position of the drawing trajectory, and the above drawing is performed step by step Recognition processing of the figure represented by the locus is performed.

(5) Display Screen Update Processing When the one-stroke gesture recognition processing is completed, the display image processing unit 50 is activated in step S40. Under the control of the display image processing unit 50, the display image update processing is as follows. It is done as follows.

  That is, information representing the corresponding operation content from the operation content database 23 based on the drawing direction of the gesture graphic and the type of the graphic identified by the direction identification unit 322 and the single stroke graphic identification unit 323, respectively. Read out. Then, based on the information indicating the read operation content, the corresponding image or video display data is read from the display image / video storage unit 24, and the television receiver 2 of the television receiver 2 is read based on the display data. The display screen is updated.

(6) Specific example of one-stroke figure identification processing Here, the angle range in the top border direction is first selected from the angle ranges in the directions of the upper side, the lower side, the right side and the left side from the center position of the drawing trajectory. In the case of performing the identification process of the drawn figure, and when the figure identification by the angle range in the upper side direction is impossible, the case of performing the identification process of the drawn figure by selecting the angle range in the bottom side direction is taken as an example explain.

(6-1) When identifying a heart-shaped figure First, as shown in FIG. 12, the angle range in the upper side direction is selected from the angle ranges in the respective directions of the upper side, the lower side, the right side, and the left side. The distance information (center point DtB information) from the center point of the drawing trajectory to the drawing trajectory is calculated and input to the SVM model in the corresponding angular range in the upper side direction. At this time, if a plurality of upper side graphic models as shown in FIG. 13 are set as the SVM model in the angle range in the upper side direction, the shape matches the “heart” of these upper side graphic models. Therefore, the one-stroke drawing figure is recognized as “heart”.

(6-2) When identifying an inverted triangle figure First, as shown in 14, an angle range in the upper side direction is selected from the angle ranges in the upper side, the lower side, the right side, and the left side, and the angle range in the upper side direction is selected. The distance information (center point DtB information) from the center point of the drawing trajectory to the drawing trajectory is calculated and input to the SVM model in the corresponding angular range in the upper side direction. At this time, assuming that a plurality of upper side figure models as shown in FIG. 15 are set as the SVM model in the angle range in the upper side direction, the upper side shape model includes the upper side shape of the drawing figure. There is no match.

  Then, select the angle range in the lower side direction, calculate the distance information (center point DtB information) from the center point of the drawing locus to the drawing locus in this angle range in the lower side direction, and the corresponding angle in the lower side direction Fill in the SVM model in range. At this time, assuming that a plurality of lower-side graphic models as shown in FIG. 16 are set as the lower-side SVM model, the shape matches the “Upside-down triangle” among these lower-side graphic models. Therefore, the one-stroke drawing figure is recognized as “Upside-down triangle”.

  As described above, the angle range in the upper side direction is sequentially selected from the angle ranges in the respective directions of the upper side, the lower side, the right side, and the left side, so that the drawing graphic identification process is performed step by step. Compared to the case where the identification process is performed for all of the angle ranges, the amount of calculation for the identification process can be reduced to shorten the identification process time.

  As described in detail above, in this embodiment, an image representing a drawing trajectory is created based on a set of drawing point position coordinates from the start to the end of the gesture stored in the finger position storage unit 25. A minimum rectangular area including an image representing a drawing locus is cut out using a frame pattern. Then, after normalizing the size of the rectangular area image of the minimum area including the cut drawing trajectory, it is represented by the drawing trajectory using a recognition algorithm applying “SVM using DtB” based on the rectangular area image. Recognize the shape to be used. At this time, the recognition processing of the figure selects the upper side, the lower side, the right side, and the left side in an angle range in order with respect to the center point of the drawing locus, and from the center point of the drawing locus to the drawing locus in the selected angle range. The distance information (center point DtB information) is calculated and input to the corresponding SVM model to identify the figure represented by the drawing trajectory until the corresponding figure can be recognized. .

  Therefore, when recognizing a gesture figure from the locus of the drawing point, the partial shape of the drawing locus is identified from the distance from the center point of the drawing locus to the drawing locus, and the gesture is based on the identified partial shape. The process of recognizing the graphic is performed in stages until the gesture graphic is recognized while sequentially selecting the angle ranges in the respective directions of the upper side, the lower side, the right side, and the left side with respect to the center point of the drawing locus. For this reason, when “Hidden Markov Machine” is used as the gesture recognition algorithm or when the conventional “SVM using DtB” is used as it is, drawing is performed from the four sides of the rectangular frame used for normalizing the drawing trajectory. Compared to the case where the distance to the locus (4-side DtB information) is used, it is possible to recognize the gesture figure with a small amount of calculation processing, thereby shortening the time required for the figure recognition process.

  In addition, since the drawing trajectory and the frame pattern size are normalized prior to the graphic recognition process, even if errors occur in the detection of drawing points or the creation of the drawing trajectory, these errors are absorbed. Thus, the figure recognition process can be performed efficiently, and the time required for the figure recognition process can be further shortened.

  The present invention is not limited to the above embodiment. For example, in the above-described embodiment, the case of recognizing a one-stroke drawing figure drawn in space by the movement of a finger has been described as an example. However, the outline shape of the figure may be recognized. For example, the contour shape of the finger when the user performs a peace sign or an OK sign as a gesture may be recognized.

  In addition, the gesture recognition area C is set as the visual field range of the camera 4, but when the visual field range of the camera 4 is large, the gesture recognition area C smaller than the visual field range is set in the visual field range of the camera 4. Also good. In this case, the size and shape of the gesture recognition area C can be arbitrarily set.

  Furthermore, the function of the one-stroke writing gesture recognition processing unit 320 is forcibly shifted to the OFF state when the drawing point of the finger is continuously undetected for a preset time. However, the present invention is not limited to this, and when a part of the set representing the finger position trajectory stored in the finger position accumulating unit 25 is missing a predetermined length or longer, the one-stroke gesture recognition processing unit 320 The function may be forcibly shifted to the OFF state.

  Further, in the above-described embodiment, the case where the function of the gesture recognition device according to the present invention is provided in the information processing device has been described as an example. However, the same function may be provided in a television receiver, a set-top box or a video radar. May be provided. In addition, the gesture recognition apparatus can be variously modified and implemented without departing from the gist of the present invention.

  In short, the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine suitably the component covering different embodiment.

  DESCRIPTION OF SYMBOLS 1 ... User, 2 ... Television receiver, 3 ... Information processing apparatus, 4 ... Camera, 5 ... Luminescent marker, 10 ... Captured image acquisition unit, 11 ... Camera image acquisition part, 20 ... Database, 21 ... Camera image storage part , 22 ... one-stroke drawing graphic storage unit, 23 ... operation content storage unit, 24 ... display image / video storage unit, 25 ... finger position storage unit, 26 ... SVM learning data storage unit, 30 ... one-stroke writing gesture recognition processing unit, 310 ... one-stroke gesture recognition function ON / OFF switching section, 311 ... finger position detection section, 312 ... one-stroke gesture recognition function ON / OFF determination / switching section, 320 ... one-stroke gesture recognition processing section, 321 ... one-stroke gesture input determination , 322... One-stroke-drawn figure direction identification section, 323... One-stroke-drawn figure identification section, 3231. 3233 ... extracted image normalization function 3234 ... center point DtB information extraction function 3235 ... mode identification function 3236 ... learning data storage function 3237 ... SVM model creation function 3238 ... figure recognition function 40 ... real-time processing generation Unit: 50... Display screen processing unit, 51... Display screen processing unit.

Claims (3)

An imaging apparatus that captures a motion of drawing a figure in a space by a gesture and outputs the image data, and a gesture recognition having a function of recognizing the figure drawn by the gesture based on the image data output from the imaging apparatus A gesture recognition device for use in a system comprising the device,
Means for capturing image data output from the imaging device at a predetermined period;
Means for detecting position information of a drawing point by the gesture from the image data each time the image data is captured, and storing the position information of the detected drawing point in a storage means;
Means for creating information representing a drawing trajectory by the gesture based on a set of drawing point position information stored in the storage means;
Based on the information representing the created drawing trajectory, the center point or centroid point of the drawing trajectory is obtained, and the distance from the obtained center point or centroid point to the drawing trajectory is obtained at a predetermined angular interval. The graphic recognition for identifying the shape of the drawing trajectory based on the set of the obtained distances and performing the process of recognizing the graphic represented by the drawing trajectory based on the identified shape of the drawing trajectory And a gesture recognition device. The graphic recognition means
Means for setting a plurality of angle ranges with respect to the drawing trajectory direction around the obtained center point or center of gravity;
For each of the set angular ranges, a distance from the center point or the center of gravity to the drawing locus is obtained at a predetermined angular interval, and a portion of the drawing locus is obtained based on the set of obtained distances. The process of identifying the shape and inferring the figure represented by the drawing locus based on the partial shape of the identified drawing locus until the figure represented by the drawing locus is recognized. The gesture recognition apparatus according to claim 1, further comprising a unit that sequentially selects and executes a range.   The program which makes the computer of the said gesture recognition apparatus perform the process of each means with which the gesture recognition apparatus of Claim 1 or Claim 2 is provided.

Images