JP2003346162A - Input system by image recognition of hand - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system capable of inputting an instruction or the like by recognizing the shape of a hand. <P>SOLUTION: Two kinds of hand shape recognition images, namely, a hand region extraction image (S204) and a hand region narrow line image (S206), are formed from an input image (S202) by a camera or the like. An apparent characteristic of the hand such as thickness of a finger is extracted from the hand region extraction image as an image characteristic, and a skeletal characteristic of the hand such as a direction of the hand or a root coordinate of the finger is extracted similarly from the hand region narrow line image. First of all, the number of fingers is recognized (S208), and when the number is 0.5 or more, the direction of the hand is determined (S210) by using the acquired image characteristic and a structural characteristic of the hand, and the finger region is extracted (S212). Then, the image characteristic is acquired (S214) by analyzing the acquired finger region extraction image and the hand shape recognition image, and it is recognized which finger is stretched (S216). Thirty- two kinds of hand shapes can be recognized in this way. <P>COPYRIGHT: (C)2004,JPO

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a human interface for computer input, and more particularly to a system which can be input by identifying a standing finger from an image of a palm.

[0002]

[Technical background] In recent years, with the development of computers, etc.,
It has made it possible to make the machinery around humans more accessible and more useful. Inventions such as virtual reality, multimedia and human interface are representative of this movement. The human interface is an invention aiming at how information can be exchanged between humans and machines in an efficient and easy-to-understand form, and a human-friendly dialogue between humans and computers In order to achieve this, it is very important to make machines understand human movements and postures. For example, in order to construct a virtual world, posture estimation for the entire human body is necessary. If remote operation of a machine is assumed, posture estimation of fingers is important. In addition, if cameras can be installed in various facilities and devices and the behavior of the user can be quantified, it will help human interface.
Among the motion recognition methods, the method of using the shape of a hand that is a part of the human body as an input means leads to realizing an interface with various devices in a natural form, and has a very important meaning. In addition, the hands are the most dexterous parts of the human body, and their shapes and movements are very abundant.
Therefore, the hand is an extremely important part in synthesizing a human image and recognizing human motion.

[0003] As a technique using the hand shape, there are a contact technique using a physical sensor such as a data glove and a marker, and a non-contact technique which performs recognition from image information.
Many contact-type methods using sensors and markers have been proposed, but the stability and high-speed processing when extracting feature points of the input image are high, but the annoyance of the observer with the physical sensor mounting is high. There is a problem in building a more natural system. On the other hand, many methods have been proposed for performing recognition from image information without using sensors or markers. As a method of performing recognition based on image information, there is a method of extracting a feature such as an edge component such as a finger or a contour from an input image, and performing shape estimation by matching the obtained component with a learned three-dimensional model. . However, these methods require a large amount of calculation at the time of matching and have a problem in processing time, and it is impossible to construct a system that operates in real time using hands as input means. In some cases, the position of the center of gravity of the hand, the direction of the hand, and the number of fingers can be recognized in real time.However, these recognize only basic information obtained from the shape of the hand, I can't tell if it is. There are systems that can recognize fingers in real time, but features are extracted and recognized using multiple cameras, creating a natural man-machine-human interface that uses hands as input devices for computers. There is no way to do it. However, in order to construct a human interface that is natural and easy for the user to use, a method that uses image information that does not use a physical sensor and does not impose a burden on the subject is more useful.

[0004]

SUMMARY OF THE INVENTION It is an object of the present invention to provide a computer by recognizing which one of five fingers stands from image information of a hand without using a sensor or a marker. And the like to provide a system capable of inputting the information.

[0005]

In order to achieve the above object, the present invention relates to an input system for recognizing a hand image, comprising: an image input means for inputting an image of a hand; Hand region extracting means for obtaining a hand region extracted image in which a hand region portion is filled from an image, hand region fine line image generating means for obtaining a hand region thin line image from the hand region extracted image, the hand region extracted image and the hand region A hand shape recognizing means for recognizing a standing finger from a fine line image and specifying a hand shape is provided, and 32 shapes are specified with one hand depending on which finger is standing. The hand region extracting means extracts a skin color portion from the image of the hand, performs a smoothing process on the skin color portion, tracks an outline of the skin color portion after the smoothing process, and has a maximum length of the outline. After a coordinate series is extracted and a smoothing process is performed on the extracted contour line, the inside of the contour line is painted out to obtain a hand region extracted image. Further, an index recognizing means for recognizing the number of fingers from the contour line is provided.
When the number of fingers is 0 or 5, the hand shape recognizing unit performs the process by the hand shape recognizing unit. It can also be excluded. The hand shape recognition means obtains a hand direction vector from the hand area thin line image, scans the hand area extracted image perpendicular to the obtained hand direction vector, obtains a hand run length, It is also possible to recognize a hand shape by extracting a finger region based on the length and extracting a finger portion from the hand region thin line image using the extracted finger region. The hand shape recognizing means may normalize and evaluate parameters for recognizing the hand shape. The present invention also includes a program for causing a computer system to configure the input system having the functions described above and a recording medium storing the program.

[0006]

Embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 shows an example of a system configuration according to an embodiment of the present invention. The input system 100 based on hand image recognition includes a computer body 110 having a built-in CPU and a hard disk, a display device 120 such as a CRT, a keyboard 112, a pointing device 114 such as a mouse, and the like. Etc. are connected to the image input device 130.
From the image input device 130, an image of a hand can be captured into a computer system and image processing can be performed. FIG. 2 is a flowchart showing a hand shape recognition process performed by this system. The hand shape recognition process will be described below with reference to this flowchart. First,
An image of a hand shape is acquired in the system by one camera 130 (S202). In this embodiment described below, there are the following constraints on the acquired input image. 1. The input image is a color image represented by the RGB color system. 2. In order to extract a flesh-color area to be a hand area, the hand is the largest flesh-color area in the image, and the hand area and other flesh-colored objects are prevented from overlapping. 3. Orient the palm as parallel to the camera lens as possible, and know whether the palm or the back of the hand is facing the camera lens. 4. All the regions that become hands are represented by the skin, and the region beyond the wrist does not intersect the image frame. 5. Make sure your fingers do not overlap. 6. It is assumed that the finger is in a stretched or unstretched state.

There are 32 types of hand shapes that satisfy these constraints and can be expressed by a human using a finger. The present invention recognizes which finger is extending for all 32 types of shapes. Details of the 32 hand shapes are shown in FIGS. 3 and 4 (the background is unified in black). In FIG. 3 and FIG. 4, for each hand shape, the position of the finger is represented by a binary number, the state where the finger is standing is 1 and the state where the finger is folded is 0 (the number in parentheses is shown in parentheses). (Decimal number) is also shown. As shown in FIG. 3, there is a one-to-one relationship between 32 kinds of hand-shaped states and binary numbers by fingers. Therefore, if the hand shape shown in FIG. 3 can be recognized, the corresponding binary number (decimal number) can be input.

FIG. 5 shows an example of an input image. FIG.
As shown in (1), an actual input image includes a background and the like that are not related to hand recognition. Next, an image for extracting a hand region is generated from the input image shown in FIG. 5 (S204). FIG. 6 shows a flowchart of an example of processing for generating an image for recognition. As shown in FIG. 6, the generation of the hand region extracted image is performed, for example, in the following procedure. 1. The input image represented by the RGB color system is converted into the HSV color system that is less affected by light than the RGB color system (S304). 2. A skin color component is extracted from the HSV color system image (S3
06), the image is binarized into a skin color component and other components to generate a binary image (S308). 3. In the obtained binary image, a region having the largest area is extracted (S310), and a region subjected to diffusion / shrinkage as a smoothing process (S322) is set as a hand region candidate.

The image obtained by the above procedure may include missing information or noise depending on the shooting environment or the like, and may cause inconvenience in the subsequent thinning processing. Therefore, the contour of the hand region candidate is also smoothed. The contour is smoothed by the following procedure. 1. The contour of the hand region candidate is extracted, and the largest contour line is set as the contour of the hand region candidate (S324). 2. The contour of the obtained hand region candidate is approximated by a polygon using the line segment approximation parameter ε, and the region approximated by the polygon is painted (S326). Here, the line segment approximation parameter ε is set for the area of the hand region candidate. 3. A figure obtained by performing a painting process on the polygon approximated area is determined as a hand area, and this image is used as a hand area extracted image (S328). The details of each process are described below.

(Extraction of Hand Area Candidate: S302) First, in order to extract a location which is a hand area in an image, skin color is extracted in the embodiment of the present invention. However, a color image represented by the RGB color system, which is an input image, is difficult for a person to directly estimate color information, and is thus difficult to handle. Therefore, it is desirable to convert to a color system having attributes such as lightness, saturation, and chromaticity that can be easily handled by human beings. Such a color system includes an HSV color system and L ^* a ^* b ^*.
There are various color systems such as a color system, but the HSV color system can stably extract skin color. Therefore, in the embodiment of the present invention,
The color system was converted from the RGB color system to the HSV color system, and the skin color was extracted.

(HSV Color System) The HSV color system has a hue H (hue) representing the type of color and a saturation S (s) representing the vividness of the color.
aturation), and lightness V (valu) representing the degree of brightness
e) consists of three elements. The conversion method nonlinearly converts the RGB color system. However, the value range of R, G, B, S, and V is [0, 1], and H has a value of [0, 2π]. How to convert from RGB color system to HSV color system
For example, "Image Analysis Handbook" supervised by Takagi and Shimoda (published by The University of Tokyo, pp. 485-491, 1991)
It is described in.

(Extraction of Hand Area) In the present invention, the threshold of the skin color area is set as 0.11 <H <0.22 0.2 <S <0.5, and the skin color is extracted using this threshold. Do (S
306). Binarization processing (S308) is performed to separate the hand region from the background, and the maximum area of the hand region is extracted.
At this time, the maximum area of the hand region, that is, the number of pixels is 1000
If it is smaller, it is determined that there is no hand region in the image, and the process is terminated. It is assumed that the image being used has a size of 320 pixels horizontally and 240 pixels vertically. For the obtained silhouette image of the largest part of the hand area, diffusion and
The result of the contraction process (S322) is set as a hand region candidate (see FIG. 7). As the algorithm of the diffusion / shrinkage processing, for example, Hasegawa et al., “Basic Techniques of Image Processing-Introduction to Technology-” (published by Gijutsu Hyoronsha 1986) can be used.

(Determination of Hand Region) The obtained hand region candidate is subjected to a contour tracing process to create a contour extraction image (S324: see FIG. 8), and the length of the contour is the maximum. Extract coordinate series. From this coordinate series, polygon approximation (S326) is performed as contour line smoothing processing. A piecewise linear approximation method is used for polygon approximation. (Piecewise linear approximation method) The piecewise linear approximation method is realized by an algorithm of the following procedure in which a certain line segment approximation parameter ε is introduced (Hasegawa et al., “Basic Techniques of Image Processing-Introduction to Techniques-” ( Technical Review, published in 1986)).
FIGS. 9A and 9B show the principle of operation.
The processing is performed in the order of (c) and (d). 1. First, from the tracking start point A of the contour graphic, a straight line segment is connected in the order of tracking, a point C farthest from the straight line segment is found, and if the maximum distance h is larger than the approximate value ε, two points are determined at that point. It is divided into a straight line AC and a straight line CB. If the maximum distance h is smaller than the approximate value ε, the procedure ends. 2. Next, for each of the divided partial lines AC and BC, 1
The division is repeated using the same procedure as described above. In the example of FIG. 9B, since the point D where the maximum distance h is larger than the approximate value ε is found again on the straight line AC, it is divided into the straight line AD and the straight line DC (see FIG. 9C). ). 3. If the maximum distance does not exceed ε with respect to any of the divided straight line segments, the dividing operation is stopped (see FIG. 9C), and the procedure is terminated as a line figure determined by a polygon connected by the dividing points (see FIG. 9C). 9 (d)). In the piecewise approximation method, the degree of smoothness can be adjusted by appropriately setting the segment approximation parameter value ε.

However, the distance between the hand and the camera (depth value)
Therefore, appropriate approximation cannot be performed with certain line segment approximation parameters because the area of the hand region on the frame is different. Therefore, in the embodiment of the present invention, each line segment approximation parameter is determined based on the area of the hand region, and appropriate approximation for reducing the influence of the depth value can be performed. Table 1 shows examples of approximate parameters used in the embodiment of the present invention.

[Table 1] Contour extraction image realized using piecewise straight line approximation method,
FIG. 10 shows an example of a polygon approximate image.

(Determination of Hand Region) The obtained polygon approximate image is subjected to a polygon approximate image filling process (S326) using a scan conversion algorithm, and the obtained image is obtained (see FIG. 11). Is determined as the hand region extraction image (S328). Scan conversion is performed, for example, by Keiichi Abe “Drawing / Document Image Processing for OA 4 Tracking of Figures” (Image Lab, pp. 52-5).
5, 1999). (Generation of Hand Region Thin Line Image) A hand region thin line image is generated using the obtained hand region extracted image (S206 in FIG. 2). The hand region extraction image is thinned. For example, Yokoi et al., “About the Topological Properties of Sampled Binary Figures” (Transactions of the Institute of Electronics, Information and Communication Engineers (D),
J56-D, 42, pp. 662-669, 1973).
The one described in the above was used. FIG. 1 shows the obtained thinned image.
In many cases, small branches are attached to a fingertip or the like as shown in FIG. 2, which causes inconvenience in subsequent processing. Therefore, a branch removal process is performed. The thinned image from which the branches have been removed is defined as a hand region thin line image (FIG. 13). (Recognition of Number of Fingers: S208) A recognition image is generated, and the number of extending fingers is obtained. A polygon approximation is performed on the contour line extraction image obtained in the hand region extraction process using a line segment approximation parameter ε having a different value (see Table 2) from the line segment approximation parameter used in the hand region extraction.

[Table 2]

The approximated image is composed of n points and n straight lines.
Is a polygon consisting of Therefore, the n points are called approximate points.
Then, as shown in FIG.₀, X₁, ...,
x_{i- 1}, X_i, ..., x_n-1And put, vector
x_ix_{i + 1}, Vector x_ix _i-1Angle θ_iWhen
I do. Where the vector x_ix_{i + 1}= (A₁, A₂),
Vector x_ix_i-1= (B₁, B₂), Θ_i
Is obtained by the following equation.

(Equation 1) This equation determines the angle formed by all approximate points of the polygon. Here, those where θ _i ≦ 60 ° are extracted as feature points and the number thereof is calculated. However, when the interior angle is obtained by using the above equation, a polygon having an interior angle of 180 ° or more is also extracted. This means that an internal angle θ of 180 ° or more is calculated as 360 ° −θ in the above formula, and a polygon whose internal angle is substantially 180 ° or more is also extracted. Therefore, as shown in FIG. 15, the triangle _{△ p i-1 p i p} i + 1 signed area (e.g., Tetsuo Asano "Computational Geometry," pp.73-79 Asakura Shoten, 19
A vertex having an internal angle of 180 ° or more is detected by using (see 1990), and a process of not extracting a corner having an internal angle of 180 ° or more as a feature point is performed. The number of fingers is recognized from the calculated number of feature points. Table 3 shows the correspondence between the number of feature points and the number of fingers. At this time, if 0 and 5 fingers are recognized, the process ends.

[Table 3] The accuracy of the recognition of the number of fingers is the same as the image 1 used in the recognition experiment.
For 500 sheets, the recognition rate is about 99%.

<Finger Area Extraction Processing>
The finger region is extracted from the types of recognition images, and image features necessary for recognition are extracted. An apparent feature of the hand such as the thickness of the finger is extracted from the hand region extraction image, and a coordinate feature which is a skeleton feature of the hand is extracted from the hand region thin line image. (Determination of Hand Direction Vector: S210) The hand direction vector is determined from the hand area thin line image (see FIG. 13). Hereinafter, a method of extracting a finger vector will be described. 1. From the hand region extraction thin line image, the end point (x _n0 , y _n0 ) (n = 0, 1,...,
N). Here, each end point exists at the corresponding fingertip. A line segment Sn from the end point to the branch point or the intersection is extracted and set as a finger candidate line segment (see FIG. 16). 2. Each line segment _{S n}
Is approximated using a piecewise linear approximation method, and the refraction point is defined as (x _nm , y _nm ). However, m is 1, from the end point side.
2, ... For a line segment that does not need to be broken-line approximation, a branch point or an intersection is extracted as a refraction point (x _n1 , y _n1 ). At this time, the value of the line segment approximation parameter ε was set to 7.0. 3. In each segment _{S n,} and the end point _{(x n0, y n0),} obtaining the respective finger vector _{v n} by using the nearest point of inflection on the end points in the approximated linear _{(x nl, y n1) (see} FIG. 17) . Here, v _n = (x
_n0 is a _{-x n1, y n0 -y n1)} . 4. Using the following equation, the sum of the directional vectors of each finger is taken as the directional vector v of the hand (see FIG. 18). Here, N is the number of fingers.

(Equation 2) In this method, a hand direction vector is required at the time of finger region extraction, and the vector in the direction from the wrist to the fingertip is more important than the direction of the entire hand region in the image. Since the hand direction vector obtained by this procedure is obtained from the finger area candidates, the direction from the wrist to the fingertip can be obtained without being affected by the degree of bending of the wrist. (Determination of finger region: S212) Image scanning is performed perpendicularly to the obtained hand direction vector, and a histogram of the run length of the hand region in the image (hereinafter referred to as run histogram) is created together with the image scanning (FIG. 19). The thickness of each finger is
It differs greatly from the thickness of the wrist and palm. Therefore, when scanning the image perpendicular to the hand direction vector, the finger, wrist,
The length of the run determined by crossing the palm differs greatly between the finger and other parts. Therefore, a run histogram as shown in FIG. 20 can be created, and a run length serving as a threshold for dividing a finger region and a region other than the finger region can be determined by a discriminant analysis method. The discriminant analysis method is described in, for example, Nobuyuki Otsu, “Automatic Threshold Selection Method Based on Discrimination and Least Square Criterion”, IEICE Transactions, J-63D, p. 3
49, 1980.

(Extraction of Finger Region) A finger region is extracted using a threshold value determined by the discriminant analysis method. Again, image scanning is performed perpendicular to the direction vector of the hand, and if the run of the hand area in the image is shorter than the threshold as shown in FIG. 21, it is extracted as a finger area. The white part in FIG. 22 is the extracted finger area. The run histograms of the finger regions shown in FIGS. 22A to 22D and the determined threshold are shown in FIG.
3 (a) to 3 (d). However, this discriminant analysis method is not always applied in all cases. Due to the characteristics of the discriminant analysis, for example, when the number of fingers is one, the threshold value may not be determined. In that case, an area smaller than the longest 1/3 of the run is extracted as a finger area. The finger region is extracted from the hand region thin line image from the obtained finger region extraction image. Extraction is
Although only the logical product of the finger region extraction image and the hand region thin line image is obtained, a number of line segments greater than the original number of fingers may be extracted. Therefore, the number of fingers is recognized (S2
Using the result of step 08), a process of removing a finger segment having a small length is performed so that the number of fingers and the finger segment coincide. The resulting image is shown in FIG.

<Recognition of Hand Shape> The following image features are extracted from images obtained by the processing or the processing steps so far, and are used as recognition parameters. A recognition function is created using these recognition parameters to recognize which finger is extended. The average length of the run of the finger region, the length of each finger line, the distance between each finger, and the position information of the finger with respect to the palm. To obtain the average length of the run of the finger area, the length of each finger line segment, the interval between each finger, and the position information of the finger with respect to the palm, which are the image features to be extracted, are digitized. That is, since the value differs depending on the position of the camera and the hand, it is necessary to normalize the value. Therefore, in order to obtain a normalized value, the extracted finger region (see FIG. 22) is subjected to image scanning perpendicular to the direction vector v of the hand as shown in FIG. 25 to create a run histogram of the finger region. The average value of the obtained run histogram is obtained, and this is set as the average length of the runs of the finger region, and is set as the normalized value T.

(Line length of each finger) Finger region extraction thin line image
In FIG. 26, as shown in FIG.
L is the length of the straight line connecting the endpoints_n(N = 1, 2,..., N)
I do. This is normalized using the average run length T of the finger region.
You. The following equation is used for the normalization, and the finger length Fl after the normalization is used. _n
(N = 1, 2,..., N).

(Equation 3)

(Interval between Fingers) The procedure for obtaining the interval between fingers will be described below. 1. In the finger region extraction thin line image, the hand direction vector v
Is used to detect the coordinates corresponding to the root of the finger of each line segment. 2. A straight line A approximating the obtained root coordinates of the finger is obtained by the least squares method (see FIG. 27A). 3. A straight line B perpendicular to the straight line A and passing through the root coordinates of the finger
_n (n = 1, 2,..., N) is obtained (see FIG. 27B). 4. Intersection point C between straight line A and straight line B _n (n = 1, 2,... N)
_n (n = 1, 2,..., N) is determined, and the length of C _n and C _{n + 1} is defined as the distance between the fingers. 5. The obtained interval is normalized using the average run length T of the finger region.

(Position Information of Finger with respect to Palm) A procedure for obtaining position information of a finger with respect to the palm will be described below. First, in order to determine the position line segment of the palm, an image scan perpendicular to the direction vector v of the hand is performed on the hand region extracted image. The part having the longest run length is detected by image scanning, and that part is defined as a palm position line segment (see FIG. 28A). Also, both end points of the obtained position line segment are referred to as an end point L and an end point R, respectively. (Determination of finger position) A perpendicular line is lowered from the root coordinates of the finger to the obtained palm position line segment, and the intersection coordinates with the palm position line segment are obtained (see FIG. 28 (b)). As shown in FIG. 28B, the position information of the finger is obtained from the distance from both ends L and R of the intersection and the position line segment of the palm. As with the other parameters, the normalization is performed using the average length T of the runs in the finger area. <Features of Recognition Parameter> Features obtained from the recognition parameters will be described, and recognition will be performed using these features (S216).

(Characteristics of Length of Each Finger) Due to the structure of the human hand, the length of each finger differs. When the number of fingers is five as shown in FIG. 29 (a), as shown in the graph of FIG. 29 (b), the characteristic that the thumb and the little finger are shorter than the length of the other fingers appears. The graph of FIG. 30 shows statistical data when five fingers are used (however, the number of data is 47). It can be seen from the statistical data that the thumb and the little finger are shorter than the other fingers. However, since this relationship is different for each hand shape shown in FIGS. 3 and 4, it is necessary to analyze data of all shapes and use it as a finger length feature. FIG.
(B) In FIG. 30, the mark x indicates the length f of the finger after normalization.
represents l _n, polygonal line portion indicates the length relationship of the fingers of the same image.

(Characteristic of Spacing Between Fingers) As shown in FIG. 31, the spacing between fingers differs depending on how the fingers are combined. For example, in FIG. 31A and FIG. 31B, the finger interval characteristics are also different due to the difference in how the fingers are combined. Therefore, by digitizing the interval between the fingers by the method described above, the combinations of the fingers can be limited and can be used for recognition. (Characteristics of finger position with respect to palm) As shown in FIG. 32, finger position information differs depending on which finger is extended. FIG. 32A shows that the distance from the end point L to the leftmost intersection is small,
The distance from the end point R to the rightmost intersection is large. From this feature, it can be seen that the probability that the little finger is extended is high, and the probability that the thumb or forefinger is extended is low. In FIG. 32B, the distance from the end point L to the leftmost intersection is small, and the distance from the end point R to the rightmost intersection is large. It can be seen that the probability that the little finger is extended is smaller than this feature, and the probability that the thumb or index finger is extended is high. In this way, it is recognized which finger is standing using each recognition parameter used as described above.

<Recognition Procedure of Which Finger is Standing> FIGS. 33 to 37 show specific examples of the recognition procedure. Recognition of which finger is standing is performed for each number of standing fingers recognized by the finger number recognition process (S208). FIG. 33 shows a recognition procedure when one finger stands, and FIG.
35, the recognition procedure when two fingers are standing, FIG.
6 shows a recognition procedure when three fingers are standing, and FIG. 37 shows a recognition procedure when four fingers are standing.
The illustrated procedure indicates a recognition result or a procedure to jump to depending on whether the condition indicated in the condition column is satisfied or not satisfied. If the recognition result is indicated, the recognition procedure ends. In these recognition procedures, the recognition result indicates the standing finger in a binary system as shown in FIGS. With the right palm facing the camera, the little finger recognizes the image on the left and the thumb on the right. When recognizing the left hand, the numbering of the fingers and the interpretation of the left and right may be changed. Assuming that the average of the finger thickness is 1, the position of the finger from the left and right ends of the hand and the length of the finger are represented.

[0026]

As described above, by changing the shape of the hand in front of the camera connected to the various devices, it is possible to recognize 32 directions by recognizing one hand and 1024 directions by using both hands.
It is possible to input to various devices using the input system of the present invention.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating an example of a system configuration according to an embodiment.

FIG. 2 is a flowchart illustrating an example of a recognition process according to the embodiment;

FIG. 3 is a diagram illustrating hand shapes recognizable by the system according to the embodiment;

FIG. 4 is a diagram illustrating hand shapes recognizable by the system according to the embodiment;

FIG. 5 is a diagram illustrating an example of an input image.

FIG. 6 is a flowchart illustrating a processing example of generating a hand region extracted image.

FIG. 7 is a diagram illustrating an example of a hand region candidate image.

FIG. 8 is a diagram showing an example of a contour line extraction image.

FIG. 9 is a diagram illustrating a piecewise linear approximation method.

FIG. 10 is a diagram illustrating an example of a polygon approximate image.

FIG. 11 is a diagram showing an example of a hand region extracted image.

FIG. 12 is a diagram illustrating an example of an image after a thinning process.

FIG. 13 is a diagram illustrating an example of a hand area thin line image.

FIG. 14 is a diagram showing polygon approximation.

FIG. 15 is a diagram illustrating detection of a vertex having an inner angle of 180 ° or more.

16 is a diagram illustrating a finger candidate segments S _n.

17 is a diagram showing a direction vector v _n of each finger.

FIG. 18 is a diagram showing a hand direction vector v.

FIG. 19 is a diagram illustrating image scanning.

FIG. 20 is a diagram showing a run histogram obtained by image scanning.

FIG. 21 is a diagram for describing extraction of a finger region using a run length histogram.

FIG. 22 is a diagram illustrating an example of a finger region extracted image.

FIG. 23 is a diagram illustrating an example of a run histogram and a threshold.

FIG. 24 is a diagram illustrating an example of extracting a finger region from a hand region thin line image.

FIG. 25 is a diagram for explaining an average length T of runs in a finger region.

FIG. 26 is a diagram showing the length of each finger.

FIG. 27 is a diagram illustrating a process for obtaining the interval between fingers.

FIG. 28 is a diagram illustrating a process of obtaining positional information of a finger with respect to a palm.

FIG. 29 is a diagram showing characteristics of the length of each finger.

FIG. 30 is another diagram showing the characteristics of the length of each finger.

FIG. 31 is a diagram illustrating a characteristic example of an interval between fingers.

FIG. 32 is a diagram illustrating an example of a position feature of a finger with respect to a palm.

FIG. 33 is a diagram illustrating an example of a recognition procedure in the case where one standing finger is used.

FIG. 34 is a diagram illustrating an example of a recognition procedure when two standing fingers are used.

FIG. 35 is a diagram illustrating an example of a recognition procedure when three standing fingers are used.

FIG. 36 is a diagram illustrating the continuation of the recognition procedure when three standing fingers are used.

FIG. 37 is a diagram illustrating an example of a recognition procedure when four standing fingers are used.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI theme coat ゛ (reference) G06T 7/00 300 G06T 7/00 300F F term (reference) 5B057 BA02 BA11 CA01 CA08 CA12 CA16 CE05 CE12 CE16 CG04 DA08 DB02 DB06 DB09 DC09 DC17 5L096 AA02 AA06 BA18 CA02 EA02 EA04 EA06 FA06 FA18 FA35 FA59 GA36 GA38 GA41 JA18

Claims (7)

[Claims] 1. An input system based on hand image recognition, comprising: an image input means for inputting an image of a hand; and a hand area extracted image in which a hand area portion is filled from a hand image from the image input means. A hand region extracting unit, a hand region thin line image generating unit that obtains a hand region thin line image from the hand region extracted image, and a hand that recognizes a standing finger from the hand region extracted image and the hand region thin line image. An input system using hand image recognition, comprising: hand shape recognition means for specifying a shape; and specifying one of 32 shapes with one hand depending on which finger is standing. 2. The input system according to claim 1, wherein the hand region extracting means extracts a flesh color portion from the image of the hand, performs a smoothing process on the flesh color portion, and performs the smoothing process on the flesh color portion. The contour of the part is traced to extract a coordinate series having the maximum length of the contour, a smoothing process is performed on the extracted contour, and then the inside of the contour is painted to obtain a hand region extracted image. An input system based on image recognition of a hand. 3. The input system according to claim 2, further comprising an index recognizing means for recognizing the number of fingers from the contour, wherein the index recognizing means selects one of 2 to 4 fingers. When it is recognized that the hand shape is identified by the hand shape recognition means, and when it is recognized that the number of fingers is 0 or 5,
An input system based on hand image recognition, wherein the processing by the hand shape recognition means is not performed. 4. The input system according to claim 1, wherein the hand shape recognizing means obtains a hand direction vector from the hand area thin line image, and obtains a hand direction vector perpendicular to the obtained hand direction vector. Scanning the hand region extraction image to determine the run length of the hand,
An input by hand image recognition, wherein a finger region is extracted based on the run length, and a hand shape is recognized by extracting a finger portion from the hand region thin line image using the extracted finger region. system. 5. The hand image recognition apparatus according to claim 1, wherein the hand shape recognition means normalizes and evaluates a parameter for recognizing the hand shape. By input system. 6. A recording medium storing a program for causing a computer system to configure the input system according to claim 1. 7. A program for causing a computer system to configure the input system according to claim 1.