JPH08328735A - Hand pointing input device - Google Patents

Abstract

PURPOSE: To perform hand pointing input without moving a television camera or performing focus adjustment. CONSTITUTION: The plural television camera are respectively provided with a fisheye lens and a CCD area image sensor. The coordinate P(x,y,z) of the tip of an index finger and the coordinate P2 (x2 , y2 , z2 ) of the base of the index finger are computed by a prescribed formula from the values of the coordinate PL (α1 , β1 ) on the CCD screen of the camera L and the coordinate PR (α1 ; β1 ,) on the CCD screen of the camera R, the three-dimensional coordinate of a point T where an extension line for connecting the coordinate P (z, y, z) and the three-dimensional coordinate P2 (x2 , y2 , z2 ) of the base of the index finger crosses a large-sized liquid crystal display is computed and a cursor is displayed at the point T. When a thumb is raised, information displayed by the large-sized liquid crystal display present at the point T is obtained as the information inputted by an object to be recognized.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a hand pointing input device, and more particularly to a hand pointing input device for inputting information by the action of a finger or an arm without touching a device.

[0002]

2. Description of the Related Art Japanese Unexamined Patent Publication No. 5-32, which is the basis of the present invention
The hand pointing input device described in Japanese Patent No. 4181 (Applicant: Takenaka Corporation) will be described with reference to FIG.

The above-mentioned hand pointing input device is a display 91 for displaying predetermined information, and an information input person's hand 95 or finger 9 pointed to the display 91.
A plurality of image pickup means 93, 94 for picking up images 6, 97, and a direction calculation for calculating the direction pointed by the hand 95 or finger 96 with respect to the display 91 based on the images picked up by these image pickup means 93, 94 Means 98C and Direction Calculation Means 98C
The position display means 98D for displaying the position on the display 91 corresponding to the direction calculated in step 9 on the display 91 by the cursor 92, the click operation detecting means 98E for detecting the click operation of the hand 95 or the finger 97, and the click operation detection. The selection means 98B is configured to select the information of the portion where the cursor 92 is located as the information instructed by the information input person when the click operation is detected by the means 98E.

According to this hand pointing input device, when the information input person points the information of the display 91 with the hand 95 or the finger 96 in front of the display 91, based on the images picked up by the image pickup means 93, 94, The direction indicated by the hand 95 or the finger 96 is calculated with respect to the display 91, and the display 91 corresponding to the calculated direction is calculated.
When the information input person confirms the position of the cursor 92 and then performs the click operation, the cursor 92 is moved to the upper position, and when the click operation is performed, the information of the portion where the cursor 92 is located is the information input. It is input as information instructed by the person. In this way, the information input person can input information to the computer or the like only by the action of the hand or finger without touching the device.

[0005]

However, in the hand pointing input device disclosed above, it is necessary to photograph the hand or finger of the information input person in order to calculate the direction pointed by the hand or finger. You have to move it and adjust the focus. Therefore, there is a problem that a delay time occurs until the cursor is displayed on the display.

Further, since the image pickup means needs a mechanical drive portion for moving the image pickup means and for adjusting the focus, there is a problem that reliability and durability tend to be low.

In view of the above facts, an object of the present invention is to provide a hand pointing input device capable of acquiring information instructed by an information input person without moving the image pickup means or adjusting the focus.

[0008]

In order to achieve the above object, the invention according to claim 1 has a display means for displaying a predetermined image and a visual field of 90 ° or more arranged at a predetermined position. A plurality of fixed-angle lenses each having an angle and an area sensor arranged at an image forming point formed by the lenses, and a plurality of images of a predetermined region including a recognition target as an information input person existing in the field of view are captured from different positions. Image pickup means, and recognition means for recognizing a finger or an arm to be recognized by processing different image pickup information picked up by the plurality of image pickup means,
Position calculation means for calculating a position on the display means on an extension of the direction indicated by the recognition target finger or arm recognized by the recognition means, and a position on the display means calculated by the position calculation means A position display means for displaying, a motion detection means for processing different imaging information newly captured by the plurality of imaging means to detect a predetermined motion by a finger or an arm to be recognized, and the motion An input information acquisition unit that acquires, when the predetermined motion is detected by the detection unit, the information displayed by the display unit at the position displayed by the position display unit as the input information input by the recognition target; , Are provided.

According to a second aspect of the present invention, a display means for displaying a predetermined image and a display means arranged at a predetermined position are provided.
A wide-angle fixed-focus lens having a viewing angle of 0 ° or more and an area sensor arranged at an image forming point by the lens are provided, and a predetermined area including a recognition target as an information input person existing in the visual field is imaged. Image pickup means, reflecting means arranged near the image pickup means for reflecting an image of a recognition target so as to form an image on the area sensor, and recognition information obtained by processing the image pickup information picked up by the image pickup means. Recognizing means for recognizing a target finger or arm, position calculating means for calculating a position on the display means on an extension line of a direction pointed by the recognized target finger or arm recognized by the recognizing means, and the position Position display means for displaying the position on the display means calculated by the calculation means, and predetermined movement by the finger or arm to be recognized by processing the image pickup information newly picked up by the image pickup means An input by which the recognition target has input the information displayed by the display means at the position displayed by the position display means when the predetermined operation is detected by the operation detection means And input information acquisition means for acquiring it as information.

According to the invention described in claim 3, a display means for displaying a predetermined image, a wide-angle fixed-focus lens arranged at a predetermined position and having a viewing angle of 90 ° or more, and a combination of the lenses. A plurality of image pickup means, each of which includes an area sensor disposed at an image point, and which picks up a predetermined region including a recognition target as an information input person existing in the visual field from different positions with a known pattern as a background; The recognition means for recognizing the finger or arm to be recognized based on the different imaging information captured by the image capturing means and the information about the known pattern, and the recognition target finger or arm recognized by the recognition means are Position calculation means for calculating the position on the display means on the extension of the pointing direction, and position display for displaying the position on the display means calculated by the position calculation means A step, a motion detection means for processing different imaging information newly imaged by the plurality of imaging means to detect a predetermined motion by the finger or arm to be recognized, and the motion detection means predetermined An input information acquisition unit that acquires the information displayed by the display unit at the position displayed by the position display unit as the input information input by the recognition target when the detected operation is detected. .

[0011]

According to the invention described in claim 1, a predetermined image is displayed on the display means. The plurality of image pickup means are arranged at predetermined positions. This predetermined position is, for example, a flat surface such as the ceiling of a room, or a ceiling and a wall.
It can be a corner formed by a surface or a corner formed by three surfaces such as a ceiling and two walls. The plurality of image pickup means are provided with a wide-angle fixed focus lens having a viewing angle of 90 ° or more and an area sensor arranged at an image forming point by the lenses. Since the wide-angle fixed focus lens has a wide viewing angle of 90 ° or more, it is possible to capture an image of a recognition target without moving a plurality of image capturing means. Further, since the height of the object to be recognized as an information input person from the floor or the ground is almost fixed, and the wide-angle fixed focus lens can have a deep depth of focus, a plurality of image pickup means are provided with a focus adjustment mechanism. Even if it is not, the recognition target image can be clearly formed on the area sensor. A plurality of image pickup means images a predetermined area including a recognition target as an information input person existing in the visual field from different positions.

The recognition means processes different pieces of image pickup information picked up by the plurality of image pickup means to recognize the finger or arm to be recognized. In order to recognize the recognition target finger or arm, for example, an assembly of a large number of cubic microspaces obtained by virtually subdividing a three-dimensional space along each of horizontal, vertical, and vertical directions. As the above, recognition may be performed by obtaining the minute area occupied by the finger or arm to be recognized from different image information captured by a plurality of image capturing means, or, for example, different by a plurality of image capturing means. The imaging information is converted into flattened image data, image data of at least the front surface, back surface, left side surface, right side surface and flat surface of the finger or arm to be recognized is obtained from the converted image data, and the obtained image It may be recognized by synthesizing data.

The position calculation means calculates the position on the display means on the extension line of the direction pointed by the finger or arm to be recognized, which is recognized by the recognition means. The position display means displays the position calculated by the position calculation means on the display means. As the display by the position display means, for example, display of a cursor, a frame, inversion, density change, etc. can be adopted. The motion detection means processes different pieces of imaged information newly imaged by the plurality of image pickup means to detect a predetermined motion by the finger or arm to be recognized. When the input information acquisition unit detects a predetermined motion by the motion detection unit, the information displayed by the display unit at the position displayed by the position display unit is acquired as the information input by the recognition target. It

As described above, since it is not necessary to move the plurality of image pickup means to adjust the focus, it is possible to quickly display the position on the display means and to improve reliability and durability.

According to the hand pointing input device of the second aspect, the predetermined image is displayed on the display means.
The reflecting means is arranged near the image pickup means. The reflecting means may be arranged along a wall, may be L-shaped, or may be curved, for example. The image of the recognition target is reflected by the reflecting means so as to form an image on the area sensor. Therefore, a plurality of recognition target images of the recognition target image formed directly and the recognition target image reflected and formed by the reflecting means are formed on the area sensor. The image capturing means captures an image of a predetermined area including the object to be recognized as the information input person existing in the visual field. The recognition means processes the imaging information imaged by the imaging means to recognize the finger or arm to be recognized. The position calculation means calculates the position on the display means on the extension line in the direction indicated by the finger or arm to be recognized, which is recognized by the recognition means. The position on the display means calculated by the position calculation means is displayed by the position display means. The motion detection means processes different pieces of imaged information newly imaged by the plurality of image pickup means to detect a predetermined motion by the finger or arm to be recognized.
When the input information acquisition unit detects a predetermined motion by the motion detection unit, the information displayed by the display unit at the position displayed by the position display unit is acquired as the information input by the recognition target. It In this way, since the recognition target image can be formed on the area sensor by the reflecting means, it is possible to obtain the input information input by the recognition target even with only one imaging means.

According to the hand pointing input device of the third aspect, a predetermined image is displayed on the display means.
A plurality of image capturing means captures images of a predetermined area including a recognition target as an information input person existing in the visual field from different positions with a known pattern as a background. The recognition means recognizes the finger or arm to be recognized based on the different imaging information captured by the plurality of imaging means and the information about the known pattern. The position calculation means calculates the position of the display means on the extension line in the direction indicated by the finger or arm to be recognized, which is recognized by the recognition means. The position display means displays the position calculated by the position calculation means on the display means. The motion detection means processes the different imaging information newly imaged by the plurality of imaging means to detect a predetermined motion by the finger or arm to be recognized. When the input information acquisition unit detects a predetermined motion by the motion detection unit, the information displayed by the display unit at the position displayed by the position display unit is acquired as the input information input by the recognition target. To be done.

[0017]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of a hand pointing input device according to the present invention will be described below with reference to the drawings.

As shown in FIG. 1, the hand pointing input device 10 includes a plurality of television cameras 30 as image pickup means arranged in an array at substantially equal intervals on a ceiling 36 of a room 34 forming a three-dimensional space. The central control unit 12 for controlling the entire hand pointing input device 10 and the room 3
Large display 26 attached to one wall 38 of the No. 4
It consists of and. Each of these TV cameras 30 and the central controller 12 are connected via a communication cable 28. A plurality of illumination lamps 32 are fixedly installed on the ceiling 36.

Each television camera 30 has a fish-eye lens 30A as a wide-angle fixed focus lens, and the viewing angle of the fish-eye lens 30A is 90 ° or more. Fisheye lenses include, for example, equidistant projections.
There are various types such as e projection type, stereoscopic projection type, equisolid angle projection type, orthographic projection type, and any fisheye lens can be used in the present embodiment, but the equidistant projection type fisheye lens is used below. It is explained as what is done.
In addition, each TV camera has a CCD (Charge-Cou).
area image sensor 30B
(Not shown in FIG. 1) is provided, and a predetermined area including the object 40 as the recognition target existing in the room 34 is imaged.

The central controller 12 comprises a bus 20. The bus 20 has a hand pointing input device 10
A CPU 14 that performs overall control processing, a ROM 16 that stores a control program that controls the entire hand-pointing input device 10, a work area that is used when the control program is executed, and imaging information captured by each video camera 30 is temporarily stored as image data. I / O for transmitting / receiving signals to / from the RAM 18 stored in
The O interface 22 is connected. The I / O interface 22 is connected to a console 24 including a display and a keyboard, a display control unit 26A that controls the large display 26, and the above-described television cameras 30.

As the large display 26, in addition to a large liquid crystal display and a large plasma display,
A display such as a large CRT display or an optical fiber display can be used.

Next, the operation of this embodiment will be described.
When a start button (not shown) provided in the central controller 12 is pressed, the hand pointing processing routine shown in FIG. 2 is executed. The hand pointing processing routine is repeatedly performed at predetermined time intervals.

First, in step 100, object classification processing is performed. In this object classification process,
The subroutine shown in FIG. 3 is executed. Step 140
Then, the image data A when the object 40 does not exist in the room 34 is read from the ROM 16, and in the next step 142, the image data B picked up by each television camera is fetched and stored in the RAM 18. In the next step 144, the difference between the image data B and the image data A is taken to determine whether the object 40 exists in the room 34 (see FIG. 6). Next, in step 146, a timer is set. An internal clock configured by the CPU 14 can be used as this timer. In the next step 148,
It is determined whether or not the time-out has occurred, and if the determination is negative, the process returns to step 148. On the other hand, if the judgment is affirmative,
In the next step 150, the image data C picked up by each television camera is fetched. Next, in step 152, the image data B stored in the RAM 18 is read out and compared with the image data C to obtain the traveling direction, and the front and back of the object 40 is judged from the obtained traveling direction (see FIG. 6), and the front and rear information is obtained. Store in RAM 18 and step 10
Return to 0.

In the next step 102, object voxel processing is performed. In this object voxel processing, the subroutine shown in FIG. 4 is executed. In step 160, the position and height of the object 40 are calculated. As shown in FIG. 7, the focal length of the equidistant projection fisheye lens 30A fixed at the point O is the focal length f, and the distance from the point O to the point Q vertically lowered to the floor surface 35 of the room 34 is a distance H, The distance from the point Q to the point P on the floor surface 35 of the object 40 is the distance R, and the height of the object 40 (the point P ′ when the tip of the object 40 in the ceiling direction is the point P ′).
Is the height h. Further, the angle formed by the point POQ is the angle θ, the angle formed by the point P′OQ is the angle θ ′, the object image on the CCD surface of the CCD area image sensor 30B is h ′, and the object image h ′ corresponds to the point P. Point p, the point imaged corresponding to point P'in the object image h'is point p ', and the image center of the CCD surface (CCD
If the distance from the center o of the plane) o to the point p is r, and the distance from the image center o of the CCD plane to the point p'is r ',
The angles θ and θ ′ and the distances r and r ′ can be calculated by the following equations (1) to (4).

Θ = tan ⁻¹ (R / H) (1) θ ′ = tan ⁻¹ {R / (H−h)} (2) r = fθ .. (3) r '= f?' (4) Therefore, the height h and the distance R can be obtained by the following equations (5) and (6).

H = H {1-tan (r / f) / tan (r ′ / f)} (5) R = Htan (r / f) (6) Note that the distance H And the focal length f are predetermined, and the equations (5) and (6) are stored in the ROM 16. Therefore, in step 160, the equation (5) is read and
The height h is calculated from the information on the CCD surfaces of the two TV cameras, and the equation (6) is read to read the CCDs of the two TV cameras.
The distance R is obtained from the information on each surface, and the two-dimensional position of the object 40 is calculated from the obtained two distances R.

Next, in step 162, the above step 1
A three-dimensional space around the position calculated in 60 in the X direction,
A voxel, which is a mesh-shaped minute region that is virtually subdivided along the Y direction and the Z direction, is set. This allows
The image data C is converted into a set of voxels. FIG. 8 conceptually shows voxels occupied by the object 40 when the object 40 is viewed from the four television cameras A, B, C and D. When the object 40 is viewed from each TV camera, the shadow R is a blind spot.
_{Although A} , R _B , R _C, and R _D occur, since the shadow part is not visible from each TV camera, these shadow parts are also set as voxels occupied by the object 40. The voxels can be subdivided up to the resolution limit of the CCD area image sensor 30B.

In the next step 162, object 4
The primary narrowing down is performed by limiting the voxels occupied by the object in the image data by the height h of 0. Figure 9
(A) to (D) show the concept of voxels when the primary narrowing-down is performed from the image data of four TV cameras.

Next, in step 166, the secondary narrowing is performed from each image data in the primary narrowing region to leave voxels in the overlapping portions of the object image (see FIG. 9), and the process returns to step 102. As a result, the shadow regions R _A , R _B , R _C, and R _D shown in FIG. 8 are excluded from the voxels occupied by the object image. FIG. 10 conceptually shows the voxels 50 occupied by the object image when the respective image data are combined.

In the next step 104, the front and rear information stored in the RAM 18 and the height of the person stored in the ROM 16, the color difference of the head, the positions of the eyes, nose, mouth, and ears, the length and position of the arm, and the toes The characteristic point information such as the orientation and the degree of freedom of the joint is read out, and it is determined in step 106 whether or not the object 40 represented by the voxel 50 is a person. In case of negative judgment, the object is not a person (information input person),
The hand pointing processing routine ends. on the other hand,
When the determination is affirmative, the voxel 5 in step 108
Parameters such as the height and thickness of the object 40 are determined from 0, and the parameters are converted into a dummy model 60 that resembles the appearance of a person according to the determined parameters. FIG. 11 shows the concept of conversion from the voxel 50 to the dummy model 60.

Next, at step 110, the dummy model 60 is called by calling the data concerning the finger and the arm from the ROM 16.
Limit the range of movement of the arm. 12A and 12B show the concept when the movable range of the arm is limited. In the next step 112, it is judged whether or not the arms of the dummy model 60 are extended. When a negative determination is made, it is determined that the hand pointing input has not been made, and the hand pointing processing routine ends. On the other hand, if the judgment is affirmative,
In step 114, it is determined whether or not the base of the index finger of the dummy model 60 can be discriminated. If the determination is affirmative, then in the next step 116, the three-dimensional coordinates of the fingertip of the index finger and the base of the index finger are calculated. On the other hand, when the determination is negative, in step 118, the three-dimensional coordinates of the fingertip of the index finger and the joint of the arm are calculated.

The three-dimensional coordinate calculation in these steps 116 and 118 will be described in detail below. To simplify the description, the calculation of the three-dimensional coordinates of the fingertip of the index finger will be described as an example.

First, two TV cameras are selected, and the two-dimensional coordinates of the index finger on the CCD surface of each of the selected two TV cameras are captured. The two television cameras can be selected in descending order of the index finger image, for example. These two selected TV cameras are referred to as camera L and camera R, respectively.

Now, with reference to FIG. 13, various amounts such as the position, distance and angle of the equidistant fisheye lens 30A, the CCD area image sensor 30B and the fingertip of the index finger will be described. The three-dimensional coordinate C of the camera L is (X, 0, Z),
The three-dimensional coordinate C ′ of the camera R is (X ′, 0, Z).
Further, the two-dimensional coordinate P _L on the CCD surface of the camera L of the fingertip of the index finger is (α ₁ , β ₁ ), the distance from the image center O _L of the CCD surface of the camera _L to the two-dimensional coordinate P _L is r, and the index finger is the index finger. of the coordinates P _R on the CCD surface of the fingertip of the camera R (α ₁ ',
beta ₁ '), the distance from the image center O _R of the CCD of the camera R to the coordinates P _R r' 2 both light when and virtual light emitted from the coordinate P _L and coordinates P _R by the lossless law of light Where P is the three-dimensional coordinate P of the fingertip of the index finger
(X, y, z). Further, the three-dimensional coordinates of the point S at which the angle between the foot of the perpendicular drawn from the three-dimensional coordinate position of the camera L and the foot of the perpendicular drawn from the point P is a right angle is S (X, 0,
z), the three-dimensional coordinates of the point S ′ at which the angle between the foot of the perpendicular drawn from the three-dimensional coordinate position of the camera R and the foot of the perpendicular drawn from the point P is a right angle is S ′ (X ′, 0, z ), The angle formed by the point PCS is the angle θ ₁ , the angle formed by the point PC ′S ′ is the angle θ ₁ ′, the angle formed by the point PSS ′ is the angle φ, and the angle formed by the point PS ′S is the angle φ ′.
And

The distance r from the image center O _L to the image on the CCD surface is obtained by the above equation (3) as r = fθ ₁ .

[0036] In addition, each α _1, β _{1 is, α 1 = fθ 1 cos (} π-φ) = - fθ 1 cosφ β 1 = fθ 1 sin (π-φ) = fθ 1 sinφ ··· (7) Is. Here, sin φ = y / {(x−X) ² + y ² } ^1/2 (8) cos φ = (x−X) / {(x−X) ² + y ² } ^1/2 , Α ₁ and β ₁ are α ₁ = −fθ ₁ (x−X) / {(x−X) ² + y ² } ^1/2 ... (9) β ₁ = fθ ₁ y / {(x -X) ² + y ² } ^1/2 ... (10) By dividing Expression (10) by Expression (9), y = (β ₁ / α ₁ ) (X−x) (11) Similarly, y = (β ₁ ′ / α ₁ ′) (X '−x) (12) Eliminating y from equation (11) and equation (12), x = (α ₁ β ₁ 'X'-α ₁ 'β ₁ X) / (α ₁ β _{1 '-α 1' β 1)} by (13) can be determined X-coordinate of the three-dimensional coordinates P.

Next, x is eliminated from the equations (11) and (13), and y = β ₁ β ₁ '(X-X') / (α ₁ β ₁ '-α ₁ ' β ₁ ) The Y coordinate of the three-dimensional coordinate P can be calculated by (14).

By the way, since θ ₁ = tan ⁻¹ [{(x−X) ² + y ² } ^1/2 / (Z−z)], β ₁ / (from equations (7) and (8) fsin φ) = tan ⁻¹ [{(x−X) ² + y ² } ^1/2 / (Z−z)] Therefore, z = Z − [{(x−X) ² + y ² } ^1/2 / tan [ (Β ₁ / f) × {(x−X) ² + y ² } ^1/2 / y] (15) Further, from formula (11), {(x−X) ² + y ² } ^1/2 = (X−X) {1+ (β ₁ / α ₁ ) ² } ^{1/2 From} formula (11) and formula (14), (x−X) = (X′−X) / {1− (α ₁ ′) / Α ₁ ) × (β ₁ / β ₁ ′)}, the formula (15) is as follows: z = Z + [(X′−X) {1+ (β ₁ / α ₁ ) ² } ^1/2 / { 1- (α ₁ ′ / α ₁ ) (β ₁ / β ₁ ′)}] / tan {(α ₁ ² + β ₁ ² ) ^1/2 / f} (16), which can be expressed as 3 Of the dimensional coordinate P The Z coordinate can be obtained.

The three-dimensional coordinates of each television camera 30 are predetermined. Therefore, steps 116, 118
Then, from the ROM 16 the equations (13), (14) and (1
6) is read out, and the coordinates P _L (α
₁ , β ₁ ) and the coordinate P on the CCD plane of the camera R
The three-dimensional coordinates P (x, y, z) of the fingertip of the index finger are calculated by substituting the values of _R (α ₁ ', β ₁ '). Similarly,
Three-dimensional coordinate P ₂ (x ₂ , y ₂ ,
z ₂ ) or the three-dimensional coordinate P ₃ (x ₃ , y ₃ , z of the arm joint)
₃ ) is calculated.

Next, in step 120, the three-dimensional coordinates P (x, y, z) of the fingertip of the index finger and the 3 of the base of the index finger are set.
An extension line connecting the dimensional coordinates P ₂ (x ₂ , y ₂ , z ₂ ) or the three-dimensional coordinates P (x, y, z) of the fingertip of the index finger and the three-dimensional coordinates P ₃ (x ₃ , y) of the arm joint. The three-dimensional coordinates of the point T at which the extension line connecting ( ₃ , z ₃ ) intersects the large display 26 are calculated.

Here, as shown in FIG. 14, 3 at the point T
The dimensional coordinate is T (X, 0, Z). Also, the point S (x,
0, z), S '(X, 0, z) and T' (x, 0, Z)
With virtual to, when each angle phi _P and theta _P an angle of the point T'PS and point SPS ', X = x + ytanθ P tanθ P = (x-x 2) / are the (y ₂ -y) , X = (xy ₂ −yx ₂ ) / y ₂ −y (17) The X coordinate of the point T can be obtained.

[0042] In addition, since it is _{= Z = z + ytanφ P tanφ} P (z-z 2) / (y 2 -y), Z = (zy 2 -yz 2) / (y 2 -y) ··· (18 ), The Z coordinate of the point T can be obtained.

Similarly, an extension line connecting the fingertip of the index finger and the joint of the arm intersects the large display 26 at a point T.
The three-dimensional coordinates _{of, X = (xy 3 -yx 3} ) / y 3 -y ··· (17) 'Z = (zy 3 -yz 3) / (y 3 -y) ··· (18)' Can be asked as

Therefore, in step 120, equation (17)
(18) or equation (17) '(18)' is called from the ROM 16 to calculate the three-dimensional coordinates of the point T.

In the next step 122, it is determined whether or not the index finger is aimed at the large display 26 by determining whether or not the point T is included in the display area of the large display 26. When the determination is negative, the index finger is not within the hand pointing input possible range, so the hand pointing processing routine ends. On the other hand, when the determination is affirmative, a cursor as a marking display is displayed at the point T on the large display 26 in the next step 124.

Next, at step 126, a timer is set,
In the next step 128, arm voxel processing is performed.
In the voxel processing of this arm, the subroutine shown in FIG. 5 is executed. In step 170, camera L and camera R
The image data D of is taken in and voxels are set in the next step 172. Next, in step 174, the above-described narrowing down is performed, and in step 176, the dummy model is converted, and the process returns to step 128. In the voxel processing of the arm, voxels are set similarly to the object voxel processing (see FIG. 4) described above, but finer than in the object voxel processing in order to determine the three-dimensional coordinate P ₂ of the base of the finger. Voxels are set.

In the next step 130, the information input person may leave from the front of the large display 26 without performing a click operation described later (without inputting information) with the index finger pointing toward the large display 26, so that a timeout occurs. It is determined whether or not When the determination is negative, the hand pointing processing routine is ended. On the other hand, when the determination is affirmative, it is determined in the next step 132 whether or not the thumb has stood up as a predetermined click action based on the voxel-processed image data D and the data about the finger or arm called from the ROM 16. Note that FIG. 15A shows a state in which the information input person points his or her index finger toward the large display 26, and FIG. 15B shows a state in which the information input person raises his or her thumb. When the determination is negative, the process returns to step 112. On the other hand, when the determination is affirmative, the information displayed on the large-sized display 26 at the position indicated by the cursor in the next step 134 is fetched as the information input by the information input person, and the hand pointing processing routine is ended.

As described above, according to the present embodiment, since the image is picked up using the wide-angle fixed focus lens 30A, it is not necessary to move the television camera or adjust the focus. For this reason,
Since it is possible to shorten the time until the cursor is displayed on the large-sized display 26 and the time until the cursor is displayed, it is possible to shorten the time for hand pointing input.

Further, since a mechanism for changing the direction of the television camera and adjusting the focus is unnecessary, the durability and reliability of the television camera can be improved.

Further, since the television camera is arranged on the ceiling of the room forming the three-dimensional space, the wall surface can be effectively used.

Further, the voxels can be subdivided up to the resolution limit of the CCD area image sensor.
Since the three-dimensional coordinates of the base of the finger or the joint of the arm can be accurately calculated, the accuracy of the position of the cursor can be improved.

Since the shadow portion is eliminated by the secondary narrowing down, the shape of the object can be accurately recognized.

In this embodiment, a plurality of TV cameras 3 are used.
Although 0 is placed on the ceiling, it may be placed near the wall or embedded in the wall, such as a two-sided corner portion composed of the ceiling and the wall, or three sides composed of the ceiling and the two-sided wall. You may arrange in the corner part of. Furthermore, equidistant projection type fisheye lens 3
0A may be directed to the center of the room.

Although the image data A is read in step 140 in this embodiment, the object may be separated from the image data B and the image data C without the step 140.

Further, although the equidistant projection type fisheye lens is used in the present embodiment, the same calculation can be performed using the equisolid angle projection type fisheye lens, the stereoscopic projection type fisheye lens or the orthographic projection type fisheye lens as described above. . Below, the formulas (1) ′ to (6) ′ corresponding to the formulas (1) to (6) when using the equisolid angle projection type fisheye lens are shown.

Θ = tan ⁻¹ (R / H) (1) ′ θ ′ = tan ⁻¹ {R / (H−h)} (2) ′ r = 2fsin ( θ / 2) ··· (3) ′ r ′ = 2fsin (θ ′ / 2) ··· (4) ′ h = H [1-tan {2sin ⁻¹ (r / 2f)} / tan {2sin ⁻¹ (r ′ / 2f)}] (5) ′ R = Htan {2sin ⁻¹ (r / 2f)} (6) ′ The first embodiment Then, the object is captured by four TV cameras, but in the case of two or more TV cameras, the three-dimensional coordinates of the object can be calculated in the same manner as above.

Next, a second embodiment of the hand pointing input device according to the present invention will be described. In this embodiment,
An example of performing hand pointing input with one television camera and one mirror is shown. Since this embodiment is substantially the same as the first embodiment, the same parts in FIGS. 1 to 15 are designated by the same reference numerals and the description thereof will be omitted.

As shown in FIG. 16, each television camera 30
1 side of the CCD area image sensor 30B
A vertically long mirror 38 is fixed to the ceiling 36 in the vertical direction (Z direction) in parallel with the end face direction (X direction). Next, various quantities such as the position, distance and angle of the equidistant projection type fisheye lens 30A, the CCD area image sensor 30B and the mirror 38 of this embodiment will be described with reference to FIGS. FIG.
Reference numeral 7 shows the details of the above-mentioned amounts when the distance between the equidistant projection type fisheye lens 30A and the CCD area image sensor 30B is negligible.

As shown in FIG. 16, the center of the upper end of the mirror 38 on the same XY plane as the CCD surface of the CCD area image sensor 30B is located at the origin O (0, 0, 0, 0, 0, 0) of the three-dimensional coordinates.
Take 0). The image center H on the CCD surface is separated from the origin O in the Y direction by a distance h, and the three-dimensional coordinate of the image center H is H.
Take (0, h, 0). Note that the three-dimensional coordinates of the fingertip of the index finger are P (x, y, z), and the light emitted from the point P is refracted by the equidistant projection type fisheye lens 30A and imaged at the point D on the CCD surface. The two-dimensional coordinate of the point D on this CCD surface is D (α
_D , β _D ). Further, the light emitted from the point P and reflected by the mirror 38 is refracted by the equidistant projection type fisheye lens 30A and imaged at the point R on the CCD surface. Point R on this CCD surface
Let the two-dimensional coordinates of R (α _R , β _R ). The mirror 38
Assuming a virtual TV camera 31 when there is no
The 3D coordinate of the image center H'of the surface is H '(0, -h, 0)
The light emitted from the point P is refracted by the virtual equidistant projection type fisheye lens 31A to form an image at the point R'on the CCD surface of the virtual CCD area image sensor 31B. It is assumed that R and the virtual point R ′ are symmetrical with respect to the mirror 38. Also, the image center H on the CCD surface
Is a distance r _D , and the distance from the image center H on the CCD surface to a point R is a distance r _R.

As shown in FIG. 16, an arbitrary point on the perpendicular line drawn from the point H in the Z direction is defined as a point R, and an arbitrary point on the perpendicular line drawn from the point H ′ in the Z direction is defined as the point R. The angle formed by the point PHR is the angle θ _D , and the angle formed by the point PH′R ′ is the angle θ _{R ′} . Further, the point represented by the three-dimensional coordinates (x, y, 0) is the point S, the distance between the point S and the point H is the distance B _D , the distance between the point S and the point H ′ is the distance B _{R ′} , the point The distance between P and the point H is defined as a distance A _D , and the distance between the point P and a point H ′ is defined as a distance A _{R ′} .

Next, the operation of this embodiment will be described.
In steps 116 and 118 of FIG. 4, one TV camera 30 is selected and the CCD corresponding to the clicked position is selected.
The values of coordinates D (α _D , β _D ) and R (α _R , β _R ) on the surface are taken in. The selection of the TV camera 30 is, for example,
The TV camera with the smallest distance r _D can be selected.

Now, the various quantities described above will be further described with reference to FIGS. The angles θ _D and θ _{R ′} are respectively θ _D = tan ⁻¹ (B _D / Q) = tan ⁻¹ [{(y−h) ² + x ² } ^1/2 / z] θ _{R ′} = tan ^−1. (BR _′ / Q) = tan ⁻¹ [{(y + h) ² + x ² } ^1/2 / z], the distances r _D and r _R can be calculated from the above equation (3) by the following equation. To be done.

R _D = f · tan ⁻¹ [{(y−h) ² + x ² } ^1/2 / z] r _R = f · tan ⁻¹ [{(y + h) ² + x ² } ^1/2 / z _{Incidentally, α D = r D cos (} π-φ D) = - r D cosφ D ··· (19) β D = r D sin (π-φ D) = r D sinφ D ··· (20) α _R = r _R cos φ _{R '} (∵ φ _R' = φ _R ) ・ ・ ・ (21) β _R = r _R sin φ _{R '} (∵ φ _{R '} = φ _R ) ・ ・ ・ (22) Also, cos φ _D = (Y−h) / {(y−h) ² + x ² } ^1/2 ... (23) sin φ _D = x / {(y−h) ² + x ² } ^1/2 ... (24) cos φ _{R '} = (y + h) / {(y + h) ² + x ² } ^1/2 ... (25) sin φ _R' = x / {(y + h) ² + x ² } ^1/2 ... (26) Therefore, equation (19) and equation (23) and equation (2
0) and the equation (24), α _D = −fθ _D (y−h) / {(y−h) ² + x ² } ^1/2 (27) β _D = fθ _D x / {(y -H) ² + x ² } ^1/2 ... (28). Eliminating fθ _D from these two equations, y = h− (α _D / β _D ) x (29) Similarly, α _R = fθ _{R ′} (y + h) / {(y + h) ² + x ² } _{^{1/2 ··· (30) β R =}} fθ R 'x / {(y + h) 2 + x 2} 1/2 ··· (31) y = -h + (α R / β R) x ··· ( 32) (29) and (32) to be able to determine the X-coordinate of _{x = 2hβ D β R / (} α D β R + α R β D) 3 -dimensional coordinate P by ... (33).

Next, by substituting the equation (33) into the equation (29), y = h (α _R β _D −α _D β _R ) / (α _D β _R + α _R β _D ) (34) ), The Y coordinate of the three-dimensional coordinate P can be obtained.

Β _D = r _D sin φ _D = f θ _D sin φ _D = f · tan ⁻¹ [{(y−h) ² + x ² } ^1/2 / z] · sin φ _D z = {(y-h) 2 + x 2} 1/2 / tan (β D / fsinφ D) = {(y-h) 2 + x 2} 1/2 / tan [(β D / f) × {( y−h) ² + x ² } ^1/2 / x] From the equations (33) and (34), {(y−h) ² + x ² } ^1/2 = 2hβ _R (α _D ² + β _D ² ) ^1/2 / (α _D β _R + α _R β _D ), so z = 2hβ _R (α _D ² + β _D ² ) ^1/2 / [(α _D β _R + α _R β _D ) × tan { (Α _D ² + β _D ² ) ^1/2 / f}] (35), the Z coordinate of the three-dimensional coordinate P can be obtained.

The image center H on the CCD surface from the mirror 38
The distance h up to is predetermined. Therefore, in step 144, the equations (33), (34) and (35) are read from the ROM 16 and the coordinate D (α _D ,
The three-dimensional coordinates P (x, y, z) of the fingertip of the index finger are calculated by substituting the values of β _D ) and the coordinates R (α _R , β _R ). Similarly, the three-dimensional coordinates P ₂ (x
₂ , y ₂ , z ₂ ) and the three-dimensional coordinate P of the arm joint
₃ (x ₃ , y ₃ , z ₃ ) is calculated.

As described above, according to this embodiment, one television camera can calculate the three-dimensional coordinates of the fingertip, the base of the finger, or the joint of the arm, and the input information can be obtained. The number of cameras can be reduced.

In this embodiment, one TV camera and one mirror are used to calculate the three-dimensional coordinates of the object. However, the mirror may be attached along the wall surface. You may use two sheets. Also, curved mirrors may be used. If the mirror is mounted along the wall surface, the space can be used more effectively than when it is mounted orthogonally to the wall, and if two mirrors are used, one more image will be formed on the CCD surface. Even if a blind spot occurs due to another object, the three-dimensional coordinates can be calculated.

Next, a third embodiment of the hand pointing input device according to the present invention will be described. In the present embodiment, an example is shown in which three-dimensional coordinates are calculated by using the background and hand pointing input is performed. The configuration of this embodiment is the first
Since the configuration is substantially the same as that of the embodiment, the same parts are designated by the same reference numerals and the description thereof is omitted.

As shown in FIG. 19A, in this embodiment, one wall surface 38 to which the large display 26 is attached is attached.
In the corner adjacent to the one wall surface 38 on the one wall surface 39 adjacent to
"1" in a rectangular box in order from the floor surface 35 to the ceiling 36
The number "9" is attached. Similarly, in the adjacent corners adjacent to the one wall surface 38 on which the large display 26 is attached on the floor surface 35, "1" to "in a rectangular box are sequentially arranged from the one wall surface side opposite to the one wall surface 39. The number "9" is attached.

Next, the operation of this embodiment will be described. In this embodiment, when a start button (not shown) provided in the central controller 12 is pressed, the hand pointing processing routine shown in FIG. 18 is executed. The hand pointing processing routine is repeatedly performed at predetermined time intervals.

First, in step 200, the two television cameras 3 when the object 40 does not exist in the space.
Image data A viewed from 0 (TV cameras A and B)
Take in. In the next step 202, the image data B captured by the television cameras A and B is taken in, and in the next step 204, it is determined whether or not there is a difference between the image data A and the image data B. If the determination is negative, the object 40 does not exist in the room 34, and therefore the step 202
Return to. On the other hand, when the determination is affirmative, the next step 206
At step, data representing the characteristic of the finger as known information given in advance regarding the finger is read from the ROM 16, and the finger is recognized at the next step 208.

In the next step 210, background information of the fingertip of the index finger and the base of the finger of the index finger is acquired. For example, as shown in FIG. 19 (A), when the television cameras A and B capture the index finger of the information input person, the finger tip of the television camera A is at the position "4" as shown in FIG. 19 (B). The background information can be obtained, and the background information in which the fingertip is at the position "6" can be obtained from the television camera B as shown in FIG.

Next, at step 212, a straight line connecting the television cameras A and B and the background is calculated. TV camera A
The three-dimensional coordinates of B and B are predetermined as shown in the first embodiment. In the above example (fingertip of the index finger), the three-dimensional coordinates of the background of the TV camera A (position "6") and the background of the TV camera B (position "4") are stored in the ROM 16 in advance. . Therefore, step 21
In 2, the three-dimensional coordinates of the television cameras A and B and the three-dimensional coordinates of the background of the television cameras A and B are read from the ROM 16, and the three-dimensional coordinates of the television camera A and the three-dimensional coordinates of the background of the television camera A are connected to each other. A three-dimensional line B connecting the three-dimensional coordinates of the three-dimensional line A and the television camera B and the three-dimensional coordinate of the background of the television camera B is calculated. Similarly, a three-dimensional line C connecting the three-dimensional coordinates of the TV camera A and the three-dimensional coordinates of the background of the base of the index finger by the TV camera A.
And a three-dimensional straight line D connecting the three-dimensional coordinates of the TV camera B and the three-dimensional coordinates of the background of the base of the index finger by the TV camera B.

At the next step 214, the intersection of the three-dimensional line A and the three-dimensional line B and the three-dimensional line C and the three-dimensional line D.
By obtaining the three-dimensional coordinates of the intersection point with, the three-dimensional coordinates of the fingertip of the forefinger and the base of the forefinger are obtained.

Next, at step 216, as shown in the first embodiment, an extension line connecting the three-dimensional coordinates of the fingertip of the index finger and the extension line connecting the three-dimensional coordinates of the base of the index finger intersects the large display 26. The three-dimensional coordinates of the point T are calculated.

At the next step 218, it is determined whether or not the index finger is aimed at the large display 26 by determining whether or not the point T is included in the display area of the large display 26. When the determination is negative, the hand pointing processing routine is ended. On the other hand, when the determination is affirmative, the cursor as a marking display is displayed on the large display 26 in the next step 220.

Next, at step 222, a timer is set,
In the next step 224, new image data captured by the television cameras A and B is captured. Next step 226
Then, it is determined whether or not the time is out, and when the determination is negative, the hand pointing processing routine is ended. On the other hand, when the determination is affirmative, it is determined in the next step 228 whether or not the thumb has risen as a predetermined click operation. When the determination is negative, the process returns to step 210. On the other hand, when the determination is affirmative, the information displayed on the large display 26 at the position indicated by the cursor in the next step 230 is taken in as the information input by the information input person, and the hand pointing processing routine is ended.

As described above, according to this embodiment, the three-dimensional coordinates of the fingertip of the index finger and the root of the index finger can be calculated from the background of the fingertip of the index finger and the root of the index finger without setting voxels.

In the present embodiment, the example of the numbers "1" to "9" is shown, but the colors shown in FIG. 20 may be used.
Characters may be used instead of numbers.

Further, as shown in FIG. 21, the timing of turning on or off the light emitting body (for example, LED) may be determined in advance and the turning on or off may be detected as a known pattern.

Next, a fourth embodiment of the hand pointing input device according to the present invention will be described. Since the structure of the present embodiment is substantially the same as that of the first embodiment, the same parts are designated by the same reference numerals, and the description thereof will be omitted.

In the present embodiment, the one wall surface 38 on which the large display 26 is attached and the corner portion adjacent to the one wall surface 38 on the one wall surface 39 adjacent to the one wall surface 38 on which the large display 26 is attached and the one wall surface 38 on which the large display 26 is attached on the floor surface 35. A picture without a repeating pattern (not shown) is attached to each of the adjacent corners adjacent to as a background. That is, "1" to "9" of the third embodiment.
Instead of the number, a picture without a repeating pattern (not shown) is attached.

Next, the operation of this embodiment will be described. In this embodiment, when a start button (not shown) provided in the central controller 12 is pressed, the hand pointing processing routine shown in FIG. 22 is executed. The hand pointing processing routine is repeatedly performed at predetermined time intervals.

First, in step 240, the image data of the television camera 30 (referred to as television camera A) when the object 40 is not present in the room is fetched from the ROM 16. In the next step 242, the image data captured by the television camera A is captured, and in the next step 244, it is determined whether or not there is a difference between the image data captured in steps 240 and 242. When the determination is negative, the process returns to step 242. On the other hand, when the determination is affirmative, the characteristic point information of the index finger is read in the next step 246.

Next, in step 248, the image data of the portion hidden by the index finger is searched from the image data of the television camera A when the object 40 is not present in the room. In the next step 250, the position information obtained by the search is acquired.

At the next step 252, the object 4
Another TV camera 30 when 0 is not in the room
The image data of the TV camera B is fetched from the ROM 16. In the next step 254, the television camera B
In step 256, the image data of the portion hidden by the index finger is retrieved from the image data of the television camera B when the object 40 is not present in the room, and is obtained in the next step 258 by the search. Get the location information.

Next, at step 260, it is judged whether or not the index finger is directed to the large display 26, and if the judgment is negative, the hand pointing processing routine is ended. On the other hand, when the determination is affirmative, the next step 26
The cursor position is calculated in 2 and the cursor is displayed in step 264.

In the next step 266, a timer is set,
In step 268, the image data captured by the television camera B is captured. In the next step 270, it is determined whether or not the time is out, and if the determination is affirmative, the hand pointing processing routine is ended. Next, in step 272, it is determined whether or not the thumb has risen. When a negative determination is made, the process returns to step 268. On the other hand, when the determination is affirmative, the information is fetched in the next step 274 and the hand pointing processing routine is ended.

In the above embodiment, the fingertip of the index finger,
Although the three-dimensional coordinates of the base of the index finger or the joint of the arm are obtained, the present invention is not limited to the index finger or the joint of the arm.

Further, in the above embodiment, the rising of the thumb is the predetermined click motion, but the click motion may be detected by extending the middle finger.

[0092]

As described above, according to the first and third aspects of the present invention, it is not necessary to move a plurality of image pickup means to adjust the focus, so that the position on the display means can be displayed promptly. It is possible to obtain the effect that the reliability and durability can be improved.

According to the second aspect of the present invention, since the recognition target image can be formed on the area sensor by the reflecting means, it is possible to obtain the input information input by the recognition target even with one image pickup means. The effect that can be obtained can be obtained.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a hand pointing input device according to the present invention.

FIG. 2 is a flowchart showing a flow of a hand pointing processing routine executed by the hand pointing input device according to the first embodiment.

FIG. 3 is a flowchart showing a flow of a subroutine of object classification processing in step 100 of FIG.

FIG. 4 is a flowchart showing a flow of a subroutine of object voxel processing in step 102 of FIG.

5 is a flowchart showing a flow of a subroutine of arm voxel processing in step 128 of FIG. 2. FIG.

FIG. 6 is an explanatory diagram illustrating a concept of separating objects.

FIG. 7 is an explanatory diagram illustrating various amounts such as the height of an object.

FIG. 8 is an explanatory diagram illustrating a relationship between a shadow portion of an object and a voxel.

9A is a voxel based on the image data of the television camera A, FIG. 9B is a voxel based on the image data of the television camera A, FIG. 9C is a voxel based on the image data of the television camera A, and FIG. It is explanatory drawing explaining the voxel by the image data of the television camera A.

FIG. 10 is an explanatory diagram illustrating the concept of voxels narrowed down by the second narrowing down.

FIG. 11 is an explanatory diagram illustrating a concept of converting voxels narrowed down by the second narrowing down to a dummy model.

12A is a plan view of the movable range of the dummy model arm, and FIG. 12B is a side view of the movable range of the dummy model arm.

FIG. 13 is a conceptual diagram illustrating various amounts when three-dimensional coordinates are calculated by two television cameras.

FIG. 14 is an explanatory diagram for explaining a three-dimensional coordinate calculation of a cursor.

FIG. 15A is an explanatory diagram illustrating a state in which the information input person points the index finger toward the large display, and FIG. 15B is an explanatory diagram illustrating information in which the information input person raises his / her thumb.

FIG. 16 is an explanatory diagram illustrating a configuration of a hand pointing input device according to a second embodiment.

FIG. 17 is an explanatory diagram for explaining various quantities such as the positions of the CCD area image sensor and the like of the second embodiment.

FIG. 18 is a flowchart showing a flow of a hand pointing processing routine executed by the hand pointing input device according to the third embodiment.

FIG. 19 (A) is an explanatory view for explaining the configuration of the hand pointing input device according to the third embodiment, and (B).
Is a conceptual diagram illustrating acquisition of background information at the position of the fingertip of the index finger and the base of the finger from the image data of the television camera A, and (C) is a diagram of the fingertip of the index finger and the base of the finger from the image data of the television camera B. It is a conceptual diagram explaining acquiring the background information in a position.

FIG. 20 is a conceptual diagram in the case of acquiring background information when the background is color.

FIG. 21 is a conceptual diagram in the case of acquiring background information when the background is a light source.

FIG. 22 is a flowchart showing the flow of a hand pointing processing routine executed by the hand pointing input device according to the fourth embodiment.

FIG. 23 is an explanatory diagram illustrating a conventional hand pointing input device.

[Explanation of symbols]

10 hand pointing input device 14 CPU (recognition means, position calculation means, motion detection means, input information acquisition means) 26 large display (display means, position display means) 30 television camera (imaging means) 30A equidistant projection fisheye lens (wide angle Fixed focus lens) 30B CCD area image sensor (area sensor) 38 Mirror (reflection means)

Claims (3)

[Claims] 1. A display unit for displaying a predetermined image, a wide-angle fixed-focus lens arranged at a predetermined position and having a viewing angle of 90 ° or more, and an area sensor arranged at an image forming point of the lens. A plurality of image pickup means for picking up a predetermined region including a recognition target as an information input person existing in the visual field from different positions, and processing different image pickup information picked up by the plurality of image pickup means. Recognition means for recognizing a finger or arm as a recognition target, and position calculation means for calculating a position on the display means on an extension line of a direction indicated by the recognition target finger or arm recognized by the recognition means. A position display means for displaying the position on the display means calculated by the position calculation means, and a different recognition target by processing different image pickup information newly picked up by the plurality of image pickup means. Motion detecting means for detecting a predetermined motion of the finger or arm, and when the predetermined motion is detected by the motion detecting means, the display means at the position displayed by the position display means is A hand pointing input device comprising: input information acquisition means for acquiring the displayed information as input information input by a recognition target. 2. A display unit for displaying a predetermined image, a wide-angle fixed focus lens having a viewing angle of 90 ° or more arranged at a predetermined position, and an area sensor arranged at an image forming point of the lens. An image pickup means for picking up an image of a predetermined area including the object to be recognized as an information input person existing in the field of view; A reflection means for reflecting the image of the object, a recognition means for recognizing the finger or arm to be recognized by processing the imaging information imaged by the imaging means, and a finger or arm to be recognized recognized by the recognition means. Position calculating means for calculating a position on the display means on an extension of the direction pointed by, position display means for displaying the position on the display means calculated by the position calculating means, and the imaging means. A motion detecting unit that processes the newly captured image information to detect a predetermined motion of the recognition target finger or arm; and when the motion detecting unit detects a predetermined motion, A hand pointing input device comprising: input information acquisition means for acquiring the information displayed by the display means at the position displayed by the position display means as the input information input by the recognition target. 3. A display unit for displaying a predetermined image, a wide-angle fixed focus lens having a viewing angle of 90 ° or more arranged at a predetermined position, and an area sensor arranged at an image forming point of the lens. A plurality of image pickup means for picking up a predetermined region including a recognition target as an information input person existing in the visual field from different positions with a known pattern as a background, and different image pickup means imaged by the plurality of image pickup means Recognition means for recognizing a finger or arm to be recognized based on imaging information and information on the known pattern, and the display on an extension line of the direction indicated by the finger or arm to be recognized recognized by the recognition means. Position calculation means for calculating a position on the means, position display means for displaying the position on the display means calculated by the position calculation means, and the plurality of image pickup means When a predetermined motion is detected by the motion detection unit, the motion detection unit that processes different newly captured image information to detect a predetermined motion by the recognition target finger or arm, In addition, a hand pointing input device comprising: input information acquisition means for acquiring information displayed by the display means at the position displayed by the position display means as input information input by the recognition target.