JP2012242901A - Operation input device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operation input device capable of detecting a motion of a target article corresponding to a hand and a forearm, and recognizing a motion close to the natural movement of a person in turning pages.SOLUTION: Distance image generation means 10 generates a motion image of a distance image of a target article including a first element corresponding to a hand and a second element corresponding to a forearm. Element extraction means 2 extracts a target surface corresponding to a palm of the first element and a region corresponding to the second element from the distance image. Motion extraction means 3 extracts changes in positions and orientations of the target surface and the second element that are extracted by the element extraction means 2. The motion extraction means 3 detects an angle with which the orientation of a normal vector on the target surface rotates around the principal axis of the second element and an angle with which the orientation of a directional vector along the principal axis of the second element changes, and determines a motion to turn pages to be made if an angle detected within a specified determination period after starting detection of an angle is within a specified detection range.

Description

  The present invention determines whether or not a target object is performing a prescribed operation by using a distance image having a pixel value as a distance to the target object, and performs an operation for performing correspondence control of the control target according to the determination result The present invention relates to an input device and a program for realizing the operation input device.

  Conventionally, an input device has been proposed in which a motion of a target is detected using a distance image having a pixel value as a distance to the target, and corresponding control of the control target is performed according to the motion of the target. (For example, refer to Patent Document 1). The technique described in Patent Document 1 obtains a minimum point of distance in a distance image, and gives an instruction to a control object according to a change in position of the minimum point. That is, the technique described in Patent Document 1 regards the minimum point of the distance as the position of the fingertip that is forwarded, and tracks the position of the fingertip, thereby enabling operation of the device without having a remote control device or the like. ing.

  In recent years, an input device that detects the movement of a fingertip or a penpoint on a two-dimensional plane by using a touch panel and operates a device in accordance with the detected movement has been put into practical use.

Japanese Patent No. 3544739

  By the way, an input device using a touch panel needs to touch the touch panel. For example, if an input device is used to operate a television receiver placed on the floor, the input device moves to a position touching the television receiver. become. Therefore, it is not possible to give a control instruction from a place away from the control target as in the case of a remote control device.

  On the other hand, if the technique described in Patent Document 1 is adopted, a control instruction can be given from a place away from the control target. However, since the technique described in Patent Document 1 regards the minimum point of the distance as the position of the fingertip and regards the change in the position of the minimum point as the change in the position of the fingertip, Less.

  Recently, electronic books and web versions of newspapers have been provided, and page turning operations are becoming indispensable in order to handle this type of content. When a page turning operation is performed on the touch panel, only a fingertip or the like is moved on a two-dimensional plane, and even if the technique described in Patent Document 1 is adopted, the movement is the same as that on the touch panel. That is, it is not possible to associate a natural movement of a person with page turning on the screen.

  The present invention relates to an operation input device capable of recognizing a movement close to a natural movement of a person when turning a page of a book or a newspaper by detecting the movement of an object corresponding to a hand and a forearm, and this It is an object to provide a program for realizing an operation input device.

  In order to achieve the above-described object, the operation input device according to the present invention includes a distance image generating unit that generates a moving image of a distance image for a first element corresponding to a hand and a second element corresponding to a forearm, An angle detection unit that detects a change in the angle of the first element around the principal axis of the two elements, and an operation determination unit that determines that a specified operation has been performed when the angle detected by the angle detection unit is within a specified detection range. It is characterized by providing.

  The operation input device according to the present invention generates a moving image of a distance image having a pixel value as a distance to the object with respect to the object including at least a first element corresponding to a hand and a second element corresponding to a forearm. An element generating means for extracting an area corresponding to the first element and an area corresponding to the second element from the distance image, and extracting a target surface corresponding to a palm in the first element; A motion extracting means for extracting a change in the position and orientation of the extracted first element and second element from the moving image of the distance image; and when the motion extracting means detects a prescribed motion of the first element and the second element. Control means for performing correspondence control to the controlled object, and the motion extracting means has a normal vector direction of the target surface around the principal axis of the second element in a three-dimensional virtual space formed by the distance image. Rotate with An event detection unit that detects a first event and a second event in which the direction of a direction vector along the main axis of the second element changes, a rotation angle in the first event detected by the event detection unit, and a second event An angle detection unit that detects a changing angle, and an angle detected by the angle detection unit within a predetermined determination time after the event detection unit starts detecting the first event and the second event is within a predetermined detection range. And an operation determination unit that determines that the prescribed operation has been performed.

  In this operation input device, the event detection unit identifies the direction of movement between the first element and the second element, and the operation determination unit identifies the type of operation according to the direction of movement identified by the event detection unit. It is preferable.

  In this operation input device, it is preferable that the motion determination unit includes a setting unit that sets a detection range for the angle detected by the angle detection unit.

  In this operation input device, the setting unit has two operation modes: a setting mode for setting a detection range, and a use mode for determining the specified operation using the detection range. Preferably, the detection range is set based on the angle detected by the angle detection unit when the prescribed operation is performed a plurality of times.

  In this operation input device, the element extracting means extracts the first element and the second element from the object using the distance image, and then extracts the first element and the second element in the plane perpendicular to the main axis in the direction along the main axis of the first element. It is preferable to measure a large amount and consider a position where the change in the measured maximum width satisfies a specified condition as a boundary between the first element and the second element.

  In this operation input device, the element extraction means extracts the first element and the second element with respect to the object using the distance image, and then, in the direction along the principal axis of the first element, the curvature in the plane perpendicular to the principal axis. It is preferable to regard the position where the measured change in curvature satisfies the specified condition as the boundary between the first element and the second element.

  In this operation input device, the element extraction means extracts the first element and the second element with respect to the object using the distance image, and then sums the first element and the second element in the direction along the main axis of the first element. It is preferable that the position divided at a specified ratio with respect to the length dimension is regarded as the boundary between the first element and the second element.

  The program according to the present invention acquires a moving image of a distance image from the distance image generation means for the first element corresponding to the hand and the second element corresponding to the forearm, and the first element and the second element. A program for causing the computer to function as an arithmetic processing unit that detects a prescribed operation of the first and second computers, wherein an angle detector that detects a change in the angle of the first element around the main axis of the second element, and an angle detector The information processing apparatus functions as an arithmetic processing device including an operation determination unit that determines that a specified operation has been performed when the detected angle is within a specified detection range.

  Further, the program according to the present invention allows a computer to generate a moving image of a distance image having a pixel value as a distance to an object with respect to an object including at least a first element corresponding to a hand and a second element corresponding to a forearm. A program for obtaining a distance image from a distance image generating means to be generated and for causing a function to function as an arithmetic processing unit that performs control of correspondence to a control target when a prescribed operation of the first element and the second element is detected. The computer extracts an area corresponding to the first element and an area corresponding to the second element from the distance image, and extracts an object plane corresponding to the palm in the first element; Action extraction means for extracting changes in position and orientation between the extracted first element and second element from the moving image of the distance image, and the action extraction means defines the first element and the second element Control means for performing correspondence control to the control object when the motion is detected, and the motion extraction means has a second direction of the normal vector of the target surface in the three-dimensional virtual space formed by the distance image. An event detector that detects a first event that rotates about the principal axis of the element and a second event that changes the orientation of the direction vector along the principal axis of the second element, and a first event detected by the event detector An angle detection unit for detecting a rotation angle and a changing angle in the second event, and an angle detected by the angle detection unit within a predetermined determination time after the event detection unit starts detecting the first event and the second event Is operated as an arithmetic processing unit including an operation determination unit that determines that the specified operation has been performed when is within a specified detection range.

  According to the configuration of the present invention, it is possible to individually detect the movement of the object corresponding to the hand and the forearm and recognize a movement close to a natural movement of a person when turning a page of a book or a newspaper. Even if the user does not learn the operation method, the user can cause the control target to perform the corresponding control only by performing a normal operation when turning the page.

It is a block diagram which shows embodiment. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above.

  As shown in FIG. 1, the operation input device described below is a distance image generation unit 10 that generates a moving image of a distance image, acquires a distance image from the distance image generation unit 10, and performs a prescribed operation of an object. And an arithmetic processing unit 20 that performs control corresponding to the control target when detected. The arithmetic processing unit 20 includes a computer as a hardware element, and implements the functions described below by executing an appropriate program on the computer.

  In the present embodiment, it is assumed that the object to be recognized by the distance image generating unit 10 includes a first element corresponding to the hand of the human body and a second element corresponding to the forearm of the human body. . However, the object may be a humanoid robot or a doll. If the first element is recognized as a hand and the second element is recognized as a forearm, whether or not the object is a human body. Does not matter. Hands and forearms are preferably exposed, but they can recognize page-turning movements even when mittens are worn on the hands or when the forearms are covered with winter clothes sleeves. Is possible. Therefore, it is possible to recognize the page turning operation not only when used indoors but also outdoors, regardless of the season.

  The distance image generation means 10 includes a sensor unit 11 that measures a distance to a target object in real space, and a distance calculation unit 12 that generates a distance image whose pixel value is the distance to the target object measured by the sensor unit 11. Prepare.

  The sensor unit 11 can employ either an active configuration or a passive configuration. The sensor unit 11 of the present embodiment projects light into a space where an object exists, receives reflected light from an object existing in the space, and uses information corresponding to a time difference from light projection to light reception to the object. It has an active configuration for detecting the distance. That is, the sensor unit 11 measures the distance to the object by the time of flight (TOF = Time Of Flight) method.

  In the following, as an example of the sensor unit 11, an active configuration is assumed in which light is emitted from a light emitting source into a space, and reflected light from an object existing in the space is received by an image sensor.

  The light source emits intensity-modulated light whose intensity changes with time, and projects the phase difference between the intensity-modulated light (reflected light) reflected by the object and received by the image sensor and the intensity-modulated light thus projected. It is used as information corresponding to the time difference from light to light reception. The modulation waveform of the modulated light is preferably a sine wave, but can also be selected from a triangular wave, a sawtooth wave, a square wave, and the like. The period of the modulated light is assumed to be constant.

  On the other hand, an image pickup device having a known configuration such as a CCD image sensor or a CMOS image sensor in which a plurality of light receiving regions (pixels) are two-dimensionally arranged is used. However, the imaging element may be designed to have a special structure suitable for the sensor unit 11. The light receiving timing of the image sensor is set in synchronization with the period of the intensity modulated light emitted by the light source. For example, the amount of light received at a timing corresponding to a phase different by 90 degrees within the period of the intensity modulated light is individually set Take out. The intensity change frequency of the intensity-modulated light is practically set to several MHz to several tens of MHz, and the amount of received light received by the image sensor is insufficient with the amount of reflected light for one period of the intensity-modulated light. Accordingly, the image sensor accumulates charges corresponding to the amount of light received for a plurality of periods (for example, 10,000 periods), and then outputs the accumulated charge as charges corresponding to the amount of light received in the phase.

  By the way, the sensor unit 11 having a configuration in which the light source emits intensity-modulated light and the image sensor receives reflected light can collectively measure the distance to the object in the entire field of view of the image sensor. Information corresponding to one distance image can be obtained in a short time. Therefore, it is possible to obtain 30 frames or more per second, as in a normal gray image or color image.

  The sensor unit 11 obtains a plurality of types (for example, four types) of charge amounts corresponding to each phase of the intensity-modulated light, and these charge amounts are input to the distance calculation unit 12 together with a signal indicating the phase of the intensity-modulated light. The

Now, it is assumed that the intensity-modulated light is a sine wave, and the sensor unit 11 can obtain charge amounts corresponding to four phases that differ by 90 degrees. When each charge amount is A0, A1, A2, and A3, the phase difference φ [rad] of light transmission / reception is expressed by the following equation.
φ = tan ⁻¹ {(A0−A2) / (A1−A3)}
If the frequency of intensity-modulated light is f [Hz], the time difference Δt from light projection to light reception is expressed as Δt = φ / 2π · f using the phase difference φ, so the speed of light is c [m / s. ], The distance to the object is expressed as c · φ / 4π · f. That is, the distance to the object can be obtained from the four types of charge amounts A0, A1, A2, and A3.

  The distance calculation unit 12 generates the distance image in which the distance is associated as the pixel value of each pixel by performing the above-described calculation for each pixel using the information obtained from the sensor unit 11. However, the obtained distance image is a distance in the direction in which the object is viewed from the sensor unit 11, and the pixel position corresponds to the angle. Since the angle corresponding to the position of the pixel is known, the distance calculation unit 12 uses the position of the pixel and the distance that is the pixel value to thereby obtain a three-dimensional orthogonal coordinate system in which the position of the object corresponds to the real space. Coordinate transformation is performed as represented by Therefore, the distance calculation unit 12 can obtain a moving image of a distance image in which the position of the object is represented in an orthogonal coordinate system. In other words, a distance image obtained by mapping a distance image in which a distance is associated with a pixel of an image sensor in a three-dimensional virtual space represented by an orthogonal coordinate system is obtained. The configuration of the distance image generation means 10 described above is well known and will not be described in detail.

  The distance image generation unit 10 gives a moving image of the distance image to the element extraction unit 2. The element extraction means 2 extracts the first element corresponding to the hand and the second element corresponding to the forearm as objects in the field of view of the distance image. In other words, it is necessary that the object including at least the first element and the second element is included in the distance image.

  In order to extract the first element and the second element from the distance image, the element extraction means 2 obtains the distribution of distances in the distance image, extracts the contour of the object from the distance image, and extracts from the forearm to the hand. Pixels in the area corresponding to the range are extracted. That is, the element extraction unit 2 recognizes the pixel group as a part of the object when the pixel group forms a cluster in the distance distribution and the pixel has continuity. Here, in the distance image, when the distance between each pair of adjacent pixels is within a specified range, it is determined that both pixels are continuous, and the shape and other areas of the region composed of continuous pixels are determined. It is determined as a part of the object Ob using the connection relationship with the region. Further, the envelope of the extracted region becomes the contour of the object Ob.

  The element extraction unit 2 classifies the pixel group recognized as a part of the object into a first element corresponding to the hand and a second element corresponding to the forearm by using another criterion. To do. The element extraction unit 2 can be operated at all times, but the element extraction unit 2 is not operated until an object is detected in a predetermined spatial region set in the field of view of the sensor unit 11. It is desirable to make it. That is, it is desirable to switch the operation mode of the arithmetic processing unit 20 to two stages.

  The element extraction unit 2 extracts the first element and the second element for each frame of the distance image. Further, the element extraction unit 2 extracts a target surface that is a surface region corresponding to the palm of the first element 21 (see FIG. 2), and normal vectors V11 and V12 corresponding to the direction of the palm from the direction of the target surface. To decide. The normal vectors V11 and V12 are obtained, for example, by obtaining the center of gravity of the target surface extracted as a palm, setting a plurality of small surface elements in the vicinity of the center of gravity, and synthesizing the normal vectors of the surface elements. The direction is adopted as the direction of the normal vectors V11 and V12. Note that the normal vectors V11 and V12 need only have an orientation and can be of any size.

  Further, the element extraction unit 2 obtains the principal axis Ax for the second element 22 (see FIG. 2), and determines the direction vectors V21 and V22 along the principal axis Ax. The principal axis Ax of the second element 22 is obtained using the pixel distribution of the part extracted as the forearm. As with the normal vectors V11 and V12, only the direction is required for the direction vectors V21 and V22, and the size is not limited.

  The first element 21 and the second element 22 extracted by the element extraction unit 2, the normal vectors V11 and V12 of the first element 21, and the direction vectors V21 and V22 of the second element 22 are input to the operation extraction unit 3. The The motion extraction unit 3 extracts changes in the position and orientation of the first element 21 and the second element 22 extracted by the element extraction unit 2 from the moving image of the distance image. Therefore, the motion extraction unit 3 includes an event detection unit 31, an angle detection unit 32, and a motion determination unit 33, as shown in FIG.

  As shown in FIG. 2, the event detection unit 31 has the orientation of the normal vectors V11 and V12 of the target surface corresponding to the palm obtained from the first element 21 in the three-dimensional virtual space formed as the distance image. A first event rotating around the main axis Ax of the second element 22 is detected. In addition, the event detection unit 31 detects a second event in which the direction of the direction vectors V21 and V22 along the main axis Ax of the second element 22 changes. That is, the event detection unit 31 determines whether or not there is a change in the direction of the normal vectors V11 and V12 obtained from the first element 21 and the direction of the direction vectors V21 and V22 obtained from the second element 22. .

  Note that FIG. 2 shows an example in which the sensor unit 11 is arranged above the human body. However, in the usage mode in which the sensor unit 11 is arranged in front of the human body, the front and back of the object are reversed. become.

  When the event detection unit 31 determines that the direction change has occurred with respect to the normal vectors V11 and V12 and the direction vectors V21 and V22, the angle detection unit 32 detects an angle corresponding to the change in direction. That is, the angle detection unit 32 detects an angle φ that is a change in the direction of the normal vectors V11 and V12 in the first event and an angle θ that is a change in the direction of the direction vectors V21 and V22 in the second event. To do. The angle φ is an angle rotated around the main axis Ax of the second element 22, and the angle θ is an angle at which the direction of the main axis Ax is changed in the virtual space.

  Since the present embodiment is intended to determine whether or not the object is moving corresponding to the page turning operation, the first event and the second event detected by the event detection unit 31 are interlocked. The condition of being is necessary. Moreover, the condition that the angle which the angle detection part 32 detects is in a regular detection range is also required. Furthermore, the condition that the time from when the detection of the first event and the second event is started until the angle reaches the detection range is within a predetermined determination time is also required. In order to determine whether or not these three conditions are satisfied, an operation determination unit 33 is provided. The operation determination unit 33 determines the above-described three conditions, and determines that a prescribed operation corresponding to page turning has been performed when all the conditions are satisfied.

  When it is determined that the operation corresponding to the page turning is performed based on the moving image of the distance image by the above-described processing, the operation extracting unit 3 confirms that the operation corresponding to the page turning is performed by the control unit 4. Notice. Upon receipt of this notification, the control means 4 performs response control to the control object 5. For example, if the control target 5 is a device that displays an electronic book or a web newspaper page on the screen of the monitor device, the updating of the page is instructed as a corresponding control to the device.

  The angle detection unit 32 also detects the direction in which the angle has changed. When the angle changes in the direction indicated by the arrow in FIG. 2, the direction in which the page is updated is reversed. For example, if the page is updated in the direction in which the page increases with respect to the angle change in the direction indicated by the arrow in FIG. 2, the page is updated in the direction in which the page decreases with respect to the angle change in the direction opposite to the arrow. . Note that if the increase and decrease of pages are reversed between vertically written pages and horizontally written pages, page turning of an electronic book or the like can be performed in the same manner as a book.

  In the above-described example, the event detection unit 31 extracts the normal vectors V11 and V12 from the first element 21, extracts the direction vectors V21 and V22 from the second element 22, and determines whether or not there is a change in direction. doing. Although it is possible to detect the page turning operation by this determination as well, it is desirable that a function for identifying the direction of movement of the first element 21 and the second element 22 is added to the event detection unit 31.

  In order to realize this function, the event detection unit 31 not only detects the change in the direction of the direction vectors V21 and V22, but also detects the direction vectors V21 and V22 with respect to the direction defined in the virtual space (for example, the direction of the coordinate axes). It is necessary to know the angle that represents the direction. Using this angle, the event detection unit 31 can detect the direction of movement of the object (the first element 21 and the second element 22) with respect to the virtual space. Further, the motion determination unit 33 can identify the type of motion according to the direction of movement of the object detected by the event detection unit 31, and can change the corresponding control of the control target 4 according to the type of motion. Become.

  For example, when the event detection unit 31 detects the movement of the object in the left-right direction, the operation determination unit 33 instructs the left and right pages to be turned. When the event detection unit 31 detects the movement of the object in the up-down direction, the operation determination unit 33 The operation can be selected so as to instruct up and down page turning. Similarly, if a page turning operation that is not left / right or up / down is necessary as control corresponding to the control target 5, the operation determination unit 33 instructs the page turning in an appropriate direction according to the moving direction of the object. Is possible.

  Of the conditions determined by the motion determination unit 33, the angle detection range varies depending on the positional relationship between the sensor unit 11 and the user. That is, it is expected that the angle for determining the prescribed operation differs depending on the position where the sensor unit 11 is installed. Therefore, it is preferable that the operation determination unit 33 includes a setting unit (not shown) that sets a detection range for the angle detected by the angle detection unit 32.

  Although this setting unit can adopt a configuration in which the detection range is manually set at the time of construction, it is preferable that the setting of the detection range is automated. When the detection range is automatically set, the setting unit is provided with two operation modes: a setting mode for setting the detection range and a use mode for determining the page turning operation using the detection range.

  In the setting mode, the user or the contractor performs the page turning operation a plurality of times, and the detection range is set using the angle detected by the angle detection unit 32 at this time. That is, the detection range may be statistically determined using the average value and the standard deviation for the angle detected by the angle detection unit 32 when the page turning operation is actually performed. Further, the detection range may be determined by giving a learning function to the setting unit and learning the angle detected by the angle detection unit 32.

  The above-described operation is excellent in that the page turning operation can be detected without strictly separating the first element 21 and the second element 22, but the first element 21 and the second element 22 are detected. If it is possible to determine the boundary, it will be easy to judge the operation.

  Therefore, it is preferable that a function for determining the boundary between the first element 21 and the second element 22 is added to the element extraction unit 2 after extracting the first element 21 and the second element 22. In order to determine the boundary, for example, the maximum widths W1 and W2 (see FIG. 3) in the plane orthogonal to the main axis Ax in the direction along the main axis Ax of the first element 21 are measured, and the measured maximum widths W1 and W2 are measured. A process is performed in which the position where the change satisfies the specified condition is regarded as the boundary between the first element 21 and the second element 22. The defining condition in this case is that the maximum widths W1 and W2 are obtained from the tip side (fingertip side) of the object along the main axis Ax (see FIG. 2), and the maximum width W1 is reached after the maximum widths W1 and W2 are minimized. , W2 is determined to be the boundary 23 on the condition that the change rate of W2 is equal to or less than a predetermined value. As shown in FIG. 3, since the maximum width W2 of the first element 21 is larger than the maximum width W1 of the second element 22, the boundary 23 can be detected under this determination condition.

  Further, the curvature κ (see FIG. 3) of the outer shape of the target object is obtained in a plane orthogonal to the main axis Ax, and the position where the curvature κ satisfies the specified condition may be set as the boundary 23. The defining condition in this case is that the curvature κ is obtained from the tip side (fingertip side) of the object along the main axis Ax (refer to FIG. 2), and the change rate of the curvature κ is predetermined after the change rate of the curvature κ is maximized. After the value becomes greater than or equal to the value, the position where the change rate of the curvature κ becomes maximum is determined as the boundary 23 on condition that the change rate of the curvature κ becomes equal to or less than a predetermined value.

  Since the maximum widths W1 and W2 and the curvature κ described above can be easily detected based on the distance image, the boundary 23 between the first element 21 and the second element 22 can be easily detected. Furthermore, in order to obtain the boundary 23 more easily, the position divided at a prescribed ratio with respect to the total length of the first element 21 and the second element 22 in the direction along the main axis Ax is the first position. It may be regarded as a boundary 23 between the element 21 and the second element 22. In general, since the ratio of various parts is measured in the human body, a statistically correct result can be obtained even if the boundary 23 is obtained by the ratio to the length dimension. In addition, since the boundary 23 is obtained if the total length dimension of the first element 21 and the second element 22 is known, the amount of calculation is smaller than when other methods are used.

2 element extraction means 3 action extraction means 4 control means 10 distance image generation means 20 arithmetic processing device 21 first element 22 second element 31 event detection part 32 angle detection part 33 action determination part

Claims (10)

  Distance image generating means for generating a moving image of the distance image for the first element corresponding to the hand and the second element corresponding to the forearm, and detecting a change in the angle of the first element around the principal axis of the second element An operation input device comprising: an angle detection unit configured to perform an operation determination unit configured to determine that a predetermined operation has been performed when the angle detected by the angle detection unit is within a predetermined detection range.   A distance image generating means for generating a moving image of a distance image having a distance to the object as a pixel value with respect to an object including at least a first element corresponding to a hand and a second element corresponding to a forearm; An element extraction means for extracting an area corresponding to the first element and an area corresponding to the second element and extracting a target surface corresponding to a palm in the first element; and the first element extracted by the element extraction means. A motion extracting means for extracting a change in position and orientation between one element and the second element from the moving image of the distance image; and the motion extracting means detects a prescribed motion between the first element and the second element. Control means for performing correspondence control to the control object when the image is extracted, and the motion extraction means has a normal vector orientation of the target surface in the three-dimensional virtual space formed by the distance image. An event detector that detects a first event that rotates about a principal axis of two elements and a second event that changes the orientation of a direction vector along the principal axis of the second element; and the event detector detects An angle detection unit that detects a rotation angle in the first event and a changing angle in the second event; and the event detection unit starts detecting the first event and the second event within a predetermined determination time. An operation input device comprising: an operation determination unit that determines that the specified operation has been performed when the angle detected by the angle detection unit is within a specified detection range.   The event detection unit identifies the direction of movement of the first element and the second element, and the operation determination unit identifies the type of operation according to the direction of movement identified by the event detection unit The operation input device according to claim 2, wherein:   The operation input device according to claim 2, wherein the motion determination unit includes a setting unit that sets the detection range for the angle detected by the angle detection unit.   The setting unit has two operation modes, a setting mode for setting the detection range and a use mode for determining the specified operation using the detection range. In the setting mode, the setting unit The operation input device according to claim 4, wherein the detection range is set based on an angle detected by the angle detection unit when the prescribed operation is performed a plurality of times.   The element extraction means extracts the first element and the second element with respect to the object using the distance image, and then extracts the first element and the second element in a direction perpendicular to the principal axis in a direction along the principal axis of the first element. 6. The method according to claim 2, wherein a position where the change in the measured maximum width satisfies a specified condition is regarded as a boundary between the first element and the second element. Operation input device.   The element extraction means extracts the first element and the second element with respect to the object using the distance image, and then has a curvature in a plane perpendicular to the principal axis in a direction along the principal axis of the first element. The operation input according to any one of claims 2 to 5, wherein a position where a change in the measured curvature satisfies a specified condition is regarded as a boundary between the first element and the second element. apparatus.   The element extraction means extracts the first element and the second element with respect to the object using the distance image, and then, in the direction along the main axis of the first element, the first element and the second element The position divided at a prescribed ratio with respect to the total length dimension is regarded as a boundary between the first element and the second element. Operation input device.   The computer obtains a moving image of the distance image for the first element corresponding to the hand and the second element corresponding to the forearm from the distance image generating means, and performs a prescribed operation of the first element and the second element. A program for causing a computer to function as an arithmetic processing unit for detection, wherein the computer detects an angle detection unit that detects a change in the angle of the first element around the main axis of the second element, and the angle detection unit detects the computer. A program that functions as an arithmetic processing unit including an operation determination unit that determines that a specified operation has been performed when the angle is within a specified detection range.   The distance from a distance image generating means for generating a moving image of a distance image having a pixel value as a distance to the object with respect to the object including at least a first element corresponding to a hand and a second element corresponding to a forearm. A program for acquiring an image and causing a computer to function as an arithmetic processing device that performs control corresponding to a control target when a prescribed operation of the first element and the second element is detected. An element extraction unit that extracts a region corresponding to the first element and a region corresponding to the second element from a distance image and extracts a target surface corresponding to a palm in the first element, and the element extraction unit extracts Action extracting means for extracting a change in position and orientation between the first element and the second element from the moving image of the distance image; and the action extracting means includes the first element. And a control unit that performs control corresponding to a control target when a prescribed motion between the second element and the second element is detected, and the motion extraction unit is configured in the three-dimensional virtual space formed by the distance image. Event detection for detecting a first event in which the direction of the normal vector of the target surface rotates around the main axis of the second element and a second event in which the direction of the direction vector along the main axis of the second element changes An angle detection unit that detects a rotation angle in the first event detected by the event detection unit and a changing angle in the second event, and the event detection unit detects the first event and the second event. A program that functions as an arithmetic processing unit including an operation determination unit that determines that the specified operation has been performed when the angle detected by the angle detection unit is within a specified detection range within a specified determination time from the start. Beam.

Images