JP2005321966A - Interface method, device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an interface method and device capable of realizing a three-dimensional pointing with excellent usability by a user. <P>SOLUTION: A three-dimensional instructing direction information calculation part 2 determines three-dimensional instructing direction information instructed by the user 6 from two or more input images. A space information registration part 4 registers information of an object in a real space possibly instructed by the user 6 in a space information database 5. A voting processing part 31 performs a ballot box voting processing corresponding to a voxel which the extension line of the direction instructed by the user crosses based on the three-dimensional instructing direction information in a ballot box data memory 33 including ballot boxes corresponding to voxels of voxel spaces obtained by dividing the real space in 1:1. A detection processing part 32 detects the voxel of the three-dimensional position instructed by the user 6 from information for a voting value recorded in each ballot box and the registered object information. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention uses an image taken by a plurality of cameras as an input image, and a position in a real space indicated by a body part (finger, arm, etc.) without being worn by an operator (user) Alternatively, the present invention relates to an interface method and apparatus for detecting an object.

  Regarding the interface between a computer and a human, there are conventional methods as described below as interface methods and apparatuses based on human three-dimensional operations.

  As a first conventional technique, there is an apparatus that attaches a sensor capable of measuring movement to a body and detects movement from sensor information. For example, Tsukada et al.'S Ubi-finger (Non-Patent Document 1) recognizes a gesture by wearing a glove with an acceleration sensor or the like attached to the hand.

  As a second conventional technique, there is a non-contact hand reader (Non-Patent Document 2) of Fukumoto et al. This method connects the coordinate position obtained in the body ("virtual projection center") with the detected coordinate position of the fingertip, and sets the point where the extended pointing axis intersects the screen as the cursor position (pointing position). It is. Since the virtual projection center position is obtained from the intersection of straight lines extending from the four corner positions of the screen to the fingertip, it is non-contact / non-wearing, but the position that can be indicated is only on the screen. The extraction of the fingertip position is realized by detecting the object closest to the screen by using two cameras and installing one at the position where the image is taken from above.

  As a third conventional method, there is a pointing pointer (Non-patent Document 3) of Kinji et al. This method uses the distance image generated using a multi-lens stereo camera to extract the position of the fingertip by detecting the object closest to the screen (screen of the display device displaying the input image), and color information The distance information is used to detect the position between the eyebrows (eyes), and the point where the extended line connecting them crosses the screen is used as the cursor position (indicated position). Although it is non-contact / non-wearing, the position that can be indicated is only on the screen.

As a fourth conventional technique, there is an arm gesture gesture interface (Non-Patent Document 4) of Yamamoto et al. This method is a method of recognizing the user's arm holding direction in real space using a server computer that performs integration processing with a plurality of stereo camera groups. The position in the real space can be pointed by non-contact and non-wearing.
Tsukada et al., “Ubi-Finger: Study on Mobile Oriented Gesture Input Device”, Transactions of Information Processing Society of Japan, 2002.12. Fukumoto et al., “Non-contact hand reader by moving image processing”, Proceedings of 7th Human Interface Symposium, pp. 427-432, 1991 Kinji et al., IEICE Technical Committee on Image Engineering, 2002.1 Yamamoto et al., “Arm gesture gesture interface using ubiquitous stereo vision”, 9th Image Sensing Symposium, 2003.

  However, the conventional method described above has the following problems.

  1) The first conventional method is a pointing method that can recognize the movement of a hand or a finger and can point to a three-dimensional position in real space. However, it is necessary to always wear some device on a body part. It lacks convenience as a practical interface device.

  2) The second and third conventional methods are pointing methods in which the user can indicate the position on the screen by a three-dimensional operation without wearing and non-contacting, but can be indicated on the screen. Therefore, it is not possible to directly indicate a three-dimensional position or object in the real space.

  3) The fourth conventional method is a pointing method in which a user can point to a three-dimensional position in the real space by a three-dimensional operation without wearing and non-contacting, and for determining the pointing position. Although voting processing is used, accuracy improvement techniques such as voting processing around the pointing direction and changing the weight of the voting value have not been used.

  An object of the present invention is to use a problem that lacks convenience due to the wearing type, a problem that can be pointed on a screen but cannot be three-dimensionally pointed into a real space, and a voting process, compared to the conventional method described above. It is an object to provide an interface method, apparatus, and program that solve the problem that the performance improvement technique by voting processing to the periphery in the pointing direction can be performed although the three-dimensional pointing to the real space can be performed.

In order to achieve the above object, the interface method of the present invention comprises:
A spatial information registration step for registering part or all of the information of the object in the real space and the additional information thereof;
From a plurality of input images taken by a plurality of cameras, the three-dimensional coordinates of the start point and end point of the body part of the operator related to the direction indicated by the operator are calculated, and in the real space indicated by the operator A three-dimensional pointing direction information calculating step for obtaining three-dimensional pointing direction information;
The intersection of the registered object and the extension line in the direction indicated by the operator, which is the three-dimensional indication position information indicated by the operator, from the obtained three-dimensional indication direction information and the registered object information Is a three-dimensional pointing position detection step for detecting information relating to the three-dimensional pointing direction information of a ballot box data memory including a ballot box corresponding to each voxel of the voxel space into which the real space is divided. The voting process step for performing voting processing on the ballot box corresponding to the voxel where the extension line in the direction indicated by the operator intersects, and the information of the voting value recorded in each ballot box in the ballot box data memory are registered. A three-dimensional designated position detecting step including a detection processing step for detecting a voxel at the three-dimensional designated position indicated by the operator from the detected object information.

  Since it is not attached, user convenience is improved. In addition, since the pointing direction of the user is detected and the pointing position is detected, three-dimensional pointing can be realized. Further, as a destination for three-dimensional pointing, not only the position on the screen but also the position in the real space can be pointed, and the application destination can be expanded. Further, by using the voting process, it is possible to accurately recognize the position that the user wants to point to. Therefore, the above problems 1) and 2) can be solved.

  According to an embodiment of the present invention, the voting process step further includes the step of performing a voting process on a voxel that is adjacent to the intersecting voxel and in which a voting value differently weighted for each voxel is preset. In the step, a voxel having a vote value integrated by a voting process for temporally continuous images exceeds a predetermined threshold is detected as a voxel at a three-dimensional designated position indicated by the operator.

  Therefore, it is possible to improve accuracy by performing voting processing on the periphery in the pointing direction and changing the weight of the voting value, so that the problem 3) can be solved.

  According to an embodiment of the invention, the weighting is such that the higher the score from the closer to the intersecting voxel.

  According to the embodiment of the present invention, the three-dimensional designated position detecting step includes a step of subtracting a vote value for temporally continuous images.

  Therefore, since the voted vote value can be reduced with the passage of time, it is possible to reduce misrecognition, so that the above problem 3) can be solved.

The present invention has the following effects.
1) According to the inventions of claims 1, 5 and 6, since it is not worn, the convenience of the user is improved, and the pointing direction can be detected by detecting the pointing direction of the user. It can be realized, and as a point of instruction for three-dimensional pointing, not only the position on the screen but also the position in the real space can be pointed, the application destination can be expanded, and the position that the user wants to point by using the voting process Can be accurately recognized.
2) According to the invention of claim 2, it is possible to improve accuracy by performing voting processing on the periphery in the pointing direction and changing the weight of the voting value.
3) According to the invention of claim 4, since the voted vote value can be reduced with time, misrecognition can be reduced and the accuracy can be improved.

  Next, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a configuration diagram of an interface apparatus according to the first embodiment of the present invention, and FIG. 2 is a flowchart showing a flow of the processing.

  The interface apparatus includes image input units 1 a and 1 b, a three-dimensional pointing direction information calculation unit 2, a three-dimensional pointing position detection unit 3, a spatial information registration unit 4, and a spatial information memory 5.

  As the image input units 1a and 1b, two (or three or more) cameras are used as shown in FIG. The camera may be a commonly used video camera or CCD camera, and may be monochrome or color. However, a color camera is required when using a method using color information described later. In the case of two cameras, the cameras are installed at such a distance that image processing by the stereo method is possible and the viewing direction (optical axis) of the camera is parallel or close to parallel in the three-dimensional space. The same applies to the case of three or more units (two included units have the same conditions).

  The three-dimensional indication direction information calculation unit 2 obtains three-dimensional information on the direction (three-dimensional indication direction) indicated by the user 6. Specifically, the three-dimensional indication direction information calculation unit 2 includes, for example, a distance image generation unit 21, a start / end point two-dimensional coordinate calculation unit 22, and a start / end point three-dimensional coordinate calculation unit 23. The three-dimensional information of the three-dimensional indication direction can be obtained.

  The distance image generation unit 21 generates a distance image from two or more input images using stereo image processing (step 101). The distance image is an image that visualizes the distance from the camera to the object. For example, the near image is brightened (larger value) and the far object is darkened (smaller value). The stereo method is also called binocular stereoscopic vision, and is a method of measuring the three-dimensional position of an object by viewing the same object from two different viewpoints, similar to human eyes.

  As an example of a specific image processing method for generating a distance image, there is a method using a commercially available product (Diglops (trinocular camera type) or Bumblebee (binocular camera type) manufactured by Point Gray Research). Each of these is an image input device incorporating two or three cameras, and can generate a distance image as an output. In addition, since the method using the stereo method is common in the field of image processing (many publications), it is possible to make it by using any two or more cameras.

  The start / end point two-dimensional coordinate calculation unit 22 uses at least one input image among the input images used in the distance image generation unit 21 to input images of two predetermined parts (start point and end point) of the user's body. The above two-dimensional coordinates are calculated (step 102). For example, the start point and the end point may be the position of the user's shoulder and the center of gravity of the hand or arm as the end point. Thus, in this case, the direction toward the center of gravity of the hand or arm with the arm extended becomes a three-dimensional instruction direction to be described later. A specific detection method on the input image when the positions of the shoulder and hand (or arm) are set as the start point and the end point will be described below. As a method for detecting the position of the hand or arm by image processing, for example, when an input image is a color image, a skin color component (a color value having an arbitrary width) is obtained from color information such as RGB in the color image. Can be specified in the range of) and labeling. Restriction information such as the size or position of the hand or arm (for example, the range of the skin color area that may be inferred from the size of the hand or arm, or the input image from the obtained multiple skin color parts The target skin color portion of the hand or arm is selected by using a restriction such as excluding a place where a hand or arm is unlikely to exist, such as the vicinity of the upper ceiling or the floor. As a specific example of how to select, if the user is usually wearing clothes, it is likely that the skin color candidate is likely to be both hands (or arms) and face, and the largest area Is considered to be a face, the skin color part having the second and third largest areas is selected as a hand or arm candidate. In the case of the usage method in which two hands (or arms) are used as user-specified positions, the positions of the center of gravity for the two (second and third largest) skin-colored part candidates are specified by the left and right hands or arms, respectively. It may be the position. For left and right selection, the left hand (arm) may be the left hand, and the right hand (arm) may be the right hand. In addition, in the case of a usage method in which only one user-specified position (one hand or one arm) is used, it is necessary to select one from two candidates. For example, if the hand is to specify the right hand (or right arm) in advance, The candidate on the right hand side of the body is likely to be the right hand (or right arm), so it can be selected as the skin color part of the right hand (or right arm), and the center of gravity position can be set as the user-specified position of the right hand (or right arm) . The same applies to the left hand (or left arm). As a method of detecting the shoulder position by image processing, for example, there is a method of first extracting the face position and then calculating the shoulder position. Specifically, first, from the result of performing the skin color extraction process, since the skin color part having the first largest area is highly likely to be a face, the skin color part is determined to be a face, and the center of gravity is obtained. Next, it can be assumed that the shoulder position is shifted from the center of gravity of the face by a certain distance downward and by a certain distance to the left and right. (There are individual differences, and the value may be changed by the user 6), and the left and right shoulder positions can be calculated from the center of gravity of the face. Further, when outputting the two-dimensional coordinates of the start point / end point, a plurality of candidate values may be obtained and a plurality of values may be output to the start / end point three-dimensional coordinate calculation unit 23. In this case, the start / end point three-dimensional coordinate calculation unit 23 is used when obtaining the start / end point three-dimensional coordinates. Thus, the two-dimensional coordinates on the input image of the start point and end point can be obtained. Although the shoulder position is the starting point here, the face position (center of gravity) obtained as described above may be used as it is. In that case, the extension line connecting the position of the face and the position of the hand (or arm) is the direction instructed by the user 6.

  The start / end point three-dimensional coordinate calculation unit 23 calculates a three-dimensional coordinate value of the start point / end point from the generated distance image information and the two-dimensional coordinates of the start point / end point (step 103). As a specific method, for example, a value (distance value) at the same position as the two-dimensional coordinate on the input image of the starting point on the distance image may be referred to and used as the starting point distance value. The same applies to the end point. In the three-dimensional real space, the conversion between the two-dimensional coordinate system and the three-dimensional coordinate system of the input image can generally be easily calculated in advance, so that the start point and the end point 2 on the input image obtained based on the conversion can be easily obtained. From the three-dimensional coordinate value and each distance value thereof, the three-dimensional coordinate value in the three-dimensional space of the start point and the end point can be obtained. Furthermore, by obtaining a three-dimensional straight line connecting the two points from the obtained two three-dimensional coordinate values of the start point and end point, the user's indication direction can be obtained. In addition, when a plurality of two-dimensional coordinates of the start point / end point are input (described by the start / end point two-dimensional coordinate calculation unit 22), a plurality of two-dimensional coordinate candidates can be selected based on the distance image information. For example, assuming that the position where the user 6 is located is only in a space restricted in advance, it is possible to remove candidates indicating a place exceeding the restriction. That is, the start / end point three-dimensional coordinate calculation unit 23 can narrow down not only the start / end point three-dimensional coordinates but also the start / end point two-dimensional coordinates. Thereby, since the erroneous detection in the start / end point two-dimensional coordinate calculation part 22 can be excluded using different information (distance image information), an improvement in accuracy can be expected.

  The spatial information registration unit 4 registers information such as an object in the real space that the user 6 may instruct in the spatial information memory 5 (step 104). As an object in the real space, for example, when the user 6 is in a room, home appliances in the room (TV, air conditioner, computer, clock, window, shelf, chair, desk, drawer, Documents, audio equipment, lighting equipment, etc.) objects, and walls, floors, ceilings, windows, etc. of the room itself can be considered as objects. Information on these objects and the like (three-dimensional position coordinate information and information on other objects) is registered and stored in advance in the spatial information data memory 5 as shown in Table 1.

  Regarding the registration of information, a sensor capable of recognizing a position for each target real object (commercially available magnetic sensor, ultrasonic sensor, infrared tag, wireless tag) is not used as fixed three-dimensional position coordinates in advance. Etc.), the position of each object can be recognized in real time. Therefore, the object information is generated from the three-dimensional position information obtained from the object, and the three-dimensional position coordinates of the object are always generated. It is also possible to update the information. In this case, even if the object is moved, the three-dimensional position information and the like can be updated in real time.

  The three-dimensional designated position detection unit 3 detects a three-dimensional position in the real space pointed to by the user 6, and includes, for example, a voting processing unit 31, a detection processing unit 32, and a voting box data memory 33. Here, a three-dimensional voxel space obtained by dividing a three-dimensional real space in advance is set (FIG. 3). For example, when a space of width 5 m × depth 5 m × height 5 m is divided into 10 parts each (ie, divided into 10 × 10 × 10 voxels), one voxel has a size of 50 cm × 50 cm × 50 cm. Corresponds to the region. Based on the coordinate information of the object registered in the spatial information registration unit 4, which voxel belongs to each object (where it is spatially located) is obtained in advance, and each registered in the spatial information data memory 5. Corresponding voxel coordinates are obtained for each object. One voxel is provided with one ballot box. Here, the number of divisions in the real space is (Xdiv, Ydiv, Zdiv), and the voxel at coordinates (X, Y, Z) is V (X, Y, Z). The ballot box is a memory that can store a numerical value (which may be an integer or a real number), and it is sufficient if there is memory for Xdiv × Ydiv × Zdiv. A memory having these memories is called a ballot box data memory. The values (voting values) stored in all the ballot boxes are initialized once (for example, the value 0 is set).

  The voting processing unit 31 first extends the three-dimensional straight line connecting the start point and the end point obtained by the start / end point three-dimensional coordinate calculation unit 23 of the three-dimensional indication direction information calculation unit 2 in the hand or arm (end point) direction. At this time, the voxel (intersection voxel) where the extension lines intersect is calculated (step 105, FIG. 4). When the extension line passes through the voxel space, it intersects a plurality of voxels, so that all the intersecting voxels are obtained. Each voting process is performed for each voting box corresponding to the obtained cross voxel (step 105). The voting process is a process of adding a predetermined voting value to the voting box. The value of the voting box of the voxel where the extension line intersects is 0 + v = v where the initial value is 0 and the voting value is v. The voxels to be voted may be all of the intersecting voxels, but among the plurality of intersecting voxels, an object is associated with the voxel by association with the spatial information data performed in advance (as described above). Only in some cases, a method of voting can be considered (FIG. 5). Thereby, it is not voted for the voxel (namely, ballot box) of the space which has nothing, and misrecognition can be reduced. A three-dimensional pointing direction information calculation process (steps 101 to 103) and a three-dimensional position detection process (steps 105 and 106) are performed for each image group that is input continuously in time. Therefore, since the voting process is also performed for each image, the voting value of each voxel in the voting box data memory 33 is integrated with time.

  The detection processing unit 32 determines in advance among the vote values accumulated in time in each voxel in the ballot box data memory 33 by voting by the voting processing unit 31 for each temporally continuous image. A voting box (that is, a voxel) having a voting value exceeding the threshold is searched (step 106). Here, the threshold value is that a voxel having the vote value is a three-dimensional designated position designated by the user 6 when the vote value accumulated by the voting process for the temporally continuous images exceeds the threshold value. It is a value for determination, and may be arbitrarily set, for example, voting value v = 1 and threshold value th = 5. When the threshold value is set higher than the one-time vote value as in this example, since a plurality of images are voted on the same ballot box (voxel) (in this case, five images), the user 6 has a certain time ( (This may be a continuous time or a discontinuous time.) By pointing, the target voxel is detected, so that the non-target voxel may be erroneously detected during the movement when the user 6 moves the arm. The detection performance can be increased significantly because the number is greatly reduced. The searching method includes a method of searching for a voting box (voxel) having a voting value exceeding a threshold value from all the voxels in the voxel space. In addition, there is a method of searching only from among the intersecting voxels. By doing so, the number of voxels to be searched can be reduced and the processing amount can be reduced. When ballot boxes (voxels) exceeding the threshold are detected, all the ballot boxes may be initialized (set to 0).

  As described above, according to the present embodiment, the user 6 extends his / her arm and points an object or the like in the three-dimensional space directly in the real space for a certain period of time, thereby indicating the indicated three-dimensional position (object). It becomes possible to detect.

[Second Embodiment]
FIG. 6 is a block diagram of the interface apparatus according to the second embodiment of the present invention, and FIG. 7 is a flowchart showing the processing flow.

  The interface device includes image input units 1 a and 1 b, a three-dimensional pointing direction information calculation unit 2, a three-dimensional pointing position detection unit 3, a spatial information registration unit 4, and a spatial information data memory 5. The three-dimensional designated position detection unit 3 includes a voting processing unit 31, a detection processing unit 32, and a voting box data memory 33. The voting processing unit 31 includes a cross voxel voting processing unit 31a and an adjacent voxel voting processing unit 31b. The image input units 1a and 1b, the three-dimensional indication direction information calculation unit 2, the spatial information registration unit 4, and the spatial information data memory 5 have the same configuration as in the first embodiment.

  As in the first embodiment, the intersecting voxel voting processing unit 31a first calculates a three-dimensional straight line connecting the start point and the end point obtained by the start / end point three-dimensional coordinate calculation unit 23 of the three-dimensional indication direction information calculation unit 2. Or extend in the arm (end point) direction. At this time, the voxel (intersection voxel) where the extension lines intersect is calculated. When the extension line passes through the voxel space, it intersects a plurality of voxels, so that all the intersecting voxels are obtained. Each voting process is performed for each voting box corresponding to the obtained cross voxel (step 107). The voting process is a process of adding a predetermined voting value to the voting box.

  The adjacent voxel voting processing unit 31b performs a voting process for each voxel for which voting is performed by the intersecting voxel voting processing unit 31a (step 108). The voxels to be voted may be, for example, all the voxels in the vicinity of 26 of the voxels in the three-dimensional voxel space, may be the voxels in the vicinity of 6, the voxels in the vicinity of 18, the four voxels in the two dimensions, or the eight voxels in the vicinity. Voxels may be used (FIG. 8). The values to be voted (voting values) may be the same for all neighbors or different values. For example, when voting to 26 neighborhoods is considered, the vote value for the central intersecting voxel is a, the vote values for the neighboring neighborhood voxels are b, and the 6 neighborhood voxels are excluded from the 18 neighborhood voxels. The neighboring voxel may be c, and the neighboring voxel obtained by removing the 18 neighboring voxels from the 26 neighboring voxels may be d. Here, a = b = c = d = 1 (or a value other than 1) may be set, or the value is increased as the voxel is closer to the center (cross voxel), for example, a = 10, b = 5, c = 2 and d = 1 (FIG. 9). If b = c = d = 0, the voting process only for the intersecting voxels, if c = d = 0, the voting process for 6 neighborhoods, and if d = 0, the voting process for 18 neighborhoods are the same. is there. By increasing the value of the voxel closer to the center, the accuracy of the recognition result by the voting process can be improved. This is in the correct direction even when the three-dimensional indication direction is shifted to the surrounding direction from the correct direction to be originally indicated due to an error included in the information of the three-dimensional indication direction obtained by the three-dimensional indication direction information calculation unit 2. This is because voting for voxels is also performed, and an improvement in accuracy can be expected. In addition, as a result, when the vote values are integrated with time, it is possible to expect an increase in speed due to the voxels in the correct direction quickly exceeding the threshold.

  A three-dimensional pointing direction information calculation process 100, a voting process (a cross voxel voting process 107, an adjacent voxel voting process 108), and a detection process 109 are performed for each image group that is input continuously in time. Therefore, since the voting process is also performed for each image, the voting value of each voxel in the voting box data memory 33 is integrated with time.

  As in the first embodiment, the detection processing unit 32 searches for a voting box (that is, a voxel) having a voting value exceeding a predetermined threshold value among the voting values of each voxel in the voting box data memory 33. (Step 106). The threshold value is used to determine that the voxel having the vote value is the three-dimensional designated position designated by the user 6 when the accumulated value obtained by the voting process on the temporally continuous images exceeds the threshold value. For example, voting values a = 10, b = 5, c = 2, d = 1, threshold th = 20, etc. may be set arbitrarily. As in the first embodiment, the searching method is to search for a voting box (voxel) having a voting value exceeding a threshold from all voxels in (the corresponding voting box) in the voxel space, There is also a way to search from within voxels. Further, it may be searched from the voted cross voxels and neighboring voxels. In this way, the number of voxels to be searched can be reduced and the processing amount can be reduced. When ballot boxes (voxels) exceeding the threshold are detected, all the ballot boxes may be initialized (set to 0).

  As described above, according to the present embodiment, the user 6 extends his / her arm and points an object or the like in the three-dimensional space directly in the real space for a certain period of time, thereby indicating the indicated three-dimensional position (object). In addition to being able to detect, it is possible to detect an object with higher accuracy and higher speed than in the first embodiment.

[Third Embodiment]
FIG. 10 is a block diagram of an interface apparatus according to the third embodiment of the present invention, and FIG. 11 is a flowchart showing the processing flow.

  The interface device includes image input units 1 a and 1 b, a three-dimensional pointing direction information calculation unit 2, a three-dimensional pointing position detection unit 3, a spatial information registration unit 4, and a spatial information data memory 5. The three-dimensional indication position detection unit 3 includes a voting processing unit 31, a detection processing unit 32, a voting box data memory 33, and a voting value subtraction processing unit 34. The voting processing unit 31 includes a cross voxel voting processing unit 31a and an adjacent voxel voting process. It consists of the part 31b. Image input unit 1a, 1b, three-dimensional pointing direction information calculation unit 2, cross voxel voting processing unit 31a, adjacent voxel voting processing unit 31b, detection processing unit 32, ballot box data memory 33, spatial information registration unit 4, spatial information data The memory 5 has the same configuration as that of the second embodiment.

  The voting value subtraction processing unit 34 performs a three-dimensional pointing direction information calculation process (step 100), a voting process (steps 107 and 108), and a detection process (for each image) for an image group that is input continuously in time. After step 106), the voting value accumulated in the ballot box data memory 33 is subtracted with the passage of time (every time the image is changed and the next step is performed).

For example, there are the following methods for the subtraction process. Let t (i) be the time when an image is input, and t (i + 1) when the next successive image is input. Subtract the voting value accumulated in an arbitrary voxel (x, y, z) in the voting data memory 33 after voting in the process at time t (i) as v (i) (x, y, z). If the value is s, for all voxel ballot boxes,
v (i + 1) = v (i) (x, y, z) -s
It can be executed by doing. When the value of the ballot box is 0, s need not be subtracted.

  In the first and second embodiments, the value of the accumulated ballot box continues to increase with time until one of the ballot boxes of each voxel is detected exceeding the threshold value, so that the user 6 votes. Although there is a problem that the accumulated value remains in the place where the user 6 does not intend to perform, the vote value subtraction process according to the present embodiment causes the voxel in the direction that the user 6 has not pointed for a while to accumulate the vote. Since the value decreases with time and settles to 0 at the end, there is an advantage that it is possible to reduce false detection that the user 6 detects unintended voxels.

In the above description, the subtraction value is -s. However, there is a method of reducing the value not by subtraction but by multiplication. For example, if the value to be multiplied is m and m is a real number between 0 and 1,
v (i + 1) = v (i) (x, y, z) × m
By performing the above, the value can be reduced with time. Although the obtained value is a real number as it is, it may be converted into an integer such as truncating after the decimal point for easy calculation. When subtraction is performed by subtraction (-s), it decreases monotonically with time. However, when it is performed by multiplication (xm), the amount of decrease decreases with time. Use them as needed.

  As described above, according to the present embodiment, the user 6 extends his / her arm and points an object or the like in the three-dimensional space directly in the real space for a certain period of time, thereby indicating the indicated three-dimensional position (object). In addition to being able to detect, it is possible to detect an object with higher accuracy and higher speed than in the first embodiment, and to reduce the possibility of erroneous detection of a voxel position that the user 6 does not intend. And accuracy can be improved.

  The functions of the interface device described above are not realized by dedicated hardware, but a program for realizing the functions is recorded on a computer-readable recording medium and recorded on the recording medium. The program may be read into a computer system and executed. The computer-readable recording medium refers to a recording medium such as a floppy disk, a magneto-optical disk, a CD-ROM, or a storage device such as a hard disk device built in the computer system. Furthermore, a computer-readable recording medium is a server that dynamically holds a program (transmission medium or transmission wave) for a short period of time, as in the case of transmitting a program via the Internet, and a server in that case. Some of them hold programs for a certain period of time, such as volatile memory inside computer systems.

It is a block diagram of the interface apparatus of the 1st Embodiment of this invention. It is a flowchart which shows the flow of a process of the interface apparatus of 1st Embodiment. It is a figure which shows the example of a voxel space. It is explanatory drawing of the example which performs a voting process to all the crossing voxels. It is explanatory drawing of the example which performs a voting process only to the crossing voxel with which the object is registered. It is a block diagram of the interface apparatus of the 2nd Embodiment of this invention. It is a flowchart which shows the flow of a process of the interface apparatus of 2nd Embodiment. It is a figure which shows the example of the voxel to vote. It is a figure which shows the weighted vote value (the figure (a)) and the example of the weighted vote value (the figure (b)). It is a figure which shows the structure of the interface apparatus of the 3rd Embodiment of this invention. It is a flowchart which shows the flow of a process of the interface apparatus of 3rd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1a, 1b Image input part 2 3D designation | designated direction information calculation part 3 3D designation | designated position detection part 4 Spatial information registration part 5 Spatial information data memory 6 User 21 Distance image generation part 22 Start / end point two-dimensional coordinate calculation part 23 Start / end point 3 Dimensional coordinate calculation unit 31 Voting processing unit 31a Cross voxel voting processing unit 31b Adjacent voxel voting processing unit 32 Detection processing unit 33 Ballot box data memory 34 Voting value subtraction processing unit 100 to 109 Steps

Claims (6)

An interface method for inputting images taken by a plurality of cameras and recognizing a position or an object in a real space indicated by an operator using a body part,
A spatial information registration step for registering part or all of the information of the object in the real space and the additional information thereof;
From a plurality of input images photographed by a plurality of cameras, the three-dimensional coordinates of the start point and end point of the body part of the operator related to the direction indicated by the operator are calculated, and in the real space indicated by the operator A three-dimensional pointing direction detecting step for obtaining three-dimensional pointing direction information of
From the obtained three-dimensional indication direction information and the registered object information, the extension line in the direction indicated by the operator, which is three-dimensional indication position information indicated by the operator, and the registered object 3D pointing position detecting step for detecting information related to the intersection of the three-dimensional pointing direction information of a ballot box data memory including a ballot box corresponding to each voxel in the voxel space obtained by dividing the real space on a one-to-one basis A voting process step for performing voting on a ballot box corresponding to a voxel intersecting with an extension line in the direction indicated by the operator, and information on voting values recorded in each ballot box in the ballot box data memory; A three-dimensional pointing position detection step including a detection step of detecting a voxel at the three-dimensional pointing position indicated by the operator from the registered object information.   The voting process step further includes a step of voting a voxel adjacent to intersecting voxels and preset with voting boxes having different weights for each voxel, and the detecting step is continuous in time The interface method according to claim 1, wherein a voxel having a voting value accumulated by a voting process for a selected image exceeds a predetermined threshold is detected as a voxel at a three-dimensional designated position indicated by the operator.   The interface method according to claim 2, wherein the weighting is such that a higher score is obtained from a side closer to the intersecting voxel.   The interface method according to claim 1, wherein the three-dimensional indication position detection step further includes a step of subtracting the vote value for temporally continuous images. An interface device for inputting images taken by a plurality of cameras and recognizing a position or an object in a real space indicated by an operator using a body part,
Spatial information registration means for registering part or all of the information of the object in the real space and the additional information thereof;
From a plurality of input images photographed by a plurality of cameras, the three-dimensional coordinates of the start point and end point of the body part of the operator related to the direction indicated by the operator are calculated, and in the real space indicated by the operator 3D pointing direction information calculating means for obtaining 3D pointing direction information of
Based on the obtained three-dimensional indication direction information and the registered object information, the extension line in the direction indicated by the operator, which is three-dimensional indication position information indicated by the operator, and the registered object A three-dimensional pointing position detecting means for detecting information related to the intersection, a ballot box data memory including a one-to-one ballot box corresponding to each voxel in the voxel space obtained by dividing the real space, and the three-dimensional pointing direction information Voting processing means for performing voting processing on the ballot box corresponding to the voxel through which the extension line in the direction indicated by the operator passes, and information on the voting value recorded in each ballot box in the ballot box data memory, An interface device comprising: three-dimensional pointing position detection means including detection processing means for detecting voxels at the three-dimensional pointing position indicated by the operator from registered object information.   An interface program for causing a computer to execute the interface method according to claim 1.

Images