JP2005084817A - Three-dimensional object attitude operation method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To operate the attitude of a three-dimensional object in real time at a low calculating cost. <P>SOLUTION: When a three-dimensional position 100 of the fingertip outside a virtual operation region 130 enters the virtual operation region 130, an operation mode is shifted to an attitude operation mode. At that time, a unit vector 141 going from a reference point 102 to the contact 140 of a border plane 131 between the three-dimensional position 100 of the fingertip and the virtual operation region 130 and the attitude of a three-dimensional object 210 are stored. During the attitude operation, the attitude of the three-dimensional object 210 stored when the operation mode is shifted to the attitude operation mode is rotated only by an angle shown by 144 around an axis 143, and defined as the present attitude of the three-dimensional object 210. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a method for manipulating the attitude of a three-dimensional object.

There is Patent Document 1 as a patent application related to the above technique, but in this patent application, only a three-dimensional position and direction indicated by an operator is detected, and a method for operating the posture of an object is described. It is not done. Moreover, although there exists patent document 2, since it is a matching system which used the hand image of various attitude | positions as a template, the attitude | position detected is discrete and calculation cost starts.
JP-A-10-326148 JP-A-8-076917

  Although it is possible in principle to detect the posture of a palm or fist and manipulate the posture of a three-dimensional object using this posture information, calculation is necessary to detect the posture of a palm or fist with sufficient accuracy. Expensive and real-time processing is impossible and impractical. Conversely, in order to enable real-time processing, the accuracy of posture detection such as palms and fists must be reduced. The degree of discreteness of the posture is large and impractical.

  An object of the present invention is to provide a three-dimensional object posture operation method and program for manipulating the posture of a three-dimensional object in a computer in real time.

According to the first aspect of the present invention, the three-dimensional object posture operation method comprises:
Set a virtual work area that is fixed in the spatial coordinate system, including a fixed reference point,
From the idle mode in which the operation mode is in the space, such as a fingertip or a fingertip with a simple marker attached, outside the virtual work area, and the posture in which the three-dimensional position is on and inside the boundary surface of the virtual work area When transitioning to the operation mode, the position of the intersection of the three-dimensional position and the boundary surface of the virtual work area, the posture of the three-dimensional object, and the plane perpendicular to the plane including the three-dimensional position, the intersection, and the reference point And a straight line passing through the reference point,
Only when the operation mode is the posture operation mode, the three-dimensional object is subjected to a rotation transformation that superimposes a unit vector from the reference point toward the intersection with a unit vector from the reference point toward a three-dimensional position, with the straight line as an axis. The resulting posture is set as the current three-dimensional object posture.

According to the second aspect of the present invention, the three-dimensional object posture operation method comprises:
In addition to the virtual work area, an internal virtual work area included in the virtual work area and fixed in the space is set,
When the operation mode is changed from the idle mode where the three-dimensional position is outside the virtual work area to the posture operation mode when the three-dimensional position is inside the internal virtual work area or in contact with the boundary surface Furthermore, the position of the intersection of the three-dimensional position and the boundary surface of the internal virtual work area, the posture of the three-dimensional object, the three-dimensional position, the plane perpendicular to the plane including the intersection and the reference point, and pass through the reference point Remember the straight line,
Only when the operation mode is the posture operation mode, the three-dimensional rotation transformation that superimposes the unit vector from the reference point to the intersection with the unit vector from the reference point to the three-dimensional position with the straight line as an axis. This is applied to the posture of the object, and the resulting posture is set as the current three-dimensional object posture.

According to the third aspect of the present invention, the three-dimensional object posture operation method comprises:
A virtual work area fixed in the space corresponding to the number of three-dimensional objects and corresponding to the three-dimensional objects on a one-to-one basis is set.
Providing a reference point fixed in the space, which is the same number as the number of the three-dimensional object and corresponds to the three-dimensional object on a one-to-one basis;
Set the same number of internal virtual work areas that are included in each virtual work area and fixed in the space as the number of the three-dimensional objects and do not intersect each other,
The operation mode changes from the idle mode in which the three-dimensional position is outside all virtual work areas to the posture operation mode when the three-dimensional position enters the inside of one internal virtual work area or touches its boundary surface. The position of the intersection between the 3D position and the boundary surface of the internal virtual work area, the posture of the 3D object corresponding to the internal virtual work area, the 3D position, the intersection, and the internal virtual work area And a straight line passing through the reference point and perpendicular to a plane including the reference point
Only when the operation mode is the posture operation mode, a rotation transformation that superimposes a unit vector from the reference point to the intersection with a unit vector from the reference point to a three-dimensional position with the straight line as an axis is a three-dimensional object. The resulting posture is set as the current three-dimensional object posture.

  According to the present invention, by moving the position of a fingertip or the like, it is possible to easily specify gripping and releasing of a virtual object or to easily operate the posture of the virtual object.

Next, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]

  In the present embodiment, by analyzing a camera image obtained by imaging a marker attached to the fingertip, the three-dimensional movement of the fingertip in the space is captured, and the posture of the three-dimensional object in the computer is manipulated. 1 to 5 show a method for detecting the three-dimensional position of the fingertip in the space.

  In FIG. 1, the camera 104 and the camera 105 are fixed to a spatial coordinate system 190, and the central axes of the cameras 104 and 105 intersect at a reference point 102 on the Z axis of the spatial coordinate system 190. The display monitor 103 is placed between the camera 104 and the camera 105. The marker 101 attached to the fingertip is a sphere painted in fluorescent red, but a light emitting diode having a wavelength of about 900 nanometers can also be used as a marker.

  In FIG. 2, images 200 and 201 are images obtained by processing the entire red channel images of the cameras 104 and 105 capturing the fingertip marker 101 with a Gaussian filter. The fingertip marker images 202 and 203 are displayed as high brightness areas.

  In FIG. 3A, the camera coordinate system 110 is fixed to the spatial coordinate system 190, and its Z axis coincides with the central axis of the camera 104. A straight line 113 is a straight line that connects the position of the pixel 112 with the highest luminance on the image plane 111 and the origin of the camera coordinate system 110. In FIG. 3B, the camera coordinate system 120 is fixed to the spatial coordinate system 190, and its Z axis coincides with the central axis of the camera 105. A straight line 123 is a straight line that connects the position of the pixel 122 with the highest luminance on the image plane 121 and the origin of the camera coordinate system 120.

  FIG. 4 shows a straight line 113 and a straight line 123 in the spatial coordinate system 190. Since the camera coordinate system 110 and the camera coordinate system 120 are fixed to the spatial coordinate system 190, the straight line 113 and the straight line 123 whose positions and orientations are specified in the respective camera coordinate systems 110 and 120 are the same in the spatial coordinate system 190. The location is also specified.

FIG. 5 shows a method for detecting the three-dimensional position of the fingertip in real space. Intersections between the straight line 113 and the straight line 123 whose positions in the spatial coordinate system 190 are identified in the shortest and the straight line 113 and the straight line 123 are defined as P ₁ and P ₂ , respectively. A midpoint P _M between P ₁ and P _{2 is set} as a three-dimensional position 100 of the fingertip in the real space.

  FIGS. 6 to 11 show a method of operating a three-dimensional object in the computer by the three-dimensional position of the fingertip in the real space.

  FIG. 6 shows a virtual work area 130 fixed to the spatial coordinate system 190 so as to include the reference point 102.

  FIG. 7 shows a state of the idle mode in which the operation mode is before the start of operation. FIG. 7A is a monitor screen on which a three-dimensional object 210 to be operated, a cursor 211 indicating the fingertip position, and an image 212 of the boundary 131 of the virtual work area 130 are displayed. FIG. 7B shows a state where the three-dimensional position 100 of the fingertip is outside the virtual work area 130.

  FIG. 8 illustrates a state in which the operation mode is changed to the posture operation mode when the three-dimensional position 100 of the fingertip outside the virtual work area 130 enters the virtual work area 130. At that time, the unit vector 141 from the reference point 102 shown in FIG. 10 toward the contact point 140 between the three-dimensional position 100 of the fingertip and the boundary surface 131 of the virtual work area 130 and the posture of the three-dimensional object 210 are stored. deep. At this time, as a method of notifying the operator of the transition of the operation mode, the display color of the three-dimensional object 210 is changed, the display color of the image 212 of the boundary 131 and the cursor 211 of the virtual work area 130 is changed, or a chime is sounded. There is a way. While the three-dimensional position 100 of the fingertip is inside the virtual work area 130 or on the boundary 131, the posture operation mode is continued.

  FIG. 9 shows a state during posture operation. Regarding the current posture of the three-dimensional object 210, the posture of the three-dimensional object 210 stored when the motion mode is changed to the posture operation mode is indicated by 144 around the axis 143 shown in FIG. The posture rotated by an angle is taken as the current posture. Here, the axis 143 is a straight line perpendicular to a plane including both the unit vector 141 shown in FIG. 10A and the unit vector 142 from the reference point 102 toward the three-dimensional position 100 of the fingertip and passing through the reference point 102. It is. An angle indicated by 144 is an angle formed by the unit vector 142 with respect to the unit vector 141.

FIG. 11 shows a state where the posture operation is completed when the three-dimensional position 100 of the fingertip comes out of the virtual work area 130 and the operation mode changes from the posture operation mode to the idle mode. This state is the same as the state shown in FIG.
[Second Embodiment]

  In the first embodiment, at the time of the operation mode transition from the idle mode to the posture operation mode described in FIG. 8 or at the time of the operation mode transition from the posture operation mode to the idle mode described in FIG. Chattering may occur when noise or the like overlaps with the detected three-dimensional position 100 of the fingertip.

  As a second embodiment, FIGS. 12 to 16 show a method of operating a three-dimensional object in a computer based on the three-dimensional position of a fingertip in real space while preventing chattering.

  FIG. 12 shows a state where the operation mode is the idle mode. As shown in FIG. 12B, in addition to the virtual work area 130 shown in FIG. 7, an internal virtual work area 150 included therein is used. It is assumed that the minimum value of the distance between the boundary surface 131 and the boundary surface 151 of both virtual work areas 130 and 150 is sufficiently larger than the level of noise superimposed on the three-dimensional position 100 of the fingertip. In the display monitor 103 shown in FIG. 12A, the image 215 of the boundary surface 131 of the virtual work area 130 is not displayed or is displayed in a light color, while the image 213 of the boundary surface 151 of the internal virtual work area 150 is displayed. Is displayed in dark color and informs the operator of the boundary surface to be tilted.

  FIG. 13 shows a state of transition from the operation mode idle mode to the posture operation mode. Only in the idle mode, when the three-dimensional position 100 of the fingertip outside the internal virtual work area 150 enters the internal virtual work area 150 or comes into contact with the boundary surface 151, the operation mode transitions to the posture operation mode. At that time, the unit vector 141 from the reference point 102 shown in FIG. 15 toward the contact point 140 between the three-dimensional position 100 of the fingertip and the boundary surface 131 of the virtual work area 130 and the posture of the three-dimensional object 210 are stored. deep. At this time, the display color of the cursor 211 is changed, the chime is sounded, or the image 213 of the boundary surface 151 of the internal virtual work area 150 is deleted or changed to a light color display, while the boundary surface 131 of the virtual work area 130 is changed. By changing the image 215 to a dark color display, the operator is notified of the transition of the operation mode, and the boundary surface to be tilted is also notified. While the three-dimensional position 100 of the fingertip is inside the virtual work area 130, the posture operation mode is continued.

  FIG. 14 shows a state during the posture operation. As for the current posture of the three-dimensional object 210, the posture of the three-dimensional object 210 stored when the operation mode and the posture operation mode are changed is indicated by 144 around the axis 143 shown in FIG. The posture rotated by an angle is taken as the current posture. Here, the axis 143 is a straight line perpendicular to a plane including both the unit vector 141 shown in FIG. 15A and the unit vector 142 from the reference point 102 toward the three-dimensional position 100 of the fingertip and passing through the reference point 102. It is. An angle indicated by 144 is an angle formed by the unit vector 142 with respect to the unit vector 141.

FIG. 16 shows a transition of the operation mode from the posture operation mode to the idle mode. Only in the posture operation mode, when the three-dimensional position 100 of the fingertip comes out of the virtual work area 130 or comes into contact with the boundary surface 131, the mode changes to the idle mode.
[Third Embodiment]

As shown in FIG. 17, instead of the internal virtual work area 150 in the second embodiment, an internal virtual work area 160 that is an area having the same shape as the three-dimensional object 210 is used.
[Fourth Embodiment]

  In the fourth embodiment, a case will be described in which there are two virtual work areas similar to the virtual work area shown in FIG. 7 of the first embodiment, and these virtual work areas intersect.

As shown in FIG. 18, the areas obtained by removing the intersecting areas of the two internal virtual work areas from the respective internal virtual work areas are defined as an internal virtual work area 150 and an internal virtual work area 550, respectively. The posture of the three-dimensional object is manipulated by the same method as in the embodiment.
[Fifth Embodiment]

  In the first embodiment, a detection engine using a marker is used as means for detecting the three-dimensional position of the fingertip. However, instead of this engine, an engine for detecting the position of the fingertip without anything is attached. It can also be used. As such a fingertip three-dimensional position detection engine, there is “Robust Finger Tracking with Multiple Cameras”, Cullen Jennings, 1999, and the like.

  FIG. 19 is a block diagram of an apparatus that implements the three-dimensional object posture operation methods of the first to fifth embodiments described above.

  The fingertip or marker position detection unit 301 detects the three-dimensional position 100 of the fingertip or the marker 101 attached to the fingertip. The area determination unit 302 determines whether the three-dimensional position 100 of the fingertip or the marker 101 is located outside the virtual work area 130 (in addition, the internal virtual work area 150 in the second embodiment) or the outside of the virtual work area 130. . The operation mode determination unit 304 determines the operation mode (idle mode, posture operation mode) from the determination search of the region determination unit 302, stores the determined operation mode in the operation mode storage unit 303, and the operation mode changes from the idle mode to the posture. When transitioning to the operation mode, the unit vector 41 and the posture of the three-dimensional object 210 are stored in the position information storage unit 305. The posture calculation unit 306 performs calculation for rotating the posture of the three-dimensional object 210 around the axis 143 by an angle 144 using the information stored in the position information storage unit 305 when the operation mode is the posture operation mode.

  The three-dimensional object posture operation method of the present invention described above records a program for realizing the function on a computer-readable recording medium, and reads the program recorded on the recording medium into a computer system. Can be executed. The computer-readable recording medium refers to a recording medium such as a floppy disk, a magneto-optical disk, a CD-ROM, or a storage device such as a hard disk device built in the computer system. Furthermore, a computer-readable recording medium is a server that dynamically holds a program (transmission medium or transmission wave) for a short period of time, as in the case of transmitting a program via the Internet, and a server in that case. Some of them hold programs for a certain period of time, such as volatile memory inside computer systems.

It is a figure which shows a mode that the fingertip marker 101 is image | photographed with two cameras 104,105. It is a figure which shows the image which processed the whole red channel image of the cameras 104 and 105 which image | photographed the fingertip marker 101 by the Gaussian filter. It is a figure which shows the straight lines 113 and 123 which connect the image 112 and 122 with the highest brightness | luminance on the image planes 111 and 121, and the origin of the camera coordinate system 120. FIG. It is a figure which shows the straight lines 113 and 123 in the spatial coordinate system 190. FIG. It is a figure which shows the method of detecting the three-dimensional position of the fingertip in real space. FIG. 4 is a diagram showing a virtual work area 130 including a reference point 102. It is a figure which shows the state whose operation mode is an idle mode. It is a figure which shows the state which operation mode changed to attitude | position operation mode. It is a figure which shows the state during attitude | position operation. FIG. 6 is a diagram illustrating a change in posture of a three-dimensional object 210. It is a figure which shows the state which operation mode changed from attitude | position mode to idle mode, and attitude | position operation was completed. It is a figure which shows the idle mode in 2nd Embodiment. It is a figure which shows the mode of the operation mode transition from idle mode in 2nd Embodiment to attitude | position operation mode. It is a figure which shows the state during attitude | position operation in 2nd Embodiment. It is a figure which shows the change of the attitude | position of the three-dimensional object 210 in 2nd Embodiment. It is a figure which shows the mode of the operation mode transition from the attitude | position operation mode in 2nd Embodiment to idle mode. It is a figure which shows the internal virtual work area used by 3rd Embodiment. It is a figure which shows the virtual work area used in 4th Embodiment. It is a block diagram of the apparatus which implements the three-dimensional object attitude | position operation method of 1st-5th embodiment.

Explanation of symbols

100 Estimated Marker Position in Real Space 101 Marker Position in Real Space 102 Attitude Operation Reference Point on Z-axis of Real Space Coordinate System 103 Display Monitor 104 Camera 105 Camera 190 Real Space Coordinate System 200 Camera Image 201 Camera image 202 Marker position on camera image 203 Marker position on camera image 110 Camera coordinate system 111 Camera imaging surface 112 Marker position on camera imaging surface 113 Origin of camera coordinate system and marker position on camera imaging surface 120 Camera coordinate system 121 Camera imaging plane 122 Marker position on camera imaging plane 123 Straight line connecting origin of camera coordinate system and marker position on camera imaging plane 130 Virtual work area 131 Boundary of virtual work area Surface 132 Virtual work area 151 Internal virtual work area Boundary surface 210 Three-dimensional object displayed on monitor 211 Cursor at marker position 212 Virtual work area displayed on monitor 140 Contact point between estimated marker position and boundary surface of virtual work area 141 From posture operation reference point to contact point Heading unit vector 142 Unit vector heading to the marker position estimated from the posture operation reference point 143 Straight line passing through the posture operation reference point perpendicular to the plane including the two unit vectors 144 Angle formed by the two unit vectors 108 Three-dimensional object 109 3 Dimensional object 301 Fingertip or marker position detection unit 302 Area determination unit 303 Operation mode storage unit 304 Operation mode determination unit 305 Position information storage unit 306 Posture calculation unit

Claims (10)

A method for manipulating a posture of a three-dimensional object according to a three-dimensional position of a predetermined object in space,
Set a virtual work area that is fixed in the spatial coordinate system, including a fixed reference point,
When the operation mode is changed from the idle mode in which the three-dimensional position is outside the virtual work area to the posture operation mode in which the three-dimensional position is on and inside the boundary surface of the virtual work area, the three-dimensional A straight line that passes through the reference point and is perpendicular to a plane including the position of the intersection of the position and the boundary surface of the virtual work area, the posture of the three-dimensional object, the three-dimensional position, the intersection, and the reference point And remember
Only when the operation mode is the posture operation mode, the rotation transformation that superimposes the unit vector from the reference point to the intersection with the unit vector from the reference point to the three-dimensional position with the straight line as an axis is the 3 A three-dimensional object posture operation method that applies to the posture of a three-dimensional object and sets the resultant posture as the current three-dimensional object posture. A method for manipulating a posture of a three-dimensional object according to a three-dimensional position of a predetermined object in space,
Including a fixed reference point, a virtual work area fixed in a spatial coordinate system, and an internal virtual work area included in the virtual work area and fixed in the space;
The operation mode is changed from the idle mode in which the three-dimensional position is outside the virtual work area to the posture operation mode in which the three-dimensional position is inside the inner virtual work area or in contact with the boundary surface. When the three-dimensional position and the boundary surface of the internal virtual work area, the posture of the three-dimensional object, the three-dimensional position, the intersection and the reference point are perpendicular to the plane, and Storing a straight line passing through the reference point;
Only when the operation mode is the posture operation mode, the three-dimensional rotation transformation that superimposes the unit vector from the reference point toward the intersection with the unit vector from the reference point toward the three-dimensional position with the straight line as an axis. A three-dimensional object posture operation method that applies to the posture of an object and sets the resulting posture as the current three-dimensional object posture.   The posture operation method according to claim 2, wherein an area formed by the shape of the three-dimensional object is used as the internal virtual work area. A method of selecting one of two or more three-dimensional objects according to a three-dimensional position of a predetermined object in space and operating the posture of the three-dimensional object,
A virtual work area fixed in the space, corresponding to the number of the three-dimensional objects and corresponding to the three-dimensional objects on a one-to-one basis;
Providing a reference point fixed in the space, the number of which is the same as the number of the three-dimensional object and corresponding to the three-dimensional object on a one-to-one basis;
Set the same number of internal virtual work areas that are included in each virtual work area and fixed in the space as the number of the three-dimensional objects and do not intersect each other,
The operation mode changes from the idle mode in which the three-dimensional position is outside all the virtual work areas, and the three-dimensional position enters the inside of one internal virtual work area or touches the boundary surface to perform the posture operation. When the mode is changed, the position of the intersection between the three-dimensional position and the boundary surface of the internal virtual work area, the posture of the three-dimensional object corresponding to the internal virtual work area, the three-dimensional position and the intersection A line perpendicular to a plane including a reference point corresponding to the internal virtual work area and passing through the reference point;
Only when the operation mode is the posture operation mode, the rotational transformation that superimposes the unit vector from the reference point to the intersection with the unit vector from the reference point to the three-dimensional position, with the straight line as an axis, A three-dimensional object posture operation method that applies to the posture of a three-dimensional object and sets the resulting posture as the current three-dimensional object posture.   As the internal virtual work area, an area formed by the shape of a three-dimensional object corresponding to the internal virtual work area, an area formed by all three-dimensional objects other than the three-dimensional object, and a shape of the three-dimensional object The method according to claim 4, wherein a region from which the intersection with the region is removed is used.   The posture operation method according to claim 1, wherein a cubic area, a rectangular parallelepiped area, or a spherical area is used as the virtual work area.   Change the display color of the cursor indicating the three-dimensional position and / or the graphic indicating the virtual work area displayed on the display monitor to indicate that it is in the posture operation mode, sound the chime, fingertip The posture operating method according to any one of claims 1 to 6, wherein the operator is informed using any one of vibrating the marker attached to the head.   The posture operation method according to claim 7, wherein the position in space of the marker attached to the fingertip is used as the position in space.   The posture operation method according to claim 7, wherein a position of the fingertip in space is used as the position in space.   A three-dimensional object posture operation program for causing a computer to execute the three-dimensional object posture operation method according to any one of claims 1 to 9.

Images