The invention relates to a method and system for force feedback device for interacting with a virtualized environment in each case according to the preambles of the independent claims.
Force feedback devices of the type mentioned are known. Such can be used for example in the field of virtual technology, wherein, for example, in addition to a visual impression in the virtualized environment, a force feedback can be experienced. The robot device may follow movements of a device which is suitable for making a force feedback tangible (handling device). Movements made by the user may, for example, be transmitted to the robotic device by means of the handling device, which may follow them. The movements can be recorded and the force feedback can be felt to the user and used, for example, for a virtual mounting installation with force feedback. The US 2004 0164960 A1
relates to a system and method for providing a tactile virtual reality to a user.
From the US 2007/0276423 A1
is a teleoperator system with telepresence known, which includes right and left controllers for the control of right and left manipulators. An operator should be enabled by an improved teleoperator system an increased sense of presence in manipulation with the manipulators.
The publication DE 10 2004 001 167 B4
describes a method for positioning virtual objects with the method steps:
- Positioning a first virtual object by means of a man-machine interface against a second virtual object taking into account acting forces, wherein the first virtual object
- is positioned against an inclined and / or concave surface of the second virtual object, wherein the first is in Lagrange contact with the second virtual object, is fixed in this position and / or
- and wherein the first virtual object and its environment are inspected for a predetermined period of time with respect to the effects of the forces.
- Compressing a first and second handle member with a hand of an operator;
- Moving a first and second end effector element in response to the compression of the handle elements according to a control relation; and changing the control relation when the handle members are in the vicinity of a closed configuration; wherein the control relation imposes a follower force on the end effector elements with an actuator such that the follower force varies in response to a misalignment between a handle separation and an end effector separation.
The publication US 2003 0132968 A1
describes a system and method for virtual interactive design and evaluation and manipulation of a vehicle mechanism using a haptic user interface. The system includes a computer system and a haptic interface that operatively communicates with the computer system. The haptic interface includes a haptic end effector device for communicating information between a user and a digital model. The user creates, modifies and dynamically evaluates dynamic properties of the vehicle mechanism interactively and kinematically using the haptic end effector.
The object of the invention is to provide an improved force feedback for interacting with a virtualized environment and to allow interacting objects with limp properties with the virtualized environment.
The object is achieved with the features of the independent claims.
Under a virtualized environment, a virtual reality and / or a remote reality that has been virtualized for transmission to another location is also understood to be a combination of a virtual reality and a remote reality. A handling device is understood to be any device which is suitable for making a force feedback tangible. In this case, the handling device may comprise one or more arbitrary elements which can be manipulated by means of human and / or animal forces, for example one or more handles, one or more pedals and / or one or more tangible cables and / or or similar. Force feedback is a feedback of a force, a moment, an acceleration and / or a vibration understood, in particular those by means of the handling device for haptic representation (capture or feel) of the same against a Users of the force feedback device are transferable. Since the article having a deformable shape is coupled to at least two robot arms and the robot arms affect the instantaneous shape of the deformable article, it can be controlled and moved and moved relatively stably.
Advantageously, according to the invention limp objects can be held in two or more positions and then performed better and more accurate in space than was previously possible. Movements that a user of the force feedback device performs for interacting with the virtualized environment can therefore be recorded at several points by means of the robotic device or the robot arms assigned to the handling device.
Advantageously, it is also possible to transmit force feedback to the handling device at more than one position, so that, for example, a user can haptically detect or feel a force feedback on both hands. Advantageously, this allows changes in an outer shape of a limp object to be recorded and correspondingly interacted with the virtualized environment. Advantageously, by means of the handling device and the robot arms associated therewith, a controller can be realized, in particular a hand controller that very realistically powers, moments, accelerations and / or oscillations in a selectable number of degrees of freedom, in particular 6 degrees of freedom, from the virtual or distant reality can represent.
In one embodiment of the invention it is provided that the handling device has a first handle associated with the first robot arm and a second handle associated with the second robot arm. Advantageously, the handles may be grasped by the user of the force feedback device to input motions or to interact with the virtualized environment. Furthermore, a robot arm itself can be part of the handling device.
In a further embodiment, it is provided that the handling device comprises the object. Advantageously, the article itself or the model of the article may be physically or reallocated to the handling device. In the latter case, the entire object can serve as a handling device, that is to say that it is advantageously handled in its actual physical spatial extent by the user of the force feedback device. The force feedback can be advantageously transferred in this case by means of the robot arms on the handling device and thus on the object itself, so that advantageously any behavior of the object can be simulated. This may, for example, be a palpable collision of the object with the virtualized environment. Furthermore, it is conceivable to simulate a weight of the object, for example if it is only a significantly lighter or heavier model of the object.
It is provided that the handling device has a first limp element associated with the first robotic arm and a second limp element associated with the second robotic arm. Advantageously, by means of the handling device, the touch of a limp element, such as a cable, a rope and / or a piece of fabric can be simulated. Accordingly, it is possible to advantageously simulate forces of limp objects occurring when interacting with the virtualized environment by means of the robot device.
In a further embodiment, it is provided that the force feedback is determined by means of a computer model of the object. Advantageously, a computer model of the object to be interacting can be deposited, which simulates changes in shape occurring during interaction with the virtualized environment and / or forces introduced into the object. Furthermore, it is conceivable to make visible and / or perceptible by means of the computer model a change in shape of the object occurring during the interaction with the virtualized environment for the user of the force feedback device.
In a further embodiment it is provided that an optical reproduction takes place for representing the virtualized environment or the object in the virtualized environment. In addition to the haptic feedback when interacting with the virtualized environment, it is advantageously possible to also return a visual impression of the interaction to the user of the force feedback device by means of the optical reproduction.
In a further embodiment it is provided that the force feedback is determined by monitoring and evaluating virtual collisions occurring when the object interacts with the virtualized environment. Advantageously, outer shapes and / or properties of these outer forms of the virtualized environment and the object can be deposited and monitored for collisions. It is possible to assign a virtual force to a potentially occurring virtual collision, which can be reported haptically back to the user by means of the force feedback. Algorithms for a haptic rendering or force calculation algorithms, for example the storage of the virtualized environment as a static scene location discrete as a voxel map and the object or the manipulated object as a point cloud, a collision detection by simple index accesses in the voxel map are feasible, so that will not be discussed further here.
In a further embodiment it is provided that the virtualized environment has a digital prototype and the object has a component of the prototype. Advantageously, such a virtual mounting installation with force feedback can be made possible, for example, to test or determine a virtual mounting path of the component in the prototype.
In a further embodiment it is provided that the at least one robotic device or the robotic device and the handling device are gravity-compensated. Advantageously, it is possible to counter the user with no gravitational forces. Alternatively, however, it is also conceivable, in particular for the case that the handling device consists of the object, to provide only the robot device itself with the gravity compensation, so that the user can also feel the actual weight of the object. Overall, a robot device for receiving the movements of the user of the force feedback device may be switched without power, wherein only in the case of a successful force feedback counter forces are transmitted to the handling device. Advantageously, therefore, the actual manipulation or interaction of the object with the virtualized environment so long powerless, as long as no virtual collision occurs.
Further advantages, features and details emerge from the following description in which an exemplary embodiment is described with reference to the drawing. The same, similar and / or functionally identical parts are provided with the same reference numerals. Show it:
- 1 a system for force feedback with two robot arms and a handling device comprising an article with elastic properties;
- 2 this in 1 shown system, wherein the handling device comprises an article with limp properties;
- 3 that in the 1 - 2 shown system, wherein the handling device comprises two cable ends;
- 4 two different detail views of a robot arm in the 1 - 3 shown force feedback device, wherein a simulation of different directions of force is shown;
- 5 that in the 1 - 3 shown system, wherein a computer model of an object is provided;
- 6 that in the 1 - 5 shown system, wherein the computer model comprises an object which has partially pliable and elastic properties and is claimed to torsion.
- 7 a system corresponding manipulation device with two robot arms, the physically physically existing object in a remote reality analogous to according to 6 manipulated object manipulated, with a video surveillance is provided.
1 shows a system for force feedback 1 or a force feedback device 1 with a robot device 3 with a first robot arm 5 and a second robot arm 7 , The robot arms 5 . 7 For example, to manipulate in a three-dimensional space with a total of 6 Degrees of freedom be designed. However, it is also possible to use the robot arms 5 . 7 the robot device 3 designed as so-called redundant robots or kinematic redundant manipulators. These can be for example 7 Degrees of freedom, in particular 7 Have axes, which, for example, human-like, so the kinematics of a human arm modeled, can be interpretable. It can also be provided several robot devices.
The robot arms 5 . 7 the robot device 3 serve for the movable storage of the handling device 9 , for recording movements of the handling device 9 and for introducing a force feedback into the handling device 9 , These are the first robot arm 5 the handling device 9 at a first allocation point 11 and the second robot arm 7 at a second allocation point 13 the handling device 9 assigned.
As shown in 1 Couples an elongated deformable object 15 with the robot arms 5 . 7 , The object 15 preferably has elastic properties, which by means of a dashed line 17 for symbolizing an outer shape of the object 15 in a bent state in 1 is shown. For manipulating the object 15 with a virtual environment, not shown, the subject 15 or the handling device 9 anywhere, for example with two hands are touched, with the robot arms 5 . 7 the robot device 3 can be powerless controlled and / or regulated, allowing a movement of the object 15 or the handling device 9 a corresponding retraction of the robot arms 5 . 7 the robot device 3 effect. Corresponding solutions for the required inverse kinematics of the robot arms 5 . 7 , In particular, a redundant inverse kinematics are known, so that will not be discussed further here at this point. In particular, the robot arms 5 . 7 the robot device 3 have at their joints position and / or torque sensors, so that in a corresponding position of the robot arms 5 . 7 a torque corresponding to a powerless state is always so einregelbar that the entire robot device 3 together with the handling device 9 or the object 15 stands still. A corresponding manipulation of the object 15 or the handling device 9 by a user of the force feedback device 1 can thus cause a powerless retreat, which advantageously means of the force feedback device 1 the positions of the two allocation points 11 . 13 are recordable or detectable. Advantageous may be the location of the allocation points 11 . 13 on the location of the object 15 be inferred. Advantageously, even with a fixed assignment of the allocation points 11 . 13 to the subject 15 a possible deflection, as by means of the line 17 implied of the object 15 be recorded. At the elongated object 15 it may, for example, be a windscreen wiper and / or a windscreen wiper linkage.
Advantageously, a required for mounting the windshield wiper linkage bending same by means of the two robot arms 5 . 7 the robot device 3 also be recorded. This can be used to advantage to the elongate object 15 to interact with a virtualized environment, such as a prototype of a non-illustrated vehicle. Advantageously, the interaction of the actual exterior shape of the object 15 as they do by means of the line 17 is indicated to be compared with the virtualized environment, for example, to detect possible collisions with the virtualized environment and from these a force feedback by means of the robot arms 5 . 7 the robot device 3 on the handling device 9 or the object 15 the handling device 9 report back. Advantageously, the object 15 touching and possibly deforming users of the force feedback device 1 a realistic haptic picture of his interaction of the object 15 obtained with the virtualized environment.
In addition, it is possible for the user of the force feedback device 1 by means of a in the 1 Not shown displays also give a visual impression of his interactions with the virtualized environment.
2 also shows the in 1 shown force feedback device 1 , In difference, an object 15 is used, which is essentially claimed only on train, such as a rope 19 , The rope 19 which also acts as a handling device 9 can serve, is for clarity in 2 shown twice and with different deflection, resulting in a flatter and a lower hanging chain line. The the by means of the robot arms 5 . 7 in the rope 19 introduced forces are in 2 represented by two different arrows, each with an arrow 21 for the different representations of the rope 19 the corresponding traction symbolizes.
To interact with the rope 19 with the virtual environment, not shown, the rope can 19 at any point, preferably relatively close to the assignment points 11 . 13 be touched by a user, thereby being one on the robot arms 5 . 7 and by means of the arrows 21 indicated force changes, so that in this case, the robot arms 5 . 7 can follow the movements of the user of the force feedback device. In addition, it is possible to use a computer model of the rope 19 or the resulting chain line 23 with a normal sag of the rope 19 deposit, wherein advantageously the force feedback and recording the movements of the handling device 9 can be refined by means of this computer model. Advantageously, for example, the user of the force feedback device, the rope can be brought together or pulled apart, resulting in a particularly deep or shallow slack. In this case, the force feedback device can be determined from the resulting distance of the assignment points 11 . 13 or their spatial position relative to each other and the computer model of the rope 19 , So its weight and its length, the force resulting for a powerless state, as by means of the arrows 21 indicated, calculated, so that advantageous when not touching the handling device 9 the robot device 1 despite different real tensile forces of the rope 19 stationary.
3 shows the in the 1 and 2 shown force feedback device 1 in contrast, a the first robot arm 5 at the first allocation point 11 associated first limp element 25 and a second robot arm 7 at the second allocation point 13 associated second limp element 27 is used. Moreover, between the assigning points 11 . 13 dotted represents a virtual catenary 29 located. Advantageously, the user can use the limp element 25 and 27 take, by means of the dotted chain line 29 symbolizing a deposited computer model of the in 2 illustrated rope 19 can be understood, advantageously by means of the robotic device 3 such a force feedback on the handling device 9 can be transmitted to the user of the force feedback device 1 a haptic representation arises as if it were a rope with the slack corresponding to the chain line 29 hold in hands.
4 shows two different detail images of the first robot arm 5 the robot device 3 , for example as in 3 shown. Also shown is a hand 31 that the handling device 9 grasps. By means of the force feedback, the user can have different sags of the simulated object 15 or the rope 19 be conveyed, what in the 4 by means of differently inclined catenary lines 29 is shown.
5 shows the in the 3 and 4 illustrated robot device 3 along with a deformed catenary 29 of the rope which can be represented by means of the computer model 19 or object 15 , The only virtual existing rope, by means of the chain line 29 in 5 implied is using a virtual environment 33 , for example, a cuboid shown dotted 35 can interact with. This puts the catenary 29 virtually across the cuboid 35 , Advantageously, the resulting tensile forces by means of the robot arms 5 . 7 the robot device 3 on the handling device 9 and on these hands 31 the user of the force feedback device 1 be transmitted. It could also be advantageous to hook or wrap around the virtual rope around the virtual cuboid 35 be reported back by means of the force feedback. This could be the direction and strength of the virtual tractive forces on the handling device 9 be transmitted.
6 shows the in the 1 - 5 shown force feedback device 1 where the computer model is a virtual item 15 represents, which has pliable properties, but also in addition to torsion claims. This could, for example, be a cable. By means of a curved arrow 37 is in 6 indicated to the user when moving the handling device 9 can be reported back on this torque or a reaction, so the force feedback. Advantageously, the means of the curved arrow 37 indicated reaction of the virtual item 15 be determined by means of the computer model. In this case, for example, an entanglement 39 of the virtual object 15 calculated and taken into account. Advantageously, by means of the force feedback device 1 Also, a very difficult to handle item, such as a cable, will interact with the virtualized environment.
Advantageously, it is possible as in 7 represented by the handling device 9 recorded movements of the user of the force feedback device 1 in a physically present, for example, remote (from the user) reality 41 understand. This can be the distant reality 41 a manipulator 43 have, for example, analogous to the robotic device 3 , in particular identical to the robot device 3 , is constructed. Regarding the 6 and 7 it can be seen that by means of the manipulator 43 in the distant reality 41 physically actually existing object 15 a torque can be introduced, the direction of the in 6 shown force feedback is opposite, which by means of a curved arrow 45 in 7 is indicated.
It is conceivable by means of a camera 47 the actual reactions of the physical object 15 the distant reality 41 record them either in combination with the computer model or instead of the computer model to the virtual reaction of the object 15 as he is for example in 6 by means of a dotted illustrated deformation line 49 is shown, to charge.
The force feedback system 1 can be used advantageously in the field of virtual technology. For this purpose, a display not shown in more detail can be provided for imparting a visual impression. The visual impression may be, for example, from computer generated data, in particular the computer model of the object 15 and a corresponding model of the virtualized reality, wherein advantageously the user of the force feedback device 1 additionally can experience a force feedback or the force feedback. This can advantageously be done, for example, to install a component in a digital prototype. For this purpose, the object 15 at the allocation points 11 . 13 the robot arms 5 . 7 the robot device 3 be associated with the robot arms 5 . 7 the robot device 3 without own force movements of the user of the force feedback device 1 can follow. Advantageously, by means of the robotic device 1 a position of the object 15 captured or recorded and in real time with virtual reality data 33 , in particular the digital prototype. If the item 15 to parts of virtual reality 33 would be advantageous to the robotic device 3 applying a collision corresponding counterforce and thus further movements of the object 15 prevent in the appropriate direction, so that advantageously creates a realistic impression.
Advantageously, by means of the robotic device 3 also not intrinsically stable components of the robotic device 3 be assigned, for example, long components such as windscreen wiper linkage or pliable components, especially cables or hoses.
For this purpose, the robot device 3 a variety of robot arms 5 . 7 respectively.
It is conceivable that the limp elements 25 and 27 the handling device 9 , as in 3 represented, for example, have small pieces of cable, in particular to be mounted with plugs and / or sockets 51 are provided. The user can then, for example, for assembly testing of the plugs and / or sockets 51 these interact with virtualized reality. Furthermore, it is conceivable the plugs and / or sockets 51 additionally or exclusively associate robot arms for force feedback and / or recording of the movements. The force feedback can do this directly in the plugs and / or sockets 51 be initiated, for example, for a mounting test or a virtual contact.
Advantageously, by means of the robot arms 5 . 7 Such limp and / or long parts at two or more allocation points 11 . 13 held and thereby advantageously also better and more accurate in the room.
The display, not shown, may have a so-called multi-side projection system. Advantageously, the force feedback device 1 be used for assembly testing and / or testing of maintenance situations. Further, it is possible to use the force feedback device 1 to use for training and / or monitoring of motor skills, for example, for training purposes, in particular rehabilitation. In addition, it is conceivable, the force feedback device 1 for manipulation tasks in the distant realm 41 to use, in particular, if these dangers.
LIST OF REFERENCE NUMBERS
- Device or system for force feedback
- robotic device
- first robot arm
- second robot arm
- handling device
- first assignment point
- second allocation point
- first limp element
- second limp element
- virtual environment
- distant reality
- deformation line
- Male / female