JP2007026117A - Automatic acquiring method for new knowledge regarding body based upon reality-based simulation, and reality-based virtual space simulator - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To estimate a dynamic function of a body based upon actual-world data. <P>SOLUTION: Disclosed is a method of automatically acquiring the new knowledge regarding a body through reality-based simulation by a virtual space simulator operating by being supplied with body data, operation object data, and subject data as actual-word data depending upon the actual world, and the virtual space simulator simulates interactions among a subject, the body, and an object of operation based upon the given subject data, given operation object data, and given subject data to acquire the new knowledge regarding the body based upon a simulation result of the interaction among the subject, body, and object of operation. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a method for automatically acquiring new knowledge about an object based on reality-based simulation and a reality-based virtual space simulator.

  The novelty of virtual reality (VR) is to provide an “experience” based on the real world, and it is an essential issue to realize a simulation faithful to real world phenomena and behavior of real objects. Through the “experience” faithful to the real world in the VR environment, it is possible to guide new “discovery” and “understanding” to the real world. Recently, tactile / force feedback devices have also been put to practical use, and in the fields of VR, CG (computer graphics), robotics, and psychology, research is actively progressing toward the realization of interfaces based on vision and touch. [Non-Patent Documents 1 to 3].

Research on tactile sensation in the field of psychology has been focused on analysis of hand function and grip. As a representative study, typical hand movement patterns EPs (Exploration Procedures) for exploring the prominent features of an object by Lederman and Klatzky, that is, the shape, stiffness, flexibility, weight, temperature, function of the object, etc. ). A haptic vision concept of “touch and see” is proposed for the purpose of automatically acquiring tactile information such as the stiffness, flexibility, weight, and mutual restraint of objects obtained by active search of these hands. [Non-Patent Documents 4 to 6].
Furthermore, based on haptic vision, research to automatically acquire a three-dimensional interactive model of a joint object having a scissor-like function from observation of real-world interactions is being carried out [Non-Patent Document 7].

  Given such an interactive model, real world interaction is possible in virtual space. Many artificially created objects are tools to achieve some purpose, that is, to achieve a function. As shown in FIG. 1, the function of the object is achieved by the interaction between the three elements of the object, the action target, and the subject. A function is defined as usefulness for a purpose of use, which means that the subject manipulates an object and changes / transitions the target of action to a different state. If any of the interactions between the three elements is missing, the function cannot be achieved.

  Research on object function estimation and recognition has been advanced in the fields of artificial intelligence and computer vision. Early research on function estimation focused only on objects (such as chairs), and estimated static functions of objects from still images [Non-Patent Document 8]. After that, the dynamic function (“cut”, “push”, “slice”, etc.) of the object is estimated by observing and analyzing the dynamic scene (subject-object interaction) where the subject uses an object such as a knife. [9] Non-patent document 9].

  However, it is difficult to observe the change of the action target from the image, and the “object-action target” interaction is not considered. A study focusing on the target of action is found in Kitahashi et al. [Non-Patent Document 10], which satisfies the mechanical characteristics of the contact state (object-action target interaction) between the object (wrench, bottle opener) and the target (nut, plug). The function was estimated by judging whether or not. However, the subject who is the operator was not considered. Thus, in the conventional method, since it is difficult to simultaneously observe the interaction between the three elements of the object, the subject, and the action target, research has been stagnant since the mid-1990s. Recently, a technique has been proposed in which both the subject and the action target are recognized in a complementary manner [Non-Patent Document 11].

H.T.Tanaka, K. Kushihama: Haptic Vision.Proc.IEEE 16th Int. Conf. On Pattern Recognition, Vol.2, Session1.7 Robotics, Vol.II, pp.852855, Aug. 2002. AM Okamura, MA Costa, ML Turner, C. Richard, and M.R.Cutkosky, “Haptic Surface Exploration,” Experimental Robotics VI, Lecture Notes in Control and Information Sciences, Vol. 250, Springer-Verlag, pp. 423-432 , 2000. D.K.Pai, K. van den Doel, D.L.James, J. Lang, J. E. Lloyd, J.L.Richmond and S.H.Yau: Scanning Physical Interaction behavior of 3D objects.in Computer Graphics (ACM SIGGRAH 2001 conference Proceedings) (2001). Shiro Tanaka, Kengo Nishimura, Hiromi T. Tanaka: "Active 3D shape estimation using shape symmetry in stable posture-Approach to estimate shape from function-", IPSJ Journal, Vol.44 No. SIG9 (CVIM7), pp.3845, 2003. Shiro Tanaka, Takeshi Tanigawa, Yoshinobu Abe, Masatsugu Uejo, Hiromi T. Tanaka: “Active Mass Estimation with Haptic Vision”, Proc. IEEE 17th Int. Conf. On Pattern Recognition (ICPR2004), 2004. Naoki Ueda, Shi-iti Hirai, Hiromi T. Tanaka: “Extracting Rheological Properties of Deformable Objects with Haptic Vision”, Proc. IEEE Int. Conf. On Robotics and Automation (ICRA2004), 2004. Masatsugu Uejyo, Hiromi T. Tanaka: “Actiev Modeling of anarticulated Object with Haptic Vision, Proc. IEEE 17th Int. Conf. On Pattern Recognition (ICPR2004), 2004. Tsutomu Yamamoto, Hirokazu Kato, Kosuke Sato, Seiji Iguchi: "Three-dimensional object recognition using functional models", IEICE Transactions, Vol.74-D II, No.5, pp.601609,1995. Z. Duric, J.A. Fayman and E. Rivin: “Function From Motion”, IEEE PAMI, Vol.18, pp.576578, 1996. H. Hattori, K. Kise, T. Kitahashi and K. Fukunaga: Recognitionof Functions of Objects Based on Models of Dynamic Functions.IPSJ, Vol.36 [10], pp.22772285, 1995. Shigeki Yoshida, Tadahiro Kitahashi: "Functional recognition of objects through human movement", IEICE, IEICE Technical Report, PRMU2002-213, pp.1318, 2003.

The present invention focuses on the fact that a virtual space simulator functions as an inference engine, and aims to automatically acquire new discoveries and knowledge about the real world (real objects) through simulations that rely on the real world. .
A more specific object of the present invention is to estimate a mechanical function as new knowledge about an object.

Here, in many cases, the function of an object cannot be determined apart from the characteristics of the subject (human) who operates the object. For example, if an object (tool) has pliers and the object acts on a wire, if the subject using the pliers is an adult, the wire can be cut with the pliers. If so, you can only grab the wire with pliers. In this way, even if the same object (tool) is used, the function of the object differs depending on the subject who uses it.
Moreover, even if the subject is the same, the function of the object varies depending on how the subject has the object (tool). In other words, the force that the subject can give to the object differs between when holding and holding the object, and as a result, the functions exhibited by the object may be different.
In view of the above, the present invention discovers and acquires new knowledge such as the mechanical function of an object through simulation based on the real world, in particular, simulation by applying an arbitrary contact force to the object by the subject. The purpose is to do.

The present invention relating to a method for automatically acquiring new knowledge about an object based on reality-based simulation,
1) As a real world data that depends on the real world, a new knowledge about an object is automatically acquired by a reality-based simulation by a virtual space simulator that is operated with the following data 1a), 1b), and 1c). Thus, the present invention is a method for automatically acquiring new knowledge about an object based on a reality-based simulation characterized by including the following steps 2) and 3).
1a) Object data used to reproduce in the virtual space an object that is used by the subject and that can act on the action target;
1b) Action target data used to reproduce an action target on which an object acts in a virtual space by a computer;
1c) Subject data relating to the characteristics of the subject using the object;
2) A simulation step of simulating a subject-object-action target interaction with a virtual space simulator based on the given subject data, the given subject data, and the given subject data.
3) A knowledge acquisition step for acquiring new knowledge about the object based on the simulation result of the interaction between the subject, the object, and the action target.

The present invention relating to a reality-based virtual space simulator is provided with the following 1a), 1b), and 1c) data as real world data based on the real world, and new knowledge about the object is automatically obtained by the reality-based simulation. A reality-based virtual space simulator characterized by including the following means 2) and 3) in order to acquire:
1a) Object data used to reproduce in the virtual space an object that is used by the subject and that can act on the action target;
1b) Action target data used to reproduce an action target on which an object acts in a virtual space by a computer;
1c) Subject data relating to the characteristics of the subject using the object;
2) Simulating means for simulating the subject-object-action target interaction by a virtual space simulator based on the given subject data, the given subject data, and the given subject data.
3) Knowledge acquisition means for acquiring new knowledge about the object based on the simulation result of the interaction between the subject, the object and the action target.

The present invention relating to the dynamic function estimation method is as follows. 1) As the real world data that depends on the real world, the following 1a), 1b), and 1c) data are given, and the dynamic function of the object is controlled by a virtual space simulator. A method for estimating the mechanical function of an object, comprising the following steps 2), 3) and 4).
1a) An object including three-dimensional shape data and mechanical data of an object used by the subject and capable of acting on the action target in order to reproduce in the virtual space by the computer the action acting on the action target Object data;
1b) Action target data including physical property data of the action target in order to reproduce the action target on which the object acts in a virtual space by a computer;
1c) Subject data relating to the characteristics of the subject using the object;
2) A gripping state estimation step of estimating the gripping state of the object represented by the given object data by the subject represented by the given subject data using a virtual space simulator.
3) An action force calculation step of calculating a force applied by the object to the action target using the virtual space simulator in the grasp state estimated in the grasp state estimation step.
4) When the acting force estimated in the acting force estimation step acts on the acting target represented by the given acting target data, the mechanical function of the object is estimated based on a change that can occur in the acting target. Function estimation step.

The subject data is preferably human characteristic data based on knowledge of anatomy, psychology, and / or ergonomics.
The subject data preferably includes gender data and / or age data of the subject.
The gripping state estimated in the gripping state estimation step preferably includes a gripping pattern that is a posture when the subject grips the object and a gripping force that is a force when the subject grips the object.

The present invention relating to the virtual space simulator is provided with the following 2) in order to estimate the mechanical function of an object given the following data 1a), 1b) and 1c) as real world data based on the real world: 3) A virtual space simulator including the means of 4).
1a) An object including three-dimensional shape data and mechanical data of an object used by the subject and capable of acting on the action target in order to reproduce in the virtual space by the computer the action acting on the action target Object data;
1b) Action target data including physical property data of the action target in order to reproduce the action target on which the object acts in a virtual space by a computer;
1c) Subject data relating to the characteristics of the subject using the object;
2) Grasping state estimation means for estimating the gripping state of the object represented by the given object data by the subject represented by the given subject data.
3) Action force calculation means for calculating the force exerted by the object on the action target in the grip state estimated by the grip state estimation means.
4) When the acting force estimated by the acting force estimation means acts on the acting target represented by the given acting target data, the mechanical function of the object is estimated based on a change that can occur in the acting target. Function estimation means.

The subject data is preferably human characteristic data based on knowledge of anatomy, psychology, and / or ergonomics.
The subject data preferably includes gender data and / or age data of the subject.
The gripping state estimated by the gripping state estimation means preferably includes a gripping pattern that is a posture when the subject grips the object and a gripping force that is a force when the subject grips the object.

  According to the present invention, new knowledge such as the function of an object can be acquired by simulation based on real world data.

[1. Introduction]
Here, based on the analysis of simulation results based on the real world, which is called the Analysis by Reality-based Simulation approach, as an example of new knowledge about the object, the joint functions (objects such as grasping, crushing, and breaking) We propose a method for estimating the function of Using an interactive model of a joint object with scissors-like functions obtained from observation of real-world “object-action” interactions in virtual space, various physical properties (shape, size, stiffness, surface By analyzing the results of a simulation in which an arbitrary contact force is applied to a subject with various human characteristics (gender, age, hand size, grip pattern) on an arbitrary target with smoothness (1) Estimate all possible mechanical functions of the object, and (2) estimate the most likely mechanical function called the main function and the gripping posture of the hand holding the object. Experimental results of estimating the mechanical function of an object with one joint such as scissors, pliers, tweezers, etc. are shown, and the effectiveness of the proposed method is verified by comparison with the results of subjective evaluation by 30 subjects. .

[2. Analysis By Reality-Based Simulation Approach]
FIG. 2 shows the concept of the Analysis By Reality-Based Simulation approach. The input to the virtual space simulator is a 3D model of the real object that is obtained from observation of real-world interaction and enables real-world interaction, physical properties of the real object (action target), anatomy, It is a human characteristic based on knowledge of psychology and ergonomics. These inputs are all information that relies on the real world, and simulations based on them represent phenomena in the real world. Therefore, the Analysis By Reality-Based Simulation approach focuses on the fact that the virtual space simulator functions as an inference engine, and automatically discovers new knowledge and knowledge about the real world (real objects) through simulations that rely on the real world. Make it possible to earn.

[3. Embodiment: Mechanical Function Estimation of Joint Object Based on Analysis By Reality-Based Simulation Approach]
French social thinker J. et al. Proudhon (18909-1865) states: `` The most basic and most common tools that humans use to do their work, that is, what they come from is leverage. ''
Thus, the lever is the most basic element for an object to recognize its function, and most studies on the function of an object assume that the object has a lever function. Therefore, it is assumed here that the target joint object having dynamic functions such as “grab”, “cut”, and “break” has leverage as an element function.

The reality-based virtual space simulator according to the present embodiment is for estimating a dynamic function of an object having a lever function (joint object), and installs a computer program that causes a computer to execute processing described later. It is constituted by.
The virtual space simulator has, as its function, a) a gripping pattern estimation processing unit that performs the processing of Step 1 below, b) a gripping force estimation processing unit that performs the processing of Step 2 below, and c) an action force calculation processing unit that performs the processing of Step 3 below. D) an action stress calculation processing unit that performs the processing of Step 4 below, e) a mechanical function estimation processing unit (knowledge acquisition processing unit) that performs the processing of Step 5 below, and f) a main function estimation processing unit that performs the processing of Step 6 below ( Knowledge acquisition processing unit).
Note that the gripping pattern estimation processing unit and the gripping force estimation processing unit are collectively referred to as a gripping state estimation processing unit, and the gripping state estimation processing unit indicates the gripping state (for example, the gripping pattern and the gripping force in the gripping pattern). To be estimated.
The gripping state estimation processing unit, the acting force calculation processing unit, and the acting stress calculation processing unit are collectively referred to as a simulation unit that simulates the subject-object-action target interaction in a virtual space.

The input to the virtual space simulator is three elements for achieving the function: (1) interactive model (object data) of object (joint object), (2) action target data, and (3) subject data. Two pieces of data are real world data based on the real world (see FIG. 2).
These data are given to the storage means of the virtual space simulator, and each processing unit of the virtual space simulator automatically performs new discovery and knowledge of the object such as function estimation of the real object based on the real world data. Process for acquisition.

(1) An interactive model of a joint object is data relating to a joint object that is a real object, and is obtained from observation of a “joint object-action target” interaction in the real world. This interactive model has three-dimensional shape data Shape of the object and mechanical data Force, Level, and Spring. Details of this interactive model will be described later.
(2) The action target data is configured to have physical characteristics (shape, size, rigidity, surface smoothness, etc.) of the action target, and when the object acts on the action target by the dynamic function, This is data that is a basis for estimating by calculation how the object changes (cuts, breaks, etc.).
(3) The subject data is composed of human characteristic information based on anatomical knowledge, human characteristic information based on psychological knowledge, and human characteristic information based on ergonomic knowledge. Can be constituted by the gender of the subject (human), age, dominant hand, hand size, hand movement, grip pattern, grip force, and the like.

FIG. 3 shows an outline of the flow of processing by the virtual space simulator.
Step 1) Three-dimensional interactive model of joint object, subject's anatomical knowledge [Noriko Kamakura: "Shape of the hand, hand movement", dentistry publishing, 1989.] (Gripping state estimation processing; gripping pattern estimation processing).
Step 2) Ergonomic knowledge using the estimated gripping pattern and subject data (subject's gender, age, dominant hand) given as the first input as arguments [Norihiro Kasai, Ken Horii, Kentaro Kotani: “Target width in picking motion Relationship between Stable Gripping Force and 1999 ”: Proceedings of the 1999 Japan Ergonomics Society Kansai Branch Conference, pp.175-178, 1999. [Virgil Mathiowetz, MS, OTR, Nancy Kashman, OTR, Gloria Volland, OTR, Karen Weber, OTR, Mary Dowe, OTS, Sandra Rogers, OTS: Grip and Pinch Strength: Normative Data for Adults, American Occupational Therapy Foundation, 1984. presume. The gripping force is a force with which the subject grips the joint object.
Step 3) From the leverage information of the three-dimensional interactive model of the joint object and the range of the gripping force obtained in Step 2, the absolute force applied to the action target and the range of the acting force are estimated.
(Step 4) The range of the applied stress is calculated from the contact area of the 3D interactive model of the joint object (contact between the joint object and the action target) and the action force range obtained in Step 4.
Step 5) All possible functions are estimated by using the working stress range obtained in Step 4 and the physical properties (stress / strain) of the working object.
Step 6) From the estimated functions, the maximum likelihood function called the main function and the posture of the hand holding the object are estimated.
Through the above steps, it becomes possible to discover and automatically acquire new knowledge about the dynamic main function and gripping pattern of the joint object corresponding to an arbitrary target and an arbitrary subject, that is, the object.

[3.1 Three-dimensional interactive model of joint object]
Based on the haptic vision [Non-Patent Document 1], a method of generating a 3D interactive model [Non-Patent Document 7] automatically acquired from observation of real-world interactions and its contents will be described. Note that the interactive model generation method is not limited to the following.
First, (1) it is assumed that the “contact” by the robot that is the subject to the object is known, and the geometric and mechanical structure and functional posture of the joint object having the lever are extracted from the observation of the subject-object interaction.
Next, (2) assuming that the physical characteristics of the action target are known, the mechanical action characteristics of the joint object having a lever are extracted from the observation of the subject-object-action target interaction. Furthermore, using these methods, we will automatically construct a 3D interactive model that enables real-world interaction in virtual space.

As shown in FIG. 4, the interactive model of the joint object includes the three-dimensional shape (Shape) of the object, the functional posture (Functional Pose), the mechanical action characteristic (Force), the lever structure (Level), and the spring characteristic of the object (Spring). Is described. As shown in FIG. 5 and FIG. 6, 3D surface shape Os, 3D volume data Ov, and barycentric coordinates Og are shown for Shape, and 3D shape symmetry plane Osp of the joint object is shown for Functional Pose, A cross-sectional symmetry contour Ocross and a symmetry axis Oaxis of the joint object are described. In the force, the contact point Fcp (contacted by the robot), the change graph of the acting force / contact force of the robot hand without the action target shown in FIG. 7 (Contact Force without acte: dotted line in FIG. 7) Fcf, with the action target Graph (Contact Force with actee: thin solid line in FIG. 7) Fcf ′, robot hand action force Fy at the yield point, Lever, lever pattern Lp, lever from initial state An angle Lθmax which is a rotation limit state, a lever fulcrum Lf, a contact plane Lcp of an articulated object to which an external force is to be applied, a support plane Lbp which is a plane to be supported for gripping the articulated object, and a space where an operation target is to be placed The action space Lis, the contact point Lip between the joint object and the action target, the lever magnification Lm, the joint object And a contact area Lia operand, the Spring, has reactive force Sk of articulated object, the reaction force of the joint object of the initial value Sif as parameters. Describe.
Here, the lever pattern is classified into five types according to the positional relationship between the fulcrum, the force point, and the action point, the force amplification of the ability, the distance amplification, and the relationship.

  With the interactive model having the above parameters, it is possible to reproduce what shape the object has in the virtual space, and how much acting force is applied to the target when the subject applies a predetermined force to the object It can be determined by calculation whether it occurs.

[3.2 Step 1: Grasping pattern estimation (gripping state estimation process 1)]
[3.2.1 Grasp pattern]
Humans have different shapes of hands holding objects depending on the purpose of the work. For example, when writing something with a pen and when picking up a falling pen, use different hand shapes. In this way, Napier, which focuses on the relationship between human intentions and hand grip patterns, is roughly classified into two types: Precision used for performing work accurately and Power used for work requiring force. [Napier, JR: “The prehensile movements of the human hand.”, Journal of Bone and Joint Surgery, 38B, 902-913, 1956.].
Napier's achievement is that it is the work purpose, not the shape or size of an object, that determines the gripping type. Since then, various gripping classifications have been proposed based on two types of Napier. Among them, as shown in FIG. 88, Iberall pays attention to the fact that the posture of the hand includes at least two forces (Opposition) on the surface of the object, and falls into the following three basic categories. [CLMackenzie, T.IberallL: THE GRASPING HAND, NORTH-HOLLAND, 1994.] [Iberall, T., Bingham, G., & Arbib, MA: Opposition space as a structuring concept for the analysis of skilled hand movements. Generation and modulation of action patterns, pp.158173, 1986.].

(1) Pad Opposition: Used when the Opposition Vector at the time of gripping is gripping parallel to the y axis of the hand palm coordinate system shown in FIG.
(2) Palm Opposition: Used when the Opposition Vector at the time of grasping is grasping perpendicular to the Palm coordinate system xy plane of the hand and performing work requiring force.
(3) Side Opposition: Opposition Vector when gripping is grip parallel to the hand palm coordinate system x-axis, located between Pad Opposition and Palm Opposition and requires accuracy or force It is used when performing work.

[3.2.2 Grasping pattern estimation process (gripping state estimation process 2)]
The virtual space simulator grasps the object represented by the subject data given as input from the three-dimensional shape of the joint object represented by the interactive model given as input by the grip pattern estimation processing unit. A grip pattern estimation process is performed.
Specifically, the grip pattern estimation processing unit is based on the interactive model, as shown in FIG. 9, 1) First, the angle between the contact surface Lcp and the base surface through the contact plane (Contact Plane) Lcp of the joint object An Opposition Vector existing on the axis of symmetry (Base Plane) with the same angle is obtained.

2) Next, the gripping pattern estimation processing unit determines whether or not the obtained Opposition Vector can be gripped with three gripping patterns based on human characteristic data based on anatomical knowledge.
The gripping pattern estimation processing unit is represented by the surface shape Os of the interactive model using data for reproducing three gripping patterns of Pad, Palm, and Side by human hand in the virtual space among the human characteristic data. The joint object is actually gripped by each gripping pattern in the virtual space, and the gripping pattern is estimated depending on whether or not the joint object can be gripped by each gripping pattern.

Specifically, as shown in FIG. 10, the joint angle of the human hand is restricted by the following anatomical findings [Noriko Kamakura: “Shape of the hand, movement of the hand”, Medical and Dental Publishing , 1989.]. That is, the motion angle θdip at the joint (first joint) Dip of the index finger of the hand, the motion angle θpip at the second joint Pip, and the motion angle θmcp at the third joint Mcp are restricted by the following equations (1) and (2). .
0 ° ≦ θmcp ≦ 90 ° (1)
θdip = 2 / 3θpip (2)

Data for reproducing the three grip patterns of Pad, Palm, and Side in the virtual space in the virtual space is based on the model of the hand of the library (with 22 joints) attached to the VHT Cyber Glove. The angles of 22 joints in one grip pattern are obtained.
Based on this data, the gripping pattern estimation processing unit changes the angle of each joint according to the anatomical knowledge of the above formulas (1) and (2) in the virtual space, and when the object is gripped, , Palm, and Side, the grip pattern is determined based on whether or not the shapes of the three grip patterns can be formed.
Note that the relative position between the hand and the joint object is set to the initial position of a plane that is perpendicular or parallel to the palm (palm) plane of the hand and the target surface Osp of the joint object.

In particular,
(1) Determination of Pad Opposition: a) Whether the first joint Dip of the hand (thumb) can contact the support plane Lbp of the joint object, or b) Index of the hand (index finger) It is determined whether or not the first joint Dip (the foremost joint of the index finger) can contact the contact plane Lcp of the joint object. If both a) and b) are possible, they can be gripped.
(2) Palm Opposition: a) The palm of the hand can contact the contact plane Lcp of the joint object, b) The second joint Pip of the finger other than the thumb of the hand or the first joint Dip is the support plane of the joint object It is determined whether or not contact with Lbp is possible. If both a) and b) are possible, they can be gripped.
(3) Side Opposition: a) Whether the first joint Dip other than the thumb of the hand can contact the contact plane Lcp of the joint object, b) Whether the second joint Pip of the index finger of the hand and the support plane Lbp of the joint object can contact Determine whether or not. If both a) and b) are possible, they can be gripped.

  Based on the above constraints, the gripping pattern estimation processing unit estimates possible gripping patterns from a total of four directions: two directions in the vertical direction and two directions in the parallel direction of the symmetry plane Osp of the joint object.

[3.3 Gripping force estimation processing]
The gripping force estimation processing unit of the virtual space simulator has gripping force data for each gripping pattern. Based on this gripping force data, the gripping force estimation processing unit calculates the gripping force in the gripping pattern estimated by the gripping pattern estimation processing unit. Do.
Gripping force data were measured using gripping force measurement results for clinical hand function evaluation [Virgil Mathiowetz, MS, OTR, Nancy Kashman, OTR, Gloria Volland, OTR, Karen Weber, OTR, Mary Dowe, OTS, Sandra Rogers, OTS: Grip and Pinch Strength: Normative Data for Adults, American Occupational Therapy Foundation, 1984.]. In the study on the left, gripping force data of 310 healthy people was measured, and 20 to 24 years old, 25 to 29 years old, 30 to 34 years old, 35 to 39 years old, and 40 to 44 years old for three holding patterns. , 45-49 years old, 50-54 years old, 55-59 years old, 60-64 years old, 65-69 years old, 70-74 years old, males and females over 75 years old, a total of 144 types of average values and standard deviation values of left and right hands It has been measured.
FIG. 11 shows gripping force data in PalmOposition as an example. Here, the range of ± standard deviation value (± SD) from the average value of the gripping force was defined as the gripping force range. With this gripping force data, the gripping force estimation processing unit can estimate the gripping force (griping force range) when the gripping pattern and the attributes of the subject (gender, age) are given.
With these findings, it is possible not only to estimate functions that are easy to use for women and the elderly, but also to estimate objects (tools) suitable for the functions.

  11 represents standard deviation, a value obtained by subtracting the standard deviation from the average value is represented by SD−, and a value obtained by adding the standard deviation to the average value is represented by SD +. Min represents the lowest value, and Max represents the highest value.

[3.5 Step4 action force calculation processing]
The action force calculation processing unit of the virtual space simulator is based on the data related to the lever structure (Lever) of the interactive model of the joint object (the lever's force point, action point, fulcrum, etc.). From the force range), an action force (action force range) that is a force applied to the action target is obtained.
Specifically, the absolute force, acting force Fac (φ) applied to the acting object is calculated from the force point Fcp, the acting point Lip, and the fulcrum Lf of the mechanical structure model (interactive model) of the joint object. As shown on the left side of FIG. 12, if the gripping force is Fag, the reaction force of the joint object in a state where the action target is not sandwiched is Fs (φ), the fulcrum / action point distance is Dac, and the fulcrum / force point distance is Dag. The acting force Fac (φ) is
Fac (φ) = (Fag−Fs (φ)) × Dag / Dac (3)
It can be obtained more.
On the other hand, the acting force may not be calculated by the above equation. This is a case where the main force does not act directly on the target of action and is stored as spring energy in a joint object such as a clothes pin or a paper pin as shown on the right in FIG. In this case, the acting force Fac (φ) is
Fac (φ) = Fs (φ) × Ds / Dac (4)
Ask more. Here, the gripping force Fag is obtained from Step 3 and is assumed to be a constant value.

[3.5 Step4 action stress calculation processing]
Based on the contact area Lia data of the interactive model of the joint object, the action stress calculation processing unit of the virtual space simulator obtains the action stress (action stress range) applied to the action target based on the action force obtained in Step 3.
Here, the stress is a force [Mpa] per unit area acting on each surface when a certain point is considered as a microcube. FIG. 13 shows a general stress / strain curve. σe represents the yield point and σb represents the ultimate stress. The deformation until reaching the yield point is elastic deformation. When the load is further increased beyond the yield point, plastic deformation begins. The stress becomes maximum at the ultimate stress, and then naturally goes to the breaking point. This ultimate stress is called tensile strength and is often used as a measure of the strength of the material.
The stress that the joint object applies to the acting object, that is, the acting stress Fs is:
Fs = acting force / contact area = Fac / Lia
Can be obtained.

[3.6 Step 5 Mechanical Function Estimation Process]
The dynamic function estimation processing unit of the virtual space simulator uses the action stress (action stress range) obtained in Step 4 and the physical property data (see FIG. 13) included in the action target data given by the input. Performs processing to estimate the functional function.
Here, we defined the mechanical function of an object according to the deformation of the target whose physical properties are known as follows.
・ Elastic deformation (working stress is smaller than the yield point): Grabbing ・ Plastic deformation (working stress is above the yield point and below the ultimate stress)… Crushing ・ Deformation exceeding the ultimate stress (above the working stress and the ultimate stress)… breaking Here, elasticity A state refers to a state in which a force is applied to a target to be deformed and then restored to its original form when the force is released. On the other hand, the plastic state means a state in which the original shape is not restored even when the force is released, but remains distorted. For example, the function of `` cutting '' the action target in the case of scissors and the function of `` punching a hole '' of a punching punch are classified as `` breaking ''.

The physical property data included in the action target data is, for example, as shown in FIG. 13 and is data that associates stress on the action target with three mechanical functions of “grab”, “crush”, and “break”. .
When the physical characteristics of a certain target and the range of the working stress obtained in Step 4 are expressed as shown in FIG. 13, the obtained working stress range spans (1) the gripping function and (2) the crushing function. I understand. That is, two dynamic functions of “grabbing” and “smashing” are estimated for a certain target depending on the range of the applied stress obtained. In FIG. 12, since “grabbing” has a wider range than “crushing”, it can be seen that “grabbing” is more likely as a function of the object.
Note that the function estimation is performed for a combination of three types of grip patterns and three types of functions, that is, nine types for one input joint object model.

[3.7 Main function estimation processing unit]
The main function estimation processing unit of the virtual space simulator gives an evaluation value to the function by a predetermined method, and obtains the one having the highest evaluation value as the main function.
We numerically give three points: 1) ease of holding, 2) efficiency of operation, and 3) ease of achieving the function, giving priority as the evaluation value, giving priority to the highest evaluation value Is the main function. Assume that the evaluation value of the ease of holding the joint object is P1, the evaluation value of the ease of use is P2, and the ease of achieving the function is P3.

1) Ease of evaluation P1
Ease of holding when the joint object is gripped can be replaced with ease of gripping. In the field of ergonomics, research is being conducted to evaluate the ease of grasping an object [Norihiro Kasai, Ken Horii, Kentaro Kotani: "Relationship between target width and stable gripping force in picking motion": 1999 Japan Human Engineering Proceedings of the Kansai Branch Conference, pp.175-178, 1999.].
As shown in FIG. 14, NOD (Natural Opposition Distance) NOD in Pad Opposition is determined to be most easily gripped at 20 to 25 mm. NDO is defined as the length of the OppositionVector in a natural state without flexor and extensor activity. We fit the graph shown in FIG. 14 to a normal distribution, and gave a maximum value 1 of the evaluation value P1 at 20-25. The evaluation value P1 approaches 0.625, 0.08, and 0 every time σ 2 leaves. σ is the standard deviation value of NOD.

2) Action efficiency P2
The action efficiency represents how efficiently the force (gripping force) applied by the subject is applied to the action target. As shown in FIG. 15, when the gripping force is Fag and the reaction force to the main body is Fagr (φ), the action efficiency P2 is (reaction force Fagr (φ) to the main body) / (main body gripping force Fag). Is calculated by
Then, the reaction force Far (φ) to the main body is divided into a reaction force Fs (φ) (FIG. 15 left) to the main body by a spring or the like built in the joint object and a reaction force Fac (φ) (FIG. 15 only). 15 right). Assuming that the distance between the fulcrum and the force point (Fag) is Dag and the distance between the fulcrum and the action point (Fag) is Dac, the reaction force from the acting object to the subject at the force point is (Dac / Dag) × Fac (φ) and I can. Therefore, P2 is
P2 = reaction force on the subject / gripping force on the subject = Fagr (φ) / Fag
= (Fs (φ) + (Dac / Dag) × Fac (φ)) / Fag (6)
Can be obtained as
The evaluation value P2 has a maximum value of 1 and indicates that the force (gripping force) applied by the subject as the value approaches 1 is efficiently applied to the target.

3) Ease of function achievement P3
When humans use tools, tools that achieve a certain function within a wide range of power are easy to handle. This is because a tool that achieves a function in a narrow force range takes time to adjust or fails to function until the function is achieved. Therefore, we increased the priority of the range of stress applied to the target of action obtained in Step 6 that includes more ranges that achieve the function. For example, as shown in FIG. 16, if the range of the stress applied to the action target is S, the range of the function “grab” is s1, and the range of the function “crush” is s2, the ratio of s1 to S and s2 to S is In comparison, since the ratio of s1 to S is large, the priority of the “grab” function is high. In FIG. 16, the evaluation value P3 of the “grab” function is
P3 = s1 / S (7)
Ask for.
The main function estimation processing unit obtains the overall evaluation value P 1 using the evaluation values P1, P2, and P3 obtained from the above. We consider that the product of all the evaluation values P1, P2 and P3 is a comprehensive evaluation value. If 1 is given to all the evaluation values P1, P2, and P3, the total evaluation value is 1, and the performance is sufficient as a tool for achieving the function. Even if the value of P3 (ease of achieving the function) is 1, if P1 (ease of holding evaluation) is 0, it cannot be said that the tool achieves the function. This is also the case when P3 and P2 are reversed. Therefore, it can be said that the evaluation value is not a sum of P1, P2 and P3 but a product.
Therefore, the main function estimation processing unit calculates the comprehensive evaluation value P as
P = P1 × P2 × P3 (8)
Calculate as

[Example]
[4. Function estimation simulation experiment]
[4.1 Experimental environment]
This system used a Windows 2000 (trademark) machine (CPU: Intel Pentium (trademark) 3.06 GHz), and the collision and position of the simulation were detected using an attached library attached to CyberGlove of Virtual technologies.
[4.2 Experimental results]
Using a 3D interactive model of the joint object acquired by the haptic vision system, a reality-based simulation was performed to test the functions of the mechanical functions "grab", "crush", and "break". Three types of joint objects were tested: 1) hand substitute, 2) hand qualities and functions expanded, and 3) new functions added.
・ Hand substitute: Laundry scissors (small) (large), paper scissors (small) (large)
・ Expansion of hand qualities and functions: tweezers (small) (large), tongs (small) (large), pliers ・ Addition of new functions: hole punch, scissors

For the virtual space simulator, 1) the three-dimensional interactive model of each joint object as object data, 2) the strain and stress characteristics of the action target as the action target data, and 3) the gender, age, dominant hand of the subject as the subject data. Information was input and main functions were estimated.
There are three types of functions, “grab”, “crush”, and “break”, for the grip patterns of Pad Opposition, Palm Opposition, and Side Opposition, respectively, and therefore function estimation is performed for a total of nine types of 3 × 3. Some of the experimental results are shown in FIGS. Here, the sex is male, the age is 20 to 25 years, the dominant hand is the right hand, the yield stress of the paper to be acted is 5.5 to 6 [MPa], the cloth is 20 to 23 [MPa], the wire or iron plate The yield stress was 400 to 420 [MPa]. 17, 18 and 19, the lower left part represents an interactive model of a joint object, the upper part represents a simulation scene for main function estimation, and the lower right part represents comprehensive evaluation values for grip patterns and functions (total of nine types).

  As shown in FIG. 17, the simulation for the clothespin (small) and the action target (paper) is in the elastic state with respect to all the grip patterns. It was estimated that there was a “grab” function. Among them, “grabbing” of the Pad Opposition (upper part of FIG. 18) has the highest evaluation value, and is estimated to be the main function. The simulation results for the clothespin (small) and the action target (cloth) were the same as those of the action target (paper). The other evaluation values are 0 because the evaluation value P3 is 0.

  As shown in FIG. 18, for a joint object pliers and an action target (wire), the PadOpposition function is “grab”, the PalmOpposition is “grab”, “crush”, and “break” all functions, and SideOpoition is “ Presumed to have a “grab” function. Among them, “grabbing” of SideOpposition (upper part of FIG. 18) has the highest evaluation value and is estimated to be the main function. The other evaluation values are 0 because the evaluation value P3 is 0.

  As shown in FIG. 19, it was estimated that all the grip patterns had a function of “breaking” with respect to the joint object drilling punch and the action target (paper). Among them, “destruct” of SideOpposition (upper part of FIG. 19) has the highest evaluation value and is estimated to be the main function. In addition, it was estimated that, for an action target (iron plate), PadOpposition has a “grab” function, PalmOpposition has a “break” function, and SideOpposition has a “grab” function. Among them, “Grabbing” of SideOpposition (upper part of FIG. 19) had the highest evaluation value and was estimated to be the main function. The other evaluation values are 0 because the evaluation value P3 is 0.

[4.3 Verification experiment]
The estimated main function was compared with the subject's subjective evaluation value. The total number of subjects was 20 to 25 years old, 25 males and 5 females. The same joint object as in the simulation experiment, and 2) the subject of action having the same stress / strain characteristics are presented to the subject, who is A) the grip pattern, B) the ranking and priority of the estimated functions, 3 ) Select possible mechanical functions (grabbing, crushing, breaking) and the application achieved by that function.
FIG. 20 shows a comparison between simulation experiment results and verification results.
The result of estimation by Analysis by Reality-Based Simulation and the result of subjective evaluation agreed with 88.0% (26 experiments in total). However, as seen in the tweezers (small) -paper, pliers-wire, punching punch-iron plate of FIG. In the case of tweezers (small) -paper, the simulation result was estimated to be “grabbing” of PadOpposition as the main function, while in the subjective evaluation experiment, “grabbing” of SideOpposition was estimated as the main function. In the simulation, it is estimated that there is a “grab” function for all grip patterns. However, when calculating an evaluation value, priority is given to a function that operates efficiently with less force. Presumed to be the main function.

In the case of a pliers-wire, the simulation result is presumed that the main function is “grab” of SideOpposition. In the subjective evaluation experiment, it was estimated that “crush” of Palm Opposition was the main function.
From the simulation results, it is estimated that there is a “collapse” function of PalmOpposition. However, in the main function evaluation in the simulation, it is estimated that the function that effectively acts on the operation target with less force is prioritized, so that it is “grabbed” of SideOpposition.

In the case of punched punch-iron plate, the simulation result was estimated to be “gripping” of SideOpposition, while the subjective evaluation experiment was estimated to be “breaking” of PalmOpposition.
When the test subject was given a punch, the drilling punch assumed that the punch had a function of “breaking”, that is, the “breaking” function. It is thought that the function was determined to be “break” of PalmOpposition. In the simulation results, it is presumed that there is also a “breaking” function of PalmOpposition. However, in simulation, the “grabbing” function of SideOpposition, which is a function that acts on the target efficiently with less force, is given priority. It is considered that the simulation results and the results of the subjective evaluation experiment did not match.

[5. Summary]
In this embodiment, based on analysis of simulation results based on the real world, called Analysis by Reality-Based Simulation approach, (1) all possible mechanical functions of joint objects with scissors-like functions are estimated. Furthermore, (2) a method for estimating the maximum likelihood mechanical function that calls the main function and the posture of the hand that holds the object has been disclosed. Experimental results of estimating the mechanical functions of seven types of joint objects such as scissors, pliers, tweezers, etc. are shown, and the effectiveness of the proposed method is verified by comparison with subjective evaluation experiments by 30 subjects.

In addition, this invention is not limited to the said embodiment. For example, new knowledge acquired by the virtual space simulator is not limited to the function of the object, but may be the three-dimensional shape of the object.
For example, given object data that includes the function of an object, action target data that is the target of the function, and data of the subject that uses the function, the subject is easy to use (executes the function). (Easy) The shape of the object may be estimated.
In other words, even an object (tool) that produces the same function has a different shape that is easy to use depending on the subject (adult, child, elderly person), so the shape of the easy-to-use object may be estimated by a virtual environment simulator.

It is explanatory drawing of the interaction of function achievement. It is explanatory drawing of an Analysis by Reality-Based Simulator approach. It is estimation processing explanatory drawing of the mechanical function of the joint object based on Analysis by Reality-Based Simulation approach. It is explanatory drawing of the three-dimensional interactive model of a joint object. It is explanatory drawing of the volume representation of a joint object and a three-dimensional shape symmetry surface. It is a cross-sectional outline shape figure in the symmetry plane of a joint object. It is an action force change graph with and without an action mode. It is explanatory drawing of three holding patterns of a hand. It is explanatory drawing of Opposition Vector. It is grasp estimation explanatory drawing in Palm Opposition. It is a grip force distribution map in a grip pattern. It is a calculation model of acting force. It is function estimation (physical property of action object, and range of action stress) process explanatory drawing. It is a graph of NOD in PadOpposition. It is explanatory drawing of the reaction force of the joint object without an action target, and the reaction force of the joint object with an action target. It is explanatory drawing of the priority by the range of gripping force. It is a main function estimation result with respect to the clothespin. It is a main function estimation result for pliers. It is a main function estimation result with respect to a punch. It is a main function estimation comparison between simulation results and subjective evaluation experiments.

Claims (10)

1) As a real world data that depends on the real world, a new knowledge about an object is automatically acquired by a reality-based simulation by a virtual space simulator that is operated with the following data 1a), 1b), and 1c). A method for automatically acquiring new knowledge about an object based on a reality-based simulation characterized by including the following steps 2) and 3).
1a) Object data used to reproduce in the virtual space an object that is used by the subject and that can act on the action target;
1b) Action target data used to reproduce an action target on which an object acts in a virtual space by a computer;
1c) Subject data relating to the characteristics of the subject using the object;
2) A simulation step of simulating a subject-object-action target interaction using a virtual space simulator based on the given subject data, the given subject data, and the given subject data.
3) A knowledge acquisition step for acquiring new knowledge about the object based on the simulation result of the interaction between the subject, the object, and the action target. 1) The following data 1a), 1b) and 1c) are given as real world data based on the real world, and in order to automatically acquire new knowledge about the object by reality-based simulation, the following 2) 3) A reality-based virtual space simulator characterized by comprising:
1a) Object data used to reproduce in the virtual space an object that is used by the subject and that can act on the action target;
1b) Action target data used to reproduce an action target on which an object acts in a virtual space by a computer;
1c) Subject data relating to the characteristics of the subject using the object;
2) Simulating means for simulating the subject-object-action target interaction by a virtual space simulator based on the given subject data, the given subject data, and the given subject data.
3) Knowledge acquisition means for acquiring new knowledge about the object based on the simulation result of the interaction between the subject, the object and the action target. 1) A method for estimating a mechanical function of an object by a virtual space simulator which operates by being given the following data 1a), 1b) and 1c) as real world data based on the real world, and includes the following 2) 3) A method for estimating the mechanical function of an object, comprising the step of 4).
1a) An object including three-dimensional shape data and mechanical data of an object used by the subject and capable of acting on the action target in order to reproduce in a virtual space by a computer the action acting on the action target data;
1b) Action target data including physical property data of the action target in order to reproduce the action target on which the object acts in a virtual space by a computer;
1c) Subject data relating to the characteristics of the subject using the object;
2) A gripping state estimation step of estimating the gripping state of the object represented by the given object data by the subject represented by the given subject data using a virtual space simulator.
3) An action force calculation step of calculating a force applied by the object to the action target using the virtual space simulator in the grasp state estimated in the grasp state estimation step.
4) When the acting force estimated in the acting force estimation step acts on the acting target represented by the given acting target data, the mechanical function of the object is estimated based on a change that can occur in the acting target. Function estimation step.   The method according to claim 3, wherein the subject data is human characteristic data based on knowledge of anatomy, psychology, and / or ergonomics.   5. The method for estimating a mechanical function of an object according to claim 3, wherein the subject data includes gender data of the subject and / or age data of the subject.   The gripping state estimated in the gripping state estimation step includes a gripping pattern that is a posture when the subject grips an object and a gripping force that is a force when the subject grips the object. Item 5. The method for estimating a mechanical function of an object according to any one of Items 3 to 4. 1) The following 1a), 1b) and 1c) data are given as real world data based on the real world, and the following means 2), 3) and 4) are included to estimate the mechanical function of the object. A virtual space simulator characterized by
1a) An object including three-dimensional shape data and mechanical data of an object used by the subject and capable of acting on the action target in order to reproduce in a virtual space by a computer the action acting on the action target data;
1b) Action target data including physical property data of the action target in order to reproduce the action target on which the object acts in a virtual space by a computer;
1c) Subject data relating to the characteristics of the subject using the object;
2) Grasping state estimation means for estimating the gripping state of the object represented by the given object data by the subject represented by the given subject data.
3) Action force calculation means for calculating the force exerted by the object on the action target in the grip state estimated by the grip state estimation means.
4) When the acting force estimated by the acting force estimation means acts on the acting target represented by the given acting target data, the mechanical function of the object is estimated based on a change that can occur in the acting target. Function estimation means.   8. The virtual space simulator according to claim 7, wherein the subject data is human characteristic data based on knowledge of anatomy, psychology, and / or ergonomics.   9. The virtual space simulator according to claim 7, wherein the subject data includes sex data of the subject and / or age data of the subject.   The gripping state estimated by the gripping state estimation means includes a gripping pattern that is a posture when the subject grips an object and a gripping force that is a force when the subject grips the object. Item 10. A virtual space simulator according to any one of Items 7 to 9.