JP5608007B2 - Surface hardness measurement device, tactile force sense presentation device, surface hardness measurement program, and tactile force sense presentation program - Google Patents

Description

  The present invention relates to a surface hardness measurement device, a tactile force sense presentation device, a surface hardness measurement program, and a tactile force sense presentation program. The present invention relates to a surface hardness measurement device, a tactile force sense presentation device, a surface hardness measurement program, and a tactile force sense presentation program for highly accurate presentation.

  2. Description of the Related Art Conventionally, devices that present information using a tactile sensation have been developed in the field of virtual reality in which a device that mechanically presents the shape of a virtual object with a force sense has been developed. In such a force sense presentation device, for example, position information of a virtual object is detected according to a user operation, and a reaction force calculated from the position information obtained by the detection and the position information of the virtual object is fed back to a finger or the like. ing. In addition, as a method for presenting the reaction force described above, a method using the tension of the wire, the air pressure, the moment of inertia of the rotating body, and the like has been developed (for example, see Non-Patent Documents 1 and 2).

  Here, “PHANToM” of the haptic device shown in Non-Patent Document 1 is a point contact type that feeds back haptic information corresponding to the movement of the user's fingertip with an arm-type actuator having 3 or 6 degrees of freedom. Presenting. In addition, “Haptic Master” shown in Non-Patent Document 2, which is another method, uses a parallel mechanism that connects a plurality of parallel actuators to transmit a reaction force. These devices can touch a virtual object and transmit the shape and hardness of the object.

  In addition to the above, a multi-finger type force sense device that gives a sense of grasping or touching an object with a plurality of fingers has been developed (for example, see Non-Patent Documents 3 to 5).

  In order to virtually reproduce information on a real object remotely through broadcasting or communication using such a haptic sense presentation device, it is necessary to acquire three-dimensional spatial coordinate information of the real object. Spatial coordinates can be obtained by scanning a laser beam and detecting reflected light to measure the distance to the object, such as a three-dimensional digitizer (see Non-Patent Document 6, for example) or a number of cameras around the actual object. A multi-view camera (see, for example, Non-Patent Document 7) or the like that is arranged in the above, appropriately processes images obtained from various directions, and generates three-dimensional coordinate data has been put into practical use.

T. T. et al. H. Massie, J.M. K. Salisbury: “The phantom haptic interface: A device for probing virtual objects,” Proc. ASME Haptic Interfaces for Virtual Environment and Teleoperator Systems In Dynamic Systems and Control 1994, vol. 1, pp. 295-301 (1994) Asano, Yano, Iwata: "Development of elemental technology for surgical simulation in a virtual environment using force display", VR University, pp. 95-98 (1996) Sato, Hirata, Kawarada: "Proposal of Spatial Interface Device SPIDAR", Theory of Science, Vol. J74-D-II, no. 7, pp. 887-894 (1991) http: // www. immersion. com / 3d / products / H. Kawasaki et al. : “Development of five-fingered haptic interface: HIRO II”, Proc. 15th Int. Conf. Artificial Reality & Teleexistence, pp. 209-214 (2005) http: // konicaminolta. jp / instruments / products / 3d / index. html Iwabuchi: “Broadcasting application of multi-view video processing technology”, The Institute of Image Information and Television Engineers, ITE Technical Report Vol. 33, no. 42, pp. 21-28, October 21, 2009

  By the way, in order to provide a highly realistic environment through broadcasting and communication with higher accuracy, it is effective to add tactile information to audiovisual information and present a sense close to a real object. Here, in order to present the real object information remotely to the tactile sense, obtain information such as the three-dimensional spatial coordinates such as the surface and corners of the real object, the roughness and hardness of the surface, and present the tactile force sense. It must be presented on the device.

  However, with conventional 3D digitizers and multi-viewpoint camera systems, even if 3D coordinate data and surface roughness data of an object can be acquired, it is not considered to sense hardness information. In order to present a tactile sensation closer to a real object, a new information sensing method using hardness as an index is required.

  The present invention has been made in view of the above-described problems, and provides a surface hardness measurement device and a tactile force sense for presenting a tactile sensation close to a real object with higher accuracy when an object is virtually presented. It is an object of the present invention to provide a presentation device, a surface hardness measurement program, and a haptic sense presentation program.

  Furthermore, in other words, the object of the present invention is to provide a surface hardness measurement device, a tactile force sense presentation device, a surface that can obtain a more natural tactile sensation as if a real object is touched in the tactile force sense presentation of a virtual object. It is to provide a hardness measurement program and a haptic sense presentation program.

  In order to solve the above problems, the present invention employs means for solving the problems having the following characteristics.

According to the first aspect of the present invention, there is provided a surface hardness measurement apparatus for measuring hardness information of a real object corresponding to a virtual object in a virtual space to be presented to a user by a tactile force sense. A three-dimensional shape measuring means for measuring a three-dimensional shape, and the ultrasonic transducer array changes a plurality of ultrasonic waves at a predetermined cycle based on the three-dimensional shape information obtained by the three-dimensional shape measuring means, and the real object Non-contact measurement of the vibration amplitude of the surface caused by a periodic change in radiation pressure when irradiating a predetermined position on the surface of the surface, and the displacement ratio or displacement amount at the position from the measured vibration amplitude of the surface A displacement measuring means that measures the viscoelastic impedance for each position measured by the real object from the displacement ratio or displacement obtained by the displacement measuring means, and the estimated viscoelastic impedance And having a measuring and controlling unit for obtaining the stiffness information of each predetermined position of the object.

  According to the first aspect of the present invention, the hardness information of the real object can be acquired with high accuracy. Thereby, when presenting an object virtually, a tactile sensation close to a real object can be presented with high accuracy.

The invention described in claim 2, in haptic presentation device provided with a surface hardness measuring apparatus according to claim 1, given the actual object by the viscoelastic impedance obtained by the measurement control unit It has a tactile force sense presenting means for performing force sense control based on the hardness information by mapping the hardness information for each position onto a virtual object in a virtual space corresponding to the real object.

  According to the invention described in claim 4, when the object is virtually presented using the surface hardness information, a tactile sensation close to the real object can be presented with high accuracy.

According to a third aspect of the present invention, the tactile force sense presenting means performs force sense control on the virtual object in correspondence with the three-dimensional shape information and the hardness information obtained by the three-dimensional shape measuring means. It is characterized by that.

The invention described in claim 4 has surface roughness measuring means for obtaining a surface shape including information on the surface roughness and roughness of the real object, and the tactile sensation presentation means is the surface roughness. Force control is performed on the virtual object in correspondence with the surface roughness information obtained by the measuring means and the hardness information.

The invention described in claim 5 is a surface hardness measurement program for causing a computer to function as the surface hardness measurement apparatus according to claim 1 .

According to the fifth aspect of the present invention, the hardness information of the real object can be acquired with high accuracy. Thereby, when presenting an object virtually, a tactile sensation close to a real object can be presented with high accuracy. Moreover, the surface hardness measurement process can be easily realized by installing the execution program in the computer.

The invention described in claim 6 is a haptic sense presentation program for causing a computer to function as the haptic sense presentation device according to any one of claims 2 to 4 .

According to the invention described in claim 6 , when the object is virtually presented using the surface hardness information, a tactile sensation close to a real object can be presented with high accuracy. Further, by installing the execution program in the computer, it is possible to easily realize the tactile sensation presentation process.

  In addition, what applied the component, expression, or arbitrary combination of the component of this invention to a method, an apparatus, a system, a computer program, a recording medium, a data structure, etc. is also effective as an aspect of this invention.

  According to the present invention, the hardness information of a real object can be acquired with high accuracy. Thereby, when presenting an object virtually, a tactile sensation close to a real object can be presented with high accuracy.

It is a figure which shows an example of schematic structure of the surface hardness measuring apparatus in this embodiment. It is a flowchart which shows an example of a surface hardness measurement processing procedure. It is a figure which shows an example of a function structure of a surface hardness measuring apparatus. It is a figure for demonstrating the ultrasonic transducer | vibrator array in this embodiment. It is a figure for demonstrating the mechanical impedance applied to this embodiment. It is a figure which shows the outline | summary of the tactile-force sense presentation apparatus using the surface hardness measuring apparatus in this embodiment. It is a flowchart which shows an example of the tactile-force sense presentation process procedure in this embodiment. It is a figure which shows the example of description of the virtual object data which has hardness and roughness information.

<About the present invention>
The present invention provides a technique for detecting the hardness of the surface of an object, which is used in a haptic sense presentation technique for virtually presenting an object. In addition to this, it is possible to detect the three-dimensional coordinates of the object and the surface of the object, and to provide a tactile sensation presentation method that presents the unevenness and a means for presenting a tactile sensation such as a rough feeling on the surface. Present virtual objects to multiple users.

  More specifically, in the present invention, in order to extract and present the surface hardness of an object, the surface viscoelastic impedance is estimated by measuring the displacement of the surface with ultrasonic radiation pressure or the like. That is, in the present invention, surface viscoelastic impedance or the like is used as an example of hardness information. Furthermore, the hardness of the surface can be experienced by presenting the estimated viscoelastic impedance to at least one tactile sensation presentation device.

  In addition to the viscoelastic impedance described above, a means for obtaining three-dimensional shape information of a computer-created object or real object with a digitizer and presenting the object shape unevenness, and fine unevenness such as a rough surface And means for acquiring roughness information and presenting a tactile sensation in combination with unevenness information of the object shape, and integrating them makes it possible to present a more realistic virtual object.

  In the present invention, the virtual object data obtained by these methods is centrally managed and provided to the tactile sensation presentation device of each client via broadcasting or the Internet, so that a plurality of users can share the same object. Can be palpated. Thereby, in the tactile force sense presentation device that virtually presents an object, it is possible to obtain a tactile sensation closer to a real object and to provide a presentation method that is easier to recognize.

  Next, a preferred embodiment of the surface hardness measurement device, the surface hardness measurement device, the tactile force sense presentation device, the surface hardness measurement program, and the tactile force sense presentation program according to the present invention having the above-described features. This will be described in detail with reference to the drawings.

<Outline configuration example of surface hardness measuring device>
FIG. 1 is a diagram illustrating an example of a schematic configuration of a surface hardness measurement apparatus according to the present embodiment. A surface hardness measurement apparatus 10 shown in FIG. 1 includes one or a plurality of ultrasonic transducer arrays 11 (in the example of FIG. 1, ultrasonic transducer arrays 11-1 and 11-2) serving as displacement measuring means, and vibrations. The measuring unit 12 (vibration measuring units 12-1 and 12-2 in the example of FIG. 1), a three-dimensional shape measuring unit 13, a measurement control unit 14, and a recording unit 15 are configured.

  In the ultrasonic transducer array 11, a large number of transducers that generate ultrasonic waves are arranged at predetermined positions at predetermined intervals. Note that the ultrasonic transducer array 11 in the present embodiment can be an existing ultrasonic transducer array. Proc. World Haptics 2009, Salt Lake City, UT, March 18-20, pp. 256-260, 2009. "and the like can be used. A specific example of the configuration of the ultrasonic transducer array 11 will be described later.

  Here, the surface hardness measurement apparatus 10 according to the present embodiment uses, for example, about 180 mm square with respect to the measurement target object 21 (for example, a real object including not only a product but also a living thing) by the ultrasonic transducer array 11. The ultrasonic waves 22 are emitted from the phased array and converged on the surface of the measurement target object 21. The diameter of the convergence spot is about 10 mm, for example. At that time, a pressure (radiation pressure) proportional to the energy density of the ultrasonic beam is applied to the target surface.

  Here, in this embodiment, the ultrasonic transducer array 11 can apply a force of about 1.6 gf to the vicinity of the convergence spot on the target surface at the maximum output. The ultrasonic transducer array 11 can change the pressure by turning on and off the ultrasonic beam at a predetermined timing.

  The vibration measuring means 12 selectively generates ultrasonic waves having arbitrary different frequencies up to about several hundred Hz at a preset timing by the ultrasonic transducer array 11, and generates them by a periodic pressure change. The vibration amplitude of the target surface of the measurement target object 21 is measured. Thereby, in this embodiment, the hardness for every position can be estimated from the ratio or displacement amount of the displacement with respect to force. In the example of FIG. 1, the vibration measuring means 12-1 and 12-2 are provided so as to be paired with the ultrasonic transducer arrays 11-1 and 11-2.

  It should be noted that the position where the force is applied, that is, the convergence position of the ultrasonic radiation pressure is set to a free position on the surface by adjusting the drive phase of each element of the ultrasonic transducer array (ultrasonic phased array) 11. can do.

  Further, the hardness at one point (attention point) at this time can be derived from the equation of viscoelastic impedance Z (ω) = F (ω) / D (ω). Note that F (ω) described above is the amplitude of the angular frequency ω component of the force applied to the surface by the radiation pressure, and D (ω) described above is the surface displacement at the angular frequency ω (at the center of the ultrasonic convergence point). (Surface displacement) amplitude.

  If the amplitude F of each frequency ω component of the force applied to the surface and the amplitude D of the surface displacement are displayed as complex numbers (phasor display), the mechanical impedance spring element, mass element and damper element are decomposed from these values. Can be described. The above-described mechanical impedance will be described later.

  The three-dimensional shape measuring unit 13 measures the three-dimensional shape of the measurement target object 21. The three-dimensional shape measuring means 13 is used before the ultrasonic wave for measuring the hardness of the measurement target object 21 is radiated from the ultrasonic transducer array 11, that is, before the hardness measurement is started or when the hardness measurement is started. Get it. These data are recorded in the recording unit 15 and the like, and the measurement control unit 14 uses the data recorded in the recording unit 15 to compare the measurement target object 21 and the ultrasonic transducer array 11 relative to each other. The focal position of the ultrasonic wave is set from the positional relationship.

  The measurement control unit 14 moves the position relative to the ultrasonic transducer arrays 11-1 and 11-2, sets the focal position of the ultrasonic wave, generates ultrasonic waves, moves relative to the vibration measurement units 12-1 and 12-2, Measurement timing, movement of the three-dimensional shape measuring means 13, shape measurement control, and the like are performed.

  That is, the measurement control unit 14 moves the ultrasonic transducer arrays 11-1 and 11-2, the vibration measurement units 12-1 and 12-2, and the three-dimensional shape measurement unit 13 by using a moving mechanism provided in advance. Then, the hardness of the entire surface of the measurement target object 21 is measured. In the present invention, the position where the hardness is measured is not limited to the entire surface described above, and may be only a predetermined region of the measurement target object 21 that is presented to the user with a tactile force sense, for example.

  Further, when there are a plurality of ultrasonic transducer arrays 11 and vibration measuring means 12, the measurement control means 14 performs control so as to measure the hardness at different positions with respect to the measurement target object 21. Thereby, for example, according to the shape or the like of the measurement target object 21, the entire hardness can be measured quickly and accurately by properly using a complicated shape portion and a gentle shape portion.

  In addition, the measurement control unit 14 checks in advance the force applied to the normal incidence surface at each focal point of the ultrasonic transducer array 11, and stores the data in the recording unit 15 so that the data can be used as appropriate. Record as. The vertical incident surface is a surface perpendicular to a straight line connecting the focal point and the center of the ultrasonic transducer array 11 among the planes passing through the focal point.

  In addition, the radiation pressure by the ultrasonic beam works most effectively when the beam is perpendicular to the surface, and the force exerted on the surface is small when it is incident obliquely. Specifically, when the angle formed between the normal line of the target surface and the incident direction is θ, the applied force decreases approximately in proportion to cos θ.

  Here, since the direction of the normal vector of the target surface is known in advance from the three-dimensional shape data obtained by the three-dimensional shape measurement means 13, the measurement control means 14 uses the net to be applied to the surface. And viscoelastic impedance Z is calculated from the vibration amplitude. Specifically, the measurement control unit 14 calculates the cosine cos θ between the normal vector at the target point on the target surface (the focal position of the ultrasonic transducer array 11) and the normal vector of the normal incidence surface, and performs vertical calculation. Multiplying cos θ by the force that should work against the incident surface (recorded in the recording means 15 or the like described above) to obtain the net force.

  Here, in the embodiment shown in FIG. 1, since a plurality of phased arrays (in the example of FIG. 1, ultrasonic transducer arrays 11-1 and 11-2) facing a plurality of directions are installed, θ is 90. When the angle approaches, the ultrasonic transducer array 11 having a smaller θ is selected and pressure is applied. In addition, the measurement control unit 14 selects and measures the vibration measuring devices 12-1 and 12-2 that have the target surface near the front if necessary.

  Here, in this embodiment, when the vibration measuring device 12 is of a type that measures only one vibration displacement, the measurement area of the vibration measuring device matches each point of interest (measurement point) on the target surface. The direction / position of the vibration measuring device is controlled by a moving mechanism, or a mirror is installed in the light path, and the direction / position is controlled by the moving mechanism. Further, the measurement target object 21 may be controlled by a moving and rotating mechanism.

  In addition, for the vibration measurement device 12 and the three-dimensional shape measurement unit 13 in the present embodiment, any optical product such as a stereo method, a laser beam cutting method, a moire fringe method, and an optical interference method can be used. The stereo system is a system in which a marker added to a projected image of a projector or a non-measurement object is measured by two optical cameras based on the same principle as human binocular vision. The laser beam cutting method scans an input object with a slit-shaped laser beam and receives the reflected light with a camera, thereby obtaining distance information from the subject on the basis of the triangulation principle and three-dimensionally. This is a method of data conversion. In addition, the moire fringe method is a method in which a grating pattern is arranged on each of a projector (light source) and a camera, and the depth is measured by observing the moire pattern. Further, the optical interference method is a method of measuring the depth by observing interference fringes generated by causing reflected light and reference light to interfere with each other, and is effective for measuring, for example, nano-order fine irregularities.

  The vibration frequency ω is selected according to the measurement object and application purpose. By measuring Z (ω) for multiple ωs, the force response to any time-varying pattern of displacement given by the user can be reproduced more faithfully within a range where linearity between the displacement and force can be assumed. Can do. However, in this case, since the measurement time becomes long, the method of selecting ω is adjusted according to the allowable measurement time. For example, one or more frequencies of 1 Hz, 10 Hz, 30 Hz, 50 Hz, and 100 Hz are selected.

<Surface hardness measurement processing procedure>
Here, the surface hardness measurement processing procedure in the present embodiment will be described using a flowchart. FIG. 2 is a flowchart showing an example of the surface hardness measurement processing procedure. As shown in FIG. 2, first, the three-dimensional shape of the target surface (entire or predetermined region) of the measurement target object 21 that is an actual object is measured before the hardness measurement is started or at the start of the hardness measurement (S01).

  Next, the focal point of the ultrasonic wave with respect to the target surface of the measurement target object 21 based on the relative positional relationship between the measurement target object 21 and the ultrasonic transducer array 11 from the three-dimensional shape result obtained by the process of S01. A position is set (S02).

  Next, ultrasonic waves of different frequencies are selectively emitted to the target surface at a preset timing (S03), and the vibration amplitude of the target surface of the measurement target object 21 generated by the periodic pressure change is measured. (S04). Moreover, the hardness of the position is estimated from viscoelastic impedance from the ratio of displacement to the force obtained by the process of S04 or the amount of displacement (S05).

  Next, it is determined whether or not the user finishes the surface hardness measurement process according to an instruction or the like (S06). If the process is not completed (NO in S06), the process returns to S01 to continue the process for other positions (target surface), and the ratio of displacement to force for each measured position of the measurement target object 21 or The viscoelastic impedance is estimated from the amount of displacement, and the hardness information for each predetermined position of the measurement target object 21 is acquired as the viscoelastic impedance. Moreover, in the process of S06, when the process is completed (YES in S06), the surface hardness measurement process is terminated.

  In the present embodiment, since the hardness information corresponds to viscoelastic impedance, the larger the value of viscoelastic impedance, the harder (the resistance value against applied force is higher).

  Thus, according to the present invention, the viscoelastic impedance Z can be measured and recorded for each point of the three-dimensional surface shape data of the target object. Further, by transmitting this data to, for example, a tactile sense presentation device, the user can virtually experience the tactile sensation (viscoelastic impedance) generated from the data.

<Functional Configuration Example of Surface Hardness Measurement Device 10>
Next, a specific functional configuration example of the surface hardness measurement apparatus 10 in the present embodiment described above will be described with reference to the drawings. FIG. 3 is a diagram illustrating an example of a functional configuration of the surface hardness measurement apparatus. The surface hardness measurement apparatus 10 shown in FIG. 3 includes a three-dimensional shape measurement unit 31, a three-dimensional shape recording unit 32, a hardness information measurement position calculation unit 33, a pressure application position control unit 34, and a vibration amplitude measurement position. Control means 35, vibration amplitude measurement means 36, applied pressure calculation means 37, pressure application cycle control means 38, pressure application means 39, hardness information calculation means 40, and hardness information recording means 41 are provided. It is configured as follows. In the example of FIG. 3, hardness information transmission means 42 for outputting the acquired surface hardness information to the haptic sense presentation device and reflecting the hardness information in the presentation information of the virtual object; Presentation means 43.

  In order to perform the operation in the present embodiment described above, the surface hardness measuring apparatus 10 is provided with the above-described function. Specifically, the three-dimensional shape measuring unit 31 measures the three-dimensional shape of the measurement target object as described above. The three-dimensional shape recording unit 32 records the three-dimensional shape information of the measurement target object measured by the three-dimensional shape information measuring unit 31. The three-dimensional shape recording unit 32 corresponds to the recording unit 15 described above.

  The hardness information measurement position calculation means 33 calculates a position for measuring the hardness information of the target surface on the object to be measured. Further, the pressure application position control unit 34 controls the position to apply pressure according to the measurement position of the hardness information calculated by the hardness information measurement position calculation unit 33.

  The vibration amplitude measurement position control means 35 controls the position at which the vibration amplitude is measured according to the measurement position calculated by the hardness information measurement position calculation means 33. The vibration amplitude measuring means 36 measures the vibration amplitude generated on the object surface by the application of pressure. Further, the applied pressure calculating means 37 inputs normal incident surface applied force data 44, which is a set of numerical values of applied force when pressure is applied perpendicularly to the surface of the measurement target object 21 set in advance. Calculate the pressure to be applied and the net force applied to the object surface.

  The pressure application cycle control means 38 controls the cycle of the applied pressure. The pressure application means 39 applies a predetermined pressure to the object. The hardness information calculation unit 40 calculates hardness information from the net force applied to the object surface calculated by the applied pressure calculation unit 37 and the measurement value measured by the vibration amplitude measurement unit 36. The hardness information recording unit 41 records the hardness information calculated by the hardness information calculating unit 40. The hardness information recording unit 41 corresponds to the recording unit 15 described above.

  In the present embodiment, the hardness information transmission unit 42 transmits the hardness information calculated by the hardness information calculation unit 40 and the hardness information recorded by the hardness information recording unit 41. The tactile sensation presentation unit 43 performs tactile sensation presentation as described later according to the hardness information transmitted from the hardness information transmission unit 42.

<About the ultrasonic transducer array 11>
Next, a specific example of the ultrasonic transducer array 11 in the present embodiment will be described. FIG. 4 is a diagram for explaining the ultrasonic transducer array in the present embodiment.

  As shown in FIG. 4A, on the radiation plane of the ultrasonic transducer array 11, a large number of array elements 51 that generate ultrasonic waves are arranged at predetermined intervals. 4A is arranged in a square shape, the shape is not particularly limited in the present invention, and may be a rectangle such as a rectangle or a circle.

  Further, for example, a position with a distance of about 200 mm with respect to the radiation surface of the ultrasonic transducer array 11 is set as a target plane, and each array element 51 arranged at the position is predetermined at a predetermined timing from a predetermined position. The ultrasonic wave is radiated in the direction of and the ultrasonic wave is radiated to the target surface of the measurement target object 21.

  That is, in the present embodiment, as described above, the site to which the force is applied, that is, the convergence position of the ultrasonic radiation pressure adjusts the drive phase of each element of the ultrasonic transducer array (ultrasonic phased array) 11. Thus, it can be set to a free position on the surface. Therefore, the hardness distribution of the surface can be measured over the entire target surface by sequentially switching the position (attention point), that is, by scanning. Note that the attention point at this time is typically set at intervals of about 10 mm in length and width as shown in FIG. 4, but it is finer depending on conditions such as the type of the object to be measured 21 and the shape of the surface. It may be set or coarsely set.

  Further, as scanning in the ultrasonic transducer array 11, for example, as shown in FIG. 4B, linear scanning (FIGS. 4B and 4A) and sector scanning (FIGS. 4B and 4B). , Zone focus (FIGS. 4B and 4C), dynamic depth focus (FIGS. 4B and 4D), and the like can be sequentially switched or combined. Measurement is performed using the zone focus shown in 4 (B) and (c), but the present invention is not limited to this, and other methods can be used.

  In the measurement, the target plane is moved by moving either or both of the ultrasonic transducer array 11 and the measurement target object 21 relative to the measurement target object 21, The measurement target object 21 can be scanned from a predetermined position.

  Here, in this embodiment, if the vibration displacement is insufficient and the measurement is hindered, the ultrasonic transducer array 11 is made larger and more transducers are driven to increase the applied force. You may let them. In that case, the distance between the ultrasonic transducer array 11 and the object is used as far as the opening of the ultrasonic transducer array 11.

  In such an ultrasonic transducer array 11, the maximum output of the system (when using ultrasonic waves) is about 1.6 gf as described above. However, if the target is not a living body, the upper limit of force is particularly high. However, if a large phased array is made and driven powerfully, considerable power can be produced.

For example, when the measurement target object 21 is applied to a human body, it is necessary to set the power of ultrasonic waves transmitted through the human body to a safety standard or less. Here, since the strictest (safe side) standard is said to be about 100 mW / cm ² , the radiation pressure corresponding to this is about 60 gf / cm ^{2, which} is a practical upper limit.

  In the present embodiment, when applying about 1.6 gf, a laser displacement meter can stably measure up to a displacement of about 1 μm, so that the hardness of the skin is about 10 times that of the skin hardness. Until then, quantitative evaluation is possible. If a force of about 60 gf can be generated, it can be measured even if it is ten times harder than that. Moreover, when the measurement target object 21 is soft, it can be measured appropriately by reducing the applied radiation pressure.

  Further, for example, as long as an ultrasonic wave of about 40 kHz is used, the convergence spot does not become about 10 mm (˜wavelength 8 mm) or less. If the frequency is increased, the convergence spot is reduced, but the attenuation distance is reduced in inverse proportion to the square of the frequency. For example, since the attenuation distance until the amplitude of the 40 kHz ultrasonic wave becomes 1 / e is 8.7 m, ultrasonic attenuation does not need to be considered much when measuring an object separated to about 1 m. However, if the frequency is increased 10 times, this distance becomes 1/100, so that only a very close object can be measured. Therefore, the practical lower limit of the convergence spot is about 10 mm or about 0.5 mm.

<About mechanical impedance>
Next, the mechanical impedance described above will be described with reference to the drawings. FIG. 5 is a diagram for explaining the mechanical impedance applied to the present embodiment. FIG. 5 shows a basic model of mechanical impedance. In this case, assuming that the generalized spring constant is mass m, spring coefficient k, and damper α, Z (ω) = F / D = −ω2m + jωε + k.

  In the above description, “impedance” is used, but in the case of impedance, the denominator is usually a speed instead of a displacement. In this case, Z (ω) = F / jωD = jωm + ε + k / jω It becomes. As the viscoelastic model, there are countless variations such as the one in which the damper α and the spring k are connected in series, and the one in which the parallel model shown in FIG. 5 is connected (in series and in parallel). Any of them can be applied.

<Tactile force sense presentation device using surface hardness measurement device>
Next, a tactile sensation presentation device using the above-described surface hardness measurement device will be described with reference to the drawings. FIG. 6 is a diagram illustrating an outline of a haptic sense presentation device using the surface hardness measurement device according to the present embodiment.

  The haptic sense presentation device 60 using the surface hardness measurement device shown in FIG. 6 acquires or generates haptic information such as the shape and hardness of the measurement target object 21 that is an actual object on the haptic sense information transmission side. Specifically, a surface hardness measuring device 61, a three-dimensional shape measuring device 62, a surface roughness measuring device (surface roughness measuring means) 63, and a measurement control means 64 are provided. And a recording means 65 and a transmission means 66. The tactile force sense presentation device 60 includes sensory transmission devices 70-1 and 70-2 and a tactile force sense control device 80 on the tactile force sense presentation side for presenting a virtual object by tactile force sense. It is configured as follows.

  The surface hardness measuring device 61 measures the hardness of the object surface in a non-contact manner as shown in the present embodiment described above.

  The three-dimensional shape measuring device 62 measures the three-dimensional shape of the entire object to be measured 21 as with the three-dimensional shape measuring means 13 in the present embodiment described above. The surface roughness measuring device 63 measures fine irregularities and roughness on the object surface. In addition, the measurement control unit 64 controls each measurement device (the surface hardness measurement device 61, the three-dimensional shape measurement device 62, and the surface roughness measurement device 63) in the present embodiment and integrates or generates data.

  In addition, the recording unit 65 records the data of each measuring device described above and the integrated or generated data in association with the measurement target object 21. The transmission unit 66 transmits data including the haptic information of the object integrated or generated by the measurement control unit 64 to one or a plurality of haptic sense presentation sides.

  In the present embodiment, by using the same method as in the present embodiment described above, the surface hardness measuring device 61 is controlled by the measurement control means, thereby controlling and acquiring the three-dimensional shape measuring device 62, and the recording device 45. The viscoelastic impedance described above can be measured for each point of the three-dimensional surface shape data of the measurement target object 21 recorded in FIG.

  Further, in the present embodiment, by controlling the surface roughness measuring device 63, fine irregularities and roughness on the surface of the measurement target object 21 that is a real object are measured for each point of the three-dimensional surface shape data. And recorded in the recording means 65. At this time, the measurement control unit 64 performs mapping by mapping the surface hardness information such as viscoelastic impedance and the surface roughness information in association with the coordinate space of the three-dimensional surface shape data as a reference, and is a real object. The unified haptic information of the measurement target object 21 is recorded in the recording means 65.

  In addition, the measurement control unit 64 reads the haptic information recorded in the recording unit 65 and transmits the haptic information to the one or more haptic control devices 80 via the transmission unit 66. That is, by outputting data including the measured haptic information of the object to the plurality of haptic control devices 80 at the same time, a plurality of users can simultaneously sense the object with the haptic sensation.

  In other words, in addition to the macro three-dimensional surface shape, by presenting the microscopic surface roughness and roughness information together with the surface hardness information, only the hardness information is presented, or Compared with the case where only the roughness information is presented, the tactile sensation peculiar to the material constituting the real object can be expressed more. Therefore, for example, there are a plurality of cases of different materials (or states) even if the hardness is actually the same, and cases of different materials (states) having the same surface roughness and different hardness. By doing so, it is possible to express a difference in material that cannot be grasped by only presentation of hardness information or presentation of roughness information, and it is possible to experience a material feeling closer to a real object with a tactile sensation.

  Note that the three-dimensional surface shape, the fine unevenness and roughness of the object surface, and the hardness information are associated with the same coordinate space or minimum component, but may be presented independently or in combination.

  Here, the three-dimensional shape measuring device 62 and the surface roughness measuring device 63 in the present embodiment may be one device capable of simultaneously measuring, and as described above, the stereo method, the laser beam cutting method, the moire fringe. Any optical product such as a method or an optical interference method may be used.

  The transmission means 66 may be any device capable of transmitting data to a plurality of devices having a receiving function by an electric signal or the like, and a wired or wireless communication facility or a broadcast facility using radio waves or the like can be used as a transmission path. .

  Further, the measurement control means 64 may virtually generate and record the same haptic information on the computer without using the measurement target object 21 that is a real object and each measurement device.

  The three-dimensional surface shape data acquired by the three-dimensional shape measuring device 62 is data in a markup language or the like when the virtual three-dimensional object is displayed on a communication network such as the Internet. For example, by expanding the data description format, the haptic information can be determined with respect to each of the smallest components such as polygons constituting the virtual object. Can be additionally described.

  In the present embodiment, the parameter representing the amplitude of the fine unevenness on the surface and the parameter representing the spread of roughness are also described for each of the smallest constituent elements such as polygons constituting the virtual object. To do. The hardness, surface roughness, and the like can be described as constants or coefficients based on data acquired by each measuring device, but the present invention is not limited to this, and polygons that constitute a virtual object, etc. Each of the minimum components may be given in the form of a function that calculates a reaction force according to the applied force, for example.

  Furthermore, in this embodiment, it is possible to associate information such as hardness and surface roughness with the three-dimensional coordinates of the surface of the virtual object as well as the minimum constituent elements constituting the virtual object. These can be described, for example, in an extended form of VRML (Virtual Reality Modeling Language) or the like, but is not limited to this in the present invention. ) An extended file format such as 3DMLW (3D Markup Language for Web) can be used.

  Further, on the haptic sense presentation side of the haptic sense presentation device 60 shown in FIG. 6, the sensory transmission devices 70-1 and 70-2 have a reaction force or the like that can restrict movement to any one of the different fingertips of the user. For example, a finger sack or the like is attached to a finger that transmits haptic feedback.

  Furthermore, the sensory transmission devices 70-1 and 70-2 shown in FIG. 6 output the position information acquired from the user's finger or palm to the haptic control device 80, and control information from the haptic control device 80. Based on the above, a predetermined operation is performed to transmit a sense to the user. Here, the above-described finger sensory transmission devices 70-1 and 70-2 are configured to include a base 71, a drive unit 72, a link mechanism 73, and a fingertip transmission unit 74.

  The base 71 (the bases 71-1 and 71-2 in the example of FIG. 6) is installed at a predetermined location on the real space on the haptic sense presentation side. The drive unit 72 (in the example of FIG. 6, the drive units 72-1 and 72-2) can be driven to rotate by 0 to 360 degrees horizontally with respect to the base 71, for example. Then, the shafts of the link mechanisms 73-1 and 73-2) can be rotated by 0 to 360 degrees perpendicular to the base 71. In addition, the drive part 72 in this embodiment can use a motor, a wire mechanism, a rotary encoder, etc., for example.

  Since the link mechanism 73 is composed of one or more rods (shafts), one of which is connected to the drive unit 72, it can rotate 0 to 360 degrees horizontally and vertically with respect to the surface of the base 71. . That is, the drive unit 72 and the link mechanism 73 can move in any direction around the base 71.

  In addition, the link mechanism 73 is connected to the fingertip transmission unit 74, and for the user's finger, the position information of the finger, the position information of the virtual object acquired from the above-described haptic information transmission side, the overall shape information, An operation for applying a tactile load such as an appropriate reaction force is performed according to the surface roughness information and the surface hardness information.

  The fingertip transmission unit 74 cannot move in any direction with respect to the fingertip according to the positional information of the fingertip and the positional relationship of the virtual object, or gives a predetermined amount of reaction force by applying a predetermined amount of reaction force. Gives an objective load. In the present embodiment illustrated in FIG. 6, the fingertip transmission unit 74 includes a finger sack (gimbal) that transmits haptic feedback information to two or more fingers (multiple fingers) of the user. That is, in the present embodiment, for example, a finger sack is attached to the user's thumb and index finger, and haptic information (tactile force load) when a virtual object is gripped is presented. The present invention is not limited to this. For example, it may be attached to at least one of the five fingers of the hand, and at least one of the fingers existing in the left and right hands is intended for both hands. It may be attached. Further, not only the finger but also the palm part or the entire hand is covered with a glove or the like, and a tactile load is presented.

  Note that, as a tactile force load such as a reaction force or pressure stimulus in the present embodiment, for example, when a virtual object is a hard cylinder, the user is based on position information and hardness information on the surface of the virtual object. A predetermined amount of tactile sensation load is applied to the finger or palm of the user to restrain the movement of the virtual object from the surface position to the inside. Also, when the virtual object is a soft cylinder, it moves to the inside of the surface of the virtual object relative to the user's finger or palm based on the position information and hardness information of the virtual object. As a result, a predetermined amount of haptic load is applied so that the movement is gradually restrained. The tactile force load includes, for example, pressure or vibration, but is not limited to this in the present invention, and includes, for example, a load caused by some sense such as an electric signal or heat.

  In addition, the sensory transmission devices 70-1 and 70-2 shown in FIG. 6 are installed corresponding to the number of fingers to be presented to the user as a tactile force sense. However, the present invention is not limited to this. For example, a plurality of fingers may be attached to the fingertip transmission unit of one sensory transmission device to transmit a sense, or a plurality of sensory transmission devices may transmit a sensory sense to one finger. In addition, the palm in the present invention described above may indicate the whole including a coupling portion with a finger, or may be one or a plurality of predetermined regions or points.

  The haptic control device 80 controls each component of the above-described sensory transmission devices 70-1 and 70-2, and the position of a fingertip or the like obtained from each of the sensory transmission devices 70-1 and 70-2. Each fingertip transmission unit 74-1 is detected by detecting information and controlling each component of each of the above-described sensory transmission devices 70-1 and 70-2 based on data sent from the transmission means 66 on the real object measurement side. , 74-2, the shape of the object, the tactile sensation of the surface such as roughness and roughness, and the hardness of the touched location are presented to the user.

  Note that the haptic control device 80 has output means such as a display as shown in FIG. 6, and a virtual object corresponding to the measurement target object 21 that is a real object as three-dimensional object presentation data on the screen. 81, and a virtual hand 82 that operates based on finger position information obtained from the sensory transmission devices 70-1 and 70-2 can also be displayed on the screen. Thereby, the user can grasp | ascertain exactly the movement condition of the hand in virtual space through vision.

  Note that the virtual object displayed on the screen is not limited to the virtual object 81 such as a cylinder as described above, and includes, for example, a wall, a ceiling, a ground, and the like in the virtual space. At this time, the virtual object is arranged in the virtual space so that it can be fixed or moved freely by the user. When presenting only to one finger or stylus, etc., in order to accurately grasp the hardness, shape, surface roughness, in the virtual space, or in the case of an object that is not fixed, It is possible to grasp the hardness, shape, and surface roughness by presenting on two or more fingers or the like and grasping the virtual object.

  On the haptic sense presentation side, haptic sensation is presented to the user's fingers and the like based on the haptic sense information. Specifically, it is presented by a sensory transmission device such as a contact type using the tension of a wire, a non-contact type using an ultrasonic wave, or the like using a pseudo traction force by an illusion.

<Tactile force sense presentation processing procedure in this embodiment>
Next, the tactile sensation presentation process procedure in the present embodiment described above will be described using a flowchart. FIG. 7 is a flowchart illustrating an example of a tactile sensation presentation process procedure according to the present embodiment.

  In FIG. 7, it is first determined whether or not there is a real object to be measured (S11). If there is a real object to be measured (YES in S11), the three-dimensional shape measuring means is controlled as described above. The three-dimensional shape of the real object is acquired and recorded (S12).

  Next, the surface hardness measuring device is controlled to acquire and record the surface hardness information of the real object (S13). Further, the recorded hardness information is read out, and mapping (association) is performed on the previously acquired minimum three-dimensional configuration element (polygon or the like) (S14). Next, the surface roughness measuring device is controlled to acquire and record the above-described surface roughness information of the real object (S15). Further, the recorded surface roughness information is read out, and mapping (association) is performed on the above-described three-dimensional minimum component, and virtual object data having hardness and roughness information is generated for each element (S16). ). A specific example of the process of S16 will be described later.

  Further, in the above-described processing of S11, when there is no real object to be measured (NO in S11), the user or the like arbitrarily sets on the computer to create three-dimensional shape information (S17), surface hardness information (S18) and surface roughness information (S19). That is, in the above-described processing, not only when there is a real object to be measured, but also when there is no real object, three-dimensional shape information creation, surface hardness information creation, and surface roughness information creation by user settings Is done. In addition, when user setting is performed, virtual object data having hardness and roughness information for each element may be directly generated without mapping, and the virtual object data may be generated by performing the process of S16 as described above. May be generated.

  Information obtained by mapping the hardness information and the surface roughness information to the three-dimensional shape information by the above processing is recorded as tactile force information (S20). Next, the recorded haptic information is read (S21), converted into a preset transmission format, and transmitted (S22).

  Next, the presentation device receives the haptic information described above (S23), and constructs a virtual environment to be provided to the user or the like in correspondence with the received haptic information (S24). Next, finger information including the position and orientation of the user's hand and fingers is acquired (S25), and the position and orientation are analyzed from the acquired finger information (S26). Further, the three-dimensional position coordinates are analyzed (S27), and the presentation force is calculated based on the transmitted tactile force sense information and the movement information of the finger of the user or the like (S28). Next, the obtained presentation power is output to the device (sensory transmission device 50) (S29).

  Next, it is determined whether or not the user ends the virtual tactile sensation presentation process based on the hand motion (S30). If the process is not finished (NO in S30), the process returns to S25 to continue the process. If the process is finished (YES in S30), the virtual haptic sense presentation process is finished.

  Note that the processing order in the flowchart described above is not limited to this in the present embodiment, and for example, a plurality of processes may be performed simultaneously in parallel.

<Example of description of virtual object data having hardness and roughness information for each element>
Here, a description example (mapping) of virtual object data having surface hardness and roughness information in the present embodiment will be described with reference to the drawings. FIG. 8 is a diagram illustrating a description example of virtual object data having hardness and roughness information. FIG. 8A shows VRML data of the described virtual object, and FIG. 8B shows an example of generation of the virtual object 81 and the polygon as the target.

  The surface of the virtual object 81 can be expressed by a minimum component such as a polygon. In FIG. 8, for example, P <b> 0 to P <b> 5 indicate vertices that constitute a part of the virtual object surface, and S <b> 0 to S <b> 3 indicate each surface that constitutes a part of the virtual object surface. Further, D1 shown in FIG. 8A indicates the three-dimensional coordinates of the vertices constituting the virtual object surface, and here, examples of the coordinate values of x, y, z in P1 are shown. D2 indicates a minimum configuration surface constituting the virtual object surface, and here, an example in which a set of vertices constituting S1 is P1, P4, and P2.

  D3 represents a function and a numerical value definition for expressing or calculating hardness information in each minimum component, and here, a calculation formula including a mass element, a damper element, and a spring element, respectively, with the hardness as viscoelastic impedance. The example which defines is shown. Further, D4 shows an example in which the coefficient in the calculation formula defined in D3 and the measured numerical data are described for each minimum component of the virtual object. Here, a set of values of the mass element m and the spring constant k is shown. Shows a description example of the hardness information on the surface S1.

  Similarly, D5 indicates a description example of the amplitude and spread degree (wavelength) of roughness information that is fine irregularities on the surface for each minimum component of the object surface. Here, the amplitude and wavelength of the fine irregularities on the surface S3. An example is shown. Also, in the roughness information, a function or the like for calculation in the data description may be defined similarly to the hardness information. Furthermore, a data string of optical patterns having fine irregularities and roughness actually acquired or a call number of the data string may be described and associated. Thereby, even if it is surface acquisition information that cannot be simply quantified, it can be pasted as it is or external data can be referred to.

  In this manner, virtual object data having hardness and roughness information can be described for each element by extending the description such as D3 to D4 to the existing description format (D1, D2) such as VRML.

<Execution program (surface hardness measurement program, tactile force presentation program)>
In the above-described embodiment, the above-described surface hardness measurement processing and haptic sense presentation processing procedure according to the present invention can be performed by a dedicated device configuration in the above-described surface hardness measurement device and haptic sense presentation device. However, it generates an execution program (tactile sensation presentation program) that can cause a computer to execute each of the above-described surface hardness measurement processing and haptic sense presentation processing, for example, in a general-purpose personal computer or workstation By installing the surface hardness measurement program and the haptic sense presentation program, the surface hardness measurement process and the haptic sense presentation process in the present invention can be realized.

  That is, the surface hardness measurement device and the tactile sensation presentation device described above are non-volatile such as a volatile storage medium such as a CPU (Central Processing Unit) and a RAM (Random Access Memory), a ROM (Read Only Memory), and the like. Storage devices, input devices such as a mouse, keyboard, and pointing device, a display unit for displaying images and data, and a computer having an interface for communicating with the outside.

  Accordingly, the above-described functions of the surface hardness measurement device and the haptic sense presentation device can be realized by causing the CPU to execute a program describing these functions. These programs can also be stored and distributed in a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), semiconductor memory, or the like.

  In other words, an execution program (surface hardness measurement program, tactile sensation presentation program) for causing a computer to execute the processes in the above-described configurations is generated, and the program is installed in, for example, a general-purpose personal computer or server Thus, the computer can function as the surface hardness measurement processing device and the tactile sensation presentation device in the above-described embodiment, and thereby, for example, the surface hardness measurement processing and touch as shown in FIG. 2 and FIG. A force sense presentation process can be realized.

  As described above, according to the present invention, the hardness information of a real object can be acquired with high accuracy without contact. Thereby, when presenting an object virtually, a tactile sensation close to a real object can be presented with high accuracy. Therefore, not only the three-dimensional shape of the object, but also the more realistic tactile sensation that the real object has, such as surface hardness and surface roughness, is acquired and transmitted, and virtually presented to multiple users. be able to.

  Specifically, in the present invention, means for measuring the displacement of the surface, means for estimating the viscoelastic impedance of the surface, means for extracting and mapping the hardness information, and controlling the force sense according to the hardness information Tactile sensation presentation means. The present invention also has means for acquiring the three-dimensional shape of the object, means for mixing with the hardness information of the object, and presents the shape and hardness of the object together.

  Furthermore, the present invention has means for acquiring fine irregularities and roughness of the surface of the object, and has means for mixing the shape, hardness and surface roughness, and presents them to the sense of touch. Thus, according to the present invention, it is possible to provide a tactile force sense presentation device and a presentation method that can obtain a more natural tactile sensation such as touching a real object.

  Although the preferred embodiment of the present invention has been described in detail above, the present invention is not limited to the specific embodiment, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed.

DESCRIPTION OF SYMBOLS 10,61 Surface hardness measuring apparatus 11 Ultrasonic transducer array 12 Vibration measuring means 13, 31 Three-dimensional shape measuring means 14, 64 Measurement control means 15, 65 Recording means 21 Object to be measured 22 Ultrasonic wave 32 Three-dimensional shape recording Means 33 Hardness information measurement position calculation means 34 Pressure application position control means 35 Vibration amplitude measurement position control means 36 Vibration amplitude measurement means 37 Applied pressure calculation means 38 Pressure application cycle control means 39 Pressure application means 40 Hardness information calculation means 41 Hard Information recording means 42 Hardness information transmission means 43 Tactile sensation presentation means 44 Normal incidence surface application force data 51 Array element 60 Tactile sensation presentation apparatus 62 Three-dimensional shape measurement apparatus 63 Surface roughness measurement apparatus 66 Transmission means 70 Sensory transmission Device 71 Base 72 Drive unit 73 Link mechanism 74 Fingertip transmission unit 80 Tactile force sense control device 81 Virtual object 82 virtual hand

Claims (6)

In a surface hardness measurement device for measuring hardness information of a real object corresponding to a virtual object in a virtual space to be presented to the user with a tactile sensation,
Three-dimensional shape measuring means for measuring the three-dimensional shape of the real object;
A period when the ultrasonic transducer array changes a plurality of ultrasonic waves at a predetermined period and irradiates a predetermined position on the surface of the real object based on the three-dimensional shape information obtained by the three-dimensional shape measuring means. A displacement measuring means for measuring the vibration amplitude of the surface caused by a change in a general radiation pressure in a non-contact manner, and measuring a displacement ratio or a displacement amount at the position from the measured vibration amplitude of the surface ;
The viscoelastic impedance at each measured position of the real object is estimated from the displacement ratio or the displacement amount obtained by the displacement measuring means, and the hardness information at each predetermined position of the real object is obtained from the estimated viscoelastic impedance. A surface hardness measuring device. In the tactile sensation presentation device provided with the surface hardness measurement device according to claim 1 ,
Force control by the hardness information by mapping the hardness information for each predetermined position of the real object by the viscoelastic impedance obtained by the measurement control means to a virtual object in a virtual space corresponding to the real object A tactile sensation presentation device comprising tactile sensation presentation means for performing the above. The tactile sensation presentation means includes
3. The tactile force sense presentation device according to claim 2 , wherein force sense control is performed on the virtual object in correspondence with the three-dimensional shape information obtained by the three-dimensional shape measurement unit and the hardness information. Surface roughness measuring means for obtaining a surface shape including information on the surface roughness and roughness of the real object,
The haptic presentation unit, according to claim 2 or 3, characterized in that the force control with respect to the virtual object in correspondence with the surface roughness information obtained and said hardness information by the surface roughness measuring device The tactile sensation presentation device according to claim 1. A surface hardness measurement program for causing a computer to function as the surface hardness measurement device according to claim 1 . A haptic sense presentation program for causing a computer to function as the haptic sense presentation device according to any one of claims 2 to 4 .

Images