WO2017175867A1 - 触覚情報変換装置、触覚情報変換方法、および、触覚情報変換プログラム - Google Patents
触覚情報変換装置、触覚情報変換方法、および、触覚情報変換プログラム Download PDFInfo
- Publication number
- WO2017175867A1 WO2017175867A1 PCT/JP2017/014570 JP2017014570W WO2017175867A1 WO 2017175867 A1 WO2017175867 A1 WO 2017175867A1 JP 2017014570 W JP2017014570 W JP 2017014570W WO 2017175867 A1 WO2017175867 A1 WO 2017175867A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- tactile
- sensation
- vibration
- unit
- tactile information
- Prior art date
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J13/00—Controls for manipulators
- B25J13/02—Hand grip control means
- B25J13/025—Hand grip control means comprising haptic means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J13/00—Controls for manipulators
- B25J13/08—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices
- B25J13/081—Touching devices, e.g. pressure-sensitive
- B25J13/084—Tactile sensors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
- G06F3/03543—Mice or pucks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
- G06F3/03547—Touch pads, in which fingers can move on a surface
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
- G06F3/0383—Signal control means within the pointing device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/0414—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using force sensing means to determine a position
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03K—PULSE TECHNIQUE
- H03K17/00—Electronic switching or gating, i.e. not by contact-making and –breaking
- H03K17/94—Electronic switching or gating, i.e. not by contact-making and –breaking characterised by the way in which the control signals are generated
- H03K17/96—Touch switches
- H03K17/9618—Touch switches using a plurality of detectors, e.g. keyboard
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03K—PULSE TECHNIQUE
- H03K2217/00—Indexing scheme related to electronic switching or gating, i.e. not by contact-making or -breaking covered by H03K17/00
- H03K2217/94—Indexing scheme related to electronic switching or gating, i.e. not by contact-making or -breaking covered by H03K17/00 characterised by the way in which the control signal is generated
- H03K2217/96—Touch switches
- H03K2217/96062—Touch switches with tactile or haptic feedback
Definitions
- the present invention relates to a haptic information conversion device, a haptic information conversion method, and a haptic information conversion program.
- Patent Document 1 a pen-type force sense presentation device that moves a fingertip by moving a reciprocating motion, a peristaltic motion, a tilting motion, a rotating motion, etc., in a movable portion that is touched by a fingertip when gripped by a user. Presenting proprioceptive sensation is disclosed.
- Non-Patent Document 1 discloses a tactile sensation presentation apparatus that creates a tactile map using pressure, vibration, and temperature as element tactile sensations and synthesizes these element tactile sensations to present an arbitrary tactile sensation.
- Non-Patent Document 2 it is disclosed that a virtual uneven feeling is presented by vibrating a vibrator provided on the nail side during a finger-sliding operation in a real environment.
- the conventional tactile sensation presentation device is an ad hoc technology specialized for presenting each tactile sensation, and there is a problem that an arbitrary tactile sensation expressed by, for example, onomatopoeia cannot be selected and output. there were.
- Patent Document 1 has a problem that although it is possible to present proper receptive sensations by movements such as reciprocation, peristalsis, tilting, and rotation, these combinations cannot present various skin sensations and tactile sensations. .
- Non-Patent Document 1 an attempt is made to create a tactile map with three axes of pressure, temperature, and vibration to present an arbitrary tactile sensation. There was a problem that it was impossible in principle.
- Non-Patent Document 2 by applying a vibration stimulus from above the nail when tracing an object with a finger, an appropriate impulse stimulus can be generated on the finger pad to reproduce the uneven edge perception.
- an appropriate impulse stimulus can be generated on the finger pad to reproduce the uneven edge perception.
- various tactile sensations cannot be presented.
- the present invention has been made in view of the above problems, and provides a tactile information conversion device, a tactile information conversion method, and a tactile information conversion program that can be used for general purposes by presenting or sensing an arbitrary tactile sensation. It is to provide.
- the haptic information conversion device of the present invention provides haptic information to an output unit side that can output physical quantities including at least electricity and including force, temperature, vibration, and / or spatiotemporal space. Therefore, in the tactile information conversion device including at least a control unit, the control unit selects a plurality of physical quantities of at least two or more of the physical quantities according to the tactile sensation to be presented and selected A creation unit that creates haptic information for presenting the predetermined tactile sensation based on a physical quantity, and an output control unit that outputs the haptic information created by the creation unit to the output unit side It is characterized by.
- the tactile information conversion device of the present invention is characterized in that, in the tactile information conversion device, the electrical output of the physical quantity is an electrical stimulus presentation of a tactile receptor.
- the tactile information conversion device of the present invention is characterized in that, in the tactile information conversion device, the tactile sensation is a psychological texture.
- the creation unit when the creation unit presents a tactile sensation due to a temporal change of force, at least the physical quantities of electricity and space-time are selected and selected. Based on the physical quantity, tactile information for presenting the predetermined tactile sensation is created.
- the tactile information conversion device of the present invention is the above-described tactile information conversion device, wherein the creating unit is more powerful than the case of a hard surface in a transition process from a non-contact state to a contact state or a body displacement process, The tactile information is created so that a stimulus of force or vibration, or an electrical, force, or vibration stimulus of a larger area than that of a hard surface is given, and the output control unit is created by the creation unit Further, a soft psychological texture is presented based on the tactile information.
- the tactile information conversion device of the present invention is the above-described tactile information conversion device, wherein the creating unit is more powerful than the case of a hard surface in a transition process from a contact state to a non-contact state or a body displacement process, The tactile information is created so that a stimulus of force or vibration, or an electrical, force or vibration stimulus of a larger area than that of a hard surface is given, and the output control unit is created by the creation unit A sticky psychological texture is presented based on the tactile information.
- the tactile information conversion device of the present invention is characterized in that, in the tactile information conversion device, the body is a finger.
- the psychological texture is a psychological amount that is integrally recognized in the brain from information obtained by a plurality of different tactile receptors of a human body. It is characterized by that.
- an input device of the present invention is characterized by including the above-described tactile information conversion device.
- the input device of the present invention includes at least a multipoint distributed pressure sensor on the surface layer, a thermal sensation sensor on the intermediate layer, and a vibration sensor on the lower layer in the above input device. It is characterized by that.
- the transmission device of the present invention is characterized by including the tactile information conversion device described above.
- a storage device of the present invention is characterized by including the above-described tactile information conversion device.
- server device of the present invention is characterized by including the above-described tactile information conversion device.
- the receiving device of the present invention is characterized by including the tactile information conversion device described above.
- an output device of the present invention is characterized by comprising the tactile information conversion device described above.
- the output device of the present invention is the above output device, wherein the surface layer has a distributed pressure presentation unit by multi-point electric tactile stimulation, an intermediate layer, a high-speed drive type thermal sensation presentation unit by a Peltier element, and a lower layer. And a vibration presenting unit in a wide frequency range.
- the interaction operation system of the present invention is an interaction operation system that teaches the operation of the robot using the robot teaching device operated by the operator, which includes the above-described tactile information conversion device.
- the robot grips an object.
- An object detection unit for detecting a finger part, an object or a property of the object, and generating object detection information, the object detection unit disposed on the finger part, and a robot drive device for driving the robot
- the robot teaching device provides an object detection sensation that provides a corresponding tactile sensation to the operator according to tactile information converted by the tactile information conversion device based on the object detection information transmitted from the robot. It is characterized by having a part.
- the tactile sensation presentation method of the present invention includes at least a control unit for providing tactile information to an output unit side that can output physical quantities including at least electricity and including force, temperature, vibration, and / or spatiotemporal space.
- a control unit for providing tactile information to an output unit side that can output physical quantities including at least electricity and including force, temperature, vibration, and / or spatiotemporal space.
- the tactile information conversion program of the present invention is for causing a computer to execute tactile information for an output unit that includes at least electricity and can output physical quantities including force, temperature, vibration, and / or time and space.
- a tactile sensation for selecting a plurality of physical quantities of at least two of the physical quantities according to a tactile sensation to be presented and presenting the predetermined tactile sensation based on the selected physical quantities A creation step for creating information, and an output control step for outputting the haptic information created in the creation step to the output unit are executed by a computer.
- the tactile information conversion device of the present invention provides tactile information at least with a control unit to provide tactile information to an output unit capable of outputting physical quantities including electricity, force, temperature, vibration, and / or spatiotemporal space.
- the control unit selects at least one physical quantity among the physical quantities according to a tactile sensation to be presented, and presents the predetermined tactile sensation based on the selected physical quantity.
- the physical quantity is selected in association with at least two axes of a rough sliding axis by force, a hard axis by vibration, and a wet and dry axis by temperature.
- the creation unit includes the physical quantity detected by the detection unit with respect to a sample with a known tactile sensation, and the tactile sensation, Learning is performed by updating in association with a biaxial map.
- the tactile information conversion device of the present invention is a tactile information conversion device including at least a detection unit and a control unit for obtaining tactile information, and the detection unit includes force, temperature, vibration, and / or time.
- a physical quantity including space can be detected, and the control unit is based on the physical quantity detected by the detection unit, and is based on at least one of a rough sliding shaft due to force, a hard shaft due to vibration, and a wet and dry shaft due to temperature.
- the tactile information for presenting the corresponding tactile sensation is acquired by associating on two axes.
- the control unit associates the physical quantity detected by the detection unit with the tactile sensation for a sample with a known tactile sensation. Learning is performed by updating the at least two-axis map.
- the output unit side that can generate and output tactile information based on physical quantities including at least electricity and including force, temperature, vibration, and / or time and space. Since the generated tactile information is output in order to provide tactile information, an effect is obtained that an arbitrary tactile sensation can be presented and used universally.
- various tactile sensations such as skin sensations that cannot be reproduced by reciprocating or rotating movements by the drive unit, and psychological feelings such as mochi-mochi and stickiness that cannot be reproduced by pressure, temperature, and vibration. Play.
- an output unit that can be output when creating tactile information based on a physical quantity including electricity, force, temperature, vibration, and / or spatiotemporal according to the tactile sensation to be presented.
- a physical quantity including electricity, force, temperature, vibration, and / or spatiotemporal according to the tactile sensation to be presented.
- FIG. 1 is a schematic diagram showing the reaction of each cell when a finger is brought into contact with the object to be slid on the object, or rested after being stopped.
- FIG. 2 is a diagram schematically showing a second method of stimulating the tactile 7 base of the physiological space by synthesizing the three bases of “force”, “vibration”, and “temperature”.
- FIG. 3 is a diagram illustrating an example of a psychological texture map in which various onomatopoeia are positioned on a triaxial map of a roughness axis, a hardness axis, and a wetness axis.
- FIG. 1 is a schematic diagram showing the reaction of each cell when a finger is brought into contact with the object to be slid on the object, or rested after being stopped.
- FIG. 2 is a diagram schematically showing a second method of stimulating the tactile 7 base of the physiological space by synthesizing the three bases of “force”, “vibration”, and “temperature”.
- FIG. 3 is a diagram
- FIG. 4 schematically illustrates a method according to the present embodiment that is synthesized based on the four bases of “force”, “vibration”, “temperature”, and “electricity” in physical space, and reproduces various tactile sensations in psychological space. It is the figure shown in.
- FIG. 5 is a block diagram illustrating an example of a configuration of a tactile sensation presentation system including a haptic information conversion apparatus according to an embodiment of the present invention.
- FIG. 6 is a perspective view showing the structure of an integrated tactile measurement module based on the tactile primary color principle.
- FIG. 7 is a perspective view showing the structure of an integrated tactile transmission module based on the tactile primary color principle.
- FIG. 8 is a diagram showing perceptual points based on the phantom sensation illusion.
- FIG. 9 is a diagram showing the appearance of the finger-mounted sensor / actuator of the telexistance robot called the above-mentioned TELESAR V developed by the present inventors.
- FIG. 10 is a diagram showing an external appearance of the tele-distance robot called TELESAR V developed by the present inventors when the finger-mounted sensor / actuator is mounted.
- FIG. 11 is a flowchart showing an example of tactile information conversion processing in the tactile sensation presentation system of the present embodiment.
- FIG. 12 is a flowchart illustrating an example of multipoint stimulation processing in the tactile information conversion device 100 of the tactile sensation presentation system of the present embodiment.
- FIG. 13 is a flowchart illustrating an example of a remote tactile sensation transmission process in the tactile sensation presentation system of this embodiment.
- FIG. 14 classifies input information of sensors, etc., into tactile sensation category information based on a category information conversion table obtained by machine learning, etc., and synthesizes and presents a plurality of physical quantities based on the classified tactile sensation category information. It is a figure which shows the function conceptual structure in the case of outputting.
- FIG. 15 is a diagram schematically showing changes in the contact surface and pressure when the hard surface, soft surface, and viscous surface are touched.
- FIG. 16 is a diagram schematically showing changes in the contact surface and pressure when large deformation occurs when a hard surface, soft surface, or viscous surface is touched.
- FIG. 17 is a diagram illustrating changes in the contact area of the hard surface, the soft surface, and the viscous surface with respect to the amount of displacement of the finger.
- FIG. 15 is a diagram schematically showing changes in the contact surface and pressure when the hard surface, soft surface, and viscous surface are touched.
- FIG. 16 is a diagram schematically showing changes in the contact surface and pressure when large deformation occurs
- FIG. 18 is a diagram schematically illustrating the configuration of the experimental apparatus used in Experiment 1 of Example 1 from the fingertip side.
- FIG. 19 is a photographic diagram showing a state before the wearing of the experimental device used in Experiment 1 of Example 1 on the finger and a state after the wearing.
- FIG. 20 is a diagram illustrating experimental conditions (Press conditions) used to present a soft feeling in Experiment 1 of the first embodiment.
- FIG. 21 is a diagram illustrating experimental conditions (Release conditions) used to present a sticky feeling in Experiment 1 of the first embodiment.
- FIG. 22 is a diagram illustrating a subjective evaluation experiment result according to Experiment 1 of the first embodiment.
- FIG. 23 is a photograph showing a state in which the experimental device used in Experiment 2 of Example 1 is worn on the finger.
- FIG. 24 is a diagram illustrating experimental conditions (press conditions) of the experiment 2 of the first embodiment.
- FIG. 25 is a diagram illustrating experimental conditions (Release conditions) of Experimental Example 2 of the first embodiment.
- FIG. 26 is a diagram illustrating a subjective evaluation experiment result according to Experiment 2 of the first embodiment.
- FIG. 27 is a diagram illustrating a configuration example of a high-density presentation device in which an electrical stimulation matrix and a mechanical stimulation matrix are combined.
- FIG. 28 is a diagram showing a range of the first embodiment (axis 1) in the above-described onomatopoeia map (FIG. 3).
- FIG. 29 is a diagram illustrating a range of tactile sensation expressed by a horizontal resistance force.
- FIG. 30 is a diagram showing a device used in the experiment.
- FIG. 31 is a diagram illustrating the results of a reproduction experiment of “feeling rough”, “feeling rough”, and “feeling bulky”.
- FIG. 32 is a diagram illustrating a range of tactile expression that is reproduced by resistance force and temperature.
- FIG. 33 is a diagram illustrating a device and a vibration waveform used in the experiment.
- FIG. 34 is a diagram showing the results of a reproduction experiment of “smooth”, “smooth”, “smooth”, and “slimy”.
- FIG. 31 is a diagram illustrating the results of a reproduction experiment of “feeling rough”, “feeling rough”, and “feeling bulky”.
- FIG. 32 is a diagram illustrating a range of tactile expression that is reproduced by resistance force and temperature.
- FIG. 33 is a diagram illustrating a device and a vibration waveform used in the experiment.
- FIG. 34 is a diagram showing the results of a reproduction experiment of “smooth”, “smooth”, “smooth”, and “slimy”.
- FIG. 35 is a diagram showing an onomatopoeia map showing a psychological space and a physical space stimulus map represented by three axes of force, vibration, and temperature.
- FIG. 36 is a diagram illustrating a configuration of an experimental apparatus (Peltier element) according to the second embodiment.
- FIG. 37 is a diagram illustrating an experimental environment according to the second embodiment and a mounting example of a Peltier element.
- FIG. 38 is a diagram schematically illustrating a position to apply a temperature stimulus when a finger is viewed from the fingertip side.
- FIG. 39 is a graph showing the experimental results of the second embodiment.
- FIG. 40 is a diagram illustrating an experimental apparatus of Experiment 2 of the second embodiment.
- FIG. 41 is a diagram schematically illustrating a position where a temperature stimulus (Thermal stimulus) is applied when a finger is viewed from the fingertip side.
- FIG. 42 is a graph showing the experimental results of Experiment 2 of Example 2 in the case of warm stimulation.
- FIG. 43 is a graph showing the experimental results of Experiment 2 of Example 2 in the case of cold stimulation.
- FIG. 44 is a diagram showing a shift in temperature sensation with respect to contact in this experiment 3 (a. Presence / absence of contact + temperature stimulation).
- FIG. 45 is a diagram showing a deviation in temperature sensation with respect to vibration in this experiment 3 (b. Presence / absence of vibration stimulation + temperature stimulation).
- a haptic information conversion device a haptic information conversion method, a haptic information conversion program, and a recording medium according to an embodiment of the present invention
- this invention is not limited by this embodiment.
- an example in which the function of the tactile information conversion device according to the present invention is connected to an input unit such as a sensor or an output unit that outputs a tactile stimulus may be described. Is not limited to this, and may be configured as a function of an independent server device that is not directly connected to the input unit or the output unit, for example.
- the present invention may be configured as a part of an input device, a transmission device, a storage device, a reception device, or an output device, or as a conversion device installed between these devices. .
- a special sensation refers to a sensation in which a corresponding special sensory organ exists, such as an eyeball for vision and an ear for hearing.
- the sense of “acceleration” is classified as a special sense in the sense that it corresponds to the ear, in particular the semicircular canal, which is the vestibule in the ear, and the sensory organs of the oval and spherical folds.
- somatic sensation means somatosensory sensation, which is roughly divided into skin sensation derived from skin (cutaneous sensation) and posture and movement derived from internal muscles and tendons. It is divided into sensation, proprioception. The proprioceptive sensation is also called self-acceptance sensation.
- tactile sensation means the whole somatic sensation of skin sensation and proprioceptive sensation
- tactile sensation in this embodiment indicates a broad sense of tactile sense.
- “tactile sensation” essentially means only contact sensation and pressure sensation among skin sensations including various sensations such as warm, cold and pain. This sense of contact and pressure corresponds to sensory organs such as Merkel cells, Meissner bodies, Patini bodies and Rufini terminals in the skin. And when the whole skin is dented or pulled, the deformation
- tactile sensation in a broad sense can be said to be a total of a wide range of sensations including not only a simple sensation corresponding to one sensory organ but whether or not it is touching, as well as a proprioceptive sensation.
- a process will be described in which a human recognizes an object, for example, an iron ball, with a broad sense of touch.
- the human When you first touch, the human knows the shape, but if you do not touch it directly with your finger, put on a thick glove on your finger and touch the iron ball through it, the arm, hand, and finger joints It can be inferred as a “sphere” from the information about how it has moved and what form it has.
- rough shape recognition is not based on the skin, but on the proper receptive sensations such as muscle spindles and Golgi receptors.
- the sense of hardness and repulsion like a spring, and the sense of resistance when moving the arm in the water, etc. are derived from the proprioceptive sensation.
- the skin sensation recognizes a fine shape pattern on the surface called a finer “texture” (texture). This sensation does not occur with thick gloves, but it is important to touch it directly with the skin. This sense is further improved in recognition accuracy by the active tactile sensation described above.
- the tactile sensation accompanied by the movement in which the proper reception and the skin sensation are integrated is called haptics (haptic perception, haptics). It is not possible to discriminate it from iron by just touching it, but the fact that it can be recognized as metal is largely due to the skin sensation that feels heat and cold in addition to texture.
- the tactile sensation may particularly refer to a skin sensation, but is not limited to this and may include a proper receptive sensation.
- FIG. 1 is a schematic diagram showing the reaction of each cell when the finger is brought into contact with the object to slide on the object, or after resting and moving away.
- the horizontal axis represents time
- the vertical axis represents the excited state for each cell.
- a real object is not limited to a flat surface as shown in the figure, and has many physical characteristics such as uneven shape, friction, heat, and elasticity. Considering the situation in which skin sensation occurs, when an object is touched, force, vibration, temperature change and the like are generated on the skin surface, which are transmitted to the inside of the skin, and the above-mentioned sensory apparatus reacts to generate a tactile sensation.
- the inventors of the present application can actually react to each sensory device regardless of the physical characteristics such as the uneven shape of the object, friction, heat, and elasticity, as long as the object is touched. I thought that the same tactile sensation would occur in humans as if they were touching an object. In other words, if tactile sensation can react with cells corresponding to the primary color as well as vision, it is not necessary to reproduce and present the real object, and only the tactile sensation can be reproduced and used as an information medium similar to vision. The inventors of the present application thought that it was possible.
- the first method is a method of obtaining a base in a physiological space and selectively firing the underlying sensory organ itself by electrical stimulation.
- base means a structural unit or an element. Since the invasive method of embedding and stimulating electrodes in all sensory organs is impractical, it stimulates sensory organs (eg tactile receptors) selectively with transcutaneous electrical stimulation from the skin surface. A method is conceivable (for example, see Japanese Patent No. 3543097 by the present inventor). This method was developed by the inventor of the present application.
- Meissner bodies can be selectively stimulated by anodic stimulation, and Merkel cells are stimulated through skin electrodes by cathodic stimulation, resembling pressure sensation. It is possible to convey a sense of feeling.
- the first method has not yet been applied as a general-purpose stimulus as a method of directly and selectively stimulating cells serving as the basis of physiological space.
- the second method is a method of selecting a base in a physical space by a method equivalent to a method based on visual RGB.
- FIG. 2 is a diagram schematically showing a second method of stimulating the tactile 7 base of the physiological space by synthesizing the three bases of “force”, “vibration”, and “temperature”.
- Merkel cells and Rufini endings are pressure and shear forces, Meissner bodies are low-frequency vibrations, Patini bodies are high-frequency vibrations, and free nerve endings respond to temperature, cold, and pain.
- the inventors of the present invention considered that the method based on “force”, “vibration”, and “temperature” still has problems for miniaturization, and further investigation is necessary. . That is, in order to present “force”, it is necessary to prepare a fixed place (grounding) for applying a force at a place different from the point where the force is applied because of the characteristic of the physical quantity called force. For example, in order to apply force to the finger pad, it is necessary to ground the part such as the back of the finger and apply force from there. In this case, when the presentation device itself is produced, the device cannot be accommodated only in the vicinity of the stimulation site. Therefore, a relatively large space is inevitably required, which is a great restriction when the device is downsized.
- FIG. 3 is a diagram illustrating an example of a psychological texture map in which various onomatopoeia are positioned on a three-axis map of a roughness axis, a hardness axis, and a wetness axis.
- FIG. 4 shows a method of the present embodiment in which various tactile sensations in the psychological space are reproduced by synthesizing based on the four bases of “force”, “vibration”, “temperature”, and “electricity” in the physical space.
- all “force” may be replaced with “electricity”, and the participation of “force” of the four bases may be zero, and synthesis may be performed based on the three bases.
- the present embodiment is not limited to this, and spatiotemporal elements of “time” and / or “space” may be added to the base. Electricity is the basis of physical space because it is a definite physical quantity, except for special cases where it feels static electricity when it comes into contact with an actual normal object.
- this electrical stimulation generates pressure sensation, vibration sensation, and even pain sensation
- the major feature of this embodiment is that it has an effect on the tactile sensation of the five bases in the psychological space depending on how the electrical stimulation is applied. That is, instead of an approach that stimulates the tactile sensation of the 7th basis of the physiological space based on the basis of the physical space, the present embodiment synthesizes a plurality of bases of the physical space including electricity, and a variety based on the 5th basis of the psychological space. It is an approach that reproduces the tactile sensation.
- a psychology expressed by onomatopoeia It is possible to reproduce the texture, in other words, the psychological quantity that is integratedly recognized in the brain from information obtained from a plurality of different tactile receptors on the human body. This makes it possible to present a tactile sensation that works on tactile sensation based on multimodal (multi-sensory format), unlike a conventional method of simply stimulating tactile sensation. A specific method for presenting the psychological texture expressed by onomatopoeia will be described later.
- the present invention it is possible to greatly widen the range of sensations that can be presented only with conventional force, vibration, and temperature, particularly by giving a sense of hardness, dryness, and so on.
- the force can be replaced by electricity, so that a miniaturized multimodal device can be made.
- One of the major features of the present embodiment is that the tactile sensation is presented based on a plurality of bases of the physical space including electricity.
- electricity is not necessarily used.
- electricity, force, temperature, vibration, and / or time-space eg, electricity, force, temperature, time of vibration and / or vibration, depending on the tactile sensation to be presented.
- the tactile information is given to the output unit that can be output.
- a physical quantity is selected in association with at least two of the hard and soft axes and the wet and dry axes according to temperature. Thereby, for example, various arbitrary psychotactile sensations on the onomatopoeia map can be reproduced.
- FIG. 5 is a block diagram showing an example of a configuration of a tactile sensation presentation system including a haptic information conversion apparatus according to an embodiment of the present invention, and conceptually shows only a part related to the present invention in the configuration.
- the tactile sensation presentation system includes a tactile information conversion device 100, an external input device 120 such as various sensors, an external output device 140 that can output a physical quantity, and an external device such as a server. 200 are connected via a network 300.
- a tactile information conversion device 100 an external input device 120 such as various sensors
- an external output device 140 that can output a physical quantity
- an external device such as a server. 200
- the tactile sensation presentation system may include a plurality of devices.
- the tactile information conversion device 100 is a personal computer, a server computer, a portable information terminal (tablet computer or the like), and the like.
- the network 300 has a function of mutually connecting the haptic information conversion device 100, the external input device 120, the external output device 140, and the external device 200, and is, for example, a wired or wireless LAN or the Internet.
- the external input device 120 is input means such as various sensors.
- a force sensor, a vibration sensor, or a temperature sensor may be used, and further, an unevenness sensor, a surface roughness sensor, a sticking sensor, a friction sensor, a wet sensor, a heat conduction sensor, a viscoelastic sensor, an acceleration sensor, and the like. May be.
- the object has physical characteristics such as uneven shape, surface roughness, friction, thermal conductivity, viscosity, elasticity, inertia, and wetness.
- the human skin sensory organs are basically concerned with the force (skin deformation) and vibration generated by the human hand when the human touches the target object by moving his / her hand.
- the optimum sensors are force sensors, vibration sensors, and temperature sensors that directly sense them.
- the robot hand may not always have those sensors. In that case, for example, conversion from an acceleration sensor to force information or vibration information, conversion from a surface roughness sensor to vibration information, or calculation by a combination of various sensors converts to force, vibration, or temperature. You may do that. It is optimal that these conversions are performed on the sensor side and communication is standard, but communication may be performed as it is, and tactile information conversion may be performed.
- the external input device 120 may be a recognition device that recognizes a user's movement or object, such as a camera, a touch panel, or a motion sensor. Specifically, the external input device 120 may recognize the movement of a person's body by an arbitrary detection unit such as a camera or a pressure sensor. For example, the external input device 120 may detect the movement of the user's body using a known gesture recognition technique, a known motion sensor, or the like. Gestures can be obtained from the user's position and movement in physical space, and can include any user movement, dynamic or static, such as arm or leg movement, or static posture.
- a capture device such as a camera captures user image data, and recognizes a user's gesture (s) from the user image data.
- the external input device 120 recognizes and analyzes gestures performed by the user in the user's three-dimensional physical space using the computer environment, interprets the user's motion data before analysis, Raw data or the like may be transmitted to the haptic information converter 100.
- the external input device 120 may be a wearable sensor that detects a finger shape, body posture, contact location, contact area, pressure, vibration, and the like.
- a sensing technology such as a globe sensor of a TELESAR system, which is a tele-distance system that can transmit a reaction force and temperature to a fingertip developed by the present inventors, may be used.
- the external input device 120 may include a plurality of two-dimensionally arranged force detection sensors, a vibration source and vibration detection sensor, a temperature sensor, a timer, and the like. Good.
- a Microsoft Kinect sensor, an Intel RealSense sensor, or the like may be used. According to these known sensing techniques, motion data such as skeleton motion data and contact data of the whole body and fingers can be obtained.
- the movement of the person is analyzed using the control means built in the sensor, or the movement and attributes of the person are analyzed by the control means of the connected computer.
- these analysis functions may be realized by the control means (processor or the like) of the external input device 120, or by the control means of the haptic information conversion device 100 that has received the raw data before analysis. It may be realized, or the analysis function may be realized by being distributed by both control means.
- a known film pressure sensor described in Non-Patent Document 2 or the like may be used.
- the external input device 120 may be used for recognizing an object from an image captured by a camera or the like and presenting a tactile sensation according to the recognized object. It may be used to present tactile stimuli at an appropriate timing according to the body posture and movement.
- the external input device 120 detects physical quantities such as forces, vibrations, temperatures, spatiotemporal changes such as temporal changes and spatial distributions thereof, tactile sensations or tactile sensations themselves, and detects the detected physical quantities, tactile sensations or It may be used to present the tactile sensation recognized based on the learning result from the detected tactile information, and the error with the sensory amount is complemented so that the actual tactile sensation is presented based on the detected tactile sense or tactile sense. May be used as basic data.
- FIG. 6 is a perspective view showing the structure of an integrated tactile measurement module based on the tactile primary color principle.
- FIG. 7 is a perspective view showing the structure of an integrated tactile transmission module based on the tactile primary color principle.
- tactile sensation is pressure / shearing force (perceived by Merkel cell / Ruffini terminal) / low frequency vibration sense (perception by Meissner body) / high frequency vibration sense (perception by Patini body) / cool sense / warmth sense / It is considered as a synthesis from the spatiotemporal relationship between each element of pain perception (each perceived by free nerve endings) and realizes the decomposition and synthesis of tactile sense.
- the inventors of the present application have developed a module that can acquire and present each element of force, vibration sense, and cold / hot sense based on the principle of the tactile color. Each of them is 24 mm long, 12 mm wide, and 6 mm high, and is suitable for contacting the finger pad.
- a 32-point distributed pressure measurement sensor 120a is arranged on the surface layer, and a thermal sensation measurement sensor 120b having a body temperature presentation function in the lower intermediate layer.
- the vibration feeling measurement sensor 120c in a wide frequency range is arranged in a further lower layer.
- a distributed pressure presentation unit 140a by 32-point electrotactile stimulation is arranged on the surface layer, and four Peltier elements are arranged in the lower intermediate layer.
- a high-speed drive type thermal sensation presentation unit 140b using a matrix is provided.
- a vibration presenting unit 140c in a wide frequency range (HiFi) is arranged in a further lower layer.
- a pressure sensation and a low frequency vibration sensation can be freely generated without physical stimulation. It is shown that the pressure sense / low frequency vibration sense can be selectively stimulated by changing the stimulus pattern.
- the stimulation electrode a thin and high-density electrotactile display using a flexible printed board is used.
- the integrated tactile measurement module of this embodiment uses a flexible substrate and a thin pressure sensor that enables multipoint measurement corresponding to the electric tactile display of the integrated tactile presentation module, and uses a high-density, multi-point pressure sensor.
- a thin sensor and display that acquire and present That is, the distributed pressure measurement sensor 120a and the distributed pressure presentation unit 140a can realize highly accurate pressure presentation and pressure measurement of 32 points in 4 rows and 8 columns at intervals of 3 mm or less.
- this module is applicable not only to a fingertip but to any part of a human body surface by miniaturizing and integrating.
- since the encoding is based on the principle of touch primary colors, general-purpose use is possible.
- the vibration presentation unit 140c of the integrated tactile sensation presentation module can provide a vibration sensation with flat characteristics in a wide vibration frequency band of 1 Hz to 1000 Hz in order to realize a vibration actuator optimal for vibration sensation presentation.
- a small vibration actuator is used.
- the thermal sensation measurement sensor 120b and the thermal sensation presentation unit 140b have a bidirectional function in the acquisition and presentation of cold sensation, that is, the sensor that acquires the cold sensation functions as a display that reproduces the body temperature equivalent to that of a human. Have.
- conventional cold / warmth transmission it is considered that the time response is low, but by utilizing human cold / warmth perception characteristics, a plurality of temperature presentation elements are arranged and controlled in a matrix to be about 1 Hz.
- the external output device 140 is not limited to the integrated tactile sense presentation module described above, and the external output device 140 is an output unit capable of outputting physical quantities including electricity, force, temperature, vibration, time and space.
- the external output device 140 may be an output device capable of outputting spatially and spatially by arranging electric stimulators, force presentation actuators, Peltier elements, vibration elements, and the like in a matrix.
- the vibrator may be a voice coil vibrator, a piezo element, or a vibration motor.
- the external output device 140 may output the above-described physical quantity using a known electrical output unit, force presentation unit, temperature presentation unit, vibration presentation unit, and spatiotemporal arrangement unit.
- the external output device 140 may include a plurality of two-dimensionally arranged pressing means, vibration generating means, heat generating means, a timer, a receiving device, and the like, such as the integrated tactile sense presentation module described above.
- the object detection sensation providing technology of the TELESAR system which is a tele-distance system that can transmit the reaction force and temperature to the fingertips, developed by the present inventors, may be used (see, for example, JP-A-2013-91114). .
- the haptic information conversion device 100 does not input / output haptics or the like in real time with the external input device 120 or the external output device 140, it is not always necessary to be connected to the network 300.
- the tactile information conversion device 100 acquires input data stored in the external input device 120 or input data stored in the external device 200 such as a server from the external input device 120. Also good.
- the external output device 140 may be connected to the tactile information conversion device 100 or the external device 200 to which the tactile information is uploaded to acquire tactile information.
- the drawing shows a block diagram showing an example of the configuration of the haptic information conversion apparatus 100 to which the present embodiment is applied, and the portion related to the present embodiment in the configuration is mainly shown. It shows conceptually.
- the tactile information conversion device 100 will be described with respect to an example including the input unit 112, the output unit 114, and the like, but is not limited thereto, and without including the input unit 112, the output unit 114, It may function as a server or the like that generates tactile information in response to a request from the outside and outputs it.
- the haptic information conversion apparatus 100 schematically includes a control unit 102 such as a processor (for example, a CPU) that performs overall control of the entire haptic information conversion apparatus 100, communication such as a router connected to a communication line, and the like.
- a communication control interface unit 104 connected to a device (not shown), an input / output control interface unit 108 connected to an input unit 112 and an output unit 114, and a storage unit 106 for storing various databases and tables
- These units are communicably connected via an arbitrary communication path.
- Each unit may be configured to be temporarily communicably connected via an arbitrary communication path as necessary for input / output by the input unit 112 and the output unit 114.
- a recording medium such as a USB memory may be configured such that each unit can temporarily exchange tactile information.
- Various databases and tables (for example, the tactile definition file 106a, the object tactile sense database 106b, etc.) stored in the storage unit 106 are small-capacity high-speed memories (for example, caches) configured using SRAM (Static Random Access Memory) or the like.
- Memory, etc. storage means such as HDD (Hard Disk Drive) and SSD (Solid State Drive), etc., and stores various programs, tables, files, databases, web pages, etc. used for various processes .
- the tactile sense definition file 106a is tactile sense defining means for defining two or more types of tactile stimuli to be presented.
- the tactile sense definition file 106a may store a temporal and / or spatial threshold with the first stimulation point.
- the haptic definition file 106a may define a first type of haptic stimulus and a second type of haptic stimulus. More specifically, the tactile definition file 106a defines a type of tactile stimulus that is more difficult to be physiologically distinguished temporally or spatially than the first type tactile stimulus as the second type tactile stimulus. Also good. Accordingly, the second type of tactile stimulus can be perceived by the first stimulus point by the processing of the output control unit 102c described later.
- the tactile definition file 106a may define a tactile stimulus of a type that is more difficult to distinguish physiologically in time or space than the second type of tactile stimulus as the first type of tactile stimulus. Accordingly, the first type of tactile stimulus can be perceived by the second stimulus point by the processing of the output control unit 102c described later. In general, it is considered that discrimination in terms of space and time becomes difficult in the order of force (particularly pressure), electricity, vibration, and temperature.
- the haptic definition file 106a defines force presentation as a first type of haptic stimulus, and defines one or more of vibration, temperature, and electrical stimulation as a second type of haptic stimulus. May be. Thereby, vibration, temperature, and / or electrical stimulation can be perceived by being localized at the stimulation point of force presentation.
- the tactile definition file 106a defines one or more of vibration, temperature, and electrical stimulus as the first type of tactile stimulus and defines force presentation as the second type of tactile stimulus. The same effect can be obtained by.
- the haptic definition file 106a may define vibration presentation as a first type of haptic stimulus and define a temperature stimulus as a second type of haptic stimulus.
- a temperature stimulus can be localized and perceived as a stimulus point of vibration presentation.
- the tactile definition file 106a defines the temperature stimulus as the first type of tactile stimulus and the vibration presentation as the second type of tactile stimulus.
- the haptic definition file 106a may define an electrical stimulus presentation as a first type of haptic stimulus and define vibration and / or temperature as a second type of haptic stimulus. Accordingly, vibration and / or temperature stimulation can be localized and perceived as a force-presenting stimulation point. The same effect can be obtained by defining the vibration and / or temperature as the first type of tactile stimulus and defining the electrical stimulus presentation as the second type of tactile stimulus by the tactile definition file 106a.
- the object tactile sensation database 106b is an object tactile sensation storage unit that stores an object and a tactile sensation in association with each other.
- the object tactile sensation database 106b should be presented when an object is recognized in the real world by a known object recognition method, or a virtual object in an augmented reality space or a virtual space according to the object. Defines the tactile sensation.
- the object tactile sensation database 106b defines psychological tactile sensations such as a psychological amount that is integratedly recognized in the brain from information obtained from a plurality of different tactile receptors of the human body as tactile sensations to be presented. Also good.
- the object tactile sensation database 106b is a tactile sensation corresponding to an object or a material using a map in which materials, objects, etc. are positioned on a predetermined quantitative axis, like the psychological tactile sensation map (onomatopoeia map) shown in FIG. May be defined.
- the object tactile sensation database 106b includes, as shown in the following table, output tactile information including electricity as a base from input tactile information not including electricity as a base, according to the object and the tactile sensation.
- this conversion table for example, in general force presentation on the skin surface, the problem that the pressure in the pushing direction can be presented but the force in the pulling direction cannot be presented is solved, and electricity is included.
- By reproducing the force in the direction drawn by the stimulus presentation it becomes possible to present various psychological textures expressed by onomatopoeia such as a feeling of necho necho.
- the object tactile sensation database 106b may store parameters such as psychotactile sensations such as onomatopoeia in association with parameters on a rough axis due to force, a rigid axis due to vibration, and / or a wet / dry axis due to temperature.
- the object tactile sensation database 106b is a physical space map in which tactile sensations are associated with a map composed of two to three axes among a rough smooth axis due to force, a hard axis due to vibration, and a wet and dry axis due to temperature. Etc. may be stored.
- the relationship between the tactile sensation and the physical quantity defined by the object tactile sensation database 106b may be stored in advance, or may be newly created or updated by the creation unit 102b or the like. For example, a map that is updated by associating the physical quantity detected by the input unit 112 with the tactile sensation with respect to a sample with a known tactile sensation and the tactile sensation in a learning process of the creation unit 102b described later.
- the object tactile sensation database 106b may be stored.
- the tactile sensation of the sample may be known and used for machine learning or the like as teacher data of the tactile sensation of the sample.
- the information stored in the tactile sense definition file 106a, the object tactile sensation database 106b, or the like may be downloaded from the external device 200 or the like periodically or each time, and the storage unit 106 temporarily or non-downloads the downloaded information. You may memorize
- the input / output control interface unit 108 controls the input unit 112 such as various sensors and the output unit 114 such as an output unit capable of outputting a physical quantity or the like.
- the input unit 112 such as various sensors may have the same function as that of the external input device 120 described above.
- the output unit 114 such as an output unit capable of outputting a physical quantity or the like may have the same function as the external output device 140 described above.
- a monitor including a home television or a touch screen monitor
- the input unit 112 a touch panel, a voice microphone, a keyboard, or the like can be used.
- the input unit 112 and the output unit 114 may be input / output means such as a touch panel in which the output unit 114 such as a liquid crystal panel and the input unit 112 such as a touch position input device are combined.
- the input / output control interface unit may be connected to a recording medium such as a USB memory to perform control to exchange data.
- the input unit may be either the input unit 112 or the external input device 120, and in this case, collectively referred to as the input units 112 and 120.
- the output unit may be either the output unit 114 or the external output device 140, and may be collectively referred to as the output unit 114, 140 or the output unit 114, 140.
- control unit 102 has a control program such as an OS (Operating System), a program that defines various processing procedures, and an internal memory for storing necessary data.
- a processor such as a CPU that performs information processing for executing various processes.
- the control unit 102 includes a presentation tactile sensation setting unit 102a, a creation unit 102b, and an output control unit 102c in terms of functional concept.
- the presentation tactile sensation setting unit 102a is a presentation tactile sensation setting unit that sets a tactile sensation for a presentation purpose.
- the presentation tactile sensation setting unit 102a may set a predetermined tactile sensation as a presentation purpose.
- the presentation tactile sensation setting unit 102a may set a tactile sensation corresponding to body movement as a presentation purpose based on motion data obtained from the external input device 120 or the input unit 112.
- the presentation tactile sensation setting unit 102a corresponds to the object from the object tactile sensation database 106b based on information on the object or virtual object recognized by the real object recognition obtained from the external input device 120 or the input unit 112.
- the tactile sensation for presentation may be set by acquiring the tactile sensation to be performed.
- the presentation tactile sensation setting unit 102a may cause the user to set a tactile sensation for presentation purposes via the external input device 120, the input unit 112, or the like.
- the presentation tactile sensation setting unit 102a performs multipoint measurement on the time axis using the flexible substrate of the input means 120 and 112 as described above, and from the contact area and pressure distribution change taking this time axis into consideration, A tactile sensation such as a sticky sensation or a sticky feeling may be determined.
- the conversion from the determined tactile sensation to the tactile information to be presented may be performed by the control unit 102 using a conversion table such as the above-described table.
- the presentation tactile sensation setting unit 102a acquires information on tactile sensors (input means 120, 112 such as a force sensor, a vibration sensor, and a temperature sensor) of a finger part of a robot such as the above-described TELESAR system, and performs machine learning or the like. According to the method, it is classified into category information related to the tactile sensation such as a plurality of onomatopoeia.
- the combination may be stored in the object tactile sensation database 106b in association with each other.
- the tactile sensation of an object at a remote location can be estimated from the information of the tactile sensor, and the estimated tactile sensation can be presented to a user at another location in a multisensory format.
- the presentation tactile sensation setting unit 102a is based on a physical quantity detected by the input unit 112 or the external input device 120, and is at least two of a rough sliding shaft by force, a hard shaft by vibration, and a wet / dry shaft by temperature. Corresponding tactile sensations (psychological tactile sensation etc.) may be detected by associating them on an axis map or a biaxial conversion table.
- the presentation tactile sensation setting unit 102a may calibrate the input units 120 and 112 and the output units 140 and 114 by a known method or the like. For example, the presentation tactile sensation setting unit 102a transmits an initial operation signal to the external output device 140, and the external output device 140 performs output according to the initial operation signal, thereby executing various calibrations. Good.
- the creation unit 102b selects at least two or more physical quantities according to the tactile sensation to be presented, and creates haptic information for presenting a predetermined tactile sensation based on the selected physical quantities.
- the creation unit 102b selects and selects at least two or more physical quantities according to the tactile sensation to be presented. Tactile information for presenting a predetermined tactile sensation may be created based on the physical quantity.
- the physical quantity includes at least electricity, and includes force, temperature, vibration, and / or spatio-temporal changes such as electricity, force, temperature, vibration, etc.
- the creation unit 102b includes at least Tactile information may be created by selecting two or more physical quantities.
- the creation unit 102b associates the physical quantity with at least two of a rough sliding axis due to force, a rigid axis due to vibration, and a wet / dry axis due to temperature. May be selected.
- the creation unit 102b may create haptic information by selecting at least electrical and spatiotemporal physical quantities when presenting a tactile sensation due to a temporal change in force.
- the creation unit 102b may select the first and second types of haptic stimuli defined in the haptic definition file 106a as a plurality of physical quantities.
- the creation unit 102b may create haptic information by selecting a plurality of physical quantities for the purpose of presentation of the tactile sensation set by the presentation tactile sensation setting unit 102a.
- the creation unit 102b refers to the object tactile sensation database 106b, determines a tactile sensation corresponding to the information of the tactile sensor obtained from the input units 120 and 112, and presents the tactile sensation to display a plurality of tactile sensations and the like Tactile information based on physical quantities may be created.
- the haptic information conversion apparatus 100 does not simply present the recorded haptic information as it is, but performs appropriate conversion according to the difference in physical movement.
- a tactile sensation that a person feels sensibly is a tactile psychological space
- a physiological element such as a tactile receptor that generates a nerve pulse constituting the tactile sensation
- the physical stimulus that activates the object is defined as the tactile physical space.
- the tactile transmission modules 120 and 112 of this embodiment are considered to have a hybrid configuration of stimulation in the physiological space of Merkel cells and Meissner bodies by electrical stimulation and stimulation in the physical space of vibration and temperature. Therefore, in order to present any tactile sensation using the tactile transmission modules 120 and 112, any “tactile sensation” in the psychological space is decomposed into the sensory elements of the tactile sense presentation modules 140 and 114, and according to body movements.
- the control unit 102 of the haptic information conversion apparatus 100 creates and generates arbitrary haptic information by editing and processing the information recorded from the real world through the processing of the creation unit 102b, the output control unit 102c, and the like.
- An algorithm that converts the tactile information recorded in the real world into new tactile information corresponding to different body movements is executed and functions as a tactile primary color encoder.
- the creation unit 102b when presenting a soft psychological texture, in other words, when presenting a “motivated feeling”, the creation unit 102b is stronger in the transition process from the non-contact state to the contact state or the body displacement process than in the case of the hard surface. It may be synthesized by a plurality of physical quantities so that an electrical, force, or vibration stimulus, or an electrical, force, or vibration stimulus having a larger area than that of a hard surface is applied.
- the creation unit 102b may give a stimulus of electrical stimulation, force, or vibration stronger than the strength of tactile stimulation received when a finger or the like is brought into contact with the hard surface. Further, the creation unit 102b may create tactile information that provides an electrical, force, or vibration stimulus having a larger area than the contact area of the tactile stimulus received when a finger or the like is brought into contact with the hard surface. Note that the transition process from the non-contact state to the contact state of the finger or the like and the body displacement process indicating the movement can be determined based on the motion data received by the external input device 120 or the input unit 112 described above. it can.
- the creation unit 102b when presenting a sticky psychological texture, in other words, when presenting “stick feeling”, performs the transition process from the contact state to the non-contact state or the body displacement process. You may synthesize
- the creation unit 102b may apply a stimulus of electrical stimulation, force, or vibration stronger than the strength of tactile stimulation received when a finger or the like is removed from the contact state from the hard surface with the same pressure.
- the creation unit 102b creates tactile information that provides an electrical stimulus, a force stimulus, or a vibration stimulus having a larger area than the contact area of the tactile stimulus received when the finger or the like is released from the hard surface by the same movement of the finger or the like. Also good.
- the transition process from the contact state to the non-contact state of the finger or the like, and the body displacement process indicating the movement can be determined based on the motion data received by the external input device 120 or the input unit 112 described above. it can.
- the creation unit 102b is not limited to the conversion from tactile sensation to physical quantity, and may perform processing for converting physical quantity to tactile sensation.
- the creation unit 102b associates the physical quantity with at least two axes of the rough slipping shaft by force, the soft shaft by vibration, and the wet and dry shaft by temperature according to the tactile sensation to be presented.
- the present invention is not limited to this, and the reverse process may be performed. That is, based on the physical quantity detected by the input unit 112 or the like, the creation unit 102b is associated with at least two axes of a rough shaft by force, a soft shaft by vibration, and a wet and dry shaft by temperature, Corresponding tactile information may be obtained. That is, if the conversion table between the physical quantity and the tactile sensation is stored in the storage unit 106, the creation unit 102b can perform both conversions from either one.
- the output control unit 102c is output control means for outputting tactile information to the output unit side such as the external output device 140 and the output unit 114. Specifically, the output control unit 102c outputs the haptic information created by the creation unit 102b to the output means 140 and 114 side. For example, the output control unit 102 c may transmit the haptic information to the external output device 140 or may output the haptic information to the output unit 114 via the input / output control interface unit 108.
- the output control unit 102c may further include a first stimulation unit 102d and a second stimulation unit 102e.
- the first stimulation unit 102d determines the first stimulation point at which the first type of tactile stimulation occurs, or generates the first type of tactile stimulation at the first stimulation point via the output units 140 and 114. It is a means of stimulation. That is, the first stimulation unit 102d may actively generate the first type of tactile stimulation at the first stimulation point via the output units 140 and 114, and the user moves relative to the object.
- the first stimulation point of the first type of tactile stimulation generated by this may be determined.
- the stimulus point means a point in time and / or a position point in space (a position on a one-dimensional line, a position on a two-dimensional plane, a position on a three-dimensional space, etc.).
- the second stimulation unit 102e is separated from the first stimulation point of the first type of tactile stimulation determined or generated by the first stimulation unit 102d within a predetermined threshold in terms of time and / or space.
- the second stimulation means generates a second type of tactile stimulation at the bistimulation point via the output units 140 and 114.
- the predetermined temporal threshold and the predetermined spatial threshold vary depending on the tactile stimulus intended for presentation, and may be stored in advance in the storage unit 106 (such as the tactile definition file 106a).
- a known two-point discrimination threshold (such as a spatial two-point discrimination threshold or a temporal two-point discrimination threshold) may be used as the predetermined threshold in terms of time and / or space.
- the first stimulus unit 102d determines or generates force presentation as the first type of tactile stimulus
- the second stimulus unit 102e One or more of vibration, temperature, and electrical stimulation may be generated at the second stimulation point as a second type of tactile stimulation.
- the first stimulation unit 102d determines or generates one or more of vibration, temperature, and electrical stimulation as the first type of tactile stimulation
- the second stimulation unit 102e The same effect can be obtained by generating force presentation at the second stimulation point as the second type of tactile stimulation.
- the first stimulus unit 102d determines or generates the vibration presentation as the first type of tactile stimulus
- the second stimulus unit 102e may generate a temperature stimulus as a second type of tactile stimulus at the second stimulus point.
- the first stimulation unit 102d determines or generates a temperature presentation as a first type of tactile stimulus
- the second stimulation unit 102e generates a vibration stimulus as a second type of tactile stimulus. The same effect can be achieved by generating the stimulation point.
- the first stimulus unit 102d determines or generates the electrical stimulus presentation as the first type of tactile stimulus, and the second stimulus unit 102e. May generate vibration and / or temperature at the second stimulus point as a second type of tactile stimulus.
- the first stimulation unit 102d determines or generates vibration and / or temperature as a first type of tactile stimulus
- the second stimulation unit 102e performs electrical stimulation as a second type of tactile stimulus.
- a similar effect can be achieved by generating the presentation at the second stimulation point.
- the stimulation points of the first stimulation point and the second stimulation point may be perception points based on the phantom sensation illusion.
- FIG. 8 is a diagram showing perceptual points based on the phantom sensation illusion.
- a good example of the latter is phantom sensation.
- a white circle in FIG. 8 represents a stimulation site, and a hatched circle represents a perception point by phantom sensation.
- Phantom sensation is a skin sensation phenomenon similar to sound localization by binaural hearing confirmed by von Bekesy.
- a stimulus image can be given to a part different from the stimulus part between the two parts by applying stimulus to the two parts under a certain condition.
- FIG. 8A by applying stimulation to three or more parts, it is possible to localize the stimulation image inside thereof. Further, as shown in FIG.
- the position of the image can be controlled by the difference in the intensity of the stimulation at the plurality of stimulation sites. This is known to occur both for vibration stimulation and for electrical stimulation (Tanie, Kazuo, Tsuji, Susumu, Komoriya, Kiyoshi, Abe, Satoshi “Position of intensity difference phantom sensation images in electrical pulse stimulation. Discrimination Characteristics ”, Transactions of the Society of Instrument and Control Engineers, Vol.15, No.4, p.505-512 (19798), and Susumu Tachi, Kazuo Tanie, Kiyoshi Komoriya and MinoruAbe: ElectricalCum Robot (MELDOG), IEEE Transactions on Biomedical Engineering, Vol. BME-32, No 7, reference pp.461-469 (1985)).
- FIG. 9 and FIG. 10 are diagrams showing the finger-mounted sensor / actuator of the telegesence robot called TELESAR V developed by the inventors of the present application and the appearance when the finger is mounted (details of device configuration and the like) JP, 2013-91114, A).
- TELESAR V the finger-mounted sensor / actuator of the telegesence robot
- vibration stimulation can be similarly presented to the electrical stimulation portion by phantom sensation.
- the phantom sensation image is localized more clearly by electrical stimulation. For this reason, for example, phantom sensation is used to localize vibration stimulation to the site of force and electrical stimulation, and the phantom sensation image of vibration stimulation is clearly displayed by force presentation or electrical presentation. Can be localized.
- a temperature stimulus when a temperature stimulus is applied from the side, it is not localized on the finger pad on one side alone, but when a vibration stimulus is presented on the finger pad, it is localized on that part. It was verified by experiment whether phantom sensation, which is localized in the center when added from the left and right, may be this case.
- the presentation is delayed due to the characteristics of the element and the thermal properties.
- a non-contact temperature sensor can be incorporated in addition to the contact-type temperature sensor.
- the tactile information conversion device 100 includes a database that stores tactile information and threshold values, an external device 200 that provides an external program such as a tactile information conversion program, and the like. It may be configured to be communicably connected via the network 300.
- the tactile information conversion device 100 may be communicably connected to the network 300 via a communication device such as a router and a wired or wireless communication line such as a dedicated line.
- the communication control interface unit 104 is a device that performs communication control between the tactile information conversion device 100 and the network 300 (or a communication device such as a router). That is, the communication control interface unit 104 has a function of communicating data with other terminals or stations via a communication line (whether wired or wireless). In the present embodiment, the communication control interface unit 104 controls communication with the external input device 120, the external output device 140, the external device 200, and the like.
- the external device 200 is connected to the tactile information conversion device 100, the external input device 120, and the external output device 140 via the network 300, and a tactile sense definition file, an object tactile sense database, etc. are connected to each terminal.
- a tactile sense definition file, an object tactile sense database, etc. are connected to each terminal.
- the external device 200 may be realized by hardware elements such as a personal computer and a server computer and software elements such as an operating system, application programs, and other data.
- the external device 200 may be configured as a WEB server, an ASP server, or the like, and the hardware configuration thereof is configured by an information processing device such as a commercially available workstation or a personal computer and an accessory device thereof. Also good.
- Each function of the external device 200 is realized by a processor such as a CPU, a disk device, a memory device, an input device, an output device, a communication control device, etc. in the hardware configuration of the external device 200 and a program for controlling them.
- the external device 200 may be realized by hardware elements such as a personal computer and a server computer and software elements such as an operating system, application programs, and other data.
- the external device 200 may be configured as a WEB server, an ASP server, or the like, and the hardware configuration thereof is configured by an information processing device such as a commercially available workstation or a personal computer and an accessory
- the haptic information conversion device 100 may be configured as an integral unit with the input device, may be configured as an integral unit with the transmission device, may be configured as an integral unit with a storage device, or may be configured as an integral unit with a server device. Alternatively, it may be configured integrally with the receiving device, or may be configured integrally with the output device.
- FIG. 11 is a flowchart showing an example of tactile information conversion processing in the tactile sensation presentation system of the present embodiment.
- the input units 120 and 112 of the tactile sensation presentation system detect a user's physical motion, and the tactile information conversion device 100 performs physical motion such as motion data by processing of the creation unit 102b.
- Information is acquired (step SA-1).
- the input units 120 and 112 may perform touch detection, detect a contact area, or detect finger displacement.
- the external output device 120 acquires body motion information
- the body motion information is transmitted to the tactile information conversion device 100, and the tactile information conversion device 100 receives the body motion information.
- the tactile sensation information conversion device 100 of the present tactile sensation presentation system performs processing by the creation unit 102b, and at least two or more of the physical quantities of electricity, force, temperature, vibration, and spatiotemporal according to the tactile sensation for the presentation purpose.
- a physical quantity is selected and tactile information for presenting a predetermined tactile sensation is created based on the selected physical quantity (step SA-2).
- the creation unit 102b may synthesize at least electrical and space-time physical quantities.
- the creation unit 102b synthesizes them as a plurality of physical quantities based on the first and second types of haptic stimuli. May be.
- the haptic information conversion device 100 of the tactile sensation presentation system outputs haptic information to the output units such as the external output device 140 and the output unit 114 (step SA-3).
- the output control unit 102c outputs the haptic information created by the creation unit 102b to the output units 140 and 114 side.
- the haptic information conversion device 100 transmits the haptic information to the external output device 140 under the control of the output control unit 102c.
- the output control unit 102c may execute processing by the first stimulation unit 102d and the second stimulation unit 102e (processing will be described later).
- the output units 140 and 114 of the present tactile sensation presentation system perform multimodal (multisensory format) output from the output units 140 and 114 based on the tactile information, and present tactile sensations that work on the tactile sensation of the user (step) SA-4).
- FIG. 12 is a flowchart illustrating an example of multipoint stimulation processing in the tactile information conversion device 100 of the tactile sensation presentation system of the present embodiment.
- the output control unit 102c of the haptic information conversion apparatus 100 sets the defined first type haptic stimulus and second type haptic stimulus with reference to the haptic definition file 106a. (Step SB-1).
- the output control unit 102c of the tactile information conversion apparatus 100 determines the first stimulation point where the first type of tactile stimulation occurs by the processing of the first stimulation unit 102d (step SB-2). More specifically, the output control unit 102c may determine the time point, the position point, and the like of the first type of tactile stimulus that is generated when the user contacts the object. Alternatively, the output control unit 102c may output stimulation information that causes the first type of tactile stimulation to be generated at the first stimulation point via the output units 140 and 114 by the processing of the first stimulation unit 102d.
- the output control unit 102c of the tactile information conversion device 100 performs temporal processing from the first stimulation point of the first type of tactile stimulation determined or generated by the first stimulation unit 102d by the processing of the second stimulation unit 102e. And / or a second stimulation point spatially separated within a predetermined threshold is set (step SB-3).
- the output control unit 102c may set the second stimulation point by referring to the threshold value stored in advance in the tactile sense definition file 106a by the processing of the second stimulation unit 102e.
- the output control unit 102c of the tactile information conversion device 100 performs the second type via the output units 140 and 114 at the second stimulation point set in Step SB-3 by the processing of the second stimulation unit 102e.
- the tactile information for generating the tactile stimulus is output (step SB-4).
- the presentation element of all of electricity, force, temperature, vibration, or any combination thereof can be placed in another part on the human skin while giving a tactile sensation to feel stimulation.
- the means by using a presentation method by phantom sensation and other methods, it is physically arranged in another place, but it is possible to present it as if it is felt in one place. .
- force presentation is placed at the point of action of the stimulus, and any stimulus, vibration, temperature, electrical stimulus, or any combination is placed physically away and the sensation of that stimulus is achieved by phantom sensation.
- any stimulus, vibration, temperature, electrical stimulus, or any combination is placed physically away and the sensation of that stimulus is achieved by phantom sensation.
- the localization image by phantom sensation is sharpened, so that the presentation sites of the primary colors of force, electricity, vibration, and temperature can be matched.
- the force presentation is placed at the point of action of the stimulus, and either the vibration, temperature, or both stimulators are placed physically apart, and the sensation of the stimulus is made by phantom sensation. It is possible to match the site of presentation of the primary colors of force, vibration, and temperature by localizing to the point of action of the force and clarifying the localization image by phantom sensation by presenting the force.
- the electrical stimulus presentation is arranged at the action point of the stimulus, and either the vibration, the temperature, or both are arranged at a physically separated place, and the force action point is obtained by phantom sensation. Can be localized.
- vibration presentation can be arranged at the action point of the stimulus, the temperature can be arranged at a physically separated place, and the vibration action point can be localized by phantom sensation.
- the vibration action point can be localized by phantom sensation.
- FIG. 13 is a flowchart illustrating an example of a remote tactile sensation transmission process in the tactile sensation presentation system of the present embodiment.
- the above-described TELESAR system may be used as a partial function of the tactile sensation presentation system (see, for example, JP-A-2013-91114).
- the presentation tactile sensation setting unit 102a includes tactile sensors (force sensors, vibration sensors, temperature sensors, and other input means 120) of the robot's fingers such as the above-described TELESAR system. 112) is acquired (step SC-1).
- the presentation tactile sensation setting unit 102a of the tactile sensation presentation system performs machine learning based on information from the tactile sensor and teacher data regarding tactile sensation of an object recognized by a camera or the like (step SC-2).
- the operator of the haptic information conversion apparatus 100 may input teacher data related to an object or a tactile sensation via the input unit 112 such as a keyboard.
- the presentation tactile sensation setting unit 102a of the present tactile sensation presentation system uses tactile sensor information characteristics (such as temporal and spatial variation patterns) as tactile sensation category information (onomatopoeia map, etc.) such as onomatopoeia as a result of machine learning or the like. Classify (step SC-3).
- tactile sensor information characteristics such as temporal and spatial variation patterns
- tactile sensation category information such as onomatopoeia map, etc.
- the presentation tactile sensation setting unit 102a of the present tactile sensation presentation system stores the classified tactile sensation category information and the tactile sensation presentation method of the present embodiment in association with each other in the object tactile sensation database 106b (step SC-5).
- the tactile display (the external output device 140 that presents a stimulus such as force, electricity, vibration, and temperature) may be stored in the object tactile sense database 106b in association with the spatiotemporal combination.
- the processing of steps SC-1 to SC-4 described above may be executed in advance as preprocessing, and the processing result may be stored in the storage unit 106 in advance.
- a tactile sensor obtained by interacting with the object by moving the finger part of the robot following the movement of the user of the robot teaching device in real time using the TELESAR system or the like.
- Information is acquired and the object tactile sensation database 106b is referred to and classified into the corresponding tactile sensation category (step SC-6).
- the robot side fingers and object detection unit of the TELESAR system can be considered as the external output device 140 and the external input device 120.
- the hand movement detection means and the tactile stimulation means on the robot teaching device side of the TELESAR system can be considered as the input unit 112 and the output unit 114.
- the creation unit 102b and the output control unit 102c of the present tactile sensation presentation system select a plurality of physical quantities according to the tactile sensation category, create tactile information, and output the tactile information to the output unit 114 (step SC-7).
- step SC-8 This makes it possible to infer the tactile sensation of an object at a remote location from the information of the tactile sensor and present the estimated tactile sensation to the user in a multisensory format such as electrical stimulation (step SC-8).
- the presentation tactile sensation setting unit 102a acquires information on tactile sensors (input means 120, 112 such as a force sensor, a vibration sensor, and a temperature sensor) of a finger part of a robot such as a TELESAR system. It is classified into a plurality of onomatopoeia-related categorical information by a method such as learning, and a tactile sensation expressed by the category information and a tactile display (output means 140 and 114 for presenting stimuli such as force, electricity, vibration, and temperature) These temporal and spatial combinations may be associated with each other and stored in the object tactile sensation database 106b.
- FIG. 14 categorizes input information such as sensors into tactile sensation category information based on a category information conversion table obtained by machine learning or the like, and combines a plurality of physical quantities based on the classified tactile sensation category information. It is a figure which shows the functional conceptual structure in the case of carrying out presentation output in this way.
- the tactile information conversion device 100 preliminarily performs machine learning on information of tactile sensors (force sensor 112-1, vibration sensor 112-2, temperature sensor 112-3) arranged on a robot hand or the like.
- a conversion table or the like is constructed so that it can be classified into a plurality of category information such as onomatopoeia.
- the sensor information may be converted from other physical quantity sensors.
- the vibration sensor information is Fourier-transformed for a short time, the frequency information is converted into the vertical axis, the horizontal axis is converted into time, and the temperature and force are similarly converted to be used as a two-dimensional image for character recognition. Deep learning may be performed using a stacked auto-encoder or the like. At that time, a sample whose onomatopoeia is known may be used as a teacher signal.
- the tactile information conversion device 100 performs various tactile signals (force information, vibration information, and the like) from the tactile sensors (force sensor 112-1, vibration sensor 112-2, temperature sensor 112-3) by processing of the identification device 102b ′. Temperature information, etc.) is classified into tactile sensation category information such as onomatopoeia using a conversion table as a learning result.
- the tactile information conversion apparatus 100 processes the classified tactile sensation category information into an adjustment signal by using the correspondence table between tactile sensation and physical quantity by the processing of the identification device 102b ′ and the adjustment device 102c ′.
- vibration can be realized by changing the frequency characteristics with a filter or changing the frequency of the Markov process.
- Learning to obtain the correspondence table between the tactile sensation and the physical quantity for example, classifies signals output from the adjusting device 102c ′ in deep learning, and the correct answer rate is higher than before passing through the adjusting device 102c ′.
- the adjustment device 102c ′ may be learned.
- the identification device 102b ′ associates with at least two of a rough shaft by force, a hard shaft by vibration, and a wet and dry shaft by temperature according to the tactile sensation to be presented. Alternatively, learning may be performed so that the optimum physical quantity is selected.
- the tactile information conversion device 100 is configured so that the input information of the force sensor 112-1, the vibration sensor 112-2, the temperature sensor 112-3, and the like is expressed by category information such as onomatopoeia. It is possible to create a mechanism for processing into a machine learning method. Thereby, when the above learning is completed, the haptic information conversion device 100 moves the robot hand or the like following the movement of the user's hand or the like in real time, and is obtained by interaction with the object at that time. From the tactile sensor information, in real time, signals from various sensors are respectively corresponded to the tactile displays (force display 114-1, electric display 114-2, vibration display 114-3, It can be presented on the temperature display 114-4).
- the same signal can be input to the identification device 102a.
- a tactile sensation that is difficult to present on the force display 114-1 such as repulsive force is displayed on the adjustment signal so as to be presented using the electric display 114-2.
- both force information and vibration information, or only vibration may be converted into electricity by encoding.
- the adjusting device 102c ′ outputs nothing but the information from the tactile sensors 112-1 to 112-4 until the identifying devices 102a ′ and b ′ recognize the tactile sensation. Then, when the tactile sensation is identified by the identification devices 102a ′ and b ′, the adjusting device 102c ′ processes the signal according to the classification of the tactile sensation of the identification result, and the user obtains a tactile sensation that is more easily comprehensible according to the actual situation. be able to. That is, by converting tactile information into tactile information, a tactile sensation represented by category information such as onomatopoeia can be obtained in real time.
- category information such as onomatopoeia
- Example 1 that demonstrates the effect of the tactile sensation presentation system of the present embodiment will be described below.
- Example 1 it was confirmed by an experiment that a tactile sensation that originally needs a sticking force, such as a sticky feeling, can be presented instead of electronic stimulation.
- Example 1 of the present embodiment an object is to present a tactile sensation such as an adhesive feeling in combination with an electric tactile sense or mechanical presentation.
- FIG. 15 is a diagram schematically showing changes in the contact surface and pressure when the hard surface, soft surface, and viscous surface are touched. The figure shows the case where the finger is viewed from the fingertip side in the body axis direction, and the double arrows indicate the width of the contact surface.
- FIG. 15 shows a case where the deformation is very small.
- FIG. 16 is a diagram schematically showing changes in the contact surface and pressure when large deformation occurs when a hard surface, soft surface, or viscous surface is touched.
- Experiment 1 of this example an experiment was performed to determine whether stickiness can be presented by changing the stimulus with respect to the “force” of the fingertip. For example, an experiment was conducted with the hypothesis that the soft surface touches in a wider area when pressed than the hard surface, and that softness is felt by giving a stronger stimulus when pressed. On the other hand, the viscous surface makes contact with a larger area than that of the hard surface at the time of pulling up, so an experiment was conducted to show whether a sticky feeling can be presented by applying a stronger stimulus at the time of pulling up.
- FIG. 17 is a diagram showing a change in the contact area of the hard surface, the soft surface, and the viscous surface with respect to the amount of displacement of the finger.
- soft surfaces soft surfaces that undergo large deformation often undergo plastic deformation and the contact area decreases, but experiments were carried out to feel softness by providing hysteresis when pressed down and pulled up. .
- FIG. 18 is a diagram schematically illustrating the configuration of the experimental apparatus used in Experiment 1 of Example 1 from the fingertip side.
- FIG. 19 is a photographic diagram showing a state before and after mounting the experimental device used in Experiment 1 of Example 1 on the finger.
- a film-like force sensor was used as the input unit 112.
- the output unit 114 a 4 ⁇ 5 matrix, an electrotactile display with a diameter of 1.4 mm in which electric stimulators are arranged at intervals of 2 mm was used.
- a finger fixing tool was used to maintain contact between the electrode for electrical stimulation and the skin.
- cathode stimulation was used as the polarity of all electrical stimulation unless otherwise specified.
- one electrode was stimulated at a time, and all 20 electrodes were stimulated with a pulse period of 60 pps (pulses per sec).
- FIG. 20 is a diagram illustrating experimental conditions (press conditions) used to present a soft feeling in Experiment 1 of the first embodiment.
- FIG. 21 is a diagram illustrating experimental conditions (Release conditions) used to present a sticky feeling in Experiment 1 of the first embodiment.
- FIG. 22 is a diagram illustrating a subjective evaluation experiment result according to Experiment 1 of the first embodiment.
- the vertical axis represents the seven-stage subjective evaluation values.
- the horizontal axis represents, from left to right, the subjective evaluation result of softness under the Press condition, the subjective evaluation result of stickiness under the Press condition, the subjective evaluation result of softness under the Release condition, and the viscosity under the Release condition.
- the subjective evaluation result of feeling is shown.
- FIG. 23 is a photograph showing a state in which the experimental device used in Experiment 2 of Example 1 is worn on the finger. As shown in FIG. 23, unlike the apparatus configuration of Experiment 1, an acceleration sensor is provided. Thereby, the inclination of the finger can be measured, and the stimulus can be changed by the movement of the finger in the air. Note that the desk and pressure sensor of Experiment 1 are not used.
- FIG. 24 is a diagram illustrating experimental conditions (press conditions) of Experiment 2 of the first embodiment.
- FIG. 25 is a diagram illustrating experimental conditions (Release conditions) of Experimental Example 2 of the first embodiment.
- the lifting operation is performed at an angle of 100 degrees or less from the vertically downward direction
- the stimulation is performed at an angle of 80 degrees or more and a constant current value adjusted by the volume. went.
- the stimulus is performed at a constant current value adjusted by the volume at an angle of 100 degrees or less from the vertically downward direction during the pushing operation and at an angle of 80 degrees or more during the lifting operation. It was.
- the stimulation range during the pushing operation is larger than that during the lifting operation
- the Release condition the stimulation range during the lifting operation is larger than that during the pushing operation. It can be said that.
- FIG. 26 is a diagram illustrating a subjective evaluation experiment result according to Experiment 2 of the first embodiment.
- the vertical axis represents the seven-stage subjective evaluation values.
- the horizontal axis represents, from left to right, the subjective evaluation result of softness under the Press condition, the subjective evaluation result of stickiness under the Press condition, the subjective evaluation result of softness under the Release condition, and the viscosity under the Release condition.
- the subjective evaluation result of feeling is shown.
- FIG. 27 is a diagram illustrating a configuration example of a high-density presentation device in which an electrical stimulation matrix and a mechanical stimulation matrix are combined.
- electrical stimulation there is a problem of sensory stability and sensory quality for long-term stimulation.
- electrical stimulation also has the advantage that a very natural sensation can be produced if a short pulse is presented.
- electrical stimulation has an advantage that it can output a stimulus (vibration sensation or the like) with high time resolution.
- mechanical stimulation has a problem of poor time responsiveness, particularly when the size is reduced as in matrix presentation.
- FIG. 27 is a diagram showing a range of the first embodiment (axis 1) in the above-described onomatopoeia map (FIG. 3).
- the expression by the force in the vertical direction can be performed as shown in the range of the thick frame in FIG.
- This region can be said to be a region that can be expressed by the duration of the force in the vertical direction. That is, this region can be expressed by presenting a vertical force for a longer time in the upward direction during the “pushing and lifting” operation of the finger.
- the tactile sensation of “mochimochi” and “greasy” on the onomatopoeia map can be distinguished and expressed by the duration of the force in the vertical direction.
- the force in the vertical direction is shown.
- the prior knowledge of the inventors of the present application Karl et al.
- Cathodic electrical stimulation pressure sensation presentation
- a resistance force Frtional Force
- FIG. 29 is a diagram illustrating a range of tactile sensation expressed by a horizontal resistance force.
- onomatopoeia In order to present the onomatopoeia on the rough side of this rough shaft, it is necessary to present resistance (for example, the difference between a rough feeling and a bulky feeling). Most of the resistance force is horizontal force, but some onomatopoeia involves vertical force as described above.
- the region on the lower left side of the rough-sliding shaft shown in FIG. 29 can be expressed mainly by a combination of resistance and vibration. However, it is considered that temperature is involved in a part of the lower right area (smooth, smooth, etc.). It was considered that this region can be expressed by presenting a stronger resistance force in the upper left direction of the figure and presenting a vibration with a lower center frequency in the lower left direction. For example, “roughness”, “gogogogo”, and “bulkyness” can be distinguished and expressed by the strength of resistance and vibration components. It is considered that the ruggedness can be expressed with a frequency lower than the roughness, and the bulkiness can be expressed with a weaker resistance than the roughness.
- FIG. 30 is a diagram showing a device used in the experiment.
- the experimental device For details of the experimental device and the like, refer to prior knowledge (Kajimoto, et al. 1999, Sato et al. 2010).
- FIG. 31 is a diagram illustrating the results of a reproduction experiment of “feeling rough”, “feeling rough”, and “feeling bulky”.
- FIG. 32 is a diagram illustrating a range of tactile expression that is reproduced by resistance force and temperature.
- the region to the right of the vertical dotted line in the figure was considered to be expressed mainly by a combination of resistance and temperature.
- the dry-wet axis can be expressed by a decrease in temperature, and in the previous research of the present inventors, the wetness of the cloth is expressed by a decrease in temperature (Sato et al., 2016). Conversely, there are no receptors that perceive humidity itself.
- the hard-flexible axis can be expressed mainly by vibration, and along the axis in the figure, the center frequency of vibration is low in the lower left region (eg, rugged), but the center frequency increases as it goes to the upper right. (Example: rough, bulky).
- the region indicated by the thick frame in FIG. 32 presents a large temperature drop in the right direction of the figure, and presents a strong resistance force or a long-time resistance force in the upper left direction (direction along the resistance force axis). It can be expressed. Thus, for example, an experiment was conducted to distinguish between “smooth”, “slimy”, “smooth”, and “smooth”. It is considered that a large temperature drop is required as the movement to the right of the region (example: Numenume vs. slimy). On the other hand, on the left side outside the region, the temperature decrease is not involved. Because the temperature drop also represents the material, it is difficult for humans to distinguish (eg, the temperature drop when touching metal). For this reason, the name of this axis is assumed to be the wet and dry axis, but it also includes the expression of material feeling (eg, smooth vs. smooth).
- FIG. 33 is a diagram illustrating a device and a vibration waveform used in the experiment.
- FIG. 34 is a diagram showing the results of a reproduction experiment of “smooth”, “smooth”, “smooth”, and “slimy”.
- FIG. 35 is a diagram illustrating an onomatopoeia map indicating a psychological space and a stimulus map of a physical space represented by three axes of force, vibration, and temperature.
- the rough-smooth axis, hard-soft axis, and dry-wet axis of the onomatopoeia map can be expressed by resistance (horizontal force, vertical force), vibration, and temperature.
- resistance horizontal force, vertical force
- vibration vibration
- temperature temperature
- the psychological space of tactile sensation can be expressed mainly by the coarse, hard and dry axes.
- using vertical and horizontal force presentation means by electrical stimulation has the advantage of not only compactness but also the naturalness of the spatial distribution of the occurrence sensation compared to mechanical stimulation.
- sticky and sticky can be expressed by vertical force
- rugged, bulky and rough can be expressed by combination of horizontal force and vibration
- slippery, smooth, slimy, slimy can be presented by combination of horizontal force and temperature
- resistance force horizontal force, vertical force
- vibration and temperature
- Example 2 According to the tactile sensation presentation system of the present embodiment will be described below.
- Example 2 it was confirmed by an experiment that perceptual points can be clarified temporally and spatially mainly for temperature presentation.
- FIG. 36 is a diagram illustrating a configuration of the experimental apparatus (Peltier element) of the second embodiment
- FIG. 37 is a diagram illustrating an experimental environment of the second embodiment and an example of mounting the Peltier element.
- Peltier elements were attached to the finger side surfaces (locations a, b, c, and d). Moreover, as shown to Fig.37 (a), skin temperature was adjusted with the hotplate. On the hot plate, a Peltier element to be touched and an aluminum plate (Platform) for resting were arranged.
- Platinum aluminum plate
- Example 2 touching a touch target at the same time as a signal was performed with an interval of 30 seconds.
- the test subject was asked to feel a randomly selected stimulus when touching the object to be touched for 2 seconds, and the stimulus intensity felt on the finger pad was answered numerically with the perceived intensity for the standard stimulus being 100.
- Three trials were performed for each condition, and the average value was adopted as the perceived intensity of the subject.
- FIG. 38 is a diagram schematically illustrating a position to apply a temperature stimulus when a finger is viewed from the fingertip side.
- the abdomen and side use two Peltier elements each, and there are two stimulation intensities (warm / strong 4 ° C / s, warm / weak 3 ° C / s, cold / strong 3 ° C / s, cold / weak 2 ° C / s s).
- a strong stimulus to Pad was used as a standard stimulus, and the intensity of the standard stimulus was confirmed every 6 trials.
- FIG. 39 is a graph showing the experimental results of the second embodiment.
- FIG. 39 (a) shows the case of warm stimulation
- FIG. 39 (b) shows the case of cold stimulation, where p ⁇ 0.05 (multiple comparison by ANOVA and Ryan method), and only significant difference from Pad (only finger pad) is shown. Show.
- both the warm (both sides) and the pad (only the abdomen) and the cold were perceived by both the hot and cold.
- Pad (only the abdomen) and Side (only the side surface) the pad weakness and the Side strength were perceived at the same intensity by the temperature stimulation. Moreover, it was perceived at the same intensity even with cold stimulation.
- Experiment 2 of Example 2 the presentation position of the warm / cool stimulus and the presence / absence of the pressure / vibration stimulus were examined.
- the purpose of Experiment 2 is to evaluate the thermal sensation perceived on the finger pad and the side surface when the finger pad is in contact with the object in advance or not in contact with the object. The experiment confirmed how the feeling was affected.
- the influence of the presentation position (one side and both sides) of the thermal stimulation on the side surface and the presence or absence of vibration stimulation were evaluated.
- the subjects were 12 women aged 19 to 21 years and all gave no prior knowledge about the experimental hypothesis.
- the skin temperature of the finger pad was previously adjusted to 32 degrees.
- FIG. 40 is a diagram illustrating an experimental apparatus of Experiment 2 of the second embodiment.
- a Peltier device was attached to the side of the finger, and the skin temperature was adjusted with a hot plate.
- a Peltier element to be touched locations e and b
- a resting aluminum plate were arranged on the hot plate.
- a vibrator was provided under the Peltier element.
- the object was touched at the same time as the signal or the finger was kept in the air at an interval of 30 seconds. Then, a randomly selected stimulus was felt for 2 seconds, and the stimulus intensity felt on the abdomen and side surfaces of the finger was answered numerically with the perceived intensity for the standard stimulus being 100. The experiment was performed for each condition.
- FIG. 41 is a diagram schematically illustrating a position where a temperature stimulus (Thermal stimulus) is applied when a finger is viewed from the fingertip side. As shown in FIG. 41, the abdomen and the side surface each use two Peltier elements.
- the stimulus intensity was set to two types of temperature / strong 4 ° C./s, warm / weak 3 ° C./s, cold / strong 3 ° C./s, and cold / weak 2 ° C./s.
- the vibration stimulus was applied to the finger pad at 200 Hz for 0.1 second immediately before the hot and cold stimulus.
- stimulation to Pad was used as a standard stimulus, and the intensity
- FIG. 42 is a graph showing the experimental results of Experiment 2 of Example 2 in the case of temperature stimulation.
- FIG. 43 is a graph showing the experimental results of Experiment 2 of Example 2 in the case of cold stimulation. Similar to FIG. 39, strong stimuli are shown as dark bar graphs and weak stimuli as light bar graphs.
- Example 3 Basic confirmation was performed on the following three items. a. Presence / absence of contact + temperature stimulation b. Existence of vibration stimulus + temperature stimulus c. Pressure stimulus + Vibration stimulus + Temperature stimulus
- the subjects were two 19-year-old women, and the finger skin temperature was adjusted to 32 degrees.
- the experimental apparatus was equipped with a Peltier device on the side of the finger, the stimulation intensity was about 3.5 ° C./s, and the stimulation time was 2 seconds.
- TECHTILE Toolkit was used, and the frequency was 200 Hz.
- Example 2 As an experimental method of Experiment 3 (a. Presence / absence of contact + temperature stimulation) of Example 2, as soon as the alarm sounded, the target object (plastic resin) was touched and temperature stimulation was started at a specific time before and after the alarm. (From -2 seconds to +2 seconds, 9 types in 0.5 second increments). Then, he asked me to answer numerically the temperature sensation felt on the finger pad (cold: -3 points to +3 points: warm). Tried twice for each condition.
- FIG. 44 is a diagram showing a shift in temperature sensation with respect to contact in this experiment 3 (a. Presence / absence of contact + temperature stimulation). As shown in FIG. 44, it was found that the temperature sensation of the finger pad was not generated when the time was shifted by about ⁇ 2 seconds. In the case of a warm sensation, it is desirable to present the temperature stimulus earlier than the vibration stimulus, and in the case of a cold sensation, the temperature stimulus is preferably presented simultaneously with the vibration stimulus. Psychophysical findings indicate that warm sensations are felt later, which may have an effect. In addition, as a subjective impression, I felt the strongest when the skin temperature changed simultaneously with the contact, and felt that I was touching a warm (or cold) plastic resin.
- FIG. 45 is a diagram showing a deviation in temperature sensation with respect to vibration in this experiment 3 (b. Presence / absence of vibration stimulus + temperature stimulus).
- the change in temperature sensation due to the time difference was small as compared with Experiment a. Due to the influence of the contact pressure with the vibrator, a certain temperature sensation may occur regardless of the presence or absence of vibration stimulation. It was the same as in Experiment a that the warm stimulus was fast and the cold stimulus was obtained simultaneously when a strong sensation was obtained.
- Table 2 is a table showing a deviation in temperature perception due to a combination of pressure stimulation and vibration stimulation in the case of cold stimulation.
- Table 3 is a table showing a temperature perception shift due to a combination of a pressure stimulus and a vibration stimulus in the case of a temperature stimulus. The effect of contact pressure was slight, and the lower the contact pressure, the easier it was to obtain a temperature sensation. Similar to Experiments a and b, it was thought that the sensation would not be weakened even if the warm stimulus was presented early.
- Example 2 From the above Example 2, the following knowledge was obtained. According to Example 2, it was found that the sense obtained by shifting the timing of the cold stimulus with respect to the tactile stimulus becomes weak. It was also found that it was difficult to obtain a sense when the time was shifted by about ⁇ 2 seconds.
- the timing of thermal stimulation with respect to tactile stimulation may depend on the time it takes to perceive thermal stimulation, and in the case of thermal stimulation, it was found that the timing obtained earlier is stronger than the simultaneous timing. .
- vibration stimulation is performed in a state where the vibrator is always in contact with the vibrator, a sense is generated. There was a need to give a context that contact with an object occurred when vibration started. The intensity of pressure stimulation did not affect the resulting sensation.
- force presentation is arranged at the action point of stimulation, and any one of vibration, temperature, electrical stimulation, or any combination of stimulators is arranged at a physically separated place, and the phantom
- the sense of stimulation can be localized at the point of action of the force by a phenomenon such as sensation.
- by clarifying the localization image by phantom sensation or the like by the presentation of force it is possible to match the presentation sites of the primary colors of force, electricity, vibration, and temperature.
- the force presentation is arranged at the point of action of the stimulus, and either the vibration, the temperature, or both stimulators are arranged physically separated from each other, and the phantom sensation etc. It becomes possible to localize the sense of stimulation to the point of action of the force. In addition, by presenting the force, the localization image by phantom sensation or the like is clarified, thereby making it possible to match the presentation sites of the primary colors of force, vibration, and temperature.
- the electrical stimulus presentation is arranged at the action point of the stimulus, and either the vibration, the temperature, or both are arranged at a physically separated place, and the action point of the force by phantom sensation or the like. Can be localized. Further, by clarifying the localization image by phantom sensation or the like by the presentation of electrical stimulation, it is possible to match the presentation sites of the primary colors of electricity, vibration, and temperature.
- vibration presentation can be arranged at the action point of the stimulus, and the temperature can be arranged at a physically distant place to be localized at the action point of the vibration by phantom sensation or the like. Further, by clearing a localization image by phantom sensation or the like by presenting vibrations, it is possible to match the presentation parts of the primary colors of vibration and temperature.
- the haptic information conversion apparatus 100 may perform processing in response to a request from a client terminal such as the external device 200 and return the processing result to the client terminal.
- all or part of the processes described as being automatically performed can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method.
- each illustrated component is functionally conceptual and does not necessarily need to be physically configured as illustrated.
- the processing functions provided in each device of the tactile information conversion device 100 are all or any part of a processor such as a CPU (Central Processing Unit) and the processor. It may be realized by a program that is interpreted and executed by, or may be realized as a hardware processor by wired logic.
- the program is recorded on a non-transitory computer-readable recording medium including programmed instructions for causing the computer to execute the method according to the present invention, which will be described later, and the tactile information conversion is performed as necessary. It is mechanically read by the device 100 or the external device 200.
- the storage unit 106 such as a ROM or an HDD (Hard Disk Drive)
- a computer program for giving instructions to the CPU in cooperation with an OS (Operating System) and performing various processes is recorded.
- This computer program is executed by being loaded into the RAM, and constitutes a control unit in cooperation with the CPU.
- the computer program may be stored in an application program server connected to the haptic information conversion device 100, the external device 200, the external input device 120, or the external output device 140 via an arbitrary network 300, It is also possible to download all or part of it if necessary.
- the program according to the present invention may be stored in a computer-readable recording medium, or may be configured as a program product.
- the “recording medium” includes a memory card, USB memory, SD card, flexible disk, magneto-optical disk, ROM, EPROM, EEPROM, CD-ROM, MO, DVD, and Blu-ray (registered trademark). It includes any “portable physical medium” such as Disc.
- program is a data processing method described in an arbitrary language or description method, and may be in any form such as source code or binary code.
- program is not necessarily limited to a single configuration, but is distributed in the form of a plurality of modules and libraries, or in cooperation with a separate program typified by an OS (Operating System). Including those that achieve the function.
- OS Operating System
- a well-known configuration and procedure can be used for a specific configuration for reading a recording medium, a reading procedure, an installation procedure after reading, and the like in each device described in the embodiment.
- the present invention may be configured as a program product in which a program is recorded on a computer-readable recording medium that is not temporary.
- Various databases and the like (tactile sense definition file 106a, object tactile sensation database 106b and the like) stored in the storage unit 106 are storage devices such as a memory device such as a RAM and a ROM, a fixed disk device such as a hard disk, a flexible disk, and an optical disk. It stores various programs, tables, databases, web page files, etc. used for various processes and website provision.
- the tactile information conversion device 100, the external device 200, the external input device 120, and the external output device 140 may be configured as an information processing device such as a known personal computer or a workstation.
- the peripheral devices may be connected.
- the tactile information conversion device 100, the external device 200, the external input device 120, and the external output device 140 are realized by installing software (including programs, data, and the like) that realizes the method of the present invention in the information processing device. May be.
- the specific form of distribution / integration of the devices is not limited to that shown in the figure, and all or a part of them may be functional or physical in arbitrary units according to various additions or according to functional loads. Can be distributed and integrated. That is, the above-described embodiments may be arbitrarily combined and may be selectively implemented.
- a tactile information conversion device including at least a control unit for providing tactile information to an output unit that can output physical quantities including at least electricity and including force, temperature, vibration, and / or space-time,
- the controller is A creation unit that selects a plurality of physical quantities of at least two or more of the physical quantities according to a tactile sensation to be presented, and creates haptic information for presenting the predetermined tactile sensation based on the selected physical quantities
- An output control unit that outputs the haptic information created by the creation unit to the output unit
- a tactile information conversion device comprising:
- the creating unit When presenting a tactile sensation due to temporal changes in force, at least the physical quantities of electricity and space-time are selected, and tactile information for presenting the predetermined tactile sensation is created based on the selected physical quantities.
- a tactile information converter When presenting a tactile sensation due to temporal changes in force, at least the physical quantities of electricity and space-time are selected, and tactile information for presenting the predetermined tactile sensation is created based on the selected physical quantities.
- the tactile information conversion device according to any one of appendices 1 to 4,
- the creating unit In the transition process from non-contact state to contact state or in the process of body displacement, stimulation of electricity, force or vibration stronger than that of hard surface, or electricity, force or vibration of larger area than that of hard surface
- the tactile information is created so that
- the output control unit A tactile information conversion device characterized by presenting a soft psychological texture based on the tactile information created by the creating unit.
- the tactile information conversion device according to any one of appendices 1 to 5,
- the creating unit In the transition process from contact state to non-contact state or body displacement process, the stimulation of electricity, force or vibration stronger than that on the hard surface, or electricity, force or vibration of a larger area than on the hard surface Create the tactile information to give a stimulus,
- the output control unit A tactile information conversion device that presents a sticky psychological texture based on the tactile information created by the creating unit.
- a tactile information conversion device As described above in detail, according to the present invention, a tactile information conversion device, a tactile information conversion method, a tactile information conversion program, and a recording that can be used universally by presenting an arbitrary tactile sensation are provided.
- Media can be provided and has industrial utility.
- remote work / remote work field for example, by moving a remote robot as its own body and experiencing it, remote work / remote work field, care / health field, skill transmission / learning field, media / broadcast technology, virtual It can be used in industrial fields such as sports, entertainment, mobile and wearable.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Mechanical Engineering (AREA)
- Robotics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Dermatology (AREA)
- Neurology (AREA)
- Neurosurgery (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- User Interface Of Digital Computer (AREA)
- Manipulator (AREA)
Abstract
Description
以下、本発明の実施形態の概要を説明するために、まず本発明の実施形態を考案するに至った背景および概要について説明し、その後、本実施形態の構成および処理等について詳細に説明する。なお、本実施形態の概要は、本発明の実施形態を考案するに至った背景および概要を示すものであり、本発明を限定するものではない。
触覚情報変換装置を含む触感提示システムの構成について図面を参照しながら説明する。図5は、本発明の実施形態にかかる触覚情報変換装置を含む触感提示システムの構成の一例を示すブロック図であり、該構成のうち本発明に関係する部分のみを概念的に示している。
つづいて、本実施の形態の触覚情報変換装置100の構成について詳細に説明する。再び図5に戻り、図示は、本実施の形態が適用される触覚情報変換装置100の構成の一例を示すブロック図を示しており、該構成のうち本実施の形態に関係する部分を中心に概念的に示している。なお、本実施の形態において、触覚情報変換装置100は、入力部112や出力部114等を備えた例について説明するが、これに限られず、入力部112や出力部114等を備えることなく、外部から要求に応じて触覚情報を作成して出力送信するサーバ等として機能してもよいものである。
次に、このように構成された本実施形態における触感提示システムの処理の一例について、以下に図11~図13を参照して詳細に説明する。
図11は、本実施形態の触感提示システムにおける触覚情報変換処理の一例を示すフローチャートである。
図12は、本実施形態の触感提示システムの触覚情報変換装置100における多点刺激処理の一例を示すフローチャートである。
ここで、図13は、本実施形態の触感提示システムにおける遠隔触感伝送処理の一例を示すフローチャートである。なお、本触感提示システムの一部機能として、上述したTELESARシステムを用いてもよい(例えば特開2013-91114号公報参照)。
ここで、本実施形態の触感提示システムの効果を実証した実施例1について以下に説明する。実施例1では、ねばねば感など本来、吸着力を必要とする触感の提示を、電子刺激で代替して提示できることを実験により確かめた。
ここで、本実施形態の触感提示システムによる実施例2について以下に説明する。
a.接触の有無+温度刺激
b.振動刺激の有無+温度刺激
c.圧刺激+振動刺激+温度刺激
さて、これまで本発明の実施の形態について説明したが、本発明は、上述した実施の形態以外にも、特許請求の範囲に記載した技術的思想の範囲内において種々の異なる実施の形態にて実施されてよいものである。
少なくとも電気を含み、力、温度、振動、および/または、時空間を含む物理量を出力可能な出力部側へ触覚情報を与えるため、制御部を少なくとも備えた触覚情報変換装置であって、
前記制御部は、
提示目的とする触感に応じて、前記物理量のうち少なくとも二つ以上の複数の物理量を選択するとともに、選択された物理量に基づいて、所定の前記触感を提示するための触覚情報を作成する作成部と、
前記作成部により作成された前記触覚情報を前記出力部側に出力する出力制御部と、
を備えたことを特徴とする、触覚情報変換装置。
付記1に記載の触覚情報変換装置において、
前記物理量のうちの電気の出力とは、触受容器の電気刺激提示であることを特徴とする、触覚情報変換装置。
付記1または2に記載の触覚情報変換装置において、
前記触感は、心理質感であることを特徴とする、触覚情報変換装置。
付記1乃至3のいずれか一つに記載の触覚情報変換装置において、
前記作成部は、
力の時間変化による触感を提示する場合、少なくとも電気および時空間の前記物理量を選択するとともに、選択された物理量に基づいて、所定の前記触感を提示するための触覚情報を作成することを特徴とする、触覚情報変換装置。
付記1乃至4のいずれか一つに記載の触覚情報変換装置であって、
前記作成部は、
非接触状態から接触状態への遷移過程または身体の変位過程において、硬表面の場合よりも強い電気、力、もしくは振動の刺激、または、硬表面の場合よりも広い面積の電気、力、もしくは振動の刺激が与えられるように前記触覚情報を作成し、
前記出力制御部は、
前記作成部により作成された前記触覚情報に基づいて、柔らかい心理質感を提示すること
を特徴とする、触覚情報変換装置。
付記1乃至5のいずれか一つに記載の触覚情報変換装置であって、
前記作成部は、
接触状態から非接触状態への遷移過程または身体の変位過程において、硬表面の場合よりも強い電気、力、もしくは振動の刺激、または、硬表面の場合よりも広い面積の電気、力もしくは振動の刺激が与えられるように前記触覚情報を作成し、
前記出力制御部は、
前記作成部により作成された前記触覚情報に基づいて、粘つく心理質感を提示すること
を特徴とする、触覚情報変換装置。
102 制御部
102a 提示触感設定部
102b 作成部
102c 出力制御部
102d 第一刺激部
102e 第二刺激部
104 通信制御インターフェイス部
106 記憶部
106a 触覚定義ファイル
106b 物体触感データベース
108 入出力制御インターフェイス部
112 入力部
114 出力部
120 外部入力装置
120a 分布型圧覚計測センサ
120b 温冷感計測センサ
120c 振動感計測センサ
140 外部出力装置
140a 分布型圧力提示部
140b 温冷感提示部
140c 振動提示部
200 外部機器
Claims (20)
- 少なくとも電気を含み、力、温度、振動、および/または、時空間を含む物理量を出力可能な出力部側へ触覚情報を与えるため、制御部を少なくとも備えた触覚情報変換装置であって、
前記制御部は、
提示目的とする触感に応じて、前記物理量のうち少なくとも二つ以上の複数の物理量を選択するとともに、選択された物理量に基づいて、所定の前記触感を提示するための触覚情報を作成する作成部と、
前記作成部により作成された前記触覚情報を前記出力部側に出力する出力制御部と、
を備えたことを特徴とする、触覚情報変換装置。 - 請求項1に記載の触覚情報変換装置において、
前記物理量のうちの電気の出力は、触受容器の電気刺激提示であることを特徴とする、触覚情報変換装置。 - 請求項1または2に記載の触覚情報変換装置において、
前記触感は、心理質感であることを特徴とする、触覚情報変換装置。 - 請求項1乃至3のいずれか一つに記載の触覚情報変換装置において、
前記身体は、指であることを特徴とする、触覚情報変換装置。 - 請求項3に記載の触覚情報変換装置において、
前記心理質感は、
人体の複数の異なる触覚受容器で得られる情報から脳内で統合的に認知される心理量であることを特徴とする、触覚情報変換装置。 - 請求項1乃至5のいずれか一つに記載の触覚情報変換装置を備えた入力装置。
- 請求項6に記載の入力装置において、
表層に、多点分布型圧覚計測センサと、
中間層に、温冷感計測センサと、
下層に、振動感計測センサと、
を少なくとも備えた入力装置。 - 請求項1乃至5のいずれか一つに記載の触覚情報変換装置を備えた送信装置。
- 請求項1乃至5のいずれか一つに記載の触覚情報変換装置を備えた記憶装置。
- 請求項1乃至5のいずれか一つに記載の触覚情報変換装置を備えたサーバ装置。
- 請求項1乃至5のいずれか一つに記載の触覚情報変換装置を備えた受信装置。
- 請求項1乃至5のいずれか一つに記載の触覚情報変換装置を備えた出力装置。
- 請求項12に記載の出力装置において、
表層に、多点電気触覚刺激による分布型圧力提示部と、
中間層に、ペルチェ素子による高速駆動型温冷感提示部と、
下層に、広周波数域の振動提示部と、
を少なくとも備えた出力装置。 - 請求項1乃至5のいずれか一つに記載の触覚情報変換装置を備えた、操作者が操作するロボット教示装置を用いてロボットに動作を教示するインタラクション操作システムにおいて、
前記ロボットは、
物体を把持するための手指部、
前記物体または前記物体の性状を検出し、物体検出情報を生成する物体検出部であって、前記手指部に配置される物体検出部、
前記ロボットを駆動させるロボット駆動装置、
を有し、
前記ロボット教示装置は、
前記ロボットから伝送された前記物体検出情報に基づき前記触覚情報変換装置により変換された触覚情報に応じて、対応する触感を前記操作者に提供する物体検知感覚提供部、
を有すること、
を特徴とするインタラクション操作システム。 - 少なくとも電気を含み、力、温度、振動、および/または、時空間を含む物理量を出力可能な出力部側へ触覚情報を与えるため、制御部を少なくとも備えた触覚情報変換装置において実行される触覚情報変換方法であって、
前記制御部において実行される、
提示目的とする触感に応じて、前記物理量のうち少なくとも二つ以上の複数の物理量を選択するとともに、選択された物理量に基づいて、所定の前記触感を提示するための触覚情報を作成する作成ステップと、
前記作成ステップにて作成された前記触覚情報を前記出力部側に出力する出力制御ステップと、
を含むことを特徴とする、触覚情報変換方法。 - 少なくとも電気を含み、力、温度、振動、および/または、時空間を含む物理量を出力可能な出力部側へ触覚情報を与えるため、コンピュータに実行させるためのプログラムであって、
提示目的とする触感に応じて、前記物理量のうち少なくとも二つ以上の複数の物理量を選択するとともに、選択された物理量に基づいて、所定の前記触感を提示するための触覚情報を作成する作成ステップと、
前記作成ステップにて作成された前記触覚情報を前記出力部側に出力する出力制御ステップと、
をコンピュータに実行させるための触覚情報変換プログラム。 - 電気、力、温度、振動、および/または、時空間を含む物理量を出力可能な出力部側へ触覚情報を与えるため、制御部を少なくとも備えた触覚情報変換装置であって、
前記制御部は、
提示目的とする触感に応じて、前記物理量のうち少なくとも一つの物理量を選択するとともに、選択された物理量に基づいて、所定の前記触感を提示するための触覚情報を作成する作成部と、
前記作成部により作成された前記触覚情報を前記出力部側に出力する出力制御部と、
を備え、
前記作成部は、
前記提示目的とする触感に応じて、力による粗滑軸、振動による硬柔軸、および、温度による乾湿軸のうちの少なくとも2軸上に対応付けて、前記物理量を選択すること
を特徴とする、触覚情報変換装置。 - 請求項17に記載の触覚情報変換装置において、
前記作成部は、
触感が既知のサンプルに対して前記検出部により検出された前記物理量と、当該触感とを、前記少なくとも2軸のマップ上に対応付けて更新することにより学習を行うことを特徴とする、触覚情報変換装置。 - 触覚情報を得るため、検出部と制御部を少なくとも備えた触覚情報変換装置であって、
前記検出部は、
力、温度、振動、および/または、時空間を含む物理量を検出可能であり、
前記制御部は、
前記検出部により検出された前記物理量に基づいて、力による粗滑軸、振動による硬柔軸、および、温度による乾湿軸のうちの少なくとも2軸上に対応付けることにより、対応する触感を提示するための前記触覚情報を取得すること、
を特徴とする、触覚情報変換装置。 - 請求項19に記載の触覚情報変換装置において、
前記制御部は、
触感が既知のサンプルに対して前記検出部により検出された前記物理量と、当該触感とを対応付けた前記少なくとも2軸のマップを更新することにより学習を行うことを特徴とする、触覚情報変換装置。
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020187026642A KR20180112847A (ko) | 2016-04-07 | 2017-04-07 | 촉각 정보 변환 장치, 촉각 정보 변환 방법, 및, 촉각 정보 변환 프로그램 |
JP2018510679A JP6875010B2 (ja) | 2016-04-07 | 2017-04-07 | 触覚情報変換装置、触覚情報変換方法、および、触覚情報変換プログラム |
CN201780021222.3A CN108885500A (zh) | 2016-04-07 | 2017-04-07 | 触觉信息转换装置、触觉信息转换方法、及触觉信息转换程序 |
US16/152,982 US10739858B2 (en) | 2016-04-07 | 2018-10-05 | Tactile information conversion device, tactile information conversion method, and tactile information conversion program |
US16/941,024 US11281296B2 (en) | 2016-04-07 | 2020-07-28 | Tactile information conversion device, tactile information conversion method, and tactile information conversion program |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2016-077692 | 2016-04-07 | ||
JP2016077692 | 2016-04-07 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/152,982 Continuation US10739858B2 (en) | 2016-04-07 | 2018-10-05 | Tactile information conversion device, tactile information conversion method, and tactile information conversion program |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2017175867A1 true WO2017175867A1 (ja) | 2017-10-12 |
Family
ID=60000417
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2017/014570 WO2017175867A1 (ja) | 2016-04-07 | 2017-04-07 | 触覚情報変換装置、触覚情報変換方法、および、触覚情報変換プログラム |
Country Status (5)
Country | Link |
---|---|
US (2) | US10739858B2 (ja) |
JP (1) | JP6875010B2 (ja) |
KR (1) | KR20180112847A (ja) |
CN (1) | CN108885500A (ja) |
WO (1) | WO2017175867A1 (ja) |
Cited By (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2019198939A (ja) * | 2018-05-18 | 2019-11-21 | Telexistence株式会社 | 触覚センサ |
JP2019536189A (ja) * | 2016-11-30 | 2019-12-12 | シューシングス インコーポレイテッドShoothings, Inc. | 手指の関節間着用型の指輪タイプのユーザ操作センシング装置 |
KR20200006251A (ko) * | 2018-07-10 | 2020-01-20 | 동의대학교 산학협력단 | 인공 신경망 기반 알고리즘을 이용한 디바이스 모니터링 방법 및 시스템 |
WO2020145386A1 (ja) * | 2019-01-11 | 2020-07-16 | ダイキン工業株式会社 | 環境条件記憶装置、環境条件記憶方法、操作装置、及び環境認識装置 |
JP2021022006A (ja) * | 2019-07-24 | 2021-02-18 | トヨタ紡織株式会社 | 情報処理装置 |
JPWO2020031483A1 (ja) * | 2018-08-08 | 2021-11-18 | ソニーグループ株式会社 | 復号装置、復号方法、プログラム |
JP2021197054A (ja) * | 2020-06-18 | 2021-12-27 | 日本電信電話株式会社 | 触覚提示物、触覚提示装置、触覚提示方法、プログラム |
US11279037B2 (en) * | 2018-05-31 | 2022-03-22 | National University Corporation Nagoya University | Force-sense visualization apparatus, robot, and force-sense visualization program |
WO2023037431A1 (ja) * | 2021-09-08 | 2023-03-16 | 豊田合成株式会社 | 触感提示装置 |
WO2023119582A1 (ja) * | 2021-12-23 | 2023-06-29 | 日本電信電話株式会社 | 見掛け力制御装置、見掛け力提示システム、見掛け力制御方法、およびプログラム |
WO2023189406A1 (ja) * | 2022-03-31 | 2023-10-05 | 株式会社栗本鐵工所 | 出力制御装置、出力制御プログラム及び出力制御方法 |
JP7415139B2 (ja) | 2019-01-11 | 2024-01-17 | ダイキン工業株式会社 | 環境条件記憶装置、環境条件記憶方法、操作装置、及び環境認識装置 |
EP3798590B1 (en) * | 2018-05-22 | 2024-03-20 | Kyocera Corporation | Measurement device and control method for measurement device |
Families Citing this family (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2018143021A1 (ja) * | 2017-02-06 | 2018-08-09 | アルプス電気株式会社 | 触覚呈示装置 |
JP6749429B2 (ja) * | 2017-02-06 | 2020-09-02 | アルプスアルパイン株式会社 | 触覚呈示装置 |
US10627906B2 (en) * | 2017-06-02 | 2020-04-21 | International Business Machines Corporation | Tactile display using microscale electrostatic accelerators |
CN108153020B (zh) * | 2018-01-02 | 2020-08-11 | 京东方科技集团股份有限公司 | 触控结构,触控面板及其制备方法、触控显示装置 |
US10795446B2 (en) * | 2018-04-25 | 2020-10-06 | Seventh Sense OÜ | Portable electronic haptic vision device |
CN109710059A (zh) * | 2018-11-27 | 2019-05-03 | 南京恩诺网络科技有限公司 | 感官信息还原方法、装置及系统 |
CN109710058A (zh) * | 2018-11-27 | 2019-05-03 | 南京恩诺网络科技有限公司 | 触觉信息录制方法及装置、系统 |
CN109739345A (zh) * | 2018-11-27 | 2019-05-10 | 南京恩诺网络科技有限公司 | 触觉还原方法、装置及系统 |
CN109447038A (zh) * | 2018-11-27 | 2019-03-08 | 南京恩诺网络科技有限公司 | 触觉信息估算方法及装置、系统 |
JP7127513B2 (ja) * | 2018-11-30 | 2022-08-30 | トヨタ自動車株式会社 | センサシステムおよびロボットハンド |
CN109782950B (zh) * | 2019-01-04 | 2022-05-17 | Oppo(重庆)智能科技有限公司 | 触摸电极控制方法、装置、存储介质及电子设备 |
WO2020210014A1 (en) * | 2019-04-11 | 2020-10-15 | Massachusetts Institute Of Technology | Sensing and control systems |
KR102265399B1 (ko) * | 2019-06-12 | 2021-06-15 | 주식회사 씨케이머티리얼즈랩 | 스테레오 촉각 제공 장치 |
CN110297546B (zh) * | 2019-07-08 | 2022-05-03 | 合肥工业大学 | 一种用于采集腕部-手指动作同步信号的装置及其标注方法 |
CH716787B1 (it) * | 2019-11-13 | 2022-10-14 | Wintecare Sa | Manipolo attuatore per un dispositivo di stimolazione neuromuscolare e relativo dispositivo di stimolazione neuromuscolare. |
WO2021248010A1 (en) * | 2020-06-06 | 2021-12-09 | Battelle Memorial Institute | High-definition electrical stimulation for enhanced spatial awareness and target alignment in weapon aiming applications |
US20210382545A1 (en) * | 2020-06-06 | 2021-12-09 | Battelle Memorial Institute | High-definition electrical stimulation garment for evoking realistic somatosensation |
WO2022046340A1 (en) | 2020-08-31 | 2022-03-03 | Sterling Labs Llc | Object engagement based on finger manipulation data and untethered inputs |
US20220374649A1 (en) * | 2021-05-20 | 2022-11-24 | Disney Enterprises, Inc. | Face swapping with neural network-based geometry refining |
TWI805134B (zh) * | 2021-12-17 | 2023-06-11 | 長庚大學 | 感光觸覺系統及其感光方法 |
EP4250060A1 (en) * | 2022-03-25 | 2023-09-27 | Nokia Technologies Oy | Apparatus, systems, methods and computer programs for transferring sensations |
WO2024101689A1 (ko) * | 2022-11-10 | 2024-05-16 | 삼성전자주식회사 | 햅틱 피드백을 출력하기 위한 전자 장치 및 그 방법 |
US11751799B1 (en) * | 2023-02-08 | 2023-09-12 | Lanny Leo Johnson | Methods and systems for diagnosing cognitive conditions |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2002132432A (ja) * | 2000-10-30 | 2002-05-10 | Toshiba Corp | 多次元触覚入出力装置 |
JP2005519387A (ja) * | 2002-03-06 | 2005-06-30 | アクセレイテ ビジネス ランチ アンド イクスペンション ゲーエムベーハー | 信号を符号化、送信および/または受信する装置 |
JP2011086068A (ja) * | 2009-10-14 | 2011-04-28 | Chugoku Electric Power Co Inc:The | 状態再現システム |
WO2012073733A1 (ja) * | 2010-12-03 | 2012-06-07 | 日本電気株式会社 | 提示装置、コミュニケーションシステム、および感性提示方法 |
JP2013091114A (ja) * | 2011-10-05 | 2013-05-16 | Kyokko Denki Kk | インタラクション操作システム |
JP2015219887A (ja) * | 2014-05-21 | 2015-12-07 | 日本メクトロン株式会社 | 電気触覚提示装置 |
Family Cites Families (25)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6016385A (en) * | 1997-08-11 | 2000-01-18 | Fanu America Corp | Real time remotely controlled robot |
US6126373A (en) * | 1997-12-19 | 2000-10-03 | Fanuc Usa Corporation | Method and apparatus for realtime remote robotics command |
US20020133264A1 (en) * | 2001-01-26 | 2002-09-19 | New Jersey Institute Of Technology | Virtual reality system for creation of design models and generation of numerically controlled machining trajectories |
US20040077975A1 (en) * | 2002-10-22 | 2004-04-22 | Zimmerman Jeffrey C. | Systems and methods for motion analysis and feedback |
US8059089B2 (en) * | 2004-05-25 | 2011-11-15 | Sony Computer Entertainment Inc. | Input device and method, and character input method |
US9900669B2 (en) * | 2004-11-02 | 2018-02-20 | Pierre Touma | Wireless motion sensor system and method |
US20060106369A1 (en) * | 2004-11-12 | 2006-05-18 | Desai Jaydev P | Haptic interface for force reflection in manipulation tasks |
CN101021745A (zh) * | 2006-12-14 | 2007-08-22 | 东南大学 | 基于多维刺激的彩色射流触觉图文显示装置及显示方法 |
US8952888B2 (en) * | 2008-05-09 | 2015-02-10 | Koninklijke Philips N.V. | Method and system for conveying an emotion |
US8390438B2 (en) * | 2008-09-24 | 2013-03-05 | St. Jude Medical, Atrial Fibrillation Division, Inc. | Robotic catheter system including haptic feedback |
JP2010102560A (ja) * | 2008-10-24 | 2010-05-06 | National Institute Of Information & Communication Technology | 把持感覚提示装置 |
JP2010287221A (ja) | 2009-05-11 | 2010-12-24 | Univ Of Tokyo | 力覚提示装置 |
KR101746453B1 (ko) * | 2010-04-12 | 2017-06-13 | 삼성전자주식회사 | 실감 효과 처리 시스템 및 방법 |
WO2012174406A1 (en) * | 2011-06-15 | 2012-12-20 | University Of Washington | Methods and systems for haptic rendering and creating virtual fixtures from point clouds |
US9026250B2 (en) * | 2011-08-17 | 2015-05-05 | Harris Corporation | Haptic manipulation system for wheelchairs |
CN102609094A (zh) * | 2012-02-24 | 2012-07-25 | 东南大学 | 电振双模多维触觉刺激器 |
KR102161723B1 (ko) * | 2013-05-07 | 2020-10-06 | 삼성디스플레이 주식회사 | 햅틱 패널 및 이를 포함하는 표시 장치 |
KR102171873B1 (ko) * | 2013-05-08 | 2020-10-29 | 삼성전자주식회사 | 햅틱 글로브 및 수술로봇 시스템 |
JP2017506169A (ja) * | 2014-02-20 | 2017-03-02 | マーク オレイニク | ロボット調理キッチン内の食品調製のための方法及びシステム |
US9330547B2 (en) | 2014-05-20 | 2016-05-03 | Immersion Corporation | Haptic effect authoring tool based on a haptification model |
US9971406B2 (en) * | 2014-12-05 | 2018-05-15 | International Business Machines Corporation | Visually enhanced tactile feedback |
GB2533572A (en) * | 2014-12-22 | 2016-06-29 | Nokia Technologies Oy | Haptic output methods and devices |
US10296086B2 (en) * | 2015-03-20 | 2019-05-21 | Sony Interactive Entertainment Inc. | Dynamic gloves to convey sense of touch and movement for virtual objects in HMD rendered environments |
US20160321880A1 (en) * | 2015-04-28 | 2016-11-03 | Immersion Corporation | Systems And Methods For Tactile Guidance |
US20170144312A1 (en) * | 2015-11-25 | 2017-05-25 | Umm Al-Qura University | Robotic surgical finger and controller with tactile feedback and robotic hand using the same |
-
2017
- 2017-04-07 JP JP2018510679A patent/JP6875010B2/ja active Active
- 2017-04-07 CN CN201780021222.3A patent/CN108885500A/zh active Pending
- 2017-04-07 KR KR1020187026642A patent/KR20180112847A/ko not_active IP Right Cessation
- 2017-04-07 WO PCT/JP2017/014570 patent/WO2017175867A1/ja active Application Filing
-
2018
- 2018-10-05 US US16/152,982 patent/US10739858B2/en active Active
-
2020
- 2020-07-28 US US16/941,024 patent/US11281296B2/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2002132432A (ja) * | 2000-10-30 | 2002-05-10 | Toshiba Corp | 多次元触覚入出力装置 |
JP2005519387A (ja) * | 2002-03-06 | 2005-06-30 | アクセレイテ ビジネス ランチ アンド イクスペンション ゲーエムベーハー | 信号を符号化、送信および/または受信する装置 |
JP2011086068A (ja) * | 2009-10-14 | 2011-04-28 | Chugoku Electric Power Co Inc:The | 状態再現システム |
WO2012073733A1 (ja) * | 2010-12-03 | 2012-06-07 | 日本電気株式会社 | 提示装置、コミュニケーションシステム、および感性提示方法 |
JP2013091114A (ja) * | 2011-10-05 | 2013-05-16 | Kyokko Denki Kk | インタラクション操作システム |
JP2015219887A (ja) * | 2014-05-21 | 2015-12-07 | 日本メクトロン株式会社 | 電気触覚提示装置 |
Non-Patent Citations (2)
Title |
---|
ANDO ET AL: "A study of the Nail-Mounted Tactile display for Augmented Reality System", INFORMATION AND COMMUNICATION ENGINEERS, vol. J87-D-II, no. 11, 1 November 2004 (2004-11-01), pages 2025 - 2033 * |
MANABU CHIKAI: "Atarashii Shokkaku Teiji Sochi no Kenkyu", 25 March 2014 (2014-03-25), pages 1 - 42, Retrieved from the Internet <URL:http://hdl.handle.net/10649/719> * |
Cited By (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2019536189A (ja) * | 2016-11-30 | 2019-12-12 | シューシングス インコーポレイテッドShoothings, Inc. | 手指の関節間着用型の指輪タイプのユーザ操作センシング装置 |
JP2019198939A (ja) * | 2018-05-18 | 2019-11-21 | Telexistence株式会社 | 触覚センサ |
EP3798590B1 (en) * | 2018-05-22 | 2024-03-20 | Kyocera Corporation | Measurement device and control method for measurement device |
US11279037B2 (en) * | 2018-05-31 | 2022-03-22 | National University Corporation Nagoya University | Force-sense visualization apparatus, robot, and force-sense visualization program |
KR20200006251A (ko) * | 2018-07-10 | 2020-01-20 | 동의대학교 산학협력단 | 인공 신경망 기반 알고리즘을 이용한 디바이스 모니터링 방법 및 시스템 |
KR102094002B1 (ko) | 2018-07-10 | 2020-04-24 | 주식회사 콜라비팀 | 인공 신경망 기반 알고리즘을 이용한 디바이스 모니터링 방법 및 시스템 |
JPWO2020031483A1 (ja) * | 2018-08-08 | 2021-11-18 | ソニーグループ株式会社 | 復号装置、復号方法、プログラム |
WO2020145386A1 (ja) * | 2019-01-11 | 2020-07-16 | ダイキン工業株式会社 | 環境条件記憶装置、環境条件記憶方法、操作装置、及び環境認識装置 |
JP7415139B2 (ja) | 2019-01-11 | 2024-01-17 | ダイキン工業株式会社 | 環境条件記憶装置、環境条件記憶方法、操作装置、及び環境認識装置 |
JP7337579B2 (ja) | 2019-07-24 | 2023-09-04 | トヨタ紡織株式会社 | 情報処理装置 |
JP2021022006A (ja) * | 2019-07-24 | 2021-02-18 | トヨタ紡織株式会社 | 情報処理装置 |
JP2021197054A (ja) * | 2020-06-18 | 2021-12-27 | 日本電信電話株式会社 | 触覚提示物、触覚提示装置、触覚提示方法、プログラム |
JP7380439B2 (ja) | 2020-06-18 | 2023-11-15 | 日本電信電話株式会社 | 触覚提示物、触覚提示装置、触覚提示方法、プログラム |
WO2023037431A1 (ja) * | 2021-09-08 | 2023-03-16 | 豊田合成株式会社 | 触感提示装置 |
WO2023119582A1 (ja) * | 2021-12-23 | 2023-06-29 | 日本電信電話株式会社 | 見掛け力制御装置、見掛け力提示システム、見掛け力制御方法、およびプログラム |
WO2023189406A1 (ja) * | 2022-03-31 | 2023-10-05 | 株式会社栗本鐵工所 | 出力制御装置、出力制御プログラム及び出力制御方法 |
Also Published As
Publication number | Publication date |
---|---|
CN108885500A (zh) | 2018-11-23 |
US20190064927A1 (en) | 2019-02-28 |
US10739858B2 (en) | 2020-08-11 |
KR20180112847A (ko) | 2018-10-12 |
US20200356176A1 (en) | 2020-11-12 |
US11281296B2 (en) | 2022-03-22 |
JP6875010B2 (ja) | 2021-05-19 |
JPWO2017175867A1 (ja) | 2019-02-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2017175867A1 (ja) | 触覚情報変換装置、触覚情報変換方法、および、触覚情報変換プログラム | |
JP6703688B2 (ja) | 触覚情報変換装置、触覚情報変換方法、および、触覚情報変換プログラム、並びに、素子配置構造体 | |
Jones | Haptics | |
Klatzky et al. | The skin and its receptors 148 pathways to cortex and major cortical areas | |
Bach-y-Rita et al. | Seeing with the brain | |
Tan et al. | Methodology for maximizing information transmission of haptic devices: A survey | |
Hatzfeld et al. | Haptics as an interaction modality | |
Asamura et al. | Necessary spatial resolution for realistic tactile feeling display | |
Hu et al. | StereoPilot: A wearable target location system for blind and visually impaired using spatial audio rendering | |
Kyung et al. | A novel interactive mouse system for holistic haptic display in a human-computer interface | |
Caldwell et al. | Multi-modal cutaneous tactile feedback | |
Eid et al. | Soa thresholds for the perception of discrete/continuous tactile stimulation | |
Wilson | Using pressure input and thermal feedback to broaden haptic interaction with mobile devices | |
Sawada | Tactile display using the micro-vibration of shape-memory alloy wires and its application to tactile interaction systems | |
Honda et al. | Development of a novel slip haptic display device based on the localized displacement phenomenon | |
Takeda et al. | Tactile actuators using SMA micro-wires and the generation of texture sensation from images | |
Ivani et al. | Vibes: Vibro-inertial bionic enhancement system in a prosthetic socket | |
Zilbershtain-Kra et al. | Active sensory substitution allows fast learning via effective motor-sensory strategies | |
Schmidmaier | Sensory substitution systems | |
Kahol et al. | Neuro-cognitively inspired haptic user interfaces | |
Weissgerber et al. | VisPad: a novel device for vibrotactile force feedback | |
Lederman et al. | TUTORIAL REVIEW haptic perception: a tutorial | |
CN113220117B (zh) | 一种用于人-计算机交互的装置 | |
ガイアーアンドレアス | Development and Implementation of a Tactile Feedback System Using End-to-end Tactile Sensor Data Projections for Wearable Human-Robot Interaction | |
Jones et al. | Lynette Jones |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
ENP | Entry into the national phase |
Ref document number: 20187026642 Country of ref document: KR Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1020187026642 Country of ref document: KR |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2018510679 Country of ref document: JP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17779244 Country of ref document: EP Kind code of ref document: A1 |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 17779244 Country of ref document: EP Kind code of ref document: A1 |