JP2011096171A - Multisensory interaction system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To give a user higher realistic sensation and actual sensation for a virtual object, by presenting four pieces of sensation information to a user by closely linking them. <P>SOLUTION: A multisensory interaction system 10 includes a computer 12 for performing the whole control, and gives a user the sensation, which the virtual object really exists, by integrating the two or more pieces of sensation information and presenting them to a user. A three-dimensional monitor 14 displays the stereoscopic video image of the virtual object. A tactile sensation device 18 receives an operation by a user and gives reaction force generated by touching the virtual object to the user. A headphone 16 and an olfactory sensation display 20 present sound and smell to the user, corresponding to the reaction force from the virtual object, that is, in conjunction with force actively given to the virtual object by the user. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a multisensory interaction system, and more particularly to a multisensory interaction system that gives a user a sense of existence of a virtual object by integrating and presenting a plurality of sensory information.

  An example of a conventional multisensory interaction system is disclosed in Patent Document 1. The system of Patent Literature 1 includes a 3D display unit, a force sense presentation unit, and a sound output unit, and generates a contact sound in real time according to the user's operation. Then, by integrating and presenting the three sensory information of visual, auditory, and tactile senses, the user is given a sense that the virtual object actually exists.

On the other hand, many systems that present olfactory information in addition to audiovisual information have been proposed. For example, Non-Patent Document 1 discloses a system in which when a hand is held over a balloon displayed on a screen, the balloon is broken and the scent is presented accordingly.
JP 2009-205626 A [G06T 17/40] http://www.iwata-kaori.jp/ “Manda Scent Museum”

  Humans acquire information from the outside world by comprehensively using the so-called five senses of vision, hearing, smell, taste and touch. Therefore, in order to give the user a sense that the virtual object actually exists, it is desirable to present more sensory information in close cooperation. However, since the technique of Patent Document 1 simply presents three sensory information of vision, hearing, and tactile sense, it is not sufficient in terms of realism and realism. Further, in the technique of Non-Patent Document 1, tactile information is not presented, and the user cannot be given a sense that a virtual object is actually present.

  Therefore, the main object of the present invention is to provide a novel multisensory interaction system.

  Another object of the present invention is to provide a multisensory interaction system capable of giving a user a higher sense of reality and realism.

  The present invention employs the following configuration in order to solve the above problems. Note that reference numerals in parentheses and supplementary explanations indicate correspondence with embodiments described later in order to help understanding of the present invention, and do not limit the present invention.

  1st invention is the multisensory interaction system which gives a user the sense that the virtual object exists by unifying and presenting a plurality of sensory information, visual information presenting means for presenting the video of the virtual object, It is a multisensory interaction system including tactile information presenting means for accepting an operation by a user and presenting a reaction force from a virtual object, and an olfactory information presenting means for presenting an odor according to the reaction force.

  In the first invention, the multisensory interaction system (10) includes visual information presenting means (12, 14, 22, 24, 26), and the visual information presenting means provides a video of a virtual object, that is, visual information to the user. Present. The tactile information presentation means (12, 18, 30) includes an operation unit (30) that accepts a user operation, and the operation unit is changed to a virtual object according to the position of the operation unit and the physical properties of the virtual object (target object). The reaction force generated by touching is calculated, and the reaction force, that is, tactile information is given to the user who operates the operation unit. The olfactory information presenting means (12, 20, 32) presents an odor, that is, olfactory information to the user according to the reaction force presented by the tactile information presenting means in accordance with the user's active movement. For example, when the reaction force exceeds a critical value, the scent is presented to the user, or the scent is presented to the user by changing the intensity of the scent or the length of time in proportion to the magnitude of the reaction force.

  According to the first invention, in addition to the visual information and the tactile information, the olfactory information is presented to the user in an interlocked manner, so that a sense that a virtual object is actually present is given to the user with a high sense of reality and realism. Can do. In addition, since the olfactory information is presented in accordance with the user's active action, the user can easily become aware of the olfactory information, and the effect of enhancing the sense of reality by the olfactory information becomes more effective.

  A second invention is dependent on the first invention, and the visual information presentation means presents a stereoscopic image of the virtual object.

  In the second invention, the visual information presenting means (12, 14, 22, 24, 26) presents a stereoscopic image of the virtual object to the user. That is, visual information is presented to the user in a state closer to the real world. Therefore, a higher sense of reality and realism can be given to the user.

  A third invention is dependent on the first or second invention, and further includes auditory information presenting means for presenting sound according to reaction force.

  In 3rd invention, an auditory information presentation means (12, 16) is further provided. The auditory information presenting means presents sound, that is, auditory information to the user according to the reaction force presented by the tactile information presenting means (12, 18, 30) in accordance with the user's active movement.

  According to the third aspect of the invention, the four sense information of vision, hearing, touch, and smell are presented to the user in association with each other, so that a higher sense of reality and realism can be given to the user.

  The fourth invention is dependent on any one of the first to third inventions, the visual information presenting means changes the shape of the virtual object according to the reaction force, and the olfactory information presenting means is the virtual object. Presents an odor when the shape changes.

  In the fourth invention, the visual information presenting means (12, 14, 22, 24, 26) reacts with the reaction force presented by the tactile information presenting means (12, 18, 30) in accordance with the active movement of the user. In response, the virtual object shape is changed and presented. The olfactory information presenting means (12, 20, 32) presents an odor when the shape of the virtual object is changed, for example, when the virtual object is ruptured or depressed.

  According to the fourth aspect of the invention, the three sensory information of vision, touch, and smell are presented to the user in closer cooperation, so that a higher sense of reality and reality can be given to the user.

  The fifth invention is dependent on any one of the first to fourth inventions, and the visual information presenting means further presents an effect image reminiscent of the generation of an odor.

  In the fifth invention, the visual information presenting means (12, 14, 22, 24, 26) displays an effect image reminiscent of the generation of an odor, such as a radial line flying from a virtual object toward the user. indicate.

  According to the fifth invention, since the user's attention is increased by displaying the effect image, the user can feel the olfactory information more strongly, and the effect of enhancing the sense of reality by the olfactory information is more effective. Become.

  According to the first invention, in addition to the visual information and the tactile information, the olfactory information is presented to the user in an interlocked manner, so that a sense that a virtual object is actually present is given to the user with a high sense of reality and realism. Can do. In addition, since the olfactory information is presented in accordance with the user's active action, the user can easily be aware of the odor, and the effect of enhancing the sense of reality by the olfactory information becomes more effective.

  Further, according to the third invention, since the four sensory information of visual, auditory, tactile and olfactory senses are closely linked to each other and presented to the user, a higher sense of reality and realism can be given to the user.

  The above object, other objects, features, and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

It is an illustration figure which shows a mode that the user is experiencing the multisensory interaction system of one Example of this invention. It is a block diagram which shows the structure of the multisensory interaction system of one Example of this invention. FIG. 3 is an illustrative view showing a memory map of a memory of the computer of FIG. 2. It is an illustration figure for demonstrating operation | movement of the multisensory interaction system of FIG. It is a flowchart which shows an example of the whole process which CPU of the computer of FIG. 2 performs. It is a flowchart which shows another example of the whole process which CPU of the computer of FIG. 2 performs.

  Referring to FIG. 1, a multisensory interaction system 10 (hereinafter simply referred to as “system 10”) according to an embodiment of the present invention includes a three-dimensional monitor 14, headphones 16, a tactile device 18, and an olfactory display 20. By integrating and presenting to the user the four sensory information of vision, hearing, touch and smell, the user is given a sensation that the virtual object is actually present.

  FIG. 2 is a block diagram showing the configuration of the system 10. As shown in FIG. 2, the system 10 includes a computer 12 that performs overall control, and the computer 12 is connected to a three-dimensional monitor 14, headphones 16, a tactile device 18, an olfactory display 20, and the like.

  The three-dimensional monitor 14 presents a stereoscopic image (visual information) of a virtual object (target object) to the user, and includes a display 22, a half mirror 24, and liquid crystal shutter glasses 26. In the three-dimensional monitor 14, a virtual object image 22 a (see FIG. 1) is displayed on the display 22 based on a signal from the computer 12, and the image 22 a is reflected by the half mirror 24. Further, the liquid crystal shutter glasses 26 are controlled so as to generate a left / right parallax based on a signal from the computer 12 sent via the emitter 28. A user wearing the liquid crystal shutter glasses 26 can view a stereoscopic image of the virtual object through the half mirror 24.

  The headphones 16 output sound (auditory information) based on signals from the computer 12. However, the sound may be presented to the user by a speaker instead of the headphone 16 or together with the headphone 16.

  The tactile device 18 is interactively linked to the computer 12 and presents the reaction force (feeling, tactile information) to the user who has touched the virtual object. The tactile device 18 includes a pen-type operation unit 30 that accepts an operation by a user (a portion other than the operation unit 30 is omitted in FIG. 1). The operation unit 30 includes a position and a physical property (hardness of a virtual object). A reaction force according to the sheath texture) is given. A user who operates the operation unit 30 can artificially touch the virtual object and can apply an external force to the virtual object. As the tactile device 18, for example, Phantom (registered trademark) Premium 1.5 can be used. However, the type of the tactile device 18 is not limited to this, and for example, a tactile device having a globe-type operation unit may be used.

  The olfactory display 20 presents a scent (fragrance, olfactory information) to the user based on a signal from the computer 12 sent via the controller 32. The controller 32 includes a PIC (Peripheral Interface Controller) microcomputer 34 and a MOS-FET 36 for driving the olfactory display 20 in accordance with an instruction signal sent from the computer 12, and the olfactory display 20 follows the instruction from the controller 32. Smell is emitted in a space-time controllable manner. As the olfactory display 20, a known one can be used. For example, it is possible to adopt a microcapsule type that breaks microcapsules encapsulating fragrance to generate fragrance and transports the fragrance with a blower fan, or releases the fragrance filled in the tank from the minute holes of the tank It is also possible to adopt an ink jet method.

  The computer 12 performs overall control of the system 10 as described above, and functions as a control device for interlocking the three-dimensional monitor 14, the headphones 16, the tactile device 18, and the olfactory display 20. The computer 12 is a general-purpose computer such as a personal computer, and includes a CPU 38, a memory 40, a display device (not shown) such as a monitor, and an input device (not shown) such as a keyboard and a mouse. The CPU 38 performs overall control of the operation of the computer 12 (system 10), and the memory 40 stores data and programs for the CPU 38 to perform processing, and is used as a working memory, a register, and the like.

  Specifically, as shown in FIG. 3, the memory 40 includes a data storage unit 50 and a program storage unit 52. The data storage unit 50 includes shape data 54 representing the three-dimensional shape of the virtual object, physical property data 56 representing the hardness (softness), brittleness and texture of the virtual object, various sound data 58, and other necessary data. Data is stored as appropriate. The program storage unit 52 also has a visual program 60 for presenting visual information, a tactile program 62 for presenting tactile information, an auditory program 64 for presenting auditory information, and an olfactory sense for presenting olfactory information. The program 66 and other necessary programs are stored as appropriate.

  Specifically, the visual program 60 is a program for generating a stereoscopic image of a virtual object based on the shape data 54 and the like. In addition, when the operation unit 30 comes into contact with the virtual object, it calculates how the virtual object deforms (changes state) based on the position of the operation unit 30, the shape data 54, the physical property data 56, and the like, This is a program for generating a stereoscopic image of the virtual object.

  The tactile sense program 62 is a program for calculating the reaction force (feel) that the virtual object gives to the operation unit 30 (that is, the user) based on the position of the operation unit 30 and the physical property data 56 and the like.

  The auditory program 64 is obtained from the sound data 58 of the data storage unit 50 according to the user's action on the virtual object performed using the tactile device 18, that is, according to the reaction force from the virtual object or the state of the virtual object. This is a program for selecting appropriate sound data.

  The olfactory program 66 releases an odor from the olfactory display 20 in accordance with the user's action on the virtual object performed using the tactile device 18, that is, in accordance with the reaction force from the virtual object or the state of the virtual object. This is a program for generating the instruction signal.

  In the system 10 having such a configuration, the stereoscopic image of the virtual object is presented to the user by the three-dimensional monitor 14. For example, the user can select an arbitrary object from a plurality of objects, and the stereoscopic image is displayed using the object selected by the user as a virtual object. And a user's operation is received by the tactile device 18, and the reaction force according to the operation is given to a user. Specifically, the haptic device 18 measures the position of the operation unit 30 and outputs the data (position information) to the computer 12. The data output to the computer 12 is processed by a program in the computer 12 together with the physical property data 56 of the virtual object stored in the memory 40, and then fed back to the tactile device 18, where the reaction force according to the state of the virtual object Is given to the operation unit 30. As a result, the user who operates the operation unit 30 can obtain a sense of actually touching the virtual object, and feels the hardness (softness), weight, and surface roughness of the virtual object. be able to.

  Further, in the system 10, the tactile device 18 is used as a trigger when the smell is presented from the olfactory display 20, and the presentation of the odor from the olfactory display 20 is controlled according to the operation state of the tactile device 18 by the user. . Specifically, processing is performed inside the computer 12 according to the user's tactile action (action) on the virtual object performed using the tactile device 18 (or according to the state of the virtual object that changes according to the tactile action). The indicated instruction signal is output to the controller 32. The controller 32 controls the olfactory display 20 via the MOS-FET 36 according to a program incorporated in the PIC microcomputer 34. As a result, the user can feel a touch (reaction force) of touching the virtual object presented by the haptic device 18 and an odor suitable for the state (shape) of the virtual object presented by the three-dimensional monitor 14.

  Similarly, in the system 10, the tactile device 18 is used as a trigger when presenting sound from the headphones 16, and the presentation of sound from the headphones 16 is controlled according to the operation state of the tactile device 18. Accordingly, the user can hear a sound suitable for the touch of the virtual object and the state of the stereoscopic video of the virtual object.

  In this way, the system 10 organically links the four sensory information of visual, auditory, tactile, and olfactory information to the user and presents the user with a sensation as if the virtual object actually exists. To give.

  Hereinafter, the operation of the system 10 will be described with a specific example of a virtual object. For example, assume a case in which the object presented as a virtual object is a balloon and the system 10 can simulate the act of breaking a balloon containing a scent inside. First, as shown in FIG. 4A, a three-dimensional image of a balloon is displayed on the three-dimensional monitor 14. At this time, when the user operates the operation unit 30 to bring the operation unit 30 close to the balloon three-dimensional image, the three-dimensional image of the operation unit 30 is also displayed on the three-dimensional monitor 14.

  Next, when the user brings the operation unit 30 closer to the balloon and the operation unit 30 and the balloon come into contact with each other as shown in FIG. 4B, the touch such as the smoothness, softness, and elasticity unique to the balloon is obtained. 18 to the user. At this time, a reaction force corresponding to the contact state (positional relationship) between the operation unit 30 and the balloon is given to the user, and a state in which the balloon is deformed is displayed. At the same time, the headphone 16 outputs a sound such as a contact sound between the operation unit 30 and the balloon, a sound of the balloon, or an appropriate sound effect.

  When the user applies an external force stronger than a predetermined value to the balloon against the reaction force applied by the tactile device 18 (that is, when the reaction force reaches a critical value at which the deformation of the balloon is limited), FIG. As shown in C), the balloon bursting is displayed and the feeling of the balloon given to the user is lost. At the same time, a balloon burst sound is output from the headphones 16, and an odor assumed to be confined in the balloon is actually emitted from the olfactory display 20.

  In this way, by presenting the four sensory information in conjunction with the user, the user receives a sensation as if a virtual balloon actually exists, and the pseudo experience of breaking the balloon is accompanied by olfactory information. And experience it with a high sense of realism.

  When presenting an odor from the olfactory display 20, as shown in FIG. 4D, an effect image that reminds the generation of the odor may be displayed on the three-dimensional monitor 14. For example, a radial point or line coming from the ruptured balloon toward the user may be displayed. As a result, the user knows that something is coming from the ruptured balloon and his attention is increased (conscious), so that the user can feel the smell more strongly.

  When the virtual object is a balloon, when the user gives an external force stronger than a predetermined value to the balloon, that is, when the reaction force by the tactile device 18 reaches a critical value, the olfactory display 20 presents an odor. However, the present invention is not limited to this, and the presentation of the scent by the olfactory display 20 can be performed linearly or stepwise according to the magnitude of the reaction force by the tactile device 18 or the like. For example, when the virtual object is a lemon, when a physical operation such as squeezing or pushing the lemon by the tactile device 18 is performed, an odor display by the olfactory display 20 is performed according to the operation state such as the squeezing condition. It is good to add strength and weakness to the time. For example, when the user's squeezing degree (pressing condition) of lemon is small, the air volume of the blower fan is reduced and a weak smell is presented for a short time. It is better to present a strong smell for a long time with a large.

  Similarly, depending on the magnitude of the reaction force by the tactile device 18 or the like (for example, in proportion), the presentation of sound from the headphones 16 can be made strong (large or small) or lengthy.

  Next, an example of the operation of the system 10 as described above will be described using a flowchart. Specifically, the CPU 38 of the computer 12 shown in FIG. 2 executes the entire process according to the flowchart shown in FIG. In the flowchart shown in FIG. 5, a specific example will be described assuming that the object presented as a virtual object is a balloon (see FIG. 4).

  With reference to FIG. 5, CPU38 of the computer 12 displays the three-dimensional image of a balloon by step S1. That is, a signal for generating a stereoscopic image of a virtual object based on the shape data 54 or the like is output to the display 22 of the three-dimensional monitor 14 and the liquid crystal shutter glasses 26 (emitter 28), and the stereoscopic image of the balloon is output to the three-dimensional monitor 14. Is displayed.

  In the next step S3, it is determined whether or not the operation unit 30 is in contact with the balloon. That is, it is determined whether the balloon and the operation unit 30 are in contact with each other based on the position information of the operation unit 30 input from the tactile device 18 and the shape data 54. If “NO” in the step S3, that is, if the balloon and the operation unit 30 are not in contact with each other, the process proceeds to a step S5. In step S5, it is determined whether or not the entire process is completed. For example, the determination is made based on whether an end command from the user is input from the input device of the computer 12. If “NO” in the step S5, that is, if the entire process is not finished, the process returns to the step S3. On the other hand, if “YES” in the step S5, for example, if an end command is input from the user, the entire processing is ended.

  If “YES” in the step S3, that is, if the balloon and the operation unit 30 are in contact with each other, the process proceeds to a step S7. In step S <b> 7, the reaction force that the balloon applies to the operation unit 30 is calculated. That is, the magnitude of the reaction force that the balloon gives to the operation unit 30 (that is, the user) is calculated based on the position information of the operation unit 30 and the physical property data 56 and the like.

  In the next step S9, it is determined whether or not the reaction force calculated in step S7 is within a critical value. That is, it is determined whether or not an external force enough to break the balloon is given to the balloon by the user.

  If “YES” in the step S9, that is, if the reaction force is within the critical value, the process proceeds to a step S11. In step S11, the stereoscopic image of the balloon is deformed, a reaction force is applied to the operation unit 30, and a contact sound is output. That is, how the balloon is deformed according to the reaction force is calculated, a signal for displaying the deformed balloon stereoscopic image is output to the three-dimensional monitor 14, and the balloon balloon deformed to the three-dimensional monitor 14 is output. Display an image. In addition, a signal for operating with the calculated reaction force is output to the haptic device 18, and a reaction force is applied to the user from the operation unit 30 of the haptic device 18. Further, a contact sound corresponding to the reaction force is selected or synthesized from the sound data 58, a signal for outputting the contact sound is output to the headphones 16, and the contact sound corresponding to the reaction force is output from the headphones 16. When the process of step S11 ends, the process returns to step S3.

  On the other hand, if “NO” in the step S9, that is, if the reaction force exceeds the critical value, the process proceeds to a step S13. In step S13, the balloon is ruptured, a burst sound is output, and an odor is presented. That is, a signal for displaying the state of the balloon bursting is output to the three-dimensional monitor 14 and the state of the balloon bursting is displayed on the three-dimensional monitor 14. Further, a plosive sound is selected or synthesized from the sound data 58, a signal for outputting the plosive sound is output to the headphones 16, and a plosive sound is output from the headphones 16. Further, an instruction signal for releasing an odor from the olfactory display 20 is generated and output to the controller 32. The controller 32 controls the olfactory display 20 in accordance with the instruction signal, and releases the odor from the olfactory display 20. When the process of step S13 ends, the entire process ends.

  Next, as another specific example, a case where the target object presented as a virtual object is a lemon will be described with reference to the flowchart shown in FIG. Also in this case, the CPU 38 of the computer 12 shown in FIG. 2 executes the entire process according to the flowchart shown in FIG. Note that the processing in step S21-27 is the same as the processing in step S1-7 in FIG. 5 except that the virtual object is a lemon, and a description thereof will be omitted.

  Referring to FIG. 6, CPU 38 of computer 12 determines in step S29 whether or not the reaction force calculated in step S27 is greater than or equal to a threshold value for deforming lemon. That is, it is determined whether or not an external force enough to deform the lemon is given to the lemon by the user. Here, in order to present the sound effect and scent by using the deformation of the lemon, the reaction force threshold at which the lemon starts to deform is the same as the reaction force threshold at which the sound effect or scent starts to be presented. However, these threshold values may be set separately for video, sound, and smell.

  If “NO” in the step S29, that is, if the reaction force is within the threshold value for deforming the lemon, the process proceeds to the step S31. In step S31, a reaction force is applied to the operation unit 30. That is, a signal for operating with the calculated reaction force is output to the haptic device 18, and a reaction force is applied to the user from the operation unit 30 of the haptic device 18. When the process of step S31 ends, the process returns to step S23.

  On the other hand, if “YES” in the step S29, that is, if the reaction force exceeds a threshold value for deforming the lemon, the process proceeds to a step S33. In step S33, the lemon is deformed, a reaction force is applied to the operation unit 30, a sound effect is output, and an odor with a strength proportional to the magnitude of the reaction force is presented. That is, a signal for displaying the deformed lemon stereoscopic image is output to the three-dimensional monitor 14, a signal for operating with the calculated reaction force is output to the tactile device 18, and a signal for outputting a sound effect is output. Output to the headphones 16. In addition, the intensity of the odor (adjusted according to the intensity of the air flow of the blower fan, for example) is determined in proportion to the magnitude of the reaction force (or stepwise), and an instruction signal for presenting the odor of that intensity is generated. To the controller 32. Also, with or without the intensity of the odor, or instead, the controller determines the length of time to present the odor in proportion to the magnitude of the reaction force, and generates an instruction signal for presenting the odor for that length of time. 32 may be output. Note that when outputting a sound effect, the volume or the type of sound may be changed in proportion to the magnitude of the reaction force. When the process of step S33 ends, the process returns to step S23.

  According to this embodiment, the auditory information and the olfactory information are presented in conjunction with the tactile information presented by the user acting on the visual information, that is, the four sensory information is closely linked to the user. Therefore, it is possible to give the user a higher sense of reality and realism with respect to the virtual object, and form a realistic virtual environment. In particular, the olfactory information is presented in conjunction with the tactile information, that is, the scent is presented in accordance with the user's active action, so the user is more conscious of the scent, and the effect of enhancing the sense of reality by the scent is more effective It will be a thing. Therefore, the system 10 of this embodiment can be suitably used for an environment where efficient education is possible by active operation, for example, a simulated experience of touching an important cultural property in a museum or a simulator for dangerous equipment operation training. .

  In the above-described embodiment, the four sensory information is integrated and presented to the user. However, depending on the object, even if a person physically works in real space (even if force is applied), a sound is generated. Since some objects do not occur, presentation of sound information may be omitted when such an object is a virtual object. However, even when no sound is actually generated, by adding a sound effect, it is possible to enhance the sense of reality and realism of the virtual object. Further, even when sound is actually generated, a sound effect different from the actual sound can be presented.

  In the above-described embodiment, a stereoscopic image of a virtual object is presented as visual information in order to give a higher sense of reality and realism to the user. However, a planar image (2D image) is presented to the user. It doesn't matter.

  Furthermore, in the above-described embodiment, the scent is presented from the olfactory display when the reaction force from the virtual object exceeds a critical value or threshold, but the present invention is not limited to this. For example, it is possible to present an odor when the reaction force is generated, that is, when the operation unit 30 (user) touches the virtual object. Further, the scent may be presented in accordance with an operation of rubbing the virtual object instead of an operation of pressing the virtual object. For example, when a virtual object is rubbed violently, an image of fire coming out of the virtual object is displayed, and an odor or sound is presented in accordance with the fire.

  In the above-described embodiment, the case where one virtual object is displayed on the three-dimensional monitor 14 is illustrated, but a plurality of virtual objects can be displayed on the three-dimensional monitor 14 at the same time. For example, when two balloons of different sizes are displayed and the large balloon is touched, the reaction force is large. When the balloon is broken, a large popping sound and a strong smell are presented. When the small balloon is touched, the reaction force is displayed. Is small, and when it is broken, a small pop and a weak smell may be presented.

DESCRIPTION OF SYMBOLS 10 ... Multisensory interaction system 12 ... Computer 14 ... Three-dimensional monitor 16 ... Headphone 18 ... Tactile device 20 ... Olfactory display 30 ... Operation part 38 ... CPU of computer
40: Computer memory

Claims (5)

A multisensory interaction system that gives users a sense of the existence of virtual objects by integrating and presenting multiple sensory information,
Visual information presentation means for presenting an image of the virtual object;
A multisensory interaction system comprising: tactile information presenting means for accepting an operation by the user and presenting a reaction force from the virtual object; and an olfactory information presenting means for presenting an odor according to the reaction force.   The multisensory interaction system according to claim 1, wherein the visual information presentation unit presents a stereoscopic image of the virtual object.   The multisensory interaction system according to claim 1, further comprising auditory information presentation means for presenting sound according to the reaction force. The visual information presenting means changes and presents the shape of the virtual object according to the reaction force,
The multisensory interaction system according to any one of claims 1 to 3, wherein the olfactory information presentation unit presents the odor when the shape of the virtual object changes.   The multisensory interaction system according to any one of claims 1 to 4, wherein the visual information presentation unit further presents an effect image reminiscent of the generation of the odor.

Images