KR102385834B1 - Five senses hardware transmission method - Google Patents

Abstract

According to the present disclosure, a five senses hardware transmission method is to operate a five sense generating device including a visual module, a taste module, an olfactory module, a tactile module, and an auditory module. The method includes: primarily setting an operating value including an operating time for at least one of five senses necessary for sensory transmission, based on information on a section, which is necessary for transmitting at least one of the five senses, in a specific image, and information on a sense corresponding to the section; and secondary operating at least one of the five sense generating devices by applying a signal for an operation based on an operating value of the at least one of the five sense generating device, which generates a sense corresponding to the section, based on the information on the sense corresponding to the section.

Description

Five senses hardware transmission method

The present invention relates to a five sense hardware delivery method, and more particularly, to a five sense hardware delivery method for delivering virtual five senses to a wearer using a five sense generating device that generates virtual five sense senses.

In general, virtual reality, augmented reality, mixed reality, etc. refer to a virtual environment similar to a real environment created by artificial technology such as a computer.

The virtual environment experience system is to immerse in the virtual environment and experience the virtual environment by illusion. Recently, games and shopping systems using the virtual environment have been actively developed. When experiencing the virtual environment, the reality of how similar the virtual environment is to the real environment is most important.

Most virtual environment experience systems use a head mounted display to provide visual and auditory information to a user to immerse themselves in the virtual environment. Accordingly, depending on the user, the feeling of experiencing the virtual environment may be insignificant, or if the experience of the same virtual environment is repeated, the reality may drop sharply.

Therefore, there is a need for a method that can additionally provide not only visual and auditory information, but also olfactory, tactile, and gustatory information in order to increase the reality when experiencing the virtual environment.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a five sense hardware delivery method capable of delivering virtual five senses to a wearer using a five sense generating device that generates virtual five senses.

The five sense hardware delivery method according to the present invention relates to a method for operating an apparatus for generating five senses including a visual module, a taste module, a olfactory module, a tactile module, and an auditory module, and includes at least one sense among the five senses in a specific image. a first step of setting an operation value including an operation time for at least one of the five senses requiring sensory transmission based on information on a section requiring transmission and information on a sense corresponding to the section; and a second to be operated by applying a signal for an operation based on the operation value of at least one of the five senses generating devices for generating a sensation corresponding to the section based on information on the sense corresponding to the section It may be characterized in that it comprises a; step.

The first step of the five sense hardware delivery method according to the present invention is to extract an image satisfying a similarity condition by comparing the specific image with a previously stored image, and based on the five sense information corresponding to the extracted image, the section It may be characterized in that it comprises the step of extracting information on and information on the sense corresponding to the section.

In the second step of the method for transmitting the five senses hardware according to the present invention, when the device for generating a sense corresponding to the section is a taste module for generating a taste, a temperature condition, a current condition, and a frequency applied to the taste module By generating an operation signal for the condition, it may be characterized in that the taste module is operated.

The temperature condition, the current condition, and the frequency condition of the five sense hardware transmission method according to the present invention may be characterized in that they are in a range of 0 to 40°C, a range of 0 to 250 μA, and a range of 0 to 800Hz, respectively.

The second step of the five sense hardware delivery method according to the present invention is to generate a specific odor filled in the cartridge of the olfactory module when the device for generating a sense corresponding to the section is a olfactory module for generating a sense of smell It may be characterized in that the olfactory module is operated by generating an operation signal for discharging a substance for a substance, and an operation signal for a heating unit of the olfactory module for heating the cartridge to help the substance diverge.

In the second step of the five sense hardware delivery method according to the present invention, when the device for generating a sense corresponding to the section is a tactile module for generating a tactile sense, the pressure generating unit of the tactile module generates pressure It may be characterized in that the tactile module is operated by generating an operation signal for generating an operation signal and an operation signal enabling generation of vibration by the vibration generating unit of the tactile module.

The tactile module of the five sense hardware delivery method according to the present invention is provided as a haptic glove worn by a wearer, the pressure generating unit is provided on the palm of the haptic glove, and the vibration generating unit is the haptic glove. It may be provided on the back of the hand, and may include a plurality of vibrators capable of controlling the intensity of vibration.

In the second step of the method for transmitting the five senses hardware according to the present invention, when the device for generating a sense corresponding to the section is a tactile module for generating a tactile sense, the operating value is applied to the fan and the heating coil of the tactile module. It may be characterized in that by generating an operation signal based on the fan, the strength and temperature of the artificial wind are adjusted.

According to the five sense hardware delivery method according to the present invention, the virtual five senses can be transmitted to the wearer by using the five sense generating device that generates the virtual five sense senses.

In addition, by extracting whether or not the virtual five senses have occurred with respect to a specific image, it is possible to transmit realistic five senses for the specific image.

1 is a block diagram for explaining a system for transmitting five senses according to an embodiment of the present invention.
FIG. 2 is a side view for explaining the HMD shown in FIG. 1 .
3 is a partially exploded front view for explaining the speaker unit shown in FIG. 2 .
4 is a plan view illustrating the artificial olfactory recognition and delivery device shown in FIG. 1 .
5 is a side view for explaining the artificial olfactory recognition and delivery device shown in FIG. 4 .
6 is a cross-sectional view for explaining the odor generating unit shown in FIG.
7 is a plan view for explaining the rotating plate shown in FIG.
8 is a cross-sectional view for explaining the wind generating unit shown in FIG.
FIG. 9 is a plan view illustrating the artificial taste delivery device shown in FIG. 1 .
10 is a plan view for explaining the transfer unit shown in FIG.
11 is a perspective view illustrating another example of the artificial taste delivery device shown in FIG. 1 .
12 is a plan view illustrating the pressure generating unit and the vibration generating unit shown in FIG. 1 .
13 is a flowchart illustrating an artificial vision and auditory transmission method according to an embodiment of the present invention.
14 is a flowchart illustrating an artificial olfactory recognition and delivery method according to an embodiment of the present invention.
15 is a flowchart illustrating an artificial taste delivery method according to an embodiment of the present invention.
16 is a flowchart illustrating an artificial tactile sense delivery method according to an embodiment of the present invention.
17 is a flowchart illustrating a method for transmitting five sense hardware according to the present invention.

Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings. However, the spirit of the present invention is not limited to the presented embodiments, and those skilled in the art who understand the spirit of the present invention may add, change, delete, etc. other elements within the scope of the same spirit, and may use other degenerative inventions or the present invention. Other embodiments included within the scope of the invention may be easily proposed, but this will also be included within the scope of the invention.

In addition, components having the same function within the scope of the same idea shown in the drawings of each embodiment will be described using the same reference numerals.

The present invention relates to a psychotherapy approach using virtual five-sensory transmission that enables psychological treatment of a wearer wearing the five-sensory generating device using a five-sensory generating device that generates virtual five senses. .

Hereinafter, the principle of generating virtual five senses including sight, hearing, smell, touch, taste, etc. by the five senses generating device will be first described, and then, a method of approaching psychotherapy using the five senses generating device will be described.

1. Five senses generating device

First, the concept of the five sense transmission system 1000 used below is equivalent to the device for generating five senses, and the term for means capable of representing vision is a concept equivalent to the vision module.

In addition, terms for means capable of representing auditory, olfactory, tactile, and gustatory senses are respectively an auditory module, a olfactory module, a tactile module, and a taste module and are equivalent concepts.

1 is a block diagram for explaining a system for transmitting five senses according to an embodiment of the present invention.

Referring to FIG. 1 , the five sense transmission system 1000 provides virtual reality, augmented reality, mixed reality, and the like, similar to a real environment created by artificial technology such as a computer using the content output from the output device 10 . to experience the environment.

The five sense transmission system 1000 may communicate with the output device 10 through a network.

The communication method of the network is not limited, and may include not only a communication method using a communication network (eg, a mobile communication network, wired online, wireless online, broadcasting network) that the network may include, but also short-range wireless communication. For example, the network includes wireless Internet technologies such as Wireless LAN (WLAN) (WiFi), Wireless broadband (Wibro) and/or World Interoperability for Microwave Access (Wimax), Bluetooth (Bluetooth), Near Field Communication (NFC) and / or short-range communication technologies such as RFID (Radio Frequency Identification) CDMA (Code Division Multiple Access), GSM (Global System for Mobile communication), LTE (Long Tern Evolution) and / or mobile communication technologies such as LTE-Advanced, and / Alternatively, it may be implemented as a network based on V2X (Vehicle to Everything) communication technology, such as V2I (Vehicle to Infra).

Setting values for operating hardware are stored in the content output from the output device 10 . The setting values may be stored in the content when the content is produced using separate software.

The hardware means means for representing the five senses of sight, hearing, smell, touch, and taste, and the set values may be signal values for displaying the content in the form of five senses on the hardware.

The five sense delivery system 1000 includes a head mounted display (hereinafter, referred to as HMD) 100 , an artificial olfactory recognition and delivery device 200 , an artificial taste delivery device 300 , and an artificial tactile delivery device 400 . ) may be included. In this case, the artificial olfactory recognition and transmission device 200 may be an artificial wind and temperature and air resistance transmission device 200 .

The HMD 100 may be connected to the output device 10 through the network, and provides an image and sound from the content to the user from the output device 10 .

FIG. 2 is a side view for explaining the HMD shown in FIG. 1 , and FIG. 3 is a partially exploded front view for explaining the speaker unit shown in FIG. 2 .

2 and 3 , the HMD 100 includes a first receiving unit 110 , a display unit 120 , a speaker unit 130 , a first control unit 140 , and a mounting unit 150 . and a mask unit 160 . The speaker unit 130 may be a bone conduction auditory transmission device.

The first receiving unit 110 receives an image setting value and a sound setting value among the setting values from the content of the output device 10 .

The display unit 120 outputs an image using the image setting value. It is provided to cover the user's eyes of the display unit 120 .

The display unit 120 is configured as a pair of left and right, and may sequentially alternately output a left screen and a right screen to implement a stereoscopic image.

The speaker unit 130 outputs sound by using the sound setting value. A pair of the speaker unit 130 may be provided so as to be placed on the user's ear.

The speaker unit 130 may include an ear cup 131 , an ear cup case 133 , a bone conduction speaker 133 , and a headband 135 .

A pair of the ear cup 131, the ear cup case 133, and the bone conduction speaker 133 is provided, respectively.

The ear cup 131 covers the user's ear, and has a receiving groove 132 for accommodating the ear in the center of the inner surface.

The ear cup case 133 is coupled to the outer surface of the ear cup 131 .

The bone conduction speaker 134 is provided in an internal space formed by the coupling of the ear cup 131 and the ear cup case 133 .

The bone conduction speaker 134 extends through the ear cup 131 to the inner surface of the ear cup 131 . At this time, the bone conduction speaker 134 is disposed along the inner circumference of the ear cup 131 .

Since the user's ear is inserted into the receiving groove 132 of the ear cup 131 , and the bone conduction speaker 134 is disposed along the inner circumference of the ear cup 131 , the bone conduction speaker 134 . ) may be in direct contact with the cranial area around the ear or the sound transmission area under the ear. Accordingly, the bone conduction speaker 134 may accurately deliver sound to the user according to the sound setting value.

In addition, the bone conduction speaker 134 generates vibration to transmit the sound. Accordingly, even when the HMD 100 is used for a long time, hearing loss can be reduced.

The headband 135 connects a pair of the ear cup cases 134 . Since the headband 135 can be worn over the user's head, the user can easily wear the speaker unit 130 using the headband 135 .

The headband 135 may be provided with a fastening groove 136 for fastening with the mounting unit 150 .

The first control unit 140 receives the image setting value from the first receiving unit 110 . Thereafter, the first control unit 140 controls the display unit 120 to output a screen corresponding to the image value. Also, the first control unit 140 controls the bone conduction speaker 133 of the speaker unit 130 to output a screen corresponding to the sound value.

The mounting unit 150 is provided to fix the display unit 120 and the speaker unit 130 , and the display unit 120 and the speaker unit 130 are mounted on the user's head.

Specifically, the mounting unit 130 may include a first mounting unit 131 mounted on the user's back head and a second mounting unit 132 mounted on the user's forehead. Since the mounting unit 130 is mounted on the user's back head and forehead, the display unit 120 and the speaker unit 130 can be stably fixed to the user's head.

The mask unit 160 is fixed to the lower surface of the display unit 120 , and is provided integrally with the display unit 120 . The mask unit 160 may block the user's mouth.

The HMD 100 may output the screen and the sound corresponding to the image setting value and the sound setting value received from the output device 10 . Accordingly, when the user experiences the virtual environment, the user can sense sight and hearing at the same time, thereby further enhancing the reality of the virtual environment experience.

4 is a plan view illustrating the artificial olfactory recognition and delivery device shown in FIG. 1 , FIG. 5 is a side view illustrating the artificial olfactory recognition and delivery device shown in FIG. 4 , and FIG. It is a cross-sectional view for explaining the odor generating unit, FIG. 7 is a plan view for explaining the rotating plate shown in FIG. 6 , and FIG. 8 is a cross-sectional view for explaining the wind generating unit shown in FIG. 4 .

4 to 8 , the artificial olfactory recognition and delivery device 200 may be connected to the output device 10 through the network, and includes a second receiving unit 210 , a odor generating unit 220 and A second control unit 230 may be included.

The second receiving unit 210 receives a smell setting value for a smell among the setting values from the content of the output device 10 through the network.

The odor generating unit 220 generates various odors under the control of the second control unit 230 .

The odor generating unit 220 may include a first housing 221 , a plurality of cartridges 224 , and an opening/closing unit 225 .

The first housing 221 has a first inlet 222 and a first outlet 223 through which air passes.

For example, the first housing 221 may have a substantially hollow cylindrical shape, and the first inlet 222 may be disposed on a lower surface and the first outlet 223 may be disposed on an upper surface, respectively.

The cartridges 224 are provided inside the first housing 221 and may be arranged to form a substantially circular shape.

Each of the cartridges 224 may be filled with a material that generates a specific odor. The material may be in a liquid state or in a solid state in the form of chips. Examples of the smell may include a forest scent, a sea scent, and a fruit scent.

The cartridges 224 may be provided to be replaceable, or may be provided to replenish the material. Accordingly, when the material is all used up, the cartridges 224 in which the material is exhausted may be replaced, or the exhausted material may be replenished.

The opening/closing part 225 is provided at the inlet of the cartridges 224 , and opens the inlet so that the material is emitted under the control of the second control unit 230 .

Specifically, the opening/closing unit 225 may include a rotating plate 226 and a motor 227 .

The rotating plate 226 has a disk shape and is disposed to cover the inlets of the cartridges 224 . The rotating plate 226 is provided with openings 226a penetrating up and down, and the inlets of the cartridges 224 may be opened using the openings 226a.

The motor 227 has a rotating shaft connected to the rotating plate 226 and rotates the rotating plate 226 by rotating the rotating shaft. As the rotating plate 226 rotates, the opening 226a may be aligned with the inlet of the cartridges 224 .

When the opening 226a is aligned with any one of the inlets of the cartridges 224 , the inlet of the cartridge 224 is opened and the odor is generated. If the opening 226a is not aligned with the inlet of the cartridges 224, the inlet of the cartridge 224 is blocked and the odor does not occur.

Meanwhile, although not shown, the opening/closing unit 225 may be provided at the inlet of the cartridges 224 to open and close the inlet. In this case, an example of the opening/closing part 225 may be a solenoid valve. In this case, the opening/closing part 225 may open the inlets one by one, or may open a plurality of the inlets at the same time.

The odor generating unit 220 may further include a first fan 228 .

The first fan 228 is provided inside the first housing 221 and rotates under the control of the second control unit 230 . Through the rotation, air is introduced from the first inlet 222 and the air is discharged from the first outlet 223 . Due to the flow of the air, the material emitted from the cartridges 224 may be rapidly ejected through the first outlet 223 .

The odor generating unit 220 may further include heating units 229 .

The heating units 229 are provided to be adjacent to the cartridge 224 inside the first housing 221 , and heat the cartridges 224 under the control of the second control unit 230 . In this case, the heating units 229 may select and heat only the cartridge 224 in which the inlet is opened among the cartridges 224 .

As the heating units 229 are heated, the material may be rapidly and actively radiated. Accordingly, the heating units 229 may help to dissipate the material.

On the other hand, when the material is in a liquid state, each of the cartridges 224 may be provided with injection units (not shown).

The material of the cartridges 224 may be supplied to the injection units by gravity, or may be supplied to the injection units by separate pressure means.

Each of the injection units may inject the material supplied from the cartridges 224 into the air. In this case, each of the spraying units may spray the material under the control of the second control unit 230 .

The second control unit 230 receives the smell setting value from the second receiving unit 210 . Thereafter, the second control unit 230 checks the smell corresponding to the smell setting value, and controls the smell generating unit 220 to generate the smell.

Specifically, the second control unit 230 controls the opening/closing unit 225 to open the inlet of the cartridge 224 that generates a odor corresponding to the odor set value.

In addition, the second control unit 230 controls the first fan 228 to rotate the first fan 228 , and controls the heating units 229 to generate the odor. ) is heated.

On the other hand, the second control unit 230 controls the opening and closing part 225 not to open the inlet of the cartridge 224, and controls the first fan 228 to rotate, so that the odor generating unit ( 220) can also function as a wind generating unit.

The artificial olfactory recognition and delivery device 200 may further include a third receiving unit 240 , a wind generating unit 250 , and a third control unit 260 .

The third receiving unit 240 receives a wind setting value for wind among the setting values from the content of the output device 10 through the network.

The wind generating unit 250 generates wind under the control of the third control unit 260 .

The wind generating unit 250 may include a second housing 251 , a second fan 254 , heating coils 255 , and a coil support member 256 .

The second housing 251 has a second inlet 252 and a second outlet 253 for air to pass therethrough.

For example, the second housing 251 may have a substantially hollow cylindrical shape, and the second inlet 252 may be disposed on a lower surface and the second outlet 253 may be disposed on an upper surface, respectively.

The second fan 254 is provided inside the second housing 251 and rotates under the control of the third control unit 260 . Through the rotation, air is introduced from the second inlet 252 and the air is discharged from the second outlet 253 to generate wind.

The heating coils 255 are provided to be adjacent to the second fan 254 inside the second housing 251 , and heat the air introduced into the second housing 251 . Accordingly, the temperature of the wind generated by the second fan 254 may be increased.

The coil support member 256 is provided inside the second housing 251 and supports the heating coils 255 in a spaced apart state. Since the heating coils 255 may be widely distributed inside the second housing 251 , the heating efficiency of the air may be increased.

The third control unit 260 receives the wind setting value from the third receiving unit 240 . Then, the third control unit 260 checks the wind corresponding to the wind set value, and controls the wind generating unit 250 to generate the wind.

Specifically, the third control unit 260 may control the second fan 254 to adjust the rotation speed of the second fan 254 . Accordingly, the strength of the wind generated by the wind generating unit 250 may be adjusted. In addition, since the third control unit 260 may adjust the temperature of the heating coils 255 , it may adjust the temperature of the wind generated by the wind generating unit 250 .

The artificial olfactory recognition and delivery device 200 may further include a mounting member 270 .

The mounting member 270 may have a necklace shape with one side open to be hung on the user's neck. At this time, the second receiving unit 210 , the odor generating unit 220 , the second control unit 230 , the third receiving unit 240 , the wind generating unit 250 , and the third control unit 260 may be fixed to the mounting member 270 .

In this case, the odor generating unit 220 and the wind generating unit 250 may be fixed to the mounting member 270 so that the fixing direction can be adjusted. Accordingly, the odor generating unit 220 and the wind generating unit 250 are fixed in a fixed direction so that the smell generated by the odor generating unit 220 and the wind generated by the wind generating unit 250 face the user's face. can be adjusted. Therefore, the smell and the wind can be stably transmitted to the user, and the user can easily recognize the smell and the wind.

As shown in FIG. 4 , the odor generating unit 220 may be provided at one end of both ends of the mounting member 270 , and the wind generating unit 250 may be provided at the other end of the mounting member 270 .

Although not shown, the odor generating unit 220 and the wind generating unit 250 may be simultaneously provided at both ends of the mounting member 270 .

In addition, the second receiving unit 210, the second control unit 230, the third receiving unit 240, the wind generating unit 250 and the third control unit 260 are integrally formed, It may be fixed to the mounting member 270 .

In the above description, it has been described that the second receiving unit 210 and the third receiving unit 240 are provided, respectively, and the second control unit 230 and the third control unit 260 are respectively provided, but one is provided to perform the functions of the second receiving unit 210 and the third receiving unit 240 , or a single control unit is provided to provide the second control unit 230 and the third control unit The function of the unit 260 may also be performed.

The artificial olfactory recognition and delivery device 200 may generate a smell corresponding to the smell set value received from the output device 10 , and a wind corresponding to the wind set value received from the output device 10 . may additionally occur. In particular, in conjunction with the content of the output device 10 , a smell may be generated or a wind may be generated at an accurate time. Accordingly, since the user can feel the sense of smell and touch when experiencing the virtual environment, the reality of the virtual environment experience can be further increased.

9 is a plan view for explaining the artificial taste delivery device shown in FIG. 1 , and FIG. 10 is a plan view for explaining the delivery unit shown in FIG. 9 .

9 and 10 , the artificial taste transmitting device 300 includes a fourth receiving unit 310 , a storage unit 320 , a transmitting unit 330 , and a fourth control unit 340 . .

The fourth receiving unit 310 receives a taste setting value for taste among the setting values from the content of the output device 10 through the network.

The storage unit 320 serves to store programs and data necessary for the operation of the artificial taste delivery device 300 . In particular, the storage unit 320 may store a temperature (°C), an intensity of a current (mA), and a frequency (Hz) corresponding to a plurality of different flavors.

As shown in Table 1 below, a mapping table in which the intensity of current (㎂), frequency (Hz), and temperature (℃), which are the conditions under which the corresponding taste is expressed, is mapped according to the type of taste and the type of subdivided taste will be stored. can

kind of taste taste Current strength (も) Frequency (Hz) Temperature (℃) Sour taste Tuscan Orange Flavor 180 800 30 Brazilian Mango Grapefruit Flavor 180 800 25 Malibu Lemon Flower Flavor 180 800 18 lime flavor 60-180 800 20-30 spicy Madagascar spice flavor 20 800 20-30 mint flavor Malibu Lemon Flower Flavor 20 400 36 mint flavor Peppermint Candy Flavor 40 800 18 sweetness chocolate flavor 180-60 descent 800 35-25 descent salty salt to taste 50 100 20-30 rise bitterness Chinese medicine taste 60-140 400 20-30 rise

At this time, it is preferable that the temperature corresponding to each taste is set within the range of 0 to 40 ℃, the intensity of the current is in the range of 0 to 250 ㎂, and the frequency is set within the range of 0 to 800 Hz.

In addition to the stored mapping table, the storage unit 320 may store a specific taste input from a user through a user interface (not shown), and setting information on temperature, current strength, and frequency to be mapped to the specific taste.

The transfer unit 330 is configured to transmit electrical stimulation by temperature stimulation and current.

Specifically, the transmitting unit 330 may transmit a temperature stimulation by a temperature corresponding to the taste setting value and an electrical stimulation by an intensity and frequency of a current corresponding to the taste setting value. The delivery unit 330 may be made of any material as long as it can deliver temperature stimulation and electrical stimulation to a specific part of the user.

However, it is preferable that the delivery unit 330 has an arrangement shape and structure that allows the user's tongue to contact the artificial taste delivery device 300 . In the present invention, the delivery unit 330 is disposed at the distal end of each finger of the haptic glove 20 so that when the user brings each finger to the tongue while wearing the haptic glove 20, the delivery unit 330 is Temperature stimulation and electrical stimulation are transmitted to the tongue so that the taste can be felt.

The haptic glove 20 may be basically made of an insulating material having a space inside which a hand can be inserted.

The delivery unit 330 may include a temperature stimulation unit 331 for temperature stimulation and an electrode 332 for electrical stimulation.

The temperature stimulation unit 331 may be implemented as a heating coil or a thermoelectric element.

The thermoelectric element uses heat absorption or heat generation due to the Peltier effect, and the n-type semiconductor and the p-type semiconductor are repeatedly connected in series, and heat absorption or heat generation is determined according to the direction in which the current flows. When a heat source is applied, holes in the p-type semiconductor and electrons in the n-type semiconductor take the energy and move to the heat sink, where heat energy is converted into electrical energy while working on an external load. can

The electrode 332 may include a pair of conductive parts, a power connector, and a lead wire. The pair of conductive parts may include any one of silver, silver chloride, titanium, gold, and platinum for conductivity. .

The conductive part may be electrically connected to a power supply to which power is supplied through a lead wire and a power connector, and each of the lead wires connecting the conductive part and the power supply may be covered with a separate insulating material so as not to be exposed to the outside.

A specific arrangement method of the temperature stimulation unit 331 and the electrode 332 in the haptic glove 20 will be described later.

The fourth control unit 340 is configured to control the overall operation of the artificial taste delivery device 300 .

Specifically, the fourth control unit 340 transmits the temperature stimulation by the temperature corresponding to the taste setting value and the electrical stimulation by the intensity and frequency of the current corresponding to the taste setting value. can control

In this case, the fourth control unit 340 transmits the temperature of the temperature stimulus, the intensity and frequency of the electric stimulus, transmitted from the transmitting unit 330 according to the taste setting value received by the fourth receiving unit 310 . It is possible to control the delivery unit 330 to adjust the.

On the other hand, the fourth control unit 340, as shown in Table 1, in a preset taste among a plurality of different tastes, the temperature or the intensity of the current varies with time within a preset range temperature stimulation or Electrical stimulation can be transmitted.

For example, in order to realize the chocolate taste among the sweetness, the intensity of the current may be decreased from 180 μA to 60 μA, and the temperature may be decreased from 35° C. to 25° C. In order to realize the taste of Chinese medicine among bitterness, the temperature may be increased from 20°C to 30°C within a current range of 60 μA to 140 μA.

In this way, when the temperature stimulation and the electrical stimulation are transmitted from the transmission unit 330 , the user may put their tongue on the transmission unit 330 to receive the temperature stimulation and the electrical stimulation. Temperature stimulation and electrical stimulation transmitted through the taste buds are perceived as a specific taste through the brain.

In the artificial taste transmitting device 300 , the fourth receiving unit 310 , the storage unit 320 , and the fourth control unit 340 are located on the wrist of the haptic glove 20 , and the temperature stimulation and electrical stimulation The delivery unit 330 for delivering the haptic glove 20 may be provided in the form of a thimble at the distal end of each finger of the haptic glove 20 .

The fourth control unit 340 may be electrically connected to the transfer unit 330 through a wire.

The transmission unit 330 may include a temperature stimulation unit 331 for transmitting a temperature stimulation and an electrode 332 for transmitting an electrical stimulation including current and frequency.

The delivery unit 330 is implemented as a flexible printed circuit board (FPCB) board for electrodes provided at the distal end of each finger of the haptic glove 20 or a ring that can be fitted on each finger of the haptic glove 20 . It can be implemented as a type.

The fourth control unit 340 positioned on the wrist of the haptic glove 20 may include a temperature control PCB for controlling the temperature stimulation unit 331 . The temperature stimulation unit 331 for transmitting the temperature stimulation is provided at the distal end of each finger of the haptic glove 20 together with the electrode 332, and may be connected to the temperature control PCB with each conductive metal of the positive and negative electrodes. .

Meanwhile, although not shown, the fourth receiving unit 310 , the storage unit 320 , and the fourth control unit 340 are mounted on the user's wrist, and the transmitting unit 330 is one of the user's fingers. At least one may be fitted in the form of a thimble.

11 is a perspective view illustrating another example of the artificial taste delivery device shown in FIG. 1 .

Referring to FIG. 11 , the artificial taste delivery device 300 may be provided in the mask unit 160 of the HMD 100 .

Specifically, the fourth receiving unit 310 , the storage unit 320 , and the fourth control unit 340 are provided inside the mask unit 160 , and the transmitting unit 330 is the mask unit ( 160) may be provided in the form of a mouthpiece that protrudes from the inner surface of the body and can be bitten by the user's mouth.

Detailed descriptions of the fourth receiving unit 310 , the storage unit 320 , the transmitting unit 330 , and the fourth control unit 340 are in the artificial taste transmitting device 300 with reference to FIGS. 9 and 10 . It is substantially the same as the description for

The user may taste the taste by contacting the tongue with the delivery unit 330 in the form of the mouthpiece. Accordingly, the user can more conveniently feel the taste.

When experiencing the virtual environment, the user can sense taste by using the artificial taste delivery device 300 as well as sight and hearing, so that the reality of the virtual environment experience can be further increased.

12 is a plan view illustrating the pressure generating unit and the vibration generating unit shown in FIG. 1 .

12 , the artificial tactile delivery device 400 includes a fifth receiving unit 410 , a pressure generating unit 420 , a vibration generating unit 430 , and a fifth control unit 440 .

The fifth receiving unit 410 receives a tactile setting value for a tactile sense among the setting values from the content of the output device 10 through the network. The tactile setting value may include a pressure setting value and a vibration setting value.

The pressure generating unit 420 generates pressure under the control of the fourth control unit 410 .

Specifically, the pressure generating unit 420 is provided on the palm side of the haptic glove 20 and transmits a pressure stimulus by a pressure corresponding to the pressure set value among the tactile set values.

The pressure generating unit 420 may include a plurality of air pads that accommodate the air and control the pressure of the air. The pressure of the air pads may be adjusted so that a pressure stimulus corresponding to the set pressure is applied to the palm of the user.

Alternatively, the pressure generating unit 420 may include a plurality of electrical pads capable of controlling the applied current. By adjusting the intensity of the current provided to the electrical pads, the pressure stimulus corresponding to the set pressure value may be applied to the palm of the user.

The vibration generating unit 430 generates vibration under the control of the fourth control unit 410 .

Specifically, the vibration generating unit 430 is provided on the back of the hand of the haptic glove 20 and transmits vibration stimulation by vibration corresponding to the vibration setting value among the tactile setting values.

The vibration generating unit 430 may include a plurality of vibrators capable of adjusting the intensity of vibration. The vibration intensity of the vibrators may be adjusted so that a vibration stimulus corresponding to the vibration setting value is applied to the user's palm.

Meanwhile, although not shown, the pressure generating unit 420 and the vibration generating unit 430 may be provided in the haptic glove 20 together with the transmitting unit 330 .

The fifth control unit 440 receives the tactile setting value from the fifth receiving unit 410 . Then, the fifth control unit 440 controls the pressure generating unit 420 and the vibration generating unit 430 to generate pressure and vibration corresponding to the tactile set value.

Specifically, the fifth control unit 440 may control the pressure generated by the pressure generating unit 420 by adjusting the pressure of the air pads or the current of the electric pads.

Also, the fifth control unit 440 may adjust the vibration intensity of the vibrators to the vibration generated by the vibration generating unit 430 .

In addition, the fifth control unit 440 may individually control the air pads, the electrical pads, or the vibrators, or may control each region disposed on the haptic glove 20 . Accordingly, the pressure stimulation by the pressure generating unit 420 or the vibration stimulation by the vibration generating unit 430 may be applied to each part of the palm or the back of the hand.

The artificial olfactory recognition and delivery device 200 , the artificial taste delivery device 300 , and the artificial tactile sensation delivery device 400 may be formed as a set with the HMD 100 .

When the user mounts the HMD 100 on the head, the artificial olfactory recognition and delivery device 200 may be hung over the user's neck, and the artificial taste delivery device 300 and the artificial tactile delivery device 400 are It can be fitted in the user's hand.

The HMD 100 outputs a screen and a sound in association with the content of the output device 10 , and the artificial olfactory recognition and delivery device 200 works with the content of the output device 10 to provide an accurate time point It may cause an odor or generate wind. In addition, the artificial taste transmitting device 300 and the artificial tactile transmitting device 400 link with the content of the output device 10 to allow the user to taste the taste at the correct time and to deliver pressure and vibration to the user's hand. can

Accordingly, when experiencing the virtual environment, the user can sense not only sight and hearing, but also smell, taste, and touch, thereby further enhancing the reality of the virtual environment experience.

Although not shown, batteries may be respectively embedded in the HMD 100 , the artificial olfactory recognition and delivery device 200 , the artificial taste delivery device 300 , and the artificial tactile delivery device 400 . The batteries may be used as power sources for the HMD 100 , the artificial olfactory recognition and delivery device 200 , the artificial taste delivery device 300 , and the artificial tactile sensation delivery device 400 , respectively.

Alternatively, the HMD 100 , the artificial olfactory recognition and delivery device 200 , the artificial taste delivery device 300 , and the artificial tactile sensation delivery device 400 may be connected to an external power source by wire.

13 is a flowchart illustrating an artificial vision and auditory transmission method according to an embodiment of the present invention.

Referring to FIG. 13 , first, an image setting value and a sound setting value among setting values are received from the output device 10 . (S110)

The output device 10 may output content in which setting values for operating hardware are stored. The set values may be stored in the image using separate software when the image is produced.

The first receiving unit 110 receives the image setting value and the sound setting value from the content of the output device 10 through network communication. In this case, the image setting value and the sound setting value may be simultaneously received.

Thereafter, the display unit 120 is controlled to output a screen corresponding to the image setting value. (S120)

The first control unit 140 receives the image setting value from the first receiving unit 110 . Thereafter, the first control unit 140 controls the display unit 120 to output a screen corresponding to the image setting value.

In addition, the speaker unit 120 is controlled to output a sound corresponding to the sound setting value. (S130)

The first control unit 140 receives the sound setting value from the first receiving unit 110 . Thereafter, the first control unit 140 controls the speaker unit 130 to output a sound corresponding to the sound setting value. At this time, the bone conduction speaker 134 of the speaker unit 130 generates vibration to transmit the sound. The bone conduction speaker 134 may be in direct contact with a portion of the skull around the user's ear or an area capable of transmitting sound under the ear. Accordingly, it is possible to accurately transmit the sound to the user and reduce the user's hearing loss.

Meanwhile, the screen output S120 and the sound output S130 may be performed simultaneously.

When the user experiences the virtual environment, the user can sense sight and hearing at the same time, thereby further enhancing the reality of the virtual environment experience.

14 is a flowchart illustrating an artificial olfactory recognition and delivery method according to an embodiment of the present invention.

Referring to FIG. 14 , first, a smell setting value for a smell among the set values is received from the output device 10 . (S210)

The output device 10 may output an image in which setting values for operating hardware are stored. The second receiving unit 210 receives the smell setting value from the content of the output device 10 through network communication.

Thereafter, the odor generating unit is controlled to generate a odor corresponding to the odor set value. (S220)

The second control unit 230 receives the smell setting value from the second receiving unit 210 . Thereafter, the second control unit 230 checks the smell corresponding to the smell setting value, and controls the smell generating unit 220 to generate the smell.

Specifically, one of the cartridges 224 is selected, and the opening/closing unit 225 opens the entrance of the cartridge 224 to generate the smell. Preferably, the opening/closing part 225 opens the inlets one by one, but in some cases, a plurality of the inlets may be opened at the same time.

The second control unit 230 controls the first fan 228 to rotate the first fan 228 , and controls the heating units 229 to heat the cartridge 224 that generates the smell. can do. By rotating the first fan 228 , the material emitted from the cartridges 224 can be quickly ejected through the first outlet 223 . As the heating units 229 heat the cartridge 224 , the substance can be rapidly and actively dissipated.

In addition, a wind setting value for wind among the setting values is received from the output device 10 . (S230)

The second receiving unit 210 receives the wind setting value from the content of the output device 10 through the network communication.

Thereafter, a wind corresponding to the set wind value is generated by controlling the wind generating unit that generates the wind. (S240)

The third control unit 260 receives the wind setting value from the third receiving unit 240 . Then, the third control unit 260 checks the wind corresponding to the wind set value, and controls the wind generating unit 250 to generate the wind.

Specifically, the wind is generated by rotating the second fan 254 under the control of the third control unit 260 .

In addition, according to the control of the third control unit 260, the rotation speed of the second fan 254 is adjusted to adjust the strength of the wind, and the temperature of the heating coils 255 is adjusted to adjust the temperature of the wind. adjust the

The reception of the smell setting value (S210) and the generation of the smell (S220) may be performed separately from the reception of the wind setting value (S230) and the generation of the wind (S240). Accordingly, the generation of the smell (S220) and the generation of the wind (S240) may be performed simultaneously.

Since the user can feel the sense of smell and touch when experiencing the virtual environment, the reality of the virtual environment experience can be further increased.

15 is a flowchart illustrating an artificial taste delivery method according to an embodiment of the present invention.

Referring to FIG. 15 , first, temperatures, current strengths, and frequencies corresponding to a plurality of different tastes are received and stored. (S310)

The temperature corresponding to the taste and the intensity and frequency of the current are stored in the storage unit 320 .

Thereafter, a taste setting value for taste among the setting values is received from the output device 10 . (S320)

The fourth receiving unit 210 receives the taste setting value from the content of the output device 10 through the network communication.

Thereafter, the temperature stimulation by temperature corresponding to the taste setting value is transmitted through the temperature stimulation unit 331 of the transmission unit 330 provided at the distal end of each finger of the haptic glove 20, and corresponding to the taste setting value An electrical stimulation by a current of the desired intensity and frequency is transmitted through the electrode 332 of the transmission unit 330 provided at the distal end of each finger of the haptic glove 20 . (S330)

When the user brings the electrode and the temperature stimulation unit constituting the delivery unit 330 to the tongue, the electric stimulation and the temperature stimulation transmitted in this way are applied to the user's tongue, so that an artificial taste can be generated and transmitted. Accordingly, it is possible to sense and control artificial senses for various tastes by delivering a stimulus in which the intensity, frequency, and temperature of electric current are fused to the user.

Since the user can feel the taste when experiencing the virtual environment, the reality of the virtual environment experience can be further increased.

16 is a flowchart illustrating an artificial tactile sense delivery method according to an embodiment of the present invention.

Referring to FIG. 16 , first, a tactile setting value for a tactile sense among setting values is received from the output device 10 . (S410)

The fifth receiving unit 410 receives the tactile setting value from the content of the output device 10 through network communication. The tactile setting value may include a pressure setting value and a vibration setting value.

Thereafter, the pressure generating unit 420 is controlled to generate a pressure corresponding to the set pressure value. (S420)

The fifth control unit 440 receives the pressure setting value from the fifth receiving unit 410 . Thereafter, the fifth control unit 440 controls the pressure generating unit 420 to generate pressure on the palm side of the haptic glove 20 .

Specifically, the pressure stimulus corresponding to the set pressure value is applied to the palm of the user by adjusting the air pressure of the plurality of air pads or the current of the plurality of electrical pads.

In this case, the air pads or the electrical pads may be individually controlled, or may be controlled for each region disposed on the haptic glove 20 . Accordingly, the pressure stimulation by the pressure generating unit 420 may be applied to each part of the palm.

Thereafter, the vibration generating unit 430 is controlled to generate vibration corresponding to the vibration setting value. (S430)

The fifth control unit 440 receives the vibration setting value from the fifth receiving unit 410 . Thereafter, the fifth control unit 440 controls the vibration generating unit 430 to generate vibrations on the back of the hand of the haptic glove 20 .

Specifically, the vibration intensity of the plurality of vibrators is adjusted so that the vibration stimulus corresponding to the vibration setting value is applied to the back of the user's hand.

In this case, the vibrators may be individually controlled or controlled for each region disposed on the haptic glove 20 . Accordingly, the vibration stimulation by the vibration generating unit 430 may be applied to each part of the back of the hand.

The generation of the pressure (S420) and the generation of the vibration (430) are preferably performed simultaneously, but may be performed separately.

The pressure and vibration may be transmitted to the user through the haptic glove 20 . Accordingly, the user can feel and control the artificial senses for various tactile senses by delivering the stimulus in which the pressure and vibration are fused.

In addition, since the user can feel the tactile sense when experiencing the virtual environment, the reality of the virtual environment experience can be further increased.

2. How to deliver the five senses hardware

The five sense hardware delivery method according to the present invention relates to a method for operating a five sense sensor generating device including a visual module, a taste module, a olfactory module, a tactile module, and an auditory module. It is about a method of generating and delivering virtual five senses for an image.

17 is a flowchart illustrating a method for transmitting five sense hardware according to the present invention.

Referring to FIG. 17 , in the five sense hardware transmission method according to the present invention, an operation value including an operation time for at least one of the five senses requiring sensory transmission is set in a first step ( S510 ), among the apparatus 1000 for generating five senses It may include a second step ( S520 ) of operating at least one and a third step ( S530 ) of operating the apparatus 1000 for generating five senses.

The first step (S510) may include extracting information on a section in which at least one sensory transmission is required among the five senses and information on a sense corresponding to the section in a specific image, and based on the extracted information to set the operating value.

Here, the specific image may be an image captured by a VR camera or an image captured by a general camera, but is not necessarily limited thereto.

Here, if the specific image is an image captured by a VR camera, the visual module may visually show the captured image to the wearer as it is without processing, but to the wearer in a processed state to some extent within the range in which the nature of the image does not change It can also be shown visually.

In the first step (S510), an image satisfying a similarity condition is extracted by comparing a specific image with a previously stored image, and information about the section and information about the section based on the five sense information corresponding to the extracted image It may include a step of extracting information about the senses.

Here, the similarity condition may be variously set based on various similarity determination criteria, and may be, for example, a similarity condition through machine learning.

Hereinafter, it is assumed that the playback time of the specific image is 5 minutes, and the images from 2 minutes to 4 minutes are images taken at a location 1 m away from the firewood while lighting a fire and eating chocolate, and in the first step (S510) will be described in more detail.

In the first step (S510), information on a section in which virtual sensory transmission is required in the specific image is extracted from 2 minutes to 4 minutes, and information on the senses corresponding to the section is visual, olfactory, Information such as touch, taste, and hearing is extracted.

Here, the sense of smell is generated when the firewood is burned, the sense of touch is generated when the ambient temperature is increased due to the firewood, the sense of smell is generated by the sound of burning wood, and the sense of taste is generated while eating chocolate.

In the first step (S510), an image that satisfies the similarity condition with the specific image from among a plurality of images stored in advance is extracted. A satisfactory image is extracted.

When an image satisfying the similarity condition is extracted as described above, an operation value including an operation time for at least one of the five senses requiring sensory transmission is set based on the information on the section and the information on the sensation. .

In the above example, the information on the section is from 2 minutes to 4 minutes, the information on the senses may be sight, smell, touch, taste, and hearing, and at least one of the five senses requiring sensory transmission is a visual module and a olfactory module. , a tactile module, a taste module, and an auditory module.

Here, the operating time may be 2 to 4 minutes, and the operating value may be an operating condition for the modules for generating a virtual sensation that the wearer can experience in the context of the specific image.

In the second step (S520), a signal for an operation based on an operation value of at least one of the five senses generating apparatus 1000 for generating a sensation corresponding to the section based on information on the sense corresponding to the section It may be a step to apply and operate.

For example, in the second step ( S520 ), when the device for generating a sense corresponding to the section is a taste module for generating a taste, the temperature condition, current condition, and frequency condition applied to the taste module are signal can be generated.

Here, the temperature condition, the current condition, and the frequency condition may be in the range of 0 to 40 °C, 0 to 250 μA, and 0 to 800 Hz, respectively.

For another example, when the device for generating a sense corresponding to the section in the second step ( S520 ) is a olfactory module for generating a sense of smell, a substance for generating a specific smell filled in the cartridge of the olfactory module It is possible to generate a signal based on the actuating value to diverge, and an actuation signal based on the actuation value of the heating part of the olfactory module for heating the cartridge to aid in the dissipation of the substance.

As another example, when the device for generating a sense corresponding to the section in the second step ( S520 ) is a tactile module for generating a tactile sense, pressure is generated by the pressure generating unit of the tactile module, and An operation signal based on the operation value may be generated to enable generation of vibration by the vibration generating unit of the tactile module.

Here, the tactile module may be provided as a haptic glove worn by the wearer, the pressure generating unit is provided on the palm of the haptic glove, and the vibration generating unit is provided on the back of the hand of the haptic glove, but the intensity of vibration It may include a plurality of adjustable vibrators.

In addition, the tactile module may be configured as a fan, etc. In this case, in the third step ( S530 ), an operation signal based on the operation value is generated to the fan and heating coil of the haptic module to generate artificial wind by the fan. The intensity and temperature of the can be controlled.

On the other hand, when a signal for operation based on the operation value is applied as described above, a third step ( S530 ) in which the apparatus 1000 for generating five senses is operated in response to the signal proceeds, and the apparatus 1000 for generating five senses proceeds. ), the wearer receives realistic five senses.

In the above, the configuration and features of the present invention have been described based on the embodiments according to the present invention, but the present invention is not limited thereto, and it is understood that various changes or modifications can be made within the spirit and scope of the present invention. It is intended that such changes or modifications will be apparent to those skilled in the art, and therefore fall within the scope of the appended claims.

10: output device 20: haptic glove
100: HMD 110: first receiving unit
120: display unit 130: speaker unit
140: first control unit 150: mounting unit
160: mask unit
200: artificial olfactory recognition and delivery device 210: second receiving unit
220: odor generating unit 230: second control unit
240: third receiving unit 250: wind generating unit
260: third control unit 270: mounting member
300: artificial taste transmitting device 310: fourth receiving unit
320: storage unit 330: transfer unit
340: fourth control unit 400: artificial tactile transmission device
410: fifth receiving unit 420: pressure generating unit
430: vibration generating unit 440: fifth control unit

Claims (8)

A method for transmitting five sense hardware for operating a device for generating five senses including a visual module, a taste module, a olfactory module, a tactile module, and an auditory module, the method comprising:
An operation value including an operation time for at least one of the five senses requiring sensory transmission is based on information on a section requiring at least one sensory transmission among the five senses and information on a sense corresponding to the section within a specific image. a first step of being established; and
A second step of applying a signal for an operation based on the operation value of at least one of the five senses generating devices for generating a sensation corresponding to the section based on the information on the sense corresponding to the section to be operated including;
The first step is
By comparing the specific image with a plurality of pre-stored images, an image satisfying a similarity condition through machine learning is extracted from among the plurality of pre-stored images, and based on the five sense information corresponding to the extracted image, within the specific image Including the step of extracting information on the sense corresponding to the section,
The second step is
When the device for generating a sense corresponding to the section is a taste module for generating a taste, it generates an operation signal for a temperature condition, a current condition, and a frequency condition applied to the taste module to operate the taste module do,
When the device for generating a sense corresponding to the section is a olfactory module for generating a sense of smell, an operation signal for emitting a substance for generating a specific odor filled in the cartridge of the olfactory module, and the emission of the substance to generate an operation signal of the heating unit of the olfactory module for heating the cartridge to help, so that the olfactory module is operated,
When the device for generating a sense corresponding to the section is a tactile module for generating a tactile sense, an operation signal for generating pressure by a pressure generating unit of the tactile module, and vibration by a vibration generating unit of the tactile module by generating an operation signal enabling the generation of the tactile module to operate,
The temperature condition, the current condition and the frequency condition are
Conditions within the range of 0 to 40 °C, the range of 0 to 250 μA, and the range of 0 to 800 Hz, respectively,
The taste module and the tactile module,
It is provided as a wearable haptic glove,
The haptic glove is
When a finger part comes into contact with the wearer's tongue while worn by the wearer, temperature stimulation and electrical stimulation are transmitted to the tongue to generate the taste,
The pressure generating unit,
It is provided on the palm of the haptic glove,
The vibration generating unit,
The five sense hardware delivery method comprising a plurality of vibrators provided on the back of the hand of the haptic glove and capable of controlling the intensity of vibration.
delete delete delete delete delete delete delete

Images