KR20200008196A - System for providing special effect based on motion recognition and method thereof - Google Patents

Abstract

Disclosed are a system for providing a special effect by using an air fan based on motion recognition and a method thereof. The system for providing a special effect by using an air fan based on motion recognition recognizes a motion of a user to control an operation of an object displayed in a display part according to the recognized motion, provides a special effect related to the corresponding object to maximize a sensory index of the user, and miniaturizes equipment and improves the mobility of the equipment to increase user convenience as well as overall operation efficiency thereof.

Description

System for providing special effect based on motion recognition based air fan and method thereof

The present invention relates to a special effect providing system and a method using a motion recognition-based air fan, and in particular, after recognizing a user's motion, controlling the operation of the object displayed on the display unit according to the recognized motion, The present invention relates to a special effect providing system using a motion recognition-based air fan that provides related special effects, and a method thereof.

Virtual Reality (VR) is a technology that emulates a specific environment or situation of reality through a computer as if the user is interacting with the surrounding environment and environment.

The virtual reality service industry using virtual reality is facing a limit to popularization despite the high growth of devices (including Samsung Gear VR, Oculus Rift, HTC VIVE, etc.).

In other words, the virtual reality service industry in which users can use virtual reality services in the fields of education, learning, games, military simulation, etc. has high atmospheric demand, but the cost of constructing a virtual reality space per unit is high, and simply a head mounted display (HMD) is used. In addition to providing the virtual reality environment used, there is a limit in expanding the user's experience, and there are inconveniences in terms of mobility and reuse of content.

Korean Patent Publication No. 10-2018-0077588

An object of the present invention, after recognizing the user's motion, controls the operation of the object displayed on the display according to the recognized motion, and provides a special effect providing system using a motion recognition-based air fan to provide a special effect associated with the object And a method thereof.

A system for providing a special effect using a motion recognition-based air fan according to an embodiment of the present invention includes a display unit including a head mounted display (HMD) and displaying a 3D content including an object whose movement is controlled according to a user input; A sensor unit detecting a user's motion; The operation of the object in the 3D content displayed on the display unit is checked based on the detected user's motion, and it is determined whether a preset special effect exists in relation to the object in the 3D content. Operation of an object in 3D content displayed on the identified display unit while generating a control signal for controlling the operation of a plurality of air fans disposed at different positions around the user when a preset special effect exists in relation to the object A control unit for controlling the operation of the object and displaying the object on the display unit; And the plurality of air fans providing a special effect based on the control signal.

As an example related to the present invention, the sensor unit may include a 360 degree camera that detects a motion of the user based on image information including the user; And a plurality of copper field recognition sensors which are worn on a user's body and detect the motion of the user according to the movement of the user.

As an example related to the present invention, the user's motion may include the direction of both legs, the movement of the legs, the direction of the arm, the rotation of the arm clockwise or counterclockwise, the fist-squeezing motion, the finger-finding motion, the finger spreading motion, Palm touch gestures, finger spread gestures, finger bend gestures, drawing shapes using hands or arms, shaking hands in either the left, right, up, down, or diagonal directions with the hands or arms, and combinations of these operations It may include at least one of the operation by.

As an example related to the present invention, the control unit may determine that the object is in accordance with an operation of an object in 3D content displayed on the identified display unit when there is no preset special effect in relation to the object in the 3D content. The operation of the control panel may be displayed on the display unit.

As an example related to the present invention, the control unit may generate the control signal including step-by-step wind intensity control information by pulse width modulation (PWM).

As an example related to the present invention, the plurality of air fans operate on the basis of the generated control signal, and the wind speed and the direction in which the wind is blown to the user according to the moving direction and the speed of the object in the virtual reality environment. It can provide special effects in the form of surround.

According to an aspect of the present invention, there is provided a method for providing a special effect using a motion recognition-based air fan, including: displaying, by a display unit, 3D content including an object whose movement is controlled according to a user input; Sensing, by the sensor unit, a user's motion; Confirming, by the controller, an operation of an object in 3D content displayed on the display unit based on the detected user's motion; Determining, by the controller, whether a preset special effect exists in relation to an object in the 3D content; As a result of the determination, when a predetermined special effect exists in relation to an object in the 3D content, the control unit generates a control signal for controlling the operation of a plurality of air fans arranged at different locations around the user; Simultaneously controlling the operation of the object in accordance with the operation of the object in the 3D content displayed on the confirmed display unit and displaying it on the display unit; And providing, by the plurality of air fans, a special effect based on the control signal.

As an example related to the present invention, when there is no preset special effect in relation to the object in the 3D content, the control unit according to the operation of the object in the 3D content displayed on the confirmed display unit, The method may further include controlling an operation of the object and displaying the same on the display unit.

The present invention, after recognizing the user's motion, by controlling the operation of the object displayed on the display according to the recognized motion, by providing a special effect related to the object, to maximize the user's sensory index, miniaturization and mobility of the equipment By enhancing the efficiency of the system, the overall system operation efficiency can be improved and the usability can be improved.

1 is a block diagram showing the configuration of a special effect providing system using a motion recognition-based air fan according to an embodiment of the present invention.
2 is a diagram illustrating an example of a control signal according to an embodiment of the present invention.
3 is a flowchart illustrating a special effect providing method using an air fan based on motion recognition according to an exemplary embodiment of the present invention.

It should be noted that the technical terms used in the present invention are merely used to describe specific embodiments, and are not intended to limit the present invention. In addition, the technical terms used in the present invention should be interpreted as meanings generally understood by those skilled in the art unless the present invention is defined in any other meaning in the present invention, and is excessively comprehensive. It shall not be construed in the sense of or in the sense of being excessively reduced. In addition, when a technical term used in the present invention is an incorrect technical term that does not accurately express the spirit of the present invention, it should be replaced with a technical term that can be properly understood by those skilled in the art. In addition, the general terms used in the present invention should be interpreted as defined in the dictionary or according to the context before and after, and should not be interpreted in an excessively reduced sense.

Also, the singular forms used in the present invention include the plural forms unless the context clearly indicates otherwise. Terms such as “consisting of” or “comprising” in the present invention should not be construed as necessarily including all of the various components or steps described in the present invention, and some of the components or some steps may not be included. It should be construed that it may further include, or further include, additional components or steps.

In addition, terms including ordinal numbers such as first and second used in the present invention may be used to describe components, but the components should not be limited by the terms. The terms are used only to distinguish one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, and the same or similar components will be given the same reference numerals regardless of the reference numerals and redundant description thereof will be omitted.

In addition, in describing the present invention, when it is determined that the detailed description of the related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, it should be noted that the accompanying drawings are only for easily understanding the spirit of the present invention and should not be construed as limiting the spirit of the present invention by the accompanying drawings.

1 is a block diagram showing the configuration of a special effect providing system 10 using a motion recognition-based air fan according to an embodiment of the present invention.

As shown in FIG. 1, the special effect providing system 10 using a motion recognition-based air fan includes a communication unit 100, a storage unit 200, a display unit 300, a voice output unit 400, and a sensor unit ( 500), the control unit 600 and the air fan 700. Not all components of the special effect providing system 10 using the motion recognition-based air fan shown in FIG. 1 are essential components, and the motion recognition-based air may be replaced by more components than those shown in FIG. The special effect providing system 10 using the fan may be implemented, or the special effect providing system 10 using the motion recognition-based air fan may be implemented using fewer components.

The communication unit 100 communicates with any component inside or any at least one terminal outside via a wired / wireless communication network. In this case, the external arbitrary terminal may include a server (not shown), another terminal (not shown), and the like. Here, the wireless Internet technologies include Wireless LAN (WLAN), Digital Living Network Alliance (DLNA), Wireless Broadband (Wibro), WiMAX (World Interoperability for Microwave Access (Wimax), HSDPA (High Speed Downlink Packet Access). ), High Speed Uplink Packet Access (HSUPA), IEEE 802.16, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), Wireless Mobile Broadband Service (WMBS), etc. In this case, the communication unit 100 transmits and receives data according to at least one wireless Internet technology in a range including Internet technologies not listed above. In addition, near field communication technologies include Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and Near Field Communication (NFC). Ultrasound Communication (USC), Visible Light Communication (VLC), Wi-Fi, Wi-Fi Direct, and the like may be included. Also, the wired communication technology may include power line communication (PLC), USB communication, Ethernet, serial communication, optical / coaxial cable, and the like.

In addition, the communication unit 100 may mutually transmit information with an arbitrary terminal through a universal serial bus (USB).

In addition, the communication unit 100 may include technical standards or communication schemes (eg, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), EV, etc.) for mobile communication. Enhanced Voice-Data Optimized or Enhanced Voice-Data Only (DO), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), LTE-A (Long Term Evolution-Advanced) and the like to transmit and receive a radio signal with a base station, the server, the other terminal and the like on a mobile communication network.

In addition, the communication unit 100 receives software (or a program) for implementing a virtual reality system transmitted from the server under the control of the control unit 600.

The storage unit 200 stores various user interfaces (UIs), graphical user interfaces (GUIs), and the like.

In addition, the storage unit 200 stores data and programs necessary for operating the special effect providing system 10 using the motion recognition-based air fan.

That is, the storage unit 200 uses a plurality of application programs or applications driven by the special effect providing system 10 using the motion recognition-based air fan or the motion recognition-based air fan. Data and instructions for the operation of the special effect providing system 10 may be stored. At least some of these applications may be downloaded from an external server via wireless communication. In addition, at least some of these applications exist on the special effects providing system 10 using the motion recognition based air fan from the time of shipment for the basic functions of the special effects providing system using the motion recognition based air fan. Can be. On the other hand, the application program is stored in the storage unit 200, is installed in the special effect providing system 10 using the motion recognition-based air fan, by the control unit 600 using the special motion recognition-based air fan It may be driven to perform an operation (or function) of the effect providing system 10.

In addition, the storage unit 200 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg, SD or XD). Memory, etc.), magnetic memory, magnetic disk, optical disk, random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), It may include at least one storage medium of PROM (Programmable Read-Only Memory). In addition, the special effect providing system 10 using the motion recognition-based air fan operates a web storage that performs a storage function of the storage unit 200 on the Internet, or is associated with the web storage. It can also work.

In addition, the storage unit 200 stores (or installs) a program for implementing (or playing / executing) a virtual reality system received through the communication unit 100.

The display unit (or display unit) 300 may display various contents such as various menu screens by using a user interface and / or a graphic user interface stored in the storage unit 200 under the control of the controller 600. have. Here, the content displayed on the display unit 300 includes various text or image data (including various information data) and a menu screen including data such as an icon, a list menu, a combo box, and the like. In addition, the display unit 300 may be a touch screen.

The display unit 300 may include a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display. The display device may include at least one of a flexible display, a 3D display, an e-ink display, and a light emitting diode (LED).

In addition, the display unit 300 may include a head mounted display (HMD) capable of displaying three-dimensional content, a display in the form of glasses, and the like.

In addition, the display unit 300 displays 2D content including 2D objects related to a specific program, 3D content including 3D objects, etc. under the control of the controller 600.

The voice output unit 400 outputs voice information included in a signal processed by the controller 600 by a predetermined signal. In this case, the voice output unit 400 may include a receiver, a speaker, a buzzer, and the like.

In addition, the voice output unit 400 outputs the guide voice generated by the controller 600.

In addition, the voice output unit 400 outputs voice information (or sound effect) corresponding to the 2D content, 3D content, etc. by the control unit 600.

The sensor unit (or motion recognition unit) 500 is a 360-degree camera for detecting a motion (or movement) of a user adjacent to the special effect providing system 10 using the motion recognition-based air fan (motion recognition sensor ( Motion Recognition Sensor). Here, the user's motion (or movement) is the direction of both legs, the movement of the legs (for example, movement, rotation, etc.), the direction of the arm, the rotation of the arm clockwise / counterclockwise, the clenching movement, the finger Pinch gesture, spread finger gesture, palm touch gesture, finger spread gesture, finger flex gesture, hand / arm figure drawing, shaking hand / arm in left, right, up and down directions, and between these gestures Operation by combination, and the like.

That is, the sensor unit 500 recognizes various known objects (or users) based on image information (eg, including a plurality of consecutive images, videos, etc.) acquired through the 360 degree camera (or camera). Apply to detect the user's motion.

In addition, the sensor unit 500 detects a user's motion based on sensing information obtained through a plurality of motion recognition sensors worn (or attached) to the user's body. The motion recognition sensor may include a sensor for recognizing a motion or a position of an object, a geomagnetism sensor, an acceleration sensor, a gyro sensor, an inertial sensor, an altimeter, Sensors, such as vibration sensors, may further include sensors related to motion recognition. In this case, the sensor unit 500 may move the user, for example, an inclined direction, an inclined angle, and / or the inclined speed of a part of the user's body (including an arm, a hand, a finger, etc.). It detects the information including the vibration direction and / or the number of vibrations in the direction, such as up, down, left, right, diagonal. Here, the detected information (the tilted direction, the tilted angle and / or the tilted speed, the vibration direction and / or the number of vibrations) are digitized through a digital signal processing process, so that the digitized information is stored in the controller 600. Is delivered).

In addition, the sensor unit 500 may be provided in the form of a wearable device, and may recognize information according to the movement of the user while being worn on the user's body (including a hand, finger, arm, leg, etc.). .

In addition, the sensor unit 500 may include one or more sensors for sensing at least one of surrounding environment information and user information surrounding the special effect providing system 10 (or user) using a motion recognition-based air fan. have. For example, the sensor unit 500 may include a proximity sensor, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, and a gravity sensor G-. sensor, gyroscope sensor, motion sensor, inertial sensor, altimeter, vibration sensor, RGB sensor, infrared sensor (IR sensor: infrared sensor), fingerprint sensor finger scan sensors, ultrasonic sensors, optical sensors (eg cameras), microphones, battery gauges, environmental sensors (eg barometers, hygrometers, thermometers, Radiation detection sensor, heat detection sensor, gas detection sensor, etc.) and a chemical sensor (eg, electronic nose, healthcare sensor, biometric sensor, etc.). Meanwhile, the special effect providing system 10 using the motion recognition-based air fan disclosed herein may use a combination of information sensed by at least two or more of these sensors.

In addition, the sensor unit 500 is a user's motion (or motion) recognition function, the voice recognition function located near the special effect providing system 10 using the motion recognition-based air fan (or a predetermined area in the VR system) The lip motion recognition function and the eye movement recognition function (or eye tracking function) may be performed.

In addition, the sensor unit 500 may include a motion recognition module for recognizing the motion of the user, the movement of the lips, and the movement of the eyes, and a voice recognition module for voice recognition.

The controller or microcontroller unit 600 executes an overall control function of the special effect providing system 10 using the motion recognition-based air fan.

In addition, the controller 600 executes the overall control function of the special effect providing system 100 using the motion recognition-based air fan by using the program and data stored in the storage unit 200. The controller 600 may include a RAM, a ROM, a CPU, a GPU, a bus, and the RAM, a ROM, a CPU, a GPU, and the like may be connected to each other through a bus. The CPU may access the storage unit 200 to boot using the O / S stored in the storage unit 200, and use various programs, contents, data, etc. stored in the storage unit 200. Various operations can be performed.

In addition, the controller 600 outputs 3D content according to a user selection through the display unit 300 and / or the voice output unit 400.

That is, when a program for providing a specific virtual reality function is selected (or executed) among programs installed in the special effect providing system 10 using the motion recognition based air fan, the controller 600 is associated with the corresponding program. The 3D content is output through the display unit 300 and / or the audio output unit 400. Here, the 3D content may include one or more objects whose movement is controlled according to a user's motion or a user input through an input unit (not shown).

In this case, when the user is wearing the HMD, the controller 600 may provide (or display / output) the corresponding 3D content displayed on the display unit 300 through the HMD. Here, the 3D content includes a 3D image / video, an associated sound effect (or voice information), and the like.

In addition, the controller 600 processes the motion information of the user's hand obtained from the 360 degree camera or the motion recognition sensor included in the sensor unit 500 to rotate the x-axis in which the palm or the finger rotates about the x-axis. After extracting the position information including the value, the y-axis rotation value in which the palm or finger rotates about the y-axis, and the z-axis rotation value in which the palm or finger is rotated about the z-axis, In operation, the converted value may be converted into a value corresponding to the motion reference information, and the converted value may be linked with an operation of a specific object in the 3D content which is being displayed (or executed).

In addition, the controller 600 confirms an operation (or movement) of an object in 3D content displayed on the display unit 300 (or the HMD) based on the detected user's motion.

In addition, the controller 600 determines whether a preset special effect exists in relation to an object in the 3D content (or whether a preset special effect exists so as to interact with the operation of the object).

As a result of the determination, when there is no preset special effect in relation to the object in the 3D content according to the detected user's motion, the controller 600 is displayed on the confirmed display unit 300 (or the HMD). The motion of the object is controlled according to the motion (or movement) of the object in the 3D content, and the control result (or the motion / movement result of the object in the 3D content) is displayed on the display unit 300 (or the HMD). do.

As a result of the determination, when a preset special effect exists in relation to an object in the 3D content according to the detected user's motion, the controller 600 controls the plurality of air fans 700 disposed at different locations around the user. To generate a control signal for controlling the operation.

In this case, as shown in FIG. 2, the control unit 600 may generate a control signal including stepwise wind strength control information by pulse width modulation (PWM).

In addition, the controller 600 controls at least one or more of the plurality of air fans 700 to provide a special effect to the user, and at the same time, the 3D displayed on the identified display unit 300 (or the HMD). The operation of the object is controlled according to the operation (or movement) of the object in the content, and the control result (or the operation / movement result of the object in the 3D content) is displayed on the display unit 300 (or the HMD). .

In addition, the controller 600 is installed in the special effect providing system 10 (or the storage unit 200) using the motion recognition-based air fan based on updated program information provided from a server (not shown). You can update the program.

As such, the special effect providing system 10 using the motion recognition-based air fan controls the operation of an object for a virtual reality experience according to a user's motion (or movement) located in a space installed in a booth shape or the like. In addition, it provides realistic information applying a multimodality system and maximizes the experience of the virtual reality.

The air foil fan 700 may be configured as a single or a plurality.

When the air fan 700 is configured in a single shape, the air fan 700 is located around a preset area where a user in the special effect providing system 10 using the motion recognition based air fan in a booth shape is to be located. While moving in accordance with the control signal generated by the control unit 600 through a movable rail (not shown) to provide a special effect (including wind, etc.) to the user.

In addition, when the air fan 700 is configured in a single form, after the air fan 700 is fixed at a specific position, the air fan based on the motion recognition according to the control signal generated by the controller 600 The special effect providing system 10 may provide a special effect (including wind, etc.) to a corresponding user located near a preset area where the user in the special effect providing system 10 is to be located.

In addition, when the air fan 700 is configured in a plurality of forms, each of the plurality of air fans 700 may be located within the special effect providing system 10 using the motion recognition-based air fan in a booth form. Each of the air fans 700 of the plurality of air fans 700 operates in accordance with a control signal generated by the controller 600, and is installed at a predetermined position around a preset area. Provide effects (such as wind, etc.).

As such, when the air fan 700 is configured in a single form, the air fan 700 may be configured to move around the user to provide a special effect to the user in any one direction and angle of 360 degrees. Can be.

In addition, when the air fan 700 is configured in a plural form in a surround form, the plurality of air fans 700 may move to a user in accordance with the moving direction, speed, etc. of an object in a virtual reality environment. You can provide surround-type special effects (e.g. special effects that take into account the speed of the wind, the direction the wind is blowing, etc.) to feel similar.

Although an embodiment of the present invention has been described to provide a special effect based on virtual reality, the present invention is not limited thereto, and a special effect according to the present invention may be provided even on a mixed reality (MR) basis.

In the embodiment of the present invention has been described to provide the user with the wind of the surround form using the air fan 700 as a special effect, but is not limited to this, special effects for other special effects other than the air fan 700. A providing unit (not shown) may be configured.

The special effect providing unit may further include a haptic interface (not shown) and the like, and not only a desire for the user through the haptic interface, but also information about the kinesthesia associated with the object (for example, a mass associated with the object, Information about the firmness, softness, reaction, etc., and the texture associated with the object (e.g. surface pattern, small shapes, softness and roughness of the surface related to the object, etc.) Tactile information), and the like.

The haptic interface (or haptic unit / tactile interface) refers to a whole system that transmits a tactile sense to a user in a broad sense, and is a hardware device that generates a tactile sense by transmitting a physical force to the user by directly contacting the user. A haptic device, a virtual environment consisting of a remote contact environment or a computer graphic and physical characteristics, and a haptic device that transmits movement of the haptic device to the virtual environment and transmits contact force with the virtual environment to the haptic device. It may be configured as a controller (not shown) (or the control unit 600).

The ultimate purpose of the haptic interface is to make the user feel the same through the haptic device through the physical characteristics of the modeled virtual environment (including AR environment, VR environment, MR environment, etc.) or real environment.

In addition, the haptic interface includes a virtual agent representing a change in position of the haptic device in the virtual environment so that the haptic device and the virtual environment exist in one environment. If the force generated by the virtual delegate hitting the virtual object is Fe and the force felt by the haptic device is Fm, then it is an ideal haptic interface system to control the two values to be equal.

In order to implement an ideal haptic interface, research on accurate modeling methods and hardware and control methods for implementing the model should be supported.

In addition, the haptic interface may be used in the same sense as the haptic device because the haptic interface is limited to only a hardware part in a narrow sense.

In addition, the haptic interface can be roughly divided into two fields in terms of hardware development, that is, in terms of developing the haptic device.

In other words, a person feels the sense of touch through two main paths, and the haptic interface is also developed in two main areas.

One of the fields of the haptic interface is related to muscle sensation, and when we feel the mass of an object, the degree of firmness and softness, and the appearance of the object, the haptic interface feels the touch through the movement of muscles and joints. Since this process is physically related to force, the haptic device has a mechanical structure such as a robot shape that is advantageous for generating a force, and the haptic device made for this purpose is called a force feedback device. do.

In addition, one of the fields of the haptic interface is related to the texture, and is a phenomenon in which the skin is directly contacted with the surface when the pattern or small shapes of the surface being contacted, the softness and roughness of the surface, and the sense of warmth and the like are felt. This process is composed of many complex physical elements, and the shape of haptic devices to reproduce the texture is very diverse. The device made for the purpose of conveying the texture is called a texture display device.

In addition, the haptic interface greatly increases the realism of a remote environment or a virtual environment through the transmission of touch, and provides a function of touching and buying a product in an online shopping mall that was only visible in a network-based environment. Any terminal and user can provide an online museum function that can touch the relics, and provide a function of conveying emotions through the touch such as shaking hands and tapping each other in a text-based messenger environment. It can provide an interactive function between them.

As such, after recognizing the motion of the user, the motion of the object displayed on the display unit may be controlled according to the recognized motion, and a special effect related to the object may be provided.

Hereinafter, a method for providing a special effect using a motion recognition based air fan according to the present invention will be described in detail with reference to FIGS. 1 to 3.

3 is a flowchart illustrating a special effect providing method using an air fan based on motion recognition according to an exemplary embodiment of the present invention.

First, the controller 600 outputs 3D content according to a user's selection through the display unit 300 and / or the voice output unit 400. Here, the 3D content may include one or more objects whose movement is controlled according to a user's motion or a user input through an input unit (not shown).

In this case, when the user is wearing the HMD, the controller 600 may provide (or display / output) the corresponding 3D content displayed on the display unit 300 through the HMD. Here, the 3D content includes a 3D image / video, an associated sound effect (or voice information), and the like.

For example, when executing a racing game based on a user input in a VR system for a racing game, the control unit 600 is associated with a first vehicle associated with the racing game in a first HMD worn by the user. The first 3D content including the object is displayed, the first 3D content is displayed on the display unit 300, and the voice information (or sound effect) corresponding to the first 3D content is output to the voice output unit 400. Output through

As another example, when executing a drone adjustment based on a user input in a VR system for drone adjustment, the controller 600 may display an object related to the first drone related to the drone adjustment through the display unit 300. The second 3D content is included and the voice information (or sound effect) corresponding to the second 3D content is output through the voice output unit 400 (S310).

Thereafter, the sensor unit 500 detects a user's motion (or movement). In this case, the sensor unit 500 detects a user's motion based on image information obtained through a 360 degree camera (not shown), or a plurality of motion recognition sensors (not shown) worn on (or attached to) the user's body. Based on the sensing information obtained through the) can detect the user's motion. Here, the user's motion (or movement) is the direction of both legs, the movement of the legs (for example, movement, rotation, etc.), the direction of the arm, the rotation of the arm clockwise / counterclockwise, the clenching movement, the finger Pinch gesture, spread finger gesture, palm touch gesture, finger spread gesture, finger flex gesture, hand / arm figure drawing, shaking hand / arm in left, right, up and down directions, and between these gestures Operation by combination, and the like.

For example, the camera included in the sensor unit 500 is a user's left and right hand movements (for example, the user's left hand is in the 9 o'clock position and the user's right hand is in the 3 o'clock position, the left hand of the user). Rotate from 9 o'clock to 12 o'clock and the user's right hand rotates from 3 o'clock to 6 o'clock / clockwise rotation of the user's left and right hand by a certain angle (eg 30 degrees) / distance ).

As another example, a gesture / movement of a user's finger through a plurality of motion recognition sensors attached to a plurality of positions such as a user's finger joint and an arm (for example, the index finger of the user's right finger moves from the reference point to the direction / 12 o'clock direction). It detects the movement up to about 20cm (S320).

Thereafter, the controller 600 checks the motion (or movement) of the object in the 3D content displayed on the display unit 300 (or the HMD) based on the detected user's motion.

In addition, the controller 600 determines whether a preset special effect exists in relation to an object in the 3D content (or whether a preset special effect exists so as to interact with the operation of the object).

For example, the controller 600 may be configured to rotate the clockwise direction of the first vehicle in the first 3D content related to the racing game based on the detected rotation of the left and right hands of the user by 30 degrees in the clockwise direction. Check the rotation direction in degrees.

In addition, the controller 600 determines whether a special effect is preset in response to the rotation of the first vehicle in the first 3D content related to the racing game based on the detected rotation of the left and right hands of the user. Determine whether or not.

As another example, the control unit 600 is based on a movement of about 20 cm in the upward direction (or vertical direction / upward direction), which is the gesture of the detected user's finger, in the first 3D content related to the drone adjustment. Check the upward movement of the drone in the vertical direction.

In addition, the controller 600 generates a special effect in response to the upward movement of the first drone in the second 3D content related to the drone adjustment based on the upward movement, which is the gesture of the detected user's finger. It is determined whether the state is a preset state (S330).

As a result of the determination, when there is no preset special effect in relation to the object in the 3D content according to the detected user's motion, the controller 600 is displayed on the confirmed display unit 300 (or the HMD). The motion of the object is controlled according to the motion (or movement) of the object in the 3D content, and the control result (or the motion / movement result of the object in the 3D content) is displayed on the display unit 300 (or the HMD). do.

For example, when the special effect is not set in advance in response to the direction rotation of the first vehicle in the first 3D content related to the racing game, based on the clockwise rotation of the left and right hands of the detected user. The controller 600 controls the first vehicle in the first 3D content related to the racing game to rotate 30 degrees clockwise based on the detected rotation of the left and right hands of the user by 30 degrees clockwise. The first 3D content including the first vehicle rotated in the clockwise direction is displayed on the first HMD worn by the user and the display unit 300, respectively. In this case, the rotation angle of the first vehicle may be variously set according to the design of the designer, and may be configured to rotate on the screen corresponding to the detected rotation angle of the left and right hands of the user in the clockwise direction.

As another example, the special effect is not preset in response to the upward movement of the first drone in the second 3D content related to the drone adjustment based on the upward movement which is the gesture of the detected user's finger. In the non-state state, the control unit 600 is configured to raise the first drone in the second 3D content related to the drone adjustment in the vertical direction based on a 20 cm upward movement, which is a gesture of the detected user's finger. The display unit 300 displays the second 3D content including the first drone that controls and rises (or takes off) in the vertical direction. In this case, the rising distance in the vertical direction of the first drone may be variously set according to a designer's design, and the preset distance (for example, 1/5) is proportional to the movement (for example, 20 cm) of the gesture of the finger. 4 cm) may be configured to rise on the screen (S340).

As a result of the determination, when a preset special effect exists in relation to an object in the 3D content according to the detected user's motion, the controller 600 controls the plurality of air fans 700 disposed at different locations around the user. To generate a control signal for controlling the operation. In this case, the control unit 600 may generate a control signal including the step-by-step wind strength control information by PWM (Pulse Width Modulation).

In addition, the plurality of air fans 700 operate based on the generated control signal, so that the user can feel similar to the actual movement according to the moving direction, speed, etc. of the object in the virtual reality environment. It provides special effects (e.g. special effects that take into account wind speed, direction of wind blowing, etc.).

In addition, the controller 600 controls at least one or more of the plurality of air fans 700 to provide a special effect to the user, and at the same time, the 3D displayed on the identified display unit 300 (or the HMD). The operation of the object is controlled according to the operation (or movement) of the object in the content, and the control result (or the operation / movement result of the object in the 3D content) is displayed on the display unit 300 (or the HMD). .

For example, when the special effect is preset in response to the rotation of the first vehicle in the first 3D content related to the racing game, based on the clockwise rotation of the left and right hands of the detected user, the controller ( 600 is an operation state of the first air fan located at 1 o'clock position among the first to fourth air fans 700 installed at the 1 o'clock, 5 o'clock, 7 o'clock and 11 o'clock directions with respect to the user Generate a first control signal for controlling the control signal to a preset first level and a second control signal for controlling the operating state of the fourth air fan located at the 11 o'clock position to a second preset level, respectively, The first control signal and the second control signal are provided to corresponding first and fourth air fans, respectively.

In addition, the first air fan provides the user with wind corresponding to the intensity of the first level based on the first control signal, and the fourth air fan is based on the second control signal. The wind corresponding to the intensity may be provided to the user so that the user may feel the wind while making a right turn.

In addition, the controller 600 rotates the first vehicle in the first 3D content related to the racing game by 30 degrees clockwise based on the detected rotation of the left and right hands of the user by 30 degrees clockwise. The first 3D content including the first vehicle rotated in the clockwise direction is displayed on the first HMD and the display unit 300 worn by the user. In this case, the rotation angle of the first vehicle may correspond to the detected rotation angle of the left and right hands of the user in the clockwise direction.

As described above, the controller 600 transmits 3D content that is rotated in the clockwise direction of the first vehicle according to the clockwise rotation of the first vehicle in the first 3D content to the first HMD and the display unit 300. At the same time, the user may be provided with the wind, which is a special effect according to the clockwise rotation of the first vehicle.

As another example, a state in which a special effect is preset in response to an upward movement of the first drone in the second 3D content related to the drone adjustment based on the upward movement, which is the gesture of the detected user's finger. When the control unit 600 is in the 1 o'clock, 5 o'clock, 7 o'clock and 11 o'clock directions with respect to the user in order to provide a vertical air flow effect according to the upward movement of the first drone in the vertical direction, respectively The eleventh and fourteenth control signals are respectively generated for controlling the operations of the first, second, third and fourth air fans among the installed first to fourth air fans 700, respectively, Control signals to fourteenth control signals are respectively provided to the corresponding first to fourth air fans.

In addition, each of the first air fan to the fourth air fan provides the user with an upward airflow effect according to the rising operation of the first drone based on the eleventh to fourteenth control signals.

In addition, the controller 600 controls the first drone in the second 3D content related to the drone adjustment to take off in the vertical direction based on a movement of about 20 cm in the upward direction, which is a gesture of the detected user's finger. The 2D 3D content including the first drone taking off (or rising) in the vertical direction is displayed on the display unit 300. In this case, the rising distance in the vertical direction of the first drone may be configured to rise on the screen by a predetermined distance (for example, 4/5 which is 1/5) in proportion to the movement of the finger gesture (for example, 20 cm). .

As described above, the controller 600 displays 3D content rising in the vertical direction of the first drone in the vertical direction of the first drone in the second 3D content on the display unit 300. The wind may be provided to the user as a special effect according to the upward movement of the first drone in the vertical direction (S350).

As described above, after recognizing a user's motion, an embodiment of the present invention controls the operation of an object displayed on the display unit according to the recognized motion, and provides a special effect related to the object, thereby providing a user's haptic index. Maximum efficiency, miniaturization and mobility of equipment can improve overall system operation efficiency and improve usability.

The above description may be modified and modified by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

10: special effect providing system using motion fan based on motion recognition
100: communication unit 200: storage unit
300: display unit 400: audio output unit
500: sensor unit 600: control unit
700: air fan

Claims (8)

A display unit including a head mounted display (HMD) and displaying 3D content including an object whose movement is controlled according to a user input;
A sensor unit detecting a user's motion;
The operation of the object in the 3D content displayed on the display unit is checked based on the detected user's motion, and it is determined whether there is a preset special effect related to the object in the 3D content. Operation of an object in 3D content displayed on the identified display unit while generating a control signal for controlling the operation of a plurality of air fans disposed at different positions around the user when a preset special effect exists in relation to the object A control unit for controlling the operation of the object and displaying the object on the display unit; And
A special effect providing system using a motion recognition-based air fan including the plurality of air fans to provide a special effect based on the control signal. The method of claim 1,
The sensor unit,
A 360 degree camera that detects a motion of the user based on image information including the user; And
A special effect providing system using a motion recognition-based air fan, characterized in that it comprises at least one of a plurality of copper field recognition sensors worn on the user's body to detect the user's motion according to the user's movement. The method of claim 1,
The operation of the user,
The direction of both legs, the movement of the legs, the direction of the arms, the rotation of the arm clockwise or counterclockwise, the fist-squeezing motion, the pinching motion, the finger spreading motion, the palm touch action, the finger spread action, the finger bend motion, the hand Or at least one of a drawing operation using an arm, a shaking operation in one of a left and right direction, a vertical direction and a diagonal direction using a hand or an arm, and an operation by a combination of these operations. Special effect providing system using motion recognition based air fan. The method of claim 1,
The control unit,
As a result of the determination, when there is no preset special effect in relation to the object in the 3D content, controlling the operation of the object according to the operation of the object in the 3D content displayed on the confirmed display unit to display the display unit on the display unit. A special effect providing system using a motion recognition based air fan. The method of claim 1,
The control unit,
A special effect providing system using a motion recognition-based air fan, characterized in that for generating the control signal including the step-by-step wind strength control information by PWM (Pulse Width Modulation). The method of claim 1,
The plurality of air fans,
It operates based on the generated control signal, characterized in that to provide the user with a special effect in consideration of the wind speed and the direction of the wind blowing in accordance with the moving direction and speed of the object in the virtual reality environment Special effect providing system using motion recognition based air fan. Displaying, by the display unit, 3D content including an object whose movement is controlled according to a user input;
Sensing, by the sensor unit, a user's motion;
Confirming, by the controller, an operation of an object in 3D content displayed on the display unit based on the detected user's motion;
Determining, by the controller, whether a preset special effect exists in relation to an object in the 3D content;
As a result of the determination, when a predetermined special effect exists in relation to an object in the 3D content, the control unit generates a control signal for controlling the operation of a plurality of air fans arranged at different locations around the user; Simultaneously controlling the operation of the object in accordance with the operation of the object in the 3D content displayed on the confirmed display unit and displaying it on the display unit; And
And providing a special effect by the plurality of air fans based on the control signal. The method of claim 7, wherein
As a result of the determination, when there is no preset special effect in relation to the object in the 3D content, the control unit controls the operation of the object according to the operation of the object in the 3D content displayed on the confirmed display unit. The method for providing a special effect using a motion recognition-based air fan further comprises the step of displaying on the display.

Images