KR102167066B1 - System for providing special effect based on motion recognition and method thereof - Google Patents

Abstract

The present invention discloses a system and method for providing special effects using an air fan based on motion recognition. That is, after recognizing the user's motion, the present invention controls the motion of an object displayed on the display unit according to the recognized motion, and provides special effects related to the object, thereby maximizing the user's experience index and miniaturization of equipment. And, by enhancing mobility, the overall system operation efficiency can be improved, and usability can be improved.

Description

TECHNICAL FIELD [System for providing special effect based on motion recognition and method thereof]

The present invention relates to a system and method for providing a special effect using an air fan based on motion recognition. In particular, after recognizing a user's motion, the motion of an object displayed on a display unit is controlled according to the recognized motion, and The present invention relates to a system and method for providing special effects using an air fan based on motion recognition that provides related special effects.

Virtual Reality (VR) is a technology that mimics a specific environment or situation in reality through a computer, making it as if the user is interacting with the actual surrounding situation or environment.

The virtual reality service industry to which such virtual reality is applied is facing certain limits in its popularization despite the high growth of devices (eg, Samsung Gear VR, Oculus Rift, HTC VIVE, etc.).

In other words, the virtual reality service industry, where users can use virtual reality services in the fields of education, learning, games, and military simulations, has high atmospheric demand, but the cost of constructing a virtual reality space per unit is expensive, and simply uses HMD (Head Mounted Display). As it is limited to providing a virtual reality environment that has been used, there is a limit to the expansion of the user's experience, and there are inconveniences in use in terms of mobility and reusability of contents.

Korean Patent Publication No. 10-2018-0077588

An object of the present invention is a system for providing special effects using an air fan based on motion recognition that recognizes a user's motion, controls the motion of an object displayed on a display unit according to the recognized motion, and provides special effects related to the object And to provide the method.

A system for providing a special effect using an air fan based on motion recognition according to an embodiment of the present invention includes a display unit including a head mounted display (HMD) and displaying 3D content including an object whose movement is controlled according to a user input; A sensor unit for detecting a user's motion; Based on the detected user's motion, the motion of the object in the 3D content displayed on the display is checked, it is determined whether a special effect preset in relation to the object in the 3D content exists, and as a result of the determination, in the 3D content When there is a pre-set special effect in relation to the object, the motion of the object in the 3D content displayed on the checked display unit while generating a control signal for controlling the operation of a plurality of air fans arranged at different locations around the user A controller configured to control an operation of the object and display the object on the display; And the plurality of air fans providing special effects based on the control signal.

As an example related to the present invention, the sensor unit may include a 360 degree camera that detects a motion of the user based on image information including the user; And at least one of a plurality of motion recognition sensors worn on the user's body to sense the user's motion according to the user's motion.

As an example related to the present invention, the user's motion may include a direction of both legs, a motion of a leg, a direction of an arm, a rotation of the arm in a clockwise or counterclockwise direction, a fist clenching motion, a pinching motion, a finger spreading motion, Palm touch operation, finger spreading operation, finger bending operation, drawing operation using hand or arm, shaking operation in any one of left and right direction using hand or arm, vertical direction and diagonal direction, and combinations of these operations At least one of the following operations may be included.

As an example related to the present invention, when, as a result of the determination, there is no pre-set special effect in relation to the object in the 3D content, the control unit may determine the object according to the operation of the object in the 3D content displayed on the By controlling the operation of the display can be displayed on the display.

As an example related to the present invention, the controller may generate the control signal including step-by-step wind intensity control information by PWM (Pulse Width Modulation).

As an example related to the present invention, the plurality of air fans operate based on the generated control signal, and the speed of the wind and the direction in which the wind blows according to the moving direction and speed of the object in the virtual reality environment to the user It can provide a special effect taking into account in a surround form.

A method of providing a special effect using an air fan based on motion recognition according to an embodiment of the present invention includes, by a display unit, displaying 3D content including an object whose movement is controlled according to a user input; Detecting a user's motion by a sensor unit; Checking, by a control unit, an operation of an object in the 3D content displayed on the display unit based on the detected user's motion; Determining, by the control unit, whether there is a special effect preset in relation to the object in the 3D content; As a result of the determination, when there is a preset special effect in relation to the object in the 3D content, the control unit generates a control signal for controlling the operation of a plurality of air fans disposed at different locations around the user; and Simultaneously controlling the motion of the object according to the motion of the object in the 3D content displayed on the checked display and displaying it on the display; And providing a special effect based on the control signal by the plurality of air fans.

As an example related to the present invention, when, as a result of the determination, there is no pre-set special effect with respect to the object in the 3D content, the control unit determines the operation of the object in the 3D content displayed on the checked display unit. The method may further include controlling the operation of the object and displaying it on the display.

After recognizing the user's motion, the present invention controls the motion of an object displayed on the display unit according to the recognized motion, and provides special effects related to the object, thereby maximizing the user's experience index, miniaturization and mobility of the device. It has the effect of enhancing overall system operation efficiency and improving user convenience.

1 is a block diagram showing the configuration of a system for providing special effects using an air fan based on motion recognition according to an embodiment of the present invention.
2 is a diagram showing an example of a control signal according to an embodiment of the present invention.
3 is a flowchart illustrating a method of providing special effects using an air fan based on motion recognition according to an embodiment of the present invention.

It should be noted that the technical terms used in the present invention are used only to describe specific embodiments, and are not intended to limit the present invention. In addition, the technical terms used in the present invention should be interpreted as generally understood by those of ordinary skill in the technical field to which the present invention belongs, unless otherwise defined in the present invention, and is excessively comprehensive. It should not be construed as a human meaning or an excessively reduced meaning. In addition, when a technical term used in the present invention is an incorrect technical term that does not accurately express the spirit of the present invention, it should be replaced with a technical term that can be correctly understood by those skilled in the art. In addition, general terms used in the present invention should be interpreted as defined in the dictionary or according to the context before and after, and should not be interpreted as an excessively reduced meaning.

In addition, the singular expression used in the present invention includes a plurality of expressions unless the context clearly indicates otherwise. In the present invention, terms such as “consisting of” or “comprising” should not be construed as necessarily including all of the various components or steps described in the invention, and some components or some steps may not be included. It should be construed that it may or may further include additional components or steps.

In addition, terms including ordinal numbers such as first and second used in the present invention may be used to describe the constituent elements, but the constituent elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of the reference numerals, and redundant descriptions thereof will be omitted.

In addition, in describing the present invention, when it is determined that a detailed description of a related known technology may obscure the subject matter of the present invention, a detailed description thereof will be omitted. In addition, it should be noted that the accompanying drawings are only for easily understanding the spirit of the present invention and should not be construed as limiting the spirit of the present invention by the accompanying drawings.

1 is a block diagram showing the configuration of a system 10 for providing a special effect using an air fan based on motion recognition according to an embodiment of the present invention.

As shown in FIG. 1, the system 10 for providing a special effect using an air fan based on motion recognition includes a communication unit 100, a storage unit 200, a display unit 300, a voice output unit 400, and a sensor unit ( 500), a control unit 600 and an air fan 700. Not all components of the special effect providing system 10 using the motion recognition-based air fan shown in FIG. 1 are essential components, and motion recognition-based air is generated by more components than the components shown in FIG. 1. A special effect providing system 10 using a fan may be implemented, or a special effect providing system 10 using an air fan based on motion recognition may be implemented with fewer components.

The communication unit 100 communicates with an internal element or at least one external terminal through a wired/wireless communication network. In this case, the external terminal may include a server (not shown), another terminal (not shown), and the like. Here, as the wireless Internet technology, wireless LAN (WLAN), DLNA (Digital Living Network Alliance), Wibro (Wireless Broadband: Wibro), Wimax (World Interoperability for Microwave Access: Wimax), HSDPA (High Speed Downlink Packet Access) ), High Speed Uplink Packet Access (HSUPA), IEEE 802.16, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), Wireless Mobile Broadband Service (WMBS), etc. In addition, the communication unit 100 transmits and receives data according to at least one wireless Internet technology in a range including Internet technologies not listed above. In addition, short-range communication technologies include Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and Near Field Communication (NFC). , Ultra Sound Communication (USC), Visible Light Communication (VLC), Wi-Fi, Wi-Fi Direct, and the like may be included. In addition, wired communication technologies may include power line communication (PLC), USB communication, Ethernet, serial communication, optical/coaxial cable, and the like.

In addition, the communication unit 100 may mutually transmit information with an arbitrary terminal through a universal serial bus (USB).

In addition, the communication unit 100 includes technical standards or communication methods for mobile communication (for example, GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), CDMA2000 (Code Division Multi Access 2000)), EV -DO (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), WCDMA (Wideband CDMA), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc.), transmits and receives radio signals to and from the base station, the server, and the other terminal.

In addition, the communication unit 100 receives software (or program) for implementing a virtual reality system transmitted from the server under the control of the controller 600.

The storage unit 200 stores various user interfaces (UI), graphic user interfaces (GUI), and the like.

In addition, the storage unit 200 stores data and programs necessary for the operation of the special effect providing system 10 using the motion recognition-based air fan.

That is, the storage unit 200 uses a plurality of application programs (application programs or applications) driven by the special effect providing system 10 using the motion recognition-based air fan, and the motion recognition-based air fan. Data and instructions for the operation of the special effect providing system 10 may be stored. At least some of these application programs may be downloaded from an external server through wireless communication. In addition, at least some of these application programs exist on the special effect providing system 10 using a motion recognition-based air fan from the time of shipment for the basic function of the special effect providing system 10 using a motion recognition-based air fan. I can. On the other hand, the application program is stored in the storage unit 200 and installed in the special effect providing system 10 using the motion recognition-based air fan, and the motion recognition-based air fan is used by the control unit 600. It may be driven to perform an operation (or function) of the effect providing system 10.

In addition, the storage unit 200 is a flash memory type (Flash Memory Type), a hard disk type (Hard Disk Type), a multimedia card micro type (Multimedia Card Micro Type), a card type memory (e.g., SD or XD Memory, etc.), magnetic memory, magnetic disk, optical disk, RAM (Random Access Memory: RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory: ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), It may include at least one storage medium among Programmable Read-Only Memory (PROM). In addition, the motion recognition-based special effect providing system 10 using an air fan operates a web storage that performs a storage function of the storage unit 200 on the Internet, or is related to the web storage. It may work.

In addition, the storage unit 200 stores (or installs) a program for implementing (or playing/executing) the virtual reality system received through the communication unit 100.

The display unit (or display unit) 300 may display various contents such as various menu screens using a user interface and/or a graphic user interface stored in the storage unit 200 under the control of the controller 600. have. Here, the content displayed on the display unit 300 includes various text or image data (including various information data) and a menu screen including data such as icons, list menus, and combo boxes. In addition, the display unit 300 may be a touch screen.

In addition, the display unit 300 includes a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display. It may include at least one of (Flexible Display), 3D display, e-ink display, and LED (Light Emitting Diode).

In addition, the display unit 300 may include a head mounted display (HMD) capable of displaying 3D content, a display in the form of glasses, and the like.

In addition, the display unit 300 displays 2D content including 2D objects related to a specific program, 3D content including 3D objects, and the like under the control of the controller 600.

The audio output unit 400 outputs audio information included in a signal processed by a predetermined signal by the control unit 600. Here, the audio output unit 400 may include a receiver, a speaker, a buzzer, and the like.

In addition, the voice output unit 400 outputs a guide voice generated by the control unit 600.

In addition, the audio output unit 400 outputs audio information (or sound effect) corresponding to the 2D content, 3D content, etc. by the control unit 600.

The sensor unit (or motion recognition unit) 500 includes a 360-degree camera for detecting a motion (or movement) of a user adjacent to the special effect providing system 10 using the motion recognition-based air fan, and a motion recognition sensor ( Motion Recognition Sensor). Here, the user's motion (or movement) is the direction of both legs, the movement of the legs (including movement, rotation, etc.), the direction of the arm, the rotation of the arm clockwise/counterclockwise, the fist clenching motion, and the finger Pinch operation, finger spreading operation, palm touch operation, finger spreading operation, finger bending operation, drawing operation using hand/arm, shaking operation in left/right/up/down direction/diagonal direction using hand/arm, between these movements It includes actions by combination, etc.

That is, the sensor unit 500 is a method for recognizing known various objects (or users) based on image information (eg, including a plurality of consecutive images, videos, etc.) acquired through the 360-degree camera (or camera). To detect the user's motion.

In addition, the sensor unit 500 detects the user's motion based on sensing information acquired through a plurality of motion recognition sensors worn (or attached) to the user's body. Here, the motion recognition sensor is a sensor that recognizes the motion or position of an object, a geomagnetism sensor, an acceleration sensor, a gyro sensor, an inertial sensor, an altimeter, It includes a sensor such as a vibration sensor, and sensors related to motion recognition may be additionally included. At this time, the sensor unit 500 is the movement of the user, for example, the inclined direction, the inclined angle and/or the inclined speed of the user's body part (including, for example, arm, hand, finger, etc.) , It detects information including the vibration direction and/or the number of vibrations in the vertical/left/right/diagonal directions. Here, the sensed information (inclined direction, inclined angle and/or inclined speed, vibration direction and/or number of vibrations) is digitized through a digital signal processing process, and the digitized information is converted to the controller 600 ).

In addition, the sensor unit 500 may be provided in the form of a wearable device and may recognize information according to the user's movement while worn on the user's body (eg, including hands, fingers, arms, legs, etc.) .

In addition, the sensor unit 500 may include one or more sensors for sensing at least one of surrounding environment information and user information surrounding the special effect providing system 10 (or user) using a motion recognition-based air fan. have. For example, the sensor unit 500 includes a proximity sensor, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, and a gravity sensor (G- sensor), gyroscope sensor, motion sensor, inertial sensor, altimeter, vibration sensor, RGB sensor, infrared sensor (IR sensor: infrared sensor), fingerprint recognition sensor ( finger scan sensor), ultrasonic sensor (ultrasonic sensor), optical sensor (for example, camera), microphone, battery gauge, environmental sensor (for example, barometer, hygrometer, thermometer, It may include at least one of a radiation sensor, a heat sensor, a gas sensor, and the like), and a chemical sensor (eg, an electronic nose, a healthcare sensor, a biometric sensor, etc.). Meanwhile, the system 10 for providing a special effect using an air fan based on motion recognition disclosed in the present specification may combine and utilize information sensed by at least two or more of these sensors.

In addition, the sensor unit 500 has a motion (or motion) recognition function, a voice recognition function located around the special effect providing system 10 using the motion recognition-based air fan (or a preset area in the VR system) , Lip movement recognition function, eye movement recognition function (or gaze tracking function), etc. may be performed.

In addition, the sensor unit 500 includes a motion recognition module for recognizing the user's motion, lip motion, or pupil motion, and a voice recognition module for voice recognition.

The controller or microcontroller unit (MCU) 600 executes the overall control function of the special effect providing system 10 using the motion recognition-based air fan.

In addition, the controller 600 executes an overall control function of the special effect providing system 100 using an air fan based on motion recognition by using programs and data stored in the storage unit 200. The control unit 600 may include RAM, ROM, CPU, GPU, and bus, and RAM, ROM, CPU, GPU, etc. may be connected to each other through a bus. The CPU can access the storage unit 200 and perform booting using the O/S stored in the storage unit 200, and use various programs, contents, data, etc. stored in the storage unit 200 Thus, various operations can be performed.

In addition, the control unit 600 outputs 3D content according to the user's selection through the display unit 300 and/or the audio output unit 400.

That is, when a program for providing a specific virtual reality function is selected (or executed) among programs installed in the special effect providing system 10 using the motion recognition-based air fan, the controller 600 The 3D content is output through the display unit 300 and/or the audio output unit 400. Here, in the 3D content, one or more objects whose movement is controlled according to a user's motion or a user input through an input unit (not shown) may be included.

In this case, when the user is wearing the HMD, the controller 600 may provide (or display/output) the corresponding 3D content displayed on the display unit 300 through the HMD. Here, the 3D content includes a 3D image/video, related sound effects (or audio information), and the like.

In addition, the control unit 600 processes the motion information of the user's hand acquired from a 360-degree camera or a motion recognition sensor included in the sensor unit 500 to rotate the x-axis in which the palm or finger rotates around the x-axis. After extracting the location information including the value, the y-axis rotation value in which the palm or finger rotates around the y-axis, and the z-axis rotation value in which the palm or finger rotates around the z-axis, the corresponding location information is preset motion reference information Compared with, the converted value may be converted into a value corresponding to the motion reference information, and the converted value may be linked with a motion of a specific object in the 3D content previously being displayed (or being executed).

In addition, the controller 600 checks the motion (or movement) of an object in the 3D content displayed on the display unit 300 (or the HMD) based on the sensed user's motion.

In addition, the control unit 600 determines whether there is a special effect preset in relation to the object in the 3D content (or whether there is a special effect preset to interact with the motion of the object).

As a result of the determination, if there is no special effect set in advance in relation to the object in the 3D content according to the detected user's motion, the controller 600 is displayed on the identified display unit 300 (or the HMD). Controls the motion of the object according to the motion (or movement) of the object in the 3D content, and displays the control result (or the motion/motion result of the object in the 3D content) on the display unit 300 (or the HMD) do.

As a result of the determination, when there is a preset special effect in relation to the object in the 3D content according to the detected user's motion, the control unit 600 includes a plurality of air fans 700 disposed at different locations around the user. Generate a control signal to control the operation of ).

In this case, as shown in FIG. 2, the control unit 600 may generate a control signal including step-by-step wind intensity adjustment information by PWM (Pulse Width Modulation).

In addition, the control unit 600 controls at least one of the plurality of air fans 700 so that a special effect is provided to the corresponding user, and at the same time, a 3D display unit displayed on the identified display unit 300 (or the HMD) The motion of the object is controlled according to the motion (or movement) of the object in the content, and the control result (or the motion/motion result of the object in the 3D content) is displayed on the display unit 300 (or the HMD). .

In addition, the control unit 600 is installed in the special effect providing system 10 (or the storage unit 200) using the motion recognition-based air fan based on updated program information provided from a server (not shown). You can update the program.

In this way, the motion recognition-based special effect providing system 10 using the air fan controls the motion of the object for virtual reality experience according to the motion (or movement) of the user located in the space installed in the form of a booth, etc. , It provides realistic information applying a multimodality system, and can maximize the experience of the virtual reality.

The air fan (air foil fan) 700 may be configured as single or plural.

When the air fan 700 is configured in a single form, the air fan 700 is located around a predetermined area in the system 10 for providing a special effect using the motion recognition-based air fan in the form of a booth. A special effect (including, for example, wind) is provided to a corresponding user while moving according to a control signal generated by the controller 600 through a movable rail (not shown).

In addition, when the air fan 700 is configured in a single shape, the air fan 700 is fixed to a specific position and then the motion recognition-based air fan according to a control signal generated by the controller 600 A special effect (including, for example, wind) may be provided to a corresponding user located around a preset area in which the user in the special effect providing system 10 is located.

In addition, when the air fan 700 is configured in a plurality of forms, each of the plurality of air fans 700 may be located by a user in the special effect providing system 10 using the motion recognition-based air fan in the form of a booth. Each is installed at a preset location around a preset area, and at least one of the plurality of air fans 700 operates according to a control signal generated by the control unit 600 to provide a special It provides effects (including wind, etc.).

In this way, when the air fan 700 is configured in a single shape, the air fan 700 is configured to move around the user, thereby providing a special effect to the user in any one direction and angle of 360 degrees. I can.

In addition, as described above, when the air fan 700 is configured in a plurality of forms in a surround form, the plurality of air fans 700 provide the user with an actual movement in accordance with the movement direction and speed of the object in a virtual reality environment. In order to feel the similarity, special effects in the form of surround (for example, special effects that take into account the speed of the wind and the direction the wind is blowing) can be provided.

In the embodiment of the present invention, providing a special effect based on virtual reality is described, but the present invention is not limited thereto, and a special effect according to the present invention may also be provided based on a mixed reality (MR).

In the embodiment of the present invention, it is described that a surround type of wind using the air fan 700 is provided to the user as a special effect, but the present invention is not limited thereto, and a special effect for special effects other than the air fan 700 It is also possible to configure a providing unit (not shown).

The special effect providing unit further includes a haptic interface (not shown), and the like, and information about not only wind to the user through the haptic interface, but also kinesthesia related to the object (for example, mass related to the object, Information on the texture (tactility, etc.) associated with the object (e.g., information on the surface pattern, small shapes, smoothness and roughness of the surface, feeling of coolness, etc. related to the object) Including), etc. may be provided.

The haptic interface (or haptic part/tactile interface) refers to the entire system that transmits tactile sensation to the user in a broad sense, and is a hardware device that generates tactile sensation through physical force transmission to the user by direct contact with the user. Haptic device, remote contact (Teletaction) environment, or a virtual environment consisting of computer graphics and physical characteristics, and the role of transmitting the movement of the haptic device to the virtual environment and the contact force with the virtual environment to the haptic device. It may be configured as a controller (not shown) (or the controller 600).

The ultimate purpose of the haptic interface is to allow the user to feel the same physical characteristics of the virtual environment (eg, AR environment, VR environment, MR environment, etc.) or the real environment through the haptic device.

In addition, in the haptic interface, a virtual agent representing a position change of the haptic device exists in the virtual environment so that the haptic device and the virtual environment exist in one environment. If the force generated by the virtual delegate hitting the virtual object is Fe, and the force felt through the haptic device by a person is Fm, it is an ideal haptic interface system to control the two values to be the same.

In order to implement an ideal haptic interface, research on an accurate modeling method and hardware and control method that can implement the model should be supported.

In addition, the haptic interface is limited to a hardware part in a narrow sense and may be used in the same meaning as the haptic device.

In addition, the haptic interface can be broadly divided into two fields from the viewpoint of hardware development, that is, from the viewpoint of developing the haptic device.

That is, a person feels the sense of touch through two main paths, and the haptic interface is also being developed by dividing into these two main fields.

One of the fields of such a haptic interface is related to muscle sensation, and when we feel the mass of an object, the degree of firmness and softness, the appearance of an object, etc., it is a phenomenon in which the sense of touch is felt through the movement of muscles and joints. Since this process is physically related to force, the haptic device has a mechanical structure such as a robot shape that is advantageous for generating force, and a haptic device made for this purpose is called a force feedback device. do.

In addition, one of the fields of haptic interfaces is related to texture, which is a phenomenon in which the skin is directly in contact with the surface when feeling the pattern or small shapes of the surface in contact, the smooth and roughness of the surface, and the feeling of cold and warmth. Since this process is composed of various complex physical elements, the shapes of haptic devices to reproduce textures are also very diverse, and a device made for the purpose of delivering such textures is called a texture display device.

In addition, the haptic interface greatly increases the realism of a remote environment or a virtual environment through the delivery of tactile sensation, and provides a function to touch and buy a product in an online shopping mall that has only been seen with the eye in a network-based environment. It is possible to provide an online museum function to touch relics, and to provide a function of conveying emotions through tactile sensations such as shaking hands and tapping shoulders in a messenger environment based on text. It can provide an interactive function between.

In this way, after recognizing the user's motion, the motion of the object displayed on the display unit can be controlled according to the recognized motion, and special effects related to the object can be provided.

Hereinafter, a method of providing a special effect using an air fan based on motion recognition according to the present invention will be described in detail with reference to FIGS. 1 to 3.

3 is a flowchart illustrating a method of providing a special effect using an air fan based on motion recognition according to an embodiment of the present invention.

First, the control unit 600 outputs 3D content according to a user selection through the display unit 300 and/or the audio output unit 400. Here, in the 3D content, one or more objects whose movement is controlled according to a user's motion or a user input through an input unit (not shown) may be included.

In this case, when the user is wearing the HMD, the controller 600 may provide (or display/output) the corresponding 3D content displayed on the display unit 300 through the HMD. Here, the 3D content includes a 3D image/video, related sound effects (or audio information), and the like.

As an example, when executing a racing game based on a user input in a VR system for a racing game, the controller 600 attaches the first HMD worn by the user to a first vehicle related to the racing game. The first 3D content including an object is displayed, the corresponding first 3D content is displayed on the display unit 300, and the audio information (or sound effect) corresponding to the first 3D content is output to the audio output unit 400 Output through

As another example, when performing drone adjustment based on a user input in a VR system for drone control, the controller 600 displays an object related to the first drone related to the drone control through the display unit 300. The included second 3D content is displayed, and audio information (or sound effect) corresponding to the second 3D content is output through the audio output unit 400 (S310).

Thereafter, the sensor unit 500 detects a user's motion (or movement). In this case, the sensor unit 500 detects a user's motion based on image information acquired through a 360-degree camera (not shown), or a plurality of motion recognition sensors (not shown) worn (or attached) to the user's body. Based on the sensing information acquired through ), the motion of the corresponding user may be detected. Here, the user's motion (or movement) is the direction of both legs, the movement of the legs (including movement, rotation, etc.), the direction of the arm, the rotation of the arm clockwise/counterclockwise, the fist clenching motion, and the finger Pinch operation, finger spreading operation, palm touch operation, finger spreading operation, finger bending operation, drawing operation using hand/arm, shaking operation in left/right/up/down direction/diagonal direction using hand/arm, between these movements It includes actions by combination, etc.

As an example, the camera included in the sensor unit 500 moves the user's left hand and right hand (for example, when the user's left hand is at 9 o'clock and the user's right hand is at 3 o'clock, the left hand of the user This rotates from 9 o'clock to 12 o'clock, and the user's right hand rotates from 3 o'clock to 6 o'clock/clockwise rotation of the user's left and right hands by a certain angle (for example, 30 degrees)/distance ).

As another example, gesture/movement of the user's finger through a plurality of motion recognition sensors attached to a plurality of locations such as the user's finger joint and arm (for example, the index finger of the user's right finger is A movement that rises about 20cm) is sensed (S320).

Thereafter, the controller 600 checks the motion (or movement) of the object in the 3D content displayed on the display unit 300 (or the HMD) based on the sensed user's motion.

In addition, the control unit 600 determines whether there is a special effect preset in relation to the object in the 3D content (or whether there is a special effect preset to interact with the motion of the object).

As an example, the control unit 600 is based on the detected rotation of the user's left hand and right hand by 30 degrees in the clockwise direction, based on the clockwise direction 30 for the first vehicle in the first 3D content related to the racing game. Check the rotational motion in the road direction.

In addition, the control unit 600 determines whether a special effect is set in advance in response to the direction rotation of the first vehicle in the first 3D content related to the racing game based on the detected rotation of the left hand and the right hand in the clockwise direction of the user. Judge whether or not.

As another example, the control unit 600 is based on a movement of about 20 cm in the upward direction (or vertical direction/upward direction), which is the detected gesture of the user's finger, and the first 3D content in the second 3D content related to the drone adjustment. Check the vertical movement of the drone.

In addition, the control unit 600 has a special effect in response to the upward motion of the first drone in the second 3D content related to the drone adjustment based on the upward movement, which is a gesture of the user's finger detected. It is determined whether is a preset state (S330).

As a result of the determination, if there is no special effect set in advance in relation to the object in the 3D content according to the detected user's motion, the controller 600 is displayed on the identified display unit 300 (or the HMD). Controls the motion of the object according to the motion (or movement) of the object in the 3D content, and displays the control result (or the motion/motion result of the object in the 3D content) on the display unit 300 (or the HMD) do.

For example, when a special effect is not set in advance in response to the direction rotation of the first vehicle in the first 3D content related to the racing game based on the detected rotation of the left hand and the right hand in the clockwise direction, the The controller 600 controls the first vehicle in the first 3D content related to the racing game to rotate in the clockwise direction by 30 degrees based on the detected rotation of the user's left hand and right hand by 30 degrees in the clockwise direction. , The first 3D content including the first vehicle rotated in the clockwise direction is displayed on the first HMD worn by the user and the display unit 300, respectively. In this case, the rotation angle of the first vehicle may be variously set according to the design of the designer, and may be configured to rotate on the screen in response to the detected rotation angle of the left and right hands of the user in the clockwise direction.

As another example, a special effect is not preset in response to the upward motion of the first drone in the second 3D content related to the drone adjustment based on the upward movement, which is a gesture of the user's finger. When not in a state, the controller 600 causes the first drone in the second 3D content related to the drone adjustment to rise in the vertical direction based on the detected movement of the user's finger, which is about 20 cm upward. Control and display the second 3D content including the first drone rising (or taking off) in the vertical direction on the display unit 300. In this case, the ascending distance in the vertical direction of the first drone may be variously set according to a designer's design, and a preset distance (for example, 1/5) in proportion to the movement of the finger's gesture (for example, 20 cm). 4cm) can be configured to rise on the screen (S340).

As a result of the determination, when there is a preset special effect in relation to the object in the 3D content according to the detected user's motion, the control unit 600 includes a plurality of air fans 700 disposed at different locations around the user. Generate a control signal to control the operation of ). In this case, the control unit 600 may generate a control signal including information on adjusting wind intensity by step by PWM (Pulse Width Modulation).

In addition, the plurality of air fans 700 are operated based on the generated control signal, so that the user can feel the similarity of moving according to the moving direction and speed of the object in a virtual reality environment. It provides special effects (for example, special effects that take into account the speed of the wind and the direction the wind is blowing in).

In addition, the control unit 600 controls at least one of the plurality of air fans 700 so that a special effect is provided to the corresponding user, and at the same time, a 3D display unit displayed on the identified display unit 300 (or the HMD) The motion of the object is controlled according to the motion (or movement) of the object in the content, and the control result (or the motion/motion result of the object in the 3D content) is displayed on the display unit 300 (or the HMD). .

For example, when a special effect is set in advance in response to the direction rotation of the first vehicle in the first 3D content related to the racing game based on the detected rotation of the left hand and the right hand in the clockwise direction, the control unit ( 600) is the operation state of the first air fan located at 1 o'clock among the first to fourth air fans 700 installed at 1 o'clock, 5 o'clock, 7 o'clock, and 11 o'clock, respectively, centered on the user. Each of the first control signals for controlling the power to the first preset level and the second control signals for controlling the operation state of the fourth air fan located at 11 o'clock to the second preset level are respectively generated, The first control signal and the second control signal are provided to the corresponding first and fourth air fans, respectively.

In addition, the first air fan provides wind corresponding to the intensity of a first level to the user based on the first control signal, and the fourth air fan provides a second level of wind based on the second control signal. The wind corresponding to the intensity may be provided to the user so that the user can feel the wind while turning right.

In addition, the control unit 600 rotates the first vehicle in the first 3D content related to the racing game in the clockwise direction by 30 degrees based on the detected rotation of the user's left hand and right hand by 30 degrees in the clockwise direction. And the first 3D content including the first vehicle rotated in the clockwise direction is displayed on the first HMD worn by the user and the display unit 300, respectively. In this case, the rotation angle of the first vehicle may correspond to a clockwise rotation angle of the detected user's left and right hands.

In this way, the controller 600 transmits 3D content that rotates clockwise of the first vehicle in the clockwise direction of the first vehicle in the first 3D content to the first HMD and the display unit 300. At the same time as each display, wind, which is a special effect according to the clockwise rotation of the corresponding first vehicle, may be provided to the user.

As another example, a state in which a special effect is preset in response to the upward motion of the first drone in the second 3D content related to the drone adjustment based on the upward movement, which is a gesture of the user's finger detected. In this case, the control unit 600 operates at 1 o'clock, 5 o'clock, 7 o'clock, and 11 o'clock, respectively, in order to provide a vertical airflow effect according to the upward motion of the first drone. Generate 11th to 14th control signals respectively for controlling the operation of the 1st, 2nd, 3rd and 4th air fans among the installed 1st to 4th air fans 700, and the generated 11th Control signals to 14th control signals are provided to the corresponding first to fourth air fans, respectively.

In addition, each of the first to fourth air fans provides the user with an upward airflow effect according to the ascending operation of the first drone based on the eleventh to fourteenth control signals.

In addition, the controller 600 controls the first drone in the second 3D content related to the drone adjustment to take off in the vertical direction based on the detected movement of the user's finger, which is about 20 cm upward, The second 3D content including the first drone taking off (or ascending) in the vertical direction is displayed on the display unit 300. In this case, the ascending distance of the first drone in the vertical direction may be configured to rise on the screen by a preset distance (for example, 1/5, 4 cm) in proportion to the movement of the finger gesture (for example, 20 cm). .

In this way, the controller 600 displays the 3D content rising in the vertical direction of the first drone in the vertical direction of the first drone in the second 3D content on the display unit 300 Wind, which is a special effect according to the upward motion of the first drone, may be provided to the user (S350).

As described above, after recognizing a user's motion, an embodiment of the present invention controls the motion of an object displayed on the display unit according to the recognized motion, and provides special effects related to the object, so that the user's experience index It can maximize the efficiency of operation of the entire system and improve the convenience of use by maximizing the size of the equipment and enhancing the mobility.

The above contents may be modified and modified without departing from the essential characteristics of the present invention by those of ordinary skill in the technical field to which the present invention belongs. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain the technical idea, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

10: Motion recognition-based system for providing special effects using an air fan
100: communication unit 200: storage unit
300: display unit 400: audio output unit
500: sensor unit 600: control unit
700: air fan

Claims (8)

A display unit including a head mounted display (HMD) and displaying 3D content including an object whose movement is controlled according to a user input;
A sensor unit detecting a user's motion;
Based on the detected user's motion, the motion of the object in the 3D content displayed on the display is checked, it is determined whether a special effect preset in relation to the object in the 3D content exists, and as a result of the determination, in the 3D content Generates a first control signal for controlling the operation of a plurality of air fans arranged at different locations around the user when there is a preset special effect in relation to the object, and provides tactile information related to the object in the 3D content to the user. A control unit that generates a second control signal to be provided and controls the operation of the object according to the operation of the object in the 3D content displayed on the identified display unit and displays it on the display unit;
The plurality of air fans providing special effects based on the first control signal; And
A system for providing special effects using a motion recognition-based air fan including a haptic interface that provides tactile information, which is information on a muscular sense related to the object, based on the second control signal. The method of claim 1,
The sensor unit,
A 360-degree camera that detects the user's motion based on image information including the user; And
A system for providing special effects using a motion recognition-based air fan, comprising at least one of a plurality of motion recognition sensors worn on a user's body and detecting the user's motion according to the user's motion. The method of claim 1,
The user's action,
Direction of both legs, movement of the legs, direction of the arm, rotation of the arm clockwise or counterclockwise, a fist-clamping motion, a pinch-up motion, a finger spreading motion, a palm touch motion, a finger straightening motion, a finger bending motion, a hand Or drawing a figure using an arm, a shaking motion in any one of a left-right direction, an up-down direction, and a diagonal direction using a hand or arm, and a motion by a combination of these motions. A system that provides special effects using an air fan based on motion recognition. The method of claim 1,
The control unit,
As a result of the determination, when there is no pre-set special effect in relation to the object in the 3D content, the operation of the object is controlled and displayed on the display unit according to the motion of the object in the 3D content displayed on the checked display unit. A system for providing special effects using an air fan based on motion recognition. The method of claim 1,
The control unit,
A system for providing a special effect using an air fan based on motion recognition, characterized in that generating the first control signal including information on stepwise wind intensity adjustment by PWM (Pulse Width Modulation). The method of claim 1,
The plurality of air fans,
Operates based on the generated first control signal, and provides the user with a special effect in consideration of the speed of the wind and the direction of blowing according to the moving direction and speed of the object in a virtual reality environment in a surround form. A system for providing special effects using an air fan based on motion recognition. Displaying, by the display unit, 3D content including an object whose movement is controlled according to a user input;
Detecting a user's motion by the sensor unit;
Checking, by a control unit, an operation of an object in the 3D content displayed on the display unit based on the detected user's motion;
Determining, by the control unit, whether a special effect preset in relation to the object in the 3D content exists;
As a result of the determination, when there is a preset special effect in relation to the object in the 3D content, the control unit generates a first control signal for controlling the operation of a plurality of air fans disposed at different locations around the user And generates a second control signal for providing tactile information related to the object in the 3D content to the user, and controls the operation of the object according to the motion of the object in the 3D content displayed on the identified display unit and displays it on the display unit The step of doing;
Providing a special effect based on the first control signal by the plurality of air fans; And
A method for providing a special effect using an air fan based on motion recognition, comprising the step of providing, by a haptic interface, tactile information, which is information on a muscular sense related to the object, based on the second control signal. The method of claim 7,
As a result of the determination, when there is no special effect preset in relation to the object in the 3D content, the control unit controls the operation of the object according to the motion of the object in the 3D content displayed on the A method for providing special effects using an air fan based on motion recognition, further comprising displaying on a display unit.

Images