KR102009753B1 - System for processing object based on virtual reality and operating method thereof - Google Patents

Abstract

The present invention relates to an object processing apparatus based on virtual reality and a method of operating the same. According to an embodiment of the present invention, in the method of operating an object processing apparatus based on a virtual reality, the first sensing data is transmitted from an object device connected through wireless communication. Receiving; Implementing the environment of the object device and the object device as an object located in the virtual reality space and the virtual reality space based on the sensing data; Receiving last sensing data from the object apparatus, and detecting a change of the object apparatus from the last sensing data; And implementing a change of the object based on the detected change. It may be applied to another embodiment.

Description

Object processing device based on virtual reality and its operation method {SYSTEM FOR PROCESSING OBJECT BASED ON VIRTUAL REALITY AND OPERATING METHOD THEREOF}

The present invention relates to an apparatus for processing an object based on virtual reality and a method of operating the same, and more particularly, to an apparatus for implementing a motion applied to an object through virtual reality and a method of operating the same.

Virtual reality (VR) provides the illusion of reality. Virtual reality technology is commonly used in computing systems in the fields of education, entertainment, training and medicine. The virtual reality environment provides a virtual space for the experience of various human sensory phenomena of sight, hearing, touch, smell (smell), and palate (taste).

Among them, tactile sense is the most dominant sensation after sight and hearing in computer-human interaction, and much research has been carried out to implement perception in virtual reality in order to provide a virtual environment with higher quality of immersion and interactivity. It is becoming.

Patent document 10-2017-0037220 describes a haptic virtual reality gaming system that can be used to help elderly and children as well as disabled people to improve their tactile texture and ability to perceive 3D shapes. A method of providing a haptic puzzle using the method, the method comprising: displaying an icon corresponding to a query object and a plurality of example objects on a display unit; Providing a user with a haptic sense of each of the query object and a plurality of example objects using a haptic device; Receiving a selection of an example object having the same texture as the query object among the plurality of example objects from the user by using an input unit; And determining whether the selected example object and the query object have the same texture. The method provides a haptic puzzle using a haptic virtual reality gaming system.

Although the haptic virtual reality gaming system discloses that the cognitive ability can be improved by distinguishing the difference of texture and providing it to the user, the haptic virtual reality gaming system feels the difference of haptic provided through the system and selects one of the displayed examples. In addition, there are still limitations in inducing the user to take active actions to improve cognitive ability.

According to various embodiments of the present disclosure for solving the conventional problems, the data receiving the phase or the movement of the object apparatus is specified and implemented in the virtual reality space. An object processing apparatus based on virtual reality and a method of operating the same may be provided.

According to various embodiments of the present disclosure, an object processing apparatus based on virtual reality that receives data measured with respect to an environment around an object apparatus and implements a virtual reality space, and a method of operating the same may be provided.

According to an embodiment of the present invention, a method of operating an object processing apparatus based on virtual reality, the method comprising: receiving first sensing data from an object apparatus connected through wireless communication; Implementing the environment of the object device and the object device as an object located in the virtual reality space and the virtual reality space based on the sensing data; Receiving last sensing data from the object apparatus, and detecting a change of the object apparatus from the last sensing data; And implementing a change in the object based on the detected change.

According to various embodiments of the present disclosure, the initial sensing data may include slope information of the object, direction information of the object, height information of the object, distance information with respect to other objects located around the object, location of the other object, and other information. It may include at least one piece of information about the direction in which the object is located.

The change may include at least one of acceleration information of the object, rotation information of the object, height change information of the object, and pressure information applied to the object.

The implementing of the change of the object based on the detected change may include at least one of a shape change of the object, a rotation of the object, and a height change of the object on the basis of the detected change. Can be implemented.

The method may further include implementing the environment of the object apparatus and the object apparatus based on the sensing data as an object located in the virtual reality space and the virtual reality space, between the object apparatus detected from the sensing data and another object. The other object and the phase of the object may be implemented in the virtual reality space based on a distance and the size of the virtual reality space.

On the other hand, according to an embodiment of the present invention, an object processing apparatus based on virtual reality, the communication unit for transmitting and receiving data in wireless communication with at least one object device; A display for outputting a virtual reality space; And receiving the first sensing data from the object device, embodying the environment of the object device and the object device as a virtual reality space and an object located in the virtual reality space based on the sensing data, and outputting the same through the display. And a controller configured to receive the last sensing data from the object device, detect a change in the object device from the last sensing data, and implement a change of the object based on the detected change and output the change to the display.

According to various embodiments of the present disclosure, the controller may include the inclination information of the object, the direction information of the object, the height information of the object, the distance information of another object located around the object, the location of the other object, and the other object. At least one piece of information on the located direction may be detected from the first sensing data.

The control unit may process the change by including at least one of acceleration information of the object, rotation information of the object, height change information of the object, and pressure information applied to the object.

The controller may implement a change of the object by implementing at least one of a shape change of the object, a rotation of the object, and a change of the height of the object based on the detected change.

The controller may implement the phases of the other object and the object in the virtual reality space based on the distance between the object apparatus and another object detected from the sensing data and the size of the virtual reality space.

According to various embodiments of the present disclosure, by creating an object in a visual virtual reality space through a processing device, and implementing a movement and a phase of the object device in a virtual reality space according to an operation of a user grasping and manipulating the object device, The user may be aware of his or her own movement, and may provide an opportunity for improving cognitive or behavioral ability.

According to various embodiments of the present disclosure, a virtual reality space may be generated through a processing device, and a phenomenon acting on an object device may be implemented as an object to which sensory effects are added. In addition to visual effects, auditory effects can also be provided, so the user can expect complex cognitive enhancement.

1 is a diagram illustrating a virtual reality provided through a system according to an embodiment of the present invention.
2 is a diagram of a virtual reality providing apparatus to which a processing apparatus or a processing apparatus is connected according to an embodiment of the present invention.
3 is a diagram illustrating components of a processing apparatus according to an embodiment of the present invention.
4 is a diagram illustrating components of an object apparatus according to an embodiment of the present invention.
5 is a flowchart of an operation of implementing and outputting a virtual reality space and an object in a processing apparatus according to an exemplary embodiment.
FIG. 6 is a flowchart illustrating an operation of requesting a change of an object implemented in a virtual reality space and processing a change in a processing apparatus according to an exemplary embodiment.
FIG. 7 is a diagram illustrating a virtual reality space and an object generated based on sensing data received from an object device in a processing device according to an exemplary embodiment.
FIG. 8 is a diagram illustrating an operation of deforming an object implemented in a virtual reality space based on sensing data received from an object device in a processing device according to an embodiment of the present disclosure.
FIG. 9 is a diagram illustrating an operation of deforming an object implemented in a virtual reality space based on sensing data received from an object device in a processing device according to an embodiment of the present disclosure.
10 is a diagram illustrating a specific configuration of an object apparatus in a system according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. At this time, it should be noted that the same components in the accompanying drawings are represented by the same reference numerals as possible. And a detailed description of known functions and configurations that can blur the gist of the present invention will be omitted.

The expression 'or' or 'at least one' used in various embodiments of the present disclosure may indicate one of the words listed together or a combination of two or more.

Terms used in various embodiments of the present disclosure are for explaining a specific embodiment and should not be construed as limiting the present disclosure. For example, the singular forms "a", "an" and "the" are intended to be used in the plural unless the context clearly indicates otherwise. It may include the meaning of.

According to various embodiments of the present disclosure, an object processing apparatus (hereinafter, referred to as a processing apparatus) based on virtual reality and a method of operating the same will be described. Here, the object processing apparatus based on the virtual reality is a device for receiving information detected through the object device in contact with the user's hand for sensory cognitive training, such as visual, tactile and / or auditory, and output to the virtual reality space It may be provided as.

In addition, according to various embodiments of the present disclosure, a system including an object device and a processing device for processing information received from the object device may be provided.

1 is a diagram illustrating a virtual reality provided through a system according to an embodiment of the present invention. Here, the system may include at least one electronic device. According to an embodiment of the present disclosure, the system may include at least one of an object device (hereinafter, referred to as an object 102 and a processing device 101 which receives an operation state of the object and outputs a visual interface to the user) that is grasped and manipulated by the user and detects a phase thereof. It includes one device.

Here, referring to FIG. 1, the object device 103 may be provided as a block having a cube shape. The object apparatus 103 includes at least one sensor therein to detect the phase of the object apparatus 103, for example, the position, movement or external force (or pressure) acting on the surface of the object apparatus 103. Can be. According to various embodiments of the present disclosure, the shape of the object device 103 is not limited to a hexahedron, and may be provided in a polyhedron shape such as tetrahedron, pentagon, triangular prism, square pillar, pentagonal pillar, triangular pyramid, square pyramid, pentagonal pyramid, and platonic solid. Or may be provided in a non-polyhedral shape, such as a sphere, torus, cylinder, cone, or the like.

The sensing data detected through the sensor of the object apparatus 103 is transmitted to the processing apparatus 101, and the processing apparatus 101 visually implements the external force acting on the object based on the received sensing data and outputs it to the display. can do. Here, the processing device 101 may be implemented as a device that provides a virtual reality to a user through a display.

According to an embodiment of the present disclosure, an operation of providing an image to a user through the display 150 by the processing apparatus 101 may be described. For example, the processor 101 may generally generate two images, which are the same or similar, to be displayed on a display to provide visual virtual reality. For example, the processing apparatus 101 may divide an image, which is generally displayed as one on the display 150, into a left image 111 and a right image 113, and display the image on the display 150. The user may use the left image 111 and the right image 113 displayed on the display 150 as a single image (eg, a virtual screen or a virtual reality space) through the structure, image processing, and / or optical processing of the processing apparatus 101. (151)).

In addition, according to an embodiment, the separated images (eg, the left image 111 and the right image 113) displayed on the display 150 by the processing apparatus 101 may include at least one of the images included in the processing apparatus 101. An image may be distorted by a lens (eg, 203 of FIG. 2). The processing apparatus 101 may inversely distort the plane image separated and displayed on the display 150 in consideration of the characteristics of the lens (eg, 203 of FIG. 2) in order to provide an undistorted image to the user. have.

In the following description, a screen configuration output from the processing apparatus 101 to the display 150 may be an image processed into a left image 111 and a right image 113 as described with reference to FIG. 1. In this case, the virtual reality space 151 shown in the space (for example, in front of the user's eyes or the field of view) is provided with a screen configuration (or image) displayed on the display 150 of the processing apparatus 101 to the user (or It can be explained by the image shown.

In addition, in displaying an object corresponding to the object apparatus 103, in order to describe a more suitable embodiment, the display may be divided into a display on the display 150 or a display in the virtual reality space 151. It means a difference between what is output through the unit and what is seen by the user's view, and should not be understood as using a different device or method.

As described above, the processing apparatus 101 generates and displays a left image and a right image in displaying an image on the display 150. However, the processing apparatus 101 generates a left image and a right image using a display method according to an exemplary embodiment. The present invention is not limited to the above, but may be applied to divided images of various methods such as a method of generating an upper image and a lower image. In addition, the processing apparatus 101 may modify an interface displayed on the display 150 based on various modes such as a stereoscopic image output function in modifying the interface of the display 150, or may process the processing apparatus 101. Based on the modified display interface, the user may provide an image of a stereoscopic screen or a stereoscopic image to the user through the processing apparatus 101.

2 is a diagram of a virtual reality providing apparatus to which a processing apparatus or a processing apparatus is connected according to an embodiment of the present invention. According to an embodiment, the processing apparatus 101 provides the user with a screen configuration displayed on the display 150, such as an image processing technique processed by the display 150 and / or an image such as optical processing of a physical lens. Based on the processing, it can be expressed as spread out before the user's eyes (or the field of view). For example, a technology for outputting an image through the processing apparatus 101 may use a technique that implements at least part of an augmented reality (AR) technology or a virtual reality (VR) technology.

Therefore, in order for the user to be provided with the screen configuration displayed on the display 150 through the processing apparatus 101, the processing apparatus 101 may be provided so as to be positioned on the head of the user, and according to various embodiments of the present disclosure. The device 101 may be described as a virtual reality providing device included in the same or similar category as a head mounted display (eg, a head mounted display (HMD)).

In providing the head mounted display type processing apparatus 101, there are various methods, but for the convenience of description, the structure having the shape as shown in FIG. 2 will be described as an embodiment. According to an embodiment, the processing apparatus 101 is implemented by being included in a specific mechanism (eg, a frame or a body, hereinafter referred to as a frame 201) that can be mounted on the head, and the display 150 is disposed in one direction of the frame 201. Can be. At least one lens 203 may be disposed on the display 150 to provide a displayed image as a virtual reality image to a user.

The frame 201 may include a control device for controlling the processing apparatus 101 on one surface, for example, on one surface perpendicular to the display 150. According to an embodiment of the present disclosure, the control device may include a key pad, a dome switch, a touch pad, a touch panel, a touch key, a joystick, and a button. (button), a wheel key (wheel key) may be composed of at least one. The touch pad may display a graphical user interface (GUI) for controlling a function of the processing apparatus 101 (for example, controlling a sound or an image).

In addition, the frame 201 of the processing apparatus 101 includes, for example, a control unit, a sensor unit (sensor module), an input unit (input device), a power supply unit (power management module), and a vibrator. It may be configured to include at least some of the adjustable optics.

According to various embodiments of the present disclosure, the processing device 101 may be provided as a separate device detachable / attached to the frame 201. In this case, the frame 201 of the virtual reality providing apparatus is connected to the processing apparatus 101, and the display 150 and the lens 203 for providing the virtual reality to the user based on data received from the processing apparatus 101. It may include at least one component.

When the processing apparatus 101 is provided to be detachable / attachable to the frame 201, the processing apparatus 101 is attached to one surface of the frame 201, mounted inside the cover 20 by using a cover formed on the frame 201, or of the frame 201. It may be provided to be inserted inside through one surface. In this case, the processing apparatus 101 may be connected to the frame 201 using a wired communication such as USB or a wireless communication such as WiFi, Bluetooth, and the like to transmit and receive data.

Hereinafter, the components of the processing apparatus according to an embodiment of the present invention will be described with reference to FIG. 3. As shown in FIG. 3, the processing apparatus 101 includes at least one component of the communication unit 301, the object apparatus 103, 303, the output unit 305, the memory 307, and the controller 309. do.

The communication unit 301 communicates with an external device such as the object device 103. To this end, the communication unit 301 may perform at least one of wired or wireless communication. The communication unit 301 may perform wireless communication such as wireless fidelity (WiFi), Bluetooth, Bluetooth low energy (BLE), and near field communication (NFC).

According to various embodiments of the present disclosure, when the processing apparatus 101 is provided to be detachable / attachable rather than integrally with the virtual reality providing apparatus, the communication unit 301 may perform wired or wireless communication with the virtual reality providing apparatus.

The input unit 303 generates input data in response to a user input to the processing apparatus 101. The input unit 303 includes at least one input means. The input unit 303 may be provided in the same or similar to the above-described control device.

The output unit 305 outputs the processing result of the control unit 309. The output unit 305 may include a device of at least one of a display and a speaker. When the output unit 305 is provided as a display, the output unit 305 outputs display data. For example, the output unit 305 may include a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, and the like. In addition, the output unit 305 may include a plurality of light emitting devices. The output unit 305 may be combined with the input unit 303 to be implemented as a touch screen. When the output unit 305 is provided as a speaker, the output unit 305 may output data received from the control unit 309 as audio.

Here, when the processing apparatus 101 is provided integrally with the virtual reality providing apparatus, the display may be provided to the display 150 of the virtual reality providing apparatus as described above. On the other hand, when the processing apparatus 101 is provided as a separate device from the virtual reality providing apparatus, the display may be formed in a part of the processing apparatus 101 separately from the display 150.

The memory 307 stores the sensing data of the object device 103 received through the communication unit 301. In addition, the memory 307 may store operation programs of the processing apparatus 101 or data processed / generated. For example, the memory 307 may store data related to the virtual reality space generated through the control unit 309 based on the received sensing data, and data related to the object created in the virtual reality space corresponding to the object device 103. Can be stored.

The memory 307 may store behavior request data (hereinafter, referred to as request data) for requesting a user to act through the object apparatus 103. Here, the memory 307 may store a graphic user interface generated to display the information related to the request data through the output unit 150.

The controller 309 receives the sensing data from the object device 103 connected through the communication unit 301. According to an embodiment of the present disclosure, the controller 309 may check information related to the space where the object device 103 is located and information related to the object device 103 from the sensing data.

Here, the information related to the space may be information related to the surrounding environment of the object device 103. For example, it may include at least some of information related to a floor area (or a terrain) on which the object apparatus 103 is placed, and other objects (or structures) placed around the object apparatus 103. According to an embodiment of the present disclosure, the other object may include at least one of an object or a device having various shapes that are distinguished from the object device 103. In addition, the information related to the other object may include at least one information of the position, direction, and distance of the other object.

In addition, the information related to the object apparatus 103 may include information related to the phase of the object apparatus 103 when the object apparatus 103 is in a stopped state. For example, the information may include at least one of altitude, tilt, and an external force acting on the surface of the object apparatus 103. If the object device 103 includes a satellite navigation system such as a global positioning system (GPS), it may include spatial coordinates.

The controller 309 may generate a virtual reality space and implement an object corresponding to the object device 103 based on the received information related to the object device 103. The controller 309 may output the generated virtual reality space and the implemented object through an output unit (hereinafter, the display 150).

According to various embodiments of the present disclosure, the controller 309 may be another object or object implemented in the virtual reality space according to the information related to the object apparatus 103 and other objects detected from the sensing data and the virtual reality space implemented based thereon. At least some of the size of or the distance between the object and another object may be determined.

Also, the information related to the object apparatus 103 may include information related to the movement of the object apparatus 103. For example, the information may include at least one of acceleration, rotation, height change of the object apparatus 103, and external force applied to the surface of the object apparatus 103. The controller 309 may implement the movement of an object located in the virtual reality space based on the information related to the movement of the object device 103. The controller 309 may output the movement of the object through the display 150.

The controller 309 may apply various three-dimensional graphics processing techniques in implementing the virtual reality space, the object, the phase of the object, and / or the movement of the object. According to an embodiment of the present disclosure, the controller 309 may be a vertex shader, a rasterizer, a pixel shader, or a Z-buffer for the virtual reality space, the object, and / or another object. 3D image processing based on at least a portion of a buffer) and a pantalla may be performed.

The controller 309 may receive the sensing data in response to the mission output on the display 150. Here, the mission may be an operation that is proposed to allow a user to implement the phase or movement of the object apparatus 103 through the object apparatus 103. The mission may be provided by outputting an image of an object corresponding to the phase or movement of the object apparatus 103 to the display 150.

In addition, the processing apparatus 101 may generate data (request data) corresponding to the sensing data in relation to the image of the provided object. Here, the request data may be provided as data in which a specified value is set for at least a portion of pressure applied to an angle (or tilt), a direction, and a height surface of the object. The controller 309 generates an image (image of the request data) for requesting that an object implemented in the virtual reality space to complete a specific position or haptic based on the requested request data and displays on the display 150. I can display it.

The controller 309 may implement a position or haptic of the object based on the sensing data received from the object apparatus 103 in response to the request data, and compare the object with the object of the request data. The controller 309 may determine that the mission is completed when the position of the implemented object or the image of the haptic and the request data is determined to be the same or similar.

Hereinafter, the components of the object apparatus according to an embodiment of the present invention will be described with reference to FIG. 4. As shown in FIG. 4, the object apparatus 103 includes at least one component of the communication unit 401, the sensor unit 403, the memory 405, and the control unit 407.

The communication unit 401 communicates with the communication unit 301 of the processing apparatus 101. To this end, the communication unit 401 may perform at least one of wired and wireless communication. According to an embodiment, the communication unit 401 may be configured to be the same as or similar to the communication unit of the processing apparatus 101. For example, the communication unit 301 may perform wireless communication such as wireless fidelity (WiFi), Bluetooth (Bluetooth), Bluetooth low energy (BLE), and near field communication (NFC).

The sensor unit 403 may include at least one sensor capable of measuring an internal / external situation of the object apparatus 103, and may measure the position and / or movement of the object apparatus 103. For example, the sensor unit 403 may include a GPS, a gyroscope, an acceleration sensor, a tilt sensor, an altitude sensor, a proximity sensor, an image sensor (or a camera), a pressure sensor (hereinafter, a pressure sensor), a magnetic sensor, an infrared sensor, It may include at least one sensor of the touch sensor.

The memory 405 may store the information measured by the sensor unit 403 and the sensing data generated based on the measured information. In addition, the feedback data received in response to the transmission of the sensing data may be stored in the memory 405. According to various embodiments of the present disclosure, the memory 405 may be configured to be the same as or similar to the memory 307 of the processing apparatus 101.

The controller 407 may acquire information about at least one of the position, coordinates, altitude, and external force acting on the surface of the object apparatus 103 through the sensor unit 403. In addition, the control unit 407, the acceleration of the movement of the object device 103 through the sensor unit 403, the direction of rotation, rotation, altitude change, coordinate change, of the external force acting on the surface of the object device 103 At least one piece of information can be obtained.

In addition, the controller 407 may acquire information related to other objects located around the object apparatus 103 through the sensor unit 403. According to an embodiment of the present disclosure, the controller 407 may acquire information related to a distance and / or a shape of a terrain or a feature located near the object apparatus 103 through the sensor unit 403.

For example, the controller 407 may include at least one other object positioned around the object apparatus 103 based on at least one of an infrared sensor, a proximity sensor, and an image sensor configured in the sensor unit 403. Related information can be measured.

The controller 407 is configured to hold the object device 103 using a user's hand, and respond to the movement, vibration, rotation, or change in force applied to the surface of the object device 103 according to the movement of the hand. The phase and / or the change of the object apparatus 103 can be measured through the sensor configured in the above.

For example, the control unit 407 measures at least one of a user's gripping, gripping position, and applied pressure through at least one of a pressure sensor and a touch sensor, which are formed inside or on the object apparatus 103. can do.

The controller 407 may generate sensing data for transmitting to the processing apparatus 101 by using the measured information (or measured value). The controller 407 may transmit the generated sensing data to the processing apparatus 101 through the communication unit 401.

In addition, the object apparatus 103 may further include a motor (or a vibration motor) for outputting various stimuli through vibration. The controller 407 may detect a phase change or movement through the sensor unit 403, or output a corresponding vibration through a motor when performing a designated command from the processing apparatus 101.

5 is a flowchart of an operation of implementing and outputting a virtual reality space and an object in a processing apparatus according to an exemplary embodiment.

In operation 501, the processing apparatus 101 receives sensing data (first sensing data) from the object apparatus 103. The received first sensing data may include location information of the object device 103. Here, the location information may include not only the location of the object device 103 but also information related to the surrounding environment of the object device 103, for example, information on other objects located around the object device 103. When the first sensing data includes information about another object, the distance and direction information with respect to the other object may be included.

According to an embodiment, the operation of receiving the first sensing data from the object apparatus 103 by the processing apparatus 101 may be an operation that is processed after the processing apparatus 101 and the object apparatus 103 are connected by communication. For example, the processing device 101 may receive the sensing data through the object device 103 after being connected to the object device 103 through Bluetooth communication. According to an embodiment of the present disclosure, the processing apparatus 101 may receive the first sensing data from the object apparatus 103 after transmitting a request for sensing data to the object apparatus 103.

In operation 503, the processing apparatus 101 generates a virtual reality space based on the received first sensing data. The processing apparatus 101 may detect information related to the surrounding environment of the object apparatus 103 and the phase of the object apparatus 103 from the first sensing data, and generate a virtual reality space based on the detected information. In addition, the processing apparatus 101 may generate an object located in the virtual reality space corresponding to the object apparatus 103.

In implementing the virtual reality space, the processing apparatus 101 may implement the terrain and the arranged feature of the space based on information related to the surrounding environment of the object device 103 included in the first sensing data. In this case, the processing apparatus 101 may implement another object detected in the vicinity of the object apparatus 103 in the virtual reality space. For example, the processor 101 may obtain information related to another object, a distance and a direction of the other object from the initial sensing data, and implement the structure in the virtual reality space based on the obtained information. In this case, the shape of the structure created in the virtual reality space may be implemented in the same shape as other corresponding objects, and may be changed according to a user's selection or a preset value.

The processing device 101 may be implemented as an object having the same shape or different shape as the object device 103 when the object device 103 is created in the virtual reality space. For example, when the object device 103 is provided in a hexahedral shape, an object implemented in the virtual reality space may be provided in a hexahedron or a tetrahedron, a pentagon, a triangular prism, a square prism, a pentagonal prism, a triangular pyramid, a square pyramid, a pentagonal pyramid It may be provided in the shape of any one of a polyhedron, such as the three-dimensional of the platonic. In addition, the processing apparatus 101 is not limited to a polyhedron in generating an object corresponding to the object apparatus 103, and has various shapes that can be held by a non-polyhedron or a user's hand such as a sphere, a torus, a cylinder, and a cone. Can be generated.

In operation 505, the processing apparatus 101 checks whether the sensing device (last sensing data) for the change of the object apparatus 103 is received from the object apparatus 103. According to an embodiment, the processing apparatus 101 may perform operation 507 when receiving the sensing data from the object apparatus 103 after the creation of the virtual reality space and the implementation of the first object, and may not receive the sensing data. In case 505, the operation may be repeated.

In operation 507, the processing apparatus 101 may detect information related to a change or changed phase of the object apparatus 103 from the received last sensing data. According to an embodiment of the present disclosure, the processing apparatus 101 may detect data related to movement, movement, and deformation (eg, an action of external force) of the virtual reality space and the object apparatus 103 received from the last sensing data.

According to various embodiments of the present disclosure, the processing device 101 may compare the information related to the object detected from previously received sensing data (eg, the first sensing data) with the information related to the object detected from the last sensing data, and compare the object device 103. ), You can see the change. In addition, the processing apparatus 101 may detect the change of the virtual reality space by comparing the previously received sensing data with the last sensing data.

In operation 509, the processing apparatus 101 may apply the confirmed change of the virtual reality space or the change information of the object apparatus 103 to output the change of the virtual reality space or the object through the implementation and display 150.

For example, the processing apparatus 101 may provide information related to the virtual reality space and the object detected from the last sensing data with respect to the virtual reality space and the object generated based on previously received sensing data (eg, the first sensing data). It can be applied to realize the change of virtual reality space or the change of object.

Here, the change of the virtual reality space or the object device 103 included in the last sensing data may be information measured for continuous operation. The processing apparatus 101 may implement and output a movement of a change in the virtual reality space or an object based on the received last sensing data.

As described above, the processing apparatus 101 generates and outputs the virtual reality space and the object based on the first sensing data received from the object apparatus 103, and subsequently the virtual reality space and the object based on the last sensing data received. You can implement and output changes in. The processor 101 may terminate the embodiment of FIG. 5 when performing operation 509, or may receive additional sensing data by performing operation 505. In this case, at least one sensing data received from the object device 103 may be included between the additionally received sensing data and the first sensing data.

FIG. 6 is a flowchart illustrating an operation of requesting a change of an object implemented in a virtual reality space and processing a change in a processing apparatus according to an exemplary embodiment. Operation 601 may be described as an operation processed after performing operation 503 of FIG. 5.

In operation 601, the processing apparatus 101 may request a user to manipulate the object apparatus 103 through the display 150. For example, the processing apparatus 101 may generate request data according to a user's input through the input unit 303 or a setting previously stored in the memory 307. The processing apparatus 101 may output the image related to the changed phase or haptic of the object previously implemented in the virtual reality space through the display 150 based on the generated request data.

Here, the operation of outputting the change of the object through the display 150 may be an operation of outputting the changed image of the object to a part of the virtual reality space implemented through the processing apparatus 101.

As described above, by outputting the changed image of the object to the virtual reality space, the user can grasp the object and induce the object device 103 to move in the same or similar to the changed image of the object output to the virtual reality space. Can be.

In operation 603, the processing apparatus 101 may determine whether to receive sensing data about a phase change and a movement of the object apparatus 103. The processing apparatus 101 may perform operation 605 when receiving the sensing data from the object apparatus 103 and repeatedly perform operation 603 when receiving the sensing data.

In operation 605, the processing apparatus 101 may implement a changed phase, movement, or haptic of the object based on the sensing data received from the object apparatus 103. For example, the processing apparatus 101 may detect information related to the phase and the movement of the object from the sensing data, and apply the detected information to the object implemented as the previous sensing data. According to an embodiment of the present disclosure, the processing apparatus 101 may implement at least some of the movement, rotation, height change, and external force applied to the surface based on the sensing data.

In operation 607, the processor 101 determines whether the change of the implemented object is the same as or similar to the movement (or position) of the previously requested object through the virtual reality space. For example, the processing apparatus 101 may include movement, rotation, height change, and surface movement of an object implemented based on sensing information and change information such as movement, rotation, height change, and external force applied to the surface. Change information such as external force applied can be compared to determine whether they are the same or similar.

The processor 101 outputs a result through operation 609 when determining that the change of the object is the same as or similar to the image of the object based on the request data, and performs operation 603 when determining that the change of the object is the same or similar. can do. The processor 101 may end the embodiment of FIG. 6 when it is determined that the change of the compared object is not the same or similar more than a specified number of times based on the received sensing data. In this case, the processing apparatus 101 may output a notification about the mission failure through the display 150.

In operation 609, the processing apparatus 101 may output a notification about the result. According to an embodiment of the present disclosure, when it is determined that the image is the same as or similar to the image of the requested object, the processing apparatus 101 may output a notification about the success of the mission through the display 150.

As described above, the change of the object described in FIG. 6 may mean a change result (for example, a phase) that is implemented by including at least a part of the tilt, direction, height, and external force applied to the surface of the object. However, without being limited thereto, it is obvious that the change of the object may be configured to include a change process such as movement, rotation, and height change.

FIG. 7 is a diagram illustrating a virtual reality space and an object generated based on sensing data received from an object device 103 in the processing device 101 according to an exemplary embodiment.

As described above, the processing apparatus 101 may generate a virtual reality space based on the sensing data received from the object apparatus 103 and implement an object corresponding to the object apparatus 103 in the implemented virtual reality space. have.

Referring to FIG. 7, the object device 103 may acquire information related to a surrounding environment through a sensor included in the object device 103. In this case, the object device 103 may check not only the terrain around the object device 103 but also a feature (eg, another object 701), and may measure information about the other object 701. For example, the object device 103 may measure information related to a distance and a direction from another object 701. The object apparatus 103 may generate sensing data using the measured information and transmit the generated sensing data to the processing apparatus 101.

The processing apparatus 101 may generate the virtual reality space 151 based on the received sensing data (first sensing data) and implement the object 713 corresponding to the object apparatus 103 in the generated virtual reality space. . The processing apparatus 101 may check information related to the other object 701, for example, distance information and / or direction information between the object apparatus 103 and the other object 701 from the received sensing data. The processing apparatus 101 may implement the other object 711 and the object 713 in the virtual reality space 151 based on the information related to the other object 701 and the generated virtual reality space 151.

FIG. 8 is a diagram illustrating an operation of deforming an object implemented in a virtual reality space based on sensing data received from the object apparatuses 103 and 803 in the processing apparatus 101 according to an exemplary embodiment. Referring to FIG. 8, the processing apparatus 101 may implement changes applied to the object apparatuses 103 and 803 based on the sensing data received.

According to an embodiment of the present disclosure, the processing apparatus 101 may detect a change applied to the object apparatuses 103 and 803 based on the sensing data received after the initial sensing data. For example, the processing apparatus 101 changes the object apparatuses 103 and 803 such as movement, rotation and height change of the object apparatuses 103 and 803 from the sensing data and external force applied to the surface of the object apparatus 103. Information can be detected.

The processing device 101 may implement the object 813 by applying change information of the object device 803 to an object implemented based on previously received sensing data. The processing apparatus 101 may visually implement 821 the external force applied to the surface of the object 813 while the object apparatus 803 is held by the user.

For example, the processing apparatus 101 may detect at least some information among the pressure, the position of the pressure, and the magnitude of the pressure applied to the surface of the object apparatus 803 according to the gripping operation of the user from the received sensing data. According to the detected information, the effect may be visually implemented in the representation of the object 813 created in the virtual reality space 151.

In addition, when the positions of the object apparatuses 103 and 803 are changed, the processing apparatus 101 may be configured to execute another object implemented in the virtual reality space 151 based on information related to another object 801 detected from the sensing data. The arrangement with 811 can be reset.

FIG. 9 is a diagram illustrating an operation of deforming an object implemented in a virtual reality space based on sensing data received from the object devices 103 and 803 in the processing apparatus 101 according to an exemplary embodiment.

Referring to FIG. 9, the processing apparatus 101 may output an image 910 of an object generated based on request data to the virtual reality space 151.

The processing device 101 may display a specific phase of the object through the display 150 and request a user to manipulate the object device 103 as the displayed object. For example, the processing apparatus 101 may generate request data related to a specific phase of the object according to a user's input or a setting previously stored in the memory 307. The processing apparatus 101 may implement the phase of the object based on the request data and output the image 910 of the request data through the display 150.

The processing apparatus 101 may receive sensing data about a change of the object apparatus 103 from the object apparatus 103 and implement movement of an object implemented in the virtual reality space 151 based on the received sensing data. . The processor 101 may compare the phase of the object 913 implemented based on the sensing data with the phase of the object implemented in the image 910 of the request data.

The processing apparatus 101 may output a notification of the success of the mission when the comparison result determines that the phase of the object 913 is the same as or similar to the phase of the object implemented in the image 910 of the request data. For example, the processing apparatus 101 may apply the designated graphic effect to the object 913 and output the notification.

10 is a diagram illustrating a specific configuration of an object apparatus in a system according to an embodiment of the present invention.

According to various embodiments of the present disclosure, the object device 103 is not limited to being provided as a block having a figure shape as described above, and a product having various shapes may be applied. Referring to FIG. 10, the object apparatus 103 may be provided to the controller 1001 that controls the virtual reality space of the processing apparatus 101 or the function of the processing apparatus 101. According to an embodiment of the present disclosure, the controller 1001 may include at least one button such as a remote controller, a joystick, and a haptic grip.

The processor 1001 may implement the shape of the controller 1001 in realizing an object corresponding to the controller 1001 in the virtual reality space, but is not limited thereto. The shape of various other objects, for example, a polyhedron, It can be implemented in a block shape such as non-polyhedra.

The controller 1001 may include a communication unit and transmit / receive data with the processing apparatus 101 through wireless communication. In addition, the controller 1001 may include a component that is the same as or similar to at least one of the components of the object apparatus 103.

As described above, the system creates an object corresponding to the object apparatus 103 in the visual virtual reality space through the processing apparatus 101, and the object apparatus according to the operation of holding and manipulating the object apparatus 103 by the user. The movement and phase of the 103 may be implemented in the virtual reality space. Through this, the system can provide a method and apparatus for training cognitive ability or behavioral ability by recognizing a user's own movement.

According to various embodiments of the present disclosure, the processing apparatus 101 receives sensing data measured by the object apparatus 103 with respect to phase and motion information about a user's gripping and manipulation of the object apparatus 103, and transmits the sensing data to the virtual reality space. In the implementation of the object, the object may be implemented by adding a visual effect or an auditory effect, or the object may be modified.

As described above, the system generates the virtual reality space through the processing apparatus 101 and implements the phenomenon acting on the object apparatus 103 as an object with the added sensory effect to compare the effect with the cognitive ability training in reality. In addition to the visual effect, it can provide not only a visual effect but also an auditory effect, so that the user can expect a complex improvement of cognitive ability.

On the other hand, the embodiments of the present invention disclosed in the specification and drawings are merely presented specific examples to easily explain the technical contents of the present invention and help the understanding of the present invention, and are not intended to limit the scope of the present invention. That is, it will be apparent to those skilled in the art that other modifications based on the technical idea of the present invention can be implemented.

101: processing device 103: object device
150: display 151: virtual reality space
301: communication unit 303: input unit
305: output unit 307: memory
109 control unit 401 communication unit
403: sensor portion 405: memory
407 control unit 1001 controller

Claims (10)

Receiving initial sensing data of the shape of the object apparatus from the object apparatus connected by wireless communication;
Implementing the object device as an object located in a virtual reality space based on the first sensing data;
Outputting a deformed image of an object created based on request data in the virtual reality space;
Receiving a final sensing data, which is a result of detecting a grip position and a gripping pressure, from the object apparatus while the user grips the object apparatus, and detecting a change in appearance of the object apparatus from the last sensing data; And
Implementing the appearance change of the object based on the detected change, and if the appearance change of the implemented object coincides with the modified image of the object, outputting a notification of the success of the object to help improve the user's cognitive ability Method of operating an object processing apparatus based on virtual reality, including. The method of claim 1,
The first sensing data may include slope information of the object, direction information of the object, height information of the object, distance information with respect to other objects located around the object, location of the other object, and direction information where the other object is located. Method of operating an object processing apparatus based on virtual reality, including at least one information. The method of claim 1,
The change may include at least one of acceleration information of the object, rotation information of the object, height change information of the object, and pressure information applied to the object. . The method of claim 1,
Implementing the change of the object based on the detected change, to implement at least one of the shape change of the object, the rotation of the object, the height change of the object based on the detected change in the virtual reality space , A method of operating an object processing apparatus based on virtual reality. The method of claim 1,
Implementing the environment of the object device and the object device based on the sensing data as a virtual reality space and an object located in the virtual reality space,
The object processing apparatus based on virtual reality, which implements the phase of the other object and the object in the virtual reality space based on a distance between the object apparatus and another object detected from the sensing data and the size of the virtual reality space. Method of operation. Communication unit for transmitting and receiving data in at least one object device and the wireless communication;
A display for outputting a virtual reality space; And
Receive initial sensing data of the shape of the object device, and implement the object device as an object located in the virtual reality space and the virtual reality space based on the sensing data to output through the display and to request data Display the transformed image of the object in the virtual reality space,
When the user grips the object device, the sensor receives the last sensing data of the grip position and the gripping pressure, detects a shape change of the object device from the last sensing data, and detects the shape change of the object based on the detected change. And output to the display,
And a control unit for outputting a notification of a successful mission when the changed shape of the object coincides with the deformed image of the object for the request data, thereby improving the cognitive ability of the user. . The method of claim 6,
The control unit may include at least one of inclination information of the object, direction information of the object, height information of the object, distance information with respect to another object located around the object, location of the other object, and direction information where the other object is located. The object processing apparatus based on virtual reality, detecting information of the first sensing data. The method of claim 6,
The controller may include at least one of acceleration information of the object, rotation information of the object, height change information of the object, and pressure information applied to the object in the change, and process the object based on virtual reality. Processing unit. The method of claim 6,
The controller is configured to implement the change of the object by implementing at least one of a shape change of the object, a rotation of the object, and a change of the height of the object based on the detected change in the virtual reality space. Object processing unit. The method of claim 6,
The controller is configured to implement the phases of the other object and the object in the virtual reality space based on a distance between the object apparatus and another object detected from the sensing data and the size of the virtual reality space. Object processing unit.

Images