KR101723828B1 - Interactive Augmented space available space expansion engines experience applied Cube system - Google Patents

Abstract

The present invention relates to an experience cube system applying spatial augmented interactive engine capable of expanding space, and more particularly, to an experience cube system applying spatial augmented interactive engine capable of expanding space, which can induce pleasure through a realistic/sensory type exhibition image applying interaction technology which enables a dynamic object (user) with an object of a virtual world in real time, and can actively perform various experiences. The system according to the present invention includes: a first cube which includes a kinect camera which recognizes and supplies a gesture and a posture, a spatial augmented reality implementing device which obtains information from the kinect camera and performs interaction, a DID device which displays interaction image information with a 3D virtual object, and a speaker which outputs interaction sound information with the 3D virtual object; and a second cube for expanding the space. The spatial augmented interactive engine includes an image information obtaining unit which obtains color image and depth image information, an image matching unit, a dynamic object recognizing unit for storing dynamic object information, a dynamic object coordinate extracting unit for extracting position information about the dynamic object, a dynamic object mesh output unit for generating a real-time lattice mesh, an augmented object generating unit for augmenting a virtual object, an augmented object event generating unit for generating an event in the augmented object, and an object animation generating unit for executing an animation corresponding to the augmented object.

Description

[0001] The present invention relates to an interactive augmented cube system,

The present invention relates to an experiential cube system to which a space enhancing interactive engine capable of expanding a space is applied, and more particularly, to an experiential cube system in which a dynamic object (user) And an experience cube system to which a space enhancing interactive engine capable of expanding a space capable of actively experiencing various experiences through pleasure through exhibition images.

VIRTUAL REALITY refers to a specific environment or situation, or technology itself, which is similar to the reality created by artificial technology using computers and the like, but these virtual environments or situations stimulate the user's five senses, , Allowing them to experience the time and the freedom to move freely between reality and imagination.

Recently, domestic and foreign companies are investing heavily in creating virtual reality devices and developing content based on virtual reality. In light of this situation, we can see that virtual reality will become the mainstream of the video industry in the near future have.

However, most of the contents development industry based on virtual reality recently focuses on the development of interesting virtual reality contents such as game entertainment, and development of educational contents such as natural learning is not enough.

In addition, existing virtual reality-based contents are mainly composed of passive contents in the form of display, and virtual experience contents through users and feedback are extremely limited.

In summary, existing experiential space is mainly composed of simple dynamic image display or indirect experience through simple interaction with touch screen.

Accordingly, the present invention can recognize spaces and objects in a real-world environment, project spaces and objects on screens, and enhance (mix) virtual contents and information so that dynamic objects (users, viewers, and visitors) can interact with virtual objects This will provide an environment in which visitors can experience the exhibition in a realistic way.

In addition, the interactive exhibition space of the complex culture space (science museum / exhibition hall / library / experience hall / theme park) to which the present invention is applied provides a space enhancement interactive service that supports real-time interaction by enhancing the feeling of immersion and realism of visitors and visitors.

Korean Patent No. 10-0812624

SUMMARY OF THE INVENTION Accordingly, the present invention has been made keeping in mind the above problems occurring in the prior art,

An object of the present invention is to provide a real-time / real-time interactive display technique in which a dynamic object (user) can interact with an object in a virtual world in real time, To be able to do.

Another object of the present invention is to provide an experience cube system that allows a second cube to be located inside a first cube and to escape to the outside when necessary, To be able to carry out experience exhibition.

According to an aspect of the present invention, there is provided an experience cube system to which a space enhancing interactive engine capable of extending a space according to an embodiment of the present invention is applied,

A Kinect camera 100 for recognizing the actual space in the experience cubes and recognizing the movements (gestures and postures) of the visitors and the visitors, and providing them as the space enhancing interactive engine configured in the experience cubes;

Acquires information (real space recognition information, experiential recognition information, experiential motion recognition information) provided from the Kinect camera and provides it to the built-in space enhancement interactive engine 250 to perform interaction between the generated three-dimensional virtual object and the viewer A space enhancer 200 for implementing the space reinforcement;

A DID device 300 for displaying the interaction image information with the 3D virtual object according to the movement of the guest provided through the space enhancement device;

And a speaker (400) for outputting interaction sound information with a 3D virtual object according to a movement of a person provided through the space enhancement implementing device. The first cube (1000)

And a second cube 2000 for moving the interior of the first cube to the outside to expand the space.

The space enhancing interactive engine 250,

An image information acquiring unit 251 for acquiring color image and depth image information from the Kinect camera 100;

The color image and the depth image obtained by the image information obtaining unit are acquired to correct the color image and the depth image, and the homography of the RGB image and the depth image is calculated to obtain the RGB camera An image matching unit 252 for processing the viewpoint of the Depth camera equally;

The matched image is acquired and a straight line is extracted. The spatial information in the matching image is recognized through the extracted straight line to distinguish the spatial, static, and dynamic regions. After extracting the feature points from the respective regions, A dynamic object recognizing unit 253 for recognizing an object represented by the region using the positional information of the region and storing the dynamic object information;

A dynamic object coordinate extraction unit 254 for extracting position information on the dynamic object using the skeleton information extracted from the Kinect camera using the depth image of the recognized dynamic object area;

A dynamic object mesh output unit (255) for acquiring position information on the dynamic object to extract skeleton information of the continuously recognized object region to generate a real time mesh mesh by estimating posture transformation;

An augmented object generating unit 256 for extracting the coordinates of the recognition space and separating the inside and the outside of the surface based on the extracted coordinates to form a surface and augmenting a virtual object on the surface formed;

A collision object is generated in the reinforcement object, a collision object is created in the dynamic object, and it is determined whether the collision object generated in the augmented object and the collision object generated in the mesh object collide with each other. An augmented object event generation unit 257 for generating augmented object event;

And an object animation generation unit 258 for acquiring an event of the augmented object and executing an animation corresponding to the augmented object.

The experiential cube system to which the space enhancing interactive engine capable of expanding the space according to the present invention is applied is a pleasure cube system in which a dynamic object (user) realizes an experience / And to provide various effects to actively perform various experiences to a plurality of visitors at the same time.

In addition, through the exhibition experience using the system of the present invention, it is possible to expect a new exhibition experience paradigm generation.

In addition, the second cube is located inside the first cube, and when necessary, the first cube is taken out of the first cube, thereby overcoming the spatial limit of the experience exhibition and providing the experience cube system to visit, It is possible to carry out the experience exhibition.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an overall configuration diagram of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied, before expansion of a second cube. FIG.
FIG. 2 is an overall configuration diagram of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied, after expansion of a second cube. FIG.
FIG. 3 is a space enhancement interactive engine block diagram of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied.
4 is a block diagram of an image matching unit of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied.
5 is a block diagram of a dynamic object recognition unit of an experience cube system to which a space enhancement interactive engine capable of expanding a space according to an embodiment of the present invention is applied.
6 is a block diagram of a dynamic object coordinate extraction unit of an experiential cube system to which a space enhancement interactive engine capable of expanding a space according to an embodiment of the present invention is applied.
FIG. 7 is a block diagram of a dynamic object mesh output unit of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied.
FIG. 8 is a block diagram of an augmented object generation unit of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied.
FIG. 9 is a block diagram of an augmented object event generating unit of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied.
10 is a block diagram of an object animation generating unit of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied.
11 is a diagram illustrating an interaction screen implemented by an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied.
12 is a skeleton screen exemplary view of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied.
FIG. 13 is a view illustrating a real-time grid mesh screen of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied.
FIG. 14 is a diagram illustrating a screen for forming a surface that distinguishes the inside and the outside of the surface of the experiential cube system to which the space enhancing interactive engine capable of expanding the space according to the embodiment of the present invention is applied.
15 is a view illustrating a real object and a virtual object screen of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied.
FIG. 16 is a diagram illustrating an example of a screen in which a virtual mesh of the experiential cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied, and is transparently displayed on a screen.
FIG. 17 is a diagram illustrating a screen image of a real object and a virtual object in an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied.

Hereinafter, an experience cube system to which a space enhancing interactive engine capable of expanding a space according to the present invention is applied will be described in detail with reference to embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an overall configuration diagram of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied, before expansion of a second cube. FIG.

As shown in Fig. 1, the system of the present invention mainly includes a Kinect camera 100; A space enhancement implementing device 200; A DID device 300; A speaker 400 and a second cube 2000 which is moved from the inside of the first cube to the outside to expand the space.

The existing experience time and space was an environment where visitors had to visit a specific place (science museum, museum, art museum, etc.) to experience it.

Accordingly, the experience cube system to which the space enhancement interactive engine of the present invention is applied is to enable the experience of space enhancing interactive contents in which the visitors can interact with virtual objects mixed with real space in various places by securing mobility to be.

Through the above-mentioned structure, the experiential cube system, which is a mobility-secured experience space, overcomes the spatial limitation of the experience exhibition, and through the experiential cube system, a variety of many visitors can experience the experience display.

In addition, a variety of space-enhancing interactive contents will allow a large number of visitors to experience at the same time, and experience the latest experience exhibited by ICT technology.

In the following, the construction means will be described in more detail.

The Kinect camera 100 recognizes the actual space in the experience cubes and recognizes the motions (gestures and postures) of the visitors and the visitors and provides them to the space enhancing interactive engine configured in the experience cubes. .

At this time, the space enhancement device 200 acquires the information (real space recognition information, experience person recognition information, and guest experience motion recognition information) provided from the Kinect camera and provides it to the built-in space enhancement interactive engine 250, Dimensional virtual object and the viewer interact with each other.

At this time, the DID device 300 formed on one side wall of the first cube displays the interaction image information with the 3D virtual object according to the movement of the guest provided through the spatial augmentation realization device.

At this time, in order to provide a stereo sound effect, a plurality of speakers 400 for outputting the interaction sound information with the 3D virtual object according to the movement of the guest provided through the space enhancement implementing device are installed at predetermined intervals of the first cube .

Meanwhile, the second cube 2000 is moved from the inside of the first cube to the outside to expand the space. To this end, the size of the second cube is made smaller than the size of the first cube, Should be inserted.

The second cube 2000 includes:

The size of the first cube is smaller than the size of the first cube so as to be stored in the first cube,

A door 2100 formed on one side so that visitors can enter and exit,

An expanding cube handle 2200 formed on one side so as to be pulled out from the first cube,

And a plurality of wheels 2300 spaced a predetermined distance apart from the first cube so as to be easily removed from the first cube when being pulled out through the expanding cube handle.

That is, as shown in FIG. 2, the extension cube handle and the wheel are formed so as to escape from the first cube to the outside.

On the other hand, the door is formed on one side so that the user can freely move, and the expansion cube handle 2200 can be pulled out from the first cube to any one side.

In the example of the present invention, although two knobs are provided, the knobs may be added or may be formed as needed.

Meanwhile, in order to facilitate movement, a plurality of wheel portions 2300 are formed on the lower surface of the first cube so as to be spaced apart from each other by a predetermined distance, and can be easily removed from the first cube when pulled out through the extension cube handle portion will be.

FIG. 3 is a space enhancement interactive engine block diagram of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied.

As shown in FIG. 3, the space enhancing interactive engine 250 of the present invention mainly includes an image information acquiring section 251; An image matching unit 252; A dynamic object recognition unit 253; A dynamic object coordinate extraction unit 254; A dynamic object mesh output unit 255; An augmented object generation unit 256; An augmented object event generation unit 257; And an object animation generation unit 258.

The image information obtaining unit 251 receives the color image and the depth image from the Kinect camera.

The image matching unit 252 acquires the color image and the depth image information obtained by the image information obtaining unit to correct the color image and the depth image, calculates the homography of the RGB image and the depth image, And performs the same processing of the viewpoints of the RGB camera and the depth camera that constitute the camera.

In addition, the dynamic object recognition unit 253 acquires an image matched by the image matching unit, extracts a straight line, and recognizes spatial information in the matching image through the extracted straight line to distinguish the spatial, static, and dynamic regions .

In this case, the feature points are extracted from the respective regions, and the dynamic object information is stored by recognizing the objects represented by the regions using the feature points of the regions and the location information in the space.

The information is stored in, for example, a memory or a database, and the memory is generally constituent means, and a detailed description thereof will be omitted.

In addition, the dynamic object coordinate extraction unit 254 extracts the position information of the dynamic object using the skeleton information extracted from the Kinect camera using the depth image of the recognized dynamic object region.

At this time, the dynamic object mesh output unit 255 obtains the position information of the dynamic object, extracts the skeleton information of the continuously recognized object region, and estimates the attitude conversion to generate a real time lattice mesh.

At this time, the augmented object generating unit 256 extracts coordinates for the recognition space, forms a surface by dividing the inside and the outside of the surface based on the extracted coordinates, and augments a virtual object on the formed surface .

At this time, the augmented object event generation unit 257 generates a collision object in the augmented object, generates a collision object in the dynamic object, and determines whether the collision object generated in the augmented object collides with the collision object generated in the mesh object When the collision occurs, an event is generated in the augmented object, and the object animation generation unit 258 acquires the event of the augmented object to execute animation corresponding to the augmented object.

That is, as shown in FIG. 11, a space and an object are recognized in the real world, and a virtual object (for example, a penguin and a water) is augmented to show a combination of real and virtual, It is possible to provide an effect of being moved or modified by interaction.

4 is a block diagram of an image matching unit of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied.

As shown in Fig. 4, the image matching unit 252,

A matching determination module 252a for determining whether the color image and the depth image match each other,

A depth / color image matching module 252b for matching the color image and the depth image if they are not matched,

A resolution correction module 252c for removing unnecessary depth coordinate information in case of matching and correcting the resolution of the color image and the depth image,

And a composite image generation module 252d for generating a composite image by mapping the corrected color image and the image of the depth image.

The matching determination module 252a acquires the image acquired by the image information acquisition unit and determines whether the color image matches the depth image. The depth / color image matching module 252b determines whether the color image matches the depth image The color image and the depth image are matched.

For example, when a difference in processing speed occurs between a color image and a depth image, a slow input image is supplemented to match the reference image.

In the case of a Kinect camera, color images can be input slowly.

On the other hand, the resolution correction module 252c removes unnecessary depth coordinate information in case of matching, and corrects the resolution of the color image and the depth image as a result of the determination.

For example, unnecessary information included in the input depth image can be removed using an algorithm such as a bilateral filter.

Meanwhile, the composite image generation module 252d generates a composite image by mapping the corrected color image and the image of the depth image.

For example, if the resolution of the color image and the depth image are the same, the image of the color image and the depth image can be mapped using the homography between the camera parameters.

5 is a block diagram of a dynamic object recognition unit of an experience cube system to which a space enhancement interactive engine capable of expanding a space according to an embodiment of the present invention is applied.

As shown in FIG. 5, the dynamic object recognizing unit 253,

A straight line extraction module 253a for extracting a straight line from the generated composite image,

A spatial recognition module 253b for recognizing spatial information in the composite image using the extracted straight line,

An object-specific region separation module 253c for separating regions of a spatial object, a static object, and a dynamic object in the composite image,

A feature point extraction module 253d for extracting feature points from the separated regions of the objects,

An object-by-object recognition module 253e for recognizing an object represented by the region using the feature points of the object region and the position information in the space,

And a dynamic object information storage module 253f for storing dynamic object information from the recognized object.

The straight line extraction module 253a extracts a straight line from the combined image of the color image and the depth image.

For example, a straight line can be extracted using a method such as Canny edge detector in a composite image of a color image and a depth image.

The spatial recognition module 253b recognizes spatial information in a synthesized (matched) image using the extracted straight line.

For example, the extracted straight line is used to find the vanishing point in the matching image, and the vector information of the planes is integrated to recognize the information of the space such as the floor, the wall, and the ceiling scene.

In addition, the object-based region separation module 253c performs a function of separating a spatial object, a static object, and a dynamic object region from the composite image.

For example, the object region can be separated by using the difference in color and depth values of each image of the matched (combined) image. At this time, each divided region is represented by a rectangular parallelepiped parallel to the floor surface and the wall surface, Position and rotation values can be calculated.

On the other hand, the feature point extraction module 253d extracts feature points from the separated regions of the object, and the per-object recognition module 253e uses the feature points of the respective object regions and the position information in the space, And recognizes the object represented by the object.

For example, an object can be recognized using a multiple classification algorithm.

The dynamic object information storage module 253f stores dynamic object information from the recognized object into a memory or a database.

For example, if the recognized object is a newly recognized object, it is possible to store the object as a newly recognized object in the memory or the database for three-dimensional information extraction of the object tracking and the real world.

6 is a block diagram of a dynamic object coordinate extraction unit of an experiential cube system to which a space enhancement interactive engine capable of expanding a space according to an embodiment of the present invention is applied.

As shown in FIG. 6, the dynamic object coordinate extraction unit 254 extracts,

An object location information extraction module 254a for extracting location information on a dynamic object using skeleton information extracted from a Kinect camera using the depth image of the recognized dynamic object area,

A position change tracking module 254b for tracking the position change using the extracted position information,

And a transformed object reconstruction module 254c for reconstructing a dynamic object for an object having a change in position using the extracted object position information.

The object position information extraction module 254a extracts the position information of the dynamic object using the skeleton information extracted from the Kinect camera using the depth image of the recognized dynamic object region.

At this time, the position change tracking module 254b tracks the position change using the extracted position information. When there is no position change, the position change tracking module 254b outputs the coordinates of the dynamic object.

Meanwhile, the transformed object reconstruction module 254c reconstructs a dynamic object for an object having a change in position using the extracted object position information when there is a position transformation.

FIG. 7 is a block diagram of a dynamic object mesh output unit of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied.

7, the dynamic object mesh output unit 255 outputs,

A skeleton information extraction module 255a for extracting skeleton information of an object region continuously recognized for comparison with input three-dimensional object coordinates,

An attitude conversion estimation module 255b for estimating an attitude conversion using the input three-dimensional object coordinates and the extracted skeleton information,

And a real-time grid mesh generation module 255c for generating a real-time grid mesh using the information on the continuously input images.

The skeleton information extraction module 255a extracts skeleton information of an object region continuously recognized for comparison with the inputted three-dimensional object coordinates, and the posture conversion estimation module 255b extracts three- The posture conversion is estimated using the skeleton information.

Also, the real-time grid-mesh generation module 255c generates the real-time grid mesh using the information on the continuously input images.

12, in an image input through a Kinect camera, for example, a Kinect camera, the experimenter becomes a dynamic object to be photographed by the camera, and information for tracking the movement of the experimenter Is skeleton (10, skeleton) information.

Here, the skeleton recognizes the human skeleton joint by dividing it into a total of 25 parts, which are grouped as one set, and divided into three axes (x, y, z) .

At this time, extracting the skeleton information means constructing a structure for fetching skeleton information obtained through the Kinect V2 camera and inputting skeleton information to the generated structure.

13, the lattice 20 refers to a set of repeated figures in the shape of a triangle, and the mesh 30 is formed by connecting the triangles generated by the lattice to each other It means to form a structure like a net.

This means that the mesh creates a shape having a mesh-like surface composed of lattices gathered like a dolphin image of the right-hand example.

FIG. 8 is a block diagram of an augmented object generation unit of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied.

8, the augmented-object generating unit 256 generates the augmented-

A recognition space coordinate extraction module 256a for extracting coordinates of the recognition space,

A spatial surface classifying module 256b for forming a surface by dividing the inside and the outside of the surface based on the extracted coordinates,

A virtual object enhancement module 256c for augmenting a virtual object on the formed surface,

And a transparent rendering module 256d for performing transparent rendering so that the mesh information of the recognized space is not visible to the user.

The recognition space coordinate extraction module 256a extracts coordinates for the recognition space, and the spatial surface classification module 256b forms the surface by dividing the inside and the outside of the surface based on the extracted coordinates.

On the other hand, as shown in FIG. 14, forming the surface by dividing the inside and the outside of the surface based on the extracted coordinate reference generates the surface of the object by using a triangle, and divides the inside and outside according to the direction of drawing the triangle .

For example, if you draw in the clockwise direction, it will be the back (inside of the surface), and if you draw it in the counterclockwise direction, it will be the front (outside of the surface).

At this time, the virtual object enhancement module 256c augments the virtual object on the formed surface.

For example, a virtual object may be positioned above the table rather than below it, or a virtual object ball may be rolled or moved along the surface.

Specifically, as shown in FIG. 15, a table as a real object 40 is recognized and a virtual mesh for interaction with the virtual object 50 is generated.

Then, a virtual object is created to recognize a virtually created mesh on a real object so that it can be displayed on the real object instead of passing through the real object.

The transparent rendering module 256d performs transparent rendering so that the mesh information of the recognized space is not visible to the user.

For example, you can create a virtual mesh surface on a recognized table to place a virtual object on the table, and make the virtual mesh surface transparent so that the actual table is visible.

Specifically, as shown in FIG. 16, a table as a real object 40 is recognized, and a virtual mesh for interaction with the virtual object 50 is generated, and then processed transparently so as not to be seen on the screen.

FIG. 9 is a block diagram of an augmented object event generating unit of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied.

9, the augmented-object-event generating unit 257 generates the augmented-

A first collision object creation module 257a for creating a collision object in the augmented object,

A second collision object creation module 257b for creating a collision object in the dynamic object,

And an event generation module 257c for determining whether the collision object generated in the augmented object and the collision object generated in the dynamic object collide with each other and generating an event in the augmented object in the event of collision.

That is, when the three-dimensional model of the augmented object and the mesh object is acquired, the collision object is generated in the augmented object by the first collision object generation module 257a, and the collision object is generated in the augmented object by the second collision object generation module 257b. (The mesh object).

That is, the collision object is created in the augmented object for interaction with the dynamic mesh object, and the collision object is created in the dynamic object for interaction with the augmented object.

In addition, the event generation module 257c determines whether the collision object generated in the augmented object and the collision object generated in the dynamic object (mesh object) collide with each other, and generates an event in the augmented object in the event of collision.

Specifically, as shown in Fig. 17, when a collision occurs between a real object and a virtual object, such as an operation through the pre-interaction image 60, the interaction image 61, and the post-interaction image 62 An event is generated when the user touches the terminal.

FIG. 10 is a block diagram of an object animation generating unit of an experience cube system to which a space enhancing interactive engine capable of expanding a space according to an embodiment of the present invention is applied.

As shown in FIG. 10, the object animation generation unit 258,

An animation generation module 258a for acquiring an event of the augmented object to generate an animation corresponding to the augmented object,

A collision event checking module 258b for checking the collision event between the virtual object of attention and the colliding object in the recognition space before the generated animation is operated,

An animation allocation module 258c to which the generated animation is allocated and connected according to a collision situation,

And an animation execution module 258d for executing the linked animation to allow a virtual object to move.

Specifically, the animation corresponding to the augmented object is generated by acquiring the event of the augmented object through the animation generation module 258a.

In addition, the collision event checking module 258b checks the collision event between the virtual object of attention and the colliding object of the recognition space before the generated animation operates.

The animation allocation module 258c allocates the animation generated according to the collision situation and connects the animation.

Thereafter, the linked animation is executed through the animation execution module 258d so that a virtual object can be moved.

That is, you can execute linked animations so that virtual objects move with vitality.

Through the above-described configuration and operation, it is possible to create pleasure through a sensory / haptic exhibition image to which a dynamic object (user) can interact with an object of a virtual world in real time by applying an interaction technique, and actively provide various experiences And the like.

It will be understood by those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. It is to be understood, therefore, that the embodiments described above are to be considered in all respects as illustrative and not restrictive.

100: Kinect camera
200: space enhancement implement
400: Speaker
1000: 1st cube
2000: second cube

Claims (3)

delete 1. An experiential cube system to which a space enhancing interactive engine capable of expanding a space is applied,
A Kinect camera 100 for recognizing the actual space in the experience cubes and recognizing the movements (gestures and postures) of the visitors and the visitors, and providing them as the space enhancing interactive engine configured in the experience cubes;
Acquires information (real space recognition information, experiential recognition information, experiential motion recognition information) provided from the Kinect camera and provides it to the built-in space enhancement interactive engine 250 to perform interaction between the generated three-dimensional virtual object and the viewer A space enhancer 200 for implementing the space reinforcement;
A DID device 300 for displaying the interaction image information with the 3D virtual object according to the movement of the guest provided through the space enhancement device;
And a speaker (400) for outputting interaction sound information with a 3D virtual object according to a movement of a person provided through the space enhancement implementing device. The first cube (1000)
And a second cube (2000) for moving the interior of the first cube to the outside to expand the space,
The space-enhancing interactive engine 250,
An image information acquiring unit 251 for acquiring color image and depth image information from the Kinect camera 100;
The color image and the depth image obtained by the image information obtaining unit are acquired to correct the color image and the depth image, and the homography of the RGB image and the depth image is calculated to obtain the RGB camera An image matching unit 252 for processing the viewpoint of the Depth camera equally;
After extracting a straight line and extracting a straight line, the spatial information in the matching image is recognized through the extracted straight line to distinguish the spatial, static and dynamic regions. After extracting the feature points from each region, A dynamic object recognizing unit (253) for recognizing the object represented by the area using the positional information and storing the dynamic object information;
A dynamic object coordinate extraction unit 254 for extracting position information on the dynamic object using the skeleton information extracted from the Kinect camera using the depth image of the recognized dynamic object area;
A dynamic object mesh output unit (255) for acquiring position information on the dynamic object to extract skeleton information of the continuously recognized object region to generate a real time mesh mesh by estimating posture transformation;
An augmented object generation unit 256 for extracting the coordinates of the recognition space and separating the inside and the outside of the surface based on the extracted coordinates to form a surface and augmenting a virtual object on the surface formed;
A collision object is generated in the reinforcement object, a collision object is created in the dynamic object, and it is determined whether the collision object generated in the augmented object and the collision object generated in the mesh object are in conflict with each other. An augmented object event generation unit 257 for generating augmented object event;
And an object animation generation unit (258) for acquiring an event of the augmented object and executing an animation corresponding to the augmented object. The experiential cube system to which a space enhancement interactive engine capable of expanding a space is applied.
1. An experiential cube system to which a space enhancing interactive engine capable of expanding a space is applied,
A Kinect camera 100 for recognizing the actual space in the experience cubes and recognizing the movements (gestures and postures) of the visitors and the visitors, and providing them as the space enhancing interactive engine configured in the experience cubes;
Acquires information (real space recognition information, experiential recognition information, experiential motion recognition information) provided from the Kinect camera and provides it to the built-in space enhancement interactive engine 250 to perform interaction between the generated three-dimensional virtual object and the viewer A space enhancer 200 for implementing the space reinforcement;
A DID device 300 for displaying the interaction image information with the 3D virtual object according to the movement of the guest provided through the space enhancement device;
And a speaker (400) for outputting interaction sound information with a 3D virtual object according to a movement of a person provided through the space enhancement implementing device. The first cube (1000)
And a second cube (2000) for moving the interior of the first cube to the outside to expand the space,
The second cube 2000 includes:
The size of the first cube is smaller than the size of the first cube so as to be stored in the first cube,
A door 2100 formed on one side so that visitors can enter and exit,
An expanding cube handle 2200 formed on one side so as to be pulled out from the first cube,
And a plurality of wheels 2300 spaced a predetermined distance apart from the first cube 2300 and configured to be able to escape from the first cube easily when being pulled out through the extension cube handle. Experimental cube system with space enhancement interactive engine.

Images