KR102074680B1 - Device and method for moving and control of real estate model house virtual space using gesture recognition module - Google Patents

Abstract

One embodiment of the present invention may use functions such as movement and interaction, menu selection while controlling a 3D virtual space in the air by hand gestures on a device from a large display to a kiosk using a gesture recognition module. An image for a model house VR is provided and an image corresponding to a user interaction can be provided to the user. That is, the user can remove the inconvenience such as manipulating the touch or the mouse and take a specific gesture by hand, so that the user can check the VR image and move from virtual space to another space, thereby increasing user convenience. .

Description

Movement and control method and device for real estate model house virtual space using gesture recognition module {DEVICE AND METHOD FOR MOVING AND CONTROL OF REAL ESTATE MODEL HOUSE VIRTUAL SPACE USING GESTURE RECOGNITION MODULE}

The present invention relates to a method and apparatus for moving and controlling a real estate model house virtual space using a gesture recognition module, and more specifically, a surrounding environment and convenience for an actual apartment that are not all realized due to space limitations in the model house. The present invention relates to a method and apparatus for moving and controlling a real estate model house virtual space using a gesture recognition module that enables users to experience houses of different facilities and different structures and through VR and AR.

In the past, people visit the model house in person to check the apartments to be completed in the future, and only the countertops and structures provided in the model house can be checked with their own eyes. It had a form to do.

To solve this inconvenience, create a 360-degree image by pasting an actual photo or an image created through 3D CG through a cyber model house using a 360-paranomic image, and using a PC, smart device, Google cardboard, etc. in the model house. The unconstructed form was also made visible.

However, it is a problem that there is a big difference from seeing the structure constructed in the actual model house, because only the space where the photo was taken, that is, it proceeds only according to the movement of the specified camera, and only the image taken 360 degrees around the camera can be checked. This existed.

The present invention is to solve the above-mentioned problems of the prior art, and a 360-degree image made of 3D graphics can be rendered in real time to observe objects in the screen in high quality, and actively select directions and move while voluntarily moving. The present invention is to provide a method and apparatus for moving and controlling a virtual space in a real estate model house using a gesture recognition module, which enables a user to control only a simple hand gesture rather than a complicated operation through gesture recognition.

However, the technical problems to be achieved by the present embodiment are not limited to the technical problems as described above, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, a method for moving and controlling a virtual space in a real estate model house using a gesture recognition module performed by a server receives an entry request for the virtual space and through a gesture recognition module Recognizing the interaction by the user's hand, the command corresponding to the interaction is searched, and an image in which the command is performed is provided in the current image. The virtual space is provided as a first person perspective, and the virtual space is a real model house implemented as 3D VR content, and includes the entire interior and the surroundings of the apartment complex, amenities within the complex, and the entire landscape. The image providing device is fixed to the ground, and is formed on a body part including a gesture recognition module for recognizing the interaction and on the top of the body part, and is moved and controlled in response to an operation recognized through the gesture recognition module. It includes a display unit including a display module for providing the user.

In addition, the apparatus for performing the movement and control method for the real estate model house virtual space using the gesture recognition module is fixed to the ground, and is formed on a body portion including a gesture recognition module for recognizing the interaction and on the top of the body portion. , A display unit including a display module for providing the user with an image that is moved and controlled in response to an operation recognized through the gesture recognition module, and receives a gesture request for entry into the virtual space, the server The request is transmitted to provide the corresponding image through the display unit, recognize the user's hand received through the gesture recognition module, and provide the image corresponding to the interaction.

One embodiment of the present invention may use functions such as movement and interaction, menu selection, etc. while controlling a 3D virtual space in the air by hand gestures on a device from a large display to a kiosk using a gesture recognition module.

In the display module, an image for a model house VR is provided and an image corresponding to a user interaction can be provided to the user. That is, the user can remove the inconvenience such as manipulating the touch or the mouse and take a specific gesture by hand, so that the user can check the VR image and move from virtual space to another space, thereby increasing user convenience. .

1 is a block diagram of a system according to an embodiment of the present invention.
2 is a block diagram showing the internal configuration of a server according to an embodiment of the present invention.
3 to 5 are actual photographs of an apparatus for performing a method of moving and controlling a virtual space in a real estate model house using a gesture recognition module according to an embodiment of the present invention.
6 is an exemplary view of a kiosk type device according to an embodiment of the present invention.
7 is a flowchart illustrating a method for moving and controlling a real estate model house virtual space using a gesture recognition module according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains can easily practice. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present invention in the drawings, parts irrelevant to the description are omitted, and like reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part is "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element in between. . Also, when a part “includes” a certain component, this means that other components may be further included rather than excluding other components, unless otherwise specified.

In the present specification, the term “unit” includes a unit realized by hardware, a unit realized by software, and a unit realized by using both. Further, one unit may be realized by using two or more hardware, and two or more units may be realized by one hardware. Meanwhile, the '~ unit' is not limited to software or hardware, and the '~ unit' may be configured to be in an addressable storage medium or may be configured to play one or more processors. Thus, as an example, '~ unit' refers to components such as software components, object-oriented software components, class components and task components, processes, functions, attributes, and procedures. , Subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, database, data structures, tables, arrays and variables. The functions provided within components and '~ units' may be combined into a smaller number of components and '~ units', or further separated into additional components and '~ units'. In addition, the components and '~ unit' may be implemented to play one or more CPUs in the device or secure multimedia card.

The "terminal" mentioned below may be embodied as a computer or portable terminal capable of accessing a server or other terminal through a network. Here, the computer, for example, a laptop equipped with a web browser (WEB Browser), desktop (desktop), laptop (laptop), VR HMD (e.g., HTC VIVE, Oculus Rift, GearVR, DayDream, PSVR, etc.) It may include. Here, VR HMD is for PC (e.g., HTC VIVE, Oculus Rift, FOVE, Deepon, etc.) and mobile (e.g. GearVR, DayDream, Storm Racing, Google Cardboard, etc.) and console (PSVR). Includes all Stand Alone models (eg Deepon, PICO, etc.) that are implemented independently. The portable terminal is, for example, a wireless communication device that guarantees portability and mobility, as well as a smart phone, a tablet PC, and a wearable device, as well as Bluetooth (BLE, Bluetooth Low Energy), NFC, RFID, and ultrasound (Ultrasonic). , Infrared, Wi-Fi (WiFi), LiFi (LiFi), and may include various devices equipped with a communication module.

In addition, "network" refers to a connection structure capable of exchanging information between each node such as terminals and servers, a local area network (LAN), a wide area network (WAN), and the Internet. (WWW: World Wide Web), wired and wireless data communication networks, telephone networks, and wired and wireless television communication networks. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasound Communication, Visible Light Communication (VLC), LiFi, and the like are included, but are not limited thereto.

In addition, in this specification, the virtual space is provided as a first person view provided through a display module of a device that performs a method for moving and controlling a virtual space in a real estate model house using a gesture recognition module such as a user terminal. By implementing 3D VR content, it can be content including the entire interior and surrounding environment of the apartment complex, amenities in the complex, and the landscape of the entire complex.

Hereinafter, a system 10 for performing a method of moving and controlling a real estate model house virtual space using a gesture recognition module according to an embodiment of the present invention will be described in detail with reference to FIG. 1.

The system 10 according to an embodiment of the present invention may include a server 100 and a terminal 200. The server 100 and the terminal 200 may be communicatively connected through a wired or wireless network.

The server 100 may search for the content of the requested virtual space according to the request of the terminal 200 and deliver the VR content of the first person view point to the user through the terminal 200 for the corresponding virtual space.

The terminal 200 is a device possessed by a user or a computing device installed in a specific place, and a real estate model house virtual space application may be installed. In addition, in this specification, the terminal 200 may include an image providing device. When a user requests to enter a specific virtual space, a request query may be transmitted to the server 100 and VR content received from the server 100 may be provided to the user.

Hereinafter, a server 100 according to an embodiment of the present invention will be described in detail with reference to FIG. 2.

The server 100 receives a query from the terminal 200, and at this time, the query enters a new virtual space, moves from the current virtual space to another virtual space, and queries for moving the viewpoint within the current virtual space. It can contain.

When the network-connected terminal 200 transmits a query for a specific request, a query is received through the communication module 110 in the server 100, and the received query requests the corresponding query through the processor 130 can confirm.

The memory 120 may store a program for performing a method of moving and controlling a real estate model house virtual space using a gesture recognition module.

The processor 130 may search for the VR content for providing to the terminal 200 according to the query from the DB 140 in the server 100, and deliver the searched contents to the terminal 200 through the communication module 110. have.

For example, when the user requests admission to the 25-pyeong apartment, the terminal 200 transmits a query requesting VR content for the 25-pyeong apartment to the server 100, and the server 100 queries the corresponding Among the contents stored in the DB 140, VR contents of the 25-pyeong apartment can be provided to the user through the terminal 200.

Hereinafter, gesture recognition according to an embodiment of the present invention will be described in detail with reference to FIGS. 3 to 5.

A method for moving and controlling a virtual space in a real estate model house using a gesture recognition module according to an embodiment of the present invention, receives an entry request for the virtual space, and interacts with a user's hand received through the gesture recognition module. Recognizing, searching for a command corresponding to the interaction, and providing an image in which the command is performed in the current image. The virtual space is provided as a first person view, and the virtual space is a 3D VR content that embodies a real model house, and is content that includes the entire interior and the surroundings of the apartment complex, amenities in the complex, and views of the entire complex. The image providing device is fixed to the ground, is formed on the top of the body portion 420 and the body portion 420 including a gesture recognition module for recognizing interactions, and moves and controls in response to motions recognized through the gesture recognition module It may include a display unit 420 that includes a display module for providing a video to the user.

In performing a movement and control method for a virtual space in a real estate model house using a gesture recognition module according to an embodiment of the present invention, an image provided by interaction through gesture recognition changes or views a viewpoint in a virtual space by an image It can be provided for movement to occur.

Referring to FIG. 3, the user spreads both hands and places the palms facing each other to create a gap between the right hand and the left hand. The gesture may be a gesture that activates the main menu screen.

That is, the image provided by the user interaction with the gesture of the user opening the palm of the hand may be a main menu image capable of inputting a selection for initial entry, or a menu image capable of moving from a space corresponding to a currently provided time to another space. You can.

When the menu is opened, it is possible to select the area within the virtual space by simply touching it, not a special operation or a commanded gesture, and a 3D image of the provided virtual space can be selected.

Referring to FIG. 4, the user is in a fist gripping motion, and the corresponding gesture may be a gesture moving forward in a virtual space.

That is, the image provided by the interaction may be an image in which the viewpoint currently provided in the virtual space moves forward. It is possible to move in the direction viewed from the 3D image of the virtual space currently viewed through the display module.

In the virtual space 3D image data provided to the user, the movement function is when moving from a wide location, for example, if the user wants to see the environment around the apartment complex, select the apartment complex environment from the menu, and the image of the apartment complex environment In this case, the image of the virtual space at the first person point of view is provided, and the user can check the surrounding environment while moving freely within the image of the virtual space called the surrounding environment of the apartment.

Moving forward in virtual space refers to moving in the direction you are looking at from the first person point of view, and the image of the virtual space is provided as a 360-degree image, so even if the user looks at the side rather than the front or looks back, it continues to be virtual. An image of a space is provided, and even if the user's body does not turn behind or rotate to the side, when the gaze moves, the server 100 changes the direction in accordance with the gaze movement in the virtual space to correspond to the corresponding interaction Can provide

Referring to FIG. 5 below, the user is taking a gesture to open the palm of the hand, and the gesture may be an interaction of a command to stop at the current position.

That is, when the user opens the palm of the hand, the image provided by the interaction of opening the palm may be a still image at the time currently provided in the virtual space.

For example, if a user receives an image for a wide virtual space and spreads his palm while moving by interaction through a fist-gesturing gesture in the virtual space area, the movement of the image at the current position, the current position, stops. , There can be.

In virtual space, stop refers to a stop in the image provided through the current display while moving in the direction you are currently looking at in the first person view. Even if the user stops moving, the surrounding trees are shaken or placed. The character's movement and other object's movements placed in the virtual space can continuously move regardless of the user's stoppage.

Hereinafter, an image providing apparatus performing a method of moving and controlling a real estate model house virtual space according to an embodiment of the present invention will be described in detail with reference to FIGS. 6 to 7.

As illustrated in FIG. 6, a display unit including a body unit 420 including a gesture recognition module and a display module provided on an upper portion of the body unit 420 that can be fixed to the ground to provide an image to a user It can be composed of 420.

The gesture recognition module included in the body portion 420 may be disposed in front of the screen on which the image is provided on the display unit 420.

As shown, the image providing device may be provided in the form of a kiosk.

A real estate model house virtual space application may be installed in the kiosk type video providing device. It may include a gesture recognition module. When a user requests to enter a specific virtual space, a request query may be transmitted to the server 100 and VR content received from the server 100 may be provided to the user.

That is, the image providing device in the form of a kiosk is a kiosk, but may perform the same function as the terminal 200.

In the kiosk type image providing apparatus according to an embodiment of the present invention, the display unit 420 is provided at the top of the body unit 420, and the body unit 420 and the rear surfaces of the display unit 420 are inclined to form an acute angle. Can be formed.

That is, the inclined shape is taken at the top of the body portion 420, and the front surface and the body portion 420 provided with a screen form an obtuse angle, and the cabinet 401 formed by the back surface and the body portion 420 forms an acute angle. Can be tilted

The gesture recognition module disposed in one area of the upper end of the body 420 may be provided at the lower end of the display unit 420 based on the display unit 420. Alternatively, the display unit 420 is provided on the upper surface of the body portion 410 is disposed, it may be provided in a position closer to the user than the display unit 420.

This may be possible when the user makes a gesture for user interaction on the video providing device, in front of the display unit.

When the display unit 420 causes a portion provided with a screen to look at the top like other touch-type kiosks, there may be a disadvantage that it does not fit the user's eye level, and may be provided at right angles or the front of the display unit 420 ( When the screen) and the body 420 form an acute angle, gesture recognition may not be properly performed.

Accordingly, the kiosk type image providing device may increase the recognition rate of the gesture recognition module by tilting the display unit 420, and allow the user to conveniently use it.

Hereinafter, a procedure for a method for moving and controlling a real estate model house virtual space using a gesture recognition module performed by a server according to an embodiment of the present invention will be described in detail with reference to FIG. 8.

First, the server 100 receives an admission request for the virtual space from the terminal 200 (S310). At this time, the query for the admission request may be a specific space selected from the menu, or may be a query for the screen for starting the first time and selecting the menu.

If it is the first time, it is possible to provide a video menu for a virtual space where admission is possible, and when a different video is requested through a menu in a situation where a video for a virtual space is currently being provided, a request is received from the terminal 200 It is possible to provide image data for a specific virtual space.

Thereafter, the user may be able to move based on the image data for the virtual space currently provided.

The gesture recognition module recognizes the user's hand and, after recognizing the hand, may recognize the interaction through the user's hand motion in consideration of the shape or direction of the hand (S320).

When a pre-planned interaction, such as clenching a fist, opening a hand, or opening a gap between both hands, is received, the server 100 may provide corresponding image data.

According to the gesture taken by the user, the gesture recognition module may search for a command corresponding to the user's interaction and provide image data on which the command requested by the interaction is performed to the user through the terminal 200.

In a further embodiment of the present invention, the interaction through a gesture in which the user takes the action of spreading one hand and holding the fist in one hand can go to the end in the current viewing direction and stop when a dead end comes out.

Or you can move to the direction you are currently looking at, and when a dead end comes out, you can move it by rotating in the direction you are spreading your hands.

That is, when both hands are recognized and different gestures are taken, various interactions are possible and various commands can be made by recognizing the motions of two different hands.

As a further embodiment of the present invention, when a 3D image of a virtual space is provided, it may be possible to divide the screen and receive two screens. By providing an independent virtual space on the screen, commands by user interaction can be performed for each screen.

One embodiment of the present invention may also be embodied in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, the computer-readable medium may include any computer storage medium. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

Although the methods and systems of the present invention have been described in connection with specific embodiments, some or all of their components or operations may be implemented using a computer system having a general purpose hardware architecture.

The above description of the present invention is for illustration only, and those skilled in the art to which the present invention pertains can understand that it can be easily modified to other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and it should be interpreted that all changes or modified forms derived from the meaning and scope of the claims and equivalent concepts thereof are included in the scope of the present invention. do.

10: system 100: server
110: communication module 130: processor
140: DB
200: terminal
400: video providing device
410: body portion 420: display unit
401: angle between the body and the back of the display

Claims (12)

In a method for moving and controlling a real estate model house virtual space using a gesture recognition module performed by a server,
Receiving a request to enter the virtual space;
Providing a real estate model house menu for the virtual space, wherein the real estate model house menu includes content including contents of the entire interior and surroundings of the apartment complex, convenience facilities in the complex, and views of the entire complex;
Receiving a selection input for any one of the real estate model house menus and providing a corresponding image;
Recognizing the interaction by one hand of the user received through the video providing device; And
Searching for a command corresponding to the interaction and providing an image in which the command is performed in the current image;
It includes,
When receiving a predetermined specific interaction among the interactions, the real estate model house menu is provided,
The virtual space,
It is provided as a first person perspective, and the virtual space is a real model house that is implemented as 3D VR content, and includes the entire interior and the surroundings of the apartment complex, amenities within the complex, and the entire landscape.
When both hands of a user having different shapes are recognized through the image providing device, a complex interaction is performed by recognizing the motions of the two hands and combining commands corresponding to the motions of each hand according to a preset method. ,
The video providing device,
A body part fixed to the ground and including a gesture recognition device for recognizing the interaction; And
A display unit formed on an upper portion of the body unit and including a display module providing an image that is moved and controlled in response to an operation recognized by the gesture recognition module to the user;
A method of moving and controlling a real estate model house virtual space using a gesture recognition module. According to claim 1,
The display unit,
The body portion and the rear surface of the display portion is formed to be inclined to form an acute angle,
A method for moving and controlling a real estate model house virtual space using a gesture recognition module.
According to claim 1,
The image provided by the interaction,
It is provided to change the viewpoint or move the viewpoint in the virtual space by the image,
A method for moving and controlling a real estate model house virtual space using a gesture recognition module.
The method of claim 3,
The interaction,
The user comprises the action of clenching a fist,
The image provided by the interaction is an image in which the view point currently provided in the virtual space moves forward,
A method for moving and controlling a real estate model house virtual space using a gesture recognition module.
The method of claim 3,
The interaction,
The user includes the operation of spreading the palm,
The image provided by the interaction is an image that stops at the time currently provided in the virtual space,
A method for moving and controlling a real estate model house virtual space using a gesture recognition module.
The method of claim 3,
The interaction,
The user comprises the operation of opening the gap between both hands,
The image provided by the interaction is a menu image that can be moved from the space corresponding to the currently provided time in the virtual space to another space,
A method for moving and controlling a real estate model house virtual space using a gesture recognition module.
In the image providing apparatus for performing a movement and control method for a real estate model house virtual space using a gesture recognition module,
A body part fixed to the ground and including an image providing device for recognizing an interaction; And
A display unit formed on an upper portion of the body portion and including a display module that provides a user with an image that is moved and controlled in response to an operation recognized by the image providing device;
Including,
When the image providing device receives a request to enter the virtual space, it transmits a request to a server to provide a corresponding image through a display unit,
Providing a real estate model house menu for the virtual space,
The real estate model house menu is provided including contents including the entire interior and surroundings of the apartment complex, amenities in the complex, and views of the entire complex,
When a selection input for any one of the real estate model house menus is received, a corresponding video is provided,
Recognize one hand of the user received through the video providing device,
Provide an image corresponding to the interaction,
When receiving a predetermined specific interaction among the interactions, the real estate model house menu is provided,
When both hands of different shapes of the user are recognized through the image providing device, a complex interaction is performed by recognizing the motions of the two hands and combining commands corresponding to the motions of each hand according to a preset method. ,
The virtual space,
It is provided as a first person perspective, and the virtual space is a real model house that is implemented as 3D VR content, which is content that includes the entire interior and surrounding environment of the apartment complex, amenities in the complex, and the entire landscape.
An image providing device that performs a movement and control method for a real estate model house virtual space using a gesture recognition module.
The method of claim 7,
The display unit,
The body portion and the rear surface of the display portion is formed to be inclined at a predetermined angle to form an acute angle,
An image providing device that performs a movement and control method for a real estate model house virtual space using a gesture recognition module.
The method of claim 7,
The video corresponding to the interaction,
It is provided to change the viewpoint or move the viewpoint in the virtual space by the image,
An image providing device that performs a movement and control method for a real estate model house virtual space using a gesture recognition module.
The method of claim 9,
The interaction,
The user comprises the action of clenching a fist,
The image corresponding to the interaction is an image in which the viewpoint currently provided in the virtual space moves forward,
An image providing device that performs a movement and control method for a real estate model house virtual space using a gesture recognition module.
The method of claim 9,
The interaction,
The user includes the operation of spreading the palm,
The image corresponding to the interaction is an image that stops at the time currently provided in the virtual space,
An image providing device that performs a movement and control method for a real estate model house virtual space using a gesture recognition module.
The method of claim 9,
The interaction,
The user comprises the operation of opening the gap between both hands,
The image corresponding to the interaction is a menu image that can be moved from the space corresponding to the currently provided time in the virtual space to another space,
An image providing device that performs a movement and control method for a real estate model house virtual space using a gesture recognition module.

Images