KR20230067307A - Method for moving in 3D modeling space using avatar - Google Patents

Abstract

The present invention relates to a method for moving in a three-dimensional modeling space using an avatar and, more specifically, to a method for moving in a three-dimensional modeling space, capable of enabling a user to feel as if the user is moving in a physical three-dimensional space by moving a three-dimensional avatar to a desired space among virtual three-dimensional modeling spaces using the three-dimensional avatar; and enabling the three-dimensional avatar to seamlessly move in the three-dimensional modeling space by dividing each moving route of the three-dimensional modeling space into a plurality of unit grids and mapping a panorama image of each of the unit grids with one single panorama image extracted from a video made by filming the moving routes. The method includes the following steps of: determining current location information in a three-dimensional modeling space; determining a current panorama image mapped with the current location information; and outputting a current synthetic image.

Description

Method for moving in 3D modeling space using avatar {Method for moving in 3D modeling space using avatar}

The present invention relates to a method for moving a 3D modeling space using an avatar, and more specifically, by using a 3D avatar to move a 3D avatar to a space where a user wants to move among a virtual 3D modeling space, the user moves in a real 3D modeling space. It can give the same sense of reality as moving in dimensional space, and each movement path in the 3D modeling space is divided into a number of unit grids, and a panoramic image for each unit grid is converted into one panoramic image extracted from a video of the movement path. A method for moving a 3D modeling space in which a 3D avatar can naturally move in the 3D modeling space without interruption by setting mapping.

In order to effectively determine a position in space and move, it is required to create a map of the space in which you are moving and to recognize your own position in space. Recognizing a location in the surrounding space and forming a map is called Simultaneous Localization And Mapping (SLAM).

SLAM creates a 3D map based on the feature data of objects located in space, and the feature data used in the SLAM algorithm is also different depending on the type of sensor used to create the map. For example, distance information obtained through a distance sensor such as ultrasound or laser, coordinate information obtained directly through a GPS device, acceleration information obtained through an IMU device, image information of a camera, and the like may be used as feature data.

The SLAM technique using image information from the camera is called Visual SLAM. Visual SLAM creates a 3D map of the surrounding environment using visual feature points extracted from the video, and determines the current location based on the created 3D map. guess

1 illustrates an example of a method for generating a 3D map using a visual SLAM technique.

Looking more specifically with reference to FIG. 1 , an image of a 3D space to be generated for a map is photographed and feature points are extracted from the photographed image (S10). In order to extract feature points, a person can directly wear wearable imaging equipment, move around the 3D space to take images, and extract feature points from the captured images, or install imaging equipment on a mobile robot to move the mobile robot into a 3D space. It is possible to capture an image and extract feature points from the captured image.

Here, as the video equipment, a mono camera, a stereo camera, a depth camera, or a combination thereof may be used. Meanwhile, the camera includes a charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS) image sensor, and an image processing module generating a 2D image by receiving an output of the image sensor.

As an element representing the shape of an object, a feature point is an important point that can describe and model an object, and as a clearly identifiable interest point, a corner, apex, and vertex are included in the feature point. applicable Methods for extracting feature points from images use algorithms such as Curvature Scale Space (CSS) and FAST-Enhanced Repeatability (FAST-ER), extract specific "points" using the Harris corner detection technique, or use the Canny edge detection technique. It is possible to extract specific "lines" using Feature points can be extracted in various ways, and a detailed description thereof will be omitted.

The extracted feature points are displayed as 2-dimensional coordinates, and a point cloud is generated by converting the feature points into 3-dimensional coordinates having depth information (S30). In order to obtain depth information from the captured image, depth information is acquired using the previous/posterior image frame based on the current image frame captured through a mono camera, or depth information is obtained using the parallax of the image frame captured through a stereo camera. Information may be obtained or depth information may be obtained using a depth camera.

The extracted feature points are continuously tracked in the image taken in the 3D space, and a 3D map of the 3D space is created by combining point clouds for the entire 3D space (S50).

In the case of generating a 3D map in this way, as shown in FIG. 2, each panoramic image is captured using a photographing device such as a camera at a specific location in the 3D space, and the captured panoramic image is displayed at the corresponding location on the 3D map. It is possible to implement a virtual 3D modeling space identical to the actual physical space by mapping to .

By providing the 3D modeling space implemented as described above to the user, the user may acquire information about the real space while moving the virtual 3D modeling space implemented identically to the real space.

The user moves in the 3D modeling space implemented as described above and acquires information in the 3D modeling space. As shown in FIG. Since the 3D modeling space is moved by selecting the movement indicator, even if information can be acquired while moving the 3D modeling space, the user himself has a problem in that the sense of reality is inferior to that of moving the 3D modeling space himself.

Moreover, since the conventional 3D modeling space is generated by mapping a panoramic image captured using a photographing device at a set photographing position (1, 2, 3, ...) in the 3D space to the same position on the 3D map, The user may search the surrounding space by moving from the capturing position 1 to the capturing position 2 in units of capturing positions of the panoramic image in the 3D modeling space, for example. Therefore, it is difficult for the user to move naturally in the 3D modeling space due to the distance between the photographing positions of the panoramic image, as if the user actually moves.

The present invention is to solve the problem of the movement method of the 3D modeling space mentioned above, and the object to be achieved by the present invention is to move the user to a position in the 3D modeling space where the user wants to move using a 3D avatar. To provide a method for moving a 3D modeling space that can give the same sense of reality as moving an actual 3D modeling space.

Another object to be achieved by the present invention is to divide each movement path in the 3D modeling space into a plurality of unit grids and capture a video of the movement path (where the video is taken along the movement path at the same speed and at the same speed). It is to provide a method for moving a 3D modeling space that can be implemented like a 3D avatar naturally moving in a 3D space by mapping the panoramic image extracted from a location) to a panoramic image for each unit grid.

In order to achieve the object of the present invention, a method for moving a 3D modeling space according to the present invention includes the steps of determining current location information in which a 3D avatar freely moving in a 3D modeling space is currently located in a 3D modeling space; A step comprising determining a current panoramic image mapped to the current location information among a plurality of panoramic images of a 3D space, and outputting a composite image composed of the 3D avatar and the current panoramic image. do.

Preferably, the 3D modeling space according to the present invention is divided into a plurality of unit grids along a movement path along which the 3D avatar can move, and the moving method of the 3D modeling space according to the present invention is a 3D modeling space among the plurality of unit grids. A step of determining a current unit grid where the avatar is located is further included, and the current unit grid is determined as current location information of the 3D avatar.

Preferably, the method of moving the 3D modeling space according to the present invention moves the next unit grid where the 3D avatar is located to the next position based on input movement information of the 3D avatar among a plurality of preliminary next unit grids adjacent to the current unit grid. Further comprising the step of determining with information, the step of determining the next panoramic image mapped to the next unit grid, and the step of outputting the next synthesized image consisting of the 3D avatar and the next panoramic image to correspond to the movement information. do.

Here, the plurality of panoramic images of the 3D space are characterized in that they are extracted from a moving picture of the 3D space while moving the photographing equipment at the same height and the same speed.

In the method for moving a 3D modeling space according to the present invention, the 3D avatar moves in the 3D modeling space at unit grid intervals according to an input user command.

Preferably, the current panoramic image mapped to the current location information includes the step of determining the number of unit grids constituting the first movement route in the 3D space, and the panoramic image constituting the moving image captured along the first movement route. is divided by the number of unit grids to be divided into panoramic images allocated for each unit grid, and mapping is performed through the step of setting one panoramic image among the allocated panoramic images for each unit grid as a panoramic image for the corresponding unit grid. to be

Here, a method of moving a three-dimensional modeling space, characterized in that one panoramic image spaced apart at a regular interval among the panoramic images allocated for each unit grid is set as a panoramic image mapped to the unit grid.

A method of moving a 3D modeling space according to the present invention moves a 3D avatar to a space where a user wants to move among a 3D modeling space using a 3D avatar, thereby providing the same sense of reality as when a user actually moves in a 3D modeling space. can give

On the other hand, in the moving method of the 3D modeling space according to the present invention, each movement path in the 3D space is divided into a plurality of unit grids, and one panoramic image of the video taken for each unit grid is mapped and set. It can be implemented by moving naturally through the dimensional modeling space.

1 illustrates an example of a method for generating a 3D map using a visual SLAM technique.
2 is a diagram for explaining a movement method in a conventional 3D modeling space.
3 is a functional block diagram for explaining an apparatus for moving an avatar in a 3D modeling space according to the present invention.
4 is a functional block diagram for explaining an example of an image mapping unit according to the present invention.
5 is a diagram for explaining an example of capturing a video using a photographing device.
6 is a diagram for explaining an example of a video in which a moving path in a 3D space is captured in the present invention.
7 is a diagram for explaining an example of implementing a 3D modeling space with different movement resolutions.
8 is a flowchart illustrating a method of moving an avatar in a 3D modeling space according to the present invention.
9 illustrates a 3D avatar moving along a movement path divided into unit grids on a 3D map.
10 illustrates an example of a preliminary next unit grid in which a 3D avatar can move.

It should be noted that technical terms used in the present invention are only used to describe specific embodiments and are not intended to limit the present invention. In addition, technical terms used in the present invention should be interpreted in terms commonly understood by those of ordinary skill in the art to which the present invention belongs, unless specifically defined otherwise in the present invention, and are excessively inclusive. It should not be interpreted in a positive sense or in an excessively reduced sense. In addition, when the technical terms used in the present invention are incorrect technical terms that do not accurately express the spirit of the present invention, they should be replaced with technical terms that those skilled in the art can correctly understand.

Also, singular expressions used in the present invention include plural expressions unless the context clearly dictates otherwise. In the present invention, terms such as "consisting of" or "comprising" should not be construed as necessarily including all of the various elements or steps described in the invention, and some of the elements or steps are included. It should be construed that it may not be, or may further include additional components or steps.

In addition, it should be noted that the accompanying drawings are only for the purpose of easily understanding the spirit of the present invention, and should not be construed as limiting the spirit of the present invention by the accompanying drawings.

3 is a functional block diagram for explaining an apparatus for moving an avatar in a 3D modeling space according to the present invention.

Looking more specifically with reference to FIG. 3 , the location information determiner 110 determines current location information based on the current unit grid where the 3D avatar is currently located in the 3D modeling space. Here, in the 3D modeling space, there are a plurality of movement paths through which the 3D avatar can move, and each movement path is divided into a plurality of unit grids.

The image determination unit 130 extracts a current panoramic image mapped to the current unit grid corresponding to the current location information from the storage unit 150 based on the current location information of the 3D avatar, and the image generator 170 From the extracted current panoramic image and the 3D avatar image, a composite image of the space where the 3D avatar is currently located in the 3D modeling space is created. In this case, the storage unit 150 stores a panoramic image extracted from a video of a moving path in a 3D space at the same speed and the same shooting angle and mapped to each unit grid constituting the moving path.

On the other hand, when a user's movement command for moving the 3D avatar is input, the location information determining unit 110 moves the 3D avatar based on the inputted avatar movement command among a plurality of preliminary next unit grids adjacent to the current unit grid. The next unit grid where is located is determined as the next location information. As shown in FIG. 10, there are a plurality of preliminary next unit grids adjacent to the current unit grid, to which the 3D avatar can move in the 3D modeling space. It is possible to determine the next unit grid to be moved in succession from the modern unit grid.

When the next location information is determined, the image determination unit 130 stores the next panoramic image mapped to the next unit grid corresponding to the next location information based on the next location information of the 3D avatar in the same manner as described above. 150), and the image generating unit 170 generates the next synthesized image for the space where the 3D avatar is located among the 3D modeling space from the extracted panoramic image and the 3D avatar image.

According to the user's movement command, synthesized images are continuously generated along the path along which the 3D avatar moves, and the output control unit 190 outputs and controls the continuously generated synthesized images to the display of the user terminal.

Here, the image mapping unit 140 extracts and sets a panoramic image for each unit grid in the 3D modeling space from a moving picture of a moving path in the 3D space, and maps the image mapping unit 140 to each unit grid. The set panoramic image is stored in the storage unit 150 . In this way, by extracting a panoramic image of the unit grid constituting the movement path from a video taken and mapping the extracted panoramic image to the unit grid and storing it in the storage unit 150, whenever the 3D avatar moves, the 3D avatar Panoramic images for the unit grid where is located can be processed quickly with fewer processes than directly extracting them from a video.

4 is a functional block diagram for explaining an example of an image mapping unit according to the present invention.

Looking more specifically with reference to FIG. 4 , the grid divider 210 divides the moving path in the 3D space according to the input total number of unit grids.

The image allocator 230 divides and allocates the panoramic images constituting the video into panoramic images corresponding to each unit grid according to the number of total unit grids constituting the movement path. Here, a moving path in a 3D space is photographed at the same speed and at the same shooting angle to generate a video. For example, if the total number of unit grids constituting the moving route is 10 and the video recording the moving route consists of 1000 panoramic images, 100 consecutive panoramic images are converted into panoramic images corresponding to each unit grid. allocated separately.

The image setting unit 250 maps and sets one panoramic image among the panoramic images allocated to each unit grid as a panoramic image for each unit grid. Here, as an example of setting the mapping of the panoramic image for the unit grid in the image setting unit 250, the image setting unit 250 randomly selects one panoramic image from among the panoramic images assigned to each unit grid and assigns it to the corresponding unit grid. It can be set by mapping it to a panoramic image.

As another example of mapping the panoramic image to the unit grid in the image setting unit 250, the image setting unit 250 selects the panoramic images spaced at the same interval among the panoramic images assigned to each unit grid to the corresponding unit grid. It can be set by mapping to a panoramic image. For example, when 100 panoramic images are allocated to each unit grid, a 50th panoramic image among the panoramic images allocated to each unit grid may be mapped and set as a panoramic image for the corresponding unit grid.

Here, a moving path in the 3D space is moved at the same speed using the photographing device 1 shown in FIG. 5 and a video is taken at the same shooting angle. A natural 3D modeling space can be implemented according to the movement of the 3D avatar by generating a composite image for .

In addition, a video is taken while moving at the same speed, and the panoramic image constituting the video is divided by the total number of unit grids of the input movement route, divided into panoramic images corresponding to each unit grid, and allocated, and one panoramic image among the allocated panoramic images By setting the mapping to the unit grid, the 3D modeling space can be easily changed to different movement resolutions. That is, if the total number of unit grids constituting the movement path is set small, it can be implemented as a 3D modeling space with low movement resolution, and if the total number of unit grids constituting the movement path is set large, 3D modeling space with high movement resolution It can be implemented as a modeling space.

By easily changing the movement resolution in this way, the 3D modeling space can be differently implemented according to the purpose for which the 3D modeling space is used or the infrastructure for implementing the 3D modeling space.

6 is a diagram for explaining an example of a video in which a moving path in a 3D space is captured in the present invention.

Referring to FIG. 6 , the photographing device 1 captures a video while moving each movement path in a 3D space at the same speed and the same photographing angle. Whenever each moving route is taken, a video (A file, B file, ..., L file) for each moving route is created. Here, since the photographing device moves at the same speed and captures the moving path, it has the same number of panoramic image frames per unit time.

7 is a diagram for explaining an example of implementing a 3D modeling space with different movement resolutions.

The 3D modeling space may be implemented with different movement resolutions depending on the purpose of use of the 3D modeling space or the performance of the infrastructure that implements the 3D modeling space, for example, hardware. When the total number of unit grids is set to be different from each other, the 3D modeling space can be implemented with different movement resolutions for the total number of unit greases.

That is, in the case of implementing a 3D modeling space with a low movement resolution, as shown in FIG. Each unit grid is divided and allocated, and one of the allocated panoramic images is mapped to a panoramic image (F1, F201, F401, F601, F801) for the unit grid and set.

On the other hand, if you want to implement a 3D modeling space with high movement resolution, set the total number of unit grids constituting the movement path (P) to 20 as shown in FIG. Each unit grid is divided and allocated, and one of the allocated panoramic images is mapped to a panoramic image (F1, F51, F101, .... F901, F951) for the unit grid and set.

The 3D avatar moves the 3D modeling space as a unit grid. When the 3D modeling space is implemented with a higher movement resolution than when the 3D modeling space is implemented with a lower movement resolution, the 3D avatar moves the 3D modeling space with a higher movement resolution. You can move naturally.

8 is a flowchart illustrating a method of moving an avatar in a 3D modeling space according to the present invention.

Looking more specifically with reference to FIG. 8 , the current location information of the 3D avatar currently located in the 3D modeling space is determined (S110). The 3D modeling space is divided into a plurality of unit grids, and the current unit grid where the 3D avatar is located in the 3D modeling space is determined as the current location information of the 3D avatar.

A current panoramic image mapped to the current unit grid of the determined current location information is determined (S130).

A synthesized image composed of the current panoramic image and the 3D avatar is generated and the generated synthesized image is displayed on the display unit of the user terminal (S150).

It is determined whether a movement command is received from the user terminal (S170). If the movement command is received, the next unit grid in which the 3D avatar is located among a plurality of preliminary next unit grids adjacent to the current unit grid is determined as the next location information according to the movement command. After determining, the next synthesized image composed of the next unit grid and the 3D avatar, which is set to be mapped to the next unit grid, is displayed on the display unit of the user terminal (S190).

9 illustrates a 3D avatar moving along a movement path divided into unit grids on a 3D map.

As shown in FIG. 9(a), the movement path of the 3D avatar on the 3D map is divided into a plurality of unit grids.

As shown in FIG. 9(b), each unit grid has a panoramic image mapped to the unit grid. When the 3D avatar moves the unit grid according to the user's movement command, the 3D avatar moves to the current position. A panoramic image mapped to the unit grid to be displayed is displayed together with the 3D avatar.

On the other hand, the above-described embodiments of the present invention can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium.

The computer-readable recording medium includes a magnetic storage medium (eg, ROM, floppy disk, hard disk, etc.), an optical reading medium (eg, CD-ROM, DVD, etc.), and a carrier wave (eg, Internet transmission through).

Although the present invention has been described with reference to the embodiments shown in the drawings, this is only exemplary, and those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the attached claims.

1: photographing device 110: location information determining unit
130: image determination unit 140: image mapping unit
150: storage unit 170: image generation unit
190: output control unit 210: grid division unit
230: image allocation unit 250: image setting unit

Claims (7)

A method for moving a 3D modeling space to which a panoramic image is mapped,
determining current location information of a 3D avatar freely moving in a 3D modeling space and currently located in the 3D modeling space;
determining a current panoramic image mapped to the current location information among a plurality of panoramic images of a 3D space; and
and outputting a current composite image composed of the 3D avatar and the current panoramic image.
According to claim 1,
The 3D modeling space is divided into a plurality of unit grids along a movement path along which the 3D avatar can move,
The method of moving the 3D modeling space
Further comprising determining a current unit grid in which the 3D avatar is located among the plurality of unit grids;
A method of moving a 3D modeling space, characterized in that the current unit grid is determined as the current location information of the 3D avatar.
The method of claim 2, wherein the method of moving the 3D modeling space
determining a next unit grid in which the 3D avatar is located as next location information based on movement information of the 3D avatar among a plurality of preliminary next unit grids adjacent to the current unit grid;
determining a next panoramic image mapped to the next unit grid; and
and outputting a next synthesized image composed of the 3D avatar and the next panoramic image to correspond to the movement information.
According to claim 3,
A method of moving a 3D modeling space, characterized in that a plurality of panoramic images of the 3D space are extracted from a video taken in the 3D space while moving the shooting equipment at the same height and the same speed.
The method of claim 4, in the movement method of the 3D modeling space
The method of moving the 3D modeling space, characterized in that the 3D avatar moves the 3D modeling space at the unit grid interval according to an input user command.
The method of claim 4, wherein the current panoramic image mapped to the current location information is
determining the number of unit grids constituting a first movement path in the 3D space;
Dividing the panoramic image constituting the moving picture taken along the first movement route by the number of unit grids into panoramic images assigned to each unit grid; and
A method of moving a 3D modeling space, characterized in that mapping through the step of setting one of the panoramic images allocated for each unit grid as a panoramic image for the corresponding unit grid.
The method of claim 6, wherein one panoramic image spaced apart at a regular interval among the panoramic images assigned to each unit grid is set as a panoramic image mapped to the unit grid.

Images