KR20210067525A - Virtual Camera Tracking System, a real-time MR environment synthesis system using it - Google Patents

Abstract

According to one embodiment of the present invention, provided are a virtual camera tracking system, and a real-time MR environment synthesizing system using the same. The virtual camera tracking system has a feature of more precisely synthesizing photographed images photographed indoors and/or outdoors and MR images. The system includes a depth camera module, a distance measuring scanner (RPLIDAR), a tracking unit, and an image photographing unit.

Description

Virtual Camera Tracking System, a real-time MR environment synthesis system using it

The present invention relates to a virtual camera tracking system and a real-time synthesizing system for a MMR environment using the same.

Augmented reality (AR) and/or virtual reality (VR) systems may create three-dimensional (3D) immersive virtual environments. A user may, for example, a helmet or other head-mounted device comprising a display, glasses or goggles - when the user looks through the glasses or goggles - gloves equipped with a sensor, including sensors This virtual environment can be experienced through interaction with various electronic devices, such as external hand held devices and other such electronic devices. Once immersed in the virtual environment, user interaction with the virtual environment may take various forms, such as, for example, physical movement and/or manipulation of an electronic device to interact, personalize, and control the virtual environment.

Laid-open Patent Publication No. 10-2017-0130580

The problem to be solved by the present invention is to provide a virtual camera tracking system and a real-time synthesizing system for a MMR environment using the same.

A virtual camera tracking system according to an embodiment of the present invention for solving the above problems and a real-time MR environment synthesizing system using the same can more precisely synthesize the captured images and MR images captured indoors and/or outdoors.

By using the virtual camera tracking system according to an embodiment of the present invention and a real-time MR environment synthesizing system using the same, it is possible to more precisely synthesize the captured images and MR images captured indoors and/or outdoors.

1 is a block diagram illustrating a virtual camera tracking system and a real-time MR environment synthesis system using the same according to an embodiment of the present invention.
2 to 4 are exemplary diagrams illustrating a connection configuration diagram and a support function between the virtual camera tracking system shown in FIG.
5 illustrates an example computing environment in which one or more embodiments disclosed herein may be implemented.

Various embodiments and/or aspects are now disclosed with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth to provide a thorough understanding of one or more aspects. However, it will also be recognized by one of ordinary skill in the art that such aspect(s) may be practiced without these specific details. The following description and accompanying drawings set forth in detail certain illustrative aspects of one or more aspects. These aspects are illustrative, however, and some of the various methods in principles of various aspects may be employed, and the descriptions set forth are intended to include all such aspects and their equivalents.

Further, various aspects and features will be presented by a system that may include a number of devices, components and/or modules, and the like. It is also noted that various systems may include additional devices, components, and/or modules, etc. and/or may not include all of the devices, components, modules, etc. discussed with respect to the drawings. must be understood and recognized.

As used herein, “embodiment”, “example”, “aspect”, “exemplary”, etc. may not be construed as an advantage or advantage in any aspect or design described above over other aspects or designs. . The terms 'component', 'module', 'system', 'interface', etc. used below generally mean a computer-related entity, for example, hardware, a combination of hardware and software, It could mean software.

In addition, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from context, "X employs A or B" is intended to mean one of the natural implicit substitutions. That is, X employs A; X employs B; or when X employs both A and B, "X employs A or B" may apply to either of these cases. It should also be understood that the term “and/or” as used herein refers to and includes all possible combinations of one or more of the listed related items.

Also, the terms "comprises" and/or "comprising" mean that the feature and/or element is present, but excludes the presence or addition of one or more other features, elements, and/or groups thereof. should be understood as not Also, unless otherwise specified or unless the context is clear as to designating a singular form, the singular in the specification and claims should generally be construed to mean "one or more." In addition, as used herein, the terms “client,” “user,” and “user” are often used interchangeably. In addition, as used herein, the terms "component" and "element" may also be used interchangeably sometimes.

Hereinafter, a virtual camera tracking system according to an embodiment of the present invention and a real-time MR environment synthesis system using the same will be described in more detail based on the accompanying drawings.

1 is a block diagram illustrating a virtual camera tracking system according to an embodiment of the present invention and a real-time MMR environment synthesis system using the same, and FIGS. 2 to 4 are the virtual camera tracking system shown in FIG. 1 and a real-time MMR environment using the same. It is an exemplary diagram explaining the configuration diagram of the connection between the synthesis systems and supporting functions.

1 to 4, the virtual camera tracking system 10 presented in the present invention is a depth camera module 11, a distance measuring scanner (RPLIDAR) 12, a tracking unit 13 and an image capturing unit 14. includes

The depth camera module 11 , the tracking unit 13 and the distance measuring scanner 12 are mounted on the image capturing unit 14 .

The depth camera module 11 recognizes position (coordinate) information of a photographing operation of the image photographing unit 14, which will be described later.

The distance measuring scanner 12 functions to scan an object located in all directions in a clockwise direction by outputting infrared rays in all directions around 360 degrees.

The tracking unit 13 performs a function of tracking the shooting operation sequence of the video camera.

The image capturing unit 14 may be a camera that captures a surrounding image.

Here, the image capturing unit 14 is a camera equipped with 4K 60F Unreal Engine 4, and the above-described absolute coordinate value engine acquisition indoor/outdoor algorithm of Unreal Engine 4, Unreal Engine 4 height value acquisition algorithm, and Unreal Engine 4 rotation value acquisition Algorithms may be mounted.

Here, the indoor algorithm is linked with the distance measuring scanner, the outdoor algorithm is linked with the coordinate recognition module, and the height value acquisition algorithm is linked with the IR sensor.

In addition, the rotation value acquisition algorithm may be linked with the gyro sensor of the user terminal (tablet PC).

Next, the MRF environment real-time synthesis system 20 is provided with the absolute coordinate values, height, rotation values, etc. of the captured image captured by the virtual camera tracking system 100 and the captured image captured indoors/outdoors, and within the captured image. It performs the function of synthesizing MR images.

Meanwhile, the virtual camera tracking system 10 disclosed herein may provide a management interface (UX/UI) for managing output data of each configuration.

5 is a diagram illustrating an example computing environment in which one or more embodiments disclosed herein may be implemented, and is an illustration of a system 1000 including a computing device 1100 configured to implement one or more embodiments described above. shows For example, computing device 1100 may be a personal computer, server computer, handheld or laptop device, mobile device (mobile phone, PDA, media player, etc.), multiprocessor system, consumer electronics, minicomputer, mainframe computer, distributed computing environments including any of the aforementioned systems or devices, and the like.

The computing device 1100 may include at least one processing unit 1110 and a memory 1120 . Here, the processing unit 1110 may include, for example, a central processing unit (CPU), a graphic processing unit (GPU), a microprocessor, an Application Specific Integrated Circuit (ASIC), Field Programmable Gate Arrays (FPGA), etc. and may have a plurality of cores. The memory 1120 may be a volatile memory (eg, RAM, etc.), a non-volatile memory (eg, ROM, flash memory, etc.), or a combination thereof. Additionally, computing device 1100 may include additional storage 1130 . Storage 1130 includes, but is not limited to, magnetic storage, optical storage, and the like. The storage 1130 may store computer readable instructions for implementing one or more embodiments disclosed herein, and other computer readable instructions for implementing an operating system, an application program, and the like. Computer readable instructions stored in storage 1130 may be loaded into memory 1120 for execution by processing unit 1110 . Computing device 1100 may also include input device(s) 1140 and output device(s) 1150 .

Here, the input device(s) 1140 may include, for example, a keyboard, mouse, pen, voice input device, touch input device, infrared camera, video input device, or any other input device, or the like. Further, the output device(s) 1150 may include, for example, one or more displays, speakers, printers, or any other output device, or the like. Also, the computing device 1100 may use an input device or an output device included in another computing device as the input device(s) 1140 or the output device(s) 1150 .

Computing device 1100 may also include communication connection(s) 1160 that enable computing device 1100 to communicate with another device (eg, computing device 1300 ).

Here, communication connection(s) 1160 may be a modem, network interface card (NIC), integrated network interface, radio frequency transmitter/receiver, infrared port, USB connection, or other for connecting computing device 1100 to another computing device. It may include interfaces. Also, the communication connection(s) 1160 may include a wired connection or a wireless connection. Each component of the aforementioned computing device 1100 may be connected by various interconnections such as a bus (eg, peripheral component interconnection (PCI), USB, firmware (IEEE 1394), optical bus structure, etc.) and may be interconnected by a network 1200 . As used herein, terms such as "component," "system," and the like, generally refer to a computer-related entity that is hardware, a combination of hardware and software, software, or software in execution.

For example, a component can be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. For example, both an application running on a controller and a controller may be a component. One or more components may reside within a process and/or thread of execution, and components may be localized on one computer or distributed between two or more computers.

Although specific embodiments according to the present invention have been described so far, various modifications are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined by the claims described below as well as the claims and equivalents.

As described above, although the present invention has been described with reference to limited embodiments and drawings, the present invention is not limited to the above-described embodiments, which are various modifications and variations from these descriptions for those of ordinary skill in the art to which the present invention pertains. Transformation is possible. Accordingly, the spirit of the present invention should be understood only by the claims described below, and all equivalents or equivalent modifications thereof will fall within the scope of the spirit of the present invention.

10: Virtual Camera Tracking System
20: MMR environment real-time synthesis system

Claims (2)

Including a depth camera module, a distance measuring scanner (RPLIDAR), a tracking unit and an image capturing unit,
The depth camera module, the tracking unit and the distance measuring scanner are mounted on the image capturing unit,
The depth camera module recognizes the position (coordinate) information of the photographing operation of the image capturing unit, and the distance measuring scanner scans the object located in the clockwise direction by outputting infrared rays in all directions around 360 degrees, and the tracking The unit tracks the shooting operation sequence of the image capturing unit, and the image capturing unit is a camera that captures surrounding images. An acquisition algorithm is mounted, the indoor algorithm is linked with a distance measuring scanner, the outdoor algorithm is linked with a coordinate recognition module, the height value acquisition algorithm is linked with an IR sensor, and the rotation value acquisition algorithm is a gyro of the user terminal linked to the sensor,
Virtual camera tracking system, characterized in that. A real-time MMR environment characterized by receiving the absolute coordinate values, height, rotation values, etc. of the captured image captured in the virtual camera tracking system of claim 1 and the indoor/outdoor captured image, and synthesizing the MR image within the captured image. synthetic system.

Images