KR20240007591A - Method for estimating positioning information of multiple stereo camera devices based on 6 dof and slam, device and program - Google Patents

Abstract

A computing device for providing a method of operating a complex sensor-based multi-tracking camera system in a virtual studio is disclosed. The computing device may include a processor including one or more cores and a memory that stores instructions executable by the processor. With the provision of this device, a tracking camera system in a virtual studio can be installed inexpensively and effectively.

Description

Method, device, and program for estimating positioning information of multiple stereo camera devices based on 6 DOF and SLAM {METHOD FOR ESTIMATING POSITIONING INFORMATION OF MULTIPLE STEREO CAMERA DEVICES BASED ON 6 DOF AND SLAM, DEVICE AND PROGRAM}

This disclosure relates to a method, device, and program for estimating positioning information of a plurality of stereo camera devices based on 6 DOF and SLAM.

In virtual studio and virtual production-based extended reality content production, a camera tracking system is essential to match the position of the virtual camera in the virtual space with the position in the real space.

The existing camera tracking system is a technology that estimates the position of the camera by detecting the position of a marker attached to the virtual studio space through a special tracking device attached to the camera in the space inside the virtual studio, or tracking optics inside the virtual studio. We installed a camera, installed a special marker that the tracking camera can detect on a separate filming camera, and applied technology that can calculate the camera's position through the position of the marker within the virtual studio space.

The method of optically detecting external markers through a tracking device attached to a camera is to attach a number of recognizable markers to the ceiling or floor depending on the size of the virtual studio space and track the camera through them. It takes a considerable amount of time for the initial installation to attach and calibrate as many markers as possible, and there is always a possibility of errors occurring in the precision of camera tracking depending on whether skilled personnel performs the installation and calibration work.

In addition, the method of installing multiple tracking optical cameras in a virtual studio and tracking the position through special markers costs a lot of money because the tracking optical cameras must be installed in proportion to the size of the virtual studio, and the tracking optical cameras are vulnerable to vibration. Due to the characteristics of the camera, it is essential to install a structure to attach a tracking device and to prevent vibration. For this reason, the above method requires a lot of cost and time for installation.

In addition, in case of previous installation due to structural changes or expansion of the virtual studio, a reinstallation process is required in the same way as the initial installation, and the resulting change cost is close to the initial installation cost. In addition, it has the disadvantage of being difficult to apply to a mobile tracking system because it requires the same amount of time for calibration and stabilization as the initial installation.

Accordingly, a cheaper and more effective method of overcoming the limitations of the prior art is needed.

Korean Patent No. 10-2121287 (Registration Date: 2020.06.04)

The purpose of the embodiment disclosed in the present disclosure is to provide a complex sensor-based multi-tracking camera system in a virtual studio by applying a relatively inexpensive tracking camera.

In addition, the purpose of the embodiment disclosed in the present disclosure is to provide a method for inexpensively and adaptively matching the position between virtual space and real space even if a change in the structure of space occurs.

The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood by those skilled in the art from the description below.

A method of operating a complex sensor-based multi-tracking camera system in a virtual studio according to an aspect of the present disclosure for achieving the above-described technical problem is to use a plurality of stereo camera devices to center the virtual studio around an origin placed in the virtual studio. When being photographed, receiving captured images from the plurality of stereo camera devices; Based on the received captured image, generating a 3D space based on the origin; estimating positioning information including spatial information and rotation angles of the plurality of stereo camera devices in the generated 3D space; Synchronizing timing for receiving information from a plurality of stereo camera devices; And based on information received from the plurality of stereo camera devices, it may include updating positioning information for the plurality of camera devices in the 3D space.

Here, generating the 3D space includes recognizing a marker located at the origin; And it may include generating a 3D coordinate space centered on the origin based on the recognized marker.

Estimating the positioning information may include calculating spatial information and rotation angles of the plurality of stereo camera devices; Cross-verifying the calculated spatial information and rotation angle based on triangulation; and outputting spatial information and rotation angles of the cross-verified plurality of stereo camera devices.

The step of synchronizing the timing may include synchronizing the signal generation timing of the plurality of stereo camera devices with each other to prevent frame synchronization errors in data when synthesizing images.

The step of updating the positioning information includes receiving inertial measurement values collected by an inertial measurement sensor from the plurality of camera devices; and a step of first estimating positioning information of the plurality of camera devices based on 6 degrees of freedom (6 DOF).

Updating the positioning information may include calculating positioning information including spatial information and rotation angles of a plurality of stereo camera devices; and secondly estimating positioning information of the plurality of camera devices based on simultaneous localization and mapping (SLAM).

Additionally, the operating method may further include performing correction on the first and second estimated positioning information of the plurality of camera devices.

Additionally, the operating method includes, after performing the correction, generating raw data for the corrected positioning information of a plurality of camera devices; performing filter calibration on the generated raw data; calculating positioning information to which the filter calibration is applied; And it may further include performing image synthesis based on the calculated positioning information.

In addition, a computer program stored in a computer-readable storage medium according to one aspect of the present disclosure may be provided, where the computer program, when executed on one or more processors, performs a method of operating a complex sensor-based multi-tracking camera system of a virtual studio. The following operations are performed for the purpose of receiving captured images from the plurality of stereo camera devices when the virtual studio is photographed centered on an origin placed within the virtual studio using a plurality of stereo camera devices. action; Based on the received captured image, generating a 3D space based on the origin; Estimating positioning information including spatial information and rotation angles of the plurality of stereo camera devices in the generated 3D space; Synchronizing timing for receiving information from a plurality of stereo camera devices; and updating positioning information about the plurality of camera devices in the 3D space based on information received from the plurality of stereo camera devices.

In addition, a computing device for providing a method of operating a complex sensor-based multi-tracking camera system of a virtual studio according to an aspect of the present disclosure includes a processor including one or more cores; and a memory that stores instructions executable by the processor.

When the instructions are executed by the processor, the processor captures images from the plurality of stereo camera devices when the virtual studio is photographed centered on an origin placed within the virtual studio using a plurality of stereo camera devices. Receive, based on the received captured image, generate a 3D space based on the origin, estimate positioning information including spatial information and rotation angles of the plurality of stereo camera devices in the generated 3D space, and estimate a plurality of positioning information including rotation angles of the plurality of stereo camera devices. It may be configured to synchronize the timing for receiving information from the stereo camera devices and update positioning information for the plurality of camera devices in the 3D space based on the information received from the plurality of stereo camera devices.

In addition to this, a computer program stored in a computer-readable recording medium for execution to implement the present disclosure may be further provided.

In addition, a computer-readable recording medium recording a computer program for executing a method for implementing the present disclosure may be further provided.

According to the means for solving the above-described problem of the present disclosure, a complex sensor-based multi-tracking camera system of a virtual studio can be provided relatively inexpensively, and the location between the virtual space and the real space can be inexpensively and adaptively even if the structure of the space changes. The method of matching provides the effect provided.

The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned may be clearly understood by those skilled in the art from the description below.

1 is a diagram schematically illustrating a complex sensor-based multi-tracking camera system of a virtual studio according to an embodiment of the present invention;
Figure 2 is a relative block diagram showing the configuration of a computing device and a camera device constituting a complex sensor-based multi-tracking camera system of a virtual studio according to an embodiment of the present invention;
Figure 3 is a representative sequence diagram showing a method of operating a complex sensor-based multi-tracking camera system in a virtual studio according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a method for expressing a virtual studio in three-dimensional coordinates using a plurality of camera devices disposed in the virtual studio according to an embodiment of the present invention;
5 is a sequence diagram showing a method for generating a 3D coordinate space according to an embodiment of the present invention;
Figure 6 is a sequence diagram showing a method for determining 3D spatial coordinates and rotation angles of a plurality of camera devices according to an embodiment of the present invention, and
Figure 7 is a sequence diagram showing a method of updating positioning information of a plurality of camera devices according to an embodiment of the present invention.

Like reference numerals refer to like elements throughout this disclosure. The present disclosure does not describe all elements of the embodiments, and general content or overlapping content between the embodiments in the technical field to which the present disclosure pertains is omitted. The term 'unit, module, member, block' used in the specification may be implemented as software or hardware, and depending on the embodiment, a plurality of 'unit, module, member, block' may be implemented as a single component, or It is also possible for one 'part, module, member, or block' to include multiple components.

Throughout the specification, when a part is said to be “connected” to another part, this includes not only direct connection but also indirect connection, and indirect connection includes connection through a wireless communication network. do.

Additionally, when a part is said to “include” a certain component, this means that it may further include other components, rather than excluding other components, unless specifically stated to the contrary.

Throughout the specification, when a member is said to be located “on” another member, this includes not only cases where a member is in contact with another member, but also cases where another member exists between the two members.

Terms such as first and second are used to distinguish one component from another component, and the components are not limited by the above-mentioned terms.

Singular expressions include plural expressions unless the context clearly makes an exception.

The identification code for each step is used for convenience of explanation. The identification code does not explain the order of each step, and each step may be performed differently from the specified order unless a specific order is clearly stated in the context. there is.

Hereinafter, the operating principle and embodiments of the present disclosure will be described with reference to the attached drawings.

FIG. 1 is a diagram schematically illustrating a complex sensor-based multi-tracking camera system 1 (hereinafter referred to as “system”) of a virtual studio according to an embodiment of the present invention.

The system 1 may include a plurality of camera devices C, a computing device 100, an image synthesis device 200, etc.

Each of the plurality of camera devices (C) (C1 to CN) is a computing device that collects data using a stereoscopic camera equipped with a plurality of lenses, an inertial measurement sensor (e.g., an IMU sensor), and a depth measurement sensor. It can be provided as (100).

Each of the plurality of camera devices C may include one or more optical cameras for photographing a subject in a virtual studio and a tracking module attached thereto to track the posture and position of the camera. The plurality of camera devices C may be installed on a shooting fixture, a shooting crane, a rail, etc., but the embodiment is not limited thereto.

The computing device 100 is a device capable of performing computer calculations by receiving various data (captured image data, inertial measurement data, depth measurement data, etc.) from a plurality of camera devices (C). When a virtual studio is captured, a three-dimensional (3D) space can be created in the virtual studio based on images taken from multiple directions.

The computing device 100 may include an Open Sound Control (OSC) protocol to acquire data at the same cycle from a plurality of camera devices (C), and each of the plurality of camera devices (C) (C1 to CN) After being searched, data can be sent and received (GET request, POST request, etc.) through the network.

The computing device 100 can create a 3D space centered on the origin, which is the reference point of the virtual studio. When receiving captured images from a plurality of camera devices C, the computing device 100 receives the captured images centered on the origin, and receives the received captured images. By recombining images in three dimensions, a 3D space can be created.

The computing device 100 provides spatial information and rotation angles for the plurality of camera devices C in 3D space based on captured images, inertial measurements, depth measurements, etc. collected from the plurality of camera devices C. Information (positioning information) can be updated.

The computing device 100 may transmit a captured image to the image synthesis device 200, and the image synthesis device 200 may synthesize the received image. The image synthesis device 200 may receive 6 DOF 3D correction data transmitted through the output interface of the computing device 100 and synthesize the background or 3D asset and the image signal received from one or more camera devices. Depending on the implementation example, the computing device 100 and the image synthesis device 200 may be the same device.

In this specification, the 'computing device 100 according to the present disclosure' includes all various devices that can perform computational processing and provide results to the user. For example, the computing device according to the present disclosure may include all of a computer, a server device, and a portable terminal, or may take the form of any one.

Here, the computer may include, for example, a laptop equipped with a web browser, a desktop, a laptop, a tablet PC, a slate PC, etc.

The server device is a server that processes information by communicating with external devices, and may include an application server, computing server, database server, file server, game server, mail server, proxy server, and web server.

The portable terminal is, for example, a wireless communication device that guarantees portability and mobility, such as PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), and PDA. (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminal, smart phone ), all types of handheld wireless communication devices, and wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted-device (HMD). may include.

Figure 2 is a relative block diagram showing the configuration of the computing device 100 and the camera device (C1, 300) that constitute the system 1 according to an embodiment of the present invention.

First, the camera device C1 (300) may include a communication unit 310, a stereo camera 320, an inertial measurement sensor 330 (IMU), and a depth measurement sensor 340.

The components shown in FIG. 2 are not essential for implementing the camera device 300 according to the present disclosure, so the camera device 300 described herein has more or fewer components than the components listed above. It can have elements.

The communication unit 310 may include one or more components that enable communication with an external device (e.g., computing device 100), for example, a broadcast reception module, a wired communication module, a wireless communication module, and short-range communication. It may include at least one of a module and a location information module.

The stereo camera 320 is a type of input unit and may include a plurality of cameras to generate a three-dimensional image.

The stereo camera 320 processes image frames, such as still images or moving images, obtained by an image sensor in shooting mode. The processed image frame may be displayed on the screen of the computing device 100 of the present disclosure or stored in memory.

An inertial measurement unit (IMU) 330 may be a sensor that uses a combination of an accelerometer, a tachometer, and sometimes a magnetometer to measure and report specific forces, angular rates, and sometimes magnetic fields.

The depth measurement sensor 340 may be used to generate a three-dimensional image by measuring the depth of an object in an image, and may be implemented based on infrared rays, but the embodiment is not limited thereto.

Next, the computing device 100 is a device for providing a method of operating a complex sensor-based multi-tracking camera system of a virtual studio, and includes a communication unit 110, an input unit 120, a display 130, a memory 150, and one It may include a processor 190 including more than one core.

The components shown in FIG. 2 are not essential for implementing the computing device 100 according to the present disclosure, so the computing device 100 described herein may have more or fewer components than the components listed above. It can have elements.

The communication unit 110 may include one or more components that enable communication with an external device, for example, at least one of a broadcast reception module, a wired communication module, a wireless communication module, a short-range communication module, and a location information module. It can be included.

The input unit 120 is for inputting image information (or signal), audio information (or signal), data, or information input from a user, and includes at least one of at least one camera, at least one microphone, and a user input unit. can do. Voice data or image data collected by the input unit 120 may be analyzed and processed as a user's control command.

The microphone processes external acoustic signals into electrical voice data. Processed voice data can be used in a variety of ways depending on the function being performed (or the application being executed) on the device. Meanwhile, various noise removal algorithms can be implemented in the microphone to remove noise generated in the process of receiving an external acoustic signal.

The user input unit is for receiving information from the user. When information is input through the user input unit, the control unit can control the operation of the device to correspond to the input information. This user input unit uses hardware-type physical keys (e.g., buttons, dome switches, jog wheels, jog switches, etc. located on at least one of the front, back, and sides of the device) and software-type touch keys. It can be included. As an example, the touch key consists of a virtual key, soft key, or visual key displayed on a touch screen-type display unit through software processing, or is displayed on the touch screen. It may be composed of touch keys placed in other parts. Meanwhile, the virtual key or visual key can be displayed on the touch screen in various forms, for example, graphic, text, icon, video or these. It can be made up of a combination of .

The display 130 displays (outputs) information processed by the device. For example, the display 130 displays execution screen information of an application (for example, an application) running on the device 100, or UI (User Interface) and GUI (Graphic User Interface) information according to this execution screen information. can be displayed.

The memory 150 can store data supporting various functions of the device and a program for the operation of the processor 190, and input/output data (e.g., music files, still images, videos, etc.) can be stored, and a number of application programs (application programs or applications) running on the device, data for operation of the device, and commands can be stored. At least some of these applications may be downloaded from an external server via wireless communication.

These memories include flash memory type, hard disk type, SSD type (Solid State Disk type), SDD type (Silicon Disk Drive type), and multimedia card micro type. , card-type memory (e.g., SD or It may include at least one type of storage medium among (only memory), PROM (programmable read-only memory), magnetic memory, magnetic disk, and optical disk. Additionally, the memory may be a database that is separate from the device, but is connected wired or wirelessly.

The processor 190 includes a memory that stores data for an algorithm for controlling the operation of components in the device or a program that reproduces the algorithm, and at least one processor that performs the above-described operations using the data stored in the memory ( (not shown) may be included. At this time, the memory and processor may each be implemented as separate chips. Alternatively, the memory and processor may be implemented as a single chip.

In addition, the processor 190 may control any one or a combination of the above-described components in order to implement various embodiments according to the present disclosure described in FIGS. 3 to 7 below on the present device.

The image synthesis device 200 may be implemented separately from the computing device 100, or may match spatial images of a virtual studio taken from various directions and map them to three-dimensional space (coordinates).

At least one component may be added or deleted in response to the performance of the components shown in FIG. 2. Additionally, it will be easily understood by those skilled in the art that the mutual positions of the components may be changed in response to the performance or structure of the system.

Meanwhile, each component shown in FIG. 2 refers to software and/or hardware components such as Field Programmable Gate Array (FPGA) and Application Specific Integrated Circuit (ASIC).

Figure 3 is a sequence diagram showing the operation method of the complex sensor-based multi-tracking camera system 1 of the virtual studio according to an embodiment of the present invention, and instructions (instrictions) are executed by the processor 190 of the computing device 100. It can be executed. FIG. 4 is a diagram illustrating a method for expressing a virtual studio in 3D coordinates using a plurality of camera devices (C1 to C3) disposed in the virtual studio according to an embodiment of the present invention, and FIG. 5 is a diagram illustrating a method of expressing the virtual studio in 3D coordinates according to an embodiment of the present invention. It is a diagram showing a method of generating a 3D coordinate space according to an embodiment of the present invention, and FIG. 6 is a sequence diagram showing a method of determining the 3D space coordinates and rotation angles of a plurality of camera devices according to an embodiment of the present invention. Figure 7 is a sequence diagram showing a method of updating positioning information of a plurality of camera devices according to an embodiment of the present invention. While explaining FIG. 3, FIGS. 4 to 7 will be referred to where necessary.

First, the processor 190 may receive captured images from a plurality of stereo camera devices when the virtual studio is photographed centered on an origin placed within the virtual studio using a plurality of stereo camera devices (S310).

Here, a marker (eg, a tag) is placed at the origin to help capture and origin recognition of a plurality of camera devices. Various marker displays such as QR code and AprilTag may be used as the marker, but the embodiment is not limited thereto.

Referring to FIG. 4, each of a plurality of camera devices (C1 to C3) capable of filming a virtual studio may have its own shooting range (C1C to C3C), centered on a marker (MK) placed at the origin of the virtual studio. You can shoot.

The plurality of camera devices C1 to C3 may use inertial measurement sensors to provide rotation angle information and depth measurement values of each device to the computing device 100 through the communication unit.

After step S310, the processor 190 may generate a 3D space based on the origin based on the received captured image (S320).

Referring to FIG. 5, the processor 190 may recognize a tag located at the origin (S321) and create a 3D coordinate space centered on the origin based on the recognized marker (S323).

At this time, the processor 190 may receive positioning information of the plurality of camera devices C1 to C3 together with the image images. Positioning information may include spatial information, rotation angle, depth measurement, etc., but the embodiment is not limited thereto.

After step S320, the processor 190 may estimate positioning information including spatial information and rotation angles of a plurality of stereo camera devices in the generated 3D space (S330).

Referring to FIG. 6, the processor 190 may calculate 3D coordinates and rotation angles of each of the plurality of camera devices (S331).

Next, the processor 190 may cross-verify the 3D coordinates and rotation angles of the plurality of camera devices based on triangulation (S333).

Afterwards, the processor 190 may return (determine) the 3D space coordinates and rotation angles of the plurality of camera devices after completing verification (S335).

However, in step S330, the processor 190 represents the virtual studio in 3D space based on the initial origin, and after the initial viewpoint, based on differences in positioning information of the camera device, changes in the environment, etc., the processor 190 displays the virtual studio in real time. The corresponding 3D space coordinates can be updated.

To this end, the processor 190 can synchronize the timing for receiving information from a plurality of stereo camera devices (S340).

The processor 190 can synchronize the information acquisition time of a plurality of camera devices based on the Open Sound Control (OSC) protocol, and for this purpose, an OSC server and an OSC client can be used.

In this way, the processor 190 can prevent frame synchronization errors in data when synthesizing images by synchronizing the signal generation timing of a plurality of stereo camera devices.

After step S340, the processor 190 may update positioning information about the plurality of camera devices in the 3D space based on information received from the plurality of stereo camera devices (S350).

Referring to FIG. 7, the processor 190 receives inertial measurement values collected by an inertial measurement sensor from the plurality of camera devices (S351), and receives a plurality of inertial measurement values based on 6 degrees of freedom (6DOF). Positioning information of the camera device may be first estimated (S352). Here, 6 DOF refers to the six movement directions of the aircraft, and can reflect all three axes (XYZ) directions and rotation directions (roll, pitch, yaw, etc.) in each axis.

The processor 190 may calculate positioning information including spatial information and rotation angles of a plurality of stereo camera devices (S353), and may second estimate the positioning information of the plurality of camera devices based on SLAM (S354).

Next, the processor 190 may perform correction on the first and second estimated positioning information of the plurality of camera devices (S355).

The processor 190 may perform the Sensor Fusion algorithm to derive coordinate data and rotation angle in 3D space. At this time, optimization can be performed through the visual-inertial odometry method, and mechanical errors due to data errors due to external environments such as light or magnetic fields among sensors or failure of specific sensors can be compensated.

In an optional embodiment, if the difference between the first estimated and second estimated positioning information of the plurality of camera devices occurs beyond a predetermined range, the processor 190 does not trust the estimated positioning information and uses 6 DOF and SLAM-based Positioning information can be re-estimated.

In an optional embodiment, the processor 190 may re-estimate the positioning information only for the specific camera device when the estimated value is different only in the positioning information for the specific camera device among the first estimated and second estimated positioning information for the plurality of camera devices. You can. If there is a difference in the re-estimated value that exceeds a predetermined range, the relevant information can be provided as a message to the administrator's terminal.

The processor 190 generates raw data for the positioning information of a plurality of calibrated camera devices (S356), performs filter calibration on the generated raw data (S357), and calculates positioning information to which the filter calibration is applied. (S358).

The processor 190 can prevent the jitter phenomenon by executing the Linear Kalman Filter Calibration algorithm. At this time, a specific pre-measured correction coefficient can be applied depending on the external environment, so filtering and correction values suitable for various studio environments can be applied to the algorithm.

The processor 190 may perform image synthesis based on positioning information calculated by itself. However, depending on the embodiment, the processor 190 may transmit the calculated positioning information and the captured image through the communication unit 110 to the image synthesis device, and image synthesis may be performed by the image synthesis device (S410).

Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium that stores instructions executable by a computer. Instructions may be stored in the form of program code, and when executed by a processor, may create program modules to perform operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

Computer-readable recording media include all types of recording media storing instructions that can be decoded by a computer. For example, there may be Read Only Memory (ROM), Random Access Memory (RAM), magnetic tape, magnetic disk, flash memory, optical data storage device, etc.

As described above, the disclosed embodiments have been described with reference to the attached drawings. A person skilled in the art to which this disclosure pertains will understand that the present disclosure may be practiced in forms different from the disclosed embodiments without changing the technical idea or essential features of the present disclosure. The disclosed embodiments are illustrative and should not be construed as limiting.

100: Computing device for providing a method of operating a complex sensor-based multi-tracking camera system in a virtual studio

Claims (10)

6 In the method of estimating positioning information of a plurality of stereo camera devices based on DOF and SLAM,
When the virtual studio is captured using the plurality of stereo camera devices centered on an origin placed within the virtual studio, receiving captured images from the plurality of stereo camera devices;
Based on the received captured image, generating a 3D space based on the origin;
estimating positioning information including spatial information and rotation angles of the plurality of stereo camera devices in the generated 3D space; and
Synchronizing timing for receiving information from the plurality of stereo camera devices; Including,
First estimating positioning information of the plurality of stereo camera devices based on 6 degrees of freedom (6 DOF); Including,
Second estimating positioning information of the plurality of stereo camera devices based on simultaneous localization and mapping (SLAM); Including,
When the difference between the first estimate and the second estimated positioning information of the plurality of stereo camera devices exceeds a preset range, the positioning information is re-estimated based on the 6 DOF and the SLAM, method. According to paragraph 1,
The step of generating the 3D space is,
Recognizing a marker located at the origin; and
generating a 3D coordinate space centered on the origin based on the recognized marker; Method, including. According to paragraph 2,
The six degrees of freedom include the X-axis direction, Y-axis direction, Z-axis direction, and rotation directions in each axis, such as roll, pitch, and yaw. According to paragraph 3,
The step of synchronizing the timing is characterized in that the information acquisition time of the plurality of stereo camera devices is synchronized based on the Open Sound Control (OSC) protocol. According to paragraph 4,
Generating raw data for positioning information of the plurality of stereo camera devices;
performing filter calibration on the generated raw data;
calculating positioning information to which the filter calibration is applied; and
performing image synthesis based on the calculated positioning information; A method further comprising: A computer program stored in a computer-readable storage medium, wherein the computer program, when executed on one or more processors, performs the following operations for performing a method for estimating positioning information of a plurality of stereo camera devices based on 6 DOF and SLAM, The above operations are:
When the virtual studio is captured using the plurality of stereo camera devices centered on an origin placed within the virtual studio, receiving captured images from the plurality of stereo camera devices;
An operation of generating a 3D space based on the origin based on the received captured image;
estimating positioning information including spatial information and rotation angles of the plurality of stereo camera devices in the generated 3D space; and
synchronizing timing for receiving information from the plurality of stereo camera devices; Including,
Comprising an operation of first estimating positioning information of the plurality of stereo camera devices based on 6 degrees of freedom (6 DOF),
An operation of secondly estimating positioning information of the plurality of stereo camera devices based on simultaneous localization and mapping (SLAM),
When the difference between the first estimate and the second estimated positioning information of the plurality of stereo camera devices occurs beyond a preset range, an operation of re-estimating the positioning information based on the 6 DOF and the SLAM, computer program. According to clause 6,
The operation of creating the 3D space is,
Recognizing a marker located at the origin; and
An operation of generating a 3D coordinate space centered on the origin based on the recognized marker; A computer program, including. In clause 7,
The six degrees of freedom include the According to clause 8,
The operation of synchronizing the timing is a computer program, characterized in that the information acquisition time of the plurality of stereo camera devices is synchronized based on the OSC (Open Sound Control) protocol. According to clause 9,
Generating raw data for positioning information of the plurality of stereo camera devices;
An operation of performing filter calibration on the generated raw data;
An operation of calculating positioning information to which the filter calibration is applied; and
An operation of performing image synthesis based on the calculated positioning information; A computer program further comprising:

Images