KR102549811B1 - Method for correcting positioning information using a plurality of stereo camera devices, computer program - Google Patents

Abstract

A computing device for providing a method of operating a multi-sensor-based multi-tracking camera system of a virtual studio is disclosed. The computing device may include a processor including one or more cores and a memory storing instructions executable by the processor. By providing this device, a tracking camera system of a virtual studio can be installed inexpensively and effectively.

Description

Correction method for positioning information using a plurality of stereo camera devices, computer program

The present disclosure relates to a multi-sensor-based multi-tracking camera system deployed in a virtual studio and an operation method thereof.

A camera tracking system is essential for matching the position of a virtual camera in a virtual space with that of a real space in the production of virtual studio and virtual production-based augmented reality content.

Existing camera tracking systems detect the location of a marker attached to the virtual studio space through a special tracking device attached to the camera in the space inside the virtual studio and estimate the position of the camera, or optical tracking inside the virtual studio. A camera was installed, and a special marker that the tracking camera could detect was installed in a separate shooting camera, and a technology that could calculate the location of the camera through the location of the marker inside the virtual studio space was applied.

The method of optically detecting external markers through the tracking device attached to the camera is to attach a number of recognizable markers to the ceiling or floor according to the size of the virtual studio space and track the camera through them. Significant time is required for the initial installation of attaching and calibrating a large number of markers of possible size, and there is a possibility of error in the accuracy of camera tracking depending on whether skilled personnel perform the installation and calibration work.

In addition, the method of installing multiple tracking optical cameras in a virtual studio and tracking the position through special markers is costly because the tracking optical cameras must be installed in proportion to the size of the virtual studio, and the tracking optics are vulnerable to vibration. Due to the characteristics of the camera, it is essential to install a structure for attaching the tracking device and a structure for preventing vibration. For this reason, the above method requires a lot of cost and time for installation.

In addition, in the case of previous installation due to structural change or expansion of the virtual studio, a reinstallation process is required in the same way as the initial installation, and the cost of the change due to this is close to the initial installation cost. In addition, there is a disadvantage in that it is difficult to apply to a mobile tracking system because it requires the same time as the initial installation for calibration and stabilization.

Accordingly, there is a need for a method of overcoming the limitations of the prior art more inexpensively and effectively.

Korean Patent Registration No. 10-2121287 (registration date: 2020.06.04)

An object of the present disclosure is to provide a complex sensor-based multi-tracking camera system of a virtual studio by applying a relatively inexpensive tracking camera.

In addition, an object of the embodiments disclosed in the present disclosure is to provide a method for matching a location between a virtual space and a real space inexpensively and adaptively even when a space structure is changed.

The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

An operating method of a multi-sensor-based multi-tracking camera system of a virtual studio according to an aspect of the present disclosure for achieving the above-described technical problem is to use a plurality of stereo camera devices around an origin disposed in the virtual studio. receiving a photographed image from the plurality of stereo camera devices when is photographed; generating a 3D space based on the origin based on the received captured image; estimating positioning information including spatial information and rotation angles of the plurality of stereo camera devices in the generated 3D space; synchronizing timing for receiving information from a plurality of stereo camera devices; and updating positioning information of a plurality of camera devices in the “3D” space based on information received from the plurality of stereo camera devices.

Here, the generating of the 3D space may include recognizing a marker located at the origin; and generating a 3D coordinate space centered on the origin based on the recognized marker.

The estimating of the positioning information may include calculating spatial information and rotation angles of the plurality of stereo camera devices; cross-verifying the calculated spatial information and rotation angle based on triangulation; and outputting the cross-verified spatial information and rotation angle of the plurality of stereo camera devices.

Synchronizing the timings may include synchronizing signal generation timings of the plurality of stereo camera devices with each other to prevent a frame sync error of data during image synthesis.

The updating of the positioning information may include receiving an inertial measurement value collected by an inertial measurement sensor from the plurality of camera devices; and first estimating positioning information of the plurality of camera devices based on 6 degrees of freedom (6 DOF).

The updating of the positioning information may include calculating positioning information including spatial information and rotation angles of a plurality of stereo camera devices; and second estimating positioning information of the plurality of camera devices based on simultaneous localization and mapping (SLAM).

In addition, the operating method may further include correcting the first estimated and second estimated positioning information of the plurality of camera devices.

In addition, the operating method may include, after the performing of the correction, generating raw data for the corrected positioning information of the plurality of camera devices; performing filter calibration on the generated raw data; Calculating positioning information to which the filter calibration is applied; and performing image synthesis based on the calculated positioning information.

In addition, a computer program stored in a computer readable storage medium according to an aspect of the present disclosure may be provided. When the computer program is executed on one or more processors, the method of operating the multi-sensor-based multi-tracking camera system of the virtual studio may be performed. In order to perform the following operations, the operations are, when the virtual studio is photographed around the origin disposed in the virtual studio using a plurality of stereo camera devices, the captured image is received from the plurality of stereo camera devices action; generating a 3D space based on the origin based on the received captured image; estimating positioning information including spatial information and rotation angles of the plurality of stereo camera devices in the generated 3D space; synchronizing timing for receiving information from a plurality of stereo camera devices; and updating positioning information of a plurality of camera devices in the “3D” space based on information received from the plurality of stereo camera devices.

In addition, a computing device for providing a method of operating a multi-sensor-based multi-tracking camera system of a virtual studio according to an aspect of the present disclosure includes a processor including one or more cores; and a memory for storing instructions executable by the processor.

When the instructions are executed by the processor, the processor, when the virtual studio is captured using a plurality of stereo camera devices around the origin disposed in the virtual studio, captures images from the plurality of stereo camera devices. receiving, based on the received captured image, generating a 3D space based on the origin, estimating positioning information including space information and rotation angles of the plurality of stereo camera devices in the generated 3D space, and plural It may be configured to synchronize timing for receiving information from the stereo camera devices of the , and update positioning information for the plurality of camera devices on the 3D space based on the information received from the plurality of stereo camera devices.

In addition to this, a computer program stored in a computer readable recording medium for execution to implement the present disclosure may be further provided.

In addition to this, a computer readable recording medium recording a computer program for executing a method for implementing the present disclosure may be further provided.

According to the above-mentioned problem-solving means of the present disclosure, a multi-sensor-based multi-tracking camera system of a virtual studio can be provided at a relatively low cost, and even if the structure of the space is changed, the position between the virtual space and the real space can be inexpensively and adaptively Provides the effect provided by the method in which is matched.

The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a diagram for schematically explaining a multi-sensor-based multi-tracking camera system of a virtual studio according to an embodiment of the present invention;
2 is a relative block diagram showing the configuration of a computing device and a camera device constituting a complex sensor-based multi-tracking camera system of a virtual studio according to an embodiment of the present invention;
3 is a representative sequence diagram illustrating an operating method of a multi-sensor-based multi-tracking camera system of a virtual studio according to an embodiment of the present invention;
4 is a view for explaining a method for expressing a virtual studio in three-dimensional coordinates using a plurality of camera devices arranged in the virtual studio according to an embodiment of the present invention;
5 is a sequence diagram illustrating a method of generating a 3D coordinate space according to an embodiment of the present invention;
6 is a sequence diagram illustrating a method of determining 3D spatial coordinates and rotation angles of a plurality of camera devices according to an embodiment of the present invention;
7 is a sequence diagram illustrating a method of updating positioning information of a plurality of camera devices according to an embodiment of the present invention.

Like reference numbers designate like elements throughout this disclosure. The present disclosure does not describe all elements of the embodiments, and general content or overlapping content between the embodiments in the technical field to which the present disclosure belongs is omitted. The term 'unit, module, member, or block' used in the specification may be implemented as software or hardware, and according to embodiments, a plurality of 'units, modules, members, or blocks' may be implemented as one component, It is also possible that one 'part, module, member, block' includes a plurality of components.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case of being directly connected but also the case of being indirectly connected, and indirect connection includes being connected through a wireless communication network. do.

In addition, when a certain component is said to "include", this means that it may further include other components without excluding other components unless otherwise stated.

Throughout the specification, when a member is said to be located “on” another member, this includes not only a case where a member is in contact with another member, but also a case where another member exists between the two members.

Terms such as first and second are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

Expressions in the singular number include plural expressions unless the context clearly dictates otherwise.

In each step, the identification code is used for convenience of description, and the identification code does not explain the order of each step, and each step may be performed in a different order from the specified order unless a specific order is clearly described in context. there is.

Hereinafter, the working principle and embodiments of the present disclosure will be described with reference to the accompanying drawings.

1 is a diagram schematically illustrating a multi-sensor-based multi-tracking camera system (1, hereinafter referred to as “system”) of a virtual studio according to an embodiment of the present invention.

The system 1 may include a plurality of camera devices C, a computing device 100 and an image synthesis device 200, and the like.

Each of the plurality of camera devices (C) (C1 to CN) transmits data collected using a stereoscopic camera equipped with a plurality of lenses, an inertial measurement sensor (eg, an IMU sensor), and a depth measurement sensor to a computing device. (100) can be provided.

Each of the plurality of camera devices C may include at least one optical camera that photographs a subject in the virtual studio and a tracking module attached to the optical camera that tracks the position and position of the camera. The plurality of camera devices C may be installed in a fixing device for photographing, a crane for photographing, a rail, etc., but the embodiment is not limited thereto.

The computing device 100 is a device capable of performing computer calculation by receiving various data (photographed image data, inertial measurement data, depth measurement data, etc.) from a plurality of camera devices C, and the plurality of camera devices C When the virtual studio is photographed by, it is possible to create a three-dimensional (3D) space of the virtual studio based on images photographed in various directions.

The computing device 100 may include an Open Sound Control (OSC) protocol to acquire data from the plurality of camera devices C at the same period, and each of the plurality of camera devices C (C1 to CN) After being retrieved, data can be sent and received (GET request, POST request, etc.) through the network.

The computing device 100 may create a 3D space centered on the origin, which is the reference point of the virtual studio, and when receiving captured images from a plurality of camera devices C, receive the captured images centered on the origin, and receive the received captured images. Images can be recombined on three dimensions to create a 3D space.

The computing device 100 obtains spatial information and rotation angle of the plurality of camera devices C in a 3D space based on captured images collected from the plurality of camera devices C, inertial measurement values, depth measurement values, and the like. Information (positioning information) can be updated.

The computing device 100 may transmit captured images to the video synthesizing device 200, and the video synthesizing device 200 may synthesize the received captured images. The image synthesis device 200 may receive 6 DOF 3D correction data transmitted through an output interface of the computing device 100 and synthesize a background or 3D asset and image signals received from one or more camera devices. Depending on implementation, the computing device 100 and the image synthesizing device 200 may be the same device.

In this specification, the 'computing device 100 according to the present disclosure' includes all of various devices capable of providing results to users by performing calculation processing. For example, a computing device according to the present disclosure may include a computer, a server device, and a portable terminal, or may be in any one form.

Here, the computer may include, for example, a laptop computer, a desktop computer, a laptop computer, a tablet PC, a slate PC, and the like equipped with a web browser.

The server device is a server that processes information by communicating with an external device, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

The portable terminal is, for example, a wireless communication device that ensures portability and mobility, and includes a Personal Communication System (PCS), a Global System for Mobile communications (GSM), a Personal Digital Cellular (PDC), a Personal Handyphone System (PHS), and a PDA. (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminal, smart phone ) and wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted-devices (HMDs). can include

2 is a relative block diagram showing the configuration of the computing device 100 and the camera devices C1 and 300 constituting the system 1 according to an embodiment of the present invention.

First, the camera device C1 or 300 may include a communication unit 310, a stereo camera 320, an inertial measurement sensor 330 (IMU), a depth measurement sensor 340, and the like.

The components shown in FIG. 2 are not essential to implement the camera device 300 according to the present disclosure, so the camera device 300 described in this specification has more or fewer components than the components listed above. can have elements.

The communication unit 310 may include one or more components enabling communication with an external device (eg, the computing device 100), for example, a broadcast reception module, a wired communication module, a wireless communication module, and short-distance communication. It may include at least one of a module and a location information module.

The stereo camera 320 is a type of input unit and may include a plurality of cameras to generate a stereoscopic image.

The stereo camera 320 processes an image frame such as a still image or a moving image obtained by an image sensor in a photographing mode. The processed image frame may be displayed on the computing device (screen of 100) of the present disclosure or stored in memory.

An inertial measurement unit (IMU) (330) may be a sensor that measures and reports a specific force, angular rate, and sometimes a magnetic field using a combination of an accelerometer, a tachometer, and sometimes a magnetometer.

The depth measurement sensor 340 may be used to generate a three-dimensional image by measuring the depth of an object in an image, and may be implemented based on infrared rays, but the embodiment is not limited thereto.

Next, the computing device 100 is a device for providing a method of operating a multi-sensor-based multi-tracking camera system of a virtual studio, and includes a communication unit 110, an input unit 120, a display 130, a memory 150, and one The processor 190 including the above cores may be included.

The components shown in FIG. 2 are not essential to implement the computing device 100 according to the present disclosure, so the computing device 100 described in this specification has more or fewer components than the components listed above. can have elements.

The communication unit 110 may include one or more components enabling communication with an external device, and for example, at least one of a broadcast reception module, a wired communication module, a wireless communication module, a short-distance communication module, and a location information module. can include

The input unit 120 is for inputting image information (or signals), audio information (or signals), data, or information input from a user, and includes at least one of at least one camera, at least one microphone, and a user input unit. can do. Voice data or image data collected by the input unit 120 may be analyzed and processed as a user's control command.

A microphone processes an external acoustic signal into electrical voice data. The processed voice data can be used in various ways according to the function (or application program being executed) being performed in the present device. Meanwhile, various noise cancellation algorithms for removing noise generated in the process of receiving an external sound signal may be implemented in the microphone.

The user input unit is for receiving information from a user, and when information is input through the user input unit, the control unit can control the operation of the device to correspond to the input information. The user input unit includes hardware physical keys (for example, a button located on at least one of the front, rear, and side surfaces of the device, a dome switch, a jog wheel, a jog switch, etc.) and a software touch key. can include As an example, the touch key is composed of a virtual key, soft key, or visual key displayed on a touch screen type display unit through software processing, or the touch screen It may be made of a touch key (touch key) disposed in a part other than the part. On the other hand, the virtual key or visual key can be displayed on the touch screen while having various forms, for example, graphic (graphic), text (text), icon (icon), video (video) or these can be made of a combination of

The display 130 displays (outputs) information processed by the present device. For example, the display 130 displays execution screen information of an application program (eg, application) driven in the device 100, or UI (User Interface) and GUI (Graphic User Interface) information according to such execution screen information. can be displayed.

The memory 150 may store data supporting various functions of the device and programs for operation of the processor 190, and input/output data (eg, music files, still images, moving images, etc.) and can store a plurality of application programs (application programs or applications) running in the device, data for operation of the device, and commands. At least some of these application programs may be downloaded from an external server through wireless communication.

These memories include a flash memory type, a hard disk type, a solid state disk type (SSD type), a silicon disk drive type (SDD type), and a multimedia card micro type. , card-type memory (for example, SD or XD memory, etc.), RAM (random access memory; RAM), SRAM (static random access memory), ROM (read-only memory; ROM), EEPROM (electrically erasable programmable read- only memory), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. In addition, the memory is separate from the present device, but may be a database connected by wire or wireless.

The processor 190 includes a memory for storing data for an algorithm or a program for reproducing the algorithm for controlling the operation of components in the device, and at least one processor for performing the above-described operation using the data stored in the memory ( not shown) may be included. In this case, the memory and the processor may be implemented as separate chips. Alternatively, the memory and the processor may be implemented as a single chip.

In addition, the processor 190 may control any one or a combination of a plurality of components described above in order to implement various embodiments according to the present disclosure described in FIGS. 3 to 7 below on the present device.

The image compositing device 200 may be implemented separately from the computing device 100 or map spatial images of a virtual studio photographed in various directions to a 3D space (coordinates) by matching each other.

At least one component may be added or deleted corresponding to the performance of the components shown in FIG. 2 . In addition, it will be easily understood by those skilled in the art that the mutual positions of the components may be changed corresponding to the performance or structure of the system.

Meanwhile, each component shown in FIG. 2 means software and/or hardware components such as a Field Programmable Gate Array (FPGA) and Application Specific Integrated Circuit (ASIC).

3 is a sequence diagram illustrating an operating method of a multi-sensor-based multi-tracking camera system 1 of a virtual studio according to an embodiment of the present invention, in which instructions are executed by the processor 190 of the computing device 100. can be executed 4 is a diagram for explaining a method for expressing a virtual studio in 3D coordinates using a plurality of camera devices (C1 to C3) arranged in the virtual studio according to an embodiment of the present invention, and FIG. 5 is a view for explaining the present invention. 6 is a sequence diagram showing a method for determining 3D space coordinates and rotation angles of a plurality of camera devices according to an embodiment of the present invention, 7 is a sequence diagram illustrating a method of updating positioning information of a plurality of camera devices according to an embodiment of the present invention. While describing FIG. 3 , FIGS. 4 to 7 will be referred to together where necessary.

First, the processor 190 may receive a photographed image from a plurality of stereo camera devices when a virtual studio is captured using a plurality of stereo camera devices around an origin disposed in the virtual studio (S310).

Here, a marker (eg, a tag) may be disposed at the origin to assist in capturing and recognizing the origin of a plurality of camera devices. As the marker, various marker displays such as QR code and AprilTag may be used, but the embodiment is not limited thereto.

Referring to FIG. 4 , each of the plurality of camera devices (C1 to C3) capable of shooting a virtual studio may have its own shooting range (C1C to C3C), centered on a marker (MK) disposed at the origin of the virtual studio. can be filmed

The plurality of camera devices C1 to C3 may provide rotation angle information, a depth measurement value, and the like of each device to the computing device 100 through a communication unit by using an inertial measurement sensor.

After the step S310, the processor 190 may generate a 3D space based on the origin based on the received photographed image (S320).

Referring to FIG. 5 , the processor 190 may recognize a tag located at the origin (S321) and create a 3D coordinate space centered on the origin based on the recognized marker (S323).

In this case, the processor 190 may receive positioning information of the plurality of camera devices C1 to C3 together with the image image. The positioning information may include spatial information, a rotation angle, a depth measurement value, and the like, but the embodiment is not limited thereto.

After step S320, the processor 190 may estimate positioning information including spatial information and rotation angles of a plurality of stereo camera devices in the generated 3D space (S330).

Referring to FIG. 6 , the processor 190 may calculate 3D coordinates and rotation angles of each of the plurality of camera devices (S331).

Next, the processor 190 may cross-verify the 3D coordinates and rotation angles of the plurality of camera devices based on triangulation (S333).

Thereafter, the processor 190 may return (determine) 3D space coordinates and rotation angles of the plurality of camera devices after verification is completed (S335).

However, in step S330, the processor 190 represents the virtual studio on the 3D space based on the initial origin, and after the initial viewpoint, based on the difference in positioning information of the camera device, change in the environment, etc., the virtual studio is displayed in real time. The corresponding 3D spatial coordinates can be updated.

To this end, the processor 190 may synchronize timing for receiving information from a plurality of stereo camera devices (S340).

The processor 190 may synchronize information acquisition times of a plurality of camera devices based on an Open Sound Control (OSC) protocol, and for this purpose, an OSC server and an OSC client may be used.

In this way, the processor 190 synchronizes signal generation timings of the plurality of stereo camera devices to prevent frame sync errors of data during image synthesis.

After step S340, the processor 190 may update positioning information for a plurality of camera devices on the "3D" space based on information received from a plurality of stereo camera devices (S350).

Referring to FIG. 7 , the processor 190 receives inertial measurement values collected by the inertial measurement sensor from the plurality of camera devices (S351), and based on 6 degrees of freedom (6DOF), a plurality of Positioning information of the camera device may be first estimated (S352). Here, 6 DOF means six motion directions of the aircraft, and can reflect all three axis (XYZ) directions and rotational directions (roll, pitch, yaw, etc.) in each axis.

The processor 190 may calculate positioning information including spatial information and rotation angles of the plurality of stereo camera devices (S353), and second estimate the positioning information of the plurality of camera devices based on SLAM (S354).

Next, the processor 190 may perform correction on the first estimated and second estimated positioning information of the plurality of camera devices (S355).

The processor 190 may derive coordinate data and a rotation angle in 3D space by performing a sensor fusion algorithm. At this time, optimization may be performed through the visual-inertial odometry method, and data errors due to external environments such as light or magnetic fields among sensors or mechanical errors due to failure of a specific sensor may be supplemented.

As an optional embodiment, if the difference between the first and second estimated positioning information of the plurality of camera devices exceeds a predetermined range, the processor 190 does not trust the estimated positioning information, and based on 6 DOF and SLAM Positioning information can be re-estimated.

As an optional embodiment, the processor 190 may re-estimate positioning information only for a specific camera device when first estimated and second estimated positioning information of a specific camera device differs only in positioning information of a plurality of camera devices. can If there is a difference exceeding a predetermined range even in the re-estimated value, the corresponding information may be provided as a message to the manager's terminal.

The processor 190 may generate raw data for the corrected positioning information of the plurality of camera devices (S356), perform filter calibration on the generated raw data (S357), and calculate positioning information to which the filter calibration is applied. (S358).

The processor 190 may prevent the jitter phenomenon by executing the Linear Kalman Filter Calibration algorithm. In this case, since a specific correction coefficient pre-measured according to an external environment may be applied, filtering and correction values suitable for various studio environments may be applied to the algorithm.

The processor 190 may perform image synthesis based on self-calculated positioning information. However, depending on the embodiment, the processor 190 may transmit the calculated positioning information and the photographed image through the communication unit 110 to the video synthesizing device, and the image synthesizing may be performed by the video synthesizing device (S410).

Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium storing instructions executable by a computer. Instructions may be stored in the form of program codes, and when executed by a processor, create program modules to perform operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

Computer-readable recording media include all types of recording media in which instructions that can be decoded by a computer are stored. For example, there may be read only memory (ROM), random access memory (RAM), magnetic tape, magnetic disk, flash memory, optical data storage device, and the like.

As above, the disclosed embodiments have been described with reference to the accompanying drawings. Those skilled in the art to which the present disclosure pertains will understand that the present disclosure may be implemented in a form different from the disclosed embodiments without changing the technical spirit or essential features of the present disclosure. The disclosed embodiments are illustrative and should not be construed as limiting.

100: Computing device for providing an operating method for a multi-sensor-based multi-tracking camera system in a virtual studio

Claims (10)

As a correction method for positioning information using a plurality of stereo camera devices,
receiving a captured image from the plurality of stereo camera devices when the virtual studio is captured using the plurality of stereo camera devices around an origin disposed in the virtual studio;
generating a 3D space based on the origin based on the received captured image;
estimating positioning information including spatial information and rotation angles of the plurality of stereo camera devices in the generated 3D space; and
synchronizing timing for receiving information from the plurality of stereo camera devices; including,
The step of estimating the positioning information,
calculating spatial information and rotation angles of the plurality of stereo camera devices;
cross-verifying the calculated spatial information and rotation angle based on triangulation; and
Outputting spatial information and rotation angles of the plurality of stereo camera devices that have been cross-verified,
Synchronizing the timing,
Synchronizing signal generation timings of the plurality of stereo camera devices with each other to prevent a frame sync error of data during image synthesis,
Updating positioning information for a plurality of camera devices in the 3D space based on information received from the plurality of stereo camera devices,
receiving inertial measurement values collected by an inertial measurement sensor from the plurality of stereo camera devices; and
First estimating positioning information of the plurality of stereo camera devices based on 6 degrees of freedom (6 DOF);
Updating the positioning information,
calculating positioning information including spatial information and rotation angles of the plurality of stereo camera devices; and
Second estimating positioning information of the plurality of stereo camera devices based on simultaneous localization and mapping (SLAM);
And performing correction on the first estimated and second estimated positioning information of the plurality of stereo camera devices. According to claim 1,
The step of generating the 3D space,
Recognizing a marker located at the origin; and
generating a 3D coordinate space centered on the origin based on the recognized marker; Including, method. According to claim 2,
The six degrees of freedom include roll, pitch, and yaw, which are rotational directions in the X-axis direction, the Y-axis direction, the Z-axis direction, and each axis. According to claim 3,
The step of synchronizing the timing may include synchronizing information acquisition times of the plurality of stereo camera devices based on an Open Sound Control (OSC) protocol. According to claim 4,
After performing the above correction,
Generating raw data for positioning information of a plurality of calibrated stereo camera devices;
performing filter calibration on the generated raw data;
Calculating positioning information to which the filter calibration is applied; and
performing image synthesis based on the calculated positioning information; Further comprising a method. A computer program stored in a computer-readable storage medium, wherein the computer program, when executed in one or more processors, performs the following operations for performing a correction method for positioning information using a plurality of stereo camera devices, wherein the Actions are:
receiving a captured image from the plurality of stereo camera devices when the virtual studio is captured using the plurality of stereo camera devices around an origin disposed in the virtual studio;
generating a 3D space based on the origin based on the received captured image;
estimating positioning information including spatial information and rotation angles of the plurality of stereo camera devices in the generated 3D space;
synchronizing timing for receiving information from the plurality of stereo camera devices; including,
The operation of estimating the positioning information,
calculating spatial information and rotation angles of the plurality of stereo camera devices;
cross-verifying the calculated spatial information and rotation angle based on triangulation; and
An operation of outputting spatial information and rotation angles of the plurality of stereo camera devices that have been cross-verified,
The operation of synchronizing the timing,
Synchronizing signal generation timing of the plurality of stereo camera devices with each other to prevent a frame sync error of data during image synthesis,
Updating positioning information for a plurality of camera devices in the 3D space based on information received from the plurality of stereo camera devices,
receiving an inertial measurement value collected by an inertial measurement sensor from the plurality of stereo camera devices; and
An operation of first estimating positioning information of the plurality of stereo camera devices based on 6 degrees of freedom (6 DOF);
The operation of updating the positioning information,
calculating positioning information including spatial information and rotation angles of the plurality of stereo camera devices; and
Second estimating positioning information of the plurality of stereo camera devices based on simultaneous localization and mapping (SLAM);
A computer program stored in a computer readable storage medium comprising an operation of correcting the first estimated and second estimated positioning information of the plurality of stereo camera devices. According to claim 6,
The operation of generating the 3D space,
recognizing a marker located at the origin; and
generating a 3D coordinate space centered on the origin based on the recognized marker; A computer program stored on a computer readable storage medium comprising a. According to claim 7,
The six degrees of freedom include roll, pitch, and yaw, which are rotational directions in the X-axis direction, the Y-axis direction, the Z-axis direction, and each axis. A computer program stored in a computer readable storage medium. According to claim 8,
The operation of synchronizing the timing is based on an OSC (Open Sound Control) protocol, characterized in that for synchronizing the information acquisition time of the plurality of stereo camera devices, a computer program stored in a computer readable storage medium. According to claim 9,
After the operation of performing the correction,
generating raw data for positioning information of a plurality of calibrated stereo camera devices;
performing filter calibration on the generated raw data;
calculating positioning information to which the filter calibration is applied; and
performing image synthesis based on the calculated positioning information; A computer program stored on a computer-readable storage medium further comprising a.

Images