KR102442637B1 - System and Method for estimating camera motion for AR tracking algorithm - Google Patents

Abstract

A method for estimating a camera motion for an AR tracking algorithm according to an embodiment is a method for estimating a camera motion for an AR tracking algorithm performed by a sequence production application executed by a processor of a computing device. The method comprises the steps of: displaying a target object on a sequence authoring interface; setting a virtual camera trajectory on the displayed target object; generating an image sequence by rendering an image viewed from the target object along the set virtual camera trajectory; and reproducing the generated image sequence.

Description

Camera motion estimation method and system for augmented reality tracking algorithm {System and Method for estimating camera motion for AR tracking algorithm}

The present invention relates to a camera motion estimation method and system for an augmented reality tracking algorithm. In detail, the present invention relates to a method of estimating a virtual camera motion with respect to an image sequence of a predetermined ground-truth that is a benchmark for benchmarking in an object tracking algorithm.

Augmented Reality (AR) is applied to the perception of a real-world view by superimposing virtual objects or media on the real-world view, for example a live video feed ( live video feed) is added or added.

Such augmented reality enables artificial or simulated information related to the real-world and its objects to be overlaid on the real-world view. Augmented reality is different from but related to virtual reality (VR), in which virtual reality replaces a real-world view with an artificial or simulated view. Augmented reality has been used in applications such as entertainment, video games, sports, and cell phone applications.

In various AR systems, a tracking system is used to track the position and orientation of various real-world objects and cameras in 3D space.

For example, the tracking system may track a real object in an image captured by the camera as viewed/captured. Various AR systems may attempt to create an augmented scene that includes a real-world scene (including various real-world objects) as captured by a camera and an overlaid virtual medium and/or objects.

To create the augmented scene, the tracking system may generate a virtual coordinate frame in which it may track or “place” representations of real-world objects.

Various AR systems may attempt to “place” various virtual objects (eg, CAD models/objects) in a coordinate frame to create an augmented scene. Virtual objects/models may have their own default or arbitrary reference frame (eg 3D position and orientation) for placing the virtual object in the tracking system's coordinate frame, and the tracking system's coordinate frame and virtual A mapping or transform between object reference frames has to be determined.

In addition, if a camera (eg, a camera that captures a real-world scene) moves, the AR system may attempt to change the view of the virtual objects. To do this with precision, the AR system may need to track the position and orientation of the camera.

As such, in order to implement AR, it can be seen that accurate augmentation is possible only when the camera and the pose (eg, position and posture) of the real object are detected and tracked at the point in time when the camera captures the real object.

Therefore, various object tracking applications for tracking real objects are being developed, and as various object tracking applications are installed and driven in various environments and various terminals, it is possible to determine whether tracking of real objects is being performed accurately according to such various parameter conditions. There is a need.

US Patent Publication US 2014-0118339 A1

In order to determine whether object tracking is being performed accurately in the object tracking application, the object tracking application captures the real object and tracks the pose of the real object with the ground-truth data that serves as a benchmark for benchmarking. ) is required.

An object of the present invention is to provide a camera motion estimation method and a system for an augmented reality tracking algorithm for generating an image sequence that is a benchmark.

In addition, an object of the present invention is to provide a camera motion estimation method and system for an augmented reality tracking algorithm that generates a trajectory of a virtual camera for generating an image sequence that is a reference for a benchmark.

Another object of the present invention is to provide a camera motion estimation method and system for an augmented reality tracking algorithm that evaluates the accuracy of an object tracking algorithm using a generated image sequence and improves tracking performance by modifying parameters.

A camera motion estimation method for an augmented reality tracking algorithm according to an embodiment is a camera motion estimation method for an augmented reality tracking algorithm performed by a sequence production application in which a processor of a computing device is executed, and displays a target object on a sequence production interface to do; setting a virtual camera trajectory on the displayed target object; generating an image sequence by rendering an image looking at the target object along the set virtual camera trajectory; and reproducing the generated image sequence.

In this case, the displaying of the target object on the sequence production interface includes: importing a pre-designed 3D model corresponding to the target object; and placing the imported 3D model at the origin of 3D spatial coordinates. may include.

In addition, the step of setting the virtual camera trajectory on the displayed target object includes: arranging a virtual camera according to a user input through the sequence creation interface; moving the placed virtual camera; The method may include setting the moved trajectory as a virtual camera trajectory.

In addition, the setting of the virtual camera trajectory on the displayed target object includes: receiving a real camera trajectory generated by photographing and tracking a real object from a camera device; and generating a virtual camera trajectory corresponding to the real camera trajectory and setting the generated virtual camera trajectory as a virtual camera trajectory for the target object.

In addition, the step of receiving the real camera trajectory generated by photographing and tracking the real object from the camera device includes: photographing the real object while the camera device moves; The method may include calculating, by a real camera, a pose value for a real object, and determining an actual camera trajectory including a change in the real camera pose value.

The generating of the virtual camera trajectory corresponding to the real camera trajectory may include converting a real camera pose value for the real object into a pose value of the virtual camera for the target object.

In addition, the generating an image sequence by rendering an image looking at the target object along the set virtual camera trajectory includes: moving a virtual camera according to the virtual camera trajectory; The method may include generating a frame image by rendering a frame image looking at the target object in a camera pose.

In addition, the step of generating an image sequence by rendering the image looking at the target object along the set virtual camera trajectory may include matching the virtual camera pose value at the time of photographing each rendered frame image with the corresponding frame image in the virtual image sequence. It may include the step of including.

In addition, the method for estimating camera motion for the augmented reality tracking algorithm of the embodiment may further include setting the generated virtual image sequence as a ground-truth as a reference for a benchmark.

In addition, the camera motion estimation method for the augmented reality tracking algorithm of the embodiment comprises the steps of: receiving an actual image sequence from an object tracking application of a terminal; The method may further include calculating an accuracy parameter.

A camera motion estimation method and system for an augmented reality tracking algorithm according to an embodiment of the present invention can quickly and accurately generate an image sequence as a reference for a benchmark.

In addition, the method and system for estimating camera motion for an augmented reality tracking algorithm according to an embodiment of the present invention realistically generate a motion (trajectory) of a virtual camera for generating an image sequence that is a benchmark as a reference as a real camera motion. can do.

In addition, the camera motion estimation method and system for an augmented reality tracking algorithm according to an embodiment of the present invention evaluate the accuracy of the object tracking algorithm by using the generated image sequence, and improve the tracking performance by modifying the parameters. .

1 is an exemplary block diagram of a camera motion estimation system for an augmented reality tracking algorithm.
2 and 3 depicts a camera device tracking and recognizing a real object in a real environment, and a user checking a screen displayed by matching augmented content with a real object through the camera device.
4 is a flowchart of a camera motion estimation method for an augmented reality tracking algorithm according to an embodiment of the present invention.
5 shows an example of a user interface of a program for generating an image sequence according to an embodiment of the present invention.
6 schematically illustrates the concept of generating a virtual camera trajectory according to an embodiment of the present invention.

Since the present invention can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. Effects and features of the present invention, and a method for achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms. In the following embodiments, terms such as first, second, etc. are used for the purpose of distinguishing one component from another, not in a limiting sense. Also, the singular expression includes the plural expression unless the context clearly dictates otherwise. In addition, terms such as include or have means that the features or components described in the specification are present, and do not preclude the possibility that one or more other features or components will be added. In addition, in the drawings, the size of the components may be exaggerated or reduced for convenience of description. For example, since the size and thickness of each component shown in the drawings are arbitrarily indicated for convenience of description, the present invention is not necessarily limited to the illustrated bar.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when described with reference to the drawings, the same or corresponding components are given the same reference numerals, and the overlapping description thereof will be omitted. .

1 is an exemplary block diagram of a camera motion estimation system for an augmented reality tracking algorithm, and FIGS. 2 and 3 show that a camera device tracks and recognizes a real object in a real environment, and matches the augmented content to the real object. It depicts the user checking the displayed screen through the camera device.

1 to 3 , a system 10 according to an embodiment of the present invention may include a computing device 100 and a camera device 200 .

- Computing device 100

In the computing device 100 according to an embodiment of the present invention, a sequence production application for generating an image sequence may be installed, and a target object in the object tracking application as a predetermined correct answer (Ground-truth) for the generated image sequence. A benchmarking application that benchmarks the tracked data (eg, a sequence according to a camera trajectory) to calculate accuracy may be installed.

In addition, the sequence production application may provide an environment for creating drawings of 3D models of various objects, and an environment for creating and editing content such as various augmented models or various types of information for various objects. The sequence creation application may provide various tools for drawing various contents, but is not limited thereto, and may include mechanisms for importing existing files including images, 2D or 3D objects.

Various Software Developer Kit (SDK) or library-type ToolKit can be applied to sequence creation application.

The computing device 100 may be a mobile device such as a smart phone or a tablet PC on which an application is installed. For example, the computing device 100 may include a smart phone, a mobile phone, a digital broadcasting device, personal digital assistants (PDA), a portable multimedia player (PMP), a tablet PC, and the like.

However, according to an embodiment, the computing device may include the desktop type computing device 100 . Here, the desktop type computing device 100 executes a map target tracking service based on wired/wireless communication, such as a fixed desktop PC in which an application is installed, a personal computer such as a laptop computer, and an ultrabook. It may include a device in which a program is installed for the purpose.

Also, in an embodiment, the computing device 100 may further include a predetermined server computing device that provides an environment for generating an image sequence or an environment for benchmarking.

The computing device 100 may include a memory, a processor assembly, a communication processor, an interface unit, an input system, and a display system.

In detail, an application is stored in the memory , and the application may store any one or more of various application programs, data, and commands for generating an image sequence.

For example, the memory may be various storage devices such as a ROM, an EPROM, a flash drive, a hard drive, and the like, and may include a web storage that performs a storage function of the memory on the Internet.

The processor assembly may include at least one processor capable of executing instructions of an application stored in a memory to perform various operations for generating an image sequence.

Such a processor assembly may be a system-on-a-chip (SOC) suitable for a computing device including a central processing unit (CPU) and/or a graphics processing unit (GPU), and an operating system (OS) and/or application program stored in a memory and the like, and may control each component mounted on the computing device.

In addition, the processor assembly, ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers (controllers), micro It may be implemented by including at least one of controllers (micro-controllers), microprocessors, and other electrical units for performing functions.

The communication processor may include one or more devices for communicating with an external device. Such a communication processor may communicate via a wireless network.

Specifically, the communication processor 130 may communicate with a device including a predetermined camera to generate an image sequence, and may communicate with various user input components such as a controller that has received user input.

Such a communication processor, the technical standards or communication methods for mobile communication (eg, LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G NR (New Radio), WIFI) or short-range communication method Data may be wirelessly transmitted/received with at least one of a base station, an external computing device, and an arbitrary server on a mobile communication network constructed through a communication device capable of performing the following operations.

The interface unit may communicatively connect the computing device to one or more other devices. Specifically, the interface unit may include a wired and/or wireless communication device compatible with one or more different communication protocols.

The computing device may be connected to various input/output devices through such an interface unit. For example, the interface unit may be connected to an audio output device such as a headset port or a speaker to output audio.

Also, for example, the interface unit may be connected to an input device such as a keyboard and/or a mouse to obtain a user input. For example, although the keyboard and/or mouse have been described as being connected through the interface unit, embodiments in which the keyboard and/or mouse are installed inside the computing device may also be included.

Such an interface unit, a wired / wireless headset port (port), an external charger port (port), a wired / wireless data port (port), a memory card (memory card) port, a port for connecting a device equipped with an identification module (port) , audio input/output (I/O) ports, video I/O (input/output) ports, earphone ports, power amplifiers, RF circuits, transceivers, and other communication circuits. may be included.

The input system may detect a user's input (eg, a gesture, voice command, actuation of a button, or other type of input). Specifically, the input system may include a predetermined button, a touch sensor and/or an image sensor for receiving user motion input, and the like.

Also, the input system may be connected to an external controller through an interface unit to receive a user's input.

The display system may output various information such as an image sequence as a graphic image.

Such displays include a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display. , a three-dimensional display (3D display), may include at least one of the electronic ink display (e-ink display).

- Camera device 200

And it may be a terminal including a camera that senses an image by photographing the camera device 200 periphery.

The camera device 200 may be installed with an object tracking application for tracking an object, and the object tracking application may provide an augmented reality environment by controlling to match and display augmented content on the tracked object.

The camera device 200 may be a mobile-type terminal capable of changing a viewing point of a camera by a user. For example, the camera device 200 may include a smart phone, a mobile phone, a digital broadcasting device, personal digital assistants (PDA), a portable multimedia player (PMP), a tablet PC, and the like.

In addition, the camera device 200 is a head mounted display (HMD) in which a user immersed in the augmented and/or virtual reality environment wearing the device can navigate the virtual environment and interact with the virtual environment through various different types of inputs. may include.

In some embodiments, the camera device 200 may include Microsoft's HoloLens, Meta's Meta1/Meta2 Glass, Google's Google Glass, Canon's MD-10, Magic Leap's Magic Leap One Creator Edition, etc. Commercial products may be used, or devices providing the same or similar functions may be used.

As shown in FIG. 2 , when the object tracking program of the camera device 200 detects the target object 30 , it calls the augmented content ac stored in the database or computing device 100 or other server or memory, and augments it. The content ac is matched with the target object 30 to be rendered and augmented, and event flags can be adjusted so that the stored interaction event can be operated.

At this time, the augmented content based on the matching relationship between the augmented content ac and the target object according to the pose of the target object 30 according to the angle at which the camera of the camera device 200 observes the target object 30 and the observation distance By matching (ac) to the target object 30, the augmented virtual model or other virtual information can be observed in various aspects and different sizes. Accordingly, the camera device 200 may display various pieces of information related to the target object 30 through the augmented content ac.

In various embodiments, the user may manipulate the augmented content ac displayed on the camera device 200 through manipulation of the camera device 200 .

In various embodiments, the object tracking program provides an interface that allows the user to move and rotate, enlarge, and reduce the displayed augmented content (ac) in the x, y, and z axes for sufficient and detailed observation of the augmented content (ac). to be able to do

Hereinafter, how the system generates an image sequence for a target object and evaluates the object tracking accuracy of an object tracking application through the image sequence will be described in detail.

- Camera motion estimation method for augmented reality tracking algorithm

In order to determine the accuracy of tracking the pose of the 3D target object, the object tracking application acquires a sequence according to the actual camera trajectory by photographing the target object along the camera trajectory according to the movement of the camera shooting point, and the actual camera trajectory The accuracy can be calculated by comparing (benchmeark) the sequence according to the predetermined correct answer (ground-truth, GT).

To this end, the more preset correct answers (GTs) close to the ideal value are set, the richer the benchmark datasets become, so more precise accuracy evaluation is possible for sequences according to various actual camera trajectories. Through precise accuracy evaluation, the tracking accuracy can also be improved by modifying the parameters of the object tracking application.

The terms “pose”, “viewpoint trajectory,” “image sequence,” and “target object” are used throughout this specification, and the term “pose” refers to “position and orientation ( position and orientation), and a three-dimensional position (e.g., X, Y, Z coordinates) and a three-dimensional orientation (e.g., roll, pitch, yaw) in three-dimensional space, i.e. 6 degrees of freedom (6DOF).

And “camera pose” may mean a relative position and direction with respect to a target object to be detected and tracked in a three-dimensional space. In an embodiment, when the target object is the origin of 6 degrees of freedom, the 6 degrees of freedom of the camera may be defined as a camera pose.

Also, "camera trajectory" means a path of a camera pose, the camera pose is changed according to the camera trajectory, and an image sequence may be obtained by photographing (or rendering) a target object.

When “camera trajectory” is defined as a line, it may mean a camera pose value on a line that is changed for each interval of frame images constituting an image sequence.

In an embodiment, the camera trajectory is set as a line, and a camera pose value at a viewpoint at which a camera moving along the line looks at a target object is determined according to frames per second (FPS), and the target at one camera pose value is determined. An image sequence can be formed by matching the frame image looking at the object.

The "image sequence" may be composed of scenes (eg, frame images) in which the camera moves according to the camera trajectory and the target object is viewed according to the camera pose value for each frame photographing time point.

In an embodiment, the "virtual image sequence" is generated by rendering a frame image (eg, a two-dimensional image) looking at a target object while moving the virtual camera according to the virtual camera trajectory and changing the virtual camera pose, and each frame image The virtual camera pose value at the time of photographing may be matched to the frame image and stored. Here, rendering means converting a scene when the target object is viewed from one pose of the virtual camera into a two-dimensional image.

In an embodiment, in the "real image sequence", a frame image is obtained by photographing a target object while a camera moves according to an actual camera trajectory, and an actual camera pose value may be matched and stored when the frame image is photographed.

"Target object" may include an object to be detected and tracked. For example, a three-dimensional model authored in CAD, a real object model obtained by photographing a real object and three-dimensional modeling, a model composed of feature points by photographing and learning a real object, and a two-dimensional identification code (eg, QR code, etc.) may include

Also, the “target object” may include a virtual object that is augmented by being registered on the tracked object. For example, it may include 3D augmented content to be augmented on an object that is authored and tracked in CAD.

4 is a flowchart of a camera motion estimation method for an augmented reality tracking algorithm according to an embodiment of the present invention, and FIG. 5 shows an example of a user interface of a program for generating an image sequence according to an embodiment of the present invention. , FIG. 6 schematically illustrates the concept of generating a virtual camera trajectory according to an embodiment of the present invention.

Hereinafter, with reference to FIGS. 4 to 6 , a sequence creation application executed in the processor of the computing device 100 generates a virtual image sequence, and benchmarks an actual image sequence based on this to benchmark the object tracking accuracy of the camera device 200 . A method of calculating , will be described in detail.

First, the sequence creation application may import a target object into the application and display it on a sequence creation interface (GUI). (S101)

The target object of the displayed target object may be, for example, a model manufactured through CAD as a three-dimensional model.

Also, the sequence creation application may call and display a pre-stored target object.

According to some embodiments, the target object may be built on another computer-aided design program (CAD). In addition, the sequence production application may import and display a target object or a 2D image produced on another computer-aided design program.

Referring to FIG. 5 , the target object to be loaded is determined by specifying the path to the file for the target object through the 3D Model affordance of the sequence creation interface (GUI) provided by the sequence production application, and the determined target object is stored in a three-dimensional space. can be displayed on the origin of

Next, the sequence creation application may set a virtual camera trajectory for the displayed target object (S103)

The sequence creation application may set a virtual camera trajectory for creating a virtual image sequence by rendering a target object.

In one embodiment, the sequence creation application moves the virtual camera after placing the virtual camera at a position in the 3D space according to a user input through the sequence creation interface, so that the path of the virtual camera is traced by the virtual camera. can be created with

For example, a user may manually input a virtual camera trajectory by moving a virtual camera by inputting a direction key on a keyboard and a mouse drag through the sequence creation interface.

In this case, the user may adjust the moving speed of the virtual camera according to an input in the sequence creation interface.

For example, the virtual camera can be moved on the X-Y axis plane through the direction keys, and the virtual camera trajectory is set by changing the pose value of the virtual camera by moving the virtual camera up and down on the Z axis through the other two keys. can

As for the virtual camera trajectory generated as described above, the virtual camera pose value at the time of rendering each frame image may be accurately designed.

The sequence creation application may set various parameters for generating a virtual sequence image as well as a virtual camera trajectory on the sequence creation interface.

Referring to FIG. 5 , by adjusting the scale of the target object as well as the file path of the target object in the 3D model, the scale may be adjusted so that the displayed target object represents the size of the corresponding real object.

In addition, by moving the displayed target object in the 3D space or aligning the posture, the target object may be corrected to be positioned on the origin of the 3D space, or another predefined texture may be set.

The sequence creation interface allows you to change the scale guides (to fit a full-scale 3D model) that can be disabled in scenes rendered from a virtual viewpoint.

Additionally, the sequence creation interface can modify parameters for standardized camera calibration patterns and toggle visualization of camera trajectories.

In addition, various renderings can be performed by selecting from predefined backgrounds and 3D environments through the sequence creation interface. For example, you can also adjust scene-specific lighting and shadow behavior.

You can also set the target FPS for recording using the FPS controller through the sequence creation interface. The motion of the virtual camera moving in the sequence creation interface can be executed at 200 FPS, but the rendering FPS for the image sequence can be set to 15. In FIG. 6 , the red virtual camera track (P _Tracked ) represents the frame rate.

In addition, the sequence creation interface may include a START/STOP button to start/stop recording of an image sequence.

When recording is activated, the sequence creation application may generate a virtual image sequence by moving the virtual camera according to the definition of the virtual camera trajectory and recording (rendering) the target object at a preset frame rate.

To export a virtual image sequence created in the sequence creation interface, you can create it as a file by specifying the path and clicking Export.

Such a virtual image sequence may be stored by matching a virtual camera pose at each frame image rendering time.

For example, the virtual image sequence may include CSV format data representing a pose of a virtual camera, a frame image, and Json format data matching the frame image and the virtual camera pose.

As described above, for the trajectory of the virtual camera manually designed by the user through the sequence creation interface, the movement of the virtual camera must be manually specified one by one. In addition, the manually designed virtual camera trajectory may not represent the realistic movement of the real camera trajectory captured while the real camera moves in the real environment.

In another embodiment, the sequence creation application may set the virtual camera trajectory based on the real camera trajectory in which the camera device captures the real object.

In detail, referring to FIG. 5 , the user may determine an actual object to be photographed by the camera device through the 2D target part in the sequence creation interface. In this case, it is preferable that the real object is an object that is easy to track, such as a two-dimensional identification code or a pre-learned two-dimensional image.

In addition, the user can track the real object by moving the camera through the object tracking application of the camera device 200 to capture the determined real object. The object tracking application may track a real object and calculate an actual camera pose value in a frame image of a captured image, and generate a change in the camera pose value according to the movement of the camera as an actual camera trajectory.

The sequence creation application may receive the real camera trajectory from the camera device 200 , generate a virtual camera trajectory corresponding to the received real camera trajectory, and display it on the sequence creation interface.

That is, referring to FIG. 6 , when a user shoots a real target object ( _TR ) while moving the camera ( _CR ) of the camera device 200, the object tracking application performs an actual target object ( _TR ) in the captured image. It is possible to calculate the pose of the real camera by tracking , and generate an actual camera trajectory (P _original ) indicating a change in the calculated actual camera pose value. In addition, an actual image sequence in which a frame image of a captured image and an actual camera pose are matched may also be generated.

The sequence production application (3D simulation) may acquire the actual camera trajectory P _original and generate a virtual camera trajectory P _Tracked of the virtual camera C _v for the target object T _v based on this. For example, the trajectory creation module (Alg 1) of the sequence production application converts the pose change value of the real camera with respect to the real object into the pose change value of the virtual camera with respect to the target object, and corresponds to the real camera trajectory (P _original ) A virtual camera trajectory (P _Tracked ) can be created.

As described above, the virtual camera trajectory generated based on the real camera trajectory can be generated according to the realistic movement of the real camera, and there is an advantage in that there is no need to manually design the virtual camera trajectory.

An offset may occur in the process of converting the actual camera trajectory to the virtual camera trajectory. Since the image sequence according to the virtual camera trajectory is rendered and generated according to the virtual camera pose, it is composed of a frame image that accurately matches the virtual camera pose. can be

Therefore, since the generated virtual image sequence has an ideal frame image corresponding to the actual camera movement, it is suitable for use as a ground truth (GT) as a reference for a benchmark.

Therefore, a sequence creation application can easily create an ideal virtual image sequence to create a benchmark dataset.

In addition, the sequence production application may reproduce the virtual image sequence generated in this way through the sequence production application. (S109)

The virtual image sequence may be input and reproduced in the export part of FIG. 5 .

In addition, the sequence production application can create a benchmark dataset by setting the generated virtual image sequence as a predetermined correct answer (GT) for the target object. (S111)

Thereafter, the sequence production application may receive an actual image sequence obtained by photographing and tracking a real object corresponding to the target object in various object tracking applications of various camera devices. (S113)

The sequence production application may benchmark the received real image sequence and a predetermined correct answer (GT) to calculate accuracy in the object tracking application of the camera device that has transmitted the real image sequence. (S115)

In addition, the sequence production application may improve the object tracking performance by modifying an algorithm parameter of the object tracking application through parameters such as calculated accuracy or by modifying the target object itself.

The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and used by those skilled in the art of computer software. Examples of the computer-readable recording medium include hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floppy disks. medium), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. A hardware device may be converted into one or more software modules to perform processing in accordance with the present invention, and vice versa.

The specific implementations described in the present invention are only examples, and do not limit the scope of the present invention in any way. For brevity of the specification, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connections or connecting members of the lines between the components shown in the drawings exemplify functional connections and/or physical or circuit connections, and in an actual device, various functional connections, physical connections that are replaceable or additional may be referred to as connections, or circuit connections. In addition, unless there is a specific reference such as "essential", "importantly", etc., it may not be a necessary component for the application of the present invention.

In addition, although the detailed description of the present invention has been described with reference to a preferred embodiment of the present invention, those skilled in the art or those having ordinary knowledge in the art will appreciate the spirit of the present invention described in the claims to be described later. And it will be understood that various modifications and variations of the present invention can be made without departing from the technical scope. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

Claims (10)

As a camera motion estimation method for an augmented reality tracking algorithm performed by a sequence production application executed by a processor of a computing device,
displaying the target object on the sequence creation interface;
setting a virtual camera trajectory on the displayed target object;
generating an image sequence by rendering an image looking at the target object along the set virtual camera trajectory; and
reproducing the generated image sequence;
The step of displaying the target object on the sequence production interface includes determining the target object, which is a pre-designed three-dimensional model corresponding to the real object, according to a user input,
setting the generated virtual image sequence as a ground-truth that is a standard for a benchmark;
Receiving a real image sequence obtained by photographing and tracking a real object corresponding to the target object from an object tracking application of the terminal;
Comparing the received actual image sequence with the predetermined correct answer, further comprising the step of calculating an accuracy parameter of the object tracking application
Camera motion estimation method for augmented reality tracking algorithm. The method of claim 1,
Displaying the target object on the sequence production interface comprises:
importing a pre-designed three-dimensional model corresponding to the target object;
arranging the imported 3D model at the origin of 3D spatial coordinates
Camera motion estimation method for augmented reality tracking algorithm. The method of claim 1,
Setting a virtual camera trajectory on the displayed target object comprises:
arranging a virtual camera according to a user input through the sequence production interface;
moving the placed virtual camera;
and setting the trajectory of the virtual camera as a trajectory of the virtual camera.
Camera motion estimation method for augmented reality tracking algorithm. The method of claim 1,
Setting a virtual camera trajectory on the displayed target object comprises:
Receiving a real camera trajectory generated by photographing and tracking a real object from a camera device;
generating a virtual camera trajectory corresponding to the real camera trajectory;
setting the generated virtual camera trajectory as a virtual camera trajectory for the target object
Camera motion estimation method for augmented reality tracking algorithm. 5. The method of claim 4,
Receiving a real camera trajectory generated by photographing and tracking a real object from the camera device,
moving the camera device and photographing a real object;
calculating, by the real camera of the camera device, a pose value for the real object for each time point at which the real object is photographed;
Determining an actual camera trajectory including a change in the actual camera pose value
Camera motion estimation method for augmented reality tracking algorithm. 6. The method of claim 5,
The step of generating a virtual camera trajectory corresponding to the real camera trajectory comprises:
converting a real camera pose value for the real object into a pose value of the virtual camera for the target object
Camera motion estimation method for augmented reality tracking algorithm. The method of claim 1,
The step of generating an image sequence by rendering an image looking at the target object along the set virtual camera trajectory comprises:
moving the virtual camera according to the virtual camera trajectory;
generating a frame image by rendering a frame image looking at the target object from a virtual camera pose at one point of view of the moving virtual camera
Camera motion estimation method for augmented reality tracking algorithm. 8. The method of claim 7,
The step of generating an image sequence by rendering an image looking at the target object along the set virtual camera trajectory comprises:
and matching the frame image with the virtual camera pose value at the time of capturing each of the rendered frame images and including the corresponding frame image in a virtual image sequence.
Camera motion estimation method for augmented reality tracking algorithm. delete delete

Images