KR102177445B1 - An apparatus for pose estimation of object using latent variable from auto encoder and method thereof - Google Patents

Abstract

Provided is a method for estimating a posture of an object using a latent variable dictionary of a patch auto-encoder, which is performed by a computing device. The method comprises the steps of: generating a plurality of latent variables corresponding to a plurality of image patches respectively from an image including a target object by using a patch auto-encoder, wherein the image patch includes a central point; determining a position on a three-dimensional model of the central point of an image patch corresponding to selected latent variables with respect to N selected latent variables (where N is a natural number equal to or greater than 2) predetermined among the plurality of latent variables by using the latent variable dictionary connecting a position on the three-dimensional model of the central point of the image patch according to a latent variable value; and determining the posture of the target object by using the position on the three-dimensional model of the central point of the image patch corresponding to the selected latent variables. According to the present invention, it is possible to select feature points with high reliability from the detection results, thereby increasing robustness of object posture estimation.

Description

Device for estimating the attitude of an object using a dictionary of latent variables of an auto encoder {AN APPARATUS FOR POSE ESTIMATION OF OBJECT USING LATENT VARIABLE FROM AUTO ENCODER AND METHOD THEREOF}

The present invention relates to image processing, and more particularly, to an apparatus and method for extracting feature points of an object included in an image and estimating a posture of the object.

When certain feature points of an object included in a 2D image are detected, knowing the positions of these feature points in the 3D model can inversely estimate the position and posture of the model for this object. For this, various traditional descriptors such as SIFT and ORB have been used, and various methods of estimating specific points through deep learning have been used. The new methods based on deep learning show superior results compared to the traditional methods, but have a disadvantage in that feature points must be specified in advance for learning.

Korean Patent Application Publication No. 2014-0134154 ("Method and terminal for extracting an object from an image", VFlap, Inc.)

An object of the present invention for solving the above-described problem is that, due to the nature of deep learning, it is robust and accurate with respect to environmental variables such as lighting and background, and features that are good for self-detection without specifying the feature points in advance as in the existing deep learning approach. The object attitude estimation method using the latent variable dictionary of the patch auto encoder is provided, which can increase the robustness of the object attitude estimation because it is possible to select only the highly reliable feature points from the detection result.

Another object of the present invention for solving the above problems is that, due to the nature of deep learning, it is robust and accurate for environmental variables such as lighting and background, and it is easy to detect by itself without designating a feature point in advance like the existing deep learning approach. The present invention provides an object attitude estimation apparatus using a dictionary of latent variables of a patch auto encoder, which can increase robustness of object attitude estimation by selecting feature points and selecting only highly reliable feature points from a detection result.

However, the problem to be solved of the present invention is not limited thereto, and may be variously extended without departing from the spirit and scope of the present invention.

An object attitude estimation method using a latent variable dictionary of a patch auto encoder according to an embodiment of the present invention for achieving the above object may be performed by a computing device, and the method includes: Generating a plurality of latent variables each corresponding to a plurality of image patches from an image containing a target object, wherein the image patch includes a central point; Using a latent variable dictionary that connects the position of the center point of the image patch according to the value of the latent variable on the 3D model, among the plurality of latent variables, N predetermined (where N is a natural number of 2 or more) selected latent variables And determining a position on the 3D model of a center point of the image patch corresponding to the selected latent variable; And determining a posture of the target object by using the position on the 3D model of the center point of the image patch corresponding to the selected latent variable.

According to an aspect, the patch auto encoder receives a plurality of training image patches included in at least one training data, converts them into latent variables respectively corresponding to the plurality of training image patches, and converts the latent variables again. It may be a deep learning-based auto encoder that has been trained to convert and output the training image patches.

According to an aspect, a center point of an image patch corresponding to the selection latent variable may be a feature point of the target object.

According to an aspect, when the N number of predetermined latent variables are selected from among the plurality of latent variables, when an area of a certain latent variable around the plurality of latent variables is mapped to a 3D area, the mapped 3D area is further There may be N latent variables selected in a narrow order.

According to an aspect, the N predetermined selection latent variables may be selected to have a higher precision compared to a case of selecting a latent variable other than the selection latent variable.

According to an aspect, when the N number of predetermined latent variables are selected from among the plurality of latent variables, when a region of a latent variable around the plurality of latent variables is mapped to a predetermined 3D region, the region of the mapped latent variable is There may be N latent variables selected in a wider order.

According to an aspect, the N predetermined selection latent variables may be selected to have robustness and universality for a wider range of input compared to a case of selecting a latent variable other than the selective latent variable.

According to one aspect, the 3D area matching the area of the surrounding potential variable located within a predetermined distance from the first potential variable is located apart from the 3D area corresponding to the first potential variable by a predetermined distance or more. In response to the determination, the first latent variable may be excluded from the selected latent variable.

According to an aspect, the N predetermined selection latent variables may be selected to exclude outliers from feature points.

According to an aspect, the determining of the posture of the target object may include estimating a position and posture of the target object by applying a Perspective N Point algorithm.

An apparatus according to another embodiment of the present invention for solving the above-described problem is an object attitude estimation apparatus using a latent variable dictionary of a patch auto encoder, the apparatus including a processor and a memory, and the processor is a patch auto encoder Generating a plurality of latent variables each corresponding to a plurality of image patches from an image including a target object, wherein the image patch includes a central point; Using a latent variable dictionary that connects the position of the center point of the image patch according to the value of the latent variable on the 3D model, among the plurality of latent variables, N predetermined (where N is a natural number of 2 or more) selected latent variables And determining a position on the 3D model of the center point of the image patch corresponding to the selected latent variable; Further, it may be configured to determine the posture of the target object by using the position on the 3D model of the center point of the image patch corresponding to the selected latent variable.

A computer-readable storage medium according to another embodiment of the present invention for solving the above-described problem is a computer-readable storage medium including instructions executable by a processor, the instructions being a latent variable dictionary of a patch auto encoder. The instructions are for performing object attitude estimation, and when the instructions are executed by the processor, the processor causes the processor to patch a plurality of images from an image including a target object, using a patch auto encoder-where the image patch is a center point Including-and generating a plurality of latent variables each corresponding to; Using a latent variable dictionary that connects the position of the center point of the image patch according to the value of the latent variable on the 3D model, among the plurality of latent variables, N predetermined (where N is a natural number of 2 or more) selected latent variables And determining a position on the 3D model of the center point of the image patch corresponding to the selected latent variable; In addition, it may be configured to determine the posture of the target object by using the position on the 3D model of the center point of the image patch corresponding to the selected latent variable.

The disclosed technology can have the following effects. However, since it does not mean that a specific embodiment should include all of the following effects or only the following effects, it should not be understood that the scope of the rights of the disclosed technology is limited thereby.

According to the method and apparatus for estimating the attitude of an object using a dictionary of latent variables of a patch auto encoder according to an embodiment of the present invention described above, due to the nature of deep learning, it is robust and accurate for environmental variables such as lighting and background, while existing deep learning Like the approach, it is possible to select feature points that are easy to detect by themselves, and to select only highly reliable feature points from the detection result without specifying the feature points in advance, thereby enhancing the robustness of object attitude estimation.

Therefore, while maintaining high accuracy and robustness, compared to other deep learning-based feature point extraction, since the feature point does not need to be designated in advance, there is no variation or loss in accuracy accordingly. In addition, since labels for a specific object are not learned, if a sufficiently large amount of data is learned, it can be immediately applied to a new object without additional learning, which can be a great advantage in terms of introduction time.

1 is a flowchart of a method for estimating an object attitude using a dictionary of latent variables of a patch auto encoder according to an embodiment of the present invention.
2 shows the learning process of the auto-encoder latent variable dictionary that can be used as a feature point.
3 shows an example of selection of feature points.
4 shows a conceptual diagram of object attitude estimation using a dictionary of latent variables.
5 is a block diagram illustrating a configuration of a computing system capable of operating as an object posture estimation apparatus using a latent variable dictionary of a patch auto encoder according to an embodiment of the present invention.

In the present invention, various modifications may be made and various embodiments may be provided, and specific embodiments will be illustrated in the drawings and described in detail.

However, this is not intended to limit the present invention to a specific embodiment, it is to be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention.

Terms such as first and second may be used to describe various elements, but the elements should not be limited by the terms. These terms are used only for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. The term and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. Should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in this application. Does not.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. In describing the present invention, in order to facilitate an overall understanding, the same reference numerals are used for the same elements in the drawings, and duplicate descriptions for the same elements are omitted.

The present invention includes a pipeline of feature point extraction and object attitude estimation using a latent variable dictionary of a patch auto encoder.

As described above, according to the method and apparatus for estimating the attitude of an object using a dictionary of latent variables of a patch auto encoder according to an embodiment of the present invention, due to the nature of deep learning, it is robust and accurate for environmental variables such as lighting and background. As with the deep learning approach of D, it is possible to select features that are good to be detected by itself, and to select only features with high reliability from the detection results, thereby enhancing the robustness of object attitude estimation.

Therefore, while maintaining high accuracy and robustness, compared to other deep learning-based feature point extraction, since the feature point does not need to be designated in advance, there is no variation or loss in accuracy accordingly. In addition, since labels for a specific object are not learned, if a sufficiently large amount of data is learned, it can be immediately applied to a new object without additional learning, which can be a great advantage in terms of introduction time.

1 is a flowchart of a method of estimating an object posture using a dictionary of latent variables of a patch auto encoder according to an embodiment of the present invention. Hereinafter, a method of estimating an object posture using a latent variable dictionary of a patch auto encoder according to an embodiment of the present invention will be described in more detail with reference to FIG. 1.

As shown in FIG. 1, an object attitude estimation method using a latent variable dictionary of a patch auto encoder according to an embodiment of the present invention includes a plurality of image patches from an image including a target object using a patch auto encoder. Here, the image patch includes a central point-generating a plurality of latent variables each corresponding to (step 110),

Using a latent variable dictionary that connects the position of the center point of the image patch according to the value of the latent variable on the 3D model, among the plurality of latent variables, N predetermined (where N is a natural number of 2 or more) selected latent variables , After determining the position on the 3D model of the center point of the image patch corresponding to the selected latent variable (step 120),

The posture of the target object may be determined (step 130) by using the position on the 3D model of the center point of the image patch corresponding to the selected latent variable.

Hereinafter, in relation to the determination of the attitude of the target object as described above, it will be described in more detail in the order of learning an auto encoder, learning a latent variable dictionary, and estimating an object attitude using a latent variable dictionary.

As noted above, the present invention extracts key points (hereinafter, also referred to as'feature points') by utilizing the latent variables of the deep learning auto encoder as a descriptor in the traditional vision and ultimately estimating the attitude of the object. Suggest a way for you.

2 shows the learning process of the auto-encoder latent variable dictionary that can be used as a feature point. As shown in FIG. 2, in the object attitude estimation method using the latent variable dictionary of the patch auto encoder according to an embodiment of the present invention, first, a patch centering on several pixels is extracted from an image of training data, and A deep neural network-based auto encoder can be trained (210). That is, the patch auto encoder receives a plurality of training image patches included in at least one training data, converts them into latent variables respectively corresponding to the plurality of training image patches, and converts the latent variables back to the training image patch. It may be a deep learning-based auto encoder that has been trained to be converted to and outputted.

According to one aspect, an autoencoder is a kind of machine learning method, and may belong to unsupervised learning. Using a neural network algorithm, learning aims to make the output value from a certain input through the neural network as close as possible to the input value.At this time, if the number of neural network neurons is greater than or equal to the dimension of the input value, the input value is If you receive it as it is and send it out, the meaning of learning disappears because it is just that. That is, in order for the autoencoder to be meaningful, the number of neurons must be smaller than the dimension of the input value, and as a result of this learning, the effect of compression that can restore the original value with a smaller number of values can be obtained. According to an aspect of the present invention, the input of the autoencoder may be an image patch, which is a fragment of an image having a predetermined size (for example, a size of 64 × 64 px), and the latent variable is a predetermined size converted from the image patch. It can be a vector. The auto encoder according to an aspect of the present invention extracts at least one point-centered patch from images (for example, a single object image) as training data, and the patch is transformed into a latent variable through a neuron of a neural network. It can be learned to output the same image patch as possible as the patch again from the variable.

Referring to FIG. 2 again, afterwards, training data may be re-entered into the learned autoencoder and a latent variable may be generated from each image patch (220). Here, as the learning data, the same learning data as the data used for learning of the auto encoder may be used. When a plurality of latent variables are generated, a 3D position dictionary according to the values of the latent variables may be generated by recording the generated latent variables and the positions of the center points of each patch on the 3D model (230).

According to an aspect of the present invention, it may be configured to select a feature point that is most detectable among pixel center points included in each of a plurality of image patches included in an image, and select only feature points with high reliability. In the selection of the above feature points, for example, based on the relationship between the latent variable transformed from the image patch and the 3D model space of the image patch, N (where N is a natural number of 2 or more) selected latent variables It can be done by deciding. According to an aspect, a center point of an image patch corresponding to a selection latent variable may be a feature point of the target object.

3 shows an example of selection of feature points. The previously created latent variable dictionary 320 can be viewed as a relationship between the latent variable map 310 and the model 330, that is, a mapping function between the latent variable space 340 and the model space 350. For example, the following three rules can be applied to select feature points (i.e., related latent variables).

1) When a specific latent variable area is mapped to a 3D area, the mapped 3D area should be narrow (high precision)

2) At this time, the area of the latent variable should be wide (robustness and universality for input)

3) Surrounding latent variable regions should not be connected to distant three-dimensional positions (outliers excluded)

That is, according to an aspect of the present invention, among the plurality of latent variables generated by the auto encoder, the area of the latent variables for the plurality of latent variables among the plurality of latent variables determined in relation to the feature point selection is a three-dimensional area. When mapped, the mapped 3D regions may be N latent variables selected in a narrower order. Accordingly, the determined N selection latent variables may be selected to have higher precision compared to the case of selecting latent variables other than the selected latent variables.

Meanwhile, according to an aspect of the present invention, the determined N selection latent variables are, among a plurality of latent variables generated by the auto encoder, when the regions of the latent variables around the plurality of latent variables are mapped to a predetermined three-dimensional region. , The area of the latent variable to be mapped may be N latent variables selected in a wider order. Therefore, it may be selected to have robustness and universality for a wider range of inputs compared to the case of selecting a latent variable other than the selective latent variable.

Here, the requirements of a narrow 3D area and a wide area of a latent variable may be considered in an integrated manner, and whether or not a weight for which of the above two requirements is to be assigned a greater weight may be applied differently depending on the implementation environment. In addition, in determining the selection latent variable, for example, latent variables satisfying a specific threshold value (for example, latent variables in which a three-dimensional region is smaller than a first threshold value and/or a latent variable space is a second threshold value. Potential variables larger than the value) may be selected first, and the selected latent variable may be determined according to the area of the 3D area and/or the space of the latent variable.

Meanwhile, according to an aspect of the present invention, a 3D area matching the area of the surrounding potential variable located within a predetermined distance from the first potential variable is separated from the 3D area corresponding to the first potential variable by a predetermined distance or more. In response to the determination that the position is located, the first latent variable may be determined to be excluded from the selected latent variable, and thus the outlier may be excluded from the feature point. As shown in FIG. 3, for example, in the case of the latent variable 360, it is possible to pass because it is not separated by a predetermined distance from the three-dimensional area, but in the case of the latent variable 370, the surrounding latent area indicated in orange is separated from the 3D area. Since it is connected to the location, it may be rejected.

4 shows a conceptual diagram of object attitude estimation using a dictionary of latent variables. With reference to FIG. 4, object attitude estimation for a new input image will be described.

After learning the auto encoder and generating the latent variable dictionary, the latent variable corresponding to each pixel is calculated (430) for a new input image, and from the latent variable dictionary, the position of each pixel in the 3D model is determined. You can create a connection to (440). According to this connection, the position and posture of the object can be estimated (460) by, for example, applying a general Perspective N Point algorithm (450). In this case, when several objects are mixed on the image or when the detection result is affected by the background, various object region extraction algorithms may be applied in advance (410) to improve performance. That is, when a plurality of objects are included in the image, after separating a single object and extracting a pixel center patch (420), a latent variable may be generated by an auto encoder.

5 is a block diagram illustrating a configuration of a computing system capable of operating as an object posture estimation apparatus using a latent variable dictionary of a patch auto encoder according to an embodiment of the present invention. An object posture estimation apparatus using a latent variable dictionary of a patch auto encoder according to an embodiment of the present invention includes a processor and a memory, and the processor uses a patch auto encoder to obtain a plurality of images from an image including a target object. Using a latent variable dictionary that creates a plurality of latent variables each corresponding to a patch-where the image patch includes a center point-and connects the position of the image patch center point on the 3D model according to the latent variable value, the For N predetermined selected latent variables among a plurality of latent variables (where N is a natural number of 2 or more), the position of the center point of the image patch corresponding to the selected latent variable on the 3D model is determined, and the selection It may be configured to determine the posture of the target object by using the position on the 3D model of the center point of the image patch corresponding to the latent variable. A more specific operation of the apparatus for estimating an object posture using a dictionary of latent variables of a patch auto encoder according to an embodiment of the present invention may follow the method for estimating an object posture according to an aspect of the present invention.

Meanwhile, referring to FIG. 5, the computing system 800 may include a flash storage 810, a processor 820, a RAM 830, an input/output device 840, and a power supply device 850. In addition, the flash storage 810 may include a memory device 811 and a memory controller 812. Meanwhile, although not shown in FIG. 8, the computing system 800 may further include ports capable of communicating with a video card, a sound card, a memory card, a USB device, or the like, or with other electronic devices. .

The computing system 800 may be implemented as a personal computer, or a portable electronic device such as a notebook computer, a mobile phone, a personal digital assistant (PDA), and a camera.

Processor 820 may perform certain calculations or tasks. Depending on the embodiment, the processor 820 may be a micro-processor or a central processing unit (CPU). The processor 820 is provided with a RAM 830, an input/output device 840, and a flash storage 810 through a bus 860 such as an address bus, a control bus, and a data bus. Communication can be performed. The flash storage 810 may be implemented using the flash storage of the embodiments shown in FIGS. 5 to 7.

According to an embodiment, the processor 820 may also be connected to an expansion bus such as a Peripheral Component Interconnect (PCI) bus.

The RAM 830 may store data necessary for the operation of the computing system 800. For example, random access memory of any type, including DRAM, mobile DRAM, SRAM, PRAM, FRAM, MRAM, and RRAM Can be used as 830.

The input/output device 840 may include an input means such as a keyboard, a keypad, and a mouse, and an output means such as a printer and a display. The power supply 850 may supply an operating voltage required for the operation of the computing system 800.

The above-described method according to the present invention may be implemented as a computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording media in which data that can be decoded by a computer system is stored. For example, there may be read only memory (ROM), random access memory (RAM), magnetic tape, magnetic disk, flash memory, optical data storage device, and the like. In addition, the computer-readable recording medium can be distributed to a computer system connected through a computer communication network, and stored and executed as code that can be read in a distributed manner.

Although described above with reference to the drawings and examples, it does not mean that the protection scope of the present invention is limited by the drawings or examples, and those skilled in the art It will be appreciated that various modifications and changes can be made to the present invention without departing from the spirit and scope.

Specifically, the described features may be implemented in digital electronic circuitry, or computer hardware, firmware, or combinations thereof. Features may be executed in a computer program product implemented in storage in a machine-readable storage device, for example, for execution by a programmable processor. And the features can be performed by a programmable processor executing a program of directives to perform the functions of the described embodiments by operating on input data and generating output. The described features include at least one programmable processor, at least one input device, and at least one output device coupled to receive data and directives from the data storage system and to transmit data and directives to the data storage system. It can be executed within one or more computer programs that can be executed on a programmable system including. A computer program includes a set of directives that can be used directly or indirectly within a computer to perform a specific action on a given result. A computer program is written in any form of a programming language, including compiled or interpreted languages, and is included as a module, element, subroutine, or other unit suitable for use in another computer environment, or as a independently operable program It can be used in any form.

Suitable processors for execution of a program of directives include, for example, both general and special purpose microprocessors, and either a single processor or multiple processors of a different type of computer. Storage devices suitable for implementing computer program directives and data implementing the described features are, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic devices such as internal hard disks and removable disks. Devices, magneto-optical disks, and all types of non-volatile memory including CD-ROM and DVD-ROM disks. The processor and memory may be integrated within application-specific integrated circuits (ASICs) or added by ASICs.

The present invention described above is described on the basis of a series of functional blocks, but is not limited by the above-described embodiments and the accompanying drawings, and various substitutions, modifications and changes within the scope not departing from the technical spirit of the present invention It will be apparent to those of ordinary skill in the art to which this invention pertains.

Combinations of the above-described embodiments are not limited to the above-described embodiments, and various types of combinations as well as the above-described embodiments may be provided according to implementation and/or need.

In the above-described embodiments, the methods are described on the basis of a flowchart as a series of steps or blocks, but the present invention is not limited to the order of steps, and certain steps may occur in a different order or concurrently with those described above. have. In addition, those of ordinary skill in the art understand that the steps shown in the flowchart are not exclusive, other steps are included, or one or more steps in the flowchart may be deleted without affecting the scope of the present invention. You can understand.

The above-described embodiments include examples of various aspects. Although not all possible combinations for representing the various aspects can be described, those of ordinary skill in the art will recognize that other combinations are possible. Accordingly, the present invention will be said to include all other replacements, modifications and changes falling within the scope of the following claims.

Claims (12)

An object attitude estimation method using a dictionary of latent variables of a patch auto encoder, performed by a computing device,
Generating a plurality of latent variables each corresponding to a plurality of image patches from an image including a target object, wherein the image patch includes a center point, using a patch auto encoder;
Using a latent variable dictionary that connects the position of the center point of the image patch according to the value of the latent variable on the 3D model, among the plurality of latent variables, N predetermined (where N is a natural number of 2 or more) selected latent variables And determining a position on the 3D model of a center point of the image patch corresponding to the selected latent variable; And
Determining a posture of the target object using a position on a 3D model of a center point of an image patch corresponding to the selected latent variable,
The N predetermined selection latent variables,
The object attitude estimation method, wherein, among the plurality of latent variables, when a region of a latent variable around the plurality of latent variables is mapped to a three-dimensional region, the mapped three-dimensional region is N latent variables selected in a narrower order.
An object attitude estimation method using a dictionary of latent variables of a patch auto encoder, performed by a computing device,
Generating a plurality of latent variables each corresponding to a plurality of image patches from an image including a target object, wherein the image patch includes a center point, using a patch auto encoder;
Using a latent variable dictionary that connects the position of the center point of the image patch according to the value of the latent variable on the 3D model, among the plurality of latent variables, N predetermined (where N is a natural number of 2 or more) selected latent variables And determining a position on the 3D model of a center point of the image patch corresponding to the selected latent variable; And
Determining a posture of the target object using a position on a 3D model of a center point of an image patch corresponding to the selected latent variable,
The N predetermined selection latent variables,
The object attitude estimation method, wherein, among the plurality of latent variables, when a latent variable region around the plurality of latent variables is mapped to a predetermined 3D region, the regions of the mapped latent variables are N latent variables selected in a wider order.
The method according to claim 1 or 2,
The object posture estimation method, wherein the center point of the image patch corresponding to the selected latent variable is a feature point of the target object.
The method according to claim 1 or 2,
The patch auto encoder,
Learning to receive a plurality of training image patches included in at least one training data, convert them into latent variables corresponding to each of the plurality of training image patches, and convert the latent variables back into the training image patches and output A deep learning-based auto encoder, an object attitude estimation method.
The method of claim 1,
The N predetermined selection latent variables,
The object attitude estimation method is selected to have a higher precision compared to the case of selecting a latent variable other than the selected latent variable.
delete The method of claim 2,
The N predetermined selection latent variables,
Compared to the case of selecting a latent variable other than the selected latent variable, the object attitude estimation method is selected to have robustness and universality for a wider range of inputs.
The method according to claim 1 or 2,
In response to a determination that the 3D area matching the area of the surrounding potential variable located within a predetermined distance from the first potential variable is located spaced apart from the 3D area corresponding to the first potential variable by a predetermined distance or more, The first latent variable is to be excluded from the selected latent variable, the object attitude estimation method.
The method of claim 8,
The N predetermined selection latent variables,
The object attitude estimation method is selected to exclude outliers from the feature points.
The method of claim 1,
The step of determining the posture of the target object,
To estimate the position and posture of the target object by applying a Perspective N Point algorithm.
An object attitude estimation apparatus using a dictionary of latent variables of a patch auto encoder, the apparatus including a processor and a memory, the processor,
Generating a plurality of latent variables each corresponding to a plurality of image patches from an image including a target object, wherein the image patch includes a central point, using a patch auto encoder;
Using a latent variable dictionary that connects the position of the center point of the image patch according to the value of the latent variable on the 3D model, among the plurality of latent variables, N predetermined (where N is a natural number of 2 or more) selected latent variables And determining a position on the 3D model of the center point of the image patch corresponding to the selected latent variable; And
It is configured to determine the posture of the target object by using the position on the 3D model of the center point of the image patch corresponding to the selected latent variable,
The N predetermined selection latent variables,
Among the plurality of latent variables, when a region of a latent variable around the plurality of latent variables is mapped to a three-dimensional region, the mapped three-dimensional region is N latent variables selected in a narrower order,
When a potential variable region around the plurality of latent variables among the plurality of latent variables is mapped to a predetermined 3D region, the region of the mapped latent variable is N latent variables selected in a wider order.
A computer-readable storage medium including instructions executable by a processor, wherein the instructions are for performing object attitude estimation using a latent variable dictionary of a patch auto encoder, and the instructions are transferred to the processor when executed by the processor. Let,
Generating a plurality of latent variables each corresponding to a plurality of image patches from an image including a target object, wherein the image patch includes a central point, using a patch auto encoder;
Using a latent variable dictionary that connects the position of the center point of the image patch according to the value of the latent variable on the 3D model, among the plurality of latent variables, N predetermined (where N is a natural number of 2 or more) selected latent variables And determining a position on the 3D model of the center point of the image patch corresponding to the selected latent variable; And
It is configured to determine the posture of the target object by using the position on the 3D model of the center point of the image patch corresponding to the selected latent variable,
The N predetermined selection latent variables,
Among the plurality of latent variables, when a region of a latent variable around the plurality of latent variables is mapped to a three-dimensional region, the mapped three-dimensional region is N latent variables selected in a narrower order,
Among the plurality of latent variables, when a latent variable region around the plurality of latent variables is mapped to a predetermined three-dimensional region, the region of the mapped latent variable is N latent variables selected in a wider order, a computer-readable storage medium .

Images