KR20210029689A - Method and Apparatus for Seamline estimation based on moving object preserving - Google Patents

Abstract

The present invention relates to a method and a device for estimating a seamline to which object movement information is applied, and a method and a device for matching an image using the same. According to the present invention, the method for matching an image using the method for estimating a seamline to which object movement information is applied comprises: a seamline estimation step of applying object movement; a seamline update step of applying an energy difference of an estimated seamline; and an image matching step of using the estimated seamline or updated seamline, wherein the seamline estimation step includes the steps of: obtaining energy representing an object and a structure of a background included in two images to be matched; and estimating the seamline passing through a low-energy portion. Therefore, the seamline which does not cause distortion can be estimated.

Description

A method and apparatus for estimating a seam line using object motion information, and a method and apparatus for image matching using the same {Method and Apparatus for Seamline estimation based on moving object preserving}

The present invention relates to a method and apparatus for estimating a seam tangent to which motion information of an object in an image is applied, and an image matching method and apparatus using the same.

With the recent rapid growth of the VR market, the demand for images having a wide viewing angle such as 360-degree images has increased. Such an image with a wide viewing angle is obtained through a process of acquiring a plurality of images photographing an overlapped spatial region using a plurality of cameras having a constant viewing angle, and matching the images of the overlapped portions with each other. In this process, an error may occur in the matched image due to parallax or the like of the acquired images. In order to correct such a matching error, image matching using spatial transformation based on feature points, image matching based on warping, and image matching based on seamline are used.

In the feature point-based spatial transform image matching technology, feature point pairs are formed using feature points between images, and images are matched by forming a spatial transform matrix based on the feature points. In addition, the warping technique mainly uses a position transformation function formed by regional and regional feature points, transforms an image using geometric projection transformation, and uses it for registration. However, if the object and the background exist at different distances from the camera, a perfectly matched image cannot be created using a single transformation matrix or transformation function, and thus the matching result image may be slightly different from the actual intended shooting result.

The seamline-based image matching technology constructs an energy function by using the difference between two images constituting the overlapping area of the image, and estimates the position to minimize it as the optimal seamline. It is a technology that prevents distortion by connecting the overlapping regions of the image. At this time, the energy function is configured to have a high value at the location of the strong contour line present in the two images to prevent the contour line from being broken in the matched image. However, the existing methods did not consider the energy that can selectively use the moving object and the background when calculating the energy in the overlapping area, so there was a case that the seam tangent line penetrating the moving object was estimated. Since the distortion of a moving object having a high visual importance can cause serious inconvenience to a user of the matched video content, a technology capable of matching the image without distortion of the moving object is required.

In the recent method of estimating and matching the seam tangent line, techniques for matching images while preserving moving objects have been attempted. In addition to the gradient and parallax information used when estimating the line of sight, the distortion of the object is defined in three categories (e.g., appearing, truncated, duplicated) and used when calculating the energy within the overlapping area, or an item representing the visual importance of humans. Methods such as assigning weights for each object when calculating energy using Saliency have been studied. However, these methods have a problem that they have a large amount of computation to be used in a real-time environment.

Accordingly, an object of the present invention is to provide a method and apparatus for estimating a seam tangent reflecting object motion information in order to solve the problems of the prior art. To this end, we intend to provide a technology that recognizes the movement of an object and estimates the line of sight. In addition, an object of the present invention is to provide an image matching method and apparatus using the method for estimating the seam tangent line.

In addition, it is an object of the present invention to add energy information in consideration of moving objects in addition to the outline information and parallax information of the image used when estimating the existing seam tangent line in an image acquired using a plurality of cameras and applied spatial transformation. It aims to provide the result of estimating and matching by using the estimated seam tangent line.

In addition, the present invention proposes an index capable of quantifying the degree of distortion of an object within an overlapped area in an image, and uses this to provide an efficient method and apparatus for estimating a seam tangent line compared to the existing method for estimating and matching the seam tangent line.

Other objects and advantages of the present invention can be understood by the following description, and will be more clearly understood by examples of the present invention. In addition, it will be easily understood that the objects and advantages of the present invention can be realized by the means shown in the claims and combinations thereof.

According to the present invention, the method of estimating a seam tangent to which object motion information is applied includes obtaining energy representing structures of objects and backgrounds included in two images to be matched, and estimating a seam tangent line passing through a portion with small energy. Includes.

In addition, the image matching method using the method of estimating a seam tangent to which object motion information is applied according to the present invention includes a step of estimating a seam tangent line by applying object motion, a step of updating a seam tangent line by applying an energy difference of the estimated seam tangent line, and the estimated seam tangent line. Or an image matching step using the updated seam tangent line, wherein the step of estimating the seam tangent line is a step of obtaining energy representing structures of objects and backgrounds included in the two images to be matched, and a seam tangent line passing through a portion having a small energy. And estimating.

According to an exemplary embodiment of the present invention, it is possible to estimate a seam tangent line that first reacts to an object moving within the overlapping area and does not cause distortion when matching.

Further, according to an embodiment of the present invention, it is possible to prevent a matching distortion phenomenon in a moving object having a high visual importance by adding energy reflecting the motion of the object to the energy item for estimating the seam tangent line.

In addition, according to an embodiment of the present invention, by using the object energy that can be separated from the background energy, a few pixels up, down, left and right energy can be used rather than the actual position of the object, so that the gaze line can be updated more quickly or slow motion is prevented. It is expected that various applications, such as selectively reinforcing energy in visible objects, etc., are possible.

1 is a diagram illustrating a general image matching method.
2 and 3 are diagrams illustrating a method of estimating a seam tangent and an image matching method using the same according to the present invention.
4 and 5 are diagrams for comparing the performance of a method for estimating a seam tangent and an image matching method using the same according to the present invention.
6 and 7 are flowcharts illustrating a method of estimating a seam tangent and an image matching method using the same according to the present invention.
8 is a diagram illustrating a method and apparatus for estimating a seam tangent line according to the present invention.
9 shows an image matching apparatus applicable to the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein.

In describing an embodiment of the present invention, if it is determined that a detailed description of a known configuration or function may obscure the subject matter of the present invention, a detailed description thereof will be omitted. Further, in the drawings, parts not related to the description of the present invention are omitted, and similar reference numerals are attached to similar parts.

In the present invention, components that are distinguished from each other are intended to clearly describe each feature, and do not necessarily mean that the components are separated. That is, a plurality of components may be integrated into one hardware or software unit, or one component may be distributed to form a plurality of hardware or software units. Therefore, even if not stated otherwise, such integrated or distributed embodiments are also included in the scope of the present invention.

In the present invention, components described in various embodiments do not necessarily mean essential components, and some may be optional components. Accordingly, an embodiment consisting of a subset of components described in the embodiment is also included in the scope of the present invention. In addition, embodiments including other components in addition to the components described in the various embodiments are included in the scope of the present invention. Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

It is assumed that the images used in the description of the present invention are images obtained by spatially transforming images acquired using a fisheye lens from a fixed camera capable of attaching a plurality of lenses for image acquisition using an equirectangular projection (ERP) technique. In order to synthesize an image having a wide viewing angle using images, a method of matching images captured by two adjacent cameras will be described. If this method is successively performed on neighboring cameras, an image having a viewing angle of up to 360 degrees can be synthesized. However, for convenience of explanation, the present invention is only described by using an ERP image as an example, and the technical idea of the present invention can be applied equally to other images.

First, a technical idea of the image matching method and the seam tangent estimation method according to the present invention will be described with reference to FIGS. 6 and 7. Referring to FIG. 6, the image matching method according to the present invention includes a seam tangent estimating step 610 applying an object motion, a seam tangent update step 620 applying an energy difference of the estimated seam tangent, and an estimated seam tangent or update. It may be configured to include an image matching step 630 using the seam tangent line. In addition, specifically, in the step 610 of estimating the seam tangent line applying the motion of the object, the step 611 is to obtain energy representing the structure of the object and the background included in the two images to be matched, and the seam tangent line passing through a portion having a small energy. It may be configured to include a step 612 of estimating. Hereinafter, a method for matching an image and a method for estimating a seam tangent according to an embodiment of the present invention will be described in detail.

1. Estimation of the seam tangent

In estimating the seam tangent, the energy representing the structure of the object and the background included in the two images to be matched is obtained, and then the seam tangent line passing through the small energy part is estimated. If the two images are matched at the estimated seam tangent position, it is possible to create a matched image in which the structure of an object having a large energy and the background does not deviate. The energy (E) used for estimating the seam tangent may be composed of the sum of the weights of the three energies as shown in [Equation 1] below.

[Equation 1]

?恙?

?蒻箚? 한다. 영상의 사이즈를 n X mx 라 하였을 때, (x, y)?若? 픽셀의 위치를 나타내는 벡터이고 t는 동영상의 프레임 시간을 나타낸다. Here, E _d ,Eg,E _o are three energy functions, and C _d , C _g , and C _o are weights multiplied by each energy term. Each energy function is described as follows. Each of the overlapped region images of two neighboring images

?恙?

?蒻箚? do. When the size of the image is n X mx, (x, y)?若? It is a vector representing the position of the pixel, and t represents the frame time of the video.

_{In addition, energy E d} reflecting that a point in the 3D space is acquired at different pixel positions of two images due to the parallax of the two cameras is given by [Equation 2] below.

[Equation 2]

Here, the energy E _d causes the same object and background having a large parallax to form a seam tangent so that they do not deviate from each other.

_{Regarding, the energy E g} reflecting that the outlines of the two images do not coincide with each other is given by the following [Equation 3].

[Equation 3]

Here, the energy E _g prevents the seam tangent line from being positioned at a portion where the outline exists during image matching. _{In addition, the energy E d} reflecting the motion of the object is given by the following [Equation 4].

[Equation 4]

_{The energy E d} derived from the above [Equation 4] prevents the seam tangent from being located in a portion where the temporal change of the pixel value is large according to the motion of the object.

라 할 때, 이를 다음 [수학식 5]와 같이 나타낼 수 있다. Here, the seam tangent line indicating the location where the two images are similar to each other and can be pasted together without noticeable distortion

When d, it can be expressed as the following [Equation 5].

[Equation 5]

는 시접선을 이루는 각 픽셀 에너지의 총합인

로 표현될 수 있다. 최적의 시접선을

이라 할 경우, 이는 아래 [수학식 6]에 의해 정의될 수 있다.Here, x ⁱ is a constant having a constant value between 1 and m, and i is the y-coordinate value of a continuous pixel from top to bottom. The energy of the line of sight

Is the sum of the energy of each pixel forming the line of sight

It can be expressed as Optimal seam tangent

In this case, it can be defined by [Equation 6] below.

[Equation 6]

을 연산한다. 이후, [수학식 7]에 의해 구해진 누적 최소 에너지 M을 활용하면 영상에서 중요도가 높은 영역을 시각적으로 확인할 수 있게 된다.Here, the optimal seam tangent line can be obtained by dynamic programming. Assuming that there is a cumulative minimum energy M of the same size as the total energy E, the first row is given the same value as the first row value of the total energy E. Then, by calculating the accumulated minimum energy from the second row to the last row through the following process, the energy of all possible sea tangent lines is calculated. Specifically, from the second row to the last row, the row is fixed first and the column is moved, and the following [Equation 7] is performed. When the last column is reached, the row is changed and again from the first column to the last column [Math Equation 7] is carried out. Consequently, when the operation is completed, the pixel position having the minimum value in the last row may be the last pixel position of the optimal seam tangent line. Therefore, looking at the 3 pixels adjacent to the upper row from that position, recording the position with the minimum value and moving forward

Computes Thereafter, by using the cumulative minimum energy M obtained by [Equation 7], it is possible to visually check the area of high importance in the image.

[Equation 7]

2. Sea tangent update

The seam tangent is estimated every frame through a series of processes using [Equation 7]. When images are matched using the seam tangent line estimated in all frames, image distortion due to frequent changes in the seam tangent position in the matching area can be perceived by the image observer. To prevent this, the seam tangent may be updated only when the energy difference between the estimated seam tangent line and the previously estimated seam tangent line in a given frame is large.

2 shows a case where a seam line update is unnecessary and a case where it is necessary. The seam tangent lines estimated at times t0 and t1 are indicated by solid and dotted lines, respectively. For example, in the case of (a) of FIG. 2, since there is no moving object, the image at time t0 and the image at time t1 are similar, and therefore, the energy difference between the seam tangent estimated at each of the two times is small. Therefore, in this case, the seam line update does not occur.

On the other hand, in the case of (b) of FIG. 2, the image of the position of the seam tangent estimated at times t0 and t1 is different due to the movement of the object, so that an energy difference occurs at the two positions of the seam tangent. Therefore, in this case, it is desirable to perform the seam tangent update. The energy difference between the seam tangent line estimated in the current frame and the position where the seam tangent line estimated in the previous frame passes is obtained as shown in [Equation 8] below.

[Equation 8]

The seam tangent to be used in a given frame using [Equation 8] may be determined as follows [Equation 9].

[Equation 9]

3. Image registration

Two neighboring images are matched using the estimated seam tangent and synthesized into one image. If the left and right images are used as they are based on the line of sight, the position of the line of sight is displayed as it is, but the image quality can be degraded.Therefore, the mask is calculated based on the line of sight, and left and right linear blending is performed based on the calculated boundary of the mask. After performing, the two images are combined to synthesize a matched image.

?恙?

의 일부이고 도 3 (c)는 선형 블렌딩에 이용되는 마스크 w(x,y,t)이다. 여기서, 마스크 w(x,y,t)는, 추정된 시접선의 왼쪽 픽셀이 1을 가지도록 하고, 오른쪽에 있는 픽셀들이 0을 가지도록 한 후, 자연스러운 정합 결과를 얻기 위하여 가우시안 필터 등을 이용하여 필터링을 수행할 수 있다. 상기 구해진 마스크 w(x,y,t)를 이용하여 정합 영상을 아래 [수학식 10] 과 같이 구할 수 있다. 3 shows an example of image matching using linear blending. 3(a) (b) are left and right images used for registration

?恙?

3(c) is a mask w(x,y,t) used for linear blending. Here, the mask w(x,y,t) is made so that the left pixel of the estimated seam tangent has 1 and the pixels on the right have 0, and then a Gaussian filter or the like is used to obtain a natural matching result. Filtering can be performed. Using the obtained mask w(x,y,t), a matched image can be obtained as shown in [Equation 10] below.

[Equation 10]

을 중첩 영역에서 추정된 복수의 시접선을 이용하여 연속적으로 정합 하여 최대 360도 시야각의 정합 영상을 합성할 수 있다. 이를 적용하여 상기 [수학식 10]을 변형하면 다음 [수학식 11]과 같이 정의할 수도 있다. Also, k ERP spatial transformed images acquired from k cameras

A matched image with a viewing angle of up to 360 degrees can be synthesized by successively matching by using a plurality of seam tangent lines estimated in the overlapping area. By applying this to modify the above [Equation 10], it may be defined as the following [Equation 11].

[Equation 11]

와 같은 해상도를 가진다.Here, k is a positive number greater than 1, and w ^k is a linear blending mask obtained from the results of estimating the seam tangent of the k-1 and k-th images. In this case, w ^k is the composite image

It has the same resolution as

4. Quantitative index for measuring distortion of matched image

Two images acquired from neighboring cameras have an overlapping area photographing the same space. Objects and backgrounds in the overlapping area exist in the left or right image without distortion or breakage based on the seam tangent line. Therefore, it is possible to measure image quality due to distortion and breakage of objects and backgrounds by comparing the image matched with the left and right images.

,

와 정합된 영상 을 이용하여 다음 [수학식 12]와 같이 연산할 수 있다. Video from two neighboring cameras

,

Using the image matched with can be calculated as shown in [Equation 12] below.

[Equation 12]

의 에러 맵은 아래 [수학식 13]으로 나타낼 수 있고, 이로부터 왜곡 정량 지표는 [수학식 14]를 이용하여 연산한다. In addition, as a result of the operation, an error map may be constructed where a difference between the matched image and the left or right image occurs. If this is expanded, the image matched using k ERP images

The error map of can be expressed by the following [Equation 13], and the distortion quantitative index is calculated using [Equation 14].

[Equation 13]

[Equation 14]

4 and 5 are diagrams for explaining the results of a comparative experiment between an embodiment of the present invention and a conventional method. The images used in the experiment were created by converting the ERP space for each frame of a video acquired from a 360-degree camera with a fisheye lens-equipped camera at 60-degree intervals, and each lens has an angle of view of 185 degrees. The images were captured in various background environments with the movement of the object, and using this, it is possible to check whether the proposed method forms a seam line that successfully preserves the moving object. The implementation and testing of the algorithm proposed in the present invention used MATLAB R2019a in a specific operating system (eg, WINDOWS 10). There are a total of 4 videos used in the algorithm test, and each video contains about 300 to 900 frames. The proposed seam tangent estimation and update technique using four types of images is compared with the conventional seam tangent estimation and update method, and a comparison experiment is performed by applying the matching image distortion measurement index to the matching result image.

The area used for estimating the seam tangent in the ERP transformed image is an overlapping area including the same space in two adjacent images. A total of six overlapping regions are used to match images acquired from a camera in which six fisheye lens cameras are arranged at intervals of 60 degrees, and the left and right images of the overlapping regions are acquired from fixed coordinates experimentally confirmed.

In the conventional method, in order to estimate the seam tangent line, only the outline and the parallax energy item among the three energy items described in the detailed description section of the present invention are used. That is, in order to perform the seam tangent line update, the seam tangent line is estimated in a new frame, and an outline energy difference for each pixel from the existing seam tangent line is calculated. The number of pixels whose calculated outline energy difference exceeds a predetermined value or more is recorded, and when the number of pixels exceeds 0.3 times the total number of pixels on the seam line, the update is performed.

4 is a part of the results of a comparison experiment, and shows a continuous frame in which an update occurs by a moving object. Here, Figs. 4 (a) and (b) show the conventional algorithm, and Figs. 4 (c) and (d) show the case of applying the method proposed in the present invention. As can be seen in Figure 4. As a result of the conventional algorithm, Figs.4 (a) and (b) show that the update operation is performed after a little distortion by the object occurs, and this is when updating, there is a difference in the outline energy value in more than a certain number of pixels. It is judged because it should be done. On the other hand, the result of the proposed algorithm of the present invention shown in Figs. 4 (c) and (d) is proposed to be updated when the total amount of change in the summation line energy exceeds a certain amount, and the energy by the moving object is higher than that of the existing algorithm. Therefore, it can be seen that the update by the moving object appears quickly.

In addition, FIG. 5A shows an energy map, FIG. 5B shows a cumulative minimum energy map, and each column of FIG. 5 corresponds to FIG. 4. That is, the maximum value normalization is applied for each image, and the region with a high value is strongly displayed. Through this, it can be confirmed that the moving object has higher energy and importance compared to the background in the algorithm proposed in the present invention.

Although the estimation method using the seam tangent line is mainly used for image matching, it is difficult to prevent distortion in the image due to the estimated seam tangent line passing through the object only with energy items that treat the energy of the object and the background equally. In order to overcome this, the present invention prevents a matching distortion phenomenon in a moving object having a high visual importance by adding energy reflecting the motion of the object to the energy item for estimating the seam tangent line. To prove this, an index that quantitatively represents the distortion of the matched image was presented, a comparison experiment was performed, and it was confirmed that the overall match quality of the video was improved through the distortion quantitative index value. Image matching techniques based on the line-of-sight line estimation use object energy that can be separated from the background energy, and use the energy of several pixels up, down, left, and right rather than the actual position of the object, so that the line-of-sight line update can occur faster, or slow motion is prevented. It is expected that various applications, such as selectively reinforcing energy in visible objects, etc., are possible.

8 shows another embodiment according to the present invention. In particular, FIG. 8 is a block diagram and a control flow diagram of a technique for estimating a seam tangent line using object motion information according to another embodiment of the present invention.

As described above, after performing spatial transformation such as ERP using images obtained from two or more cameras to match images, a method of intersecting and matching images based on the line of sight is mainly used. However, in the commonly used seam tangent estimation technique, when a moving object approaches the seam tangent line, it cannot estimate the seam tangent line reflecting the motion information, and as a result, the seam tangent line passes through the object and often causes distortion of the matched image. Since the distortion of the moving object causes inconvenience to the users of the matched video content, it is necessary to develop a technology for estimating the seam tangent line formed avoiding the moving object.

Referring to the embodiment of FIG. 8, the apparatus for estimating a seam tangent line of the present invention includes a gradient calculation unit 801, 802, a difference image calculation unit 811, 812, a dilation calculation unit 821, and a shake calculation unit 831. , An energy sum calculation unit 841 and a time tangent estimating unit 851. The functions and actions of each component will be described in detail as follows.

연산을 수행한다.First, the gradient calculation units 801 and 802 calculate energy based on a gradient between input images using partial derivatives. Here, the gradient calculator 801 calculates energy Eg(t-1) at time t-1, and the gradient calculator 802 calculates energy Eg(t) at time t. ^{Specifically, when receiving the left and right images I A} and I ^B corresponding to the overlapping region as inputs, the gradient calculation units 801 and 802 calculate partial derivatives in the x-axis and y-axis directions.

Perform the operation.

연산 수행한다.The difference image calculation units 811 and 812 calculate a difference between input images. _{Here, the difference image calculation unit 811 is to obtain the difference image energy E o} (t) from the results of the above-described gradient calculation units 801 and 802, and the difference image calculation unit 812, the input image x _L (t ) And the difference image energy E _d (t) _{between x R (t).} Specifically, the difference image calculation units (811, 812) for the left and right images

Perform the operation.

연산 수행한다. 여기서 p 는 적은 값의 상수이고, i,j 는 영상의 픽셀 위치를 의미한다. _{The Dilation calculation unit 821 calculates energy E od} (t) obtained by slightly moving the image left and right from the result of the difference image calculation unit 811. Specifically, the Dilation calculation unit 821 is for a given energy image E _o

Perform the operation. Here, p is a small-valued constant, and i,j is the pixel position of the image.

를 수행하여 E_ohs를 계산하고, 위 아래로

연산 수행한다. 이때 p_h, p_v는 상기 기술된 p 보다 큰 상수이며, i,j 는 영상의 픽셀 위치이다. _{In addition, the Shake calculation unit 831 calculates energy E os} (t) obtained by moving the image largely up, down, left and right from the result of the difference image calculation unit 811. Specifically, the Shake calculation unit 831 left and right for a _{given energy image E o}

To _{calculate E ohs} , up and down

Perform the operation. In this case, p _h and p _v are constants greater than p described above, and i, j are pixel positions of the image.

을 가장 첫 행부터 아래로 내려가며 수행하여 이를 이용하여 합 에너지 연산 수행한다. 여기서 i,j 는 영상의 픽셀 위치이다.The energy sum calculation unit 841 calculates the sum energy E(t) by adding the _{energy values E d} (t), E _od (t), and E _{os (t) obtained in the above-described process.} Specifically, when the energy sum calculation unit 841 adds by using the weighted sum of the aforementioned energies, and the resulting image is e,

The sum energy calculation is performed by going down from the first row and using it. Where i,j is the pixel position of the image.

In addition, the seam tangent line estimating unit 851 estimates the seam tangent line by using a low energy region from the obtained sum energy E(t). Specifically, the seam tangent estimating unit 851 continuously records the location with the smallest value among the pixel values of the top left, the top right, and the top from the location having the smallest value in the bottom row of the completed energy E. Advance to and estimate the seam tangent line.

9 is a diagram illustrating an image matching apparatus using a method for estimating a seam tangent to which object motion information is applied, as another embodiment according to the present invention. For example, the image matching device 1100 according to an embodiment of the present invention may include a memory 1110, a processor 1120, a transmission/reception unit 1130, and other peripheral devices 1140. In this case, as an example, the memory 1110 may be a non-removable memory or a removable memory. In addition, as an example, the peripheral device 1240 may be a display, a GPS, or other peripheral device, and is not limited to the above-described embodiment. In addition, as an example, the image matching apparatus 1100 may include a wired/wireless communication circuit or a communication interface like the transmission/reception unit 1130, and may perform communication with an external device based on this. The processor 1120 may perform an image matching time tangent estimation process and/or an image matching process using the above-described object motion information.

Various embodiments of the present disclosure are not listed in all possible combinations, but are intended to describe representative aspects of the present disclosure, and matters described in the various embodiments may be applied independently or may be applied in combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. For implementation by hardware, one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general purpose It may be implemented by a processor (general processor), a controller, a microcontroller, a microprocessor, or the like. For example, the processor may be implemented as software written in a computer-readable programming language, and may take various forms including the general-purpose processor. It is obvious that it may be disclosed as hardware consisting of one or more combinations.

The scope of the present disclosure is software or machine-executable instructions (for example, operating systems, applications, firmware, programs, etc.) that allow an operation according to a method of various embodiments to be executed on a device or a computer, and such software or It includes a non-transitory computer-readable medium (non-transitory computer-readable medium) which stores instructions and the like and is executable on a device or a computer.

The present invention described above, for those of ordinary skill in the art to which the present invention pertains, various substitutions, modifications and changes are possible within the scope of the technical spirit of the present invention, so that the scope of the present invention is described above. It is not limited by one embodiment and the accompanying drawings.

1110: memory
1120: processor
1130: transmitting and receiving unit
1140: peripheral device

Claims (1)

The step of estimating the seam tangent line applying object motion,
A seam tangent update step by applying the energy difference of the estimated seam tangent line, and
Including an image matching step using the estimated seam tangent line or the updated seam tangent line,
The seam tangent estimating step includes obtaining energy representing structures of objects and backgrounds included in the two images to be matched, and estimating a seam tangent line passing through a portion having a small energy. Image matching method using estimation method.

Images