KR101838207B1 - Apparatus for distinguishing similarity, and calculation method for calculation metrix correlation distance - Google Patents

Abstract

Provided are a similarity determination apparatus and a distance calculation method. The similarity determination apparatus can improve the reliability of a distance calculation result by reflecting a frame sequence length difference between images which can act as a noise component when calculating a distance between feature matrixes representing a feature dimension for each image frame as a matrix. The similarity determination apparatus comprises: a processing part; a contribution calculation part; and a distance calculation part.

Description

[0001] APPARATUS FOR DISTINGUISHING SIMILARITY, AND CALCULATION METHOD FOR CALCULATION [0002] METRIX CORRELATION DISTANCE [

The present invention relates to a method for reflecting a difference in frame sequence length between images in calculating a correlation distance between feature matrices representing a feature dimension for each frame of the image by a matrix.

Similarity is a measure of the degree of resemblance between individuals. Similarity in pattern recognition is expressed numerically, and geometrically refers to a value that is inversely proportional to the distance between two points in the same space .

Generally, data for pattern recognition is given as a video signal, that is, an image. In order to analyze such an image, it is necessary to display the image in a form easy to recognize.

Accordingly, the image can be represented by a matrix of two-dimensional matrix data, that is, a feature matrix for each frame, and pattern recognition in such an image is a method for determining the similarity between feature matrices, that is, a correlation distance between feature matrices And the like.

In order to calculate the distances between the feature matrices, the sizes of the two feature matrices must be equal to each other, since the frame sequence of the image is preserved in the form of a row in the feature matrix, ≪ / RTI >

However, when the frame sequence lengths are matched with each other, a redundant frame exists in the image, and such a duplicated frame has a problem of acting as a noise component in the distance calculation.

The present invention has been made in view of the above circumstances, and an object to be achieved by the present invention is to provide a method and apparatus capable of acting as a noise element in calculating a correlation distance between feature matrices, The frame sequence length difference between the images.

According to another aspect of the present invention, there is provided a similarity determination apparatus for determining a similarity between a comparison image and a reference image, A processing unit for processing frame sequence alignment so that a frame sequence length becomes the same between an image and the reference image; Wherein at least one of a row and a column of each of a first feature matrix representing the comparison image and a second feature matrix representing the reference image is generated based on a frame sequence length difference between the comparison image and the reference image A calculating unit for calculating a contribution to be made; And a calculation unit for calculating a distance between the first feature matrix and the second feature matrix by reflecting the contribution to data values of rows and columns of the first feature matrix and the second feature matrix, respectively .

More specifically, the calculating unit may calculate the contribution of each of the first feature matrix and the second feature matrix to have a row and a column, respectively, as the frame sequence length difference becomes larger between the comparison image and the reference image. do.

More specifically, each of the comparison image and the reference image is a difference image obtained by subtracting a pixel value of each frame from a pixel value of a neighboring previous frame.

According to an aspect of the present invention, there is provided a distance calculation method for determining a similarity degree between a comparison image and a reference image, wherein when there is a frame sequence length difference between the comparison image and the reference image, A processing step of processing frame sequence alignment so that a frame sequence length becomes equal between an image and the reference image; A calculating step of calculating a contribution degree of each of a first feature matrix representing the comparison image and a second feature matrix representing the reference image based on a frame sequence length difference between the comparison image and the reference image; And calculating distances between the first feature matrix and the second feature matrix by reflecting the contribution of data values of a row and a column of each of the first feature matrix and the second feature matrix And a calculation step.

More specifically, the calculating step calculates a lower contribution degree that a row and a column are included in each of the first feature matrix and the second feature matrix as the frame sequence length difference becomes larger between the comparison image and the reference image .

More specifically, each of the comparison image and the reference image is a difference image obtained by subtracting a pixel value of each frame from a pixel value of a neighboring previous frame.

The distance calculation method according to an embodiment of the present invention is characterized by having the form of a computer program stored in a computer-readable recording medium, which is implemented to execute each step of the above-described method.

A distance calculation method according to an embodiment of the present invention is characterized by having the form of a computer-readable recording medium including a command for executing each step of the above-described method.

According to the similarity determination apparatus and the distance calculation method according to the present invention, in calculating the distances between the feature matrices representing the feature dimensions for each frame of the image by the matrix, the difference in frame sequence length between the images, , The reliability of the distance calculation result can be enhanced.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram showing a similarity determination environment according to an embodiment of the present invention; FIG.
FIG. 2 is an exemplary diagram for explaining frame sequence length difference according to an embodiment of the present invention; FIG.
3 is a schematic configuration diagram of a similarity determination device according to an embodiment of the present invention;
4 is an exemplary diagram for explaining a contribution according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating an operation flow in the similarity determination device according to an embodiment of the present invention; FIG.

Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

FIG. 1 illustrates a similarity determination environment according to an embodiment of the present invention.

1, the similarity degree determination environment according to an embodiment of the present invention is characterized in that a degree of similarity is determined by calculating a distance between two feature matrices (X _i , X _j ) having a T-by-D size And a similarity determination device 100 for calculating the similarity degree.

Here, a feature matrix defined according to an embodiment of the present invention is a representation of one image (moving image) in the form of a matrix, and can be defined as, for example,

[Equation 1]

For example, if the length of the data is 'T ₁ ', the feature matrix 'A' can be expressed as [Formula 2] below.

[Equation 2]

Here, 'T' denotes a data sequence length, that is, a frame sequence of an image, and D denotes a number of feature (object) dimensions to be pattern recognition objects in a frame.

That is, in the feature matrix according to an exemplary embodiment of the present invention, a row corresponds to one frame in an image, and a column refers to a feature dimension.

Accordingly, the feature matrix according to an exemplary embodiment of the present invention can be defined as a feature vector in which one frame is a feature vector and is sequentially stacked according to a frame sequence.

Further, in one embodiment of the present invention, the distance between two feature matrices' X _i 'and X _j ' is defined as follows.

[Equation 3]

Also, the values defined by [Equation 3] can be expressed as in Equation 4 below.

[Equation 4]

Here, 'X _{i, d} ' and 'X _{j, d} ' are the dth column vectors of the two feature matrices, and 'X ^{i, t} ' and 'X ^{j, t} ' Which means a row vector.

On the other hand, in the feature matrix according to an embodiment of the present invention to a sequence of frames of the image to calculate the inter-row (Row) feature matrix (X _i, X _j) as preserved in the form of a distance two feature matrix (X _i , And X _j need to be equal to each other.

Here, matching the sizes of the two feature matrices (X _i , X _j ) equally can be understood as matching the frame sequence lengths between the images corresponding to the two feature matrices (X _i , X _j ) Can be processed through DTW (Dynamic Time Warping).

However, when the frame sequence lengths are matched with each other, there is a redundant frame in the image. The overlapping frame acts as a noise element in the distance calculation and is expressed by Equation 3 and Equation 4, _row 'and' d _col 'calculations.

As a result, it is necessary to minimize the influence of the overlapping frames generated by the dynamic time alignment (DTW) processing in order to improve the reliability of the distance calculation between the feature matrices (X _i , X _j ).

For reference, FIG. 2 shows the frame sequence length of each operation of the three operation databases, and it can be confirmed that there is a difference in the average frame sequence length for each operation.

Table 1 below shows the frame sequence length difference between the same class samples and the frame sequence length difference with other class samples in each database in the following Table 1, A difference in frame sequence length between the frame sequence length and the frame sequence length can be confirmed.

Data (video) Frame sequence length difference between the same classes (average) Frame sequence length difference with other classes (average) Cambridge Hand Gesture 14.0881 15.2780 ChaLearn 9.7056 13.2517 SKIG 5.7627 13.1014

Accordingly, in an embodiment of the present invention, a method for reflecting the difference in frame sequence length in calculating the distances between the feature matrices (X _i , X _j ) is proposed. Hereinafter, the similarity determination device 100 The configuration will be described in detail.

FIG. 3 shows a schematic configuration of a similarity determination apparatus 100 according to an embodiment of the present invention.

3, the similarity determination apparatus 100 according to an exemplary embodiment of the present invention includes a processing unit 110 for processing frame sequence alignment, a calculation unit 120 for calculating a contribution, and a feature matrix X _i , X _j ), and a calculation unit 130 for calculating the inter-vehicle distance.

All or at least a part of the similarity determination apparatus 100 including the processing unit 110, the calculation unit 120 and the calculation unit 130 may be implemented in the form of a hardware module or a software module, Modules may be implemented in a combined form.

Here, the software module may be understood as, for example, a command executed by a processor that processes an operation in the similarity determination apparatus 100, and the command may be stored in a memory in the similarity determination apparatus 100 or in a form stored in a separate memory Lt; / RTI >

As a result, the similarity determination apparatus 100 according to an embodiment of the present invention can calculate the distance between two feature matrices X _i and X _j by reflecting the frame sequence length difference through the above-described configuration. Hereinafter, Will be described in more detail with reference to FIG.

Hereinafter, for convenience of explanation, the feature matrix 'X _i ' is referred to as a 'first feature matrix' in which the feature dimension for each frame of the comparison image is expressed by a matrix, and the feature matrix 'X _j ' And a second feature matrix in which the feature dimension of each reference frame of the reference image is expressed by a matrix.

In addition, according to an exemplary embodiment of the present invention, a comparison image and a reference image expressed by a first feature matrix X _i and a second feature matrix X _j , respectively, Value subtracted from the difference image.

For example, from an image composed of T + 1 frames, a difference image composed of T frames can be obtained through subtraction of pixel values between frames.

The processing unit 110 performs a function of processing frame sequence alignment.

In more detail, when there is a frame sequence length difference between the comparison image and the reference image in relation to the determination of the similarity degree between the comparison image and the reference image, the processing unit 110 determines that the frame sequence length is the same between the comparison image and the reference image, Process the alignment.

At this time, the processing unit 110 processes the frame sequence alignment between the comparison image and the reference image through a dynamic time warping (DTW) method, thereby making the frame sequence length equal between the comparison image and the reference image.

The calculation unit 120 calculates the contribution degree associated with the determination of the degree of similarity.

More specifically, when the frame sequence alignment is performed between the comparison image and the reference image, the calculating unit 120 calculates the difference between the first feature matrix X _i and the second feature matrix X _i based on the frame sequence length difference between the comparison image and the reference image. The contribution degree of at least one of the row and the column of each of the feature matrices X _j is calculated.

At this time, the calculation unit 120 checks the frame sequence length difference between the comparison image and the reference image to calculate the contribution, and the difference of the frame sequence length can be defined as an absolute value as shown in [Equation 5] below.

[Equation 5]

'로 한정되며, 이때의 값은 아래 [수식 6]와 같이 정의될 수 있다.As described above, the frame sequence length difference that is verified is scaled based on the first feature matrix X _i , and the frame sequence length difference, which has various values,

Quot ;, and the value at this time can be defined as " (6) "

[Equation 6]

Finally, when the scaling of the difference of the frame sequence length is completed, the calculating unit 120 calculates the first characteristic matrix X _i and the second characteristic matrix X _j through the following equation (7) reflecting the scaled value: It is possible to calculate the contribution that at least one of each row and column has.

[Equation 7]

As a result, it can be seen that the contribution calculated according to an exemplary embodiment of the present invention has a value between 1/4 and 1/2 as shown in FIG. 4, and the frame sequence length difference between the comparison image and the reference image is It can be understood that the larger the value, the closer to 1/4 is calculated.

For reference, the fact that the frame sequence length difference between the comparison image and the reference image is large indicates that as the frame sequence length difference between the comparison image and the reference image increases, the number of overlapping frames increases in the frame sequence alignment process .

In one embodiment of the present invention, as the frame sequence length difference between the comparison image and the reference image increases, the contribution of at least one of a row and a column is calculated to be low, And reflects the effect of overlapping frames.

The calculation unit 130 calculates a distance between the feature matrices.

More specifically, when the calculation of the contribution of the rows and columns of the first feature matrix X _i and the second feature matrix X _j is completed, the calculation unit 130 calculates the first feature matrix X _i , second feature matrix by reflecting the calculated contribution for the data values having a row and a column of the (X _j), the first characteristic matrix is calculated a distance between the (X _i) and the second feature matrix (X _j).

At this time, the calculation unit 130 obtains the first feature matrix X _i and the second feature matrix X _j ( _i , _j ) through a method that reflects the calculated contribution to the calculation formula defined by Expressions 3 and 4, ). The similarity function equation for reflecting the contribution can be defined as [Equation 8].

[Equation 8]

On the other hand, according to the above [formula 8], the final effect of the first feature matrix (X _i) and the second feature matrix (X _j) contribution (υ, 1/4 ~ 1 /2) is reflected in the line from each from the contribution (1-υ, 1/4 ~ 1/2) may verify that is larger than the effect which the frame is advanced [formula 3] to be duplicated in the frame sequence alignment process, [formula 4] is reflected column 'd _col' more 'd _row' to further greatly reduce the influence of 'd _row' so effect a more significant impact on the calculated friction.

As described above, according to the configuration of the similarity degree determination unit 100 in accordance with one embodiment of the present invention, a first feature image of the characteristic dimension of the comparative image to the matrix the matrix (X _i) and matrix characteristics dimension of the reference image (X _j ) by reflecting the difference in frame sequence length between the images that can act as a noise component, it is possible to reduce the number of overlapping frames generated due to the frame sequence alignment between the comparison image and the reference image Since the influence can be minimized, the effect of increasing the reliability of the distance calculation result can be achieved.

Hereinafter, an operation flow in the similarity determination apparatus 100 according to an embodiment of the present invention, that is, a method of calculating a distance between feature matrices for determining similarity, will be described with reference to FIG.

First, in accordance with steps 'S10' to 'S30', when there is a frame sequence length difference between a comparison image and a reference image in relation to the determination of similarity between the comparison image and the reference image, The frame sequence alignment is processed such that the frame sequence length becomes equal.

At this time, the processing unit 110 processes the frame sequence alignment between the comparison image and the reference image through a dynamic time warping (DTW) method, thereby making the frame sequence length equal between the comparison image and the reference image.

If the frame sequence alignment is performed between the comparison image and the reference image, the calculating unit 120 limits the frame sequence length difference between the comparison image and the reference image to a value within a specific range in accordance with steps 'S40' and 'S50' And the contribution of at least one of the row and column of each of the first feature matrix X _i and the second feature matrix X _j is calculated by reflecting the finally scaled value, .

At this time, the calculation unit 120 calculates the number of rows and columns of the first feature matrix X _i and the second feature matrix X _{j as} the frame sequence length difference between the comparison image and the reference image increases, The contribution of at least one of them is calculated to be low.

For reference, the fact that the frame sequence length difference between the comparison image and the reference image is large indicates that as the frame sequence length difference between the comparison image and the reference image increases, the number of overlapping frames increases in the frame sequence alignment process .

Therefore, in one embodiment of the present invention, as the frame sequence length difference between the comparison image and the reference image is increased, the contribution of at least one of the row and the column is calculated to be low, By reflecting the contribution, the effect of overlapping frames is canceled.

When the calculation of the contribution of the rows and columns of the first feature matrix X _i and the second feature matrix X _j is completed, the calculation unit 130 calculates the contribution of the first feature matrix X _i and the second feature matrix X _j according to steps S60 and S70 The difference between the first feature matrix X _i and the second feature matrix X _j is calculated by reflecting the calculated contribution to the data values of the rows and columns of the feature matrix X _i and the second feature matrix X _j . Calculate the distance.

If the frame sequence length difference does not exist between the comparison image and the reference image in step S20, the calculation unit 130 calculates the first feature matrix _Xi and the second feature matrix _Xj , The distance between the first feature matrix X _i and the second feature matrix X _j can be calculated by equally reflecting the predefined basic contribution (for example, 0.5) to the data values of the rows and columns of the first feature matrix X _i .

As described above, according to the operation flow of the similarity determination apparatus 100 according to an embodiment of the present invention, the feature matrix of the comparison image is expressed by a matrix of a first feature matrix X _i , And the second feature matrix X _j expressing the first feature matrix X _j by reflecting the difference in frame sequence length between the images that can act as a noise component, Since the influence of the frame can be minimized, the effect of improving the reliability of the distance calculation result can be achieved.

It should be understood that the functional operations and subject matter implementations described herein may be implemented as digital electronic circuitry, or may be embodied in computer software, firmware, or hardware, including the structures disclosed herein, and structural equivalents thereof, . Implementations of the subject matter described herein may be implemented as one or more computer program products, i. E. One or more modules relating to computer program instructions encoded on a type of program storage medium for execution by, or control of, the operation of the processing system Can be implemented.

The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that affects the machine readable propagation type signal, or a combination of one or more of the foregoing.

As used herein, the term " system "or" device "encompasses any apparatus, apparatus, and machine for processing data, including, for example, a programmable processor, a computer or a multiprocessor or computer. The processing system may, in addition to the hardware, comprise code that forms an execution environment for a computer program upon request, such as, for example, code comprising a processor firmware, a protocol stack, a database management system, an operating system, .

A computer program (also known as a program, software, software application, script or code) may be written in any form of programming language, including compiled or interpreted language, a priori or procedural language, Components, subroutines, or other units suitable for use in a computer environment. A computer program does not necessarily correspond to a file in the file system. The program may be stored in a single file provided to the requested program, or in multiple interactive files (e.g., a file storing one or more modules, subprograms, or portions of code) (E.g., one or more scripts stored in a markup language document). A computer program may be deployed to run on multiple computers or on one computer, located on a single site or distributed across multiple sites and interconnected by a communications network.

On the other hand, computer readable media suitable for storing computer program instructions and data include semiconductor memory devices such as, for example, EPROM, EEPROM and flash memory devices, such as magnetic disks such as internal hard disks or external disks, Non-volatile memory, media and memory devices, including ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuits.

Implementations of the subject matter described herein may include, for example, a back-end component such as a data server, or may include a middleware component, such as an application server, or may be a web browser or a graphical user, for example a user, who may interact with an implementation of the subject- Front-end components such as client computers with interfaces, or any combination of one or more of such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication, such as, for example, a communications network.

While the specification contains a number of specific implementation details, it should be understood that they are not to be construed as limitations on the scope of any invention or claim, but rather on the description of features that may be specific to a particular embodiment of a particular invention Should be understood. Likewise, the specific features described herein in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments, either individually or in any suitable subcombination. Further, although the features may operate in a particular combination and may be initially described as so claimed, one or more features from the claimed combination may in some cases be excluded from the combination, Or a variant of a subcombination.

It is also to be understood that although the present invention is described herein with particular sequence of operations in the drawings, it is to be understood that it is to be understood that it is to be understood that all such illustrated acts have to be performed or that such acts must be performed in their particular order or sequential order, Can not be done. In certain cases, multitasking and parallel processing may be advantageous. Also, the separation of the various system components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the described program components and systems will generally be integrated together into a single software product or packaged into multiple software products It should be understood that

As such, the present specification is not intended to limit the invention to the specific terminology presented. Thus, while the present invention has been described in detail with reference to the above examples, those skilled in the art will be able to make adaptations, modifications, and variations on these examples without departing from the scope of the present invention. The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

According to the similarity determination apparatus and the distance calculation method according to an embodiment of the present invention, a correlation distance calculation is performed between feature matrices representing a feature dimension for each frame of an image by reflecting a frame sequence length difference between images It is not only the use of the related technology but also the possibility of commercialization or operation of the applied device as well as the possibility of being practically and practically possible because it exceeds the limit of the existing technology.

100:
110: processing unit 120:
130:

Claims (8)

In case of a similarity between the comparison image and the reference image, if there is a difference in frame sequence length between the comparison image and the reference image, the frame sequence alignment is processed so that the frame sequence length becomes equal between the comparison image and the reference image A processor;
Wherein at least one of a row and a column of each of a first feature matrix representing the comparison image and a second feature matrix representing the reference image is generated based on a frame sequence length difference between the comparison image and the reference image A calculating unit for calculating a degree of contribution of the first feature matrix and the second feature matrix to each other as the frame sequence length difference between the comparison image and the reference image increases; And
And a calculation unit for calculating a distance between the first feature matrix and the second feature matrix by reflecting the contribution to data values of rows and columns of the first feature matrix and the second feature matrix, . delete The method according to claim 1,
Wherein each of the comparison image and the reference image comprises:
Wherein the difference image is a difference image obtained by subtracting a pixel value of each frame from a pixel value of a neighboring previous frame. In case of a similarity between the comparison image and the reference image, if there is a difference in frame sequence length between the comparison image and the reference image, the frame sequence alignment is processed so that the frame sequence length becomes equal between the comparison image and the reference image Processing step;
Wherein at least one of a row and a column of each of a first feature matrix representing the comparison image and a second feature matrix representing the reference image is generated based on a frame sequence length difference between the comparison image and the reference image Calculating a contribution of the first feature matrix and the second feature matrix to each other as the frame sequence length difference between the comparison image and the reference image increases; And
And calculating the distance between the first feature matrix and the second feature matrix by reflecting the contribution to data values of the rows and columns of the first feature matrix and the second feature matrix, respectively . delete 5. The method of claim 4,
Wherein each of the comparison image and the reference image comprises:
Wherein the difference image is a difference image obtained by subtracting a pixel value of each frame from a pixel value of a neighboring previous frame. A computer program stored in a computer readable recording medium, the computer program being embodied to execute the steps of claim 4 or 6. A computer-readable recording medium containing instructions for executing the steps of claim 4 or claim 6.

Images