RU2535184C2 - Method and apparatus for detecting local features on image - Google Patents

Abstract

FIELD: physics, computer engineering.

SUBSTANCE: invention relates to means of processing digital images. In the method, an image in the RGB colour space is converted to a grayscale; processed with Laws energy characteristics by folding the matrix of the source image with nuclei of the type of detecting an image in the form of ripples, detection of spots, detection of waves and detection of edges; for each of the obtained image, spots, corners and line edges are detected by a SURF method to find a local point using Hessian matrix.

EFFECT: detection of local features on an image.

2 cl, 2 dwg

Description

The invention relates to methods for processing digital images and can be used in registration and surveillance systems, multimedia applications working with visual data.

, $$j=\overline{\mathrm{1,}J}$$

- столбцы и строки изображения, $$k=\overline{1.3}$$

- цветовой канал. При реализации метода поиска локальных особенностей используется изображение в градациях серого, т.е. каждая точка представлена уровнем яркости в диапазоне от 0 (черный) до 255 (белый), с промежуточными значениями, представляющими различные уровни серого. Соответственно такое изображение обозначим, как S_i,j.A simplified mathematical image model is a model in the RGB color space in the form of an array S _{i, j, k} , where $$i=\overline{\mathrm{one,}I}$$

, $$j=\overline{\mathrm{one,}J}$$

- columns and rows of the image, $$k=\overline{1.3}$$

- color channel. When implementing the method of searching for local features, an image in grayscale is used, i.e. each dot is represented by a brightness level ranging from 0 (black) to 255 (white), with intermediate values representing different gray levels. Accordingly, we denote such an image as S _{i, j} .

The main problem to be solved is the detection of local features in the image with certain texture properties.

Despite the fact that the image represents a simple structure - the matrix of two-dimensional numbers contains more information about the observed scene. Retrieving structured information from this scene is a daunting task. When it comes to the sequence of images, the task becomes even more complicated, as spatio-temporal relationships between frames appear. Thus, devices and methods are required that will extract and analyze the information embedded in the video sequence. One such approach is to search, track, and map point features on a frame sequence. This is one of the easiest ways to extract scene dynamics information. Several points tracked in a video sequence can provide a wealth of information.

Simplified methods for detecting local features in the image can be divided into the following groups:

1) Ways to detect angles.

2) Methods for detecting points.

3) Methods for spot detection.

4) Methods for detecting edges.

An analysis of existing detection methods shows that these methods work only with a certain type of local features, which is their main drawback. Using methods for detecting angles based on the Moravek method leads to the fact that if the angle is not directed towards the neighboring one, then it will not be detected as a point feature. The Harris detector is not invariant to zooming and the Shi-Tomashi angle search algorithm is very sensitive to noise and to choosing a scale level. The use of methods based on point detection shows that the SURF (Speeded Up Robust Features) method does not work with objects of simple shape and without pronounced boundaries, and the SIFT (Scale Invariant Feature Transform) method does not work with mirror surfaces and is sensitive to changes in lighting. Using edge detection methods based on the Kirsch operator requires large computational costs, which limits its use in real time. Methods for detecting spots (areas) based on the MSER (Maximally Stable Extremal Regions) region detector leads to the fact that it is not scale invariant with low contrast, but normalizes all 6 parameters of affine transformations. The FAST area detector (Features accelerated segment testing) detects local features quickly enough.

A known method and device for detecting an object in the image [US Patent No. 6711293, IPC G06K 9/68]. The invention relates to methods for recognizing objects in machine vision systems.

The main technical task is to recognize objects in the image using an invariant scale function. At the first stage, the image is convolved with the Gaussian function, then the Gauss pyramid is used to construct the difference image. At the second stage, local extrema are found by comparing each pixel of the image with several neighboring pixels of a given scale. At the next stage, an area around the points of extremum is distinguished. At the fourth, the direction of the gradients of a certain neighborhood is calculated, and at the fifth stage, for each extremum point, a local descriptor is calculated.

The signs of the method-analogue, coinciding with the signs of the proposed technical solution, are as follows:

- invariance to zooming;

- finding local extremes.

The disadvantages of the known method and device that implements it are:

- low accuracy of determining local descriptors.

The reasons that impede the achievement of the required technical result are as follows:

- since the Gaussian function has a limited set of invariant functions characterizing the image features described by feature vectors, and thus the vector data are less informative.

A known method of recognition of objects [Patent No. 2438174 C1, IPC G06K 9/68]. The invention relates to methods for recognizing objects in machine vision systems, television surveillance systems, information and control systems of robotic complexes.

The main technical problem solved by the claimed invention is to create a method that can improve recognition accuracy by improving the stability of the detectors of key areas in the image and increasing the number of invariant characteristics of these detectors. The method is as follows: the input image is minimized with the given Green function:

$$U\left(y,\tau |\; |\; |y_0,{\tau }_0\right)=\frac{\sqrt{\lambda }}{\sqrt{D\pi \left(\mathrm{one}-e^{-2\lambda \left(\tau -{\tau }_0\right)}\right)}}*\exp \left[-\frac{\sqrt{\lambda }}{D\pi \left(\mathrm{one}-e^{-2\lambda \left(\tau -{\tau }_0\right)}\right)}{\left(y-y_0{e}^{-\lambda \left(\tau -{\tau }_0\right)}\right)}^2\right]$$

for the evolutionary operator of the Ornstein-Uhlenbeck process for various values of the parameters τ, λ.

, $${\partial }_{\tau }U\left(y,\tau \right)={\partial }_y\left(\lambda yU\left(y,\tau \right)\right)+\frac{\mathrm{<figure-callout\; id="2"\; label="D"\; filenames="00000028.png"\; state="\{\{state\}\}">D</figure-callout>}}{2}{\partial }_{yy}U\left(y,\tau \right)$$

,

where τ, λ are the parameters used to determine convolutions at various scales;

D is a constant that is selected empirically for various categories of images;

y = (a, b) is the spatial time function, the essence of the coordinate (a, b) of the pixel in the image.

The resulting convolutions are subtracted from each other to obtain a finite-difference approximation of the first derivative of the convolution of the input image with the filter. When searching for a local extremum of a given convolution, the corresponding first derivatives are equated to zero. All local extremes are found and adaptive threshold filtering is performed to cut off minor features. Points distinguished in this way serve as centers of neighborhoods for which arbitrary descriptors are constructed.

The signs of the method-analogue, coinciding with the signs of the proposed technical solution, are as follows:

- changing the symmetry set by choosing the corresponding Green's function, to obtain feature vectors for the same image that reflect various properties of the image, thereby increasing the information content of the image.

The disadvantages of the known method and device that implements it are:

- detection of features in the form of local areas of extremum.

The reasons that impede the achievement of the required technical result are as follows:

- the use of the convolution method with the Green function does not allow to detect various types of textures.

A known method for highlighting the contours of moving objects [Patent No. 2466456 C2, IPC G06K 9/50]. The invention relates to the field of pattern recognition and can be used in vision systems for solving problems of preliminary image processing.

The main technical problem solved by the claimed invention is to increase the accuracy of determining the areas of moving objects and increase the speed of selection of image contours.

The block diagram of the device includes:

- selection of the contours of moving objects, including the detection of moving pixels according to the principle of inter-frame subtraction;

- determining the direction of motion of the detected pixels, taking into account the direction of motion of adjacent pixels;

- the formation of moving objects by combining adjacent pixels with one direction and the presence of pixels in an eight-connected neighborhood;

- spatial differentiation of detected objects by the Sobel operator:

$$\begin{array}{l}{G}_r=〈G\left(x,y\right),v\left(x,y\right)〉=f_{ed}\left(S\left(k\right),I\left(k\right)\right)=\\ 〈\sqrt{{\left(\frac{\delta I\left(x,y\right)}{\partial x}\right)}^2}+{\left(\frac{\delta I\left(x,y\right)}{\partial y}\right)}^2,arctg\left(\alpha \right)〉\end{array}$$

$$\alpha =\left(\raisebox{1ex}{$\left(\frac{\partial I\left(x,y\right)}{\partial y}\right)$}\!\left/ \!\raisebox{-1ex}{$\left(\frac{\partial I\left(x,y\right)}{\partial x}\right)$}\right.\right),\forall \left(x,y\right)\in S\left(k\right)$$

- skeletonization of contour lines in a gradient image by suppressing non-maximum brightness points, threshold processing of a skeletal gradient image based on the maximum and minimum of entropy.

The signs of the method-analogue, coinciding with the signs of the proposed technical solution, are as follows:

- selection of the contours of objects in the image.

The disadvantages of the known method and device that implements it are:

- low accuracy of determining local descriptors.

The reasons that impede the achievement of the required technical result are as follows:

- spatial differentiation by the Sobel operator in the presence of textured objects leads to significant false detections.

A known method for detecting singular points and a device that implements it [US Patent No. 2007/0071289 A1, IPC G06K 9/00]. The invention relates to the detection of point features and a method for detecting local features of a face.

The main technical problem solved by the claimed invention is to create a device and method for correctly detecting local features in a face image with one point normalization and multi-point normalization of pattern recognition.

The structural diagram of a device for detecting local features includes:

- a storage device configured to store the first pattern for the first local point of the object, the second pattern for the second local point, and the third pattern for the combination of the first and second local points;

- image input device;

- block candidate;

- the first pattern recognition unit, extracts a plurality of third candidates from the plurality of first candidates based on similarities between the first candidate and the first template, and also extracts a plurality of fourth candidates from the plurality of second candidates based on similarities between each second candidate and the second template;

- the second pattern recognition unit, creates many of the first combinations of every third candidate, and every fourth candidate is extracted by the second combination of the many first combinations based on the third similarity between the first combination and the third template.

The signs of the method-analogue, coinciding with the signs of the proposed technical solution, are as follows:

- detection of features in the form of local areas of extremum.

The disadvantages of the known method and device that implements it are:

- the invention is intended only for the detection of point features of local features of the face.

A known method for detecting angles based on the Moravec method [Moravec N (1980). "Obstacle Avoidance and Navigation in the Real World by a Seeing Robot Rover"]. This method checks every pixel in the image to determine if that pixel is an angle by looking at areas in the pixel area. Similarity is determined by taking the sum of the squared differences between the two sections. A smaller number indicates a greater similarity.

The method is as follows:

1. The fact that the homogeneous and ribbed portions of the image is self-similar is used.

2. In each pixel, the self-similarity function of neighborhoods with a different radius for this pixel is considered.

3. A self-similarity map is built.

4. Local mismatch maxima of various pixel neighborhoods are selected.

5. If the local maximum is greater than the threshold, then the point is classified as angular:

, $$E_{x,y}={\displaystyle {\sum }_{u,v}{w}_{u,v}{\left[I_{x+u,y+u}-I_{u,v}\right]}^2}$$

,

where I is the image intensity; w is a window that defines an image area with a coefficient of unity.

The signs of the method-analogue, coinciding with the signs of the proposed technical solution, are as follows:

- detection of local features in the form of angles.

The disadvantages of this method are:

- if the angle is not directed towards the adjacent one, then it will not be detected as a point feature.

A known method of detecting angles using the Hough transform [Shapiro L., J. Stockman. Computer vision; Per from English. - M .: BINOM. Laboratory of Knowledge, 2006. - S. 413]. This Hough transform method uses a battery array whose dimension corresponds to the number of unknown parameters in the equation of the family of desired curves. Angle regions in the image can be detected by searching for pairs of selected edge segments E1 and E2 that satisfy the conditions listed below:

- approximating straight lines for the sets of boundary points E1 and E2 intersect at the point [u, v], the coordinates of which are defined in the actual coordinate system of the image;

- the point [u, v] lies close to the extreme points of the sets E1 and E2;

- the directions of the gradients E1 and E2 are symmetrical about the axis of symmetry of these two edges.

The original edge segments are extracted using the Hough transform by tracing the boundaries and then approximating them by straight lines or some other suitable algorithm. For each pair of segments ([d1, θ1], [d2, θ2]) satisfying the above conditions, the four elements (d1, θ1], [d2, θ2], [u, v], α) are added to the set of candidates for angles (where α is the value of the angle). The formed set of angular characteristic features is used for various purposes, for example, for constructing any high-level descriptions.

Battery array A is used in the Hough algorithm to check each pixel in the image and its surroundings. Determines whether a sufficiently pronounced edge is present in a given pixel. If present, then the parameters of the desired curve passing through the given pixel are calculated. After evaluating the parameters of the line in this pixel, they are sampled to obtain the corresponding values of M and B, and the value of the array element A [M, B] increases. After processing the pixels, local maxima are searched in the battery array. The points of local maxima correspond to the parameters of the most probable lines in the image.

The signs of the method-analogue, coinciding with the signs of the proposed technical solution, are as follows:

- detection of local features in the form of angles.

The disadvantages of this method are:

- the above conditions determine only angles of type 'L';

- the second condition excludes from consideration the angles of type 'T', 'X' and 'Y';

- the battery array allows you to determine the parameters of the infinitely long straight or curved lines, but with its help it is impossible to determine where in the image the line segments begin and end.

A known method of Blob detection based on Laplacian Gaussian [Lindeberg T. (1998). "Feature detection with automatic scale selection". International Journal of Computer Vision 30 (2): pp.77-116], which is based on a one-time digital operation of band-pass filtering of the spatial function of an image using the Laplacian Gaussian operator.

This method uses the commutativity property of the linear Gauss and Laplace operators:

, $$L\times \left(G\times I\right)=\left(L\times G\right)\times I=LoC\times I$$

,

where × is the convolution operator and the possibility of combining them into one by changing the order of differentiation and smoothing operations. The combined operator is called the Laplacian Gaussian operator LoG (LoG - Laplacian of Gaussian) and is a convolution of the Laplace and Gaussian operators.

The signs of the method-analogue, coinciding with the signs of the proposed technical solution, are as follows:

- detection of local features in the form of regions.

The disadvantages of this method are:

- Gauss and Laplace operators are non-directional;

- sensitivity to brightness changes in the parallel direction, which reduces the signal-to-noise ratio.

A known method for detecting edges by the Kirsch method [Shapiro L., J. Stockman. Computer vision; Per from English. - M .: BINOM. Laboratory of Knowledge, 2006. - S. 413]. The Kirsch operator is a nonlinear edge detector that finds the maximum edge thickness in several directions. Taking the mask, the operator rotates it in 8 main directions of the compass: north, northwest, west, southwest, south, southeast, east, northeast.

The method is as follows:

The Kirsch method works with a two-dimensional 3 × 3 aperture of the following form:

A0 A1 A2 A7 F A3 A6 A5 A4

First, all the values of the variables S _i and T _{i are found} , where i varies from 0 to 7, two sums are calculated, one of which contains 3 elements, sequentially selected during the round-trip of the extreme elements included in the window, the second sum includes the remaining five elements:

$$S_i={\displaystyle \sum _{j=i}^{\left(i+2\right)\mathrm{mod}8}{A}_j}$$

$$T_i={\displaystyle \sum _{j=i+3}^{\left(i+7\right)\mathrm{mod}8}{A}_j}$$

Further, the values of the sums are compared using the weighted difference modulo, which determines the presence or absence of contrast in the considered direction:

$$Y_i=|\; |\; |5S_i-3T_i|\; |\; |$$

The maximum contrast difference is selected from all directions, the direction is changed by calculating all possible differences, sequentially changing the starting position of the initial element (i):

$$F\text{'}\text{'}=\underset{i=0.7}{\max \left\{Y_i\right\}}$$

The resulting value is written to the point with the value F, the filter shifts to the right.

The features of the analog device, matching the features of the claimed technical solution, are as follows:

- detection of local features in the form of boundaries.

The disadvantages of this method are:

- A large amount of time is required to calculate local features.

A known method for detecting points using the SIFT method (Scale Invariant Feature Transform) [David G. Lowe "Distinctive image features from scale-invariant keypoints" International Journal of Computer Vision, 60, 2 (2004), pp. 91-110]. The main step in the detection of singular points is the construction of a pyramid of Gaussians (Differences of Gaussian, DoG).

The method is as follows: a point is considered singular if it is a local extremum of the difference of the Gaussians. To search for extrema we will use the following method. In each image from the DoG pyramid, local extremum points are searched. Each point of the current DoG image is compared with its eight neighbors and with nine neighbors in the DoG, located one level higher and lower in the pyramid. If this point is larger (less) than all neighbors, then it is taken as a point of local extremum.

The next step will be a couple of checks on the suitability of the extremum point for the key role. First of all, the coordinates of the singular point are determined with subpixel accuracy. This is achieved by approximating the DoG function by the second-order Taylor polynomial taken at the point of the calculated extremum.

, $$D\left(x\right)=D+\frac{\partial D^T}{\partial x}x+\frac{\mathrm{one}}{2}{x}^T\frac{{\partial }^{2}D}{\partial x^2}x$$

,

here D is the DoG function, X = (x, y, sigma) is the displacement vector with respect to the decomposition point, the first DoG derivative is the gradient, the second DoG derivative is the Hessian matrix.

The extremum of the Taylor polynomial is found by calculating the derivative and equating it to zero. As a result, we obtain the displacement of the point of the calculated extremum relative to the exact

, $$\hat{x}=-\frac{{\partial }^2{D}^{-\mathrm{one}}}{\partial x^2}\frac{\partial D}{\partial x}$$

,

больше 0.5*шаг сетки в этом направлении, то это означает, что на самом деле точка экстремума была вычислена неверно и нужно сдвинуться к соседней точке в направлении указанных компонент. Для соседней точки все повторяется заново. Если таким образом выходим за пределы октавы, то следует исключить данную точку из рассмотрения.All derivatives are calculated by finite difference formulas. If one of the components of the vector $$\hat{X}$$

more than 0.5 * grid step in this direction, this means that in fact the extremum point was calculated incorrectly and you need to move to an adjacent point in the direction of the indicated components. For a neighboring point, everything repeats again. If in this way we go beyond the octave, then this point should be excluded from consideration.

When the position of the extremum point is calculated, the DoG value is compared with the threshold at the point by the formula:

$$D\left(\hat{x}\right)=D+\frac{\mathrm{one}}{2}\frac{\partial D^T}{\partial x}\hat{x}$$

If this check fails, then the point is excluded from consideration, as a point with low contrast.

The signs of the method-analogue, coinciding with the signs of the proposed technical solution, are as follows:

- detection of local features in the form of regions.

The disadvantages of this method are:

- does not work with reflective surfaces.

Closest to the invention relates to a method of SURF (Speeded Up Robust Features) [Bay H., Tuytelaars T. (2006) Ess A. Speeded-Up Robust Features (SURF) in LNCS 3951 (ECCV'06), vol. 1, 2006, pp. 404-417]. The SURF method allows you to find local points using the Hessian matrix and well detects spots and angles. Hessian is rotation invariant, but not scale invariant. Therefore, the SURF method uses multiscale filters to find the Hessians.

For each local point, the direction of the maximum brightness change (gradient) and the scale taken from the scale factor of the Hessian matrix are considered. The gradient at the point is calculated using Haar filters. The detection of singular local points in the SURF method uses a binarized approximation of the Laplacian-Gaussian.

The features of the prototype method, which coincides with the features of the proposed technical solution, are as follows:

- detection of local features in the form of areas and angles.

The disadvantages of this method are:

- partially invariant to lighting and position of optical cameras;

- Does not work with reflective surfaces and a pronounced geometric structure.

Closest to the invention is a method and system for detecting faces [Patent No. 2008145913/28, IPC G06K 9/00]. Consider a prototype device involves the following operations:

1) The mapping of gradients of the input image is performed, the result of which is the calculation of changes in the intensity (or color components) of neighboring pixels or groups and estimates the direction of the largest change at each point in the image;

2) A search is made for circular arcs on gradient maps, the main result of which is to obtain a multidimensional weight map of possible locations of the face, taking into account the geometric parameters of the arcs: radii, length and location;

3) A search is made for local features in the incoming image and gradient maps, the result of the work will be a set of selected local features with their geometric characteristics;

4) The construction of a set of hypotheses of possible positions of persons is carried out, the output information is a set of hypotheses, each of which determines the possible position of a person in the original image, taking into account its size and orientation;

5) Verification of the constructed hypotheses is carried out.

A device for detecting faces includes: an input image registration (memory) unit; block for constructing gradient maps of the incoming image; block search for circular arcs on gradient maps; block search for local features in the incoming image and gradient maps; a block for constructing a set of hypotheses of possible positions of persons; preprocessing unit; block face detector; hypothesis verification block.

The disadvantages of the known prototype device are:

- detecting certain local features in order to detect the location of the face in the image.

The reasons that impede the achievement of the required technical result are as follows:

- the SURF method can act as a detection of local features, respectively, such a method detects only local features in the form of regions.

The essence of the proposed method for detecting local features in the image is as follows (figure 1).

In the first step, the original image, which is represented by the model in the RGB color space, is converted to grayscale to implement a method for searching for local features.

At the second step, a local feature of interest is selected by processing the Laws textural energy characteristics [Laws K. Textured image segmentation, Ph.D. dissertation, University of Southern California. 130 pp. - 1980].

To calculate the energy characteristics, a set of 22 masks with dimensions 5x5 is used. Then the energy characteristics of each pixel of the analyzed image are presented in the form of a vector and 9 numbers. The following vectors are used to calculate masks:

$$L5\left(Level\right)=\left[\begin{array}{ccccc}\mathrm{one}& \mathrm{four}& 6& \mathrm{four}& \mathrm{one}\end{array}\right]$$

$$E5\left(Edge\right)=\left[\begin{array}{ccccc}-\mathrm{one}& -2& 0& 2& \mathrm{one}\end{array}\right]$$

$$S5\left(Spot\right)=\left[\begin{array}{ccccc}-\mathrm{one}& 0& 2& 0& -\mathrm{one}\end{array}\right]$$

$$E5\left(Ripple\right)=\left[\begin{array}{ccccc}\mathrm{one}& -\mathrm{four}& 6& -\mathrm{four}& \mathrm{one}\end{array}\right]$$

$$E5\left(Wave\right)=\left[\begin{array}{ccccc}-\mathrm{<figure-callout\; id="2"\; label="one"\; filenames="00000028.png"\; state="\{\{state\}\}">one</figure-callout>}& 2& 0& -2& \mathrm{one}\end{array}\right]$$

The name of the vector describes their purpose. Vector L5 is designed to calculate a symmetric weighted local mean value. Vector E5 is for detecting edges, S5 is for detecting spots, R5 is for detecting an image in the form of ripples, and W5 is for detecting waves. Two-dimensional masks are calculated by multiplying pairs of vectors. For example, to get the mask E5L5, you need to multiply the vector E5 by L5:

$$\left[\begin{array}{c}-\mathrm{one}\\ -2\\ 0\\ 2\\ \mathrm{one}\end{array}\right]\times \left[\begin{array}{ccccc}\mathrm{one}& \mathrm{four}& 6& \mathrm{four}& \mathrm{one}\end{array}\right]=\left[\begin{array}{ccccc}-\mathrm{one}& -\mathrm{four}& -6& -\mathrm{four}& \mathrm{one}\\ -2& -8& -12& -8& -2\\ 0& 0& 0& 0& 0\\ 2& 8& 12& 8& 2\\ \mathrm{one}& \mathrm{four}& 6& \mathrm{four}& \mathrm{one}\end{array}\right]$$

Each texture energy map is a full-sized image that represents the results of processing the input image using the kth mask.

After receiving 22 energy cards, some symmetric pairs are combined and as a result 13 final cards are built. Each symmetric pair of cards is replaced by an average card. For example, the mask E5L5 characterizes the content of horizontal edges, Leller - vertical. The average of these two cards will characterize the presence on the image of the edges of both of these types. We will list 13 final energy cards: Leller / E5L5, L5S5 / S5L5, W5W5, L5W5 / W5L5, L5R5 / R5L5, E5Е5, ESWSIWSES, E5S5 / Seller, E5R5 / Raser, RSWSIW5RS, S5S5, S5R5555S5R55555S5R5555S5R55S5R55S5R55S5R55S5R55S5R55S5R55S5R55S5R5555S5R555S5R5555S5R5555S5R55S5R55555S5R55S5R555555

Thus, using Loves filters, for any pixel we get a description of the texture characteristics in a certain neighborhood of the selected pixel.

In the third step, the resulting image is processed by the SURF method. The SURF method allows you to find local points using the Hessian matrix and well detects spots, corners and edges of lines. Hessian is rotation invariant, but not scale invariant. Therefore, the SURF method uses multiscale filters to find the Hessians.

For each local point, the direction of the maximum brightness change (gradient) and the scale taken from the scale factor of the Hessian matrix are considered. The gradient at the point is calculated using Haar filters.

To efficiently calculate the Hessian and Haar filters, the integral representation of the images is used.

The integral representation is a matrix, the dimension of which coincides with the dimension of the original image, and the elements are calculated by the formula:

$$II\left(x,y\right)={\displaystyle \sum _{i=0y=0}^{i\le x,j\le y}I\left(x,y\right)}$$

where I (i, j) is the brightness of the pixels of the original image.

Having an integrated matrix, you can very quickly calculate the sum of the brightness of the pixels of arbitrary rectangular areas of the image by the formula:

$$SumOf\mathrm{Re}ct=II\left(A\right)+II(C)-II(B)-II(D)$$

where ABCD is the rectangle of interest.

Detection of singular local points in the SURF method uses a binarized approximation of the Gaussian Laplacian:

. $$\det \left(H{}_{appros}\right)=D_{xx}{D}_{yy}-{\left(0.9D_{xy}\right)}^2$$

.

The proposed method allows to detect various types of local features due to preliminary processing by the energy characteristics of Loves.

The device for detecting local features in the image (Fig. 2) contains a 2D camera 1, the output of which is connected to the input of the image registration unit 2, the output of which is connected to the following inputs: the input of the gradient mapping unit 3, the output of which is connected to the input of the image convolution unit with the core edge detection 4.1, to the input of the image convolution unit with the image detection core in the form of ripples 4.2, to the input of the image convolution unit with the spots detection kernel 4.3, to the input of the image convolution unit with the waves detection kernel 4.4, outputs of which ryhs are connected to the inputs of the search blocks of local features 5.1, 5.2, 5.3, 5.4, respectively, in each channel, the outputs of which are connected to the inputs of the storage blocks 6.1, 6.2, 6.3, 6.4; the synchronization of the device is provided by the clock generator 7.

A device for detecting local features in an image is implemented as follows. The image from the camera enters the image registration unit, which acts as a memory, then the image is converted to grayscale in the gradient mapping unit and enters in parallel into four channels, in each of which the image matrix is collapsed with different cores that characterize the texture features of the image. The resulting texture energy maps, in turn, are sent in parallel to the inputs of the local feature search blocks, in which the local image features are searched for using the SURF detection method. The result of the blocks will be a set of distinguished local features with their geometric characteristics.

A device for detecting local features in the image works as follows. The image from the 2D camera 1 is transmitted to the image registration unit 2, from which the image enters the gradient mapping block 3, the result of which is a change in the color image in grayscale. The resulting image in grayscale is simultaneously sent to: the input of block 4.1, the result of which is a texture energy map of convolution of the image with the edge detection core; to the input of block 4.2, the result of which is a texture energy map of convolution of the image with the image detection core in the form of ripples; to the input of block 4.3, the result of which is a texture energy map of convolution of the image with the spot detection core; to the input of block 4.4, the result of which is a texture energy map of convolution of the image with the wave detection core. The resulting texture energy maps, in turn, are sent simultaneously to the inputs of the local feature search blocks 5.1, 5.2, 5.3, 5.4, respectively, in which the local features of the image are searched for using the SURF detection method. A set of distinguished local features with different texture and geometric characteristics is stored in blocks 6.1, 6.2, 6.3, 6.4.

The technical result is the detection of local features in the image with certain texture properties.

Claims (2)

1. A method for detecting local features in an image, which consists in searching for local features using the SURF method, characterized in that the resulting image in the RGB color space is converted to grayscale, and then processed by the Loves energy characteristics using convolution of the source image matrix with detection type kernels image in the form of ripples, spot detection, wave detection, and edge detection, for each received image, local features such as spots are detected, ly, SURF method of finding local line edge point using the Hessian matrix. 2. A device for detecting local features in an image containing a 2D camera, the output of which is connected to the input of the image registration unit, the output of which is connected to the input of the gradient mapping block; characterized in that the device contains four image processing channels by convolution of the matrix of the specified image, the output of the gradient mapping block is connected to the inputs of the feature blocks characterizing the convolution of the image matrix with the cores whose outputs are connected to the inputs of the local feature search blocks in each channel, respectively, the outputs of which connected to the inputs of the storage units, synchronization of all units of the device is carried out by a clock generator.

Images