WO2017181892A1

WO2017181892A1 - Foreground segmentation method and device

Info

Publication number: WO2017181892A1
Application number: PCT/CN2017/080274
Authority: WO
Inventors: 邓硕; 马华东; 罗圣美; 傅慧源; 刘培业
Original assignee: 中兴通讯股份有限公司
Priority date: 2016-04-19
Filing date: 2017-04-12
Publication date: 2017-10-26
Also published as: CN107305691A

Abstract

A foreground segmentation method and device, the method comprising: respectively extracting local characteristic information about two input images, and matching key points according to the extracted local characteristic information; screening a mismatching point from matching points among obtained key points to obtain all the correct matching points; performing cluster analysis to obtain, from all the correct matching points, a characteristic point group on a foreground object; and according to the obtained characteristic point group, using an image segmentation method to obtain the foreground object in the image. The embodiments of the present invention improve the accuracy of a foreground object in an image, reduce foreground processing time, and improve the efficiency of image processing.

Description

Foreground segmentation method and device

Technical field

This document relates to, but is not limited to, image processing technology, and in particular to a foreground segmentation method and apparatus.

Background technique

Foreground Extraction refers to extracting foreground objects of arbitrary shape from a static image or a burst of video images. The foreground extraction technique in the related art requires the user to mark foreground pixels or regions, and by analyzing the pixels of the region, Get a rough outline of the target in the image.

Currently, the most commonly used foreground extraction schemes include the following:

1) Significance detection

By extracting the saliency map model of the image from the global features such as color, brightness, and direction of the image, it is possible to reflect the region of the image that is most likely to cause user interest and best represent the image content. The problem of significant feature detection comes from computer simulation of human vision in order to achieve the ability of the human eye to select objects. Low-level vision plays an important role in the saliency detection model, such as color, direction, brightness, texture, and edges.

The human eye is more sensitive to the color information of the image than other visual features, so the statistics of the color features are especially important in computer vision. Two methods for color feature calculation are widely used in saliency detection: the first is to create a color histogram and then compare the difference between the histograms; the second is to block the image and put each image The internal color average is compared with other color patches to obtain color saliency; brightness is also the most basic visual feature in the image. In the saliency detection model, when the luminance feature is calculated, the luminance component of the local feature region is extracted. Statistical values are used to represent the overall brightness characteristics of the region, and then the brightness of the image is obtained by comparison with other regions; the directional features reflect the essential features of the surface of the object, and the directional feature calculation in the saliency detection of the image is mainly Gabor energy method. It can well simulate the multi-channel and multi-resolution features of the human visual system.

The salient feature is based on the global features of the image, which can well simulate the features of the human eye's region of interest, but has the following disadvantages: First, the selection of salient regions is very subjective, due to the needs of different users, the same image is of interest. The region may have greater differences; secondly, significant The global feature based on the image is less robust to local changes in the target. And in the application, the method needs to manually intervene to mark the global feature block of the target area. In the case of a simple image processing, the method has a practical space. However, with the development of search engines and networks, the capacity of data has exploded, and a small number of image processing methods are far from meeting the urgent needs of users, and it is difficult to show qualified results in a huge image database because of manual intervention.

2) Frame difference method

The motion regions in the image are typically extracted using a difference between adjacent frame images in the sequence of images. The image sequence of two adjacent frames is grayed out, and then corrected in the same coordinate system. When the difference operation is performed, the background portion where the gradation does not change will be clipped off. Since the region of interest is mostly a moving target, the contour of the region where the gradation changes, that is, the approximate contour of the region of interest, can be obtained through a difference operation. Thereby determining the foreground image.

The adjacent frame difference method can well solve the foreground extraction problem in the simple scene video sequence, but since the adjacent frame difference method requires input of consecutive video adjacent frame sequences, it is difficult to apply to the processing of still images. in. Secondly, for complex backgrounds or changing backgrounds, the frame difference method is less robust.

It can be seen from the above that the method for acquiring the approximate foreground region based on the image salient features proposed for the still image utilizes the global feature of the image, and cannot take into account the local details of the image, and the robustness is poor. Due to the complexity of the background, the similarity of the image, etc., the foreground contour of the object may have small flaws, so it is necessary to improve the accuracy of the algorithm again.

Summary of invention

The following is an overview of the topics detailed in this document. This Summary is not intended to limit the scope of the claims.

The embodiment of the invention provides a foreground segmentation method and device, which can improve the accuracy of automatic foreground segmentation of image matching.

The embodiment of the invention discloses a foreground segmentation method, which comprises:

Extracting local feature information of two input images respectively, and matching key points according to the extracted local feature information;

Screen out the mismatched points from the matching points of the obtained key points to get all the correct matching points;

Use cluster analysis to derive feature point groups on foreground targets from all correct matching points;

According to the obtained feature point group, the image segmentation algorithm is used to obtain the foreground target in the image.

Optionally, in the foregoing foreground segmentation method, extracting local feature information of the two input images includes:

The two images input by the user are grayed out, and the local feature information of the image is extracted by using the accelerated robust feature SURF algorithm.

Optionally, in the foregoing foreground segmentation method, performing key point matching according to the extracted local feature information includes:

And determining, according to the extracted local feature information, a matching point corresponding to a key point in the first input image in the second input image by using a neighbor algorithm.

Optionally, in the foregoing foreground segmentation method, the incorrect matching points are filtered out from the matching points of the obtained key points, and all correct matching points are obtained:

Configuring a scale ratio SR and a direction OA, and calculating a match between a key point in the first input image and a key point in the first input image in the second input image according to the result obtained by the key point matching The scale ratio of the points and the direction scale, and construct the following two-dimensional array:

P={<SR ₁ , OA ₁ >, <SR ₂ , OA ₂ >...<SR _n , OA _n >}

among them,

OA _n =θ _n -θ _n '

Perform screening of matching points;

Wherein the ratio of s _n to s _{n '} is a scale ratio of a key point in the first input image to a matching point of the key point in the first input image in the second input image, and a logarithm is obtained to obtain a scale ratio; _n and θ _^n, the difference of the first input key image and key image point of the first input matching points in the second direction of the input image difference.

Optionally, in the foregoing foreground segmentation method, using cluster analysis to derive feature point groups on the foreground target from all the correct matching points includes:

The following algorithm is used to randomly select the cluster centroids of the k clusters as

μ ₁ , μ ₂ ,...,μ _k ∈R ⁿ ,

For each of the samples i, the distance to the k seed points is calculated, and the point closest to the seed point μ _n belongs to the μ _n point group, wherein the Euclidean distance in the 128-dimensional SIFT feature space is calculated according to the following formula:

Moving the μ _n seed point of each point group to the center of the μ _n point group,

Repeating the calculation of the distance of each μ _n seed point until the center of each class is gradually stabilized, obtaining a pre-attraction group and a background seed point group, and obtaining the obtained pre-attraction group and background seed point group as the feature point group;

Wherein, S _i represents a one-dimensional feature of the SIFT feature, and R ⁿ represents that the heart points of the selected K clusters belong to a set of n clusters randomly taken from the point set.

The embodiment of the invention further discloses a foreground segmentation device, comprising:

The first unit is configured to separately extract local feature information of the two input images, and perform matching of the key points according to the extracted local feature information;

The second unit is configured to filter out the wrong matching points from the matching points of the obtained key points to obtain all correct matching points;

The third unit is configured to use cluster analysis to derive feature point groups on the foreground target from all the correct matching points;

The fourth unit is configured to obtain a foreground target in the image by using an image segmentation algorithm according to the obtained feature point group.

Optionally, in the foregoing foreground segmentation device, the first unit is configured to extract local feature information of the two input images, including:

Optionally, in the foregoing foreground segmentation device, the first unit is configured to perform key point matching according to the extracted local feature information, including:

Determining the first input of the two input images using a neighbor algorithm based on the extracted local feature information The key points in the image are the corresponding matching points in the second input image.

Optionally, in the foregoing foreground segmentation device, the second unit is configured to filter out the wrong matching points from the matching points of the obtained key points, and obtain all correct matching points including:

P={<SR ₁ , OA ₁ >, <SR ₂ , OA ₂ >...<SR _n , OA _n >}

among them,

OA _n =θ _n -θ _n '

Perform screening of matching points.

Optionally, in the foregoing foreground segmentation device, the third unit is configured to use the cluster analysis to derive the feature point group on the foreground target from all the correct matching points, including:

μ ₁ , μ ₂ ,...,μ _k ∈R ⁿ ,

The distance of each μ _n seed point is repeatedly calculated until the center of each class is gradually stabilized, and the front spot group and the background seed point group are obtained, and the obtained front spot group and background seed point group are taken as the feature point group.

Compared with the related art, the technical solution provided by the embodiment of the present invention includes: separately extracting local feature information of two input images, performing key point matching according to the extracted local feature information; and screening from the matching points of the obtained key points. Get all the correct matching points except the wrong matching points; use poly The class analysis derives the feature point group on the foreground target from all the correct matching points; according to the obtained feature point group, the image segmentation algorithm is used to obtain the foreground target in the image. The embodiment of the invention improves the accuracy of the foreground object in the image, reduces the time of foreground processing, and improves the efficiency of image processing. The technical solution of the embodiment of the invention can objectively obtain the foreground target in the image, so that the result is more accurate and intuitive, can replace the traditional human-computer interaction method, reduces the overall time, improves the efficiency, and can be obtained in the experimental data set. Good experimental results. At the same time, the problem of local feature information loss in the image is solved, and the robustness of the method is improved. Compared with the adjacent frame difference method, especially for static images, the accuracy of the foreground segmentation contour is improved.

Other aspects will be apparent upon reading and understanding the drawings and detailed description.

BRIEF abstract

1 is a flowchart of a foreground segmentation method according to an embodiment of the present invention;

2 is a schematic diagram of extracting local feature information of an image according to an embodiment of the present invention;

3 is a schematic diagram of an image obtained after cluster analysis processing according to an embodiment of the present invention;

4 is a schematic diagram showing a separation of a foreground object and a background according to an embodiment of the present invention;

5 is a schematic diagram of test images and foreground segmentation results using an embodiment of the present invention;

FIG. 6 is a structural block diagram of a foreground segmentation apparatus according to an embodiment of the present invention.

Detailed

Embodiments of the present application will be described in detail below with reference to the accompanying drawings. It should be noted that, in the case of no conflict, the features in the embodiments and the embodiments in the present application may be arbitrarily combined with each other.

Example 1

The inventors of the present application have found that the automatic foreground segmentation scheme in the related art mainly focuses on feature extraction of video continuous frames, or combined with user intervention and global feature foreground image extraction. However, the embodiment of the present invention proposes that the dual image joint automatic foreground extraction method can be adopted, that is, the local features of the image are extracted, the foreground region is obtained through feature point matching and cluster analysis, and then the image segmentation algorithm is used to implement the segmentation method for the foreground image. Among them, the local features include: some features that appear locally, which can be stably present and have some points that are well distinguishable. Different from global features such as variance and color, local features can better summarize the information carried by the image and reduce the amount of calculation. Improve the anti-interference ability of the algorithm.

Based on the foregoing, the embodiment provides a foreground segmentation method, as shown in FIG. 1 , including:

Step 100: Extract local feature information of two input images respectively, and perform matching of key points according to the extracted local feature information;

Optionally, extracting local feature information of the two input images includes:

The two images input by the user are grayed out, and the local feature information of the image is extracted using the accelerated robust feature (SURF) algorithm.

Optionally, performing key point matching according to the extracted local feature information includes:

Step 200: Screen out the wrong matching points from the matching points of the obtained key points to obtain all correct matching points;

Optionally, the wrong matching points are filtered out from the matching points of the obtained key points, and all correct matching points are obtained:

P={<SR ₁ , OA ₁ >, <SR ₂ , OA ₂ >...<SR _n , OA _n >}

among them,

OA _n =θ _n -θ _n '

Perform screening of matching points;

Step 300, using cluster analysis to derive features on the foreground target from all correct matching points Point group

Optionally, using cluster analysis to derive the feature point groups on the foreground target from all the correct matching points includes:

The following algorithm is used to randomly select the heart points of k clusters as:

μ ₁ , μ ₂ ,...,μ _k ∈R ⁿ ,

Wherein, S _i represents a one-dimensional feature of the SIFT feature, and R ⁿ represents that the heart points of the selected K clusters belong to a set of n clusters randomly taken from the point set. That is, R ⁿ indicates that the heart points of the K clusters are selected, and all belong to the clusters clustered by randomly taking n from a large point set.

For each of the samples i, the distance to the k seed points is calculated, and the point closest to the seed point μ _n belongs to the μ _n point group, wherein the Euclidean in the 128-dimensional scale-invariant feature transform SIFT feature space is calculated according to the following formula distance:

Step 400: Obtain a foreground target in the image by using an image segmentation algorithm according to the obtained feature point group.

Wherein, step 100 can be summarized as feature matching; comprising two parts operation: 1. extracting local feature information of two input images respectively; and performing key point matching according to the extracted local features;

The local features of the image involved in this embodiment are different from the global features of the image, and are features that appear locally. In the case where the object is occluded, some feature points that are still stable can easily and accurately describe the features of the clothing image, such as Harris, SIFT, SURF, FAST (the image matching method existing in the related art).

Step 200 can be summarized as a matching point screening;

Wherein, the wrong matching point can be filtered according to the scale ratio of the matching point and the rotation direction ratio;

Step 300 can be summarized as foreground image extraction.

Wherein, cluster analysis is first used to obtain a group of feature points on the foreground target;

Then, according to the obtained feature point group, the image segmentation algorithm is used to obtain the foreground target in the image.

It should be noted that the cluster analysis involved in the embodiment refers to a process of classifying data into different classes or clusters, and the data in the same cluster has a high similarity; and between different clusters, The data is very different. It is an unsupervised learning that does not rely on pre-defined classes or labeled training examples, such as k-means (already existing in the related art).

The image segmentation involved in the present embodiment is a technique and process for dividing an image into a plurality of specific regions having unique properties and proposing objects of interest. Such as based on threshold segmentation, region-based segmentation, edge-based segmentation, and segmentation based on specific theory.

The cluster analysis is used in this embodiment because the analysis can convert the abstract key point information into the foreground area, thereby providing support for the next image segmentation technology, and the joint application of the image matching technology and the segmentation technology to achieve the The traditional artificial interaction of image segmentation technology has improved.

In the embodiment of the present invention, a clustering algorithm is performed on the original features of the original input image to obtain a suggested region of the foreground object, and finally the overall foreground segmentation of the image is performed by the graph cut method. include:

Step 1: Enter image feature matching; includes:

Local feature extraction: The image input by the user is grayed out. The local feature information of the image is extracted by using the SURF (Speed Up Robust Feature) feature. FIG. 2 is a schematic diagram of extracting local feature information of the image according to an embodiment of the present invention. As shown in FIG. 2, the image input by the user is grayed out and used. SURF feature extraction obtains local feature information of the image.

Matching of key points: In the matching of key points, we use the neighbor algorithm to determine the matching points of the key points in the input image A in the image B. Taking the K-Nearest Neighbor (KNN) classification algorithm as an example, first set a parameter K. Calculate the key point feature in image A and the Euclidean distance in B, and maintain a queue of size K in order of distance from largest to smallest for storing the nearest neighbor training element. The training element ancestor is traversed, the distance between the key point of the current element ancestor and the A key point is calculated, and the obtained distance is compared with the maximum distance Lmax. If L>=Lmax, discard this A ancestor, traversing the next ancestor. If L<=Lmax, the ancestor of the largest distance is deleted, and the ancestor is added to enter the k queue. After the traversal is completed, the matching points in the B picture of the same category as the key points in A can be obtained.

The second step: screening of matching points;

There are many errors in the key points of the preliminary matching. Because of the complexity, diversity and similarity of the target background, the embodiment of the present invention screens the results obtained in the first step to obtain better results. The foreground target matches the point area.

In this step, we propose a unique key point screening method, in which we set two measurement parameters: scale ratio SR (Scale Rate) and direction OA (Orientation Rate). According to the result of the key point matching, the scale ratio and the direction ratio of the key points in the graph A and the key points in the graph A in the graph A are calculated, and a two-dimensional array is constructed:

P={<SR ₁ , OA ₁ >, <SR ₂ , OA ₂ >...<SR _n , OA _n >}

OA _n =θ _n -θ _n '

The matching points are filtered by the constructed two-dimensional data. In the matching process of image feature points, it is found that the feature points on the same object tend to maintain the same scale change and direction change. Therefore, by processing P, the region with large distribution of the two-dimensional array can be obtained. For the area of the matching point on the foreground target, the interference point of the background (wrong matching point) is removed in this way.

The third step: foreground image extraction;

First, cluster analysis of matching points is performed.

This step is a core step of the embodiment of the present invention. The method of the embodiment of the present invention applies the method of data clustering analysis to the homogeneity analysis of the key points, and the image feature matching method and the image segmentation method can be well organic. Combination of. Through the screening of the previous step, the combination c ⁽ⁱ⁾ of matching key points in image A is obtained. Due to the complexity of the image background, it is highly probable that the matching key points contain interference matching points similar to the foreground target key points. In order to provide the foreground pixel seed for the next image segmentation, the embodiment of the present invention uses the K-means clustering analysis algorithm to group the key points obtained in the previous step to obtain the key points of the foreground target and improve the image segmentation. The degree of accuracy. Different from the original K-means analysis algorithm, the clustering method of the embodiment of the present invention does not use the 128-dimensional sift feature of the key point according to the distance feature of the point, and analyzes the Euclidean distance of the key point in the SIFT feature space. The method of the embodiment of the invention can better analyze the same attribute of the feature points, thereby obtaining a more accurate foreground suggestion area. The following describes the analysis steps of K-means in the related technology:

The K-means algorithm clusters the samples x ⁽ⁱ⁾ into k clusters, and the clusters belong to unsupervised learning. The user does not need to provide the category labeling of the samples. The algorithm is described as follows:

Randomly select the cluster centroids of the k clusters as

μ ₁ , μ ₂ ,...,μ _k ∈R ⁿ ,

For each of the samples i, the distance to the k seed points is calculated. If the point c ⁽ⁿ⁾ is closest to the seed point μ _n , then c ⁽ⁿ⁾ belongs to the μ _n point group. In the present invention, it is necessary to calculate the Euclidean distance in the 128-dimensional SIFT feature space:

Next, move the μ _n seed point of each point group to the center of the point group.

Sn is the scale information of the matching point.

Repeat steps 2 and 3 until the center of each class is gradually stabilized.

After cluster analysis, the embodiment of the present invention can obtain a foreground group and a background seed point group for marking a foreground area and a background area in the image. FIG. 3 is a schematic diagram of an image obtained after cluster analysis processing according to an embodiment of the present invention. As shown in FIG. 3, the front attraction group and the background seed point group are used to mark the foreground area and the background area in the image, respectively.

After the center is gradually stabilized, the embodiment of the present invention performs foreground extraction.

In this embodiment, the image segmentation algorithm in the related art is used to cut and extract the target contour of the image with the foreground and background regions.

First, an undirected graph G=<V, E> is used to represent the image A to be segmented, and V and E are respectively a set of vertex and edge. In this undirected graph, there are two types of edges and vertices: the first is a common fixed point for each pixel in the image. Fixed points for every two fields (corresponding to two neighborhood images in the figure) The connection of the prime is an edge, which is n-links. In addition to ordinary vertices, there are two additional terminal vertices called S (source: source point) and T (sink: sink point). Such vertices have connections to each of the normal vertices, which are called t-links.

Next, assign weights to each edge. If the image is segmented into L, the energy of the image can be expressed as: E(L)=αR(L)+B(L), where R(L) is the region term, B( L) is the boundary term. E(L) represents the weight, also called the energy function. The goal of image segmentation is to optimize the energy function to reach the minimum value.

The weights of the regional items are as follows:

R(L)=R _x (l _x )

The item weight of the area represents the weight of the t-links edge. The higher the probability that the point belongs to S or T, the greater its weight, and vice versa.

The weights of the boundary terms are as follows:

B(L)=B _<x,y> ·δ(l _x ,l _y )

The boundary term represents the weight of the n-links edge. When the similarity of two adjacent pixels is higher, the weights of the edges connected by the two points are higher.

After assigning the weights of each edge, the min cut algorithm is used to find the smallest edges, and the breaks of the edges are just such that the target and the background are separated. FIG. 4 is a schematic diagram of the foreground target and the background separated according to an embodiment of the present invention. As shown in Figure 4, after assigning weights to each edge, the smallest edges are found, and the edges are broken so that the target and background are separated.

The following data is combined with the experiment as follows:

1. Data set: In the experiment, the paired images can be randomly selected from the dataset of CMU-Cornell as the test set of the method, and because the dataset of the target contained in the image is open sourced by CMU-Cornell, so as to provide The truth contour map is used as a test set for method accuracy.

2. Experimental setup: In the experiment, the crossover ratio is used as the evaluation parameter of the experimental results. The Intersection Rate is as follows:

Where P' is the foreground image taken out in this embodiment, and P is the true outline of the image, which can be passed The experimental results are compared with the Truth_ground of the same object in the open source dataset, and the correct pixel point ratio is obtained to evaluate the accuracy of the method.

3. Experimental Results: Experimental Results As shown in FIG. 5, with the embodiment of the present invention, the foreground target and the background segmentation process are realized, and the embodiment of the present invention can obtain the approximate outline of the foreground image.

4. Analysis of results: As shown in the figure, most of the foreground contours can be guaranteed, but due to the similarity between the background and the target, the outline of the foreground target contains small flaws, but the overall accuracy can reach about 85%.

Example 2

This embodiment provides a foreground segmentation device, as shown in FIG. 6, including:

The first unit is configured to extract local feature information of the two input images, including:

The first unit is configured to perform key point matching according to the extracted local feature information, including:

Optionally, the second unit is configured to filter out the wrong matching points from the matching points of the obtained key points, and obtaining all correct matching points includes:

Configuring a scale ratio SR and a direction OA, and calculating a match between a key point in the first input image and a key point in the first input image in the second input image according to the result obtained by the key point matching The scale ratio of the points and the direction scale, and construct a two-dimensional array as follows:

P={<SR ₁ , OA ₁ >, <SR ₂ , OA ₂ >...<SR _n , OA _n >}

among them,

OA _n =θ _n -θ _n '

Perform screening of matching points.

Optionally, the third unit uses the cluster analysis to derive the feature point groups on the foreground target from all the correct matching points, including:

μ ₁ , μ ₂ ,...,μ _k ∈R ⁿ ,

For the above-mentioned device, the method of the above-mentioned Embodiment 1 can be implemented. For the other operations of the device in the foregoing device, refer to the corresponding content of Embodiment 1, and details are not described herein again.

As can be seen from the above embodiments, the technical solution of the present application utilizes image features and is applied to the core problem of automatic foreground extraction of still images. Compared with the related technology, the feature points of the two images are proposed. Through the matching of the feature points, the contour of the region of interest is obtained through cluster analysis. Finally, the image segmentation algorithm is used to automatically extract the foreground target of the still image. Especially suitable for still image data, with high accuracy.

The embodiment of the invention further provides a computer storage medium, wherein the computer storage medium stores the meter The computer executable instructions are used to execute the foreground segmentation method described above.

An embodiment of the present invention further provides a foreground segmentation apparatus, including: a memory and a processor; wherein

The processor is configured to execute program instructions in the memory;

Program instructions perform the following operations on the processor read:

One of ordinary skill in the art will appreciate that all or a portion of the above steps may be performed by a program to instruct related hardware, such as a processor, which may be stored in a computer readable storage medium, such as a read only memory, disk or optical disk. Wait. Alternatively, all or part of the steps of the above embodiments may also be implemented using one or more integrated circuits. Correspondingly, each module/unit in the foregoing embodiment may be implemented in the form of hardware, for example, by implementing an integrated circuit to implement its corresponding function, or may be implemented in the form of a software function module, for example, being executed by a processor and stored in a memory. Programs/instructions to implement their respective functions. The invention is not limited to any specific form of combination of hardware and software.

The embodiments disclosed in the present application are as described above, but the descriptions are only for the purpose of understanding the present application, and are not intended to limit the present application, such as the specific implementation method in the embodiments of the present invention. Any modifications and changes in the form and details of the embodiments may be made by those skilled in the art without departing from the spirit and scope of the disclosure. The scope defined by the appended claims shall prevail.

Industrial applicability

The above technical solution improves the accuracy of the foreground target in the image, reduces the time of foreground processing, and improves the efficiency of image processing.

Claims

A method of foreground segmentation, including:

Extracting local feature information of two input images respectively, and matching key points according to the extracted local feature information;

Screen out the mismatched points from the matching points of the obtained key points to get all the correct matching points;

Use cluster analysis to derive feature point groups on foreground targets from all correct matching points;

According to the obtained feature point group, the image segmentation algorithm is used to obtain the foreground target in the image.
The foreground segmentation method according to claim 1, wherein the extracting the local feature information of the two input images comprises:

The two images input by the user are grayed out, and the local feature information of the image is extracted by using the accelerated robust feature SURF algorithm.
The foreground segmentation method according to claim 2, wherein the matching of the key points according to the extracted local feature information comprises:

And determining, according to the extracted local feature information, a matching point corresponding to a key point in the first input image in the second input image by using a neighbor algorithm.
The foreground segmentation method according to claim 2 or 3, wherein the false matching points are filtered out from the matching points of the obtained key points, and all the correct matching points are obtained:

Configuring a scale ratio SR and a direction OA, and calculating a match between a key point in the first input image and a key point in the first input image in the second input image according to the result obtained by the key point matching The scale ratio of the points and the direction scale, and construct the following two-dimensional array:

P={<SR 1 , OA 1 >, <SR 2 , OA 2 >...<SR n , OA n >}

among them,
OA n =θ n -θ n '

Perform screening of matching points;

Wherein, the ratio of s n to s n ' is a scale ratio of a key point in the first input image and a matching point in the first input image in the second input image, and a logarithm is obtained to obtain a scale ratio; The difference between n and θ' n is the difference in direction between the key point in the first input image and the matching point in the second input image of the key point in the first input image.
The foreground segmentation method according to claim 4, wherein said using cluster analysis to derive feature point groups on the foreground object from all of the correct matching points comprises:

The following algorithm is used to randomly select the heart points of k clusters as:

μ 1 , μ 2 ,...,μ k ∈R n ,

For each of the samples i, the distance to the k seed points is calculated, and the point closest to the seed point μ n belongs to the μ n point group, wherein the Euclidean in the 128-dimensional scale-invariant feature transform SIFT feature space is calculated according to the following formula distance:

Moving the μ n seed point of each point group to the center of the μ n point group,

Repeating the calculation of the distance of each μ n seed point until the center of each class is gradually stabilized, obtaining a pre-attraction group and a background seed point group, and obtaining the obtained pre-attraction group and background seed point group as the feature point group;

Wherein, S i represents a one-dimensional feature of the SIFT feature, and R n represents that the heart points of the selected K clusters belong to a set of n clusters randomly taken from the point set.
A foreground segmentation device comprising:

The first unit is configured to separately extract local feature information of the two input images, and perform matching of the key points according to the extracted local feature information;

The second unit is configured to filter out the wrong matching points from the matching points of the obtained key points to obtain all correct matching points;

The third unit is configured to use cluster analysis to derive feature point groups on the foreground target from all the correct matching points;

The fourth unit is configured to obtain a foreground target in the image by using an image segmentation algorithm according to the obtained feature point group.
The foreground segmentation apparatus according to claim 6, wherein the first unit is configured to extract local feature information of the two input images, including:

The two images input by the user are grayed out, and the local feature information of the image is extracted by using the accelerated robust feature SURF algorithm.
The foreground segmentation apparatus according to claim 7, wherein the matching of the first unit by the first unit to perform key points according to the extracted local feature information comprises:

And determining, according to the extracted local feature information, a matching point corresponding to a key point in the first input image in the second input image by using a neighbor algorithm.
The foreground segmentation apparatus according to claim 7 or 8, wherein said second unit is arranged to filter out the mismatched points from the matching points of the obtained key points, and obtaining all correct matching points comprises:

Configuring a scale ratio SR and a direction OA, and calculating a match between a key point in the first input image and a key point in the first input image in the second input image according to the result obtained by the key point matching The scale ratio of the points and the direction scale, and construct the following two-dimensional array:

P={<SR 1 , OA 1 >, <SR 2 , OA 2 >...<SR n , OA n >}

among them,
OA n =θ n -θ n '

Perform screening of matching points.
The foreground segmentation apparatus according to claim 9, wherein said third unit is arranged to derive a cluster of feature points on the foreground object from all of the correct matching points using cluster analysis:

The following algorithm is used to randomly select the heart points of k clusters as

μ 1 , μ 2 ,...,μ k ∈R n ,

For each of the samples i, the distance to the k seed points is calculated, and the point closest to the seed point μ n belongs to the μ n point group, wherein the Euclidean distance in the 128-dimensional SIFT feature space is calculated according to the following formula:

Moving the μ n seed point of each point group to the center of the μ n point group,

The distance of each μ n seed point is repeatedly calculated until the center of each class is gradually stabilized, and the front spot group and the background seed point group are obtained, and the obtained front spot group and background seed point group are taken as the feature point group.