WO2023155393A1

WO2023155393A1 - Feature point matching method and apparatus, electronic device, storage medium and computer program product

Info

Publication number: WO2023155393A1
Application number: PCT/CN2022/111862
Authority: WO
Inventors: 甄佳楠; 周晓巍; 孙佳明; 张思宇
Original assignee: 上海商汤智能科技有限公司
Priority date: 2022-02-17
Filing date: 2022-08-11
Publication date: 2023-08-24
Also published as: CN114519794A

Abstract

A feature point matching method and apparatus, an electronic device, and a storage medium. A first feature point on a target image, a first coordinate representing the position of the first feature point on the target image, and a first descriptor representing features of the first feature point are determined (S101). Meanwhile, at least one second feature point on a reference image, a second coordinate representing the position of each second feature point on the reference image, and second descriptors representing features of the second feature points are determined (S102). According to the first descriptor and each second descriptor, first screening is performed on the at least one second feature point, so as to obtain at least one candidate feature point (S103); and according to the first coordinate and the second coordinate of each candidate feature point, second screening is performed on the at least one candidate feature point, so as to obtain a target feature point, on the reference image, matching the first feature point (S104).

Description

Feature point matching method, device, electronic device, storage medium and computer program product

Cross References to Related Applications

This disclosure claims the priority of the Chinese patent application number 202210145598.8 submitted on February 17, 2022, the applicant is Zhejiang Shangtang Technology Development Co., Ltd., and the application name is "feature point matching method and device, electronic equipment and storage medium". The entirety of the application is incorporated by reference into this disclosure.

technical field

The present disclosure relates to the field of computer technology, and in particular to a feature point matching method, device, electronic equipment, storage medium and computer program product.

Background technique

Two-dimensional feature point matching between images is an important algorithm in computer vision. The existing two-dimensional feature point matching methods are usually based on local image feature extraction. Similar positions are incorrectly matched, and the matching result is not accurate enough.

Contents of the invention

Embodiments of the present disclosure provide a feature point matching method, device, electronic device, storage medium and computer program product, aiming at improving the accuracy of feature point matching between two images.

An embodiment of the present disclosure provides a feature point matching method, including: determining a first feature point on a target image, and a first coordinate and a first descriptor of the first feature point, the first coordinate characterizing the The position of the first feature point on the target image, the first descriptor characterizes the feature of the first feature point; determine at least one second feature point on the reference image, and each of the second feature points The second coordinates and the second descriptor, the second coordinates characterize the position of the second feature point on the reference image, the second descriptor characterizes the feature of the second feature point; according to the The first descriptor and each of the second descriptors perform a first screening on at least one of the second feature points to obtain at least one candidate feature point; according to the first coordinates and each of the candidate feature points The second coordinates perform a second screening on at least one of the candidate feature points to obtain a target feature point that matches the first feature point.

In a possible implementation manner, the first screening is performed on at least one of the second feature points according to the first descriptor and each of the second descriptors to obtain at least one candidate feature point, including : Determine the similarity parameters of each of the second descriptors and the first descriptors respectively; perform a first screening on at least one second feature point according to the similarity parameters to obtain at least one candidate feature point.

In a possible implementation manner, the respectively determining the similarity parameters of each of the second descriptors and the first descriptors includes: respectively determining the similarity parameters of each of the second descriptors and the first descriptors Sub-distance vectors to obtain the similarity parameters of each of the second descriptor and the first descriptor.

In a possible implementation manner, the first screening of at least one second feature point according to the similarity parameter to obtain at least one candidate feature point includes: determining the similarity with the first descriptor A second descriptor whose parameter is smaller than the first threshold; and a second feature point corresponding to the second descriptor whose similarity parameter is smaller than the first threshold is used as a candidate feature point.

In a possible implementation manner, the first screening of at least one of the second feature points according to the similarity parameters to obtain at least one candidate feature point includes: corresponding to each of the second descriptors The similarity parameters are sorted from small to large, and the second feature points corresponding to the first preset number of second descriptors are obtained as candidate feature points.

In a possible implementation manner, the second screening is performed on at least one of the candidate feature points according to the first coordinates and the second coordinates of each of the candidate feature points to obtain The target feature point for point matching includes: performing a second screening on at least one of the candidate feature points according to the first coordinates, the second coordinates, and the relative positional relationship to obtain a target feature point that matches the first feature point. A target feature point; the relative positional relationship is the relative positional relationship between the target image and the reference image.

In a possible implementation manner, according to the first coordinates, the second coordinates, and the relative positional relationship between the target image and the reference image, performing a second operation on at least one of the candidate feature points Screening to obtain a target feature point matching the first feature point includes: determining the first position of the target image, and determining the second position of the reference image; according to the first position, the second determining the estimated epipolar line of the first feature point on the reference image based on the epipolar geometry based on the position and the first coordinate; The shortest distance performs a second screening on at least one of the candidate feature points to obtain a target feature point matching the first feature point.

In a possible implementation manner, at least one of the candidate feature points is screened a second time according to the shortest distance between the second coordinates of each of the candidate feature points and the estimated epipolar line to obtain the The target feature point matched by the first feature point includes: determining the number of candidate feature points whose shortest distance is smaller than a second threshold among the at least one candidate feature point; the shortest distance is the second coordinate and the estimated The shortest distance of the epipolar line; when the number of candidate feature points whose shortest distance is less than the second threshold is one, use the candidate feature points whose shortest distance is less than the second threshold as target feature points; in the shortest distance When the number of candidate feature points smaller than the second threshold is greater than one, the target feature point is determined among the candidate feature points whose shortest distance is smaller than the second threshold.

In a possible implementation manner, the determining the target feature point among the candidate feature points whose shortest distance is less than a second threshold includes: determining the target feature point among the candidate feature points whose shortest distance is less than a second threshold The candidate feature point with the largest similarity parameter is the target feature point.

An embodiment of the present disclosure provides a feature point matching device, including: a first information determining part configured to determine a first feature point on a target image, and first coordinates and a first descriptor of the first feature point , the first coordinate characterizes the position of the first feature point on the target image, the first descriptor characterizes the feature of the first feature point; the second information determination part is configured to determine a reference image At least one second feature point on , and the second coordinates and the second descriptor of each of the second feature points, the second coordinates represent the position of the second feature point on the reference image, so The second descriptor characterizes the feature of the second feature point; the first screening part is configured to perform the first screening on at least one second feature point according to the first descriptor and each of the second descriptors One screening to obtain at least one candidate feature point; the second screening module part is configured to perform a second round of at least one candidate feature point according to the first coordinates and the second coordinates of each of the candidate feature points Screening to obtain target feature points matching the first feature point.

In a possible implementation manner, the first screening part includes: a parameter determination subsection configured to respectively determine similar parameters of each of the second descriptor and the first descriptor; the first screening subsection A part configured to perform a first screening on at least one second feature point according to the similarity parameter to obtain at least one candidate feature point.

In a possible implementation manner, the parameter determination subpart includes: a parameter determination part configured to respectively determine a distance vector between each of the second descriptors and the first descriptor, and obtain each of the Similar parameters of the second descriptor and the first descriptor.

In a possible implementation manner, the first screening subpart includes: a descriptor screening part configured to determine a second descriptor whose similar parameter to the first descriptor is smaller than a first threshold; the first The candidate point determining part is configured to use the second feature point corresponding to the second descriptor whose similarity parameter is smaller than the first threshold as a candidate feature point.

In a possible implementation manner, the first screening subpart includes: a second candidate point determination part configured to sort the similar parameters corresponding to each of the second descriptors from small to large, and obtain the previous The second feature points corresponding to the preset number of second descriptors are candidate feature points.

In a possible implementation manner, the second screening part includes: a second screening subsection configured to, according to the first coordinates, the second coordinates and the relative positional relationship, select at least one of the candidate features Points are screened for the second time to obtain target feature points matching the first feature point; the relative positional relationship is the relative positional relationship between the target image and the reference image.

In a possible implementation manner, the second screening subsection includes: a position determining part configured to determine a first position of the target image and determine a second position of the reference image; an epipolar line determining part , configured to determine an estimated epipolar line of the first feature point on the reference image based on epipolar geometry according to the first position, the second position and the first coordinate; the feature point screening part, It is configured to perform a second screening on at least one of the candidate feature points according to the shortest distance between the second coordinates of each of the candidate feature points and the estimated epipolar line, to obtain a target matching the first feature point Feature points.

In a possible implementation manner, the feature point screening part includes: a distance determination subsection configured to determine the number of candidate feature points whose shortest distance among the at least one candidate feature point is smaller than a second threshold; The shortest distance is the shortest distance between the second coordinate and the estimated epipolar line; the first feature point determining subsection is configured to be one when the number of candidate feature points whose shortest distance is less than a second threshold is one, The candidate feature points whose shortest distance is less than a second threshold are used as target feature points; the second feature point determining subsection is configured to, when the number of candidate feature points whose shortest distance is less than a second threshold is greater than one, A target feature point is determined among candidate feature points whose shortest distance is smaller than a second threshold.

In a possible implementation manner, the second feature point determination subsection includes: a third feature point determination subsection configured to determine the similarity among the candidate feature points whose shortest distance is smaller than a second threshold. The candidate feature point with the largest parameter is the target feature point.

An embodiment of the present disclosure provides an electronic device, including: a processor; a memory configured to store instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

An embodiment of the present disclosure provides a computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the foregoing method is implemented.

An embodiment of the present disclosure provides a computer program product, the computer program product includes a non-transitory computer storage medium storing a computer program, and the above method is implemented when the computer program is read and executed by a computer.

In the embodiment of the present disclosure, the first feature point on the target image, the first coordinate representing the position of the first feature point on the target image, and the first descriptor representing the feature of the first feature point are determined. At the same time, at least one second feature point on the reference image, second coordinates characterizing the position of each second feature point on the reference image and a second descriptor characterizing the characteristics of the second feature point are determined. And according to the first descriptor and each second descriptor, at least one second feature point is screened for the first time to obtain at least one candidate feature point, and then according to the first coordinate and the second coordinate of each candidate feature point, at least A candidate feature point is screened for the second time to obtain the target feature point matching the first feature point on the reference image. The embodiments of the present disclosure can perform two matchings according to the local features and the positions of the feature points, and accurately match the points that record the same information in the two images.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

Description of drawings

The accompanying drawings here are incorporated into the description and constitute a part of the present description. These drawings show embodiments consistent with the present disclosure, and are used together with the description to explain the technical solution of the present disclosure.

FIG. 1 shows a flowchart of a feature point matching method according to an embodiment of the present disclosure;

Fig. 2 shows a schematic diagram of a target image according to an embodiment of the present disclosure;

Fig. 3 shows a schematic diagram of a reference image according to an embodiment of the present disclosure;

FIG. 4 shows a flow chart of a feature point matching method according to an embodiment of the present disclosure;

FIG. 5 shows a flowchart of a feature point matching method according to an embodiment of the present disclosure;

FIG. 6 shows a flowchart of a feature point matching method according to an embodiment of the present disclosure;

Fig. 7 shows a schematic diagram of determining an estimated epipolar line according to an embodiment of the present disclosure;

Fig. 8 shows a schematic diagram of an estimated epipolar line according to an embodiment of the present disclosure;

FIG. 9 shows a flowchart of a feature point matching method according to an embodiment of the present disclosure;

FIG. 10 shows a schematic diagram of a process of determining target feature points according to an embodiment of the present disclosure;

Fig. 11 shows a flowchart of a feature point matching method according to an embodiment of the present disclosure;

Fig. 12 shows a schematic diagram of a feature point matching device according to an embodiment of the present disclosure;

Fig. 13 shows a schematic diagram of an electronic device according to an embodiment of the present disclosure;

Fig. 14 shows a schematic diagram of another electronic device according to an embodiment of the present disclosure.

Detailed ways

Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures indicate functionally identical or similar elements. While various aspects of the embodiments are shown in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

The term "at least one" herein means any one of a variety or any combination of at least two of a variety, for example, including at least one of A, B, and C, may mean including from A, B, and Any one or more elements selected from the set formed by C.

In addition, in order to better illustrate the present disclosure, numerous implementation details are given in the following specific implementation manners. It will be understood by those skilled in the art that the present disclosure may be practiced without certain of the implementation details. In some embodiments, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the gist of the present disclosure.

In a possible implementation manner, the feature point matching method in the embodiment of the present disclosure may be executed by an electronic device such as a terminal device or a server. Wherein, the terminal device may be user equipment (User Equipment, UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (Personal Digital Assistant, PDA), handheld device, computing device, vehicle-mounted device, Any fixed terminal or mobile terminal such as wearable devices. The server can be a single server or a server cluster composed of multiple servers. Any electronic device can implement the feature point matching method of the embodiments of the present disclosure by calling the computer-readable instructions stored in the memory by the processor.

The embodiments of the present disclosure are used to match feature points that record the same information on two two-dimensional images. For example, in the process of collecting images from different angles of the target object, two two-dimensional images are obtained, and the feature points used to record the same position of the target object on the two two-dimensional images are matched.

Fig. 1 shows a flowchart of a feature point matching method according to an embodiment of the present disclosure. As shown in FIG. 1 , the feature point matching method of the embodiment of the present disclosure includes the following steps S101 to S104.

Step S101. Determine a first feature point on a target image, as well as first coordinates and a first descriptor of the first feature point.

In a possible implementation manner, the target image is an image obtained by collecting a target object, and the target object may be any living or inanimate object such as animals, people, furniture, and decorations. And there is at least one feature point on the target image, and each feature point is a local area on the target image. At least one feature point on the target image can be obtained through object recognition, that is, identifying the target object in the target image, and determining at least one feature point on the target object for characterizing local features of the target object. When it is necessary to perform feature point matching on each feature point on the target image, one feature point may be first determined as the first feature point, and the first feature point is matched. When the current first feature point matching is completed, another feature point is re-determined as the first feature point, and then the feature point matching is performed until the entire feature point matching process is completed.

In a possible implementation manner, while determining the first feature point on the target image, first coordinates and a first descriptor of the first feature point are also determined. Wherein, the first coordinate represents the position of the first feature point on the target image, and the first descriptor represents the feature of the first feature point, for example, may include color feature and texture feature. The first coordinates can be represented by two-dimensional coordinates or three-dimensional coordinates. In the case where the first coordinates are represented by two-dimensional coordinates, the target coordinate system of the target image may be determined first, and then the coordinates of the first feature point in the target coordinate system may be determined as the first coordinates. The target coordinate system can be any preset coordinate system. For example, the coordinate system with the lower left corner of the target image as the origin of the coordinate system, the bottom side as the x-axis, and the left side as the y-axis is determined as the target coordinate system. In the case where the first coordinate is represented by three-dimensional coordinates, the representation mode of the plane where the target image is located in the preset target three-dimensional coordinate system can be first determined according to the position of the image acquisition device of the target image and the shooting angle during the target image acquisition process , and then determine the first coordinates according to the position of the first feature point in the target image. The target three-dimensional coordinate system can be preset.

In a possible implementation manner, the first descriptor may be determined by extracting a preset feature at the location of the first feature point, and determining a feature vector as the first descriptor according to the distribution of the preset feature. The preset features can be set according to actual needs, and can include at least one of color features and texture features. The first descriptor can be any feature descriptor, for example, it can be a histogram of oriented gradients (Histogram of Oriented Gradients, HOG) feature descriptor. The process of determining the first descriptor can be to determine the gradient change of each pixel in the horizontal and vertical directions in the position of the first feature point, and determine the magnitude and direction of each pixel according to the gradient change to obtain the Generate a histogram of the gradient magnitude and direction of . Then the feature vector is obtained through histogram normalization as the first descriptor.

Fig. 2 shows a schematic diagram of a target image according to an embodiment of the present disclosure. As shown in FIG. 2 , the target image 200 in this embodiment of the present disclosure may be an image obtained by capturing a target object 201 at a first angle. In a possible implementation manner, the first feature point 202 representing the feature position in the target object 201 may be determined by means of object recognition or the like. At the same time, the first coordinates are determined according to the position of the first feature point 202 in the target image 200 , and the first descriptor used to characterize the feature of the first feature point 202 is determined according to the local characteristics of the position of the first feature point 202 .

Step S102. Determine at least one second feature point on the reference image, as well as the second coordinates and the second descriptor of each second feature point.

In a possible implementation manner, the reference image is also an image obtained by collecting a target object, and the pose of the target object in the reference image is different from that in the target image. There is at least one second feature point on the reference image, and each second feature point is a local area on the reference image. The second feature points on the reference image can be obtained through object recognition, that is, identifying the target object in the target image, and determining at least one second feature point on the target object for characterizing the local features of the target object. The second feature point on the reference image is used for matching with the first feature point, so as to determine a target feature point representing the same position of the target object as the first feature point among the plurality of second feature points.

In a possible implementation manner, while determining the second feature points on the reference image, the second coordinates and the second descriptor of each second feature point are also determined. Wherein, the second coordinate represents the position of the corresponding second feature point on the reference image, and the second descriptor represents the feature of the corresponding second feature point, for example, may include color feature and texture feature. The second coordinates may be represented by two-dimensional coordinates or three-dimensional coordinates. In the case that the second coordinates are represented by two-dimensional coordinates, the target coordinate system of the reference image may be determined first, and then the coordinates of the second feature point in the target coordinate system may be determined as the second coordinates. The target coordinate system can be any preset coordinate system. For example, the coordinate system with the lower left corner of the reference image as the origin of the coordinate system, the bottom side as the x-axis, and the left side as the y-axis is determined as the target coordinate system. In the case where the second coordinates are represented by three-dimensional coordinates, the representation mode of the plane where the reference image is located in the preset target three-dimensional coordinate system can be first determined according to the position of the image acquisition device of the reference image and the shooting angle during the reference image acquisition process , and then determine the second coordinates according to the position of the second feature point in the reference image. The target three-dimensional coordinate system may be preset, and it is the same three-dimensional coordinate system as the target three-dimensional coordinate system where the first coordinate is located.

In a possible implementation manner, the second descriptor may be determined by extracting a preset feature at the location of the second feature point, and determining a feature vector as the second descriptor according to the distribution of the preset feature. The preset features can be set according to actual needs, and can include at least two of color features and texture features. The determination method of the second descriptor is the same as that of the above-mentioned first descriptor, and the included feature content is also the same.

Fig. 3 shows a schematic diagram of a reference image according to an embodiment of the present disclosure. As shown in FIG. 3 , the reference image 300 in the embodiment of the present disclosure may be an image obtained by capturing the target object 201 at a second angle. The second feature point 302 representing at least one feature position in the target object 201 may be determined by means of object recognition or the like. At the same time, determine the corresponding second coordinates according to the position of each second feature point 302 in the reference image 300 in the reference image 300, and then determine the corresponding second feature according to the local features at the position of each second feature point 302 The second descriptor of the point 302 feature.

Step S103 , performing a first screening on at least one second feature point according to the first descriptor and each of the second descriptors to obtain at least one candidate feature point.

In a possible implementation, the first feature point and each second feature point may be first matched according to the local features of the feature point, and at least one second feature that may match the first feature point may be screened out points as candidate feature points. Since the local feature of each feature point is identified according to the corresponding descriptor, that is, the second feature point can be screened for the first time directly according to the first descriptor and each second descriptor. Two feature points are used as candidate feature points. In the case that all the second feature points fail to pass the screening, the feature point matching process may be ended, and it is determined that there is no second feature point matching the first feature point on the current reference image.

Fig. 4 shows a flowchart of a feature point matching method according to an embodiment of the present disclosure. As shown in FIG. 4 , based on FIG. 1 , step S103 may be implemented through steps S401 to S402 , which will be described in conjunction with the steps shown in FIG. 4 .

Step S401, respectively determining the similarity parameters of each second descriptor and the first descriptor;

Step S402, performing a first screening on at least one second feature point according to the similarity parameter to obtain at least one candidate feature point.

In the process of feature point matching in the embodiment of the present disclosure, each second feature point can be screened once based on the similar parameters between each second descriptor and the first descriptor, so that the local feature and the first feature The second feature point with a higher point similarity is used as a candidate feature point, and the first screening of the second feature point is completed.

In a possible implementation manner, step S4011 may be used to realize the determination of similar parameters of each second descriptor and the first descriptor respectively.

Step S4011, obtain the distance vector between each second descriptor and the first descriptor, and obtain the similarity parameters between each second descriptor and the first descriptor.

In a possible implementation, the size of the similarity parameter between two features is inversely proportional to the matching degree of the local features of the two feature points, that is, the larger the similarity parameter, the lower the matching degree. Wherein, both the first descriptor and the second descriptor are vectors, and a similar parameter of the first descriptor and the second descriptor may be a distance between vectors. The distance vector can be obtained by calculating the vector inner product of the first descriptor and the second descriptor.

In a possible implementation manner, after determining the similarity parameters between the first descriptor and each second descriptor, the above-mentioned first screening of at least one second feature point according to the similarity parameters can be realized through steps S4021 to S4022 , to get at least one candidate feature point.

Step S4021, determining a second descriptor whose similar parameter to the first descriptor is smaller than the first threshold;

Step S4022, taking the second feature point corresponding to the second descriptor whose similarity parameter is smaller than the first threshold as a candidate feature point.

In the process of feature point matching in the embodiment of the present disclosure, because the second descriptor is screened based on the preset first threshold, the screening of the second feature point is completed, and the local feature is similar to the first feature point The second feature point with a higher degree is used as a candidate feature point to complete the first screening of the second feature point.

In a possible implementation manner, step S4023 may also be used to implement the above-mentioned first screening of at least one second feature point according to similar parameters to obtain at least one candidate feature point.

Step S4023, sort the similar parameters corresponding to each second descriptor from small to large, and obtain the second feature points corresponding to the previous preset number of second descriptors as candidate feature points. The preset quantity is a positive integer greater than or equal to one.

The first screening process is based on the degree of similarity between the second feature point and the local feature of the first feature point, that is, at least one candidate feature point similar to the local feature of the first feature point is obtained through screening.

Step S104, performing a second screening on at least one of the candidate feature points according to the first coordinates and the second coordinates of each of the candidate feature points, to obtain a target feature point matching the first feature point.

In a possible implementation, after the first screening of the second feature point based on the local feature to obtain at least one candidate feature point, the second screening of the at least one candidate feature point based on the position feature is performed to determine the reference The target feature point in the image that matches the first feature point. That is to say, the second screening can be performed according to the first coordinates and the second coordinates of each candidate feature point to obtain target feature points whose local features are similar to the first feature point and whose positions match the first feature point.

Fig. 5 shows a flowchart of a feature point matching method according to an embodiment of the present disclosure. As shown in FIG. 5 , based on FIG. 1 , step S104 may be implemented through step S501 , which will be described in conjunction with the steps shown in FIG. 5 .

In a possible implementation, the second screening of each candidate feature point can be implemented based on the relative positional relationship between the target image and the reference image, that is, step S501, according to the first coordinates, the second coordinates and the relative positional relationship, A second screening is performed on at least one candidate feature point to obtain a target feature point matched by the first feature point; the relative positional relationship is the relative positional relationship between the target image and the reference image.

Fig. 6 shows a flowchart of a feature point matching method according to an embodiment of the present disclosure. As shown in FIG. 6 , based on FIG. 5 , step S501 may be implemented through steps S601 to S603 , which will be described in conjunction with the steps shown in FIG. 6 .

Step S601, determine the first position of the target image, and determine the second position of the reference image;

Step S602, determining the estimated epipolar line of the first feature point on the reference image based on the epipolar geometry according to the first position, the second position and the first coordinate;

Step S603 , performing a second screening on at least one candidate feature point according to the shortest distance between the second coordinates of each candidate feature point and the estimated epipolar line, to obtain a target feature point matching the first feature point.

In a possible implementation manner, the first position of the target image is the position where the image acquisition device is located when the target image is acquired, and the second position of the reference image is the position where the image acquisition device is located when the reference image is acquired. The first position and the second position may be represented by coordinates of the image acquisition device in a preset target three-dimensional coordinate system.

In a possible implementation, the process of determining the estimated epipolar line may first determine the first position of the target image and the second position of the reference image, and determine the first coordinate and each second coordinate in the three-dimensional coordinates of the target The three-dimensional coordinates in the system are expressed as the first three-dimensional coordinates and the second three-dimensional coordinates respectively. Determine possible positions of the position of the target object represented by the first feature point in the target three-dimensional coordinate system according to the connection line between the first position and the first three-dimensional coordinates, and project each possible position according to the second position of the reference image Estimated epipolar lines are obtained on the reference image. And the shortest distance from each second three-dimensional coordinate to the estimated epipolar line can be calculated for the second screening. Wherein, when the first coordinate and the second coordinate are three-dimensional coordinates in the target three-dimensional coordinate system, the first three-dimensional coordinate is the first coordinate, and the second three-dimensional coordinate is the second coordinate. In the case where the first coordinates are two-dimensional coordinates in the two-dimensional coordinate system corresponding to the target image, and the second coordinates are two-dimensional coordinates in the two-dimensional coordinate system corresponding to the reference image, the three-dimensional coordinates of the target image and the reference image can be The position in the coordinate system transforms the first coordinate and each second coordinate into first and second three-dimensional coordinates in three dimensions.

Fig. 7 shows a schematic diagram of determining an estimated epipolar line according to an embodiment of the present disclosure. As shown in Figure 7, in the process of determining the estimated epipolar line of the first feature point on the reference image 702, the first position of the image acquisition device in the target three-dimensional coordinate system can be determined first in the process of acquiring the target image 701 703, and during the process of acquiring the reference image 702, a second position 704 of the image acquisition device in the target three-dimensional coordinate system. Then, connect the first position 703 and the second position 704 to obtain a target intersection point 708 between the connecting line and the reference image 702 in the target three-dimensional coordinate system. According to the first position 703 and the first coordinate 705 representing the position of the first feature point in the target three-dimensional coordinate system, the target line segment 706 extending outward from the target image 701 plane is determined, and one end point of the target line segment 706 is the first feature point, and the target The positions on the line segment 706 are the possible positions in the three-dimensional coordinate system of the position represented by the first feature point on the target object. The target line segment 706 is projected onto the reference image 702 to determine an estimated epipolar line 707 with the target intersection point 708 as one end point and the other end point located at the edge of the reference image 702 . Wherein, each point in the estimated epipolar line 707 is a point representing the position of the target object which may be the same as the position of the first feature point representing the target object in spatial position.

Fig. 8 shows a schematic diagram of an estimated epipolar line according to an embodiment of the present disclosure. As shown in FIG. 8 , after determining the estimated epipolar line 801 on the reference image 800 , a second screening can be performed according to the shortest distance from the second coordinates of each candidate feature point 802 to the estimated epipolar line 801 . Among them, the function representation of the predicted epipolar line 801 and the coordinate representation of each candidate feature point 802 need to be transformed into the same coordinate system.

Fig. 9 shows a flowchart of a feature point matching method according to an embodiment of the present disclosure. As shown in FIG. 9 , based on FIG. 6 , step S603 may be implemented through steps S701 to S702 , which will be described in conjunction with the steps shown in FIG. 9 .

In a possible implementation manner, the process of performing the second screening according to the shortest distance from the second coordinates of each candidate feature point to the estimated epipolar line may be implemented according to a preset second threshold.

Step S901. Determine the number of candidate feature points whose shortest distance is less than a second threshold among the at least one candidate feature point; the shortest distance is the shortest distance between the second coordinate and the estimated epipolar line:

Step S902. When the number of candidate feature points whose shortest distance is less than the second threshold is one, use the candidate feature points whose shortest distance is less than the second threshold as target feature points

Step S903 , if the number of candidate feature points whose shortest distance is less than a second threshold is greater than one, determine a target feature point among candidate feature points whose shortest distance is less than a second threshold.

In a possible implementation, the second threshold represents the maximum distance between the second feature point that may match the first feature point and the estimated epipolar line, so as to filter out the second feature point that matches the first feature point from the spatial position. Feature points. In the case that the number of candidate feature points with the shortest distance greater than the second threshold is only one, it can be directly determined that the candidate feature points that pass the screening match the first feature point in terms of spatial position and local features, and determine the candidate feature point is the target feature point. When the number of candidate feature points with the shortest distance greater than the second threshold is greater than one, it can be considered that there are multiple candidate feature points whose spatial position and local features match the first feature point. The candidate feature point with the highest matching probability is selected as the target feature point. Exemplarily, in the case that all candidate feature points fail the second screening, the feature point matching process may be ended, and it is determined that there is no second feature point matching the first feature point on the current reference image.

In a possible implementation, since the local feature is the main evaluation parameter in the feature point matching process, the target feature point can be determined according to the similarity parameters between multiple candidate feature points and the first feature point after the second screening . That is to say, when the number of candidate feature points whose shortest distance is greater than the second threshold is greater than one, the above-mentioned determination of the target feature point among the candidate feature points whose shortest distance is smaller than the second threshold can be implemented through step S9031.

Step S9031, among the candidate feature points whose shortest distance is smaller than the second threshold, determine the candidate feature point with the largest similarity parameter as the target feature point.

Fig. 10 shows a schematic diagram of a process of determining target feature points according to an embodiment of the present disclosure. As shown in FIG. 10 , in the embodiment of the present disclosure, after the target image and the reference image are determined, a first feature point 1001 to be matched on the target image and at least one second feature point 1002 to be matched on the reference image are determined. The first screening is performed according to the local feature similarity parameters of the first feature point 1001 and each second feature point 1002 , and at least one candidate feature point 1003 is obtained from at least one second feature point 1002 . A second screening is performed according to the matching degree of each candidate feature point 1003 with the first feature point 1001 in space, and the candidate feature points 1003 that pass the screening are determined as matching feature points 1004 . Determine the number of matching feature points 1005, and determine the target feature point matching the first feature point 1001 according to the number of matching feature points 1005. Wherein, when the number of matching feature points 1005 is equal to 1, the matching feature point 1004 can be directly determined as the target feature point 1006, that is, the candidate feature point 1003 that has passed the second screening is the target feature point. When the number of matching feature points 1005 is greater than 1, the matching feature point 1004 with the largest similarity parameter can be determined as the target feature point 1007, that is, the candidate feature point 1003 with the highest local feature similarity parameter with the first feature point 1001 is the target feature point.

In the embodiment of the present disclosure, when performing feature point matching, the first screening is performed according to the similarity degree of the local features of the feature points, and then the second screening is performed according to the spatial position of the feature points, which solves the problem of matching results only based on local features. The problem of large deviation can accurately match the points that record the same information in the two images.

Fig. 11 shows a flowchart of a feature point matching method according to an embodiment of the present disclosure. As shown in FIG. 11 , the feature point matching method of the embodiment of the present disclosure includes the following steps S1101 to S104.

Step S1101, performing nearest neighbor search on the feature points.

In a possible implementation, for a feature point on the target image (corresponding to the first feature point in the above-mentioned embodiment), first find at least one of the closest descriptors of the feature point on the reference image through nearest neighbor search. Reference feature points (corresponding to the second feature point in the above embodiment), and sort each reference feature point according to the distance of the descriptor, as a candidate feature point.

Step S1102, performing geometric constraint screening on the candidate feature points.

In a possible implementation, after obtaining at least one candidate feature point, firstly, combining the coordinates of the feature point on the target image and the relative positional relationship between the target image and the reference image, the feature is found through epipolar geometry Points on the reference image correspond to the estimated epipolar lines. Then, determine the distance between the candidate feature point and the estimated epipolar line. Finally, based on the distance between each candidate feature point and the estimated epipolar line, each candidate feature point is screened according to a preset threshold, and the candidate feature points whose distance from the estimated epipolar line is smaller than the threshold are retained.

In a possible implementation manner, the process of screening each candidate feature point may be implemented by calculating the epipolar geometric error between the candidate feature point and the feature point in batches. When the number of candidate feature points is fixed, the epipolar geometric error between the candidate feature point and the feature point can be determined by matrix operation with high computational efficiency.

Step S1103, selection of matching points.

In a possible implementation manner, the matching point corresponding to the feature point (corresponding to the target feature point in the above embodiment) is a reference feature point whose descriptor is closest to the feature point among the candidate feature points. In the case that all candidate feature points are eliminated in step S1102, it is considered that no reference feature point corresponding to the feature point is found in the reference image. In the embodiment of the present disclosure, based on the prior of the relative pose between the target image and the reference image, constraints are constructed for the nearest neighbor matching to guide the nearest neighbor matching of the feature points between the two images and the reference feature points. In the process of nearest neighbor matching, for a feature point in the target image, a series of candidate feature points with the closest feature distance in the reference image are first determined, and then the candidate feature points conform to the epipolar line constraint range, and The closest reference feature point of the feature is used as the matching point of the feature point. Therefore, the mismatch between feature points and reference feature points caused by the similarity of local features can be effectively reduced.

It can be understood that the above-mentioned method embodiments mentioned in this disclosure can all be combined with each other to form a combined embodiment without violating the principle and logic. Those skilled in the art can understand that, in the above method in the specific implementation manner, the specific execution order of each step should be determined according to its function and possible internal logic.

In addition, the present disclosure also provides feature point matching devices, electronic equipment, computer-readable storage media, and programs, all of which can be used to implement any of the feature point matching methods provided in the present disclosure. For corresponding technical solutions and descriptions, please refer to the method section corresponding record.

FIG. 12 shows a schematic diagram of a feature point matching device according to an embodiment of the present disclosure. As shown in FIG. The first screening part 1203 and the second screening part 1204 .

The first information determining part 1201 is configured to determine the first feature point on the target image, as well as the first coordinates and the first descriptor of the first feature point, the first coordinates represent the first feature point in The position on the target image, the first descriptor characterizes the feature of the first feature point;

The second information determining part 1202 is configured to determine at least one second feature point on the reference image, as well as the second coordinates and the second descriptor of each of the second feature points, the second coordinates representing the first The position of the two feature points on the reference image, the second descriptor characterizes the features of the second feature point;

The first screening part 1203 is configured to perform a first screening on at least one of the second feature points according to the first descriptor and each of the second descriptors to obtain at least one candidate feature point;

The second screening part 1204 is configured to perform a second screening on at least one of the candidate feature points according to the first coordinates and the second coordinates of each of the candidate feature points to obtain target feature points.

In a possible implementation manner, the first screening part 1203 includes: a parameter determination subsection configured to respectively determine similar parameters of each of the second descriptor and the first descriptor; The subsection is configured to perform a first screening on at least one of the second feature points according to the similarity parameter to obtain at least one candidate feature point.

In a possible implementation manner, the second screening part 1204 includes: a second screening subsection configured to, according to the first coordinates, the second coordinates and the relative positional relationship, screen at least one of the candidate The feature points are screened for the second time to obtain target feature points matching the first feature points; the relative positional relationship is the relative positional relationship between the target image and the reference image.

In some embodiments, the functions or parts included in the apparatus provided by the embodiments of the present disclosure may be configured to execute the methods described in the above method embodiments, and for specific implementation, refer to the descriptions of the above method embodiments.

Embodiments of the present disclosure also provide a computer-readable storage medium, on which computer program instructions are stored, and the above-mentioned method is implemented when the computer program instructions are executed by a processor. Computer readable storage media may be volatile or nonvolatile computer readable storage media.

An embodiment of the present disclosure also proposes an electronic device, including: a processor; a memory configured to store instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

An embodiment of the present disclosure also provides a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes the above method.

Electronic devices may be provided as terminals, servers, or other forms of devices.

Fig. 13 shows a schematic diagram of an electronic device 1300 according to an embodiment of the present disclosure. For example, the electronic device 1300 may be a terminal such as a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, or a personal digital assistant.

Referring to FIG. 13 , electronic device 1300 may include one or more of the following components: processing component 1301, memory 1302, power supply component 1303, multimedia component 1304, audio component 1305, input/output (Input/Ouput, I/O) interface 1306, A sensor component 1307, and a communication component 1308.

The processing component 1301 generally controls the overall operations of the electronic device 1300, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1301 may include one or more processors 1309 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 1301 may include one or more components that facilitate interaction between processing component 1301 and other components. For example, processing component 1301 may include a multimedia portion to facilitate interaction between multimedia component 1304 and processing component 1301 .

The memory 1302 is configured to store various types of data to support operations at the electronic device 1300 . Examples of such data include instructions for any application or method operating on the electronic device 1300, contact data, phonebook data, messages, pictures, videos, and the like. The memory 1302 can be realized by any type of volatile or non-volatile storage device or their combination, such as Static Random-Access Memory (Static Random-Access Memory, SRAM), Electrically Erasable Programmable Read-Only Memory (Electrically -Erasable Programmable Read-Only Memory, EEPROM), Erasable Programmable Read-Only Memory (Erasable Programmable Read-Only Memory, EPROM), Programmable Read-Only Memory (Programmable read-only memory, PROM), Read Only Memory (Read -Only Memory, ROM), magnetic memory, flash memory, magnetic disk or optical disk.

The power supply component 1303 provides power to various components of the electronic device 1300 . Power components 1303 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for electronic device 1300 .

The multimedia component 1304 includes a screen providing an output interface between the electronic device 1300 and the user. In some embodiments, the screen may include a liquid crystal display (Liquid Crystal Display, LCD) and a touch panel (Touch Panel, TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or swipe action, but also detect a duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 1304 includes at least one of the following: a front camera and a rear camera. When the electronic device 1300 is in an operation mode, such as a shooting mode or a video mode, at least one of the front camera and the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.

The audio component 1305 is configured to output and input audio signals, or to output audio signals, or to input audio signals. For example, the audio component 1305 includes a microphone (Microphone, MIC), which is configured to receive external audio signals when the electronic device 1300 is in operation modes, such as calling mode, recording mode and voice recognition mode. Received audio signals may be stored in memory 1302 or sent via communication component 1308 . In some embodiments, the audio component 1305 also includes a speaker configured to output audio signals.

The I/O interface 1306 provides an interface between the processing component 1301 and the peripheral interface part, and the above peripheral interface part may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.

Sensor assembly 1307 includes one or more sensors configured to provide various aspects of status assessment for electronic device 1300 . For example, the sensor component 1307 can detect the open/close state of the electronic device 1300, the relative positioning of components, such as the display and the keypad of the electronic device 1200, the sensor component 1307 can also detect the electronic device 1300 or a Changes in position of components, presence or absence of user contact with electronic device 1300 , electronic device 1300 orientation or acceleration/deceleration and temperature changes in electronic device 1300 . Sensor assembly 1307 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. The sensor assembly 1307 may also include a light sensor, such as a Complementary Metal-Oxide-Semiconductor (CMOS) or Charge Coupled Device (CCD) image sensor, configured for use in imaging applications. In some embodiments, the sensor component 1307 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

The communication component 1308 is configured to facilitate wired or wireless communication between the electronic device 1300 and other devices. The electronic device 1300 can access a wireless network based on a communication standard, such as a wireless network (Wireless Network), a second-generation mobile communication technology (Second Generation, 2G) or a third-generation mobile communication technology (3rd-Generation, 3G), or their The combination. In an exemplary embodiment, the communication component 1308 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1308 also includes a Near Field Communication (NFC) portion to facilitate short-range communication. For example, the NFC part can be based on Radio Frequency Identification (RFID) technology, Infrared Data Association (Infrared Data Association, IrDA) technology, Ultra Wide Band (Ultra Wide Band, UWB) technology, Bluetooth (Bluetooth, BT) technology and other technology to achieve.

In an exemplary embodiment, the electronic device 1300 may be implemented by one or more Application Specific Integrated Circuits (Application Specific Integrated Circuit, ASIC), Digital Signal Processor (Digital Signal Processor, DSP), Digital Signal Processing Device (Digital Signal Processing Device , DSPD), Programmable Logic Device (Programmable Logic Device, PLD), Field Programmable Gate Array (Field Programmable Gate Array, FPGA), controller, microcontroller, microprocessor, or other electronic component implementation, configured to execute the above method.

In an exemplary embodiment, there is also provided a non-volatile computer-readable storage medium, such as a memory 1302 including computer program instructions, which can be executed by the processor 1309 of the electronic device 1300 to implement the above method.

FIG. 14 shows a schematic diagram of another electronic device 1400 according to an embodiment of the present disclosure. For example, the electronic device 1400 may be provided as a server. Referring to FIG. 14 , an electronic device 1400 includes a processing component 1401 , and also includes one or more processors, and a memory resource represented by a memory 1402 configured to store instructions executable by the processing component 1401 , such as application programs. An application program stored in memory 1402 may include one or more portions each corresponding to a set of instructions. In addition, the processing component 1401 is configured to execute instructions to perform the above method.

The electronic device 1400 may also include a power component 1403 configured to perform power management of the electronic device 1400 , a wired or wireless network interface 1404 configured to connect the electronic device 1400 to a network, and an I/O interface 1405 . Electronic device 1400 can operate based on the operating system stored in memory 1402, such as Microsoft server operating system (Windows Server™), based on graphical user interface operating system (Mac OS X™), multi-user multi-process computer operating system (Unix™), free and An open source Unix-like operating system (LinuxTM), an open source Unix-like operating system (FreeBSDTM), or similar.

In an exemplary embodiment, there is also provided a non-volatile computer-readable storage medium, such as a memory 1402 including computer program instructions, which can be executed by the processing component 1401 of the electronic device 1400 to implement the above method.

The present disclosure includes at least one of the following: a system, a method, and a computer program product. A computer program product may include a computer readable storage medium having computer readable program instructions thereon for causing a processor to implement various aspects of the present disclosure.

A computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. Examples (non-exhaustive list) of computer-readable storage media include: portable computer discs, hard drives, Random-Access Memory (RAM), Read-Only Memory (ROM), erasable Programmable Read-Only Memory (Erasable Programmable Read-Only Memory, EPROM) or flash memory, Static Random-Access Memory (Static Random-Access Memory, SRAM), Portable Compression Disk Read-Only Memory (Compact Disc Read-Only Memory, CD-ROM ), Digital Versatile Disc (DVD), memory sticks, floppy disks, mechanically encoded devices, such as punched cards or raised structures in grooves with instructions stored thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.

The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a respective computing/processing device, or downloaded to an external computer or external storage device over a network, such as at least one of the Internet, a local area network, a wide area network, and a wireless network . The network may include at least one of: copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers. A network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

Computer program instructions for performing the operations of the present disclosure may be assembly instructions, Instruction Set Architectures (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or in the form of one or more source or object code written in any combination of programming languages, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as the “C” language or similar programming languages. Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In cases involving a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it can be connected to an external computer such as use an Internet service provider to connect via the Internet). In some embodiments, electronic circuits, such as programmable logic circuits, field programmable gate arrays (Field Programmable Gate Array, FPGA) or programmable logic arrays (Programmable Logic Array, PLA), the electronic circuit can execute computer-readable program instructions, thereby implementing various aspects of the present disclosure.

Aspects of the present disclosure are described herein with reference to at least one of flowchart illustrations and block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of at least one of the flowchart illustrations and block diagrams, and combinations of blocks in the flowchart illustrations and block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that when executed by the processor of the computer or other programmable data processing apparatus , producing an apparatus for realizing the functions/actions specified in one or more blocks in the flowchart and block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause at least one of computers, programmable data processing devices and other devices to work in a specific way, so that the computer-readable The medium then includes an article of manufacture including instructions for implementing various aspects of the functions/acts specified in at least one block in the flowcharts and block diagrams.

It is also possible to load computer-readable program instructions into a computer, other programmable data processing device, or other equipment, so that a series of operational steps are performed on the computer, other programmable data processing device, or other equipment to produce a computer-implemented process , so that instructions executed on computers, other programmable data processing devices, or other devices implement the functions/actions specified in at least one block in the flowcharts and block diagrams.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a portion, a program segment, or a portion of an instruction that includes one or more Executable instructions. In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It is also to be noted that each block of at least one of the block diagrams and flowcharts, and combinations of blocks of at least one of the block diagrams and flowcharts, may be implemented with a dedicated hardware-based computer that performs specified functions or actions. system, or it may be implemented by a combination of special purpose hardware and computer instructions.

The computer program product can be specifically realized by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) etc. wait.

The above descriptions of the various embodiments tend to emphasize the differences between the various embodiments, and the same or similar points can be referred to each other.

Those skilled in the art can understand that in the above method of specific implementation, the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possible The inner logic is OK.

If the disclosed technical solution involves personal information, the products applying the disclosed technical solution have clearly notified the personal information processing rules and obtained the individual's independent consent before processing personal information. If the disclosed technical solution involves sensitive personal information, the products applying the disclosed technical solution have obtained individual consent before processing sensitive personal information, and at the same time meet the requirement of "express consent". For example, at a personal information collection device such as a camera, a clear and prominent sign is set up to inform that it has entered the scope of personal information collection, and personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed to agree to the collection of his personal information; or On the personal information processing device, when the personal information processing rules are informed with obvious signs/information, personal authorization is obtained through pop-up information or by asking individuals to upload their personal information; among them, the personal information processing rules may include Information such as the information processor, the purpose of personal information processing, the method of processing, and the type of personal information processed.

Having described various embodiments of the present disclosure above, the foregoing description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and alterations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terminology used in this disclosure is intended to best explain the principle of each embodiment, practical application or improvement of technology in the market, or to enable other ordinary skilled in the art to understand each embodiment disclosed in this disclosure .

Industrial Applicability

In the embodiment of the present disclosure, by determining the first feature point on the target image, as well as the first coordinate representing the position of the first feature point on the target image and the first descriptor representing the feature of the first feature point; at the same time, determining the reference At least one second feature point on the image, as well as second coordinates representing the position of each second feature point on the reference image and a second descriptor representing the characteristics of the second feature point; according to the first descriptor and each second description Performing a first screening on at least one second feature point to obtain at least one candidate feature point, and then performing a second screening on at least one candidate feature point according to the first coordinates and the second coordinates of each candidate feature point to obtain a reference The target feature point on the image that matches the first feature point. The embodiments of the present disclosure can perform two matchings according to the local features and the positions of the feature points, and accurately match the points that record the same information in the two images.

Claims

A feature point matching method, the method is performed by an electronic device, and the method includes:

determining a first feature point on the target image, as well as first coordinates and a first descriptor of the first feature point, the first coordinates representing the position of the first feature point on the target image, the The first descriptor characterizes the feature of the first feature point;

determining at least one second feature point on the reference image, and second coordinates and second descriptors of each of the second feature points, the second coordinates representing the position of the second feature point on the reference image position, the second descriptor characterizes the feature of the second feature point;

performing a first screening on at least one of the second feature points according to the first descriptor and each of the second descriptors to obtain at least one candidate feature point;

Performing a second screening on at least one of the candidate feature points according to the first coordinates and the second coordinates of each of the candidate feature points to obtain a target feature point matching the first feature point.
The method according to claim 1, wherein the first screening is performed on at least one of the second feature points according to the first descriptor and each of the second descriptors to obtain at least one candidate feature point ,include:

determining the similarity parameters of each of the second descriptor and the first descriptor respectively;

At least one second feature point is screened for the first time according to the similarity parameter to obtain at least one candidate feature point.
The method according to claim 2, wherein said determining similar parameters of each of said second descriptor and said first descriptor respectively comprises:

Determining the distance vectors between each of the second descriptors and the first descriptor respectively to obtain similar parameters between each of the second descriptors and the first descriptor.
The method according to claim 2 or 3, wherein the first screening of at least one second feature point according to the similarity parameters to obtain at least one candidate feature point includes:

determining a second descriptor whose similar parameter to the first descriptor is smaller than a first threshold;

The second feature point corresponding to the second descriptor whose similarity parameter is smaller than the first threshold is used as a candidate feature point.
The method according to claim 2 or 3, wherein the first screening of at least one second feature point according to the similarity parameters to obtain at least one candidate feature point includes:

The similar parameters corresponding to each of the second descriptors are sorted from small to large, and the second feature points corresponding to the previous preset number of second descriptors are obtained as candidate feature points.
The method according to any one of claims 2 to 5, wherein the second screening is performed on at least one of the candidate feature points according to the first coordinates and the second coordinates of each of the candidate feature points , to obtain the target feature point matched with the first feature point, including:

According to the first coordinates, the second coordinates and the relative positional relationship, at least one of the candidate feature points is screened a second time to obtain a target feature point matching the first feature point; the relative positional relationship is the relative positional relationship between the target image and the reference image.
The method according to claim 6, wherein, according to the first coordinates, the second coordinates, and the relative positional relationship between the target image and the reference image, the second step is performed on at least one of the candidate feature points. Secondary screening to obtain target feature points matching the first feature point, including:

determining a first location of the target image, and determining a second location of the reference image;

determining an estimated epipolar line of the first feature point on the reference image based on epipolar geometry according to the first position, the second position, and the first coordinate;

According to the shortest distance between the second coordinates of each of the candidate feature points and the estimated epipolar line, at least one of the candidate feature points is screened a second time to obtain a target feature point that matches the first feature point. .
The method according to claim 7, wherein the second screening is performed on at least one of the candidate feature points according to the shortest distance between the second coordinates of each of the candidate feature points and the estimated epipolar line, Obtaining the target feature point matched with the first feature point, including:

determining the number of candidate feature points whose shortest distance is less than a second threshold among the at least one candidate feature point; the shortest distance is the shortest distance between the second coordinate and the estimated epipolar line;

When the number of candidate feature points whose shortest distance is less than a second threshold is one, using the candidate feature points whose shortest distance is less than a second threshold as target feature points;

If the number of candidate feature points whose shortest distance is smaller than a second threshold is greater than one, determine a target feature point among candidate feature points whose shortest distance is smaller than a second threshold.
The method according to claim 8, wherein said determining a target feature point among candidate feature points whose shortest distance is less than a second threshold comprises:

Among the candidate feature points whose shortest distance is smaller than a second threshold, determine the candidate feature point with the largest similarity parameter as the target feature point.
A feature point matching device, said device comprising:

The first information determining part is configured to determine a first feature point on the target image, as well as first coordinates and a first descriptor of the first feature point, the first coordinates characterize the first feature point at the The position on the target image, the first descriptor characterizes the feature of the first feature point;

The second information determining part is configured to determine at least one second feature point on the reference image, as well as the second coordinates and the second descriptor of each of the second feature points, the second coordinates representing the second The position of the feature point on the reference image, the second descriptor characterizes the feature of the second feature point;

The first screening part is configured to perform a first screening on at least one of the second feature points according to the first descriptor and each of the second descriptors to obtain at least one candidate feature point;

The second screening part is configured to perform a second screening on at least one of the candidate feature points according to the first coordinates and the second coordinates of each of the candidate feature points, to obtain the matching of the first feature point Target feature points.
The apparatus according to claim 10, wherein the first screening part is further configured to determine similar parameters of each of the second descriptor and the first descriptor respectively;

At least one second feature point is screened for the first time according to the similarity parameter to obtain at least one candidate feature point.
The device according to claim 11, wherein the first screening part is further configured to separately calculate a distance vector between each of the second descriptors and the first descriptor to obtain each of the second descriptors sub and similar parameters of the first descriptor.
The device according to claim 11 or 12, wherein the first screening part is further configured to determine a second descriptor whose similar parameter to the first descriptor is smaller than a first threshold;

The second feature point corresponding to the second descriptor whose similarity parameter is smaller than the first threshold is used as a candidate feature point.
The device according to claim 11 or 12, wherein the first screening part is further configured to sort the similar parameters corresponding to each of the second descriptors from small to large, and obtain the first preset number of the The second feature point corresponding to the second descriptor is a candidate feature point.
The device according to any one of claims 11 to 14, wherein the second screening part is further configured to, according to the first coordinates, the second coordinates and the relative positional relationship, at least one of the candidate The feature points are screened for the second time to obtain target feature points matching the first feature points; the relative positional relationship is the relative positional relationship between the target image and the reference image.
The apparatus of claim 15, wherein the second screening portion is further configured to determine a first position of the target image, and determine a second position of the reference image;

determining an estimated epipolar line of the first feature point on the reference image based on epipolar geometry according to the first position, the second position, and the first coordinate;

According to the shortest distance between the second coordinates of each of the candidate feature points and the estimated epipolar line, at least one of the candidate feature points is screened a second time to obtain a target feature point that matches the first feature point. .
The apparatus according to claim 16, wherein the second screening part is further configured to determine the number of candidate feature points whose shortest distance is smaller than a second threshold among the at least one of the candidate feature points; the shortest distance is The shortest distance between the second coordinate and the estimated epipolar line;

When the number of candidate feature points whose shortest distance is less than a second threshold is one, using the candidate feature points whose shortest distance is less than a second threshold as target feature points;

If the number of candidate feature points whose shortest distance is smaller than a second threshold is greater than one, determine a target feature point among candidate feature points whose shortest distance is smaller than a second threshold.
The device according to claim 17, wherein the second screening part is further configured to, among the candidate feature points whose shortest distance is smaller than a second threshold, determine the candidate feature point with the largest similarity parameter as the target feature point .
An electronic device comprising:

processor;

memory configured to store processor-executable instructions;

Wherein, the processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1-9.
A computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 9 is realized.
A computer program product, the computer program product comprising a non-transitory computer storage medium storing a computer program, when the computer program is read and executed by a computer, the method according to any one of claims 1 to 9 is realized.