WO2022142206A1 - 图像匹配的方法及装置、电子设备及车辆 - Google Patents
图像匹配的方法及装置、电子设备及车辆 Download PDFInfo
- Publication number
- WO2022142206A1 WO2022142206A1 PCT/CN2021/102929 CN2021102929W WO2022142206A1 WO 2022142206 A1 WO2022142206 A1 WO 2022142206A1 CN 2021102929 W CN2021102929 W CN 2021102929W WO 2022142206 A1 WO2022142206 A1 WO 2022142206A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- pixel point
- matching
- matched
- pixel
- standard
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000006243 chemical reaction Methods 0.000 claims abstract description 148
- 230000009466 transformation Effects 0.000 claims description 34
- 238000006073 displacement reaction Methods 0.000 claims description 21
- 230000006870 function Effects 0.000 claims description 21
- 230000005484 gravity Effects 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 13
- 238000009795 derivation Methods 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 10
- 230000011218 segmentation Effects 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 4
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 12
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 239000000284 extract Substances 0.000 description 5
- 238000013507 mapping Methods 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000001427 coherent effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000000513 principal component analysis Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
Definitions
- the present application relates to the technical field of image processing, and in particular, to an image matching method and apparatus, electronic device and vehicle.
- Vision-based positioning technology is more and more widely used due to its high-precision characteristics.
- vision-based positioning technology is used in intelligent transportation and other fields.
- vision-based positioning technology it is necessary to accurately determine the matching relationship between the pixels on the object in the two images, and then use the matching relationship between the pixels to achieve positioning.
- the above-mentioned matching process has the defects of low matching accuracy and low efficiency.
- the embodiments of the present application provide at least an image matching method and apparatus, electronic device, and vehicle, so as to improve the accuracy and efficiency of pixel point matching in an image.
- an embodiment of the present application provides an image matching method, including: acquiring a pixel point set to be matched corresponding to the target object in the image to be matched, and a standard pixel point set corresponding to the target object in the standard image; determining Conversion relationship information between the pixel point set to be matched and the standard pixel point set; pixel points to obtain multiple matching point pairs; based on the multiple matching point pairs, determine matching relationship information between the pixel points in the standard pixel point set and the pixel points in the to-be-matched pixel point set.
- the pixel points of the contour of the target object are used, which can effectively reduce the number of pixels to be processed for matching and improve the matching efficiency.
- the transformation relation information includes at least one derivable transformation relation item.
- the derivable transformation relationship information can realize the optimization of the transformation relationship information on the basis of currently obtained matching point pairs, thereby helping to improve the accuracy of pixel point matching of target objects in different images.
- the to-be-matched image includes a scene image
- the standard image includes a pre-made map
- the method further includes: determining geographic location information of the target object based on the matching relationship information; The geographic location information of the target object determines the geographic location information of the device that captures the scene image.
- the positioning accuracy of the target object can be improved, so that the positioning accuracy of the device that captures the scene image can be improved.
- the acquiring the pixel point set to be matched corresponding to the target object in the image to be matched and the standard pixel point set corresponding to the target object in the standard image includes: acquiring the pixel point set in the image to be matched The set of pixels to be matched corresponding to the contour of the target object and the standard set of pixels corresponding to the contour of the target object in the standard image.
- the acquiring the pixel point set to be matched corresponding to the contour of the target object in the image to be matched and the standard pixel point set corresponding to the contour of the target object in the standard image includes: acquiring the target The to-be-matched image and the standard image of the object; extract the to-be-matched pixel point set corresponding to the contour of the target object from the semantic segmentation image corresponding to the to-be-matched image; extract the The standard pixel point set corresponding to the outline of the target object.
- the pixel points of the contour of the target object are extracted from the semantic segmentation map of the image to be matched, and the pixels of the contour of the target object are extracted from the two-dimensional projection image of the standard image, which can improve the efficiency of pixel point extraction and Accuracy.
- the conversion relationship information includes at least one of a first conversion relationship item related to rotation information, a second conversion relationship item related to displacement information, and a third conversion relationship item related to scaling information;
- the determining the conversion relationship information between the pixel point set to be matched and the standard pixel point set includes at least one of the following: based on the orientation information of the pixel point set to be matched and the orientation of the standard pixel point set information, determine the rotation information, and determine the first conversion relationship item based on the rotation information; determine the displacement information based on the center of gravity of the pixel point set to be matched and the center of gravity of the standard pixel point set, and Determine the second conversion relationship item based on the displacement information; determine the scaling information based on the image area corresponding to the pixel point set to be matched and the image area corresponding to the standard pixel point set, and determine the scaling information based on the scaling information
- the third conversion relationship term is determined.
- the conversion relationship information includes three conversion relationship items that can be derived. Compared with the conversion relationship that cannot be derived in the prior art, the success rate of pixel point matching can be improved; This method can more accurately determine the values of the three conversion relation items, so that the accuracy of pixel point matching can be improved.
- the extracting the standard pixel point set corresponding to the contour of the target object from the two-dimensional projection image corresponding to the standard image includes: extracting the target object from the standard image the corner point information of the contour; perform upsampling processing on the corner points corresponding to the extracted corner point information to obtain a three-dimensional contour point set; project each pixel point in the three-dimensional contour point set to the corresponding two-dimensional projection image In the plane of , the standard pixel point set is obtained.
- the selecting pixel points that match the pixel points in the standard pixel point set from the pixel point set to be matched to obtain a plurality of matching point pairs includes: using the to-be-matched pixel point set Pixel point set, construct a K-dimensional search tree; wherein, K is equal to 2; for each pixel point in the standard pixel point set, the K-dimensional search tree is traversed to select and match from the pixel point set to be matched.
- the pixel points that match the pixel points are used to form a matching point pair with the pixel points and the acquired pixel points.
- the set of pixels to be matched is constructed into a K-dimensional data structure tree, and the tree structure can improve the speed of traversing the pixels in the set of pixels to be matched, and further improve the efficiency of pixel matching.
- the pixel points that match the pixel points in the standard pixel point set are selected from the pixel point set to be matched to obtain a plurality of matching point pairs, Including: using the conversion relationship information, mapping each pixel point in the standard pixel point set to the coordinate system corresponding to the pixel point set to be matched; for each pixel point in the standard pixel point set after the coordinate system conversion , select the pixel point closest to the pixel point from the set of pixel points to be matched, and use the pixel point and the closest pixel point to form a matching point pair.
- the matching method of pixel point mapping and finding the closest point can improve the accuracy of pixel point matching.
- using the pixel point and the nearest pixel point to form a matching point pair includes: determining that the distance between the pixel point and the nearest pixel point is not greater than a matching threshold.
- the noise in the pixel point to be matched can be effectively reduced, the accuracy of pixel point matching can be improved, and the conversion relationship information can be improved at the same time. Update accuracy.
- the method further includes: according to the descending order of the distance between the two pixel points in each matching point pair, The matching point pairs are sorted; the matching threshold is updated based on the distance between the two pixel points in the matching point pairs with the preset sorting order.
- the matching threshold is updated by using the distance between two pixels in the matching point pair with a preset sorting order, and the updated matching threshold can be used to more effectively reduce the influence of noise in the set of pixels to be matched.
- the accuracy of pixel point matching is improved, and at the same time, the accuracy of updating the conversion relationship information by matching point pairs can be improved.
- the method includes: based on the conversion relationship information, respectively selecting a pixel point matching each pixel point in the standard pixel point set from the pixel point set to be matched, to obtain a plurality of matching point pairs; if the iteration stop condition is not reached In the case of , update the conversion relationship information based on the plurality of matching point pairs, and return the conversion relationship information based on the to-be-matched pixel point set to select and each of the standard pixel point set respectively. Steps for matching pixels to pixels.
- the conversion relationship information is updated by using the matching point pairs obtained by matching, so that more accurate conversion relationship information can be obtained, and a more accurate affine transformation relationship can be obtained, and then the more accurate conversion relationship information can be used to improve the subsequent
- the accuracy of the determined matching point pair can improve the accuracy of subsequent pixel point matching.
- the iterative stop condition includes: an average distance between two pixel points in each of the matching point pairs is less than a first preset threshold.
- the conversion relationship information is updated and the iteration is continued to determine a matching point pair with higher matching accuracy; in the matching point pair, two pixels When the mean value of the distance between points is relatively small, the iteration is stopped.
- the matching point pair obtained at this time is relatively accurate. Even if the iteration continues to obtain new matching point pairs, the accuracy will not be significantly improved, and computing resources will be wasted.
- the iteration stop condition includes: when the difference value corresponding to the difference value information between the conversion relationship information corresponding to the round iteration and the conversion relationship information corresponding to the previous round iteration is smaller than the second preset threshold .
- the conversion relationship information when the conversion relationship information changes greatly, it indicates that the accuracy of the matching point pair determined by using the current conversion relationship information is low, and it is necessary to continue the iteration after updating the conversion relationship information to improve the accuracy of pixel point matching;
- the iteration is stopped, and the matching point pair obtained at this time is relatively accurate. Even if the conversion relationship information is continued to be iteratively obtained to obtain a new matching point pair, the accuracy will not be significantly improved, and computing resources will be wasted.
- the iterative stop condition includes: the execution times of selecting the pixel points matching each pixel point in the standard pixel point set from the pixel point set to be matched respectively is greater than the number of execution times. Three preset thresholds.
- the accuracy of the determined matching point pairs is low, and it is necessary to continue to iterate after updating the conversion relationship information, so as to improve the accuracy of pixel point matching; If there are too many, stop the iteration, and the matching point pairs obtained at this time are already relatively accurate. Even if the iteration continues to obtain new matching point pairs, the accuracy will not be significantly improved, and computing resources will be wasted.
- the updating the conversion relationship information based on the plurality of matching point pairs includes: constructing a residual function with the conversion relationship information as a variable based on the plurality of matching point pairs; A derivation operation is performed on the residual function, and the updated conversion relationship information is determined with the value of the residual function approaching zero as a target.
- the conversion relationship information is updated with the goal that the value of the residual function tends to zero, which can improve the accuracy of the matching point pair determined by using the conversion relationship.
- the present application provides an image matching device, comprising: a pixel point extraction module, configured to obtain a pixel point set to be matched corresponding to a target object in a to-be-matched image, and a standard corresponding to the target object in the standard image a pixel point set; a transformation processing module for determining the conversion relationship information between the to-be-matched pixel point set and the standard pixel point set; a matching module for converting from the to-be-matched pixel point set based on the conversion relationship information
- the point set selects the pixel points that match the pixel points in the standard pixel point set to obtain a plurality of matching point pairs; the relationship forming module is used to determine the pixels in the standard pixel point set based on the plurality of matching point pairs. Matching relationship information between the point and the pixel points in the set of pixel points to be matched.
- embodiments of the present application provide an electronic device, including: a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the processing The processor and the memory communicate through a bus, and the machine-readable instructions, when executed by the processor, perform the steps of the image matching method described above.
- an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is run by a processor, the steps of the above image matching method are executed.
- an embodiment of the present application provides a vehicle, including the electronic device provided in the third aspect.
- FIG. 1 shows a flowchart of a method for image matching provided by an embodiment of the present application
- FIGS. 2A, 2B, and 2C show schematic diagrams of standard pixel point sets in an embodiment of the present application
- FIG. 3 shows a schematic diagram of the contour corresponding to the standard pixel point set and the contour corresponding to the pixel point set to be matched in the embodiment of the present application;
- FIG. 5 shows a schematic diagram of a K-D tree provided by an embodiment of the present application.
- FIG. 6 shows a schematic diagram of an image matching apparatus provided by an embodiment of the present application.
- FIG. 7 shows a schematic diagram of an electronic device provided by an embodiment of the present application.
- the present application provides an image matching method and device.
- the present application uses the pixels of the contour of the target object, which can effectively reduce the matching requirements.
- the conversion relationship information used is an affine relationship that can be derived, which is beneficial to improve the accuracy of pixel point matching of target objects in different images.
- the image matching method provided by the embodiments of the present application is completed by a device with computing capability, such as a mobile terminal device. Specifically, as shown in FIG. 1 , the image matching method provided by the embodiment of the present application may include the following steps:
- S110 Acquire a pixel point set to be matched corresponding to the target object in the image to be matched, and a standard pixel point set corresponding to the target object in the standard image.
- a preliminary object matching needs to be performed between the to-be-matched image and the standard image to obtain the object existing in both the to-be-matched image and the standard image, that is, the above-mentioned target object.
- the target object After obtaining the target object, it is still impossible to use the target object to achieve positioning, and it is necessary to further determine the matching relationship information of the pixels on the target object in the two images, that is, the following multiple matching point pairs and the corresponding affine transformation relationship , that is, the following conversion relationship information.
- Visual positioning can be achieved only by using the matching relationship information and affine transformation relationship of the pixels of the target object in the two images.
- the pixel points in the above-mentioned pixel point set to be matched correspond to the target object in the image to be matched, and the number of pixels is relatively large; the pixel points of the standard pixel point set correspond to the target object in the standard image, and the number is relatively small.
- a matching pixel point can be found from the pixel point set to be matched.
- a dense point cloud such as an object in an image
- its effective information is usually located at the contour, such as a road sign on a road, so only the pixels at the contour of the object can be matched, without the need for Match all pixels on the object.
- this step can be performed as follows: acquiring the pixel point set to be matched corresponding to the contour of the target object in the image to be matched, and the standard pixel point set corresponding to the contour of the target object in the standard image.
- the pixels corresponding to the contour of the target object can be extracted, which can effectively reduce the number of pixels that need to be processed for matching and improve the efficiency of matching without losing the effective information of the target object. The positioning accuracy is affected.
- the above-mentioned images to be matched may include scene images, and the standard images may include pre-made maps, so that visual positioning can be realized based on the matching relationship information and conversion relationship information determined in the present application.
- the standard image is projected onto a two-dimensional plane to obtain its corresponding two-dimensional projected image, and then a standard pixel set corresponding to the contour of the target object is extracted from the two-dimensional projected image.
- a standard pixel set corresponding to the contour of the target object is extracted from the two-dimensional projected image.
- only useful corner points can be stored on the standard image, so the image projected onto the two-dimensional projection plane also includes only the projection points corresponding to the above-mentioned corner points.
- the point is upsampled, and the projected point and the upsampled point are connected, and the pixels on the connection are considered as pixels on the contour of the target object.
- the corner point information of the contour of the target object can be extracted from the standard image first; the corner points corresponding to the extracted corner point information can be upsampled to obtain a three-dimensional contour point set; Each pixel point in the point set is projected onto a plane corresponding to the two-dimensional projection image to obtain the standard pixel point set.
- FIG. 2A , FIG. 2B , and FIG. 2C that is, the standard pixel point set 21 , the standard pixel point set 22 , and the standard pixel point set 23 obtained by the above method.
- the extraction of the pixel points of the contour of the target object from the semantic segmentation map of the image to be matched and the extraction of the pixel points of the contour of the target object from the two-dimensional projection image of the standard image can improve the efficiency and accuracy of pixel point extraction.
- the conversion relationship information is used to map the standard pixel point set to the plane where the pixel point set to be matched is located. distance, and determine matching point pairs.
- the transformation relationship information may be a derivable affine transformation relationship.
- the transformation relationship information includes at least one derivable transformation relationship item.
- the conversion relationship information in this application may include a first conversion relationship item related to rotation information, a second conversion relationship item related to displacement information, a third conversion relationship item related to scaling information, and the like.
- the conversion relationship information or affine transformation relationship formed by the above-mentioned rotation information, displacement information, and scaling information, etc. can be as shown in the following formula, wherein the rotation information includes a rotation angle, and the scaling information includes a scaling ratio:
- A represents the above-mentioned conversion relationship information or affine transformation relationship
- s represents the scaling ratio, which can specifically be the ratio of the image area of the target object in the to-be-matched image to the image area of the target object in the standard image
- ⁇ represents the rotation angle, the rotation angle can specifically be the angle between the direction of the target object in the image to be matched and the direction of the target object in the standard image
- a represents the horizontal ratio, and the horizontal ratio can specifically be the width of the image to be matched and the standard image.
- the difference between the x-axis coordinate of the object and the x-axis coordinate of the target object in the standard image; ty represents the y-axis displacement, which may specifically be the y-axis coordinate of the target object in the image to be matched and the target object in the standard image.
- the difference between the y-axis coordinates of . tx and ty constitute the above displacement information.
- the above conversion relationship information may further include the lateral ratio a and the inclination angle ⁇ . Generally, set the tilt angle to 0 and the lateral ratio to 1.
- the above image area, direction and coordinates are all determined by using the position coordinates of the corresponding point set on the same two-dimensional image.
- the conversion relationship information puts the rotation angle, scaling ratio, lateral ratio, tilt angle, and displacement information in different matrices, so that there is usually only one variable in each matrix, and in special cases, there are at most one variable. Two, and the rest are filled with 0 and 1, so that the derivation is easy, and the results obtained by derivation of each matrix have corresponding physical meanings. Only by using the derivation results with physical meaning can the transformation relationship information be better optimized, and the subsequent pixel mapping and pixel matching accuracy based on the transformation relationship information can be improved. As shown in FIG.
- the contour 31 corresponding to the standard pixel point set and the contour 32 corresponding to the pixel point set to be matched are not converted by the ordinary Euclidean transformation relationship of rotation-translation, but by the above-mentioned affine transformation relationship. converted.
- the conversion relationship information can be used first to map each pixel in the standard pixel set to the coordinate system corresponding to the to-be-matched pixel set.
- the standard pixel set can be The coordinates of the pixel points are multiplied by the matrix A corresponding to the above conversion relationship information to obtain the coordinates of the pixel point in the coordinate system corresponding to the set of pixel points to be matched:
- [x y 1] T represents the coordinates of a pixel in the standard pixel set
- [u v 1] T represents the coordinates of the pixel in the coordinate system corresponding to the pixel set to be matched.
- the pixel point closest to the pixel point is selected from the pixel point set to be matched, and the pixel point and the nearest pixel point are used to form A matching point pair.
- the matching method of pixel point mapping and finding the closest point can improve the accuracy of pixel point matching.
- a set of pixels to be matched can be used to construct a K-dimensional search tree, that is, a K-D tree; where K is equal to 2, which represents the dimension of the divided space.
- the above binary tree can be traversed to select the pixel point matching the pixel point from the to-be-matched pixel point set, and use the The pixel point and the acquired pixel point form a matching point pair.
- the established K-D number structure is shown in Figure 5.
- the coordinates obtained by each pixel in the set of pixels to be matched constitute each leaf node in FIG. 5 .
- the root node and intermediate nodes store some spatial division information, such as division dimensions and division values.
- the K-D tree when using the K-D tree for pixel point matching, it does not traverse all nodes, and does not calculate the distance from all the pixel points in the set of pixels to be matched, so the calculation amount of traversal can be reduced and the matching efficiency can be improved.
- the complexity of traversal search can be reduced from N to log N.
- N represents the number of pixels in the set of pixels to be matched. It can be seen that the above tree structure can improve the speed of traversing the pixels in the set of pixels to be matched, and further improve the efficiency of pixel matching.
- the distance between two pixels in a matching point pair is not greater than the matching threshold. For example, a matching point pair whose distance between two pixel points in a matching point pair is greater than a matching threshold may be eliminated, so as to improve the accuracy of pixel point matching.
- the above matching threshold can be set according to actual application scenarios and application requirements, and can also be updated in the process of pixel point matching, so as to further improve the accuracy of pixel point matching.
- multiple matching point pairs and conversion relationship information may be used as matching relationship information between the pixel points in the standard pixel point set and the pixel points in the to-be-matched pixel point set.
- the above embodiment uses the pixel points of the contour of the target object to perform matching, which can effectively reduce the number of pixel points that need to be processed for matching and improve the matching efficiency.
- the conversion relationship information used is an affine relationship that can be derived, that is, the optimization of the conversion relationship information can be realized on the basis of the currently obtained matching point pairs, so that there are It is beneficial to improve the accuracy of pixel point matching of target objects in different images.
- the image to be matched may include a scene image, and the standard image may include a pre-made map. Then, the following steps can be used to realize positioning: determining the geographic location information of the target object based on the matching relationship information; determining the geographic location information of the device capturing the scene image based on the geographic location information of the target object.
- the shooting parameter information of the device that shoots the scene image should also be combined.
- the above-mentioned use of relatively accurate matching relationship information can improve the positioning accuracy of the target object, thereby improving the positioning accuracy of the device that captures the scene image.
- the above-determined geographic location information can be applied to the fields of automatic driving, intelligent driving, robots, and the like.
- Step S130 is performed only once, and the determined matching point pairs are rough and have poor accuracy. In order to improve the accuracy of pixel point matching, the above-mentioned step S130 needs to be performed multiple times. Specifically, step S130 can be implemented by the following steps:
- Step 1 Based on the conversion relationship information, select a pixel point matching each pixel point in the standard pixel point set from the to-be-matched pixel point set to obtain a plurality of matching point pairs.
- the method for determining the matching point pair here is the same as the method in the foregoing embodiment, and will not be repeated here.
- Step 2 In the case where the iteration stop condition is not reached, update the conversion relationship information based on the plurality of matching point pairs, and return the conversion relationship information based on the set of pixels to be matched.
- step 1 before step 1 is iteratively executed, the conversion relationship information needs to be updated, and then the new conversion relationship information is used to perform pixel point matching to generate a new matching point pair.
- the conversion relationship information can be updated using the following steps:
- a residual function is constructed with the conversion relationship information as a variable; the derivation operation is performed on the residual function, and the value of the residual function tends to zero as the goal, and the updated conversion relationship information.
- L represents the value of the residual function
- i represents the ith matching point pair
- m represents the number of matching point pairs
- q i represents the coordinates of the pixel points in the standard pixel set in the ith matching point pair
- p i Indicates the coordinates of the pixels in the set of pixels to be matched in the i-th matching point pair.
- the updated conversion relationship information can be determined with the value of the residual function approaching zero as the goal, so that the optimal conversion relationship information can be obtained, Using the optimal conversion relationship can improve the accuracy of pixel point matching.
- the initial value of the conversion relationship information can be calculated using the following steps:
- the Principal Component Analysis (PCA) method can be used first to process the pixel point set to be matched and the standard pixel point set respectively to obtain the orientation information of the pixel point set to be matched and the orientation information of the standard pixel point set.
- the orientation information of the above-mentioned pixel point set to be matched is the orientation of the target object in the image to be matched
- the orientation information of the standard pixel point set is the orientation of the target object in the standard image.
- the included angle between the direction of the target object in the image to be matched and the direction of the target object in the standard image is calculated, and the included angle ⁇ is obtained as the above-mentioned rotation information.
- the center of gravity of the pixel point set to be matched and the center of gravity of the standard pixel point set may be determined first, and then the above displacement information may be determined based on the displacement between the two centers of gravity. Specifically, two displacement amounts between the centers of gravity can be used as the above-mentioned displacement information.
- the center of gravity of the pixel point set to be matched can be obtained by weighting and summing the coordinates of each pixel point in the pixel point set to be matched.
- the center of gravity of the standard pixel point set can weight the coordinates of each pixel point in the standard pixel point set. Ask and get.
- the displacement amounts tx and ty between the two centers of gravity can be calculated.
- the image area corresponding to the pixel point set to be matched and the image area corresponding to the standard pixel point set may be determined first, and then the scaling information may be determined based on the ratio s of the two image areas. Specifically, the above-mentioned ratio s may be used as the above-mentioned scaling information.
- the image area corresponding to the above-mentioned pixel point set to be matched may specifically be the image area of the target object in the to-be-matched image, which may be determined by using the coordinates of each pixel point in the to-be-matched pixel point set.
- the image area corresponding to the standard pixel point set may specifically be the image area of the target object in the standard image, which may be determined by using the coordinates of each pixel point in the standard point set.
- the initial value of the inclination angle ⁇ in the conversion relationship information may be set to 0 degrees, and the initial value of the lateral ratio a in the conversion relationship information may be set to 1.
- the accurate initial value of the conversion relationship information can improve the iteration speed, that is, improve the speed of pixel point matching, and improve the success rate of pixel point matching, so as to avoid falling into the local optimal solution in each iteration and causing the matching failure.
- the conversion relationship information includes rotation information, displacement information, scaling information, etc., which constitute an affine transformation relationship that can be derived. Compared with the affine transformation relationship that cannot be derived in the prior art, it can better The transformation relationship information is optimized, and the high-quality transformation relationship information can be used to improve the success rate of pixel point matching; in addition, this embodiment uses the morphological method to more accurately determine the value in the affine transformation relationship, so that the pixel point matching can be improved. The accuracy of point matching.
- the initial value of the conversion relationship information is the first iteration, that is, the conversion relationship information used in the first step is executed for the first time, and the conversion relationship information needs to be updated before the subsequent step 1 is executed.
- the matching threshold can be updated using the matching point pairs obtained after each iteration, which can be implemented by the following steps:
- the distance between two pixel points in the matching point pair with the preset sorting order may be used as the updated matching threshold.
- Scheme 1 based on the distance between two pixels in each pair of matching points, determine the mean value of the distance between two pixels in the pair of matching points; when the mean value is greater than or equal to the first preset threshold In this case, it is determined that the iterative stop condition is not reached; when the mean value is less than the first preset threshold, it is determined that the iteration stop condition is reached, the iteration is stopped, and the currently obtained multiple matching point pairs are used as the final matching point pairs , the current conversion relationship information is used as the final conversion relationship information.
- the conversion relationship information is updated and then the iteration is continued to determine the matching point pair with higher matching accuracy; If the average value of the distance between them is relatively small, the iteration is stopped, and the matching point pairs obtained at this time are relatively accurate. Even if the iteration continues to obtain new matching point pairs, the accuracy will not be significantly improved, and computing resources will be wasted.
- Scheme 2 Determine the difference information between the conversion relationship information corresponding to the current iteration and the conversion relationship information corresponding to the previous iteration; in the case that the difference corresponding to the difference information is greater than or equal to a second preset threshold , it is determined that the iteration stop condition is not reached; in the case that the difference value corresponding to the difference information is less than the second preset threshold, it is determined that the iteration stop condition is reached, the iteration is stopped, and the currently obtained multiple matching point pairs are used as the final The matching point pair of , the current conversion relationship information is used as the final conversion relationship information.
- the conversion relationship information when the conversion relationship information changes greatly, it means that the matching point pair determined by the current conversion relationship information has a low accuracy, and it is necessary to update the conversion relationship information and continue to iterate to improve the accuracy of pixel matching; when the conversion relationship information changes When the value is small, the iteration is stopped, and the matching point pair obtained at this time is relatively accurate. Even if the conversion relationship information is used to iterate to obtain a new matching point pair, the accuracy will not be significantly improved, and computing resources will be wasted.
- Scheme 3 Determine the execution times of selecting the pixel points matching each pixel point in the standard pixel point set from the to-be-matched pixel point set; when the execution times are less than or equal to a third preset threshold In the case of , it is determined that the iteration stop condition is not reached; when the number of executions is greater than the third preset threshold, it is determined that the iteration stop condition is reached, the iteration is stopped, and the currently obtained multiple matching point pairs are used as the final match Point-to-point, the current conversion relationship information is used as the final conversion relationship information.
- the present application also provides an image matching apparatus, which is applied to a terminal for performing the above image matching method, and can achieve the same or similar beneficial effects.
- the apparatus can perform all steps of the image matching method in the above-mentioned embodiments, so the same steps are not repeated here.
- the image matching apparatus provided by this application includes:
- the pixel point extraction module 610 is configured to obtain the pixel point set to be matched corresponding to the target object in the to-be-matched image, and the standard pixel point set corresponding to the target object in the standard image.
- the transformation processing module 620 is configured to determine transformation relationship information between the pixel point set to be matched and the standard pixel point set.
- the matching module 630 is configured to select, based on the conversion relationship information, pixel points that match the pixel points in the standard pixel point set from the to-be-matched pixel point set to obtain a plurality of matching point pairs.
- the relationship forming module 640 is configured to, based on the plurality of matching point pairs, determine matching relationship information between the pixel points in the standard pixel point set and the pixel points in the to-be-matched pixel point set.
- the transformation relationship information includes at least one derivable transformation relationship term.
- the image to be matched includes a scene image
- the standard image includes a pre-made map
- the image matching apparatus further includes a positioning module 650, configured to: determine the geographic location information of the target object based on the matching relationship information; determine to shoot the scene image based on the geographic location information of the target object The geographic location information of the device.
- the pixel point extraction module 610 is configured to obtain a set of pixels to be matched corresponding to the contour of the target object in the image to be matched, and a set of standard pixels corresponding to the contour of the target object in the standard image.
- the pixel point extraction module 610 obtains the pixel point set to be matched corresponding to the contour of the target object in the image to be matched, and the standard pixel point set corresponding to the contour of the target object in the standard image, use In: acquiring the to-be-matched image and the standard image including the target object; extracting the to-be-matched pixel point set corresponding to the contour of the target object from the semantic segmentation image corresponding to the to-be-matched image; from the two-dimensional corresponding to the standard image A standard pixel set corresponding to the contour of the target object is extracted from the projection image.
- the conversion relationship information includes at least one of a first conversion relationship item related to rotation information, a second conversion relationship item related to displacement information, and a third conversion relationship item related to scaling information.
- the transformation processing module 620 is configured to: based on the orientation information of the pixel point set to be matched and the Orientation information of the standard pixel point set, determine the rotation information, and determine the first conversion relationship item based on the rotation information; and/or, based on the center of gravity of the pixel point set to be matched and the standard pixel point set and/or, based on the image area corresponding to the pixel point set to be matched and the image corresponding to the standard pixel point set area, the scaling information is determined, and the third conversion relationship term is determined based on the scaling information.
- the pixel point extraction module 610 extracts the standard pixel point set corresponding to the contour of the target object from the two-dimensional projection image corresponding to the standard image
- the following steps are used to: extract the standard pixel point set from the standard image Extracting corner point information of the contour of the target object; performing up-sampling processing on the corner points corresponding to the extracted corner point information to obtain a three-dimensional contour point set; projecting each pixel point in the three-dimensional contour point set to the In the plane corresponding to the two-dimensional projection image, the standard pixel point set is obtained.
- the matching module 630 when the matching module 630 selects the pixel points matching the pixel points in the standard pixel point set from the to-be-matched pixel point set to obtain a plurality of matching point pairs, the matching module 630 is used for: using the Describe the pixel point set to be matched, and construct a K-dimensional search tree; wherein, K is equal to 2; for each pixel point in the standard pixel point set, the K-dimensional search tree is traversed to obtain from the pixel point to be matched.
- K is equal to 2
- the K-dimensional search tree is traversed to obtain from the pixel point to be matched.
- the matching module 630 selects, based on the conversion relationship information, pixel points that match the pixel points in the standard pixel point set from the pixel point set to be matched to obtain a plurality of matching point pairs is used for: using the conversion relationship information to map each pixel in the standard pixel set to the coordinate system corresponding to the pixel set to be matched; for each pixel in the standard pixel set after the coordinate system conversion A pixel point is selected, the pixel point closest to the pixel point is selected from the set of pixel points to be matched, and a matching point pair is formed by using the pixel point and the closest pixel point.
- the matching module 630 when using the pixel point and the nearest pixel point to form a matching point pair, is configured to: determine that the distance between the pixel point and the nearest pixel point is not greater than match threshold.
- the matching module 630 selects, based on the conversion relationship information, pixel points that match the pixel points in the standard pixel point set from the pixel point set to be matched to obtain a plurality of matching point pairs is used to: based on the conversion relationship information, select the pixel points matching each pixel point in the standard pixel point set from the to-be-matched pixel point set, respectively, to obtain a plurality of matching point pairs; In the case of the iteration stop condition, the conversion relationship information is updated based on the plurality of matching point pairs, and the conversion relationship information is returned based on the selection from the set of pixels to be matched and the set of standard pixels. Steps for matching pixels to pixels.
- the matching module 630 is further configured to: when it is determined that the iteration stop condition is not reached: determine two of the matching point pairs based on the distance between the two pixel points in each matching point pair The mean value of the distances between the pixel points; in the case that the mean value is greater than or equal to the first preset threshold, it is determined that the iteration stop condition is not reached.
- the matching module 630 is further configured to: when it is determined that the iteration stop condition is not reached: determine the difference information between the conversion relationship information corresponding to the current iteration and the conversion relationship information corresponding to the previous iteration ; in the case that the difference value corresponding to the difference value information is greater than or equal to the second preset threshold, it is determined that the iteration stop condition is not reached.
- the matching module 630 is further configured to: when it is determined that the iteration stop condition is not met: determine the selection of the pixel points from the set of to-be-matched pixel points that match the pixel points in the standard pixel point set The execution times of the pixel point; when the execution times are less than or equal to a third preset threshold, it is determined that the iteration stop condition has not been reached.
- the matching module 630 when updating the conversion relationship information based on the plurality of matching point pairs, is configured to: construct a residual based on the plurality of matching point pairs using the conversion relationship information as a variable Difference function; perform a derivation operation on the residual function, and determine the updated conversion relationship information with the goal that the value of the residual function tends to zero.
- the matching module 630 is further configured to: according to the distance between the two pixel points in each matching point pair in descending order, The plurality of matching point pairs are sorted; and the matching threshold is updated based on the distance between two pixel points in the matching point pairs having a preset sorting order.
- an embodiment of the present application further provides an electronic device 700.
- a schematic structural diagram of the electronic device 700 provided by the embodiment of the present application includes a processor 71, a memory 72, and a bus 73.
- the memory 72 is used to store the execution instructions, including the memory 721 and the external memory 722; the memory 721 here is also called the internal memory, and is used to temporarily store the operation data in the processor 71 and the data exchanged with the external memory 722 such as the hard disk.
- the processor 71 performs data exchange with the external memory 722 through the memory 721, and when the electronic device 700 is running, the processor 71 and the memory 72 communicate through the bus 73, so that the processor 71 executes the following instructions: Obtain the contour corresponding to the target object in the image to be matched.
- the conversion relationship information between the to-be-matched pixel set and the standard pixel set determines the conversion relationship information between the to-be-matched pixel set and the standard pixel set; based on the conversion relationship information, select the pixel points that match the pixel points in the standard pixel point set from the pixel point set to be matched, and obtain multiple matching point pairs; based on the multiple matching point pairs and the conversion relationship information, Determine the matching relationship information between the pixel points in the standard pixel point set and the pixel points in the to-be-matched pixel point set.
- the embodiments of the present application further provide a vehicle, including the electronic device further provided by the embodiments of the present application.
- the vehicles provided in the embodiments of the present application may include intelligent vehicles, including fully automatic control vehicles, and manually controlled vehicles with partial intelligent functions, and the partial intelligent functions may perform any image matching method provided in the embodiments of the present application.
- Embodiments of the present application further provide a computer-readable storage medium on which a computer program is stored, and when the computer program is run by a processor, the steps of the image matching method described in the above method embodiments are executed.
- the storage medium may be a volatile or non-volatile computer-readable storage medium.
- the computer program product of the image matching method provided by the embodiments of the present application includes a computer-readable storage medium storing program codes, and the instructions included in the program codes can be used to execute the image matching methods described in the above method embodiments. For details, refer to the above method embodiments, which will not be repeated here.
- the embodiments of the present application further provide a computer program, which implements any one of the methods in the foregoing embodiments when the computer program is executed by a processor.
- the computer program product can be specifically implemented by hardware, software or a combination thereof.
- the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.
- each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
- the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution.
- the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.
- the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
一种图像匹配的方法及装置、电子设备及车辆。获取待匹配图像中目标对象对应的待匹配像素点集、和标准图像中所述目标对象对应的标准像素点集（110），确定所述待匹配像素点集和所述标准像素点集之间的转换关系信息（120），并基于所述转换关系信息从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点，以得到多个匹配点对（130）。然后，可基于所述多个匹配点对，确定所述标准像素点集中的像素点与所述待匹配像素点集中的像素点之间的匹配关系信息（140）。
Description
相关申请的交叉引用
本专利申请要求于2020年12月29日提交的、申请号为202011597941.X、发明名称为“图像匹配的方法及装置、电子设备及车辆”的中国专利申请的优先权，该申请以引用的方式并入文本中。
本申请涉及图像处理技术领域，具体而言，涉及一种图像匹配的方法及装置、电子设备及车辆。
基于视觉的定位技术以其高精度的特性，应用越来越广泛，例如基于视觉的定位技术应用于智能交通等领域中。在基于视觉的定位技术中，首先需要精确确定两张图像中物体上像素点之间的匹配关系，继而利用像素点之间的匹配关系实现定位。目前，上述匹配过程中存在匹配精度低、效率低的缺陷。
发明内容
本申请实施例至少提供一种图像匹配的方法及装置、电子设备及车辆，以提高图像中像素点匹配的准确度和效率。
第一方面，本申请实施例提供了一种图像匹配的方法，包括：获取待匹配图像中所述目标对象对应的待匹配像素点集，和标准图像中目标对象对应的标准像素点集；确定所述待匹配像素点集和所述标准像素点集之间的转换关系信息；基于所述转换关系信息，从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点，得到多个匹配点对；基于所述多个匹配点对，确定所述标准像素点集中的像素点与所述待匹配像素点集中的像素点的匹配关系信息。该方面在对不同图像中的目标对象进行匹配的时候，利用的是目标对象的轮廓的像素点，能够有效降低匹配所需处理的像素点的数量，提高匹配的效率。
在一种可能的实施方式中，所述转换关系信息包括至少一个可求导的转换关系项。
该实施方式，可求导的转换关系信息，能够在当前得到匹配点对的基础上，实现了对转换关系信息的优化，从而有利于提高不同图像中的目标对象的像素点匹配的精准度。
在一种可能的实施方式中，所述待匹配图像包括场景图像，所述标准图像包括预制地图；所述方法还包括：基于所述匹配关系信息，确定所述目标对象的地理位置信息；基于所述目标对象的地理位置信息，确定拍摄所述场景图像的设备的地理位置信息。
该实施方式，利用较为精准的匹配关系信息，能够提高目标对象的定位精度，从而 能够提高拍摄场景图像的设备的定位精度。
在一种可能的实施方式中，所述获取待匹配图像中目标对象对应的待匹配像素点集，和标准图像中所述目标对象对应的标准像素点集，包括：获取待匹配图像中所述目标对象的轮廓对应的待匹配像素点集，和标准图像中目标对象的轮廓对应的标准像素点集。
在一种可能的实施方式中，所述获取待匹配图像中目标对象的轮廓对应的待匹配像素点集，和标准图像中所述目标对象的轮廓对应的标准像素点集，包括：获取包括目标对象的待匹配图像和标准图像；从所述待匹配图像对应的语义分割图像中提取所述目标对象的轮廓对应的待匹配像素点集；从所述标准图像对应的二维投影图像中提取所述目标对象的轮廓对应的标准像素点集。
该实施方式，从待匹配图像的语义分割图中提取目标对象的轮廓的像素点，以及，从标准图像的二维投影图像中提取目标对象的轮廓的像素点，能够提高像素点提取的效率和准确度。
在一种可能的实施方式中，所述转换关系信息包括关于旋转信息的第一转换关系项、关于位移信息的第二转换关系项和关于缩放信息的第三转换关系项中的至少一种；所述确定所述待匹配像素点集和所述标准像素点集之间的转换关系信息，包括以下至少一项：基于所述待匹配像素点集的朝向信息和所述标准像素点集的朝向信息，确定所述旋转信息，并基于所述旋转信息确定所述第一转换关系项；基于所述待匹配像素点集的重心和所述标准像素点集的重心，确定所述位移信息，并基于所述位移信息确定所述第二转换关系项；基于所述待匹配像素点集对应的图像面积和所述标准像素点集对应的图像面积，确定所述缩放信息，并基于所述缩放信息确定所述第三转换关系项。
该实施方式，转换关系信息包括能够求导的三个转换关系项，相比于现有技术中不能求导的变换关系，能够提高像素点匹配的成功率；另外，该实施方式利用形态学的方法，较为准确地确定了三个转换关系项中的取值，从而能够提高像素点匹配的精准度。
在一种可能的实施方式中，所述从所述标准图像对应的二维投影图像中提取所述目标对象的轮廓对应的标准像素点集，包括：从所述标准图像中提取所述目标对象的轮廓的角点信息；对提取的所述角点信息对应的角点进行上采样处理，得到三维轮廓点集；将所述三维轮廓点集中的各个像素点投影到所述二维投影图像对应的平面中，得到所述标准像素点集。
该实施方式，通过对标准图像中存储的角点进行上采样，能够得到目标对象的较为连贯和准确的轮廓信息，从而能够得到目标对象的较为准确的轮廓像素点。
在一种可能的实施方式中，所述从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点，得到多个匹配点对，包括：利用所述待匹配像素点集，构造K维搜索树；其中，K等于2；针对所述标准像素点集中的每个像素点，对所述K维搜索树进行遍历，以从所述待匹配像素点集中选取与该像素点相匹配的像素点，利用该 像素点和所述获取的像素点组成一个匹配点对。
该实施方式，将待匹配像素点集构造成K维的数据结构树，利用该树状结构，能够提高遍历待匹配像素点集中像素点的速度，进一步提高像素点匹配的效率。
在一种可能的实施方式中，所述基于所述转换关系信息，从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点，得到多个匹配点对，包括：利用所述转换关系信息，将所述标准像素点集中的各个像素点映射到所述待匹配像素点集对应的坐标系中；针对坐标系转换后的标准像素点集中的每个像素点，从所述待匹配像素点集中筛选与该像素点距离最近的像素点，并利用该像素点和所述最近的像素点组成一个匹配点对。
该实施方式，通过像素点映射和寻找最近点的匹配方式，能够提高像素点匹配的精准度。
在一种可能的实施方式中，利用该像素点和所述最近的像素点组成一个匹配点对包括：确定该像素点和所述最近的像素点之间的距离不大于匹配阈值。
该实施方式，通过确定该像素点和所述最近的像素点之间的距离不大于匹配阈值，能够有效降低待匹配像素点集中的噪声，提高像素点匹配的精准度，同时能够提高转换关系信息更新的准确度。
在一种可能的实施方式中，在得到所述多个匹配点对之后，还包括：按照所述各匹配点对中两个像素点之间的距离从大到小的顺序，将所述多个匹配点对进行排序；基于具有预设排序次序的匹配点对中两个像素点之间的距离，更新所述匹配阈值。
该实施方式，利用具有预设排序次序的匹配点对中两个像素点之间的距离更新匹配阈值，利用该更新后的匹配阈值，能够更加有效的降低待匹配像素点集中的噪声的影响，提高像素点匹配的精准度，同时能够提高利用匹配点对更新转换关系信息的准确度。
在一种可能的实施方式中，所述基于所述转换关系信息，从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点，得到多个匹配点对，包括：基于所述转换关系信息，从所述待匹配像素点集中分别选取与所述标准像素点集中的每个像素点相匹配的像素点，得到多个匹配点对；在未达到迭代停止条件的情况下，基于所述多个匹配点对更新所述转换关系信息，并返回所述基于所述转换关系信息，从所述待匹配像素点集中分别选取与所述标准像素点集中的每个像素点相匹配的像素点的步骤。
该实施方式，利用匹配得到的匹配点对来更新转换关系信息，能够得到更加准确的转换关系信息，及能够得到更加准确的仿射变换关系，继而利用该更加准确的转换关系信息，能够提高后续确定的匹配点对的精准度，即能够提高后续像素点匹配的精准度。
在一种可能的实施方式中，所述迭代停止条件包括：每个所述匹配点对中两个像素点之间的距离均值小于第一预设阈值。
该实施方式，在匹配点对中两个像素点之间的距离的均值较大时，更新转换关系信息后继续迭代，以确定匹配精度更高的匹配点对；在匹配点对中两个像素点之间的距离的均值比较小时，停止迭代，此时得到的匹配点对已经较为精确，即使继续迭代得到新的匹配点对，精度也不会显著提高，并且会浪费计算资源。
在一种可能的实施方式中，所述迭代停止条件包括：当轮迭代对应的转换关系信息与上一轮迭代对应的转换关系信息之间的差值信息对应的差值小于第二预设阈值。
该实施方式，在转换关系信息变化较大时，表示利用当前转换关系信息确定的匹配点对的精度较低，需要更新转换关系信息后继续迭代，以提高像素点匹配的精度；在转换关系信息变化较小时，停止迭代，此时得到的匹配点对已经较为精确，即使继续利用转换关系信息迭代得到新的匹配点对，精度也不会显著提高，并且会浪费计算资源。
在一种可能的实施方式中，所述迭代停止条件包括：所述从所述待匹配像素点集中分别选取与所述标准像素点集中的每个像素点相匹配的像素点的执行次数大于第三预设阈值。
该实施方式，在迭代得到匹配点对的次数较少时，确定的匹配点对的精度较低，需要更新转换关系信息后继续迭代，以提高像素点匹配的精度；迭代得到匹配点对的次数较多时，停止迭代，此时得到的匹配点对已经较为精确，即使继续迭代得到新的匹配点对，精度也不会显著提高，并且会浪费计算资源。
在一种可能的实施方式中，所述基于所述多个匹配点对更新所述转换关系信息，包括：基于所述多个匹配点对，以所述转换关系信息为变量构造残差函数；对所述残差函数进行求导操作，以所述残差函数的值趋于零为目标，确定更新后的转换关系信息。
该实施方式，以所述残差函数的值趋于零为目标更新转换关系信息，能够提高利用转换关系确定的匹配点对的精度。
第二方面，本申请提供了一种图像匹配的装置，包括：像素点提取模块，用于获取待匹配图像中目标对象对应的待匹配像素点集，和标准图像中所述目标对象对应的标准像素点集；变换处理模块，用于确定所述待匹配像素点集和所述标准像素点集之间的转换关系信息；匹配模块，用于基于所述转换关系信息，从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点，得到多个匹配点对；关系形成模块，用于基于所述多个匹配点对，确定所述标准像素点集中的像素点与所述待匹配像素点集中的像素点之间的匹配关系信息。
第三方面，本申请实施例提供了一种电子设备，包括：处理器、存储器和总线，所述存储器存储有所述处理器可执行的机器可读指令，当电子设备运行时，所述处理器与所述存储器之间通过总线通信，所述机器可读指令被所述处理器执行时执行如上述图像匹配的方法的步骤。
第四方面，本申请实施例提供了一种计算机可读存储介质，其上存储有计算机程序， 该计算机程序被处理器运行时执行如上述图像匹配的方法的步骤。
第五方面，本申请实施例提供了一种车辆，包括上述第三方面提供的电子设备。
为使本申请的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。
为了更清楚地说明本申请实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍。这些附图示出了符合本申请的实施例，并与说明书一起用于说明本申请的技术方案。应当理解，以下附图仅示出了本申请的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1示出了本申请实施例所提供的图像匹配的方法的流程图；
图2A、图2B、图2C示出了本申请实施例中标准像素点集的示意图；
图3示出了本申请实施例中标准像素点集对应的轮廓和待匹配像素点集对应的轮廓的示意图；
图4示出了本申请实施例所提供的图像匹配的方法中确定转换关系信息的初值的流程图；
图5示出了本申请实施例所提供的K-D树的示意图；
图6示出了本申请实施例所提供的图像匹配的装置的示意图；
图7示出了本申请实施例所提供的一种电子设备的示意图。
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。通常在此处附图中描述和示出的本申请实施例的组件可以以各种不同的配置来布置和设计。因此，以下对在附图中提供的本申请的实施例的详细描述并非旨在限制要求保护的本申请的范围，而是仅仅表示本申请的选定实施例。基于本申请的实施例，本领域技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本申请保护的范围。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释。
本文中术语“和/或”，仅仅是描述一种关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外， 本文中术语“至少一种”表示多种中的任意一种或多种中的至少两种的任意组合，例如，包括A、B、C中的至少一种，可以表示包括从A、B和C构成的集合中选择的任意一个或多个元素。
在视觉定位技术中，需要匹配两张图像中的像素点。目前在像素点匹配的过程中使用的变换关系无法求导以及初始的变换关系不够准确，导致像素点对匹配成功率低，精准度差；同时，定位技术中计算量大，匹配效率无法保证。针对上述技术缺陷，本申请提供了一种图像匹配的方法及装置，本申请在对不同图像中的目标对象进行匹配的时候，利用的是目标对象的轮廓的像素点，能够有效降低匹配所需处理的像素点的数量，提高匹配的效率。另外，本申请在进行像素点匹配的时候，利用的转换关系信息是能够求导的仿射关系，有利于提高不同图像中的目标对象的像素点匹配的精准度。
下面对本申请中图像匹配的方法及装置、电子设备、存储介质进行说明。
本申请实施例提供的图像匹配的方法由具有运算能力的设备完成，例如移动终端设备等。具体地，如图1所示，本申请实施例提供的图像匹配的方法可以包括如下步骤：
S110、获取待匹配图像中目标对象对应的待匹配像素点集，和标准图像中所述目标对象对应的标准像素点集。
这里，在执行此步骤之前，首先需要在待匹配图像和标准图像之间进行初步的对象匹配，得到同时存在于待匹配图像和标准图像中的物体，即上述目标对象。在得到目标对象之后，还是无法利用该目标对象实现定位，还需要进一步确定两张图像中，目标对象上的像素点的匹配关系信息，即下述多个匹配点对以及对应的仿射变换关系，即下述转换关系信息。利用两张图像中目标对象的像素点的匹配关系信息和仿射变换关系才能实现视觉定位。
上述待匹配像素点集中的像素点对应于待匹配图像中的目标对象，数量较多；标准像素点集的像素点对应于标准图像中的目标对象，数量较少。本申请可以为标准像素点集中的每个像素点，从待匹配像素点集中寻找相匹配的像素点。
在实际应用中，对密集点云，比如图像中的某一对象，其有效的信息通常位于轮廓处，例如道路上的路标，因此可以仅对对象的轮廓处的像素点进行匹配即可，不用将对象上的全部像素点进行匹配。
则本步骤可以执行为：获取待匹配图像中目标对象的轮廓对应的待匹配像素点集，和标准图像中所述目标对象的轮廓对应的标准像素点集。在这种实施方式中，提取的可以是目标对象的轮廓对应的像素点，这样能够有效降低匹配所需处理的像素点的数量，提高匹配的效率，而不会丢失目标对象的有效信息，对定位精度造成影响。
在实际应用中，上述待匹配图像可以包括场景图像，所述标准图像可以包括预制地图，这样基于本申请确定的匹配关系信息和转换关系信息即可实现视觉定位。
在具体实施时，需要首先获取包括目标对象的待匹配图像和标准图像，再对待匹配图像进行语义分割，得到其对应的语义分割图像，并从得到的语义分割图像中提取所述目标对象的轮廓对应的待匹配像素点集。
将标准图像投影到二维平面上，得到其对应的二维投影图像，再从二维投影图像中提取所述目标对象的轮廓对应的标准像素点集。实际上，标准图像上可以只存储有用的角点，因此投影到二维投影平面上的图像中也只包括上述角点对应的投影点，为了提高标准像素点集中像素点的数量，可以将投影点进行上采样，并将投影点和上采样得到的点进行连接，连线上的像素点都认为是目标对象的轮廓上的像素点。
在实际应用中也可先从所述标准图像中提取所述目标对象的轮廓的角点信息；对提取的角点信息对应的角点进行上采样处理，得到三维轮廓点集；再将三维轮廓点集中的各个像素点投影到所述二维投影图像对应的平面中，得到所述标准像素点集。如图2A、图2B、图2C所示，即为利用上述方法得到的标准像素点集21、标准像素点集22、标准像素点集23。
通过对标准图像中存储的角点进行上采样，能够得到目标对象的较为连贯和准确的轮廓信息，从而能够得到目标对象的较为准确的轮廓像素点。
上述从待匹配图像的语义分割图中提取目标对象的轮廓的像素点，以及从标准图像的二维投影图像中提取目标对象的轮廓的像素点，能够提高像素点提取的效率和准确度。
S120、确定所述待匹配像素点集和所述标准像素点集之间的转换关系信息。
这里，转换关系信息用于将标准像素点集映射到待匹配像素点集所在的平面上，位于同一平面上的待匹配像素点集和所述标准像素点集之间才能计算像素点之间的距离，以及确定匹配点对。转换关系信息可以为可求导的仿射变换关系，示例性地，转换关系信息包括至少一个可求导的转换关系项。具体地，本申请中的转换关系信息可以包括关于旋转信息的第一转换关系项、关于位移信息的第二转换关系项和关于缩放信息的第三转换关系项等。
示例性地，上述旋转信息、位移信息以及缩放信息等形成的转换关系信息或仿射变换关系可以如下述公式所示，其中旋转信息包括旋转角，缩放信息包括缩放比：
式中，A表示上述转换关系信息或仿射变换关系；s表示缩放比，该缩放比具体可以是待匹配图像中目标对象的图像面积与标准图像中目标对象的图像面积的比值；θ表示旋转角，该旋转角具体可以是待匹配图像中目标对象的方向与标准图像中目标对象的方向之间的夹角；a表示横向比，该横向比具体可以是待匹配图像与标准图像的宽度的比值，ψ表示倾斜角，该倾斜角具体可以是待匹配图像中目标对象相对于标准图像中的 目标对象，倾斜的角度；tx表示x轴位移量，该位移量具体可以是待匹配图像中目标对象的x轴坐标与标准图像中目标对象的x轴坐标之间的差值；ty表示y轴位移量，该位移量具体可以是待匹配图像中目标对象的y轴坐标与标准图像中目标对象的y轴坐标之间的差值。tx和ty组成上述位移信息。
通过上面的公式可知，上述转换关系信息还可以包括横向比a和倾斜角ψ。一般的，设置倾斜角为0，横向比为1。
上述图像面积、方向和坐标均是利用对应的点集在同一张二维图像上的位置坐标确定的。
根据上面的公式可知，转换关系信息将旋转角、缩放比、横向比、倾斜角、位移信息分别放在不同的矩阵中，这样每个矩阵中的变量一般情况下只有一个，特殊情况下最多有两个，其余由0和1填充，这样容易进行求导，并且对各个矩阵进行求导得到的结果具有对应的物理意义。利用具有物理意义的求导结果才能更好的优化转换关系信息，才能提高后续基于转换关系信息进行像素点映射和像素点匹配的精度。如图3所示，标准像素点集对应的轮廓31与待匹配像素点集对应的轮廓32之间不是利用普通的旋转-平移的欧式变换关系进行转换的，而是利用上述仿射变换关系进行转换的。
上述普通的旋转-平移的欧式变换关系如下述公式所示：
上述欧式变换关系由于矩阵中变量很多，造成求导困难，并且非常可能无法得到求导结果。即使得到求导结果，该求导结果也不具有明确的物理意义，用这样的求导结果无法很好的优化转换关系信息，从而会降低后续像素点匹配的精确度。
S130、基于所述转换关系信息，从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点，得到多个匹配点对。
这里，具体可以首先利用所述转换关系信息，将所述标准像素点集中的各个像素点映射到所述待匹配像素点集对应的坐标系中，具体如下公式所示可以是将标准像素点集中的像素点的坐标与上述转换关系信息对应的矩阵A相乘，得到该像素点在待匹配像素点集对应的坐标系中的坐标：
式中，[x y 1]
T表示标准像素点集中的一像素点的坐标，表示[u v 1]
T该像素点在待匹配像素点集对应的坐标系中的坐标。
之后，针对坐标系转换后的标准像素点集中的每个像素点，从所述待匹配像素点集 中筛选与该像素点距离最近的像素点，并利用该像素点和所述最近的像素点组成一个匹配点对。
通过像素点映射和寻找最近点的匹配方式，能够提高像素点匹配的精准度。
上面在寻找匹配点对的过程中，对于每个标准像素点集中的像素点，都要遍历一遍待匹配像素点集中的像素点，才能找到与其匹配的像素点，效率比较低。为了提高遍历效率，可以利用待匹配像素点集，构造具有K维搜索树，即K-D树；其中，K等于2，表示划分的空间的维度。那么在寻找匹配点对的过程中，对于每个标准像素点集中的像素点，可以对上述二叉树进行遍历，以从所述待匹配像素点集中选取与该像素点相匹配的像素点，利用该像素点和所述获取的像素点组成一个匹配点对。
示例性的，如果待匹配像素点中各个像素点的坐标为(2，3)、(5，4)、(4，7)、(8，1)、(7，2)和(9，6)，那么建立的K-D数结构如图5所示。待匹配像素点集中的各个像素点得到坐标构成图5中的各个叶子节点。根结点和中间结点存放一些空间划分信息，例如划分维度、划分值。
在从待匹配像素点集中选取与标准像素点集中某一像素点相匹配的像素点时，首先将该像素点的坐标Q与K-D树中的各个节点存储的数值进行比较，即将Q对应于节点中的K维度上的值与节点中存储的K维度上的中值m进行比较，若Q对应于节点中的K维度上的值小于m，则访问对应节点的左子树，否则访问对应节点的右子树。达到叶子结点时，计算Q与叶子结点上保存的坐标之间的距离，并将距离最小的坐标对应的像素点作为与该像素点匹配的像素点。
可见，利用K-D树进行像素点匹配时，并不遍历所有的节点，不与待匹配像素点集中所有的像素点分别计算距离，因此可以减少遍历的计算量，提高匹配效率。根据K-D树空间二叉树的特性，可以将遍历查找的复杂度从N降低到log N。这里，N表示待匹配像素点集中的像素点数量，可见，利用上述树状结构，能够提高遍历待匹配像素点集中像素点的速度，进一步提高像素点匹配的效率。
从语义分割图像中提取的目标对象的轮廓的待匹配像素点集中，不可避免的会存在噪声，为了降低待匹配像素点集中的噪声对像素点匹配的影响，可以在进行像素点匹配时，确定匹配点对中两个像素点之间的距离不大于匹配阈值。例如，可以将匹配点对中两个像素点之间的距离大于匹配阈值的匹配点对剔除，以提高像素点匹配的精准度。
上述匹配阈值可以根据实际的应用场景和应用需求设置，在像素点匹配的过程中也可以进行更新，以进一步提高像素点匹配的准确度。
S140、基于所述多个匹配点对，确定所述标准像素点集中的像素点与所述待匹配像素点集中的像素点之间的匹配关系信息。
这里，具体地可以将多个匹配点对和转换关系信息，作为所述标准像素点集中的像素点与所述待匹配像素点集中的像素点之间的匹配关系信息。
上述实施例利用目标对象的轮廓的像素点进行匹配，能够有效降低匹配所需处理的像素点的数量，提高匹配的效率。另外，上述实施例在进行像素点匹配的时候，利用的转换关系信息是能够求导的仿射关系，即能够在当前得到匹配点对的基础上，实现了对转换关系信息的优化，从而有利于提高不同图像中的目标对象的像素点匹配的精准度。
在进行视觉定位的场景中，上述待匹配图像可以包括场景图像，所述标准图像可以包括预制地图。那么可以利用如下步骤实现定位：基于所述匹配关系信息，确定所述目标对象的地理位置信息；基于所述目标对象的地理位置信息，确定拍摄所述场景图像的设备的地理位置信息。
在基于目标对象的地理位置信息，确定拍摄所述场景图像的设备的地理位置信息时，还要结合拍摄所述场景图像的设备的拍摄参数信息。
上述利用较为精准的匹配关系信息，能够提高目标对象的定位精度，从而能够提高拍摄场景图像的设备的定位精度。
上述确定的地理位置信息可以应用于自动驾驶领域、智能驾驶、机器人等领域中。
只执行一次步骤S130，确定的匹配点对粗糙，精准度差，为了提高像素点匹配的精准度，需要多次执行上述步骤S130。具体地，步骤S130可以利用如下步骤实现：
步骤一、基于所述转换关系信息，从所述待匹配像素点集中分别选取与所述标准像素点集中的每个像素点相匹配的像素点，得到多个匹配点对。
这里确定匹配点对的方法与上述实施例中的方法相同，不再赘述。
步骤二、在未达到迭代停止条件的情况下，基于所述多个匹配点对更新所述转换关系信息，并返回所述基于所述转换关系信息，从所述待匹配像素点集中分别选取与所述标准像素点集中的每个像素点相匹配的像素点的步骤，也即，返回步骤一。
此步骤在迭代执行步骤一之前，需要更新转换关系信息，之后利用新的转换关系信息进行像素点匹配，生成新的匹配点对。
示例性地，可以利用如下步骤更新转换关系信息：
基于所述多个匹配点对，以所述转换关系信息为变量构造残差函数；对所述残差函数进行求导操作，以所述残差函数的值趋于零为目标，确定更新后的转换关系信息。
上述残差函数具体可以如下所示：
式中，L表示残差函数的值，i表示第i个匹配点对，m表示匹配点对的数量，q
i表示第i个匹配点对中标准像素点集中的像素点的坐标，p
i表示第i个匹配点对中待匹配素点集中的像素点的坐标。
在具体实施时，通过对所述残差函数进行求导操作，能够以所述残差函数的值趋于零为目标，确定更新后的转换关系信息，从而能够得到最优的转换关系信息，利用该最优的转换关系能够提高像素点匹配的精度。
如图4所示，对转换关系信息进行优化之前，转换关系信息的初值可以利用如下步骤计算：
S410、基于所述待匹配像素点集的朝向信息和所述标准像素点集的朝向信息，确定所述旋转信息。
这里可以首先利用主成分分析(Principal Component Analysis，PCA)方法，对待匹配像素点集和标准像素点集分别进行处理，得到待匹配像素点集的朝向信息和标准像素点集的朝向信息。上述待匹配像素点集的朝向信息即为待匹配图像中目标对象的方向，标准像素点集的朝向信息即为标准图像中目标对象的方向。之后，计算待匹配图像中目标对象的方向与标准图像中目标对象的方向之间的夹角，并将得到夹角θ作为上述旋转信息。
S420、基于所述待匹配像素点集的重心和所述标准像素点集的重心，确定所述位移信息。
这里可以首先确定所述待匹配像素点集的重心和所述标准像素点集的重心，再基于两个在重心之间的位移量，确定上述位移信息。具体地可以将两个在重心之间的位移量作为上述位移信息。
示例性地，待匹配像素点集的重心可以对待匹配像素点集中各个像素点的坐标进行加权求和得到，同样地，标准像素点集的重心可以对标准像素点集中各个像素点的坐标进行加权求和得到。
基于两个重心的坐标可以计算两个重心之间的位移量tx和ty。
S430、基于所述待匹配像素点集对应的图像面积和所述标准像素点集对应的图像面积，确定所述缩放信息。
这里可以首先确定待匹配像素点集对应的图像面积和所述标准像素点集对应的图像面积，再基于两个图像面积的比值s确定上述缩放信息。具体可以将上述比值s作为上述缩放信息。
上述待匹配像素点集对应的图像面积具体可以是待匹配图像中目标对象的图像面积，可以利用待匹配像素点集中各个像素点的坐标来确定。
同样地，标准像素点集对应的图像面积具体可以是标准图像中目标对象的图像面积，可以利用标准点集中各个像素点的坐标来确定。
转换关系信息中的倾斜角ψ的初值可以设置为0度，转换关系信息中的横向比a的初值可以设置为1。
准确的转换关系信息初值能够提高迭代速度，即提高像素点匹配的速度，以及提高像素点匹配的成功率，避免在每次迭代中陷入局部最优解而导致匹配失败。
上述实施例，转换关系信息包括旋转信息、位移信息和缩放信息等，构成了能够求导的仿射变换关系，相比于现有技术中不能求导的仿射变换关系，能够更好的对转换关系信息进行优化，利用优质的转换关系信息能够提高像素点匹配的成功率；另外，该实施方式利用形态学的方法，较为准确地确定了仿射变换关系中的取值，从而能够提高像素点匹配的精准度。
上述转换关系信息的初值是在第一次迭代，即第一次执行上述步骤一中使用的转换关系信息，并在后续执行上述步骤一之前，都需要更新转换关系信息。
在每一次迭代之后，得到多个匹配点对，此时为了提高匹配的精准度，以及更好地基于匹配点对对转换关系信息进行优化，在进行像素点匹配时，需要利用匹配阈值确定两个像素点之间的距离不大于该匹配阈值。例如，可以从多个匹配点对中剔除两个像素点之间距离较远的匹配点对。为了能够更加有效的降低待匹配像素点集中的噪声的影响，提高像素点匹配的精准度，可以利用每次迭代后得到的匹配点对更新匹配阈值，具体可以利用如下步骤实现：
按照当前匹配点对中两个像素点之间的距离从大到小的顺序，将所述多个匹配点对进行排序；基于具有预设排序次序的匹配点对中两个像素点之间的距离，更新所述匹配阈值。
这里，可以将具有预设排序次序的匹配点对中两个像素点之间的距离，作为更新后的匹配阈值。
在实际应用中，迭代需要一个终止条件，不能一直迭代下去，下面给出了几种终止方案：
方案一，基于每个所述匹配点对中两个像素点之间的距离，确定匹配点对中两个像素点之间的距离的均值；在所述均值大于或等于第一预设阈值的情况下，确定未达到所述迭代停止条件；在均值小于第一预设阈值的情况下，确定达到所述迭代停止条件，停止迭代，将当前得到的多个匹配点对作为最终的匹配点对，当前的转换关系信息作为最终的转换关系信息。
该方式，在匹配点对中两个像素点之间的距离的均值较大时，更新转换关系信息后继续迭代，以确定匹配精度更高的匹配点对；在匹配点对中两个像素点之间的距离的均值比较小时，停止迭代，此时得到的匹配点对已经较为精确，即使继续迭代得到新的匹配点对，精度也不会显著提高，并且会浪费计算资源。
方案二，确定当轮迭代对应的转换关系信息与上一轮迭代对应的转换关系信息之间的差值信息；在所述差值信息对应的差值大于或等于第二预设阈值的情况下，确定未达到所述迭代停止条件；在差值信息对应的差值小于第二预设阈值的情况下，确定达到 所述迭代停止条件，停止迭代，将当前得到的多个匹配点对作为最终的匹配点对，当前的转换关系信息作为最终的转换关系信息。
该方式，在转换关系信息变化较大时，表示利用当前转换关系信息确定的匹配点对的精度较低，需要更新转换关系信息后继续迭代，以提高像素点匹配的精度；在转换关系信息变化较小时，停止迭代，此时得到的匹配点对已经较为精确，即使继续利用转换关系信息迭代得到新的匹配点对，精度也不会显著提高，并且会浪费计算资源。
方案三，确定所述从所述待匹配像素点集中分别选取与所述标准像素点集中的每个像素点相匹配的像素点的执行次数；在所述执行次数小于或等于第三预设阈值的情况下，确定未达到所述迭代停止条件；在执行次数大于第三预设阈值的情况下，确定达到所述迭代停止条件，停止迭代，将当前得到的多个匹配点对作为最终的匹配点对，当前的转换关系信息作为最终的转换关系信息。
该方式，在迭代得到匹配点对的次数较少时，确定的匹配点对的精度较低，需要更新转换关系信息后继续迭代，以提高像素点匹配的精度；迭代得到匹配点对的次数较多时，停止迭代，此时得到的匹配点对已经较为精确，即使继续迭代得到新的匹配点对，精度也不会显著提高，并且会浪费计算资源。
对应于上述图像匹配的方法，本申请还提供了一种图像匹配的装置，应用于上述执行图像匹配的方法的终端上，并且能够取得相同或相似的有益效果。该装置能够执行上述实施例中的图像匹配的方法的所有步骤，因此相同的步骤中这里不再赘述。具体地，如图6所示，本申请提供的图像匹配的装置包括：
像素点提取模块610，用于获取待匹配图像中目标对象对应的待匹配像素点集，和标准图像中所述目标对象对应的标准像素点集。
变换处理模块620，用于确定所述待匹配像素点集和所述标准像素点集之间的转换关系信息。
匹配模块630，用于基于所述转换关系信息，从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点，得到多个匹配点对。
关系形成模块640，用于基于所述多个匹配点对，确定所述标准像素点集中的像素点与所述待匹配像素点集中的像素点的匹配关系信息。
在一些实施例中，所述转换关系信息包括至少一个可求导的转换关系项。
在一些实施例中，所述待匹配图像包括场景图像，所述标准图像包括预制地图。相应地，所述图像匹配装置还包括定位模块650，用于：基于所述匹配关系信息，确定所述目标对象的地理位置信息；基于所述目标对象的地理位置信息，确定拍摄所述场景图像的设备的地理位置信息。
在一些实施例中，像素点提取模块610，用于获取待匹配图像中目标对象的轮廓对 应的待匹配像素点集，和标准图像中所述目标对象的轮廓对应的标准像素点集。
在一些实施例中，所述像素点提取模块610在获取待匹配图像中目标对象的轮廓对应的待匹配像素点集，和标准图像中所述目标对象的轮廓对应的标准像素点集时，用于：获取包括目标对象的待匹配图像和标准图像；从所述待匹配图像对应的语义分割图像中提取所述目标对象的轮廓对应的待匹配像素点集；从所述标准图像对应的二维投影图像中提取所述目标对象的轮廓对应的标准像素点集。
在一些实施例中，所述转换关系信息包括关于旋转信息的第一转换关系项、关于位移信息的第二转换关系项和关于缩放信息的第三转换关系项中的至少一种。相应地，所述变换处理模块620在确定所述待匹配像素点集和所述标准像素点集之间的转换关系信息时，用于：基于所述待匹配像素点集的朝向信息和所述标准像素点集的朝向信息，确定所述旋转信息，并基于所述旋转信息确定所述第一转换关系项；和/或，基于所述待匹配像素点集的重心和所述标准像素点集的重心，确定所述位移信息，并基于所述位移信息确定所述第二转换关系项；和/或，基于所述待匹配像素点集对应的图像面积和所述标准像素点集对应的图像面积，确定所述缩放信息，并基于所述缩放信息确定所述第三转换关系项。
在一些实施例中，所述像素点提取模块610在从所述标准图像对应的二维投影图像中提取所述目标对象的轮廓对应的标准像素点集时，用于：从所述标准图像中提取所述目标对象的轮廓的角点信息；对提取的所述角点信息对应的角点进行上采样处理，得到三维轮廓点集；将所述三维轮廓点集中的各个像素点投影到所述二维投影图像对应的平面中，得到所述标准像素点集。
在一些实施例中，所述匹配模块630在从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点，得到多个匹配点对时，用于：利用所述待匹配像素点集，构造K维搜索树；其中，K等于2；针对所述标准像素点集中的每个像素点，对所述K维搜索树进行遍历，以从所述待匹配像素点集中选取与该像素点相匹配的像素点，利用该像素点和所述获取的像素点组成一个匹配点对。
在一些实施例中，所述匹配模块630在基于所述转换关系信息，从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点，得到多个匹配点对时，用于：利用所述转换关系信息，将所述标准像素点集中的各个像素点映射到所述待匹配像素点集对应的坐标系中；针对坐标系转换后的标准像素点集中的每个像素点，从所述待匹配像素点集中筛选与该像素点距离最近的像素点，并利用该像素点和所述最近的像素点组成一个匹配点对。
在一些实施例中，所述匹配模块630在利用该像素点和所述最近的像素点组成一个匹配点对时，用于：确定该像素点和所述最近的像素点之间的距离不大于匹配阈值。
在一些实施例中，所述匹配模块630在基于所述转换关系信息，从所述待匹配像 素点集中选取与所述标准像素点集中的像素点相匹配的像素点，得到多个匹配点对时，用于：基于所述转换关系信息，从所述待匹配像素点集中分别选取与所述标准像素点集中的每个像素点相匹配的像素点，得到多个匹配点对；在未达到迭代停止条件的情况下，基于所述多个匹配点对更新所述转换关系信息，并返回所述基于所述转换关系信息，从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点的步骤。
在一些实施例中，所述匹配模块630还用于在确定未达到所述迭代停止条件时：基于每个所述匹配点对中两个像素点之间的距离，确定匹配点对中两个像素点之间的距离的均值；在所述均值大于或等于第一预设阈值的情况下，确定未达到所述迭代停止条件。
在一些实施例中，所述匹配模块630还用于在确定未达到所述迭代停止条件时：确定当轮迭代对应的转换关系信息与上一轮迭代对应的转换关系信息之间的差值信息；在所述差值信息对应的差值大于或等于第二预设阈值的情况下，确定未达到所述迭代停止条件。
在一些实施例中，所述匹配模块630还用于在确定未达到所述迭代停止条件时：确定所述从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点的执行次数；在所述执行次数小于或等于第三预设阈值的情况下，确定未达到所述迭代停止条件。
在一些实施例中，所述匹配模块630在基于所述多个匹配点对更新所述转换关系信息时，用于：基于所述多个匹配点对，以所述转换关系信息为变量构造残差函数；对所述残差函数进行求导操作，以所述残差函数的值趋于零为目标，确定更新后的转换关系信息。
在一些实施例中，在得到所述多个匹配点对之后，所述匹配模块630还用于：按照各所述匹配点对中两个像素点之间的距离从大到小的顺序，将所述多个匹配点对进行排序；基于具有预设排序次序的匹配点对中两个像素点之间的距离，更新所述匹配阈值。
对应于上述图像匹配的方法，本申请实施例还提供了一种电子设备700，如图7所示，为本申请实施例提供的电子设备700结构示意图，包括处理器71、存储器72、和总线73。存储器72用于存储执行指令，包括内存721和外部存储器722；这里的内存721也称内存储器，用于暂时存放处理器71中的运算数据，以及与硬盘等外部存储器722交换的数据，处理器71通过内存721与外部存储器722进行数据交换，当电子设备700运行时，处理器71与存储器72之间通过总线73通信，使得处理器71执行以下指令：获取待匹配图像中目标对象的轮廓对应的待匹配像素点集，和标准图像中所述目标对象的轮廓对应的标准像素点集；确定所述待匹配像素点集和所述标准像素点集之间的转换关系信息；基于所述转换关系信息，从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点，得到多个匹配点对；基于所述多个匹配点对和所 述转换关系信息，确定所述标准像素点集中的像素点与所述待匹配像素点集中的像素点的匹配关系信息。
本申请实施例还提供一种车辆，包括本申请实施例还提供的电子设备。
本申请实施例提供的车辆可以包括智能车辆，包括全自动控制车辆，以及具有部分智能功能的手动控制车辆，该部分智能功能可以执行本申请实施例提供的任一图像匹配的方法。
本申请实施例还提供一种计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器运行时执行上述方法实施例中所述的图像匹配的方法的步骤。其中，该存储介质可以是易失性或非易失的计算机可读取存储介质。
本申请实施例所提供的图像匹配的方法的计算机程序产品，包括存储了程序代码的计算机可读存储介质，所述程序代码包括的指令可用于执行上述方法实施例中所述的图像匹配的方法的步骤，具体可参见上述方法实施例，在此不再赘述。
本申请实施例还提供一种计算机程序，该计算机程序被处理器执行时实现前述实施例的任意一种方法。该计算机程序产品可以具体通过硬件、软件或其结合的方式实现。在一个可选实施例中，所述计算机程序产品具体体现为计算机存储介质，在另一个可选实施例中，计算机程序产品具体体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统和装置的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，又例如，多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个处理器可执行的非易失的计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用 以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上所述实施例，仅为本申请的具体实施方式，用以说明本申请的技术方案，而非对其限制，本申请的保护范围并不局限于此，尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化，或者对其中部分技术特征进行等同替换；而这些修改、变化或者替换，并不使相应技术方案的本质脱离本申请实施例技术方案的精神和范围，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应所述以权利要求的保护范围为准。
Claims (18)
- 一种图像匹配的方法，包括：获取待匹配图像中目标对象对应的待匹配像素点集，和标准图像中所述目标对象对应的标准像素点集；确定所述待匹配像素点集和所述标准像素点集之间的转换关系信息；基于所述转换关系信息，从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点，得到多个匹配点对；基于所述多个匹配点对，确定所述标准像素点集中的像素点与所述待匹配像素点集中的像素点之间的匹配关系信息。
- 根据权利要求1所述方法，其特征在于，所述转换关系信息包括至少一个可求导的转换关系项。
- 根据权利要求1或2所述方法，其特征在于，所述待匹配图像包括场景图像，所述标准图像包括预制地图，所述方法还包括：基于所述匹配关系信息，确定所述目标对象的地理位置信息；基于所述目标对象的地理位置信息，确定拍摄所述场景图像的设备的地理位置信息。
- 根据权利要求1或2所述方法，其特征在于，所述获取待匹配图像中目标对象对应的待匹配像素点集，和标准图像中所述目标对象对应的标准像素点集，包括：获取待匹配图像中目标对象的轮廓对应的待匹配像素点集，和标准图像中所述目标对象的轮廓对应的标准像素点集。
- 根据权利要求4所述方法，其特征在于，所述获取待匹配图像中目标对象的轮廓对应的待匹配像素点集，和标准图像中所述目标对象的轮廓对应的标准像素点集，包括：获取包括目标对象的待匹配图像和标准图像；从所述待匹配图像对应的语义分割图像中提取所述目标对象的轮廓对应的待匹配像素点集；从所述标准图像对应的二维投影图像中提取所述目标对象的轮廓对应的标准像素点集。
- 根据权利要求1至5任一项所述方法，其特征在于，所述转换关系信息包括关于旋转信息的第一转换关系项、关于位移信息的第二转换关系项和关于缩放信息的第三转换关系项中的至少一种；所述确定所述待匹配像素点集和所述标准像素点集之间的转换关系信息，包括以下至少一项：基于所述待匹配像素点集的朝向信息和所述标准像素点集的朝向信息，确定所述旋 转信息，并基于所述旋转信息确定所述第一转换关系项；基于所述待匹配像素点集的重心和所述标准像素点集的重心，确定所述位移信息，并基于所述位移信息确定所述第二转换关系项；基于所述待匹配像素点集对应的图像面积和所述标准像素点集对应的图像面积，确定所述缩放信息，并基于所述缩放信息确定所述第三转换关系项。
- 根据权利要求5所述方法，其特征在于，所述从所述标准图像对应的二维投影图像中提取所述目标对象的轮廓对应的标准像素点集，包括：从所述标准图像中提取所述目标对象的轮廓的角点信息；对提取的所述角点信息对应的角点进行上采样处理，得到三维轮廓点集；将所述三维轮廓点集中的各个像素点投影到所述二维投影图像对应的平面中，得到所述标准像素点集。
- 根据权利要求1至7任一项所述方法，其特征在于，所述从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点，得到多个匹配点对，包括：利用所述待匹配像素点集，构造K维搜索树；其中，K等于2；针对所述标准像素点集中的每个像素点，对所述K维搜索树进行遍历，以从所述待匹配像素点集中选取与该像素点相匹配的像素点，利用该像素点和所述获取的像素点组成一个匹配点对。
- 根据权利要求1至8任一项所述方法，其特征在于，所述基于所述转换关系信息，从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点，得到多个匹配点对，包括：利用所述转换关系信息，将所述标准像素点集中的各个像素点映射到所述待匹配像素点集对应的坐标系中；针对坐标系转换后的标准像素点集中的每个像素点，从所述待匹配像素点集中筛选与该像素点距离最近的像素点，并利用该像素点和所述最近的像素点组成一个匹配点对。
- 根据权利要求9所述方法，其特征在于，利用该像素点和所述最近的像素点组成一个匹配点对，包括：确定该像素点和所述最近的像素点之间的距离不大于匹配阈值。
- 根据权利要求10所述方法，其特征在于，在得到所述多个匹配点对之后，还包括：按照各所述匹配点对中两个像素点之间的距离从大到小的顺序，将所述多个匹配点对进行排序；基于具有预设排序次序的匹配点对中两个像素点之间的距离，更新所述匹配阈值。
- 根据权利要求1至11任一项所述方法，其特征在于，所述基于所述转换关系信息，从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点，得到多个匹配点对，包括：基于所述转换关系信息，从所述待匹配像素点集中分别选取与所述标准像素点集中的每个像素点相匹配的像素点，得到多个匹配点对；在未达到迭代停止条件的情况下，基于所述多个匹配点对更新所述转换关系信息，并返回所述基于所述转换关系信息，从所述待匹配像素点集中分别选取与所述标准像素点集中的每个像素点相匹配的像素点的步骤。
- 根据权利要求12所述方法，其特征在于，所述迭代停止条件包括以下任一：每个所述匹配点对中两个像素点之间的距离的均值小于第一预设阈值；当轮迭代对应的转换关系信息与上一轮迭代对应的转换关系信息之间的差值信息对应的差值小于第二预设阈值；所述从所述待匹配像素点集中分别选取与所述标准像素点集中的像素点相匹配的每个像素点的执行次数大于第三预设阈值。
- 根据权利要求12所述方法，其特征在于，所述基于所述多个匹配点对更新所述转换关系信息，包括：基于所述多个匹配点对，以所述转换关系信息为变量构造残差函数；对所述残差函数进行求导操作，以所述残差函数的值趋于零为目标，确定更新后的转换关系信息。
- 一种图像匹配的装置，包括：像素点提取模块，用于获取待匹配图像中目标对象的轮廓对应的待匹配像素点集，和标准图像中所述目标对象的轮廓对应的标准像素点集；变换处理模块，用于确定所述待匹配像素点集和所述标准像素点集之间的转换关系信息；匹配模块，用于基于所述转换关系信息，从所述待匹配像素点集中选取与所述标准像素点集中的像素点相匹配的像素点，得到多个匹配点对；关系形成模块，用于基于所述多个匹配点对，确定所述标准像素点集中的像素点与所述待匹配像素点集中的像素点之间的匹配关系信息。
- 一种电子设备，包括处理器、存储器和总线，所述存储器存储有所述处理器可执行的机器可读指令，当电子设备运行时，所述处理器与所述存储器之间通过总线通信，所述机器可读指令被所述处理器执行时执行如权利要求1至14任一所述的图像匹配的方法的步骤。
- 一种计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器运行时执行如权利要求1至14任一所述的图像匹配的方法的步骤。
- 一种车辆，其特征在于，包括如权利要求16所述的电子设备。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011597941.X | 2020-12-29 | ||
| CN202011597941.XA CN112734837B (zh) | 2020-12-29 | 2020-12-29 | 图像匹配的方法及装置、电子设备及车辆 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2022142206A1 true WO2022142206A1 (zh) | 2022-07-07 |
Family
ID=75611393
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2021/102929 WO2022142206A1 (zh) | 2020-12-29 | 2021-06-29 | 图像匹配的方法及装置、电子设备及车辆 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112734837B (zh) |
| WO (1) | WO2022142206A1 (zh) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116625385A (zh) * | 2023-07-25 | 2023-08-22 | 高德软件有限公司 | 路网匹配方法、高精地图构建方法、装置及设备 |
| CN116977471A (zh) * | 2023-08-04 | 2023-10-31 | 广州柏视医疗科技有限公司 | 一种生成合成计算机断层扫描图像的方法和设备 |
| CN116977471B (zh) * | 2023-08-04 | 2024-06-07 | 广州柏视医疗科技有限公司 | 一种生成合成计算机断层扫描图像的方法和设备 |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112734837B (zh) * | 2020-12-29 | 2024-03-22 | 上海商汤临港智能科技有限公司 | 图像匹配的方法及装置、电子设备及车辆 |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190244054A1 (en) * | 2014-07-09 | 2019-08-08 | Slyce Acquisition Inc. | Systems, methods, and devices for image matching and object recognition in images |
| CN110287943A (zh) * | 2019-07-03 | 2019-09-27 | 厦门美图之家科技有限公司 | 图像的对象识别方法、装置、电子设备及存储介质 |
| CN110598795A (zh) * | 2019-09-17 | 2019-12-20 | 展讯通信（上海）有限公司 | 图像差异检测方法及装置、存储介质、终端 |
| CN111091590A (zh) * | 2019-12-18 | 2020-05-01 | Oppo广东移动通信有限公司 | 图像处理方法、装置、存储介质及电子设备 |
| CN112734837A (zh) * | 2020-12-29 | 2021-04-30 | 上海商汤临港智能科技有限公司 | 图像匹配的方法及装置、电子设备及车辆 |
-
2020
- 2020-12-29 CN CN202011597941.XA patent/CN112734837B/zh active Active
-
2021
- 2021-06-29 WO PCT/CN2021/102929 patent/WO2022142206A1/zh active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190244054A1 (en) * | 2014-07-09 | 2019-08-08 | Slyce Acquisition Inc. | Systems, methods, and devices for image matching and object recognition in images |
| CN110287943A (zh) * | 2019-07-03 | 2019-09-27 | 厦门美图之家科技有限公司 | 图像的对象识别方法、装置、电子设备及存储介质 |
| CN110598795A (zh) * | 2019-09-17 | 2019-12-20 | 展讯通信（上海）有限公司 | 图像差异检测方法及装置、存储介质、终端 |
| CN111091590A (zh) * | 2019-12-18 | 2020-05-01 | Oppo广东移动通信有限公司 | 图像处理方法、装置、存储介质及电子设备 |
| CN112734837A (zh) * | 2020-12-29 | 2021-04-30 | 上海商汤临港智能科技有限公司 | 图像匹配的方法及装置、电子设备及车辆 |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116625385A (zh) * | 2023-07-25 | 2023-08-22 | 高德软件有限公司 | 路网匹配方法、高精地图构建方法、装置及设备 |
| CN116625385B (zh) * | 2023-07-25 | 2024-01-26 | 高德软件有限公司 | 路网匹配方法、高精地图构建方法、装置及设备 |
| CN116977471A (zh) * | 2023-08-04 | 2023-10-31 | 广州柏视医疗科技有限公司 | 一种生成合成计算机断层扫描图像的方法和设备 |
| CN116977471B (zh) * | 2023-08-04 | 2024-06-07 | 广州柏视医疗科技有限公司 | 一种生成合成计算机断层扫描图像的方法和设备 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112734837A (zh) | 2021-04-30 |
| CN112734837B (zh) | 2024-03-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109410321B (zh) | 基于卷积神经网络的三维重建方法 | |
| CN110223348B (zh) | 基于rgb-d相机的机器人场景自适应位姿估计方法 | |
| US20220262039A1 (en) | Positioning method, electronic device, and storage medium | |
| CN108537848B (zh) | 一种面向室内场景重建的两级位姿优化估计方法 | |
| US20190147245A1 (en) | Three-dimensional object detection for autonomous robotic systems using image proposals | |
| US9754377B2 (en) | Multi-resolution depth estimation using modified census transform for advanced driver assistance systems | |
| WO2022142206A1 (zh) | 图像匹配的方法及装置、电子设备及车辆 | |
| JP2021524115A (ja) | 対象三次元検出及びスマート運転制御方法、装置、媒体並びに機器 | |
| CN108010123B (zh) | 一种保留拓扑信息的三维点云获取方法 | |
| CN109829353B (zh) | 一种基于空间约束的人脸图像风格化方法 | |
| CN115205489A (zh) | 一种大场景下的三维重建方法、系统及装置 | |
| CN109101981B (zh) | 一种街景场景下基于全局图像条纹码的回环检测方法 | |
| CN113012210A (zh) | 深度图的生成方法、装置、电子设备和存储介质 | |
| CN110910433A (zh) | 一种基于深度学习的点云匹配方法 | |
| WO2022205618A1 (zh) | 定位方法、行驶控制方法、装置、计算机设备及存储介质 | |
| Kong et al. | Local stereo matching using adaptive cross-region-based guided image filtering with orthogonal weights | |
| WO2021142843A1 (zh) | 图像扫描方法及装置、设备、存储介质 | |
| CN114898041A (zh) | 一种基于光度误差的改进icp方法 | |
| Phalak et al. | DeepPerimeter: Indoor boundary estimation from posed monocular sequences | |
| Tanner et al. | DENSER cities: A system for dense efficient reconstructions of cities | |
| Zhang | Image feature extraction algorithm in big data environment | |
| WO2023109069A1 (zh) | 一种图像检索方法及装置 | |
| Yang et al. | Ground plane matters: Picking up ground plane prior in monocular 3d object detection | |
| KR102449031B1 (ko) | 딥러닝을 이용한 실내 위치 측위 방법 | |
| CN115482268A (zh) | 一种基于散斑匹配网络的高精度三维形貌测量方法与系统 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21912962 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 21912962 Country of ref document: EP Kind code of ref document: A1 |
|
| 32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 28.11.2023) |