CN111666959A - Vector image matching method and device - Google Patents

Abstract

The invention provides a vector image matching method and a vector image matching device, relates to the technical field of electronic information, and can solve the problem that a flight device cannot be positioned in a flight area with poor electromagnetic wave signals. The specific technical scheme is as follows: acquiring at least one basic image shot by the flying device, wherein the basic image is an image of the ground shot by the flying device from top to bottom in a top view; carrying out vector extraction on at least one basic image to obtain at least one vector image; and matching at least one vector image with a prestored reference image to determine the position area of the flying device in the reference image. The present disclosure is for use in flying device positioning.

Description

Vector image matching method and device

technical field

The present disclosure relates to the technical field of electronic information, and in particular, to a vector image matching method and device.

Background technique

With the development of flight device technology, flight device technology has been applied to many fields, such as aerial photography, transportation, monitoring, and the like. During the flight of the flying device, it is usually necessary to locate the flying device, for example, through GPS (English: Global Positioning System, GPS) positioning, or through the network positioning, but these positioning methods need to ensure that they can receive/transmit electromagnetic wave signals. In some places, in the flight area with poor electromagnetic wave signal, it is impossible to locate the flight device.

SUMMARY OF THE INVENTION

Embodiments of the present disclosure provide a vector image matching method and device, which can solve the problem that the flight device cannot be positioned in a flight area with poor electromagnetic wave signals. The technical solutions are as follows:

According to a first aspect of the embodiments of the present disclosure, there is provided a vector image matching method, the method comprising:

Acquiring at least one basic image taken by the flying device, where the basic image is an image of the ground taken by the flying device from top to bottom in a bird's-eye view;

Perform vector extraction on at least one basic image to obtain at least one vector image;

Matching at least one vector image with a pre-stored reference image to determine the position area of the flying device in the reference image.

The vector image is obtained by extracting the vector from the basic image, and the vector image is used for matching, the matching is more accurate, and the vector image matching calculation amount is small, which greatly reduces the calculation amount of complex image matching. Realize fast and accurate positioning of the flying device.

In one embodiment, matching at least one vector image with a pre-stored reference image to determine the location area of the flying device includes:

Obtain the feature value of the first reference object in the target vector image according to the target vector image;

comparing the feature value of the first reference object with the feature value of at least one second reference object in the reference image;

Determining at least one second reference object with the same characteristic value as the first reference object as the target reference object;

The position area of the flying device in the reference image is determined according to the position of the target reference object in the reference image.

In one embodiment, the first reference object includes an intersection in the target vector image, and the feature value of the first reference object includes at least one of the number of roads, the orientation of adjacent intersections, and the number of preset reference objects.

In one embodiment, performing vector extraction on at least one basic image to obtain at least one vector image, including:

Extract feature values from each basic image according to preset features;

The corresponding vector image is generated according to the feature value of each basic image.

In one embodiment, matching at least one vector image with a pre-stored reference image includes:

Stitching at least one vector image according to the time sequence of shooting to obtain a stitched image;

Match the stitched image to the reference image.

According to a second aspect of the embodiments of the present disclosure, there is provided a vector image matching device, the vector image matching device comprising: an acquisition module, an extraction module, and a matching module;

Wherein, the acquisition module is configured to acquire at least one basic image taken by the flying device, where the basic image is an image of the ground taken by the flying device from top to bottom from a top-down perspective;

an extraction module, configured to perform vector extraction on at least one basic image to obtain at least one vector image;

The matching module is used for matching at least one vector image with a pre-stored reference image to determine the position area of the flying device in the reference image.

In one embodiment, the matching module includes: a first feature value unit, a comparison unit, a determination unit, and a position unit;

Wherein, the first feature value unit is used to obtain the feature value of the first reference object in the target vector image according to the target vector image;

a comparison unit, configured to compare the feature value of the first reference object with the feature value of at least one second reference object in the reference image;

a determination unit, configured to determine a second reference object with the same characteristic value as the first reference object in at least one second reference object as a target reference object;

The position unit is used for determining the position area of the flying device in the reference image according to the position of the target reference object in the reference image.

In one embodiment, the first reference object includes an intersection in the target vector image, and the feature value of the first reference object includes at least one of the number of roads, the orientation of adjacent intersections, and the number of preset reference objects.

In one embodiment, the extraction module includes a second feature value unit and a vector image unit;

a second feature value unit for extracting feature values from each basic image according to preset features;

The vector image unit is used to generate a corresponding vector image according to the feature value of each basic image.

In one embodiment, the matching module includes a splicing unit and a matching unit;

a splicing unit, used for splicing at least one vector image according to the time sequence of shooting to obtain a spliced image;

The matching unit is used to match the stitched image with the reference image.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure.

1 is a flowchart of a vector image matching method provided by an embodiment of the present disclosure;

2 is a schematic diagram of a vector extraction effect provided by an embodiment of the present disclosure;

3 is a schematic diagram of a road vector extraction network provided by an embodiment of the present disclosure;

FIG. 4 is a characteristic schematic diagram of a first reference object provided by an embodiment of the present disclosure;

5 is a structural diagram of a vector image matching apparatus provided by an embodiment of the present disclosure;

6 is a structural diagram of a vector image matching apparatus provided by an embodiment of the present disclosure;

7 is a structural diagram of a vector image matching apparatus provided by an embodiment of the present disclosure;

FIG. 8 is a structural diagram of a vector image matching apparatus provided by an embodiment of the present disclosure.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.

An embodiment of the present disclosure provides a vector image matching method, which is applied to a vector image matching apparatus. As shown in FIG. 1 , FIG. 1 is a flowchart of a vector image matching method provided by an embodiment of the present disclosure. The vector image matching method includes the following steps:

101. Acquire at least one basic image captured by the flight device.

The base image is an image of the ground taken from the top down by the flying device in a top-down view. The flying device may be a drone.

102. Perform vector extraction on at least one basic image to obtain at least one vector image.

In one embodiment, performing vector extraction on at least one basic image to obtain at least one vector image, including:

Extract feature values from each basic image according to preset features; generate corresponding vector images according to the feature values of each basic image.

For example, a fully convolutional network architecture can be used to implement pixel-by-pixel classification of the target image, as shown in FIG. 2 , which is a schematic diagram of a vector extraction effect provided by an embodiment of the present disclosure to determine whether the target pixel in the basic image belongs to a road , the target pixel is any pixel in the basic image, if the target pixel belongs to the road, the eigenvalue of the target pixel is marked as 1, if the target pixel does not belong to the road, the eigenvalue of the target pixel is marked as 0, and finally the basic The image is extracted to obtain the feature value, and then a vector image is generated according to the feature value of each pixel. In the vector image, only the shape of the road is displayed. The eigenvalues can also be the same as vector images, that is, the eigenvalues are directly used as vector images. Of course, in FIG. 2 , the extracted feature is a road as an example for illustration, and the extracted feature may also include a river, a building, etc., which is not limited in the present disclosure.

Further, here, taking the vector extraction of a road as an example, the process of vector extraction is described in detail through the following three steps.

The first step is to use a fully convolutional network to achieve pixel-level image segmentation of the basic image. Under the full convolution framework, the network structure can be adaptively adjusted for different imaging heights and different imaging sensors by adding jump-layer connections from different low-level features to high-level features, so that the designed network can cover different heights. Space-based platform.

The algorithm used for image segmentation in this step can be a highly adaptive image segmentation algorithm, which is based on a fully convolutional network and embeds the conversion model of height information and region candidate frame size information into the segmentation network; at the same time, the VGG16 feature extraction network The feature map obtained by conv5_3 through deconvolution and the channel series fusion of the conv4_3 layer feature map replaces the original conv4_3 feature map, and then performs channel series operation with fc7 and conv7_2 to further enrich the semantic information of the feature map and improve the detection accuracy. The network structure is shown in Figure 3. In Figure 3, the left side is the input image and height information. Considering the actual application, here is an image with a size of 1024 × 768 as an example. After the VGG16 feature extraction network, the conv5_3 feature layer is subjected to a 2 × 2 deconvolution operation and The conv4_3 channels are fused in series to obtain the fused feature map, and then a convolution operation is performed with fc7 and conv7_2 respectively. After connecting the three different feature map channels in series, a 1×1 convolution operation and normalization are performed to obtain The number of channels is 512 and the feature map size is 38×38. In the above model, the detector part still adopts the basic structure of FSSD. By performing multiple convolution operations on the obtained feature maps, six feature maps with different receptive fields and sizes are generated.

The network structure encodes features through a convolutional network, and then decodes through deconvolution and upsampling, which belongs to a deep encoder-decoder network with highly adaptive imaging capabilities. By adjusting the depth of the network, the network depth is reduced to obtain higher segmentation accuracy of road details.

For example, the image captured by air-to-ground imaging can be normalized (assuming a size of 512×512), using a fully convolutional network, after several layers of convolution and several max pooling, the air-to-ground image can be completed. image encoding. Due to convolution and pooling, the resulting feature map is usually much smaller than the original image (assuming 1/4). In order to achieve pixel-level segmentation of the image, the feature map is used as the input layer of the decoding network. After several upsampling, the feature map with the same size as the original image is obtained. Finally, a special convolution is performed on the feature map with the same size as the original image. The convolution kernel size is a convolutional layer of 1×1, and the output is mapped into a probability map of pixel classification with a size of 512×512×1. So far, the design of the road pixel-level segmentation network is completed.

The second step is to create a road segmentation dataset and train network parameters. Set the air-to-ground road segmentation data set, mainly including satellite imaging data sets, UAV aerial photography data sets, etc., and form air-to-ground road segmentation data with a certain scale and covering different imaging heights by labeling the original data. set. Then, on the basis of the data set, the fully convolutional network designed above is trained to obtain high-precision network weights, and the weight matrix is sent to the airborne computing platform.

The third step is automatic road extraction based on the actual flight image of the road extraction network. After the road extraction network structure and its weight files are sent to the airborne computing and processing platform, the air-based flight platform performs flight photography and automatic road extraction, and obtains the road topology structure corresponding to the current position of the UAV in real time. are described and stored in the form of

103. Match at least one vector image with a pre-stored reference image to determine a position area of the flying device in the reference image.

In one embodiment, the reference image is a vector map of the area where the flying device flies, that is, according to steps 101 and 102, the area where the flying device is flying is photographed and the vector is extracted, and then the vector of the entire area is obtained by splicing image.

In one embodiment, matching at least one vector image with a pre-stored reference image to determine the location area of the flying device includes:

Obtain the feature value of the first reference object in the target vector image according to the target vector image; compare the feature value of the first reference object with the feature value of at least one second reference object in the reference image; compare the at least one second reference object with The second reference object with the same feature value of the first reference object is determined as the target reference object; the position area of the flying device in the reference image is determined according to the position of the target reference object in the reference image.

The target vector image is any one of at least one vector image, and the first reference object is any one of the reference objects in the target vector image. Here, the first reference object is only used as an example for description, and does not represent any limitation. Further, in one embodiment, the first reference object includes an intersection in the target vector image, and the feature value of the first reference object includes at least one of the number of roads, the orientation of adjacent intersections, and the number of preset reference objects.

For example, taking the first reference object and the second reference object being an intersection as an example, as shown in FIG. 4 , FIG. 4 is a schematic diagram of the characteristics of a first reference object provided by an embodiment of the present disclosure. (that is, a certain intersection in the target vector image) The feature value includes the number of roads and the orientation of adjacent intersections; the number of roads is the number of roads connected to the intersection, and the orientation of adjacent intersections refers to the relative distance of the intersection with the closest distance to the intersection. The orientation of the intersection, as shown in Figure 4, the closest intersection to the intersection is located in the due east (right), the orientation starts from the east, and is numbered in clockwise order, for example, the east is marked as 1, and the southeast is marked as 2 , the south is marked as 3, the southwest is marked as 4, the west is marked as 5, the northwest is marked as 6, the north is marked as 7, and the northeast is marked as 8. The intersection as shown in Figure 3, the closest intersection to the intersection is in the east , so it is recorded as 1, then the eigenvalue of the intersection is [4,1]. The second reference object with the same feature value is determined as the target reference object in at least one second reference object of the reference image according to the feature value, and the position area of the target vector image in the reference image can be determined.

In one embodiment, matching at least one vector image with a pre-stored reference image includes:

The at least one vector image is stitched according to the time sequence of shooting to obtain a stitched image; the stitched image is matched with the reference image.

In the vector image matching method provided by the embodiments of the present disclosure, a vector image is obtained by extracting a vector from a basic image, and the vector image is used for matching, so that the matching is more accurate, and the vector image matching operation amount is small, which greatly reduces the operation in complex image matching. It can realize fast and accurate positioning of the flying device in the area with poor electromagnetic wave signal.

Based on the vector image matching method described in the above-mentioned embodiment corresponding to FIG. 1 , an embodiment of the present disclosure provides a vector image matching apparatus for executing the vector image matching method described in the above-mentioned embodiment corresponding to FIG. 1 , as shown in FIG. 5, the vector image matching device 50 includes: an acquisition module 501, an extraction module 502, and a matching module 503;

Wherein, the acquiring module 501 is configured to acquire at least one basic image taken by the flying device, where the basic image is an image of the ground taken by the flying device from top to bottom from a top-down perspective;

an extraction module 502, configured to perform vector extraction on at least one basic image to obtain at least one vector image;

The matching module 503 is configured to match at least one vector image with a pre-stored reference image to determine the position area of the flying device in the reference image.

In one embodiment, as shown in FIG. 6 , the matching module 503 includes: a first feature value unit 5031, a comparison unit 5032, a determination unit 5033 and a position unit 5034;

Wherein, the first feature value unit 5031 is used to obtain the feature value of the first reference object in the target vector image according to the target vector image;

a comparison unit 5032, configured to compare the feature value of the first reference object with the feature value of at least one second reference object in the reference image;

a determining unit 5033, configured to determine a second reference object having the same characteristic value as the first reference object in at least one second reference object as a target reference object;

The position unit 5034 is configured to determine the position area of the flying device in the reference image according to the position of the target reference object in the reference image.

In one embodiment, the first reference object includes an intersection in the target vector image, and the feature value of the first reference object includes at least one of the number of roads, the orientation of adjacent intersections, and the number of preset reference objects.

In one embodiment, as shown in FIG. 7 , the extraction module 502 includes a second feature value unit 5021 and a vector image unit 5022;

The second feature value unit 5021 is used to extract feature values from each basic image according to preset features;

The vector image unit 5022 is configured to generate a corresponding vector image according to the feature value of each basic image.

In one embodiment, as shown in FIG. 8 , the matching module 503 includes a splicing unit 5035 and a matching unit 5036;

The splicing unit 5035 is used for splicing at least one vector image according to the time sequence of shooting to obtain a spliced image;

A matching unit 5036, configured to match the stitched image with the reference image.

The vector image matching device provided by the embodiment of the present disclosure obtains a vector image by extracting a vector from a basic image, and uses the vector image to perform matching, the matching is more accurate, and the vector image matching calculation amount is small, which greatly reduces the calculation of complex image matching. It can realize fast and accurate positioning of the flying device in the area with poor electromagnetic wave signal.

Based on the vector image matching method described in the above-mentioned embodiment corresponding to FIG. 1 , an embodiment of the present disclosure further provides a computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a read-only memory (English: Read Only Memory, ROM), random access memory (English: Random Access Memory, RAM), CD-ROM, magnetic tape, floppy disk and optical data storage device, etc. The storage medium stores computer instructions for executing the vector image matching method described in the embodiment corresponding to FIG. 1 , which will not be repeated here.

Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or techniques in the technical field not disclosed by the present disclosure . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

Claims (10)

1. A vector image matching method, the method comprising:

acquiring at least one basic image shot by a flying device, wherein the basic image is an image of the ground shot by the flying device from top to bottom in a top view;

performing vector extraction on the at least one basic image to obtain at least one vector image;

and matching the at least one vector image with a prestored reference image to determine the position area of the flying device in the reference image.

2. The method of claim 1, wherein matching the at least one vector image to a pre-stored reference image to determine the location area of the flying device comprises:

acquiring a characteristic value of a first reference object in a target vector image according to the target vector image;

comparing the characteristic value of the first reference object with the characteristic value of at least one second reference object in the reference image;

determining a second reference object which is the same as the characteristic value of the first reference object in the at least one second reference object as a target reference object;

and determining the position area of the flying device in the reference image according to the position of the target reference object in the reference image.

3. The method of claim 2,

the first reference object comprises an intersection in the target vector image, and the characteristic value of the first reference object comprises at least one of the number of roads, the position of the adjacent intersection and the number of preset reference objects.

4. The method of claim 1, wherein vector extracting the at least one base image to obtain at least one vector image comprises:

extracting a characteristic value in each basic image according to preset characteristics;

and generating a corresponding vector image according to the characteristic value of each basic image.

5. The method according to any of claims 1-4, wherein matching the at least one vector image with a pre-stored reference image comprises:

splicing the at least one vector image according to the shooting time sequence to obtain a spliced image;

and matching the spliced image with the reference image.

6. A vector image matching apparatus, characterized in that the vector image matching apparatus comprises: the device comprises an acquisition module, an extraction module and a matching module;

the acquisition module is used for acquiring at least one basic image shot by a flying device, wherein the basic image is an image of the ground shot by the flying device from top to bottom in a top view;

the extraction module is used for carrying out vector extraction on the at least one basic image to obtain at least one vector image;

the matching module is used for matching the at least one vector image with a prestored reference image and determining the position area of the flying device in the reference image.

7. The apparatus of claim 6, wherein the matching module comprises: the device comprises a first characteristic value unit, a comparison unit, a determination unit and a position unit;

the first characteristic value unit is used for acquiring a characteristic value of a first reference object in a target vector image according to the target vector image;

the comparison unit is used for comparing the characteristic value of the first reference object with the characteristic value of at least one second reference object in the reference image;

the determining unit is used for determining a second reference object which is the same as the characteristic value of the first reference object in the at least one second reference object as a target reference object;

the position unit is used for determining the position area of the flying device in the reference image according to the position of the target reference object in the reference image.

8. The apparatus of claim 7,

the first reference object comprises an intersection in the target vector image, and the characteristic value of the first reference object comprises at least one of the number of roads, the position of the adjacent intersection and the number of preset reference objects.

9. The apparatus of claim 6, wherein the extraction module comprises a second feature value unit and a vector image unit;

the second characteristic value unit is used for extracting a characteristic value in each basic image according to preset characteristics;

and the vector image unit is used for generating a corresponding vector image according to the characteristic value of each basic image.

10. The apparatus according to any one of claims 6-9, wherein the matching module comprises a splicing unit and a matching unit;

the splicing unit is used for splicing the at least one vector image according to a shooting time sequence to obtain a spliced image;

the matching unit is used for matching the spliced image with the reference image.

Images