CN116524364B - Heterogeneous multi-source data processing system and method based on feature recognition - Google Patents
Heterogeneous multi-source data processing system and method based on feature recognition Download PDFInfo
- Publication number
- CN116524364B CN116524364B CN202310747972.6A CN202310747972A CN116524364B CN 116524364 B CN116524364 B CN 116524364B CN 202310747972 A CN202310747972 A CN 202310747972A CN 116524364 B CN116524364 B CN 116524364B
- Authority
- CN
- China
- Prior art keywords
- target
- unmanned aerial
- aerial vehicle
- dimensional object
- features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000012545 processing Methods 0.000 title claims abstract description 18
- 230000011218 segmentation Effects 0.000 claims abstract description 21
- 238000012216 screening Methods 0.000 claims abstract description 10
- 230000000712 assembly Effects 0.000 claims abstract 3
- 238000000429 assembly Methods 0.000 claims abstract 3
- 239000013598 vector Substances 0.000 claims description 32
- 238000003672 processing method Methods 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 6
- 230000004927 fusion Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000013481 data capture Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000013480 data collection Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/17—Terrestrial scenes taken from planes or by drones
Abstract
The invention provides a heterogeneous multi-source data processing system and method based on feature recognition, and belongs to the technical field of data recognition and image processing. The method comprises the step S110: acquiring heterogeneous multi-source data to be processed; s120: performing feature recognition on the near-field camera of the handheld terminal to determine a plurality of target recognition features; s130: performing region segmentation on the satellite remote sensing image graph to obtain at least one first target region; s140: counting the first number of target identification features contained in each first target area; s150: screening at least one second target area based on the first number; s160: and determining a second target three-dimensional object matched with the plurality of target recognition features from the aerial image of the unmanned aerial vehicle. The system comprises a handheld terminal, an unmanned aerial vehicle and a satellite remote sensing image database, wherein the handheld terminal and the unmanned aerial vehicle are both provided with shooting assemblies. The invention can realize the characteristic recognition and matching of multi-source heterogeneous images with different scales.
Description
Technical Field
The invention belongs to the technical field of data identification and image processing, and particularly relates to a heterogeneous multi-source data processing system and method based on feature identification.
Background
The rapid development of the internet of vehicles technology makes intelligent auxiliary driving technology possible from practical application. Intelligent auxiliary driving requires support of corresponding hardware levels (such as laser radar and vehicle-mounted cameras), and support of software levels and cloud databases is also indispensable. Among these, it is an important loop that map data capture and navigation environments be generated in real-time to generate driving decisions.
Map data capture includes environmental data collection from multiple sources, and real-time generation of navigation environments requires fusion of multi-source heterogeneous environmental data. In this process, it is necessary to acquire a plurality of image data with the same target feature of different scales in advance to perform feature matching, and after the feature matching is output, it is necessary to instruct the image acquisition apparatus to continue image data acquisition in the target region based on the output result.
In the prior art, the above-mentioned process is generally based on a test vehicle or a tester performing multiple acquisitions according to a predetermined route to obtain image data to be fused. The target object to be calibrated or the target area to be subjected to image acquisition is determined empirically, so that the whole process has high subjectivity, and the self-adaptability of the feature matching result output by the current fusion image data to other similar target objects or target areas is not strong. Meanwhile, the mode of carrying out multiple collection through a preset route also leads to lower efficiency of the whole image processing and collection process.
Therefore, a scheme for rapidly realizing the characteristic identification and matching of multi-source heterogeneous images with different scales is required to be provided so as to meet the requirements of image acquisition, fusion and characteristic matching in the intelligent Internet of vehicles era.
Disclosure of Invention
In order to solve the technical problems, the invention provides a heterogeneous multi-source data processing system and method based on feature recognition.
In a first aspect of the invention, a heterogeneous multi-source data processing method based on feature recognition is proposed, which is applicable to electronic devices, in particular image processing devices.
The method comprises the following steps:
s110: acquiring heterogeneous multi-source data to be processed, wherein the heterogeneous multi-source data to be processed comprises a satellite remote sensing image, an unmanned aerial vehicle high-altitude shooting image and a handheld terminal near-field shooting image;
s120: performing feature recognition on the near-field camera image of the handheld terminal, and determining a plurality of target recognition features;
s130: based on the target recognition features, carrying out region segmentation on the satellite remote sensing image map to obtain at least one first target region; each of the first target areas includes at least one of the target identification features;
s140: counting a first number of target identification features contained in each first target area;
s150: screening out at least one second target region based on the first number;
s160: and determining a second target three-dimensional object matched with the target recognition features from the aerial image of the unmanned aerial vehicle.
Specifically, the plurality of target recognition features determined in the step S120 include two-dimensional features and three-dimensional features; the two-dimensional features include shape features and the three-dimensional features include topology features.
The step S140 specifically includes: counting a second number of two-dimensional features contained in each first target area and a third number of three-dimensional features contained in each first target area;
the step S150 specifically includes: and if the second number of the two-dimensional features contained in the first target area is larger than a first threshold value and the third number of the three-dimensional features contained in the first target area is larger than a second threshold value, the first target area is taken as the second target area.
After the step S150, before the step S160, the method further includes:
and controlling the unmanned aerial vehicle to shoot the second target area, and obtaining at least one unmanned aerial vehicle high-altitude shooting picture.
The step S160 specifically includes:
generating at least one feature vector based on the plurality of object recognition features;
and determining a second target three-dimensional object matched with the first target three-dimensional object from the aerial image of the unmanned aerial vehicle based on the feature vector.
In a second aspect of the present invention, another heterogeneous multi-source data processing method based on feature recognition is also provided, the method comprising the steps of:
s610: shooting a first target three-dimensional object through the handheld terminal to obtain a near-field shooting image of the handheld terminal;
s620: performing feature recognition on the near-field camera image of the handheld terminal, and determining a plurality of target recognition features;
s630: acquiring a satellite remote sensing image map, wherein the satellite remote sensing image map comprises an area where the first target three-dimensional object is located;
s640: based on the target recognition features, carrying out region segmentation on the satellite remote sensing image map to obtain at least one first target region; each of the first target areas includes at least one of the target identification features;
s650: counting a first number of target identification features contained in each first target area;
s660: screening out at least one second target region based on the first number;
s670: controlling the unmanned aerial vehicle to shoot the second target area, and obtaining at least one unmanned aerial vehicle high-altitude shooting picture;
s680: and determining a second target three-dimensional object matched with the first target three-dimensional object from the aerial image of the unmanned aerial vehicle.
The step S680 specifically includes:
acquiring at least one feature vector of the first target three-dimensional object;
and determining a second target three-dimensional object matched with the first target three-dimensional object from the aerial image of the unmanned aerial vehicle based on the feature vector.
In order to implement the method of the first aspect or the second aspect, in a third aspect of the present invention, a heterogeneous multi-source data processing system based on feature recognition is provided, where the system includes a handheld terminal, a drone, and a satellite remote sensing image database, and the handheld terminal and the drone are both configured with a shooting component, so as to implement the method of the first aspect or the second aspect.
In a fourth aspect of the present invention, a heterogeneous multi-source data processing system based on feature recognition is provided, the system includes a handheld terminal, an unmanned aerial vehicle and a satellite remote sensing image database, and the handheld terminal and the unmanned aerial vehicle are both configured with a shooting component;
specifically, on the functional component, the system further comprises a feature recognition component, a region segmentation component, a feature statistics component, an unmanned aerial vehicle control component and a matching output component;
the handheld terminal shoots a fourth target area through the shooting component to obtain a near-field shooting image of the handheld terminal, and the near-field shooting image of the handheld terminal comprises a fifth target three-dimensional object;
the characteristic recognition component performs characteristic recognition on the near-field camera image of the handheld terminal and determines a plurality of target recognition characteristics;
acquiring a satellite remote sensing image map from the satellite remote sensing image database based on the fourth target area, wherein the satellite remote sensing image map takes the fourth target area as the center;
the region segmentation component performs region segmentation on the satellite remote sensing image map based on the plurality of target identification features to obtain at least one sixth target region; each of the sixth target areas includes at least one of the target identification features;
the feature statistics component counts a first number of target identification features contained in each sixth target area, and screens out at least one seventh target area based on the first number;
the unmanned aerial vehicle control assembly controls the unmanned aerial vehicle to shoot the seventh target area, and at least one unmanned aerial vehicle high-altitude shooting image is obtained;
and the matching output component determines an eighth target three-dimensional object matched with the fifth target three-dimensional object from the aerial image of the unmanned aerial vehicle and outputs the eighth target three-dimensional object.
The matching output component further comprises a vector extraction component;
the vector extraction component is used for extracting at least one feature vector of the fifth target three-dimensional object;
and the matching output component determines an eighth target three-dimensional object matched with the fifth target three-dimensional object from the aerial image of the unmanned aerial vehicle based on the feature vector and outputs the eighth target three-dimensional object.
According to the technical scheme, the multi-source heterogeneous images with different scales comprise a satellite remote sensing image, an unmanned aerial vehicle high-altitude shooting image and a handheld terminal near-field shooting image; firstly, carrying out feature recognition on the near-field camera image of the handheld terminal, and determining a plurality of target recognition features; then, based on the plurality of target identification features, carrying out region segmentation on the satellite remote sensing image map to obtain at least one first target region; each of the first target areas includes at least one of the target identification features; and then, counting the first quantity of target identification features contained in each first target area, and screening at least one second target area based on the first quantity, so that a second target three-dimensional object matched with the plurality of target identification features can be determined from the aerial image of the unmanned aerial vehicle.
In the process, the unmanned aerial vehicle is controlled to shoot the second target area, and at least one unmanned aerial vehicle high-altitude shooting image is obtained.
Therefore, the technical scheme of the invention is based on the scheme of rapidly realizing the characteristic recognition and matching of the multi-source heterogeneous images with different scales when the multi-scale images are acquired and matched with the characteristic recognition, and can meet the requirements of image acquisition, fusion and characteristic matching in the intelligent Internet of vehicles age.
Further advantages of the invention will be further elaborated in the description section of the embodiments in connection with the drawings.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a main flow chart of a heterogeneous multi-source data processing method based on feature recognition according to one embodiment of the present invention
FIG. 2 is a main flow chart of a heterogeneous multi-source data processing method based on feature recognition according to another embodiment of the present invention
FIG. 3 is a schematic diagram of the architecture of a heterogeneous multi-source data processing system of the method of FIG. 1 or FIG. 2
FIG. 4 is a schematic diagram illustrating a portion of the internal components of the heterogeneous multi-source data processing system of FIG. 3
Description of the embodiments
The invention will be further described with reference to the drawings and detailed description.
FIG. 1 illustrates a main flow diagram of a heterogeneous multi-source data processing method based on feature recognition according to one embodiment of the present invention.
The method shown in fig. 1 includes steps S110-S160, each of which is implemented as follows:
s110: acquiring heterogeneous multi-source data to be processed, wherein the heterogeneous multi-source data to be processed comprises a satellite remote sensing image, an unmanned aerial vehicle high-altitude shooting image and a handheld terminal near-field shooting image;
s120: and carrying out feature recognition on the near-field camera image of the handheld terminal, and determining a plurality of target recognition features.
In the embodiment, a first target three-dimensional object is shot through a handheld terminal, so that a near-field shooting image of the handheld terminal is obtained; thus, the target recognition object is the first target three-dimensional object;
as a specific example, the first target three-dimensional object may be a road marker, such as a speed limit sign, tunnel entrance, height limit pole, traffic light, or the like.
In order to characterize different three-dimensional objects, it is necessary to extract the target recognition features of the three-dimensional objects.
The plurality of target recognition features determined in the step S120 include two-dimensional features and three-dimensional features; the two-dimensional features include shape features and the three-dimensional features include topology features.
Shape is the most important property of a three-dimensional object, the model can be described more finely than color, texture and the shape is dimensionless, any rotation, translation does not change its properties. Calculating a shape model from the characteristics of the whole three-dimensional object, providing global information such as area, volume, perimeter, radius and the like, and not only needing no initial segmentation step but also no pretreatment, realizing rapidness and conforming to the technical scheme and application field Jing Xuqiu of the invention;
the usual shapes are area and Volume, taking a three-dimensional polygonal model as an example, these features are calculated as follows:
v is a vertex coordinate vector containing triangular faces i.
In the formula, the three-dimensional polygonal model has N triangular faces;
on the plane, for the ith triangle face, three vertex coordinates thereof are respectively expressed as；
Correspondingly, in three dimensions of xyz,coordinate component values representing the 0 th vertex coordinate vector (the IDs of three vertexes are marked as 0, 1 and 2) of the ith triangle face in xyz three directions; />Representing the coordinate component values of the 1 st vertex coordinate vector of a certain triangle surface in xyz three directions; />Representing the coordinate component values of the 2 nd vertex coordinate vector of a certain triangle surface in xyz three directions;
topology is also an important feature of three-dimensional objects, and a large number of entities need to describe the spatial relationships of the three-dimensional objects in order to properly represent the three-dimensional object content. The local characteristics are calculated using spatial relationships without segmentation models, and the spatial relationships are defined according to model features.
The method for representing the three-dimensional object features by the spatial topological structure of the three-dimensional object mentioned in the embodiment includes a multi-resolution Reeb graph method and a skeleton graph method.
S130: based on the target recognition features, carrying out region segmentation on the satellite remote sensing image map to obtain at least one first target region; each of the first target areas includes at least one of the target identification features;
in step S130, at least one satellite remote sensing image map needs to be acquired, where the satellite remote sensing image map includes an area where the first target three-dimensional object is located;
more specifically, based on a fourth target area where the first target three-dimensional object is located, acquiring a satellite remote sensing image map from the satellite remote sensing image database, wherein the satellite remote sensing image map is centered on the fourth target area;
s140: counting a first number of target identification features contained in each first target area;
s150: screening out at least one second target region based on the first number;
the step S140 specifically includes: counting a second number of two-dimensional features contained in each first target area and a third number of three-dimensional features contained in each first target area;
the step S150 specifically includes: and if the second number of the two-dimensional features contained in the first target area is larger than a first threshold value and the third number of the three-dimensional features contained in the first target area is larger than a second threshold value, the first target area is taken as the second target area.
And then, controlling the unmanned aerial vehicle to shoot the second target area, and obtaining at least one unmanned aerial vehicle high-altitude shooting picture.
S160: and determining a second target three-dimensional object matched with the target recognition features from the aerial image of the unmanned aerial vehicle.
The step S160 specifically includes:
generating at least one feature vector based on the plurality of object recognition features;
and determining a second target three-dimensional object matched with the first target three-dimensional object from the aerial image of the unmanned aerial vehicle based on the feature vector.
As a more specific example, taking the multi-resolution Reeb graph method for characterizing the three-dimensional target object as an example, the step S160 specifically includes:
based on the target recognition features, firstly extracting key topological points, constructing a multi-resolution Reeb graph based on the key topological points to maintain the topological structure features of the three-dimensional object, and then at each key topological point, rendering an image by mapping the three-dimensional object onto a plane tangential to a bounding sphere at the topological point; for each multi-resolution graph node, a method is used to extract view features from the rendered image.
In order to further reduce the comparison time and the storage overhead, the extracted view features are quantized and encoded by using a word bag method, so that the feature vectors are obtained.
It can be seen that for each three-dimensional object a unique feature vector can be obtained according to the method described above, and if the feature vector similarity of two three-dimensional objects reaches a preset criterion, the two three-dimensional objects form a matched three-dimensional object.
In the above embodiment, in order to construct a multi-resolution map of a three-dimensional object, it is necessary to use at each point on the set of key topological pointsFunction defines a height value, +.>The mathematical form of the function is as follows:
is the geodesic distance between point v and other points, s is the set of all topological points.
The extracted view features are quantized and encoded by using a word bag method, which belongs to the prior art, and the invention does not specifically develop the method.
The method described in fig. 1 mainly relates to the processing after the existing multi-source heterogeneous image data of different scales.
In yet another aspect, referring to fig. 2, a heterogeneous multi-source data processing method based on feature recognition of this embodiment mainly involves the control of the data acquisition itself.
Specifically, the method comprises the following steps:
s610: shooting a first target three-dimensional object through the handheld terminal to obtain a near-field shooting image of the handheld terminal;
s620: performing feature recognition on the near-field camera image of the handheld terminal, and determining a plurality of target recognition features;
s630: acquiring a satellite remote sensing image map, wherein the satellite remote sensing image map comprises an area where the first target three-dimensional object is located;
s640: based on the target recognition features, carrying out region segmentation on the satellite remote sensing image map to obtain at least one first target region; each of the first target areas includes at least one of the target identification features;
s650: counting a first number of target identification features contained in each first target area;
s660: screening out at least one second target region based on the first number;
s670: controlling the unmanned aerial vehicle to shoot the second target area, and obtaining at least one unmanned aerial vehicle high-altitude shooting picture;
s680: and determining a second target three-dimensional object matched with the first target three-dimensional object from the aerial image of the unmanned aerial vehicle.
It will be appreciated that some of the steps of the method of figure 2 correspond to the method of figure 1. For example, step S680, similar to the identification and matching of step S160, also uses a multi-resolution Reeb map to maintain the topological features of the three-dimensional object to generate feature vectors for matching the three-dimensional object.
Namely, the step S680 specifically includes:
acquiring at least one feature vector of the first target three-dimensional object;
and determining a second target three-dimensional object matched with the first target three-dimensional object from the aerial image of the unmanned aerial vehicle based on the feature vector.
The method described in fig. 1 or fig. 2 may be implemented by a system comprising a plurality of image acquisition devices. Accordingly, further embodiments include a feature recognition-based heterogeneous multi-source data processing system comprising a handheld terminal, a drone, and a satellite remote sensing image database, both of which are configured with a photographing assembly such that the system can be used to implement all of the steps of a feature recognition-based heterogeneous multi-source data processing method described in fig. 1 or fig. 2.
In particular, the system architecture can be seen in fig. 3.
The handheld terminal is communicated with the unmanned aerial vehicle and the satellite remote sensing image database, and the handheld terminal can call the corresponding satellite remote sensing image from the satellite remote sensing image database so as to control the shooting path of the unmanned aerial vehicle.
More specifically, referring to fig. 4, fig. 4 shows a functional principle of a part of components of a heterogeneous multi-source data processing system based on feature recognition, where the system includes a handheld terminal, an unmanned aerial vehicle and a satellite remote sensing image database, and the handheld terminal and the unmanned aerial vehicle are both configured with shooting components.
The system also comprises a feature recognition component, a region segmentation component, a feature statistics component, an unmanned aerial vehicle control component and a matching output component;
the handheld terminal shoots a fourth target area through the shooting component to obtain a near-field shooting image of the handheld terminal, and the near-field shooting image of the handheld terminal comprises a fifth target three-dimensional object;
the characteristic recognition component performs characteristic recognition on the near-field camera image of the handheld terminal and determines a plurality of target recognition characteristics;
acquiring a satellite remote sensing image map from the satellite remote sensing image database based on the fourth target area, wherein the satellite remote sensing image map takes the fourth target area as the center;
the region segmentation component performs region segmentation on the satellite remote sensing image map based on the plurality of target identification features to obtain at least one sixth target region; each of the sixth target areas includes at least one of the target identification features;
the feature statistics component counts a first number of target identification features contained in each sixth target area, and screens out at least one seventh target area based on the first number;
the unmanned aerial vehicle control assembly controls the unmanned aerial vehicle to shoot the seventh target area, and at least one unmanned aerial vehicle high-altitude shooting image is obtained;
and the matching output component determines an eighth target three-dimensional object matched with the fifth target three-dimensional object from the aerial image of the unmanned aerial vehicle and outputs the eighth target three-dimensional object.
The matching output component further comprises a vector extraction component;
the vector extraction component is used for extracting at least one feature vector of the fifth target three-dimensional object;
and the matching output component determines an eighth target three-dimensional object matched with the fifth target three-dimensional object from the aerial image of the unmanned aerial vehicle based on the feature vector and outputs the eighth target three-dimensional object.
It will be appreciated that for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, with reference to the description of method embodiments in part.
The method can be applied to the Internet of vehicles terminal. At this time, the hand-held terminal mentioned in the foregoing embodiment may be modified to a vehicle-mounted image pickup terminal, thereby obtaining a terminal near-field image pickup diagram;
the method can be applied to a vehicle-mounted terminal, and the vehicle-mounted terminal comprises a camera terminal.
Accordingly, further embodiments of the present invention further include:
a heterogeneous multi-source data processing method based on feature recognition is applied to a vehicle-mounted terminal, and the vehicle-mounted terminal comprises a camera terminal.
The method comprises the following steps:
the method comprises the steps that a vehicle-mounted terminal obtains heterogeneous multi-source data to be processed, wherein the heterogeneous multi-source data to be processed comprises a satellite remote sensing image, an unmanned aerial vehicle high-altitude shooting image and a terminal near-field shooting image;
the vehicle-mounted terminal performs feature recognition on the terminal near-field camera image and determines a plurality of target recognition features;
based on the target identification features, the vehicle-mounted terminal obtains a corresponding satellite remote sensing image map; performing region segmentation on the satellite remote sensing image map to obtain at least one first target region; each of the first target areas includes at least one of the target identification features;
counting a first number of target identification features contained in each first target area;
screening out at least one second target region based on the first number;
the vehicle-mounted terminal controls the unmanned aerial vehicle to shoot the second target area, and at least one unmanned aerial vehicle high-altitude shooting image is obtained;
and the vehicle-mounted terminal determines a second target three-dimensional object matched with the target recognition features from the aerial image of the unmanned aerial vehicle.
1-2, the "handheld terminal" need only be replaced with the "vehicle-mounted terminal";
it will be appreciated that any mobile terminal (including handheld terminals, vehicle mount terminals) may implement the corresponding steps and schemes of the method of fig. 1 or 2, as long as the camera assembly (corresponding to a near field camera of the mobile terminal) is configured.
Aiming at multi-source heterogeneous images with different scales, including a satellite remote sensing image, an unmanned aerial vehicle high-altitude shooting image and a mobile terminal near-field shooting image, the invention firstly carries out feature recognition on the mobile terminal near-field shooting image to determine a plurality of target recognition features; then, based on the plurality of target identification features, carrying out region segmentation on the satellite remote sensing image map to obtain at least one first target region; each of the first target areas includes at least one of the target identification features; next, counting a first number of target recognition features contained in each first target area, and screening at least one second target area based on the first number, so that a second target three-dimensional object matched with the plurality of target recognition features can be determined from the aerial image of the unmanned aerial vehicle; in the above process, the mobile terminal (including the vehicle-mounted terminal) controls the unmanned aerial vehicle to shoot the second target area, and at least one unmanned aerial vehicle high-altitude shooting image is obtained. Therefore, when the multi-scale image is acquired and matched with the feature recognition, the acquisition scheme and the acquisition path are determined based on the scheme of quickly realizing the feature recognition and the matching of the multi-source heterogeneous images with different scales, and meanwhile, the three-dimensional object is quickly and accurately matched based on the three-dimensional topological structure, so that the method can meet the requirements of image acquisition, fusion and feature matching in the intelligent Internet of vehicles.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments. In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present specification. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Claims (6)
1. A heterogeneous multi-source data processing method based on feature recognition, which is characterized by comprising the following steps:
s110: acquiring heterogeneous multi-source data to be processed, wherein the heterogeneous multi-source data to be processed comprises a satellite remote sensing image and a near-field camera image of a handheld terminal;
shooting and acquiring a first target three-dimensional object through the handheld terminal by the near-field camera of the handheld terminal;
the satellite remote sensing image map comprises an area where the first target three-dimensional object is located;
s120: performing feature recognition on the near-field camera image of the handheld terminal, and determining a plurality of target recognition features;
s130: based on the target recognition features, carrying out region segmentation on the satellite remote sensing image map to obtain at least one first target region; each of the first target areas includes at least one of the target identification features;
s140: counting a first number of target identification features contained in each first target area;
s150: screening out at least one second target region based on the first number; controlling the unmanned aerial vehicle to shoot the at least one second target area, and obtaining at least one unmanned aerial vehicle high-altitude shooting picture;
s160: generating at least one feature vector based on the plurality of object recognition features; and determining a second target three-dimensional object matched with the first target three-dimensional object from the aerial image of the unmanned aerial vehicle based on the feature vector.
2. A heterogeneous multi-source data processing method based on feature recognition as claimed in claim 1, wherein,
the plurality of target recognition features determined in the step S120 include two-dimensional features and three-dimensional features; the two-dimensional features include shape features and the three-dimensional features include topology features.
3. A heterogeneous multi-source data processing method based on feature recognition as claimed in claim 2, wherein,
the step S140 specifically includes: counting a second number of two-dimensional features contained in each first target area and a third number of three-dimensional features contained in each first target area;
the step S150 specifically includes: and if the second number of the two-dimensional features contained in the first target area is larger than a first threshold value and the third number of the three-dimensional features contained in the first target area is larger than a second threshold value, the first target area is taken as the second target area.
4. A heterogeneous multi-source data processing method based on feature recognition, which is characterized by comprising the following steps:
s610: shooting a first target three-dimensional object through the handheld terminal to obtain a near-field shooting image of the handheld terminal;
s620: performing feature recognition on the near-field camera image of the handheld terminal, and determining a plurality of target recognition features;
s630: acquiring a satellite remote sensing image map, wherein the satellite remote sensing image map comprises an area where the first target three-dimensional object is located;
s640: based on the target recognition features, carrying out region segmentation on the satellite remote sensing image map to obtain at least one first target region; each of the first target areas includes at least one of the target identification features;
s650: counting a first number of target identification features contained in each first target area;
s660: screening out at least one second target region based on the first number;
s670: controlling the unmanned aerial vehicle to shoot the second target area, and obtaining at least one unmanned aerial vehicle high-altitude shooting picture;
s680: determining a second target three-dimensional object matched with the first target three-dimensional object from the aerial image of the unmanned aerial vehicle;
the step S680 specifically includes:
acquiring at least one feature vector of the first target three-dimensional object;
and determining a second target three-dimensional object matched with the first target three-dimensional object from the aerial image of the unmanned aerial vehicle based on the feature vector.
5. The heterogeneous multi-source data processing system based on the feature recognition comprises a handheld terminal, an unmanned aerial vehicle and a satellite remote sensing image database, wherein the handheld terminal and the unmanned aerial vehicle are both provided with shooting assemblies.
6. The heterogeneous multi-source data processing system based on feature recognition comprises a handheld terminal, an unmanned aerial vehicle and a satellite remote sensing image database, wherein the handheld terminal and the unmanned aerial vehicle are both provided with shooting assemblies;
the method is characterized in that:
the system also comprises a feature recognition component, a region segmentation component, a feature statistics component, an unmanned aerial vehicle control component and a matching output component;
the handheld terminal shoots a fourth target area through the shooting component to obtain a near-field shooting image of the handheld terminal, and the near-field shooting image of the handheld terminal comprises a fifth target three-dimensional object;
the characteristic recognition component performs characteristic recognition on the near-field camera image of the handheld terminal and determines a plurality of target recognition characteristics;
acquiring a satellite remote sensing image map from the satellite remote sensing image database based on the fourth target area, wherein the satellite remote sensing image map takes the fourth target area as the center;
the region segmentation component performs region segmentation on the satellite remote sensing image map based on the plurality of target identification features to obtain at least one sixth target region; each of the sixth target areas includes at least one of the target identification features;
the feature statistics component counts a first number of target identification features contained in each sixth target area, and screens out at least one seventh target area based on the first number;
the unmanned aerial vehicle control assembly controls the unmanned aerial vehicle to shoot the seventh target area, and at least one unmanned aerial vehicle high-altitude shooting image is obtained;
the matching output component determines an eighth target three-dimensional object matched with the fifth target three-dimensional object from the aerial image of the unmanned aerial vehicle and outputs the eighth target three-dimensional object;
the matching output component further comprises a vector extraction component;
the vector extraction component is used for extracting at least one feature vector of the fifth target three-dimensional object;
and the matching output component determines an eighth target three-dimensional object matched with the fifth target three-dimensional object from the aerial image of the unmanned aerial vehicle based on the feature vector and outputs the eighth target three-dimensional object.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310747972.6A CN116524364B (en) | 2023-06-25 | 2023-06-25 | Heterogeneous multi-source data processing system and method based on feature recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310747972.6A CN116524364B (en) | 2023-06-25 | 2023-06-25 | Heterogeneous multi-source data processing system and method based on feature recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116524364A CN116524364A (en) | 2023-08-01 |
| CN116524364B true CN116524364B (en) | 2023-09-26 |
Family
ID=87405048
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310747972.6A Active CN116524364B (en) | 2023-06-25 | 2023-06-25 | Heterogeneous multi-source data processing system and method based on feature recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116524364B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117195054B (en) * | 2023-09-15 | 2024-03-26 | 苏州优鲜生网络科技有限公司 | Cross-node data identification method and system based on clusters |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103020945A (en) * | 2011-09-21 | 2013-04-03 | 中国科学院电子学研究所 | Remote sensing image registration method of multi-source sensor |
| CN106295613A (en) * | 2016-08-23 | 2017-01-04 | 哈尔滨理工大学 | A kind of unmanned plane target localization method and system |
| CN115526795A (en) * | 2022-09-23 | 2022-12-27 | 中北大学 | Unmanned aerial vehicle image shadow compensation method based on region matching and color migration |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113592882B (en) * | 2021-08-09 | 2022-06-03 | 中国林业科学研究院资源信息研究所 | Crown extraction method based on multi-source remote sensing of unmanned aerial vehicle |
-
2023
- 2023-06-25 CN CN202310747972.6A patent/CN116524364B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103020945A (en) * | 2011-09-21 | 2013-04-03 | 中国科学院电子学研究所 | Remote sensing image registration method of multi-source sensor |
| CN106295613A (en) * | 2016-08-23 | 2017-01-04 | 哈尔滨理工大学 | A kind of unmanned plane target localization method and system |
| CN115526795A (en) * | 2022-09-23 | 2022-12-27 | 中北大学 | Unmanned aerial vehicle image shadow compensation method based on region matching and color migration |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116524364A (en) | 2023-08-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109461211B (en) | Semantic vector map construction method and device based on visual point cloud and electronic equipment | |
| JP7326720B2 (en) | Mobile position estimation system and mobile position estimation method | |
| CN112396650B (en) | Target ranging system and method based on fusion of image and laser radar | |
| JP2019087229A (en) | Information processing device, control method of information processing device and program | |
| CN109520500B (en) | Accurate positioning and street view library acquisition method based on terminal shooting image matching | |
| CN111882612A (en) | Vehicle multi-scale positioning method based on three-dimensional laser detection lane line | |
| CN112233177B (en) | Unmanned aerial vehicle pose estimation method and system | |
| WO2022078240A1 (en) | Camera precise positioning method applied to electronic map, and processing terminal | |
| CN114413881B (en) | Construction method, device and storage medium of high-precision vector map | |
| KR101261409B1 (en) | System for recognizing road markings of image | |
| GB2520338A (en) | Automatic scene parsing | |
| WO2004095374A1 (en) | Video object recognition device and recognition method, video annotation giving device and giving method, and program | |
| CN111652934A (en) | Positioning method, map construction method, device, equipment and storage medium | |
| CN115439424A (en) | Intelligent detection method for aerial video image of unmanned aerial vehicle | |
| CN116524364B (en) | Heterogeneous multi-source data processing system and method based on feature recognition | |
| CN113989450A (en) | Image processing method, image processing apparatus, electronic device, and medium | |
| CN109063549B (en) | High-resolution aerial video moving target detection method based on deep neural network | |
| CN112308913A (en) | Vision-based vehicle positioning method and device and vehicle-mounted terminal | |
| CN113537047A (en) | Obstacle detection method, obstacle detection device, vehicle and storage medium | |
| CN112257638A (en) | Image comparison method, system, equipment and computer readable storage medium | |
| CN114241039A (en) | Map data processing method and device, storage medium and electronic equipment | |
| Jiao et al. | Individual building rooftop and tree crown segmentation from high-resolution urban aerial optical images | |
| CN112766068A (en) | Vehicle detection method and system based on gridding labeling | |
| Wu et al. | Derivation of Geometrically and Semantically Annotated UAV Datasets at Large Scales from 3D City Models | |
| KR20220062709A (en) | System for detecting disaster situation by clustering of spatial information based an image of a mobile device and method therefor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |