CN115248773A - Map data monitoring and verifying method and component type map data service cloud platform - Google Patents

Map data monitoring and verifying method and component type map data service cloud platform Download PDF

Info

Publication number
CN115248773A
CN115248773A CN202110470378.8A CN202110470378A CN115248773A CN 115248773 A CN115248773 A CN 115248773A CN 202110470378 A CN202110470378 A CN 202110470378A CN 115248773 A CN115248773 A CN 115248773A
Authority
CN
China
Prior art keywords
data
change rate
data volume
version
volume change
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110470378.8A
Other languages
Chinese (zh)
Inventor
张凤梅
张彦锋
赵风萍
郭秉须
何淑辉
徐心照
韦力
姜河义
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Navinfo Co Ltd
Original Assignee
Navinfo Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Navinfo Co Ltd filed Critical Navinfo Co Ltd
Priority to CN202110470378.8A priority Critical patent/CN115248773A/en
Publication of CN115248773A publication Critical patent/CN115248773A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3668Software testing
    • G06F11/3672Test management
    • G06F11/368Test management for test version control, e.g. updating test cases to a new software version
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3668Software testing
    • G06F11/3672Test management
    • G06F11/3692Test management for test results analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/29Geographical information databases

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Databases & Information Systems (AREA)
  • Remote Sensing (AREA)
  • Data Mining & Analysis (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The utility model discloses a map data monitoring and verification method and a component type map data service cloud platform, wherein the method mainly comprises the steps of obtaining the data volume of data elements in a version database to be tested for updating an electronic map; determining a corresponding data volume change rate according to the data volume, obtaining a change rate set corresponding to the data volume change rate, and performing normal distribution analysis on the change rate set to obtain a first change rate set conforming to normal distribution and a second change rate set not conforming to normal distribution in the change rate set; outputting the data volume change rate larger than a preset threshold value as a first abnormal value in normal distribution corresponding to the first change rate set; and carrying out extreme value analysis on the second change rate set, and outputting the data volume change rate exceeding the preset threshold range as a second abnormal value. Therefore, by implementing the technical scheme disclosed by the invention, the influence of the data base on abnormal value detection can be avoided, data redundancy can be avoided, and the data analysis workload is reduced.

Description

Map data monitoring and verifying method and component type map data service cloud platform
Technical Field
The disclosure relates to the field of navigation map data analysis, in particular to a map data monitoring and verification method and a component type map data service cloud platform.
Background
At present, the data volume of each version of map is huge, the statistics of data elements exceeds 2000 items, and 7 ten thousand records are counted in each version according to 33 provinces in the country at present.
When the data of each large version map is updated, the data of the map of the previous version needs to be combined, all data contents and data elements need to be compared in a difference mode, and the data quality is guaranteed by positioning abnormal changes. The traditional data version difference analysis method adopts a mode of comparing the change percentages of two versions of map data, and when the percentage change amplitude exceeds a limited range, difference elements are extracted for further analysis.
When the data version analysis is carried out by using the method, the data elements are different, the situation difference is large, and when the data base number of some elements is small and the variation is small, the percentage is large; when the data base number of some elements is large and a large amount of change exists, the percentage is small; under the condition, the abnormal analysis data extracted from each version of map has a lot of redundant data, so that the workload of analyzing the data is increased; meanwhile, the abnormal change of part of element data with large base numbers is omitted, so that control has a vulnerability.
Disclosure of Invention
The disclosure mainly provides a map data monitoring and verification method and system and a component type map data service cloud platform, and aims to solve the problem that redundant data greatly increases data analysis workload and data base number when a map version is updated, and further influences abnormal value detection.
To this end, according to a first aspect of the present disclosure, a map data monitoring verification method is disclosed, the method comprising the processes of:
acquiring the data volume of target data elements in a database of the version to be updated of the electronic map;
calculating corresponding data volume change rate according to the data volume of the same target data element of the updated version to be detected and the compared version, and forming a change rate set corresponding to the data volume change rate by using the data volume change rate of the same target data element between different versions in a target administrative division;
performing normal distribution analysis on the change rate set to obtain a first change rate set which accords with normal distribution and a second change rate set which does not accord with the normal distribution in the change rate set;
in normal distribution corresponding to the first change rate set, outputting the change rate of the data volume which is greater than a preset threshold value in the first change rate set as a first abnormal value;
performing extreme value analysis on the data volume change rate in the second change rate set, and outputting the data volume change rate exceeding a preset threshold range in the second change rate set as a second abnormal value;
and outputting an abnormality monitoring result and an analysis report according to the first abnormal value and/or the second abnormal value.
As an optional implementation manner, the above process of determining a corresponding data volume change rate according to the data volume and obtaining a change rate set corresponding to the data volume change rate further includes:
calculating the data volume change rate between the data volumes of the corresponding data elements between every two adjacent versions from the beginning of updating the version to be detected, and obtaining an inter-version change rate set of the data volume change rate between the versions; and/or the presence of a gas in the gas,
and calculating the data volume change rate between the current data volume corresponding to each data element in the updated version to be tested under each area unit and the corresponding data volume in the previous map version to obtain an inter-area change rate set of the data volume change rates among the area units.
As an optional implementation manner, the above process of performing extremum analysis on the data volume change rates in the second change rate set further includes:
acquiring a historical change extreme value corresponding to the target data element, adding or subtracting a corresponding historical change average value according to the historical change extreme value corresponding to each data volume change rate in the second change rate set, and setting the preset threshold range;
and outputting the data volume change rate exceeding the preset threshold range in the second change rate set as the second abnormal value.
As an optional implementation manner, the obtaining of the data volume of the data elements in the database of the version to be tested for updating the electronic map includes:
inputting the data volume corresponding to each data element into the electronic map updating version database to be tested, and calculating the data volume change rate corresponding to the data element;
if the data volume change rate is vacant, completing the missing data volume change rate according to the data volume change rate of the corresponding data element in the updated version to be tested;
and if the data volume change rate has errors, correcting the data volume change rate with errors according to the historical average value of the data volume change rate.
As an optional implementation manner, the map data monitoring and verifying method may further include:
classifying the data volume change rate of the data elements in at least one historical version according to the category of the data elements, calculating a correlation coefficient between every two data volume change rates in each category, and when the correlation coefficient is larger than a strong correlation threshold value, forming a data element group by two data elements corresponding to the correlation coefficient; according to the IDs of the two data elements in the data element group, extracting the data volume change rate of the target data element corresponding to the data element ID in each map version in the database of the version to be updated of the electronic map, and generating a correlation change rate set;
according to the correlation change rate set, performing fitting calculation on the data volume change rate of the strongly correlated target data elements, and determining a confidence interval according to the result of the fitting calculation and a preset confidence coefficient;
and judging whether the data volume change rate of the target data element extracted from the updated version to be tested exceeds the confidence interval or not, and outputting the data volume change rate exceeding the confidence interval as a third abnormal value.
As an optional implementation manner, the map data monitoring and verifying method may further include:
performing anomaly analysis according to the first anomaly value, the second anomaly value and/or the third anomaly value to obtain an anomaly result and an analysis report, and updating and verifying the version database to be detected of the electronic map; further comprising:
performing anomaly analysis on the first and/or second outliers comprises: if the cause of the first abnormal value and/or the second abnormal value is a data problem, the data quantity of the data element corresponding to the first abnormal value and/or the second abnormal value is counted again, the first abnormal value and/or the second abnormal value in the version database to be tested of the electronic map is updated through the calculated data quantity change rate, and the first abnormal value and/or the second abnormal value and the corresponding abnormal cause analysis report are output, wherein the data problem comprises the problem of the abnormal data quantity change rate caused by the abnormal workflow; if the cause of the first abnormal value and/or the second abnormal value is a non-data problem, updating the first abnormal value and/or the second abnormal value in the version database to be updated of the electronic map according to the historical record average value corresponding to the first abnormal value and/or the second abnormal value, and outputting the first abnormal value and/or the second abnormal value and the corresponding abnormal cause analysis report, wherein the non-data problem comprises a problem of abnormal data volume change rate caused by workflow change;
performing anomaly analysis on the third anomaly comprises: if the cause of the third anomaly is a data problem, the data quantity of the data element corresponding to the third anomaly is counted again, the electronic map is updated according to the calculated data quantity change rate to update the third anomaly in the version database to be tested, and the third anomaly and the corresponding abnormal cause analysis report are output; and if the reason of the third abnormal value is a non-data problem, updating the third abnormal value in the version database to be tested by using the historical record average value corresponding to the third abnormal value, and outputting the third abnormal value and a corresponding abnormal reason analysis report.
According to a second aspect of the present disclosure, there is also disclosed a map data monitoring and verification system, comprising:
the data import module is used for acquiring the data volume of the target data elements in the electronic map updating version database to be tested;
the difference statistical module is used for calculating corresponding data volume change rates according to the data volumes of the same target data element of the updated version to be detected and the compared version, and forming a change rate set corresponding to the data volume change rates by using the data volume change rates of the same target data element among different versions in a target administrative division;
the first analysis module is used for carrying out normal distribution analysis on the change rate set to obtain a first change rate set which accords with normal distribution and a second change rate set which does not accord with the normal distribution in the change rate set;
the abnormal monitoring module is used for taking the change rate of the data volume which is greater than a preset threshold value in the first change rate set as a first abnormal value in normal distribution corresponding to the first change rate set; the extreme value analysis is carried out on the data volume change rate in the second change rate set, and the data volume change rate which exceeds a preset threshold range in the second change rate set is used as a second abnormal value;
and the output verification module is used for outputting an abnormal monitoring result and an analysis report according to the first abnormal value and/or the second abnormal value, and updating and verifying the database of the to-be-detected version of the electronic map.
According to a third aspect of the present disclosure, there is also disclosed a component-based map data service cloud platform, comprising:
the user interaction component is used for providing a mode access platform comprising Web, API or SDK;
the unified authentication component is used for identity authentication, functional authority management, access control and/or dynamic configuration;
the database component is used for storing electronic map data, charging key point coordinates, high-speed line units which are pre-divided into high-speed lines according to the charging key point coordinates and toll corresponding to the high-speed line units;
the product editing assembly is configured with an independent data editing space and used for generating a customized data product based on editing, converting, fusing, checking, differentiating, batch processing of data in the database assembly and/or combination of imported reference data;
the product release component is used for performing version management, authority management, data increment release, real-time push, service online, flow control and service offline of data and services;
the service customizing component is used for providing customizable map data products and services according to users and requirements, and comprises a charging unit which is used for matching a passing path with the high-speed local line unit, extracting each high-speed local line unit corresponding to the passing path and outputting the charging unit;
and the statistical analysis component is used for executing the map data monitoring and verifying method disclosed by any scheme and is used for carrying out usage statistics, hotspot data analysis, operation and maintenance monitoring and early warning.
As an optional implementation manner, the unified authentication component further includes:
the load balancer is used for monitoring a network port and dynamically configuring network resources;
the API gateway module is used for carrying out API routing and access control;
and the authentication module is used for carrying out identity authentication and functional authority management on the user.
According to a fourth aspect of the present disclosure, there is also disclosed a computer apparatus, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor;
wherein the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to cause the at least one processor to perform the map data monitoring and verification method disclosed in any one of the preceding aspects.
Compared with the prior art, the technical scheme of this disclosure can reach the beneficial effect include:
through implementing this disclosed technical scheme, not only can avoid the influence that the data cardinal number caused to unusual value detection, can also avoid data redundancy, reduce data analysis work load.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present disclosure, and for those skilled in the art, other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a schematic flowchart of a map data monitoring and verifying method according to an embodiment of the present disclosure;
FIG. 2 is a schematic diagram of an example provided by an embodiment of the present disclosure;
FIG. 3 is a diagram illustrating a standard deviation of inter-domain rate of change sets according to an embodiment of the disclosure;
FIG. 4 is a diagram illustrating a standard deviation of an inter-version rate of change set according to an embodiment of the disclosure;
fig. 5 is a schematic flowchart of a map data monitoring and verifying method according to a second embodiment of the disclosure;
fig. 6 is a schematic diagram of a confidence interval in a map data monitoring and verifying method provided in the second embodiment of the present disclosure;
FIG. 7 is a schematic diagram illustrating a map data monitoring and verification system according to an embodiment of the present disclosure;
fig. 8 is a schematic composition diagram of a component map data service cloud disclosed in an embodiment of the present disclosure.
With the foregoing drawings in mind, certain embodiments of the disclosure have been shown and described in more detail below. The drawings and written description are not intended to limit the scope of the disclosed concepts in any way, but rather to illustrate the disclosed concepts to those skilled in the art by reference to specific embodiments.
Detailed Description
Preferred embodiments of the present disclosure are described in detail below with reference to the accompanying drawings so that the advantages and features of the present disclosure can be more readily understood by those skilled in the art, and thus the scope of the present disclosure is more clearly and clearly defined.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of another like element in a process, method, article, or apparatus that comprises the element.
The map data monitoring and verifying method provided by the disclosure is applicable to the following scenes: according to the difference of the update elements of the map data product line and the difference of the update periods of the products, the technical method can be used for the abnormal analysis of the provincial data elements, the abnormal analysis of national element data, the abnormal analysis of single element update and the abnormal analysis during the update of cloud service type products. Aiming at different service objects, the same technical method is adopted to extract different model parameters according to the updating change condition, so that different map element updating scenes can be supported, and the method has universality.
When each map version is updated, the prior art needs to combine the data elements of the previous map version and obtain the difference data elements by comparing the data change percentage between the two map versions, and when the data base number of some elements is small and there is little change, the percentage is larger; when the data cardinality of some elements is large and a large amount of change exists, the percentage of the elements is small; under the condition, redundant data of abnormal analysis data extracted from each version exist, and the workload of analyzing the data is increased; meanwhile, the abnormal change of part of element data with large base numbers is omitted, so that the detection has a vulnerability.
Method embodiment
In view of the above technical problems, the map data monitoring and verification method provided by the present disclosure can not only avoid the influence of the data base on the detection of abnormal values, but also avoid data redundancy, and reduce the workload of data analysis.
The map data monitoring and verifying method comprises the following steps:
s1: acquiring the data volume of target data elements in a database of the version to be updated of the electronic map;
s2: calculating corresponding data volume change rate according to the data volume of the same target data element of the updated version to be detected and the compared version, and forming a change rate set corresponding to the data volume change rate by using the data volume change rate of the same target data element between different versions in the target administrative division;
s3: performing normal distribution analysis on the change rate set to obtain a first change rate set which accords with normal distribution and a second change rate set which does not accord with the normal distribution in the change rate set;
s4: in normal distribution corresponding to the first change rate set, outputting the data volume change rate larger than a preset threshold value in the first change rate set as a first abnormal value;
s5: carrying out extreme value analysis on the data volume change rate in the second change rate set, and outputting the data volume change rate which exceeds the preset threshold range in the second change rate set as a second abnormal value;
s6: and outputting an abnormality monitoring result and an analysis report according to the first abnormal value and/or the second abnormal value.
According to the method, the data volume change rate is taken as a reference basis, normal distribution analysis and correlation analysis are simultaneously carried out on data elements in a version database to be tested for updating an electronic map, and extreme value analysis is carried out on the data elements which do not conform to the normal distribution analysis; when abnormal values are detected in the three different dimensions, multi-dimensional marking is carried out, and whether the abnormality in the different dimensions is reasonable or not is analyzed subsequently. The map data monitoring and verifying method mainly comprises the steps of obtaining the data volume of data elements in a version database to be tested for updating an electronic map; and counting the data in the data elements in the map database to obtain the data volume of the data elements, and inputting the data volume of the data elements into the electronic map database to be updated. Determining a corresponding data volume change rate according to the data volume, obtaining a change rate set corresponding to the data volume change rate, performing normal distribution analysis on the change rate set to obtain a first change rate set which accords with normal distribution and a second change rate set which does not accord with the normal distribution in the change rate set, and outputting the data volume change rate in the first change rate set which is larger than a preset threshold value as a first abnormal value in the normal distribution corresponding to the first change rate set; and calculating the data volume change rate corresponding to the data volume in the database of the version to be updated of the electronic map, then completing the data volume change rate missing in the database, and correcting the wrong data volume change rate. Carrying out extreme value analysis on the data volume change rate of the second change rate set which does not conform to normal distribution analysis, and outputting the data volume change rate in the second change rate set which exceeds a preset threshold range as a second abnormal value; and obtaining a change rate set by using the data volume change rate, and obtaining a first abnormal value and a second abnormal value through normal distribution analysis and extreme value analysis. And performing correlation analysis on the data volume change rate of the data elements in the version database to be updated of the electronic map, obtaining a third abnormal value by solving a confidence interval, and performing abnormal analysis on the abnormal values obtained under the three different dimensions to obtain an abnormal result and an analysis report.
The following provides a detailed description of the technical solutions of the present disclosure and how to solve the above technical problems with further embodiments. Several further embodiments may be combined with each other below, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present invention will be described below with reference to the accompanying drawings.
Example one
Referring to fig. 1, which is a schematic flowchart illustrating an embodiment of a method for monitoring and verifying map data according to the present disclosure, the method may include the following steps:
step S101, acquiring the data volume of the target data elements in the electronic map updating version database to be tested.
Step S102, calculating corresponding data volume change rates according to the data volumes of the same target data element of the updated version to be detected and the compared version, and forming a change rate set corresponding to the data volume change rates by using the data volume change rates of the same target data element between different versions in the target administrative division.
Step S103, carrying out normal distribution analysis on the change rate set to obtain a first change rate set which accords with normal distribution and a second change rate set which does not accord with the normal distribution in the change rate set.
Step S104, in normal distribution corresponding to the first change rate set, the change rate of the data volume larger than a preset threshold value in the first change rate set is used as a first abnormal value to be input.
In this embodiment, the same data element, under the same region, has only one recorded value in the same map version. In different regions, the data elements have large differences, the analysis according to the recorded values has large errors, and the change rate of data updating shows a stable rule on the periodic change of the history, so the data volume change rate of the same data element between versions is selected as a standard variable for analysis. And the data volume change rate is obtained by dividing the difference value of the data volume of the same data element in the same region in the current map version and the data volume of the same data element in the previous map version by the data volume of the previous map version and multiplying the data volume by percentage.
It should be noted that the data refers to actual map data, and includes all map data elements and related attribute information; the data volume is used for counting map data, so that the quantity of data attributes of data elements in each map in the map data is conveniently known; the data amount change rate represents a change in the number of attributes or data elements in the map data between adjacent versions. As the data element for "gate" recorded in fig. 2, in the two map versions, the statistics of attributes of the driving gate are 12655 (currently updated version to be measured) and 12505 (previous history version), respectively, and the change rate of the driving gate can be calculated to be (12655-12505)/12505 × 100% according to the data amount.
In an optional embodiment, the process of determining the corresponding data volume change rate according to the data volume and obtaining the change rate set corresponding to the data volume change rate further includes, starting from updating the version to be tested, calculating the data volume change rate between the data volumes of the corresponding data elements between every two adjacent versions, and obtaining the inter-version change rate set of the data volume change rate between the versions; and/or calculating the data volume change rate between the current data volume corresponding to each data element in the updated version to be tested under each area unit and the corresponding data volume in the previous map version to obtain an inter-area change rate set of the data volume change rates among the area units.
In this optional embodiment, the area units are represented by provinces according to the regional conditions in the target range, that is, the inter-area rate set is an inter-province rate set. The data volume change rate describing data updating is used as the change rate set, so that the change rule of data in historical periodic change can be mastered, and the abnormal problem can be positioned conveniently.
In an alternative embodiment, the region unit is represented by a province, and the normal distribution analysis is performed on the inter-province change rate set of the inter-province data volume change rate corresponding to each data element and the inter-version change rate set of the corresponding inter-version data volume change rate respectively. After a first change rate set corresponding to the change rate set conforming to normal distribution is subjected to normal distribution standardization, calculating the standard deviation of the inter-provincial change rate set in the first change rate set, outputting the data volume change rate which is more than three times of the standard deviation in the inter-provincial change rate set as a first abnormal value, and marking the first abnormal value detected by the dimensionality as 1_1; and simultaneously calculating the standard deviation of the inter-version change rate set in the first change rate set, outputting the data volume change rate of which the standard deviation is more than three times of the standard deviation in the inter-version change rate set as a first abnormal value, and marking the first abnormal value detected by the dimension as 1_2.
Optionally, the preset threshold may be three times of standard deviation respectively corresponding to different rate sets.
In an alternative embodiment, the change rate of the data amount in the data element in the same data element, between different provinces, and in the two previous and next map versions can be used as the change rate set of the data amount change rate between the provinces, as shown in fig. 3; in different historical map versions, the change rate of the data quantity in the data elements in the two previous map versions is taken as the change rate set of the data quantity change rate between the two versions, as shown in fig. 4, the two change rate sets are respectively subjected to normal distribution drawing, and the data elements meeting the normal distribution need to be verified and confirmed manually based on the updating condition of the data in the same data element. For data elements meeting normal distribution, data updating in the real world is relatively regular, such as data elements of roads, backgrounds and related road types, speed limits, traffic limits, vehicle information, direction boards and the like, further data analysis and parameter selection are required for the data elements, a triple standard deviation can be used as an index for judging whether the data elements exceed a preset threshold, and the data volume change rate exceeding the index is output as a first abnormal value.
It should be noted that, based on the update of data between different versions in the same data element, the data element corresponding to the change rate set satisfying the normal distribution can be confirmed by a computer program or manual verification. Since the update characteristic of the data element is that data is being added in the forward direction, the normal distribution of the change rate set corresponding to the data amount change rate of the same data element tends to extend toward one end infinitely.
In an optional embodiment, the rule setting module performs abnormal value detection on the data volume change rate in the map version to be detected according to the importance degree of the data elements. For data elements with high importance and large influence range, an S-level rule module is arranged, for example, when the change rate of the data volume in the data elements in the map version to be tested is less than three times of standard deviation of the data elements, and the change rate of the data volume in the same data element in the historical map version is also more than three times of standard deviation of the data elements, an S1_1 rule module is arranged; when the data reduction amount in the same data element in the historical map version exceeds three times of standard deviation of a data amount change rate record value in the historical map version, an S1_2 rule module is set; when the data quantity of the data elements in the map version to be detected in different provinces is reduced and the data quantity of any province is not increased, an S2_1 rule module is arranged; the S2_2 rule module is set when there is a decrease in the amount of data in a certain kind of data element. Setting an A-level rule module for other data elements, for example, when the data elements in the historical map version are not changed and the data volume change rate of the data elements in the map version to be detected is changed, setting an A1_1 rule module; when the data volume change rate of the data elements in the historical map version is changed and the change range exceeds three times of standard deviation of the data volume change rate of the data elements in the historical map version, an A1_2 rule module is arranged; when the data volume of the data elements in more than two historical map versions in the historical map version is increased and not reduced, and the data volume of the data elements in the map version to be detected is reduced, an A2_1 rule module is arranged; and when the data volume change rate of the data elements in the historical map version is changed and the change range of the data volume change rate of the data elements in the map version to be detected exceeds the extreme point of the historical change, setting an A2_2 rule module.
Step S105, performing extremum analysis on the data volume change rates in the second change rate set, and outputting the data volume change rates exceeding the preset threshold range in the second change rate set as second abnormal values.
And step S106, outputting an abnormality monitoring result and an analysis report according to the first abnormal value and/or the second abnormal value.
In an optional embodiment, the process of performing extremum analysis on the data volume change rates in the second change rate set that does not conform to the normal distribution analysis includes setting a preset threshold range according to a historical change extremum corresponding to each data volume change rate in the second change rate set and a historical change average corresponding to the data volume change rate, and outputting the data volume change rate that exceeds the preset threshold range in the second change rate set as a second abnormal value.
In the present embodiment, in the data amount change rates in the same second change rate set, the sum of the history change maximum value corresponding to the data amount change rate in the second change rate set and the history change average value thereof is set as the upper limit of the preset threshold range, and the difference between the history change minimum value and the history change average value thereof is set as the lower limit of the preset threshold range. And in the map version to be detected, outputting the data volume change rate in the corresponding second change rate set exceeding the preset threshold range as a second abnormal value. The abnormal problem of the second change rate set with unstable change is convenient to locate by defining the abnormality of the data elements in a mode of adopting a history change extreme value and a history change average value.
In an alternative embodiment, there is instability with respect to real-world variations of certain data elements, i.e., not satisfying a normal distribution, such as a particular kind of poi, agent store, etc.; according to the method, the history change extreme points of the data volume change rates of the data elements are calculated, the corresponding history change average values are added and subtracted to be used as a preset threshold range, the data volume change rate exceeding the preset threshold range in the corresponding data elements in the map version to be detected is output as a second abnormal value, the second abnormal value detected in the dimension is marked as 2_1, and multi-dimensional abnormal analysis can be conveniently carried out on the abnormal values in the follow-up process.
As an optional implementation manner, the map data monitoring and verifying method may further include:
classifying the data volume change rate of the data elements in at least one historical version according to the category of the data elements, calculating a correlation coefficient between every two data volume change rates in each category, and when the correlation coefficient is larger than a strong correlation threshold value, forming a data element group by two data elements corresponding to the correlation coefficient; according to the IDs of the two data elements in the data element group, extracting the data volume change rate of the target data element corresponding to the data element ID in each map version in the database of the version to be updated of the electronic map, and generating a correlation change rate set;
according to the correlation change rate set, performing fitting calculation on the data volume change rate of the strongly correlated target data elements, and determining a confidence interval according to the result of the fitting calculation and a preset confidence coefficient;
and judging whether the data volume change rate of the target data element extracted from the updated version to be tested exceeds the confidence interval or not, and outputting the data volume change rate exceeding the confidence interval as a third abnormal value.
As an optional implementation manner, the map data monitoring and verifying method may further include: and performing anomaly analysis according to the first anomaly value, the second anomaly value and/or the third anomaly value to obtain an anomaly result and an analysis report, and updating and verifying the version database to be detected of the electronic map.
The above embodiments are explained in the following developments:
example two
Referring to fig. 5, which shows a flowchart of a map data monitoring and verification method provided in the second embodiment of the present disclosure, the method may further include the following steps.
Step S500, acquiring the data volume of the data elements in the electronic map updating version database to be tested, including inputting the data volume of the data elements of each map version into the electronic map updating version database to be tested, and calculating and obtaining the data volume change rate of the data volume; if the data volume change rate of the data elements in the historical map version is vacant, completing the missing data volume change rate according to the data volume change rate of the corresponding data elements in the updated version to be detected; and if the data volume change rate of the data elements in the historical map version has errors, correcting the data volume change rate with errors according to the historical average value of the data volume change rate.
In this embodiment, the data preprocessing operation maintains the integrity of the data, does not cull the abnormal data amount change rate, and replaces or averages the noise data with the correct value as much as possible.
In an optional embodiment, as for the operation of data preprocessing, firstly, a data volume warehousing operation is performed, and the original data volume in each map version is subjected to data volume warehousing processing to obtain a data volume change rate; then, the missing data volume change rate is supplemented, because the statistics of each data element in the historical map version are not identical, the data volume change rate in the data elements in the historical map version needs to be supplemented, and the data volume change rate needing to be deleted is not processed; and finally, cleaning historical abnormal values of the data volume change rate, cleaning the data volume change rate with problems appearing in the historical map version, and replacing the data volume change rate with a correct value or smoothing the average value to reduce the influence on the average value of the data volume change rate in the data elements.
In an optional embodiment, the process of completing the data volume change rate of the historical version according to the data volume change rate in the map version to be detected further includes, based on the map version to be detected, comparing the data volume change rate of all the data elements in each historical map version with the data volume change rate of the corresponding divided data elements in the map version to be detected, completing only the data volume change rate lacking in the data elements in each historical map version, and not deleting the redundant data volume change rate.
In this optional embodiment, the process of completing the historical version data according to the data in the map version to be tested may further include, based on the map version to be tested, supplementing a data volume change rate in the data elements in the previous map version, and then, based on the completed previous map version, supplementing a data volume change rate in the data elements in the previous map version. For example, there are three existing map versions, the third version is a map to be updated, the first version and the second version are history map versions, the second version supplements the data volume change rate in the data elements thereof according to the third version, and the first version supplements the data volume change rate in the data elements thereof according to the supplemented second version.
Step 501, performing normal distribution analysis on the data volume change rate in the first change rate set.
Step 502, performing extremum analysis on the data volume change rates in the second change rate set.
In this embodiment, please refer to the relevant description in step S101 to step S106 in the method shown in fig. 1 for further implementation processes and technical principles of step S501 to step S502, which are not described herein again.
Step S503, according to the correlation analysis, extracting the correlation change rate set further includes extracting the data quantity change rate of the data elements corresponding to the data element ID in each map version in the version database to be updated in the electronic map as the correlation change rate set according to the data element ID of each data element in the predetermined data element group with correlation; fitting calculation is carried out on the data volume change rate in the correlation change rate set, and a confidence interval is determined according to the result of the fitting calculation and the preset confidence level; and outputting the data volume change rate exceeding the confidence interval in the updated version to be tested as a third abnormal value.
In an optional embodiment, the data volume change rates of the data elements in at least one historical version are classified according to the categories of the data elements, so as to obtain a third change rate set containing the data volume change rates of different categories; and calculating a correlation coefficient between every two data volume change rates in each category in the third change rate set, wherein when the correlation coefficient is greater than a correlation threshold value, two data elements corresponding to the correlation coefficient form a data element group.
In the present embodiment, a data element having a strong correlation is obtained from the data amount change rate of the data element in at least one history version, a corresponding data element is extracted from each map version using the attribute ID of the data element having a strong correlation, a straight line is fitted to the data element using the data amount change rate of the data element, a confidence interval is obtained, and an abnormal value is detected. Firstly, according to the types of the data elements, obtaining a third change rate set of the data volume change rates of different types; and calculating a correlation coefficient between every two data volume change rates in each category in the third change rate set, and obtaining a data element group with correlation when the correlation coefficient is greater than a correlation threshold value. And taking two data elements with correlation in the third change rate set as a data element group, extracting the data quantity change rate of the corresponding data elements in the map version to be tested and the historical map version according to the IDs of the two data elements in the data element group to obtain a correlation change rate set, performing straight line fitting calculation on the correlation change rate set, determining a confidence interval according to the result of the fitting calculation and the preset confidence level, and outputting the data quantity change rate exceeding the confidence interval in the updated version to be tested as a third abnormal value.
In the present embodiment, the categories of the data elements mainly include map elements and attribute data, such as road mileage of each level, function level, number of intersections, construction road mileage, number of index points of rice villages, lake area, green space area, and the like. Classifying the data elements according to the categories of the data elements, and solving the correlation between the two data elements according to the same category more accurately; and extracting a correlation change rate set corresponding to the data element group according to the data element ID, and manually verifying and judging the output data element group to exclude the data element group which does not actually have correlation, so that the detection of the abnormal value is more accurate.
In an optional embodiment, the process of outputting the data volume change rate exceeding the confidence interval as the third anomaly further includes sequentially judging whether the data volume change rate corresponding to each data element group in the map version to be tested is within the confidence interval, outputting the data volume change rate not within the confidence interval as the third anomaly, and marking the third anomaly detected in the dimension as 3_1, so as to facilitate subsequent multi-dimensional anomaly analysis according to the abnormal value.
In an optional embodiment, firstly, data elements in a version database to be detected of the electronic map are updated and classified into a plurality of large categories according to roads, backgrounds and poi; and solving correlation coefficients for the data quantity change rates of the data elements in each category pairwise, and outputting the two corresponding data elements with strong correlation. For example, there are 600 groups of data element groups for which the correlation coefficient is greater than the correlation threshold. Carrying out manual verification and judgment on the output data element group, and excluding the combination which does not have strong correlation actually; the data elements with strong correlation relationship have strong correlation in the map data, such as intersection-intersection limit, intersection-traffic light, and poi-background surface of functional type, a fitting straight line is calculated according to the data quantity change rate in two data elements in the data element group, namely y = ax + b, and the confidence interval is confirmed by adjusting the variable values a and b. And when the data volume change rate in the data elements in the map version to be tested exceeds the confidence interval, reporting an abnormal value. According to the feature of the data change in the map data element, the change interval of the partial data element is not negative, and as shown in fig. 6, the actual confidence interval is a light gray area.
In this alternative embodiment, a confidence interval is determined according to the corresponding data in the data element groups in the historical map version, the corresponding data amount change rate in each data element group in the map version to be tested is substituted into the confidence interval, and the data amount change rate which is not in the confidence interval is output as an abnormal value.
Optionally, the present disclosure uses the pearson correlation coefficient to find a confidence interval with a preset confidence level of 95%, and the correlation coefficient threshold value is a certain further value greater than 80%.
In an optional embodiment, after the data volume of each map version is recorded into the electronic map to update the version database to be tested for completion and replacement, the data volume change rate is subjected to differential operation, which includes calculating a difference and a percentage, and then analysis is performed based on differential data. For the data elements for establishing the correlation analysis, the data quantity change rate of the historical map versions of the data elements a and the data elements b is firstly acquired, the data quantity change rate in the acquired data elements a is set as x-axis data, the data quantity change rate in the acquired data elements b is set as y-axis data, then correlation coefficients including but not limited to a Pearson correlation coefficient, a Spanish rank line correlation coefficient and a Kendall correlation coefficient of the data quantity change rate in the data elements a and the data quantity change rate in the data elements b are respectively calculated, under the premise that the data elements a and the data elements b have strong correlation, straight lines are fitted according to points corresponding to the x-axis data and the y-axis data respectively, a confidence interval is calculated on the basis of triple standard deviation corresponding to the data elements a and the data elements b, and then whether the data quantity change rate in the data elements a and the data quantity change rate in the data elements b in the map version to be tested are in the confidence interval is sequentially judged, and the data change rate which is not in the data elements a and the data elements b are not in the confidence interval is taken as a third constant output value.
In the above embodiment, the first data of the x-axis and the first data of the y-axis correspond to a point, the second data of the x-axis and the second data of the y-axis correspond to a point, and so on, the data of the x-axis and the data of the y-axis correspond to a plurality of points, and the points are subjected to straight line fitting.
EXAMPLE III
The embodiment discloses a map data monitoring and verification method, which comprises the steps of obtaining the data volume of data elements in a database of a version to be tested for updating an electronic map; determining a corresponding data volume change rate according to the data volume, obtaining a change rate set corresponding to the data volume change rate, and performing normal distribution analysis on the change rate set to obtain a first change rate set conforming to normal distribution and a second change rate set not conforming to normal distribution in the change rate set; in normal distribution corresponding to the first change rate set, taking the data volume change rate which is greater than a preset threshold value in the first change rate set as a first abnormal value, carrying out extreme value analysis on the data volume change rate in the second change rate set, and taking the data volume change rate which is greater than the preset threshold value range in the second change rate set as a second abnormal value; carrying out anomaly analysis on the first abnormal value and/or the second abnormal value, if the cause of the first abnormal value and/or the second abnormal value is a data problem, carrying out statistics on the data quantity of the data elements corresponding to the first abnormal value and/or the second abnormal value again, updating the first abnormal value and/or the second abnormal value in the version database to be tested by updating the electronic map according to the calculated data quantity change rate, and outputting the analysis report of the first abnormal value and/or the second abnormal value and the corresponding anomaly cause, wherein the data problem comprises the problem of abnormal data quantity change rate caused by the abnormal workflow; if the reason for generating the first abnormal value and/or the second abnormal value is a non-data problem, updating the first abnormal value and/or the second abnormal value in the version database to be tested by using the historical record average value corresponding to the first abnormal value and/or the second abnormal value, and outputting an analysis report of the first abnormal value and/or the second abnormal value and the corresponding abnormal reason, wherein the non-data problem comprises a problem of abnormal data volume change rate caused by workflow change.
In this embodiment, the workflow change includes a data acquisition planning scheme, a data manufacturing process change, and the like, where sudden increase and sudden decrease of an abnormal value may have a relationship with the data acquisition planning scheme, the data manufacturing process change, and the like, and is a normal data change and is a non-data problem, and if the abnormal value is the non-data problem, the abnormal value is generally replaced by a historical record average value of a data volume change rate, so as to update the version database to be updated of the electronic map; the abnormal workflow comprises misoperation, wherein the change of an abnormal value does not accord with factors such as a planning scheme and a manufacturing process change, but is caused by misoperation (man-made or program) of a certain link, and the like, which is a data problem, if the abnormal value is a data problem, data elements of the actual map of the version to be detected are corrected, and the corrected statistical value and the data volume change rate obtained through calculation are normal values and are updated into the database of the version to be detected of the electronic map.
In an optional embodiment, according to a data element ID of each data element in a predetermined data element group with correlation, extracting a data amount change rate of the data element corresponding to the data element ID in each map version in an electronic map updating version database to be tested to serve as a correlation change rate set, performing fitting calculation on the data amount change rate in the correlation change rate set, determining a confidence interval according to a result of the fitting calculation and a preset confidence level, and taking the data amount change rate exceeding the confidence interval in the updating version to be tested as a third abnormal value; performing anomaly analysis on the third anomaly value, if the cause of the third anomaly value is a data problem, re-counting the data quantity of the data elements corresponding to the third anomaly value, updating the third anomaly value in the version database to be tested by the electronic map according to the calculated data quantity change rate, and outputting the third anomaly value and a corresponding anomaly cause analysis report; and if the cause of the third abnormal value is a non-data problem, updating the third abnormal value in the version database to be tested by using the historical record average value corresponding to the third abnormal value, and outputting the third abnormal value and a corresponding abnormal cause analysis report.
In an optional embodiment, the map data monitoring and verifying method further includes performing correlation analysis on data elements in the version database to be tested for updating the electronic map to obtain a third anomaly value. Classifying the data elements in the version database to be tested updated by the electronic map according to the element meanings of the data elements to obtain a third change rate set of the data volume change rates of different classes; and calculating a correlation coefficient between every two data volume change rates in each category in a third change rate set, taking every two data elements with correlation in the third change rate set as a data element group when the correlation coefficient is greater than a correlation threshold value, extracting the data volume change rates of the corresponding data elements in the map version to be tested and the historical map version according to the IDs of the two data elements in the data element group to obtain the correlation change rate set, performing straight line fitting calculation on the correlation change rate set, determining a confidence interval according to the result of the fitting calculation and the preset confidence level, and outputting the data volume change rate exceeding the confidence interval in the updated version to be tested as a third abnormal value.
In an optional embodiment, after analyzing the data volume change rate in the data elements in the version database to be updated of the electronic map, obtaining a first abnormal value, a second abnormal value and/or a third abnormal value, respectively and manually analyzing the first abnormal value, the second abnormal value and/or the third abnormal value, if the cause of the abnormality is determined to be a data problem, updating the data volume in the version database to be updated of the electronic map and the corresponding data volume change rate, counting the data elements corresponding to the abnormal values again to obtain the data volume, then calculating to obtain a new data volume change rate as a normal value, and replacing the electronic map with the normal value to update the corresponding abnormal value in the version database to be updated; and if the cause of the abnormality is determined to be a non-data problem, replacing the historical record average value of the data volume change rate of the data element corresponding to the abnormal value with the historical record average value of the data volume change rate of the data element corresponding to the abnormal value to update the corresponding abnormal value in the version database to be tested. For example, the first abnormal value is manually analyzed, and if the reason causing the abnormality of the first abnormal value is determined as the data problem, the corresponding first abnormal value in the database is updated by the normal value corresponding to the first abnormal value; and if the reason causing the first abnormal value to be abnormal is determined as a non-data problem, updating the corresponding first abnormal value in the database by using the historical record average value corresponding to the first abnormal value. Manually analyzing the second abnormal value, and if the reason causing the second abnormal value to be abnormal is determined as a data problem, updating the corresponding second abnormal value in the database by using the normal value corresponding to the second abnormal value; and if the reason causing the second abnormal value to be abnormal is determined as a non-data problem, updating the corresponding second abnormal value in the database by using the historical record average value corresponding to the second abnormal value. Manually analyzing the third abnormal value, and if the reason causing the abnormality of the third abnormal value is determined as a data problem, updating the corresponding third abnormal value in the database by using the normal value corresponding to the third abnormal value; and if the reason causing the abnormality of the third abnormal value is determined as a non-data problem, updating the corresponding third abnormal value in the database by the historical record average value corresponding to the third abnormal value.
Here, the implementation of the above method is further illustrated by the following examples:
1. data pre-processing
(1) Data warehousing, namely performing data warehousing on the original data of each version;
(2) And completing data, wherein statistics of all elements in the historical version data are not completely the same, and the historical data needs to be completed.
(3) And cleaning historical abnormal values, cleaning problematic recorded values of the historical versions, and replacing the problematic recorded values with correct values to reduce the influence on average values.
2. Selecting analytical variables
The data elements have great difference among different provinces, and great error exists when analysis is carried out according to the recorded data. However, the change rate of the data update shows a relatively stable rule in the periodic change of the history, so the change rate between the same element versions is selected as the standard variable. Rate of change = (current version-last version)/last version 100%;
3. selecting data analysis dimensions
(1) Normal distribution to obtain standard deviation
Data of the change rate of the same element, different provinces and front and back versions are used as a set, normal distribution drawing is conducted on the data set, and the element meeting normal distribution is verified and confirmed manually based on the updating condition of the data. In the map element updating characteristic, data is generally added in a forward direction, so that a normal distribution tends to extend towards one end infinitely.
Data elements meeting normal distribution, real world updating rules, such as road, background and related road types, speed limit, traffic limit, vehicle information, direction board and other elements, further data analysis and parameter selection are carried out on the elements, three times of standard deviation is used as an index for measuring whether standard change is exceeded, and the record exceeding the index is output as an abnormal record.
Standard deviation ofCalculated using this formula:
Figure BDA0003045130800000161
the measure of the anomaly change can be calculated by the formula:
Figure BDA0003045130800000162
when Z =3, it is three times the standard deviation as an index for measuring whether the standard variation is exceeded.
The method for calculating the standard deviation of the historical data and the method for calculating the standard deviation of the inter-provincial data can refer to the foregoing embodiments, and are not described herein again.
(2) Data correlation confidence interval
In the step, the elements in the library are divided into a plurality of categories according to roads, backgrounds and poi; and (3) calculating correlation coefficients of every two elements in each category, and outputting the elements with strong correlation, wherein 600 groups of the elements with the correlation of more than 80 percent exist.
Carrying out manual verification and judgment on the output element group, and excluding combinations which do not actually have correlation; elements with strong correlations exist in the map data, such as (intersection-traffic limit, intersection-traffic light, poi-background of functional categories).
A fitted straight line, y = ax + b, is calculated from the two element data of the element group and a confidence interval is confirmed by adjusting the variable values a, b.
And when the confidence interval is exceeded, reporting an abnormal record, and according to the map element data change characteristic and the attribute related change characteristic, the change interval of part of elements is not negative. As shown, the actual confidence interval is a yellow region.
(3) Solving for abnormal records according to historical extreme values of data elements
There is instability in realistic changes for certain elements that do not meet a normal distribution, such as a particular kind of poi, agency; the extreme points of the historical change of the elements are calculated, the average value of the historical change of the elements is added and subtracted to be used as a standard change range, and the out-of-range is used as abnormal output.
Product examples
In order to implement the technical solution of the foregoing method embodiment, this embodiment discloses a map data monitoring and verifying system, as shown in fig. 7, which includes the following components:
the data import module is used for acquiring the data volume of the target data elements in the electronic map updating version database to be tested;
the difference statistical module is used for calculating the corresponding data volume change rate according to the data volume of the same target data element of the updated version to be detected and the compared version, and forming a change rate set corresponding to the data volume change rate by using the data volume change rate of the same target data element between different versions in the target administrative division;
the first analysis module is used for carrying out normal distribution analysis on the change rate set to obtain a first change rate set which accords with normal distribution and a second change rate set which does not accord with the normal distribution in the change rate set;
the abnormal monitoring module is used for taking the data volume change rate which is greater than a preset threshold value in the first change rate set as a first abnormal value in normal distribution corresponding to the first change rate set; the extreme value analysis is carried out on the data volume change rate in the second change rate set, and the data volume change rate which exceeds a preset threshold range in the second change rate set is used as a second abnormal value;
and the output verification module is used for outputting an abnormality monitoring result and an analysis report according to the first abnormal value and/or the second abnormal value, and updating and verifying the to-be-detected version database of the electronic map.
In this embodiment, according to the difference of the line update elements of the map data product and the difference of the update periods of the map data product, the anomaly analysis of the provincial data elements, the anomaly analysis of the national element data, the anomaly analysis of the single element update, and the anomaly analysis during the cloud service type product update can be performed. Aiming at different service objects, different model parameters are extracted by adopting the same technical method according to the updating change condition, different map elements can be supported to update scenes, and the method has universality.
Correspondingly, this embodiment further discloses a component-based map data service cloud platform, as shown in fig. 8, which includes the following components:
the user interaction component is used for providing a mode access platform comprising Web, API or SDK;
the unified authentication component is used for identity authentication, functional authority management, access control and/or dynamic configuration;
the database component is used for storing electronic map data, charging key point coordinates, high-speed local line units which are pre-divided into high-speed local lines according to the charging key point coordinates and toll corresponding to the high-speed local line units;
the product editing assembly is configured with an independent data editing space and used for generating a customized data product based on data in the editing, converting, fusing, checking, differentiating and batch processing database assembly and/or combining with imported reference data;
the product release component is used for performing version management, authority management, data increment release, real-time push, service online, flow control and service offline of data and services;
the service customizing component is used for providing customizable map data products and services according to users and requirements, and comprises a charging unit which is used for matching the passing path with the high-speed local line unit, extracting each high-speed local line unit corresponding to the passing path and outputting the charging unit;
and the statistical analysis component is used for executing the map data monitoring and verifying method disclosed by any scheme and is used for carrying out usage statistics, hotspot data analysis, operation and maintenance monitoring and early warning.
As an optional implementation manner, the unified authentication component further includes:
the load balancer is used for monitoring a network port and dynamically configuring network resources;
the API gateway module is used for carrying out API routing and access control;
and the authentication module is used for carrying out identity authentication and functional authority management on the user.
According to a fourth aspect of the present disclosure, there is also disclosed a computer apparatus, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor;
the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the map data monitoring and verifying method disclosed in any one of the above aspects.
The map data monitoring and verifying system, the component type map data service cloud platform and the computer device provided by the disclosure can be used for executing the map data monitoring and verifying method described in any of the above embodiments, and the implementation principle and the technical effect are similar, and are not described herein again.
In an alternative embodiment, the functional units in a data statistics module of the present disclosure may be directly in hardware, in a software module executed by a processor, or in a combination of both.
A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium.
The Processor may be a Central Processing Unit (CPU), other general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), other Programmable logic devices, discrete Gate or transistor logic, discrete hardware components, or any combination thereof. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.
EXAMPLE five
Referring to fig. 8, there is shown a structure of a map cloud platform, which mainly includes:
a map element database for storing data in data elements of respective map versions;
and the data management platform comprises the map data monitoring and verifying system described in the embodiment and is used for processing the data in the map element database.
The map cloud platform provided by the present disclosure can be used for executing the map data monitoring and verification method described in any of the above embodiments, and the implementation principle and the technical effect are similar, which are not described herein again.
In the several embodiments provided in the present disclosure, it should be understood that the disclosed modules and methods may be implemented in other ways. For example, the above-described module embodiments are merely illustrative, and for example, a division of a unit is only one type of division of a logic function, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of modules or units through some interfaces, and may be in an electrical, mechanical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
The above description is only an example of the present disclosure, and not intended to limit the scope of the present disclosure, and all structural equivalents that can be used in the present disclosure and drawings, and directly or indirectly applied to other related technical fields, are included in the scope of the present disclosure.

Claims (10)

1. A map data monitoring and verification method is characterized by comprising the following steps:
acquiring the data volume of target data elements in a database of the version to be updated of the electronic map;
calculating corresponding data volume change rate according to the data volume of the same target data element of the updated version to be detected and the compared version, and forming a change rate set corresponding to the data volume change rate by using the data volume change rate of the same target data element between different versions in a target administrative division;
performing normal distribution analysis on the change rate set to obtain a first change rate set which accords with normal distribution and a second change rate set which does not accord with the normal distribution in the change rate set;
in normal distribution corresponding to the first change rate set, outputting the data volume change rate larger than a preset threshold value in the first change rate set as a first abnormal value;
performing extreme value analysis on the data volume change rate in the second change rate set, and outputting the data volume change rate exceeding a preset threshold range in the second change rate set as a second abnormal value;
and outputting an abnormality monitoring result and an analysis report according to the first abnormal value and/or the second abnormal value.
2. The map data monitoring and verification method according to claim 1, wherein the process of determining a corresponding data volume change rate according to the data volume and obtaining a change rate set corresponding to the data volume change rate further comprises:
calculating the data volume change rate between the data volumes of the corresponding data elements between every two adjacent versions from the updating of the version to be detected to obtain an inter-version change rate set of the data volume change rate between the versions; and/or the presence of a gas in the gas,
and calculating the data volume change rate between the current data volume corresponding to each data element in the updated version to be tested under each area unit and the corresponding data volume in the previous map version to obtain an inter-area change rate set of the data volume change rates among the area units.
3. The map data monitoring and verification method according to claim 1, wherein the process of performing extremum analysis on the data volume change rates in the second set of change rates further comprises:
acquiring a historical change extreme value corresponding to the target data element, adding or subtracting a corresponding historical change average value according to the historical change extreme value corresponding to each data volume change rate in the second change rate set, and setting the preset threshold range;
and outputting the data volume change rate exceeding the preset threshold range in the second change rate set as the second abnormal value.
4. The map data monitoring and verification method as claimed in claim 1, wherein the obtaining of the data amount of the data elements in the database of the version to be updated of the electronic map comprises:
inputting the data volume corresponding to each data element into the electronic map updating version database to be tested, and calculating the data volume change rate corresponding to the data element;
if the data volume change rate is vacant, completing the vacant data volume change rate according to the data volume change rate of the corresponding data element in the updated version to be tested;
and if the data volume change rate has errors, correcting the data volume change rate with errors according to the historical average value of the data volume change rate.
5. The map data monitoring and verification method according to any one of claims 1 to 4, further comprising:
classifying the data volume change rate of the data elements in at least one historical version according to the category of the data elements, calculating a correlation coefficient between every two data volume change rates in each category, and when the correlation coefficient is larger than a strong correlation threshold value, forming a data element group by two data elements corresponding to the correlation coefficient;
according to the IDs of the two data elements in the data element group, extracting the data volume change rate of the target data element corresponding to the data element ID in each map version in the database of the version to be updated of the electronic map, and generating a correlation change rate set;
according to the correlation change rate set, performing fitting calculation on the data volume change rate of the strongly correlated target data elements, and determining a confidence interval according to the result of the fitting calculation and a preset confidence coefficient;
and judging whether the data volume change rate of the target data element extracted from the updated version to be tested exceeds the confidence interval or not, and outputting the data volume change rate exceeding the confidence interval as a third abnormal value.
6. The map data monitoring and verification method according to any one of claims 1 to 5, further comprising:
performing anomaly analysis according to the first anomaly value, the second anomaly value and/or the third anomaly value to obtain an anomaly result and an analysis report, and updating and verifying the version database to be detected of the electronic map; further comprising:
performing anomaly analysis on the first and/or second outliers comprises: if the cause of the first abnormal value and/or the second abnormal value is a data problem, the data quantity of the data element corresponding to the first abnormal value and/or the second abnormal value is counted again, the first abnormal value and/or the second abnormal value in the version database to be tested of the electronic map is updated through the calculated data quantity change rate, and the first abnormal value and/or the second abnormal value and the corresponding abnormal cause analysis report are output, wherein the data problem comprises the problem of the abnormal data quantity change rate caused by the abnormal workflow; if the cause of the first abnormal value and/or the second abnormal value is a non-data problem, updating the first abnormal value and/or the second abnormal value in the electronic map updating version database to be tested according to the historical record average value corresponding to the first abnormal value and/or the second abnormal value, and outputting the first abnormal value and/or the second abnormal value and the corresponding abnormal cause analysis report, wherein the non-data problem comprises a problem of abnormal data volume change rate caused by workflow change;
performing anomaly analysis on the third anomaly value includes: if the cause of the third anomaly is a data problem, the data quantity of the data element corresponding to the third anomaly is counted again, the electronic map is updated according to the calculated data quantity change rate to update the third anomaly in the version database to be tested, and the third anomaly and the corresponding abnormal cause analysis report are output; and if the reason of the third abnormal value is a non-data problem, updating the third abnormal value in the version database to be tested by using the historical record average value corresponding to the third abnormal value, and outputting the third abnormal value and a corresponding abnormal reason analysis report.
7. A map data monitoring and verification system, comprising:
the data import module is used for acquiring the data volume of the target data elements in the electronic map updating version database to be tested;
the difference statistical module is used for calculating corresponding data volume change rates according to the data volumes of the same target data element of the updated version to be detected and the compared version, and forming a change rate set corresponding to the data volume change rates by using the data volume change rates of the same target data element among different versions in a target administrative division;
the first analysis module is used for carrying out normal distribution analysis on the change rate set to obtain a first change rate set which accords with normal distribution and a second change rate set which does not accord with the normal distribution in the change rate set;
the abnormal monitoring module is used for taking the data volume change rate which is greater than a preset threshold value in the first change rate set as a first abnormal value in normal distribution corresponding to the first change rate set; the extreme value analysis is carried out on the data volume change rate in the second change rate set, and the data volume change rate which exceeds a preset threshold range in the second change rate set is used as a second abnormal value;
and the output verification module is used for outputting an abnormal monitoring result and an analysis report according to the first abnormal value and/or the second abnormal value, and updating and verifying the database of the to-be-detected version of the electronic map.
8. A component-based map data service cloud platform, comprising:
the user interaction component is used for providing a mode access platform comprising Web, API or SDK;
the unified authentication component is used for identity authentication, functional authority management, access control and/or dynamic configuration;
the database component is used for storing electronic map data, charging key point coordinates, high-speed line units which are pre-divided into high-speed lines according to the charging key point coordinates and toll corresponding to the high-speed line units;
the product editing assembly is configured with an independent data editing space and used for generating a customized data product based on editing, converting, fusing, checking, differentiating, batch processing of data in the database assembly and/or combination of imported reference data;
the product release component is used for performing version management, authority management, data increment release, real-time push, service online, flow control and service offline of data and services;
the service customizing component is used for providing customizable map data products and services according to users and requirements, and comprises a charging unit which is used for matching a passing path with the high-speed local line unit, extracting each high-speed local line unit corresponding to the passing path and outputting the charging unit;
the statistical analysis component is used for executing the map data monitoring and verifying method as claimed in any one of claims 1 to 6, and is used for carrying out usage statistics, hot spot data analysis, operation and maintenance monitoring and early warning.
9. The modular map data service cloud platform of claim 8, wherein the unified authentication component further comprises:
the load balancer is used for monitoring a network port and dynamically configuring network resources;
the API gateway module is used for carrying out API routing and access control;
and the authentication module is used for carrying out identity authentication and functional authority management on the user.
10. A computer device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor;
wherein the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to cause the at least one processor to perform the map data monitoring verification method of any one of claims 1-6.
CN202110470378.8A 2021-04-28 2021-04-28 Map data monitoring and verifying method and component type map data service cloud platform Pending CN115248773A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110470378.8A CN115248773A (en) 2021-04-28 2021-04-28 Map data monitoring and verifying method and component type map data service cloud platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110470378.8A CN115248773A (en) 2021-04-28 2021-04-28 Map data monitoring and verifying method and component type map data service cloud platform

Publications (1)

Publication Number Publication Date
CN115248773A true CN115248773A (en) 2022-10-28

Family

ID=83695801

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110470378.8A Pending CN115248773A (en) 2021-04-28 2021-04-28 Map data monitoring and verifying method and component type map data service cloud platform

Country Status (1)

Country Link
CN (1) CN115248773A (en)

Similar Documents

Publication Publication Date Title
US8577776B2 (en) Risk and reward assessment mechanism
CN113393211A (en) Method and system for intelligently improving automatic production efficiency
CN112565422B (en) Method, system and storage medium for identifying fault data of power internet of things
CN111428095B (en) Graph data quality verification method and graph data quality verification device
CN112102074B (en) Score card modeling method
WO2024067358A1 (en) Efficiency analysis method and system for warehouse management system, and computer device
CN111680420A (en) Simulation system dynamics model for industrial policy influence and implementation method thereof
CN111338876A (en) Fault mode and influence analysis decision method, system and storage medium
CN117539677A (en) Perception enabling platform of internet of things
CN117540826A (en) Optimization method and device of machine learning model, electronic equipment and storage medium
CN113610575B (en) Product sales prediction method and prediction system
CN117311295B (en) Production quality improving method and system based on wireless network equipment
CN112508440B (en) Data quality evaluation method, device, computer equipment and storage medium
CN111882289B (en) Device and method for measuring and calculating project data auditing index interval
US7689952B2 (en) System and method for determining and visualizing tradeoffs between yield and performance in electrical circuit designs
CN112860672A (en) Method and device for determining label weight
CN112598326A (en) Model iteration method and device, electronic equipment and storage medium
CN117195451A (en) Bridge monitoring data restoration method based on graph theory
CN115248773A (en) Map data monitoring and verifying method and component type map data service cloud platform
CN108549974B (en) CIME power grid model evaluation method based on analytic hierarchy process
CN110287272A (en) A kind of configurable real-time feature extraction method, apparatus and system
CN105404608A (en) Formula analysis based complicated index set calculation method and system
CN106681967A (en) Meter changing operation method based on data backtracking mechanism
US11983623B1 (en) Data validation for automatic model building and release
CN118171090B (en) Block chain-based carbon emission monitoring and checking method, device, equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination