CN118114183A - Urban information multi-source data fusion method and device, storage medium and electronic system - Google Patents
Urban information multi-source data fusion method and device, storage medium and electronic system Download PDFInfo
- Publication number
- CN118114183A CN118114183A CN202410506411.1A CN202410506411A CN118114183A CN 118114183 A CN118114183 A CN 118114183A CN 202410506411 A CN202410506411 A CN 202410506411A CN 118114183 A CN118114183 A CN 118114183A
- Authority
- CN
- China
- Prior art keywords
- data
- information
- city
- source
- space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007500 overflow downdraw method Methods 0.000 title claims description 6
- 238000000034 method Methods 0.000 claims abstract description 48
- 230000004927 fusion Effects 0.000 claims abstract description 31
- 238000007781 pre-processing Methods 0.000 claims abstract description 20
- 238000007499 fusion processing Methods 0.000 claims abstract description 6
- 238000012545 processing Methods 0.000 claims description 89
- 238000004364 calculation method Methods 0.000 claims description 35
- 238000001514 detection method Methods 0.000 claims description 34
- 239000011159 matrix material Substances 0.000 claims description 24
- 230000008569 process Effects 0.000 claims description 17
- 238000000605 extraction Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 13
- 238000010276 construction Methods 0.000 claims description 11
- 230000008859 change Effects 0.000 claims description 10
- 230000002159 abnormal effect Effects 0.000 claims description 7
- 238000001914 filtration Methods 0.000 claims description 7
- 238000010606 normalization Methods 0.000 claims description 6
- 230000005856 abnormality Effects 0.000 claims description 4
- 238000005516 engineering process Methods 0.000 abstract description 4
- 230000000694 effects Effects 0.000 abstract description 2
- 230000000875 corresponding effect Effects 0.000 description 25
- 238000007726 management method Methods 0.000 description 11
- 238000012544 monitoring process Methods 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 6
- 230000007613 environmental effect Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000002123 temporal effect Effects 0.000 description 3
- 238000000546 chi-square test Methods 0.000 description 2
- 238000013523 data management Methods 0.000 description 2
- 238000007418 data mining Methods 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000009430 construction management Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000010223 real-time analysis Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 231100000279 safety data Toxicity 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2433—Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/10—Pre-processing; Data cleansing
- G06F18/15—Statistical pre-processing, e.g. techniques for normalisation or restoring missing data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Tourism & Hospitality (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Educational Administration (AREA)
- Development Economics (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Probability & Statistics with Applications (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a method, a device, a storage medium and an electronic system for urban information multi-source data fusion, and relates to the technical field of urban multi-source data fusion technology. The method comprises the following steps: acquiring multi-source city data; preprocessing the multi-source city data to obtain first multi-source city data; extracting attribute information of the first multi-source city data to obtain city data attribute information; and carrying out fusion processing on the data space-time information and the data attribute through a preset CIM model to obtain the target city information. The method solves the problem of low precision of the urban multi-source data fusion, and further achieves the effect of improving the precision and efficiency of the urban multi-source data fusion.
Description
Technical Field
The embodiment of the invention relates to the field of communication, in particular to a city information multi-source data fusion method, a device, a storage medium and an electronic system.
Background
The smart city is a comprehensive complex giant system with complex elements, various applications, interaction and continuous evolution, and is designed to be integrally planned, and CIM organizes various city information into a systematic integral system which runs through the whole life cycle process of city construction management.
At present, the problem of inaccurate data acquisition, preprocessing and real-time analysis still exists in the application of a CIM model, and a complete multi-source heterogeneous data fusion analysis method and system are not formed yet; meanwhile, as the urban physical-digital space social sensing big data has the characteristics of various sources, various data formats, scattered storage, large data quantity and the like, the special forms provide challenges for collaborative expression, information aggregation, information derivation and increment of data and data mining.
Disclosure of Invention
The embodiment of the invention provides a method and a device for merging urban information multi-source data, which are used for at least solving the problem of low data merging precision in the related technology.
According to one embodiment of the present invention, there is provided a city information multisource data fusion method, including:
Acquiring multi-source city data, wherein the multi-source city data are acquired through information acquisition equipment corresponding to a plurality of city information management systems and are uploaded to a target storage platform;
preprocessing the multi-source city data to obtain first multi-source city data, wherein the preprocessing comprises data filtering processing and data normalization processing which are sequentially executed according to preset rules;
Extracting attribute information of the first multi-source city data to obtain city data attribute information, wherein the city data attribute information comprises data attribute and data space-time information;
and carrying out fusion processing on the data space-time information and the data attribute through a preset CIM model to obtain the target city information.
In an exemplary embodiment, the fusing the data space-time information and the data attribute through the preset CIM model to obtain the target city information includes:
carrying out data coding processing on the data space-time information and the data attribute;
and carrying out dynamic hooking processing on the data space-time information and the data attribute according to a data coding processing result so as to obtain the target city information.
In an exemplary embodiment, comparing the multi-source city data uploaded to the target storage platform with pre-stored model history data in the target storage platform, and judging the change degree of the collected multi-source city data relative to the model history data;
giving unique identification to the multi-source city data with the variation degree exceeding a threshold value, and preprocessing and extracting attribute information;
The unique identifier comprises a space-time area of corresponding data in the CIM model, and when in dynamic hooking processing, the space-time information and attribute information corresponding to the data are hooked to corresponding positions of the CIM model.
In an exemplary embodiment, before the dynamically hooking the data temporal and spatial information with the data attribute according to the data encoding processing result, the method further includes:
carrying out data matching processing on the data space-time information and the data attribute according to a data coding processing result;
And carrying out space-time consistency detection on the data matching processing result, and carrying out dynamic hooking processing on the data space-time information and the data attribute according to the data coding processing result under the condition that the space-time consistency detection result meets the preset consistency condition.
In an exemplary embodiment, after said data encoding process is performed on said data spatiotemporal information and said data attributes, said method further comprises:
Constructing a multi-source data matrix according to a preset construction rule and a data coding processing result;
Detecting element missing values and performing consistency calculation on the multi-source data matrix, wherein the consistency calculation at least comprises correlation calculation and chi-square check calculation;
And under the condition that the element missing value detection and/or consistency calculation result does not meet the preset detection condition, determining that the city data attribute information is abnormal.
In one exemplary embodiment, the dynamic hooking process includes:
calculating the weight coefficient of each piece of data according to the viscosity of the data type;
For data with higher viscosity with other data types, increasing the weight coefficient of the data; otherwise, reducing the weight coefficient;
and accessing the data link endowed with the weight system into a preset CIM model.
According to another embodiment of the present invention, there is provided a city information multi-source data fusion apparatus including:
the urban data acquisition module is used for acquiring multi-source urban data, wherein the multi-source urban data are acquired through information acquisition equipment corresponding to a plurality of urban information management systems and are uploaded to the target storage platform;
The preprocessing module is used for preprocessing the multi-source city data to obtain first multi-source city data, wherein the preprocessing comprises data filtering processing and data normalization processing which are executed according to preset rules;
The information extraction module is used for carrying out attribute information extraction processing on the first multi-source city data to obtain city data attribute information, wherein the city data attribute information comprises data attributes and data space-time information;
And the data fusion module is used for carrying out fusion processing on the data space-time information and the data attribute through a preset CIM model so as to obtain the target city information.
In one exemplary embodiment, the data fusion module includes:
The data coding unit is used for carrying out data coding processing on the data space-time information and the data attribute;
And the dynamic hooking unit is used for carrying out dynamic hooking processing on the data space-time information and the data attribute according to the data coding processing result so as to obtain the target city information.
In an exemplary embodiment, the apparatus further comprises:
The data matching subunit is used for carrying out data matching processing on the data space-time information and the data attribute according to the data coding processing result before carrying out dynamic hooking processing on the data space-time information and the data attribute according to the data coding processing result;
And the consistency detection subunit is used for carrying out space-time consistency detection on the data matching processing result, and carrying out dynamic hooking processing on the data space-time information and the data attribute according to the data coding processing result under the condition that the space-time consistency detection result meets the preset consistency condition.
In an exemplary embodiment, the apparatus further comprises:
The matrix construction subunit is used for constructing a multi-source data matrix according to a preset construction rule and a data coding processing result after the data space-time information and the data attribute are subjected to data coding processing;
the matrix calculation subunit is used for detecting element missing values and calculating consistency of the multi-source data matrix, wherein the consistency calculation at least comprises correlation calculation and chi-square check calculation;
And the abnormality judgment subunit is used for determining that the city data attribute information is abnormal under the condition that the element missing value detection and/or the consistency calculation result does not meet the preset detection condition.
According to a further embodiment of the invention, there is also provided a computer readable storage medium having stored therein a computer program, wherein the computer program is arranged to perform the steps of any of the method embodiments described above when run.
According to a further embodiment of the invention, there is also provided an electronic system comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.
According to the invention, the city data is connected into the library through the CIM model, so that the problems of similarity and inconsistency of data in the cross-domain in geometric position, attribute semantics, logic and the like are solved, the problem of low fusion precision of the multi-source city data is solved, and the effect of improving the fusion precision and efficiency of the multi-source city data is achieved.
Drawings
FIG. 1 is a flow chart of a method of urban information multisource data fusion according to an embodiment of the present invention;
fig. 2 is a block diagram of a city information multi-source data fusion apparatus according to an embodiment of the present invention.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings in conjunction with the embodiments.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order.
In this embodiment, a method for merging city information multi-source data is provided, and fig. 1 is a flowchart of a method for merging city information multi-source data according to an embodiment of the present invention, as shown in fig. 1, where the flowchart includes the following steps:
Step S11, multi-source city data are acquired, wherein the multi-source city data are acquired through information acquisition equipment corresponding to a plurality of city information management systems and are uploaded to a target storage platform;
In this embodiment, the existing urban data acquisition system depends on a single information management system, which limits the diversity of data sources and the comprehensive utilization efficiency of data, and uploads relevant data to a unified target storage platform, so that unified processing and calling of urban multi-source data can be facilitated, and the data management capability is improved.
The information acquisition equipment used for acquiring the related city data can be a sensor, a camera, GPS positioning equipment and the like, and is used for acquiring various city data such as traffic flow, environment monitoring data, public safety information and the like in real time; in order to adapt to the data, the target storage platform should have the characteristics of high capacity, high speed and high security so as to meet the storage and management requirements of a large amount of city data, and meanwhile, a standardized data service interface needs to be provided for an external system or application to call, so that the sharing and application of the data are realized.
For example, a large city is taken as an example, and the city is provided with a plurality of city information management systems such as traffic, environment, public security and the like. Each system collects relevant data in real time through respective data collection devices such as traffic monitoring cameras, air quality monitoring stations, emergency alarms and the like. The analyzed data is then uploaded to a data storage and management system that uses distributed storage techniques to ensure the security and reliability of the data. Meanwhile, the system also provides data backup and disaster recovery functions so as to prevent data loss or damage. Finally, through the data application service interface, an external system or application can conveniently call and utilize the data to provide data support for city management, decision support, public service and the like.
In a more preferable scheme, after the information acquisition devices corresponding to the urban information management systems acquire urban information data, the urban information data are cached in the target storage platform, and the urban information data are compared with the historical data prestored in the platform through the data identification module embedded in the target storage platform.
Since the data are acquired in real time, it is necessary to determine whether the change of the real-time data is significant, and if the data with significant change indicate that the data may contain important new information, the data should be considered; data that does not change much can be ignored.
In a specific embodiment, the degree of data change can be estimated by the following equation:
wherein, Is the rate of change, D Real time is the value of the real-time data, D Model is the value already in the model, and if the rate of change is greater than a preset threshold, the data needs to be considered for subsequent processing.
And (3) carrying out unique identification on the city information data with the change degree exceeding a preset threshold, wherein the identification comprises the time for generating the data and the difference between the time and the existing numerical value in the model, including updating, adding or deleting the data item.
Step S12, preprocessing the multi-source city data to obtain first multi-source city data, where the preprocessing includes data filtering processing and data normalization processing that are sequentially executed according to a preset rule.
In this embodiment, since the sources of the city data are different, the corresponding data formats and data amounts are also different, so that the data needs to be preprocessed to facilitate the subsequent processing.
The preset rule may be a rule for filtering the data, a rule for normalizing the data, or the like.
For example, the collected data is firstly processed by a data preprocessing unit to remove noise and abnormal values, then the data is converted into a uniform format so as to facilitate subsequent fusion analysis, and then valuable information and modes are extracted from a large amount of data through advanced data analysis technologies such as machine learning and data mining; in addition, the multi-source heterogeneous space-time data can be filtered and filtered, unreasonable data can be removed, homonymous and heteronymous synonyms can be eliminated, consistency can be checked, redundant data can be deleted, and data merging can be performed.
It can be appreciated that in the process of preprocessing the data, only the city information assigned with the unique identifier in step S11 is processed, so as to reduce the consumption of computing resources.
And step S13, carrying out attribute information extraction processing on the first multi-source city data to obtain city data attribute information, wherein the city data attribute information comprises data attributes and data space-time information.
In this embodiment, the attribute information extraction generally includes two parts, namely attribute information identification and information extraction, where the attribute information identification is mainly used for data attribute, and generally includes two parts, namely feature extraction and attribute classification, where the feature extraction and attribute classification can be implemented by an unsupervised neural network algorithm, such as a K-mean series algorithm; the feature extraction is to identify key attributes in the data, such as traffic flow, air quality index, temperature, humidity and the like, by analyzing the data; attribute classification includes classifying the identified attributes, such as classifying environmental monitoring data, traffic data, public safety data, energy consumption, population density, etc., respectively; and the extraction of the data space-time information comprises the extraction of time information (including time stamp, time period and the like of data acquisition) from the data and the extraction of the space information (such as geographic position (longitude and latitude), regional division and the like) of the data.
Before extracting the related data attribute, the city units can be divided by a GIS algorithm for further extracting the data attribute, the city is divided into a plurality of city units which are geographically independent according to the management level according to the city, and even the city in the mountain area with stronger third dimension can be further divided according to the height.
And S14, carrying out fusion processing on the data space-time information and the data attribute through a preset CIM model to obtain the target city information.
In this embodiment, the CIM model generally includes a plurality of layers, each layer representing a different aspect of the city, such as a physical layer, a social layer, an economic layer, etc., and then matching the extracted spatio-temporal information with corresponding elements in the CIM model according to data attributes to ensure that the data can be accurately correlated to corresponding locations and points in the model, where fusion algorithms that can be applied include (but are not limited to) fusion algorithms such as weighted averages, maximum likelihood estimates, data fusion frameworks, etc., to ensure accuracy and reliability of fusion results; meanwhile, the fused data is further integrated and optimized to eliminate inconsistency and improve the usability of the data. Post-processing of the data, such as smoothing, trend analysis, pattern recognition, etc., may be required to refine the valuable information and then present the fused brace in the form of targeted city information, which may include visual presentation, report generation, etc., to support the city management system.
In addition, it should be explained that, since the foregoing determination is performed on the obtained city information, only the city information data with the degree of change exceeding the preset value needs to be processed later, during the process of data fusion and hooking, the part of data may only correspond to a part of the area in the CIM model, so when a unique identifier is given to the data, the identifier includes a space-time area corresponding to the part of data in the CIM model.
Through the method, when the part of data is fused and hung, the space-time information and the attribute information corresponding to the part of data can be quickly positioned to the corresponding space-time area in the model through a space-time index technology and the unique identifier, and are hung to the corresponding position of the CIM model.
Through the steps, as the city data is fused and hung through the CIM model, the similarity and the inconsistency of data in the cross-domain of the city multi-source data in the aspects of geometric position, attribute semantics, logic and the like are solved, the problem of low fusion precision of the multi-source city data is solved, and the fusion precision and the efficiency of the multi-source city data are improved.
The main execution body of the above steps may be, but not limited to, a base station, a terminal, and the like.
In an optional embodiment, the fusing the data temporal-spatial information and the data attribute through a preset CIM model to obtain the target city information includes:
and step S141, performing data encoding processing on the data space-time information and the data attribute.
In this embodiment, standardized coding formats, such as ISO 8601 date and time representation methods and WGS 84 geographic coordinate systems, may be used for data spatiotemporal information. For data attributes such as traffic flow, air quality index, etc., corresponding coding rules can be adopted according to the type and characteristics of the data attributes. For example, predefined codes or identifiers may be used to represent different types of traffic flow data. It should be noted that, no matter what coding mode is adopted, the format of the final coding result needs to be unified, so that subsequent processing is facilitated, for example, a unified sequence format [ a#b#c$d ] may be adopted, where each data includes 3 segments of ordinary codes a to C and a segment of identifier or special code D.
And step S144, carrying out dynamic hooking processing on the data space-time information and the data attribute according to the data coding processing result so as to obtain the target city information.
In this embodiment, the purpose of performing the dynamic hooking process is to correlate the data temporal-spatial information and the data attribute in the CIM model according to the encoding result; for example, traffic flow data at a specific time point is hung on a corresponding road network node in the CIM model, so that the matching of the encoded space-time information and attribute information is realized, and the traffic flow data is hung on a corresponding position of the CIM model. The dynamic hooking process allows the data to be flexibly updated and maintained in the CIM model, and real-time performance and accuracy of the target city information are ensured.
In an alternative embodiment, before the dynamically hooking the data temporal and spatial information with the data attribute according to the data encoding processing result, the method further includes:
and step S142, carrying out data matching processing on the data space-time information and the data attribute according to the data coding processing result.
In this embodiment, on the one hand, the purpose of matching the encoding processing result with the data temporal-spatial information and the corresponding data attribute is to verify whether the data encoding result can be successfully matched with the related data attribute and the data temporal-spatial information, so as to ensure the accuracy of the encoding result and ensure the correct association between the data temporal-spatial information and the data attribute; on the other hand, the method is also used for facilitating the subsequent processing according to the matching processing result and directly according to the attribute or the encoding result of the space-time information, so that the data processing efficiency is greatly improved.
The data matching process may be implemented by searching for keywords in the data, using a similarity measure method, or applying a machine learning algorithm. For example, according to the encoding result, the encoding corresponding to the environmental monitoring data collected at the same time point is matched with the encoding corresponding to the corresponding geographic position information.
And step S143, carrying out space-time consistency detection on the data matching processing result, and judging whether the space-time consistency detection result meets a preset consistency condition.
And step S144 is executed under the condition that the preset consistency condition is met, namely, the data space-time information and the data attribute are dynamically articulated according to the data coding processing result.
In this embodiment, after the data matching process is completed, space-time consistency detection is performed. The consistency detection aims at verifying whether the matched data are consistent in time and space, namely whether the data conform to the actual urban operation condition; spatio-temporal consistency detection may be performed by comparing temporal and spatial information in the data with known baseline data or model predictions. For example by analyzing the consistency of traffic flow data with historical traffic patterns or checking whether the environmental monitoring data is consistent with weather conditions.
And fusing the data time-space information and the data attribute which are successfully matched and are consistent in time and space into the CIM model, and then, dynamically hooking the CIM model by updating corresponding data elements in the CIM model in real time. For example, the matched traffic flow data is updated to the road network nodes of the CIM model, or the environmental monitoring data is updated to the corresponding geographic locations in the model.
It should be noted that, the dynamic hooking process is only executed when the space-time consistency detection result meets the preset consistency condition.
In addition, the foregoing process of implementing dynamic hooking through updating corresponding data elements in the CIM model in real time specifically includes the following steps:
s1441, calculating the weight coefficient of each data according to the viscosity of the data type.
The weight coefficient is generally used for reflecting the reliability or accuracy of the data in the data fusion, and is helpful for adjusting and optimizing the weight of specific data in the model so as to improve the overall quality of the fused data. For example, for data of which accuracy is known to be relatively high for some data sources, the compensation data may be increased accordingly; for some data with updated value, higher weight coefficients can be given; in addition, some data with stronger integrity can also be given a higher weight coefficient.
For some data, the weight coefficient of the data needs to be calculated through the viscosity of the data and other data, for example, for the traffic flow data, the data may be influenced by the data of air quality, geographic position and the like; it is therefore necessary to calculate the weight coefficients thereof, wherein the weight coefficients are calculated by the following method:
1) Determining a main sequence and a secondary sequence; where the primary sequence is the primary data sequence to be analyzed, such as traffic flow data, and the secondary sequence is other data sequences that may affect the primary sequence, such as air quality, geographic location, etc.
2) Calculating a correlation coefficient:
；
Where x 0 (k) is the (k) th data point of the main sequence; x i (k) is the (k) th data point of the secondary sequence; ρ is a resolution factor, typically taking a value of 0.5.
3) Calculating a viscosity coefficient; the viscosity coefficient is the comprehensive evaluation of the association coefficient and reflects the overall association degree of the auxiliary sequence and the main sequence; the calculation method is as follows:
；
Where n is the number of data points.
S1442, for a certain data type with higher viscosity with other data types, increasing the weight coefficient; and otherwise, reducing the weight coefficient.
Such as in the example above, the primary sequence x 0 is traffic flow; one of the auxiliary sequences x 1 is air quality; the correlation coefficient of the traffic flow and the air quality is found to be 0.8 through calculation, and the correlation coefficient of the traffic flow and the air quality can be considered to be higher, so that a higher weight coefficient can be set for x 1, for example, the higher weight coefficient can be achieved by multiplying a factor larger than 1, the factor can be determined according to the intensity of viscosity, for example, the factor of 1.5, 1.6 or 1.8 can be given to the air quality for the traffic flow and the air quality with the correlation coefficient of 0.8.
S1443, hanging the data endowed with the weight system into a preset CIM model.
In the above embodiment, since the quality and the correlation of different data sources are considered in the process of data fusion through the weight coefficient, the influence of the data sources is properly adjusted, so that more reliable or more relevant data has larger influence in the final fusion information, and the accuracy and the usability of the fusion result are improved.
In an alternative embodiment, after said data encoding process is performed on said data spatiotemporal information and said data attributes, said method further comprises:
Step S1411, constructing a multi-source data matrix according to a preset construction rule and a data coding processing result.
In this embodiment, the construction rule determines the distribution mode of the elements in the multi-source data matrix and the types of the matrix, and filling the multi-source data matrix according to the data encoding processing result can determine the data correlation of the multi-source data in the same calculation region, so that it is beneficial to determine the relationship between the multi-source data as soon as possible; meanwhile, the construction of the data matrix can also take the time sequence characteristic and the spatial distribution of the data into consideration so as to ensure the multidimensional integration of the data.
Step S1412, performing element deficiency value detection and consistency calculation on the multi-source data matrix, wherein the consistency calculation at least includes correlation calculation and chi-square check calculation.
In this embodiment, element missing value detection is performed on the constructed multi-source data matrix H to identify missing or incomplete portions that may exist in the data; the correlation calculation is used for evaluating the degree of correlation between data attributes, and the degree of correlation can be judged by similarity or correlation coefficient, and the following formula can be specifically referred to; and the chi-square test calculation is used for evaluating the consistency of data distribution so as to judge whether the adjacent space-time data has abnormality or not:
；
In the middle of (a) ,/>) Is the position coordinates of the elements in the multisource data matrix H,/>And/>And the element coordinate mean values are respectively shown, and r is a correlation coefficient.
Step S1413, determining that the city data attribute information is abnormal if the element deficiency value detection and/or the consistency calculation result does not satisfy the preset detection condition.
In this embodiment, the detection condition may be that the correlation coefficient is lower than a preset threshold, or that the result of the chi-square test indicates that the data distribution deviates significantly from the expectation.
From the description of the above embodiments, it will be clear to a person skilled in the art that the method according to the above embodiments may be implemented by means of software plus the necessary general hardware platform, but of course also by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The embodiment also provides a city information multi-source data fusion device, which is used for realizing the above embodiment and the preferred implementation manner, and the description is omitted. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
Fig. 2 is a block diagram of a city information multi-source data fusion apparatus according to an embodiment of the present invention, as shown in fig. 2, the apparatus includes:
the city data acquisition module 21 is configured to acquire multi-source city data, where the multi-source city data is acquired by information acquisition devices corresponding to a plurality of city information management systems, and is uploaded to a target storage platform;
a preprocessing module 22, configured to preprocess the multi-source city data to obtain first multi-source city data, where the preprocessing includes a data filtering process and a data normalization process that are performed according to a preset rule in sequence;
An information extraction module 23, configured to perform attribute information extraction processing on the first multi-source city data to obtain city data attribute information, where the city data attribute information includes a data attribute and data space-time information;
the data fusion module 24 is configured to fuse the data space-time information and the data attribute through a preset CIM model, so as to obtain the target city information.
In an alternative embodiment, the data fusion module 24 includes:
The data coding unit is used for carrying out data coding processing on the data space-time information and the data attribute;
And the dynamic hooking unit is used for carrying out dynamic hooking processing on the data space-time information and the data attribute according to the data coding processing result so as to obtain the target city information.
In an alternative embodiment, the apparatus further comprises:
The data matching subunit is used for carrying out data matching processing on the data space-time information and the data attribute according to the data coding processing result before carrying out dynamic hooking processing on the data space-time information and the data attribute according to the data coding processing result;
And the consistency detection subunit is used for carrying out space-time consistency detection on the data matching processing result, and carrying out dynamic hooking processing on the data space-time information and the data attribute according to the data coding processing result under the condition that the space-time consistency detection result meets the preset consistency condition.
In an alternative embodiment, the apparatus further comprises:
The matrix construction subunit is used for constructing a multi-source data matrix according to a preset construction rule and a data coding processing result after the data space-time information and the data attribute are subjected to data coding processing;
the matrix calculation subunit is used for detecting element missing values and calculating consistency of the multi-source data matrix, wherein the consistency calculation at least comprises correlation calculation and chi-square check calculation;
And the abnormality judgment subunit is used for determining that the city data attribute information is abnormal under the condition that the element missing value detection and/or the consistency calculation result does not meet the preset detection condition.
It should be noted that each of the above modules may be implemented by software or hardware, and for the latter, it may be implemented by, but not limited to: the modules are all located in the same processor; or the above modules may be located in different processors in any combination.
Embodiments of the present invention also provide a computer readable storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the method embodiments described above when run.
In one exemplary embodiment, the computer readable storage medium may include, but is not limited to: a usb disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory RAM), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing a computer program.
An embodiment of the invention also provides an electronic system comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.
In an exemplary embodiment, the electronic system may further include a transmission device connected to the processor, and an input/output device connected to the processor.
Specific examples in this embodiment may refer to the examples described in the foregoing embodiments and the exemplary implementation, and this embodiment is not described herein.
It will be appreciated by those skilled in the art that the modules or steps of the invention described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may be implemented in program code executable by computing devices, so that they may be stored in a storage device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than that shown or described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention should be included in the protection scope of the present invention.
Claims (12)
1. The city information multisource data fusion method is characterized by comprising the following steps of:
Acquiring multi-source city data, wherein the multi-source city data are acquired through information acquisition equipment corresponding to a plurality of city information management systems and are uploaded to a target storage platform;
preprocessing the multi-source city data to obtain first multi-source city data, wherein the preprocessing comprises data filtering processing and data normalization processing which are sequentially executed according to preset rules;
Extracting attribute information of the first multi-source city data to obtain city data attribute information, wherein the city data attribute information comprises data attribute and data space-time information;
and carrying out fusion processing on the data space-time information and the data attribute through a preset CIM model to obtain the target city information.
2. The method for merging city information multisource data according to claim 1, wherein the merging the data space-time information and the data attribute through the preset CIM model to obtain the target city information comprises:
carrying out data coding processing on the data space-time information and the data attribute;
and carrying out dynamic hooking processing on the data space-time information and the data attribute according to a data coding processing result so as to obtain the target city information.
3. The urban information multi-source data fusion method according to claim 2, wherein the multi-source urban data uploaded to the target storage platform is compared with the model history data prestored in the target storage platform, and the degree of change of the acquired multi-source urban data relative to the model history data is judged;
giving unique identification to the multi-source city data with the variation degree exceeding a threshold value, and preprocessing and extracting attribute information;
The unique identifier comprises a space-time area of corresponding data in the CIM model, and when in dynamic hooking processing, the space-time information and attribute information corresponding to the data are hooked to corresponding positions of the CIM model.
4. The method for merging city information multi-source data according to claim 2, wherein before the dynamically hooking the data space-time information with the data attribute according to the data encoding result, the method further comprises:
carrying out data matching processing on the data space-time information and the data attribute according to a data coding processing result;
And carrying out space-time consistency detection on the data matching processing result, and carrying out dynamic hooking processing on the data space-time information and the data attribute according to the data coding processing result under the condition that the space-time consistency detection result meets the preset consistency condition.
5. The method of claim 2, wherein after said data encoding process is performed on said data temporal-spatial information and said data attributes, said method further comprises:
Constructing a multi-source data matrix according to a preset construction rule and a data coding processing result;
Detecting element missing values and performing consistency calculation on the multi-source data matrix, wherein the consistency calculation at least comprises correlation calculation and chi-square check calculation;
And under the condition that the element missing value detection and/or consistency calculation result does not meet the preset detection condition, determining that the city data attribute information is abnormal.
6. The method for merging city information multi-source data according to claim 4, wherein the dynamic hooking process comprises:
calculating the weight coefficient of each piece of data according to the viscosity of the data type;
For data with higher viscosity with other data types, increasing the weight coefficient of the data; otherwise, reducing the weight coefficient;
and accessing the data link endowed with the weight system into a preset CIM model.
7. Urban information multisource data fusion device, characterized by comprising:
the urban data acquisition module is used for acquiring multi-source urban data, wherein the multi-source urban data are acquired through information acquisition equipment corresponding to a plurality of urban information management systems and are uploaded to the target storage platform;
The preprocessing module is used for preprocessing the multi-source city data to obtain first multi-source city data, wherein the preprocessing comprises data filtering processing and data normalization processing which are executed according to preset rules;
The information extraction module is used for carrying out attribute information extraction processing on the first multi-source city data to obtain city data attribute information, wherein the city data attribute information comprises data attributes and data space-time information;
And the data fusion module is used for carrying out fusion processing on the data space-time information and the data attribute through a preset CIM model so as to obtain the target city information.
8. The apparatus of claim 7, wherein the data fusion module comprises:
The data coding unit is used for carrying out data coding processing on the data space-time information and the data attribute;
And the dynamic hooking unit is used for carrying out dynamic hooking processing on the data space-time information and the data attribute according to the data coding processing result so as to obtain the target city information.
9. The apparatus for merging urban information multisource data according to claim 8, wherein said apparatus further comprises:
The data matching subunit is used for carrying out data matching processing on the data space-time information and the data attribute according to the data coding processing result before carrying out dynamic hooking processing on the data space-time information and the data attribute according to the data coding processing result;
And the consistency detection subunit is used for carrying out space-time consistency detection on the data matching processing result, and carrying out dynamic hooking processing on the data space-time information and the data attribute according to the data coding processing result under the condition that the space-time consistency detection result meets the preset consistency condition.
10. The apparatus for merging urban information multisource data according to claim 9, wherein said apparatus further comprises:
The matrix construction subunit is used for constructing a multi-source data matrix according to a preset construction rule and a data coding processing result after the data space-time information and the data attribute are subjected to data coding processing;
the matrix calculation subunit is used for detecting element missing values and calculating consistency of the multi-source data matrix, wherein the consistency calculation at least comprises correlation calculation and chi-square check calculation;
And the abnormality judgment subunit is used for determining that the city data attribute information is abnormal under the condition that the element missing value detection and/or the consistency calculation result does not meet the preset detection condition.
11. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program, wherein the computer program is arranged to execute the method of any of the claims 1to6 when run.
12. An electronic system comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to run the computer program to perform the method of any of the claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410506411.1A CN118114183A (en) | 2024-04-25 | 2024-04-25 | Urban information multi-source data fusion method and device, storage medium and electronic system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410506411.1A CN118114183A (en) | 2024-04-25 | 2024-04-25 | Urban information multi-source data fusion method and device, storage medium and electronic system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118114183A true CN118114183A (en) | 2024-05-31 |
Family
ID=91220968
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410506411.1A Pending CN118114183A (en) | 2024-04-25 | 2024-04-25 | Urban information multi-source data fusion method and device, storage medium and electronic system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118114183A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118365149A (en) * | 2024-06-20 | 2024-07-19 | 山东高速股份有限公司 | Highway network risk prediction method, device, equipment, storage medium and product |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116483666A (en) * | 2023-03-08 | 2023-07-25 | 中国电力科学研究院有限公司 | Multi-source heterogeneous alarm information fusion method and system based on space-time correlation |
| CN116522272A (en) * | 2023-02-17 | 2023-08-01 | 中国测绘科学研究院 | Multi-source space-time data transparent fusion method based on urban information unit |
| CN117034189A (en) * | 2023-08-01 | 2023-11-10 | 绵阳市智慧城市产业发展有限责任公司 | Multi-source internet of things data fusion method |
-
2024
- 2024-04-25 CN CN202410506411.1A patent/CN118114183A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116522272A (en) * | 2023-02-17 | 2023-08-01 | 中国测绘科学研究院 | Multi-source space-time data transparent fusion method based on urban information unit |
| CN116483666A (en) * | 2023-03-08 | 2023-07-25 | 中国电力科学研究院有限公司 | Multi-source heterogeneous alarm information fusion method and system based on space-time correlation |
| CN117034189A (en) * | 2023-08-01 | 2023-11-10 | 绵阳市智慧城市产业发展有限责任公司 | Multi-source internet of things data fusion method |
Non-Patent Citations (2)
| Title |
|---|
| 刘军 等: "智能仓储环境监控", 28 February 2021, 机械工业出版社, pages: 48 * |
| 杨伯钢 等: "城市房屋管理地理信息系统技术与应用", 31 January 2017, 中国地图出版社, pages: 88 - 93 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118365149A (en) * | 2024-06-20 | 2024-07-19 | 山东高速股份有限公司 | Highway network risk prediction method, device, equipment, storage medium and product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN118114183A (en) | Urban information multi-source data fusion method and device, storage medium and electronic system | |
| CN109656793A (en) | A kind of information system performance stereoscopic monitoring method based on multi-source heterogeneous data fusion | |
| CN111176953B (en) | Abnormality detection and model training method, computer equipment and storage medium | |
| CN109241223B (en) | Behavior track identification method and system | |
| CN115865649B (en) | Intelligent operation and maintenance management control method, system and storage medium | |
| CN111209317A (en) | Knowledge graph abnormal community detection method and device | |
| CN113326177B (en) | Index anomaly detection method, device, equipment and storage medium | |
| CN115358155A (en) | Power big data abnormity early warning method, device, equipment and readable storage medium | |
| CN111767192B (en) | Business data detection method, device, equipment and medium based on artificial intelligence | |
| CN112446637A (en) | Building construction quality safety online risk detection method and system | |
| CN111259167B (en) | User request risk identification method and device | |
| CN114880312B (en) | Flexibly-set application system service data auditing method | |
| CN113554361A (en) | Comprehensive energy system data processing and calculating method and processing system | |
| CN117172753A (en) | Intelligent hydropower station inspection system and method based on mobile internet | |
| CN111340975A (en) | Abnormal data feature extraction method, device, equipment and storage medium | |
| CN117151726A (en) | Fault repairing method, repairing device, electronic equipment and storage medium | |
| KR101960755B1 (en) | Method and apparatus of generating unacquired power data | |
| CN117591860A (en) | Data anomaly detection method and device | |
| CN117141265A (en) | Operation monitoring system and method for intelligent wireless charging pile | |
| CN111833171B (en) | Abnormal operation detection and model training method, device and readable storage medium | |
| CN111798237A (en) | Abnormal transaction diagnosis method and system based on application log | |
| CN118312891B (en) | Method, device, equipment, medium and program product for monitoring vibration type | |
| CN118172711B (en) | AI big data intelligent management method and system | |
| CN114913670B (en) | Personnel going out of sea safety management system based on safety code management | |
| CN117273765B (en) | Multistage dealer circulation data processing method and system based on automatic check |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |