CN116189436B - Multi-source data fusion algorithm based on big data - Google Patents
Multi-source data fusion algorithm based on big data Download PDFInfo
- Publication number
- CN116189436B CN116189436B CN202310262880.9A CN202310262880A CN116189436B CN 116189436 B CN116189436 B CN 116189436B CN 202310262880 A CN202310262880 A CN 202310262880A CN 116189436 B CN116189436 B CN 116189436B
- Authority
- CN
- China
- Prior art keywords
- data
- traffic
- fusion
- information
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 119
- 238000004422 calculation algorithm Methods 0.000 title claims abstract description 21
- 238000004458 analytical method Methods 0.000 claims abstract description 23
- 238000004364 calculation method Methods 0.000 claims abstract description 23
- 238000011156 evaluation Methods 0.000 claims abstract description 23
- 238000007781 pre-processing Methods 0.000 claims abstract description 10
- 238000007499 fusion processing Methods 0.000 claims abstract description 6
- 238000000034 method Methods 0.000 claims description 12
- 238000013507 mapping Methods 0.000 claims description 9
- 238000011160 research Methods 0.000 claims description 8
- 238000012544 monitoring process Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 7
- 238000007667 floating Methods 0.000 claims description 6
- 230000001174 ascending effect Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 description 6
- 238000005065 mining Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000007500 overflow downdraw method Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0125—Traffic data processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Traffic Control Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a multi-source data fusion algorithm based on big data, which relates to the technical field of data fusion and comprises the following steps: inputting traffic initial data sets of a plurality of data sources, and uniformly setting codes of the data sets as GBK codes; preprocessing the traffic initial data set; the preprocessing is performed by eliminating unqualified information in the traffic initial data set; extracting features of the preprocessed traffic data information to obtain traffic feature data; performing observation coefficient GF analysis on the traffic characteristic data to be fused, and performing fusion on the traffic characteristic data to be fused according to the data fusion terminals distributed with the corresponding quantity of the observation coefficients GF; the data fusion terminal collects a plurality of traffic characteristic data for fusion and outputs traffic evaluation data information; in the data fusion process, the calculation power occupation condition of the data fusion terminal is monitored and analyzed, whether the calculation power resource of the data fusion terminal needs to be redistributed or not is judged, and the data fusion efficiency is improved.
Description
Technical Field
The invention relates to the technical field of data fusion, in particular to a multi-source data fusion algorithm based on big data.
Background
In recent years, intelligent transportation systems have attracted attention from a large number of researchers. The intelligent transportation system is used as a comprehensive application product, and the related technology comprises information technology, communication technology, control technology, computer technology, perception technology and the like. On the other hand, technological progress has led to an increase in the variety of traffic data, and traffic flow detection devices that appear on urban roads are increasingly diverse. However, the data-driven intelligent traffic system is limited by the data quality of the existing urban road traffic data, and multi-source traffic data with uneven quality may have disastrous effects on later traffic signal management.
As a necessary premise for realizing accurate traffic control under the background of big data, the quality of the data fusion method design in the urban traffic big data greatly influences the quality of the input data of the later decision algorithm. Therefore, the design of an efficient data fusion system is an important point for improving the data quality and simplifying traffic data. However, the fusion data source types of most data fusion methods are single, and in the traffic field, abnormal situations are inevitably caused by the influence of environmental factors and the like on each traffic detection device, so that the accuracy of final data fusion is affected. Therefore, the invention provides a multi-source data fusion algorithm based on big data.
Disclosure of Invention
The present invention aims to solve at least one of the technical problems existing in the prior art. Therefore, the invention provides a multi-source data fusion algorithm based on big data.
To achieve the above object, an embodiment according to a first aspect of the present invention proposes a multi-source data fusion algorithm based on big data, comprising the steps of:
step one: inputting traffic initial data sets of a plurality of data sources, uniformly setting codes of the data sets to GBK codes, and marking data of different rows by adopting ID attributes in a first column of a file;
step two: preprocessing the traffic initial data set; the preprocessing is performed by eliminating unqualified information in the traffic initial data set;
step three: extracting features of the preprocessed traffic data information to obtain traffic feature data; the method comprises the following steps: firstly, different data in different data sources in any time period or any road section are acquired; obtaining traffic characteristic data according to the obtained different data;
step four: performing observation coefficient GF analysis on the traffic characteristic data to be fused, and performing fusion on the traffic characteristic data to be fused according to the data fusion terminals distributed with the corresponding quantity of the observation coefficients GF; wherein the fusion is based on an HFCM clustering algorithm;
step five: the data fusion terminal collects a plurality of traffic characteristic data for fusion to generate traffic fusion data; dividing urban traffic into a plurality of areas, acquiring road section traffic flow information of each area, carrying out centralized processing and analysis evaluation on the road section traffic flow information and traffic fusion data, and outputting traffic evaluation data information;
step six: in the data fusion process, monitoring and analyzing the calculation power occupation condition of the data fusion terminal, and calculating to obtain a calculation power saturation coefficient KY; and judging whether the computing power resources of the data fusion terminal need to be reallocated.
Further, the observation coefficient GF analysis is carried out on the traffic characteristic data to be fused, specifically:
acquiring a time section corresponding to traffic characteristic data to be fused, and calling a research attraction value of the corresponding time section to be YG; counting the data size of traffic characteristic data to be fused as D1;
calculating an observation coefficient GF of the traffic characteristic data to be fused by using a formula GF=YG×g1+D1×g2; wherein g1 and g2 are coefficient factors.
Further, determining the distribution number of the data fusion terminals as L1 according to the observation coefficient GF; the method comprises the following steps: obtaining a mapping relation table of a pre-constructed observation coefficient range and a terminal quantity threshold value; the mapping relation table of the observation coefficient range and the terminal quantity threshold value is generated based on a first configuration operation performed by the data fusion terminal in response to a user; and determining the terminal number threshold corresponding to the observation coefficient GF as L1 based on the observation coefficient GF and a mapping relation table of the observation coefficient range and the terminal number threshold.
Further, the method further comprises the following steps: the traffic evaluation data information is accessed and monitored, research attraction value analysis is carried out according to the access record, and the specific analysis steps are as follows:
acquiring access records of traffic evaluation data information within a preset time; the access record comprises an access start time and an access end time; acquiring a time section corresponding to traffic evaluation data information;
counting the access times of the time section as C1 for the same time section; accumulating the access time length of each access to obtain the total access time length ZT; the study attraction value YG of the time section was calculated using the formula yg=c1×a1+zt×a2, where a1, a2 are coefficient factors.
Further, the calculation power occupation condition of the data fusion terminal is monitored and analyzed, and the calculation power occupation condition is specifically:
from the initial moment, acquiring the calculated force occupancy rate of the data fusion terminal according to a preset interval and marking the calculated force occupancy rate as Nc, and establishing a graph of the change of the calculated force occupancy rate Nc along with time;
when the curve graph is in the ascending stage, deriving the curve graph to obtain an occupancy rate change rate curve graph; marking the real-time calculation power occupancy rate change rate of the data fusion terminal as Vt;
comparing the Vt with a preset rate threshold; if Vt is larger than a preset rate threshold, the data fusion terminal is busy in data fusion, and corresponding curve segments are intercepted in corresponding graphs for marking;
counting the number of marked curve segments as R1 in preset time, integrating all the marked curve segments with time to obtain marked reference energy WE, and calculating by using a formula WR=R1×d1+WE×d2 to obtain an operation heat value WR of the data fusion terminal, wherein d1 and d2 are coefficient factors;
acquiring the current calculation force occupancy rate of the data fusion terminal as Nt, and calculating to obtain a calculation force saturation coefficient KY of the core node by using a formula KY=Nt×d3+WR×d4, wherein d3 and d4 are coefficient factors;
comparing the calculated force saturation coefficient KY with a preset saturation threshold; if KY is larger than a preset saturation threshold, judging that the computational power resources of the data fusion terminal are insufficient, and generating a computational power expansion signal; to remind the manager to expand the computing power resources of the data fusion terminal.
Further, the plurality of data sources comprise mobile phone real-time moving speed information acquired from a mobile phone GPS, vehicle speed information of a road section where the road condition camera is located, and real-time vehicle position and running speed information acquired from a floating vehicle GPS.
Further, the disqualification information comprises traffic data information with the instantaneous speed of the vehicle being larger than a reasonable value, traffic data information with different video sampling time and storage time and traffic data information with the longitude and latitude of the vehicle exceeding a reasonable range in GPS positioning.
Further, the traffic fusion data comprise average speed information of vehicles and floating vehicles of the road section where all mobile phones and road condition cameras are located at any time and on any road section; the centralized processing specifically includes summarizing road section traffic flow information and traffic fusion information of each area, namely summarizing the traffic fusion information corresponding to each road section one by one, so as to evaluate the traffic data information of the area where the evaluation is located.
Compared with the prior art, the invention has the beneficial effects that:
1. according to the invention, traffic initial data sets of a plurality of data sources are input, codes of the data sets are uniformly set to be GBK codes, and ID attributes are adopted in a first column of a file to identify and distinguish data of different rows; preprocessing the traffic initial data set, and eliminating unqualified information; extracting features of the preprocessed traffic data information to obtain traffic feature data; performing observation coefficient GF analysis on the traffic characteristic data to be fused, and performing fusion on the traffic characteristic data to be fused according to the data fusion terminals distributed with the corresponding quantity of the observation coefficients GF; the data fusion efficiency is improved; wherein the fusion is based on an HFCM clustering algorithm; the method is used for mining the value of the multi-source heterogeneous data and realizing interconnection, intercommunication, exchange and sharing of the multi-source heterogeneous data;
2. the data fusion terminal acquires a plurality of traffic characteristic data for fusion to generate traffic fusion data, then divides urban traffic into a plurality of areas, acquires road section traffic flow information of each area, performs centralized processing and analysis evaluation on the road section traffic flow information and the traffic fusion data, and outputs traffic evaluation data information; access monitoring is carried out on the traffic evaluation data information, and research attraction value analysis is carried out according to access records; in the data fusion process, monitoring and analyzing the calculation power occupation condition of the data fusion terminal, and calculating to obtain a calculation power saturation coefficient KY; and judging whether the computing power resources of the data fusion terminal need to be redistributed or not, and improving the data fusion efficiency.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic block diagram of a multi-source data fusion algorithm based on big data according to the present invention.
Detailed Description
The technical solutions of the present invention will be clearly and completely described in connection with the embodiments, and it is obvious that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As shown in fig. 1, a multi-source data fusion algorithm based on big data includes the following steps:
step one: inputting traffic initial data sets of a plurality of data sources, uniformly setting codes of the data sets as GBK codes, and marking and distinguishing data of different rows by adopting ID attribute in a first column of a file so as to avoid the problem of repeated reading;
the plurality of data sources comprise mobile phone real-time moving speed information acquired from a mobile phone GPS, vehicle speed information of a road section where a road condition camera is located, and real-time vehicle position and running speed information acquired from a floating vehicle GPS;
step two: preprocessing the traffic initial data set; preprocessing is performed by eliminating unqualified information in the traffic initial data set; the unqualified information comprises traffic data information with the instantaneous speed of the vehicle being greater than a reasonable value, traffic data information with different video sampling time and storage time, and traffic data information with the longitude and latitude of the vehicle exceeding a reasonable range in GPS positioning; the unqualified information is mainly unreasonable information or information with overlarge data errors in traffic data information acquired by various data sources;
step three: extracting features of the preprocessed traffic data information to obtain traffic feature data; the method comprises the following steps: firstly, different data in different data sources in any time period or any road section are acquired; obtaining traffic characteristic data according to the obtained different data;
step four: performing observation coefficient GF analysis on the traffic characteristic data to be fused, and performing fusion on the traffic characteristic data to be fused according to the data fusion terminals distributed with the corresponding quantity of the observation coefficients GF; wherein the fusion is based on an HFCM clustering algorithm; the method is used for mining the value of the multi-source heterogeneous data and realizing interconnection, intercommunication, exchange and sharing of the multi-source heterogeneous data; the specific analysis steps are as follows:
acquiring a time section corresponding to traffic characteristic data to be fused, and calling a research attraction value of the corresponding time section to be YG; counting the data size of traffic characteristic data to be fused as D1;
calculating an observation coefficient GF of the traffic characteristic data to be fused by using a formula GF=YG×g1+D1×g2; wherein g1 and g2 are coefficient factors;
determining the distribution quantity of the data fusion terminals as L1 according to the observation coefficients GF; the method comprises the following steps:
obtaining a pre-constructed mapping relation table of the observation coefficient range and the terminal quantity threshold, wherein the mapping relation table of the observation coefficient range and the terminal quantity threshold is generated based on a first configuration operation performed by a data fusion terminal in response to a user;
based on the observation coefficient GF and a mapping relation table of the observation coefficient range and the terminal number threshold value, determining that the terminal number threshold value corresponding to the observation coefficient GF is L1;
step five: the data fusion terminal collects a plurality of traffic characteristic data for fusion and outputs traffic evaluation data information; the method comprises the following specific steps:
acquiring a plurality of traffic characteristic data for fusion to generate traffic fusion data; the traffic fusion data comprise average speed information of vehicles and floating vehicles of the road section where all mobile phones and road condition cameras are located at any time and on any road section;
dividing urban traffic into a plurality of areas, acquiring road section traffic flow information of each area, carrying out centralized processing and analysis evaluation on the road section traffic flow information and traffic fusion data, and outputting traffic evaluation data information;
the centralized processing is to summarize the road section traffic flow information and the traffic fusion information of each area, namely summarize the traffic fusion information corresponding to each road section one by one, so as to evaluate the traffic data information of the area;
step six: in the data fusion process, monitoring and analyzing the calculation power occupation condition of the data fusion terminal, and calculating to obtain a calculation power saturation coefficient KY; judging whether the computing power resource of the data fusion terminal needs to be redistributed or not, and improving the data fusion efficiency; the specific analysis steps are as follows:
from the initial moment, acquiring the calculated force occupancy rate of the data fusion terminal according to a preset interval and marking the calculated force occupancy rate as Nc, and establishing a graph of the change of the calculated force occupancy rate Nc along with time;
when the curve graph is in the ascending stage, deriving the curve graph to obtain an occupancy rate change rate curve graph; marking the real-time calculation power occupancy rate change rate of the data fusion terminal as Vt;
comparing the Vt with a preset rate threshold; if Vt is larger than a preset rate threshold, the data fusion terminal is busy in data fusion, and corresponding curve segments are intercepted in corresponding graphs for marking;
counting the number of marked curve segments as R1 in preset time, integrating all the marked curve segments with time to obtain marked reference energy WE, and calculating by using a formula WR=R1×d1+WE×d2 to obtain an operation heat value WR of the data fusion terminal, wherein d1 and d2 are coefficient factors;
acquiring the current calculation force occupancy rate of the data fusion terminal as Nt, and calculating to obtain a calculation force saturation coefficient KY of the core node by using a formula KY=Nt×d3+WR×d4, wherein d3 and d4 are coefficient factors;
comparing the calculated force saturation coefficient KY with a preset saturation threshold; if KY is larger than a preset saturation threshold, judging that the computational power resources of the data fusion terminal are insufficient, and generating a computational power expansion signal; the management personnel is reminded to expand the computing power resource of the data fusion terminal, so that the data fusion efficiency is improved;
the further technical scheme is that the method further comprises the following steps: the traffic evaluation data information is accessed and monitored, research attraction value analysis is carried out according to the access record, and the specific analysis steps are as follows:
acquiring access records of traffic evaluation data information within a preset time; the access record comprises an access start time and an access end time; acquiring a time section corresponding to traffic evaluation data information;
counting the access times of the time section as C1 for the same time section; accumulating the access time length of each access to obtain the total access time length ZT; the study attraction value YG of the time section was calculated using the formula yg=c1×a1+zt×a2, where a1, a2 are coefficient factors.
The above formulas are all formulas with dimensions removed and numerical values calculated, the formulas are formulas which are obtained by acquiring a large amount of data and performing software simulation to obtain the closest actual situation, and preset parameters and preset thresholds in the formulas are set by a person skilled in the art according to the actual situation or are obtained by simulating a large amount of data.
The working principle of the invention is as follows:
when the multi-source data fusion algorithm based on big data works, traffic initial data sets of a plurality of data sources are input, codes of the data sets are uniformly set to be GBK codes, and ID attributes are adopted in a first column of a file to identify and distinguish data of different rows; preprocessing the traffic initial data set, and eliminating unqualified information; extracting features of the preprocessed traffic data information to obtain traffic feature data; performing observation coefficient GF analysis on the traffic characteristic data to be fused, and performing fusion on the traffic characteristic data to be fused according to the data fusion terminals distributed with the corresponding quantity of the observation coefficients GF; the data fusion efficiency is improved; wherein the fusion is based on an HFCM clustering algorithm; the method is used for mining the value of the multi-source heterogeneous data and realizing interconnection, intercommunication, exchange and sharing of the multi-source heterogeneous data;
the data fusion terminal collects a plurality of traffic characteristic data for fusion to generate traffic fusion data, then urban traffic is divided into a plurality of areas, road section traffic flow information of each area is obtained, and the road section traffic flow information and the traffic fusion data are processed in a centralized manner, analyzed and evaluated, and traffic evaluation data information is output; access monitoring is carried out on the traffic evaluation data information, and research attraction value analysis is carried out according to access records; in the data fusion process, monitoring and analyzing the calculation power occupation condition of the data fusion terminal, and calculating to obtain a calculation power saturation coefficient KY; and judging whether the computing power resources of the data fusion terminal need to be redistributed or not, and improving the data fusion efficiency.
In the description of the present specification, the descriptions of the terms "one embodiment," "example," "specific example," and the like, mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The preferred embodiments of the invention disclosed above are intended only to assist in the explanation of the invention. The preferred embodiments are not intended to be exhaustive or to limit the invention to the precise form disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best understand and utilize the invention. The invention is limited only by the claims and the full scope and equivalents thereof.
Claims (4)
1. The multi-source data fusion algorithm based on big data is characterized by comprising the following steps:
step one: inputting traffic initial data sets of a plurality of data sources, uniformly setting codes of the data sets to be GBK codes, and marking and distinguishing data of different rows by adopting ID attributes in a first column of a file;
step two: preprocessing the traffic initial data set; the preprocessing is performed by eliminating unqualified information in the traffic initial data set;
step three: extracting features of the preprocessed traffic data information to obtain traffic feature data; the method comprises the following steps: firstly, different data in different data sources in any time period or any road section are acquired; obtaining traffic characteristic data according to the obtained different data;
step four: performing observation coefficient GF analysis on the traffic characteristic data to be fused, and performing fusion on the traffic characteristic data to be fused according to the data fusion terminals distributed with the corresponding quantity of the observation coefficients GF; wherein the fusion is based on an HFCM clustering algorithm; the specific analysis steps are as follows:
acquiring a time section corresponding to traffic characteristic data to be fused, and calling a research attraction value of the corresponding time section to be YG; counting the data size of traffic characteristic data to be fused as D1;
calculating an observation coefficient GF of the traffic characteristic data to be fused by using a formula GF=YG×g1+D1×g2; wherein g1 and g2 are coefficient factors;
determining the distribution quantity of the data fusion terminals as L1 according to the observation coefficients GF; the method comprises the following steps:
obtaining a mapping relation table of a pre-constructed observation coefficient range and a terminal quantity threshold value; the mapping relation table of the observation coefficient range and the terminal quantity threshold value is generated based on a first configuration operation performed by the data fusion terminal in response to a user;
based on the observation coefficient GF and a mapping relation table of the observation coefficient range and the terminal number threshold value, determining that the terminal number threshold value corresponding to the observation coefficient GF is L1;
step five: the data fusion terminal collects a plurality of traffic characteristic data for fusion to generate traffic fusion data; dividing urban traffic into a plurality of areas, acquiring road section traffic flow information of each area, carrying out centralized processing and analysis evaluation on the road section traffic flow information and traffic fusion data, and outputting traffic evaluation data information;
step six: in the data fusion process, monitoring and analyzing the calculation power occupation condition of the data fusion terminal, and calculating to obtain a calculation power saturation coefficient KY; judging whether the computing power resources of the data fusion terminal need to be redistributed or not; the specific analysis steps are as follows:
from the initial moment, acquiring the calculated force occupancy rate of the data fusion terminal according to a preset interval and marking the calculated force occupancy rate as Nc, and establishing a graph of the change of the calculated force occupancy rate Nc along with time;
when the curve graph is in the ascending stage, deriving the curve graph to obtain an occupancy rate change rate curve graph; marking the real-time calculation power occupancy rate change rate of the data fusion terminal as Vt;
comparing the Vt with a preset rate threshold; if Vt is larger than a preset rate threshold, the data fusion terminal is busy in data fusion, and corresponding curve segments are intercepted in corresponding graphs for marking;
counting the number of marked curve segments as R1 in preset time, integrating all the marked curve segments with time to obtain marked reference energy WE, and calculating by using a formula WR=R1×d1+WE×d2 to obtain an operation heat value WR of the data fusion terminal, wherein d1 and d2 are coefficient factors;
acquiring the current calculation force occupancy rate of the data fusion terminal as Nt, and calculating to obtain a calculation force saturation coefficient KY of the core node by using a formula KY=Nt×d3+WR×d4, wherein d3 and d4 are coefficient factors;
comparing the calculated force saturation coefficient KY with a preset saturation threshold; if KY is larger than a preset saturation threshold, judging that the computational power resources of the data fusion terminal are insufficient, and generating a computational power expansion signal; to remind the manager to expand the computing power resource of the data fusion terminal;
the multi-source data fusion algorithm further comprises: the traffic evaluation data information is accessed and monitored, research attraction value analysis is carried out according to the access record, and the specific analysis steps are as follows:
acquiring access records of traffic evaluation data information within a preset time; the access record comprises an access start time and an access end time; acquiring a time section corresponding to traffic evaluation data information;
counting the access times of the time section as C1 for the same time section; accumulating the access time length of each access to obtain the total access time length ZT; the study attraction value YG of the time section was calculated using the formula yg=c1×a1+zt×a2, where a1, a2 are coefficient factors.
2. The big data based multi-source data fusion algorithm of claim 1, wherein the plurality of data sources includes real-time mobile speed information of the mobile phone obtained from the mobile phone GPS, vehicle speed information of the road section where the road condition camera is located, and real-time vehicle position and traveling speed information obtained from the floating vehicle GPS.
3. The big data based multi-source data fusion algorithm of claim 1, wherein the disqualification information includes traffic data information that the instantaneous speed of the vehicle is greater than a reasonable value, traffic data information that the video sampling time and the storage time are different, and traffic data information that the longitude and latitude of the vehicle in GPS positioning exceeds a reasonable range.
4. The multi-source data fusion algorithm based on big data according to claim 1, wherein the traffic fusion data comprises average speed information of vehicles and floating vehicles of the road section where all mobile phones and road condition cameras on any road section acquire at any time; the centralized processing specifically includes summarizing road section traffic flow information and traffic fusion information of each area, namely summarizing traffic fusion information corresponding to each road section, so as to evaluate traffic data information of the area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310262880.9A CN116189436B (en) | 2023-03-17 | 2023-03-17 | Multi-source data fusion algorithm based on big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310262880.9A CN116189436B (en) | 2023-03-17 | 2023-03-17 | Multi-source data fusion algorithm based on big data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116189436A CN116189436A (en) | 2023-05-30 |
| CN116189436B true CN116189436B (en) | 2023-12-29 |
Family
ID=86434580
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310262880.9A Active CN116189436B (en) | 2023-03-17 | 2023-03-17 | Multi-source data fusion algorithm based on big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116189436B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116628451B (en) * | 2023-05-31 | 2023-11-14 | 江苏华存电子科技有限公司 | High-speed analysis method for information to be processed |
| CN117390495B (en) * | 2023-12-04 | 2024-02-20 | 江苏睿希信息科技有限公司 | Multi-source data risk management system and method based on big data |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8369622B1 (en) * | 2009-10-29 | 2013-02-05 | Hsu Shin-Yi | Multi-figure system for object feature extraction tracking and recognition |
| WO2021135879A1 (en) * | 2020-09-07 | 2021-07-08 | 平安科技（深圳）有限公司 | Vehicle data monitoring method and apparatus, computer device, and storage medium |
| CN114399006A (en) * | 2022-03-24 | 2022-04-26 | 山东省计算中心（国家超级计算济南中心） | Multi-source abnormal composition image data fusion method and system based on super-calculation |
| CN115359332A (en) * | 2022-09-05 | 2022-11-18 | 深圳海星智驾科技有限公司 | Data fusion method and device based on vehicle-road cooperation, electronic equipment and system |
-
2023
- 2023-03-17 CN CN202310262880.9A patent/CN116189436B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8369622B1 (en) * | 2009-10-29 | 2013-02-05 | Hsu Shin-Yi | Multi-figure system for object feature extraction tracking and recognition |
| WO2021135879A1 (en) * | 2020-09-07 | 2021-07-08 | 平安科技（深圳）有限公司 | Vehicle data monitoring method and apparatus, computer device, and storage medium |
| CN114399006A (en) * | 2022-03-24 | 2022-04-26 | 山东省计算中心（国家超级计算济南中心） | Multi-source abnormal composition image data fusion method and system based on super-calculation |
| CN115359332A (en) * | 2022-09-05 | 2022-11-18 | 深圳海星智驾科技有限公司 | Data fusion method and device based on vehicle-road cooperation, electronic equipment and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116189436A (en) | 2023-05-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116189436B (en) | Multi-source data fusion algorithm based on big data | |
| CN107798876B (en) | Road traffic abnormal jam judging method based on event | |
| CN108417033B (en) | Expressway traffic accident analysis and prediction method based on multi-dimensional factors | |
| CN106777093A (en) | Skyline inquiry systems based on space time series data stream application | |
| CN106708989A (en) | Spatial time sequence data stream application-based Skyline query method | |
| CN113570867B (en) | Urban traffic state prediction method, device, equipment and readable storage medium | |
| CN111445369A (en) | Urban large-scale gathering activity intelligence early warning method and device based on L BS big data | |
| CN112185108B (en) | Urban road network congestion mode identification method, equipment and medium based on space-time characteristics | |
| CN115171385A (en) | Traffic incident detection system based on millimeter wave radar and video linkage | |
| CN108681791B (en) | People flow density prediction method, device and storage medium | |
| CN113256978A (en) | Method and system for diagnosing urban congestion area and storage medium | |
| CN112330953A (en) | Traffic control method, device and equipment based on cloud data and storage medium | |
| CN113516850B (en) | Pipeline traffic flow data acquisition method based on space syntactic analysis | |
| CN108597224B (en) | Method and system for identifying to-be-improved traffic facilities based on space-time trajectory data | |
| CN108460424B (en) | Spatial motion trajectory identification method based on line point conversion and cluster comparison | |
| CN116362036A (en) | High-speed rail equipment fault early warning method and system | |
| JPWO2023195147A5 (en) | ||
| CN114564521A (en) | Method and system for determining working time period of agricultural machine based on clustering algorithm | |
| CN113421067A (en) | Shared bicycle brand identification and quantity management method and system | |
| CN114117121A (en) | Data acquisition method for smart city | |
| CN112153464A (en) | Smart city management system | |
| CN111752991A (en) | Meter data statistical method and device and meter statistical data acquisition method | |
| CN110097756B (en) | Intelligent traffic management method based on multi-source data fusion technology | |
| CN116401944B (en) | Intelligent traffic management system based on big data | |
| Karnati et al. | Data Mining Algorithms for Traffic Interruption Detection. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20231208 Address after: B02, 15th Floor, Jinqiu International Building, No. 6 Zhichun Road, Haidian District, Beijing, 100000 Applicant after: BEIJING LOGOS DATA TECHNOLOGY Co.,Ltd. Address before: Room 410, 4th Floor, No. 7 Pulan Road, Shengli Village, Shiwan Town Street, Chancheng District, Foshan City, Guangdong Province, 528000 Applicant before: Foshan Zhonghe Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |