CN115098601A - Highway data receiving, processing and storing method and system - Google Patents

Abstract

A highway data receiving, processing and storing method and a system thereof comprise the following steps: receiving monitoring data of the highway; classifying and extracting the monitoring data according to a classification rule to obtain extracted data; setting the extracted data into a corresponding classification database to obtain stored data; and the classification rule is established according to the data format of the classification database. The method is based on the background that provincial toll stations are cancelled to realize one national network, and due to the fact that various toll portals, toll lanes and overtaking control systems in the expressway have a plurality of manufacturers and different protocol standards, corresponding protocol development is often needed for the manufacturers, and a large amount of repetitive work is caused.

Description

Highway data receiving, processing and storing method and system

Technical Field

The application relates to a method and a system for receiving, processing and storing highway data.

Background

Under the prospect of eliminating provincial toll stations to realize one network in China, various toll portals, toll lanes and overtaking control systems in the expressway have a plurality of manufacturers and different protocol standards, and corresponding protocol development is often required for the manufacturers, so that a large amount of repetitive labor is caused. Some storage methods by data analysis are available, but the overall efficiency is low, and errors are likely to occur in the process of unloading, and the method is not suitable for being used in the field with high requirements of highway data.

Disclosure of Invention

In order to solve the above problems, the present application discloses, on one hand, a method for receiving, processing and storing highway data, comprising the following steps:

receiving monitoring data of a highway;

classifying and extracting the monitoring data according to a classification rule to obtain extracted data;

setting the extracted data into a corresponding classification database to obtain stored data;

and the classification rule is established according to the data format of the classification database. The method is based on the background that provincial toll stations are cancelled to realize one national network, and due to the fact that various toll portals, toll lanes and overtaking control systems in the expressway have a plurality of manufacturers and different protocol standards, corresponding protocol development is often needed for the manufacturers, and a large amount of repetitive work is caused.

Preferably, the method further comprises a process of creating a classification database and classification rules, comprising the following steps: and (4) establishing a classification database according to the monitoring data, and then determining a classification rule according to the data structure of the classification database.

Preferably, the classification extraction is obtained by comprehensive judgment according to data sources and data self-identification analysis.

Preferably, the comprehensive evaluation is performed as follows: the data source acquires and confirms according to an original source provided by the monitoring data, and performs pre-classification on the monitoring data according to historical information of the monitoring data provided by the original source;

performing data matching on target data and data in a classified database of a type obtained by pre-classification by extracting the target data in the monitoring data, and performing data matching on the target data and the data in other classified databases if the matching rate of the data matching is lower than a threshold value until the matching rate of the data matching is not lower than the threshold value;

if the classification databases meeting the requirements are not obtained in the process, if the matching rate of at least one of the classification databases exceeds 60%, the extracted data is written into the classification database with the highest matching rate, and the extracted data is marked to obtain the standard data. The comprehensive evaluation of the application judges the data type through the data source, can ensure the reduction of the required time when the data is judged and the accuracy of the classification of the monitoring data, obtains the standard data and provides a basis for the effective expansion of the classification database.

Preferably, the data matching includes a numerical matching and a data representation matching.

Preferably, the method further comprises analyzing and monitoring specific data in the stored data: calculating to obtain a median of the stored data, and determining that the data in the range of 30% above and below the median are common data; taking the common data at the boundary as a base point, carrying out outward expansion calculation, if the numerical value increase and decrease change rate of the two sides of the common data is lower than 10% of that of the common data serving as the calculation base point, marking the adjacent stored data as the common data, repeating the steps until more than 10% of the stored data are found or all the check calculation is completed, and marking more than 10% of the stored data as specific data; and performing source verification and data extraction accuracy verification on the specific data. According to the method and the device, the data extraction accuracy is judged through self judgment of internal data, so that addition and accumulation of specific data are avoided, and the accuracy of finally obtained storage data is improved.

Preferably, the method further comprises the following processing procedures of the identification data: and when the data volume of the standard and different data in the same classification database exceeds the data volume threshold or the percentage of the standard and different data in the same classification database exceeds the percentage threshold, establishing a new classification database through the data source of the standard and different data and the meaning of the standard and different data.

Preferably, the monitoring data comprises lane monitoring data, gantry monitoring data and overload monitoring data.

Preferably, the database is a JSON database.

On the other hand, the application also discloses a highway data receiving, processing and storing system, which comprises the following modules:

the data receiving module is used for receiving monitoring data of the highway;

the data extraction module is used for classifying and extracting the monitoring data according to a classification rule to obtain extracted data;

the data classification module is used for setting the extracted data into a corresponding classification database;

and the configuration module is used for setting the classification database and the classification rules.

This application can bring following beneficial effect:

1. the method is based on the background of canceling provincial toll stations to realize one national network, and as various toll portals, toll lanes and overtaking control systems in the expressway have a plurality of manufacturers and different protocol standards, corresponding protocol development is often required to be carried out aiming at the manufacturers, so that a large amount of repetitive labor is caused;

2. the comprehensive evaluation of the application judges the data type through the data source, can ensure the reduction of the time required for judging the data and the accuracy of monitoring data classification, obtains the standard and different data and provides a basis for the effective expansion of a classification database;

3. according to the method and the device, the data extraction accuracy is judged through self judgment of internal data, so that addition and accumulation of specific data are avoided, and the accuracy of finally obtained storage data is improved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

FIG. 1 is a schematic view of example 1;

FIG. 2 is a schematic view of example 2.

Detailed Description

In order to clearly illustrate the technical features of the present solution, the present application will be explained in detail through the following embodiments.

In a first embodiment, as shown in fig. 1, a method for receiving, processing and storing highway data includes the following steps:

s101, receiving monitoring data of a highway;

the monitoring data comprises lane monitoring data, portal monitoring data and overload control monitoring data.

S102, classifying and extracting monitoring data according to classification rules to obtain extracted data, wherein the classification rules are established according to data formats of a classification database;

the method also comprises a process of creating a classification database and classification rules, and comprises the following steps:

and newly building a classification database according to the monitoring data, and then determining a classification rule according to a data structure of the classification database.

The classified extraction is obtained by comprehensive judgment according to data sources and data self-identification analysis.

The comprehensive evaluation is carried out according to the following modes:

the data source acquires and confirms according to an original source provided by the monitoring data, and performs pre-classification on the monitoring data according to historical information of the monitoring data provided by the original source; for example, if the historical information shows that the traffic lane monitoring data is more than the traffic lane monitoring data, the traffic lane monitoring data is classified according to the historical information;

performing data matching on target data and data in a classified database of a type obtained by pre-classification by extracting the target data in the monitoring data, and performing data matching on the target data and the data in other classified databases if the matching rate of the data matching is lower than a threshold value, if the threshold value is 80%, until the matching rate of the data matching is not lower than the threshold value;

if the classification databases meeting the requirements are not obtained in the process, if the matching rate of at least one of the classification databases exceeds 60%, the extracted data is written into the classification database with the highest matching rate, and the extracted data is marked to obtain the standard and different data.

The data matching comprises numerical value matching and data representation matching. If the same value or 5% above or below the value exists, the matching rate may be defined as 100%, and if there is no data within the range, or if it exceeds 5%, the matching rate is set to 90%, 85%, or the like according to the degree of the exceeding.

And (3) processing the labeled data:

when the data volume of the marked data in the same classification database exceeds a data volume threshold (for example, more than 10000) or the percentage of the marked data in the same classification database exceeds a percentage threshold (for example, more than 20%, that is, the whole data is 10000, and the marked data is 2000), a new classification database is established through the data source and the self meaning of the marked data.

S103, setting the extracted data into a corresponding classification database to obtain stored data;

analytical monitoring of specific data among the stored data:

calculating to obtain a median of the stored data, and determining that the data in the range of 30% above and below the median are common data;

taking the common data at the boundary as a base point, carrying out outward expansion calculation, if the numerical value increase and decrease change rate of the two sides of the common data is lower than 10% of that of the common data serving as the calculation base point, marking the adjacent stored data as the common data, repeating the steps until more than 10% of the stored data are found or all the check calculation is completed, and marking more than 10% of the stored data as specific data;

if the value of the normal data at the boundary is 100, the value of the nearest normal data is 101, and the value to be verified is 99, the following calculation is performed: [ (100-99) - (101-99) ]/100 ═ 0, then mark it as normal data, and do the validation back and forth.

The source verification (namely, the original source) and the verification of the data extraction accuracy are performed on the specific data (for the data accuracy, manual data extraction can be performed, and data re-extraction can also be performed by means of re-extraction and the like).

It will be appreciated that the database is a JSON database.

In a second embodiment, as shown in fig. 2, a highway data receiving, processing and storing system includes the following modules:

a data receiving module 201, configured to receive monitoring data of an expressway;

the data extraction module 202 is configured to perform classification extraction on the monitoring data according to a classification rule to obtain extracted data;

the data classification module 203 is used for setting the extracted data into a corresponding classification database;

and the configuration module 204 is used for setting the classification database and the classification rules.

The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art to which the present application pertains. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims (10)

1. A highway data receiving, processing and storing method is characterized in that: the method comprises the following steps:

receiving monitoring data of the highway;

classifying and extracting the monitoring data according to a classification rule to obtain extracted data;

setting the extracted data into a corresponding classification database to obtain stored data;

and the classification rule is established according to the data format of the classification database.

2. The highway data receiving, processing and storing method according to claim 1, wherein the highway data receiving, processing and storing method comprises the following steps: the process of creating a classification database and classification rules is also included, including the steps of:

and newly building a classification database according to the monitoring data, and then determining a classification rule according to a data structure of the classification database.

3. The highway data receiving, processing and storing method according to claim 1, wherein the highway data receiving, processing and storing method comprises the following steps: the classified extraction is obtained by comprehensive judgment according to data sources and data self-identification analysis.

4. The highway data receiving, processing and storing method according to claim 3, wherein the highway data receiving, processing and storing method comprises the following steps: the comprehensive evaluation is carried out according to the following modes:

the data source acquires and confirms according to an original source provided by the monitoring data, and performs pre-classification on the monitoring data according to historical information of the monitoring data provided by the original source;

performing data matching on target data and data in a classified database of a type obtained by pre-classification by extracting the target data in the monitoring data, and performing data matching on the target data and the data in other classified databases if the matching rate of the data matching is lower than a threshold value until the matching rate of the data matching is not lower than the threshold value;

if the classification databases meeting the requirements are not obtained in the process, if the matching rate of at least one of the classification databases exceeds 60%, the extracted data is written into the classification database with the highest matching rate, and the extracted data is marked to obtain the standard and different data.

5. The method for receiving, processing and storing the highway data according to claim 4, wherein the method comprises the following steps: the data matching comprises numerical value matching and data representation matching.

6. The method for receiving, processing and storing the highway data according to claim 1, wherein the method comprises the following steps: also included is the analytical monitoring of specific data among the stored data:

calculating to obtain a median of the stored data, and determining that the data in the range of 30% above and below the median are common data;

taking the common data at the boundary as a base point, carrying out outward expansion calculation, if the numerical value increase and decrease change rate of the two sides of the common data is lower than 10% of that of the common data serving as the calculation base point, marking the adjacent stored data as the common data, repeating the steps until more than 10% of the stored data are found or all the check calculation is completed, and marking more than 10% of the stored data as specific data;

and performing source verification and data extraction accuracy verification on the specific data.

7. The highway data receiving, processing and storing method according to claim 3, wherein the highway data receiving, processing and storing method comprises the following steps: the method also comprises a processing procedure of the marking data:

and when the data volume of the standard and different data in the same classification database exceeds the data volume threshold or the percentage of the standard and different data in the same classification database exceeds the percentage threshold, establishing a new classification database through the data source of the standard and different data and the meaning of the standard and different data.

8. The highway data receiving, processing and storing method according to claim 1, wherein the highway data receiving, processing and storing method comprises the following steps: the monitoring data comprises lane monitoring data, portal monitoring data and overload control monitoring data.

9. The highway data receiving, processing and storing method according to claim 1, wherein the highway data receiving, processing and storing method comprises the following steps: the database is a JSON database.

10. A highway data receiving, processing and storing system is characterized in that: the system comprises the following modules:

the data receiving module is used for receiving monitoring data of the highway;

the data extraction module is used for classifying and extracting the monitoring data according to a classification rule to obtain extracted data;

the data classification module is used for setting the extracted data into a corresponding classification database;

and the configuration module is used for setting the classification database and the classification rules.

Images