Disclosure of Invention
The technical problem to be solved by the invention is to provide a chart data resource management method and system based on data verification, which can effectively improve the data accuracy and management efficiency of chart data and improve the security of chart data storage.
In order to solve the technical problem, the first aspect of the invention discloses a chart data resource management method based on data verification, which comprises the following steps:
acquiring a plurality of chart data corresponding to a target ocean area;
determining the credibility parameter of each chart data according to each chart data and the corresponding data acquisition parameter;
cross-verifying all the chart data according to the credibility parameter to obtain verified chart data corresponding to the target ocean area;
and dividing the verification chart data to obtain a plurality of data parts, and storing the plurality of data parts to a plurality of server nodes.
As an optional implementation manner, in the first aspect of the present invention, the data acquisition parameters include a sensing data acquisition device parameter, a sensing data acquisition position, a model data calculation device parameter, a model data calculation time and a model data calculation energy consumption; the sensor data acquisition device parameters or the model data calculation device parameters include a device model number, a device performance parameter, and a device location.
As an optional implementation manner, in the first aspect of the present invention, the determining, according to each of the chart data and the corresponding data acquisition parameter, a reliability parameter of each of the chart data includes:
Inputting the chart data and the corresponding data acquisition parameters into a trained reliability prediction neural network for each chart data to obtain reliability prediction probability corresponding to the chart data; the credibility prediction neural network is obtained through training a training data set comprising a plurality of training chart data and corresponding data acquisition parameter labels and credibility labels;
Calculating the average value of the sea chart difference degree between the sea chart data and all other sea chart data to obtain a difference degree parameter corresponding to the sea chart data;
calculating a complexity difference between the data complexity corresponding to the chart data and a reference complexity threshold corresponding to the target ocean area;
Calculating the product of the reliability prediction probability corresponding to the chart data and the first weight and the second weight to obtain reliability parameters of the chart data; the first weight is inversely proportional to the variability parameter; the second weight is inversely proportional to the complexity difference.
As an optional implementation manner, in the first aspect of the present invention, the chart difference degree is calculated by the following rule:
For the two sea chart data, calculating the boundary difference degree between land boundary objects in the two sea chart data;
calculating the regional difference degree between the marine barrier regional objects of the two sea chart data;
Calculating a float difference degree between the float objects of the two sea chart data;
Calculating a weighted sum value among the boundary difference degree, the region difference degree and the floating object difference degree to obtain a chart difference degree between two chart data; the boundary difference degree, the region difference degree and the weight corresponding to the floating object difference degree are sequentially reduced.
As an optional implementation manner, in the first aspect of the present invention, the data complexity is calculated by the following rule:
Calculating the data type number of all data in the chart data;
Determining all trivial objects in the sea chart data; the size of the data volume of the trivial object is lower than a data volume threshold or the size of the area of the object is lower than an area threshold;
Calculating the total number of objects of all the trivial objects;
calculating the discrete degree of the object position distribution of all the trivial objects;
Calculating the weighted sum value of the data type quantity, the object total quantity and the discrete degree to obtain the data complexity corresponding to the chart data; wherein the degree of discretization, the total number of objects, and the weight corresponding to the number of data types decrease in order.
In a first aspect of the present invention, the cross-verifying all the chart data according to the reliability parameter to obtain verified chart data corresponding to the target ocean area includes:
Screening all the chart data with the credibility parameter larger than a first credibility threshold value to obtain a plurality of preferable chart data;
carrying out data average calculation on all the preferred chart data to obtain basic verification chart data;
rejecting all the chart data with the credibility parameter smaller than a second credibility threshold;
Screening all the chart data with the credibility parameter smaller than the first credibility threshold and larger than the second credibility threshold to obtain a plurality of swinging chart data;
judging whether the total quantity of all the swing chart data is larger than a preset quantity threshold value, if not, determining the basic verification chart data as verification chart data corresponding to the target ocean area;
And if so, correcting the basic verification chart data based on the swing chart data to obtain verification chart data corresponding to the target ocean area.
In a first aspect of the present invention, the correcting the basic verification chart data based on the rocking chart data to obtain verification chart data corresponding to the target ocean area includes:
Calculating boundary average values of all the land boundary objects in the swing chart data and the land boundary objects in the basic verification chart data so as to correct the land boundary objects in the basic verification chart data;
ignoring all floating objects in the rocking chart data;
Judging whether the regional difference degree between the swing chart data and the marine obstacle region object of the basic verification chart data is larger than a difference degree threshold value or not for each swing chart data, if not, ignoring the marine obstacle region object in the swing chart data, and if yes, determining that the swing chart data is obstacle-related chart data;
calculating weighted regional boundary average values of all the marine barrier region objects in the barrier-related chart data and the marine barrier region objects in the basic verification chart data so as to correct land boundary objects in the basic verification chart data; wherein the mean computation weight corresponding to each sea obstacle region is proportional to the reliability parameter of the corresponding sea chart data.
In a first aspect of the present invention, the dividing the verification chart data into a plurality of data portions, storing the plurality of data portions in a plurality of server nodes includes:
Dividing the verification chart data according to a preset region subdivision rule corresponding to the target ocean region to obtain a plurality of region chart data;
dividing each regional chart data according to a preset data type dividing rule to obtain a plurality of regional data subdivision data;
Acquiring the equipment data transmission rate of the current computing equipment and the node space and the node transmission distance corresponding to each server node;
setting an objective function to the maximum number of different areas and different data types corresponding to the allocation data corresponding to each server node;
setting limiting conditions, wherein the limiting conditions comprise that more distribution data of the server nodes with larger node space, fewer distribution data of the server nodes with larger node transmission distance and the sum of transmission time of all distribution data are smaller than a preset time threshold; the transmission time is calculated according to the equipment data transmission rate and the node transmission distance of the corresponding server node;
Based on a dynamic programming algorithm, carrying out iterative allocation calculation on all the area data subdivision data according to the objective function and the limiting condition until an optimal allocation scheme is obtained; the distribution scheme comprises server nodes corresponding to different regional data subdivision data;
And sending each area data subdivision data to a corresponding server node for storage.
The second aspect of the embodiment of the invention discloses a chart data resource management system based on data verification, which comprises the following steps:
the acquisition module is used for acquiring a plurality of chart data corresponding to the target ocean area;
the determining module is used for determining the credibility parameter of each chart data according to each chart data and the corresponding data acquisition parameter;
The verification module is used for carrying out cross verification on all the chart data according to the credibility parameter so as to obtain verification chart data corresponding to the target ocean area;
And the storage module is used for dividing the verification chart data to obtain a plurality of data parts and storing the plurality of data parts to a plurality of server nodes.
As an optional implementation manner, in the second aspect of the present invention, the data acquisition parameters include a sensing data acquisition device parameter, a sensing data acquisition position, a model data calculation device parameter, a model data calculation time and a model data calculation energy consumption; the sensor data acquisition device parameters or the model data calculation device parameters include a device model number, a device performance parameter, and a device location.
In a second aspect of the present invention, the determining module determines, according to each of the chart data and the corresponding data acquisition parameter, a specific mode of the reliability parameter of each of the chart data, including:
Inputting the chart data and the corresponding data acquisition parameters into a trained reliability prediction neural network for each chart data to obtain reliability prediction probability corresponding to the chart data; the credibility prediction neural network is obtained through training a training data set comprising a plurality of training chart data and corresponding data acquisition parameter labels and credibility labels;
Calculating the average value of the sea chart difference degree between the sea chart data and all other sea chart data to obtain a difference degree parameter corresponding to the sea chart data;
calculating a complexity difference between the data complexity corresponding to the chart data and a reference complexity threshold corresponding to the target ocean area;
Calculating the product of the reliability prediction probability corresponding to the chart data and the first weight and the second weight to obtain reliability parameters of the chart data; the first weight is inversely proportional to the variability parameter; the second weight is inversely proportional to the complexity difference.
As an alternative embodiment, in the second aspect of the present invention, the chart difference degree is calculated by the following rule:
For the two sea chart data, calculating the boundary difference degree between land boundary objects in the two sea chart data;
calculating the regional difference degree between the marine barrier regional objects of the two sea chart data;
Calculating a float difference degree between the float objects of the two sea chart data;
Calculating a weighted sum value among the boundary difference degree, the region difference degree and the floating object difference degree to obtain a chart difference degree between two chart data; the boundary difference degree, the region difference degree and the weight corresponding to the floating object difference degree are sequentially reduced.
As an alternative embodiment, in the second aspect of the present invention, the data complexity is calculated by the following rule:
Calculating the data type number of all data in the chart data;
Determining all trivial objects in the sea chart data; the size of the data volume of the trivial object is lower than a data volume threshold or the size of the area of the object is lower than an area threshold;
Calculating the total number of objects of all the trivial objects;
calculating the discrete degree of the object position distribution of all the trivial objects;
Calculating the weighted sum value of the data type quantity, the object total quantity and the discrete degree to obtain the data complexity corresponding to the chart data; wherein the degree of discretization, the total number of objects, and the weight corresponding to the number of data types decrease in order.
In a second aspect of the present invention, the method for cross-verifying all the chart data according to the reliability parameter to obtain verified chart data corresponding to the target ocean area includes:
Screening all the chart data with the credibility parameter larger than a first credibility threshold value to obtain a plurality of preferable chart data;
carrying out data average calculation on all the preferred chart data to obtain basic verification chart data;
rejecting all the chart data with the credibility parameter smaller than a second credibility threshold;
Screening all the chart data with the credibility parameter smaller than the first credibility threshold and larger than the second credibility threshold to obtain a plurality of swinging chart data;
judging whether the total quantity of all the swing chart data is larger than a preset quantity threshold value, if not, determining the basic verification chart data as verification chart data corresponding to the target ocean area;
And if so, correcting the basic verification chart data based on the swing chart data to obtain verification chart data corresponding to the target ocean area.
In a second aspect of the present invention, the specific manner in which the verification module corrects the basic verification chart data based on the rocking chart data to obtain the verification chart data corresponding to the target ocean area includes:
Calculating boundary average values of all the land boundary objects in the swing chart data and the land boundary objects in the basic verification chart data so as to correct the land boundary objects in the basic verification chart data;
ignoring all floating objects in the rocking chart data;
Judging whether the regional difference degree between the swing chart data and the marine obstacle region object of the basic verification chart data is larger than a difference degree threshold value or not for each swing chart data, if not, ignoring the marine obstacle region object in the swing chart data, and if yes, determining that the swing chart data is obstacle-related chart data;
calculating weighted regional boundary average values of all the marine barrier region objects in the barrier-related chart data and the marine barrier region objects in the basic verification chart data so as to correct land boundary objects in the basic verification chart data; wherein the mean computation weight corresponding to each sea obstacle region is proportional to the reliability parameter of the corresponding sea chart data.
In a second aspect of the present invention, the method for dividing the verification chart data into a plurality of data portions by the storage module, and storing the plurality of data portions to a plurality of server nodes includes:
Dividing the verification chart data according to a preset region subdivision rule corresponding to the target ocean region to obtain a plurality of region chart data;
dividing each regional chart data according to a preset data type dividing rule to obtain a plurality of regional data subdivision data;
Acquiring the equipment data transmission rate of the current computing equipment and the node space and the node transmission distance corresponding to each server node;
setting an objective function to the maximum number of different areas and different data types corresponding to the allocation data corresponding to each server node;
setting limiting conditions, wherein the limiting conditions comprise that more distribution data of the server nodes with larger node space, fewer distribution data of the server nodes with larger node transmission distance and the sum of transmission time of all distribution data are smaller than a preset time threshold; the transmission time is calculated according to the equipment data transmission rate and the node transmission distance of the corresponding server node;
Based on a dynamic programming algorithm, carrying out iterative allocation calculation on all the area data subdivision data according to the objective function and the limiting condition until an optimal allocation scheme is obtained; the distribution scheme comprises server nodes corresponding to different regional data subdivision data;
And sending each area data subdivision data to a corresponding server node for storage.
The third aspect of the invention discloses another chart data resource management system based on data verification, which comprises:
a memory storing executable program code;
A processor coupled to the memory;
The processor invokes the executable program code stored in the memory to perform some or all of the steps in the chart data resource management method based on data verification disclosed in the first aspect of the present invention.
A fourth aspect of the present invention discloses a computer storage medium storing computer instructions which, when invoked, are adapted to perform part or all of the steps of the data verification-based chart data resource management method disclosed in the first aspect of the present invention.
Compared with the prior art, the embodiment of the invention has the following beneficial effects:
the reliability parameter can be determined based on the data acquisition parameter and the chart data, then the accurate chart data corresponding to the region is obtained based on the reliability parameter through cross verification, and then the segmentation and the distributed storage are carried out so as to facilitate the subsequent cross verification and the safety protection, thereby effectively improving the data accuracy and the management efficiency of the chart data and improving the safety of the chart data storage.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terms first, second and the like in the description and in the claims and in the above-described figures are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, apparatus, article, or device that comprises a list of steps or elements is not limited to the list of steps or elements but may, in the alternative, include other steps or elements not expressly listed or inherent to such process, method, article, or device.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
The invention discloses a chart data resource management method and system based on data verification, which can determine reliability parameters based on data acquisition parameters and chart data, then perform cross verification based on the reliability parameters to obtain accurate chart data corresponding to an area, and then perform segmentation and distributed storage to facilitate subsequent cross verification and safety protection, so that the data accuracy and management efficiency of chart data can be effectively improved, and the safety of chart data storage is improved. The following will describe in detail.
Referring to fig. 1, fig. 1 is a flowchart of a chart data resource management method based on data verification according to an embodiment of the present invention. The chart data resource management method based on data verification described in fig. 1 can be applied to a data processing system/data processing device/data processing server (wherein the server comprises a local processing server or a cloud processing server). As shown in fig. 1, the chart data resource management method based on data verification may include the following operations:
101. and acquiring a plurality of chart data corresponding to the target ocean area.
102. And determining the credibility parameter of each chart data according to each chart data and the corresponding data acquisition parameter.
103. And performing cross verification on all the chart data according to the credibility parameters to obtain verified chart data corresponding to the target ocean area.
104. Dividing the verification chart data to obtain a plurality of data parts, and storing the plurality of data parts to a plurality of server nodes.
Therefore, the embodiment of the invention can determine the reliability parameter based on the data acquisition parameter and the chart data, then perform cross-validation based on the reliability parameter to obtain accurate chart data corresponding to the region, and then perform segmentation and distributed storage to facilitate subsequent cross-validation and safety protection, so that the data accuracy and management efficiency of chart data can be effectively improved, and the safety of chart data storage is improved.
As an alternative embodiment, in the step, the data acquisition parameters include a sensing data acquisition device parameter, a sensing data acquisition position, a model data calculation device parameter, a model data calculation time and a model data calculation energy consumption; the sensor data acquisition device parameters or model data calculation device parameters include device model number, device performance parameters, and device location.
Therefore, through the optional embodiment, the content of the data acquisition parameters is clarified, so that the relevant parameters of the sensing data based on the calculation of the chart data and the relevant parameters of the model calculation of the calculated chart data can better represent the calculation relevant characteristics of the chart data for later determining the credibility of the chart data, thereby assisting in realizing the improvement of the data accuracy and management efficiency of the chart data and improving the safety of chart data storage.
As an optional embodiment, in the step, determining the reliability parameter of each chart data according to each chart data and the corresponding data acquisition parameter includes:
For each chart data, inputting the chart data and corresponding data acquisition parameters into a trained credibility prediction neural network to obtain credibility prediction probability corresponding to the chart data; optionally, the credibility prediction neural network is obtained through training a training data set comprising a plurality of training chart data and corresponding data acquisition parameter labels and credibility labels;
Calculating the average value of the sea chart difference degree between the sea chart data and all other sea chart data to obtain a difference degree parameter corresponding to the sea chart data;
Calculating a complexity difference between the data complexity corresponding to the chart data and a reference complexity threshold corresponding to the target ocean area;
Calculating the product of the reliability prediction probability corresponding to the chart data and the first weight and the second weight to obtain the reliability parameter of the chart data; optionally, the first weight is inversely proportional to the degree of difference parameter; the second weight is inversely proportional to the complexity difference.
Therefore, through the optional embodiment, the reliability parameter of the sea chart data can be accurately calculated based on the reliability probability of the neural network prediction and the weighting rule based on the sea chart difference degree and the complexity, so that the reliability parameter is used for cross verification of the sea chart data in the follow-up process, the data accuracy and the management efficiency of the sea chart data are improved, and the safety of sea chart data storage is improved.
As an alternative embodiment, in the above step, the chart difference degree is calculated by the following rule:
For the two sea chart data, calculating the boundary difference degree between land boundary objects in the two sea chart data;
calculating the regional difference degree between the marine barrier regional objects of the two sea chart data;
Calculating a float difference degree between the float objects of the two sea chart data;
calculating a weighted sum value among the boundary difference degree, the region difference degree and the floating object difference degree to obtain a chart difference degree between two chart data; the weights corresponding to the boundary difference degree, the region difference degree and the floating object difference degree are sequentially reduced.
Therefore, through the optional embodiment, detailed calculation rules of the sea chart difference degree are clarified, the sea chart data difference degree can be accurately calculated, and reliability parameters of the sea chart data can be accurately calculated in the follow-up process, so that improvement of data accuracy and management efficiency of the sea chart data is achieved in an auxiliary mode, and safety of sea chart data storage is improved.
As an alternative embodiment, in the above step, the data complexity is calculated by the following rule:
calculating the data type number of all data in the chart data;
Determining all trivial objects in the chart data; optionally, the size of the data volume of the trivial object is below the data volume threshold or the size of the object area is below the area threshold;
calculating the total number of objects of all trivial objects;
Calculating the discrete degree of the object position distribution of all the trivial objects;
Calculating the weighted sum value of the data type quantity, the object total quantity and the discrete degree to obtain the data complexity corresponding to the chart data; wherein the degree of discretization, the total number of objects and the weight corresponding to the number of data types are sequentially reduced.
Therefore, through the optional embodiment, the detailed calculation rule of the data complexity is clarified, and the data complexity of the sea chart data can be accurately calculated so as to be used for accurately calculating the reliability parameter of the sea chart data in the follow-up process, thereby assisting in realizing improvement of the data accuracy and management efficiency of the sea chart data and improving the security of sea chart data storage.
As an optional embodiment, in the step, cross-verifying all chart data according to the reliability parameter to obtain verified chart data corresponding to the target ocean area includes:
screening out all chart data with reliability parameters larger than a first reliability threshold value to obtain a plurality of preferable chart data;
Carrying out data average calculation on all the preferable chart data to obtain basic verification chart data;
rejecting all chart data with reliability parameters smaller than a second reliability threshold;
Screening out all chart data with reliability parameters smaller than the first reliability threshold and larger than the second reliability threshold to obtain a plurality of swinging chart data;
judging whether the total quantity of all the swing chart data is larger than a preset quantity threshold value, if not, determining the basic verification chart data as verification chart data corresponding to the target ocean area;
If yes, correcting the basic verification chart data based on the swing chart data to obtain verification chart data corresponding to the target ocean area.
Therefore, through the optional embodiment, the chart data with different credibility can be determined based on the threshold screening rule of the credibility parameter, so that accurate data cross-validation calculation is performed, more accurate validation chart data corresponding to the ocean area is obtained, and therefore data accuracy and management efficiency of the chart data are improved.
As an optional embodiment, in the step, correcting the basic verification chart data based on the rocking chart data to obtain verification chart data corresponding to the target ocean area includes:
calculating boundary average values of land boundary objects in all the swing chart data and land boundary objects in the basic verification chart data so as to correct the land boundary objects in the basic verification chart data;
Ignoring floating objects in all rocking chart data;
Judging whether the regional difference degree between the sea obstacle region objects of the swing chart data and the basic verification chart data is larger than a difference degree threshold value or not for each swing chart data, if not, neglecting the sea obstacle region objects in the swing chart data, and if so, determining that the swing chart data is obstacle-related chart data;
Calculating weighted regional boundary average values of the marine obstacle region objects in all the obstacle-related chart data and the marine obstacle region objects in the basic verification chart data so as to correct land boundary objects in the basic verification chart data; wherein the mean calculation weight corresponding to each sea obstacle region is proportional to the credibility parameter of the corresponding chart data.
Therefore, according to the alternative embodiment, the basic verification chart data can be corrected based on the correction rule of the data objects of different types in the swing chart data, so that more accurate verification chart data corresponding to the ocean area can be obtained, and the data accuracy and the management efficiency of the chart data are improved.
As an optional embodiment, in the step, splitting the verification chart data to obtain a plurality of data portions, and storing the plurality of data portions in a plurality of server nodes, including:
Dividing the verification chart data according to a preset regional subdivision rule corresponding to the target ocean region to obtain a plurality of regional chart data;
Dividing each region chart data according to a preset data type dividing rule to obtain a plurality of region data subdivision data;
Acquiring the equipment data transmission rate of the current computing equipment and the node space and the node transmission distance corresponding to each server node;
Setting an objective function to the maximum number of different areas and different data types corresponding to the allocation data corresponding to each server node;
Setting limiting conditions, wherein the limiting conditions comprise that the server nodes with larger node space have more distribution data, the server nodes with larger node transmission distance have fewer distribution data, and the sum of transmission time of all distribution data is smaller than a preset time threshold; optionally, the transmission time is calculated according to the equipment data transmission rate and the node transmission distance of the corresponding server node;
Based on a dynamic programming algorithm, carrying out iterative allocation calculation on all area data subdivision data according to an objective function and a limiting condition until an optimal allocation scheme is obtained; optionally, the allocation scheme includes server nodes corresponding to the subdivision data of the different area data;
and sending the subdivision data of each region data to a corresponding server node for storage.
Therefore, through the above optional embodiment, the chart data can be subdivided according to the region and the data type to obtain a plurality of subsets, and then the dynamic programming algorithm model is constructed based on the data diversity index and the limitation of the server node, so as to calculate the optimal allocation scheme, realize the distributed storage of the chart data, thereby realizing the improvement of the data accuracy and the management efficiency of the chart data and the improvement of the safety of the chart data storage.
Referring to fig. 2, fig. 2 is a schematic diagram of a chart data resource management system based on data verification according to an embodiment of the present invention. The chart data resource management system based on data verification described in fig. 2 can be applied to a data processing system/data processing device/data processing server (wherein the server comprises a local processing server or a cloud processing server). As shown in fig. 2, the data verification-based chart data resource management system may include:
the acquiring module 201 is configured to acquire a plurality of chart data corresponding to the target ocean area.
The determining module 202 is configured to determine a reliability parameter of each chart data according to each chart data and a corresponding data acquisition parameter.
And the verification module 203 is configured to cross-verify all the chart data according to the reliability parameter, so as to obtain verified chart data corresponding to the target ocean area.
The storage module 204 is configured to divide the verification chart data to obtain a plurality of data portions, and store the plurality of data portions to a plurality of server nodes.
Therefore, the embodiment of the invention can determine the reliability parameter based on the data acquisition parameter and the chart data, then perform cross-validation based on the reliability parameter to obtain accurate chart data corresponding to the region, and then perform segmentation and distributed storage to facilitate subsequent cross-validation and safety protection, so that the data accuracy and management efficiency of chart data can be effectively improved, and the safety of chart data storage is improved.
As an alternative embodiment, the data acquisition parameters include a sensing data acquisition device parameter, a sensing data acquisition location, a model data calculation device parameter, a model data calculation time, and a model data calculation power consumption; the sensor data acquisition device parameters or model data calculation device parameters include device model number, device performance parameters, and device location.
Therefore, through the optional embodiment, the content of the data acquisition parameters is clarified, so that the relevant parameters of the sensing data based on the calculation of the chart data and the relevant parameters of the model calculation of the calculated chart data can better represent the calculation relevant characteristics of the chart data for later determining the credibility of the chart data, thereby assisting in realizing the improvement of the data accuracy and management efficiency of the chart data and improving the safety of chart data storage.
As an optional embodiment, the determining module determines, according to each chart data and the corresponding data acquisition parameter, a specific manner of the credibility parameter of each chart data, including:
For each chart data, inputting the chart data and corresponding data acquisition parameters into a trained credibility prediction neural network to obtain credibility prediction probability corresponding to the chart data; optionally, the credibility prediction neural network is obtained through training a training data set comprising a plurality of training chart data and corresponding data acquisition parameter labels and credibility labels;
Calculating the average value of the sea chart difference degree between the sea chart data and all other sea chart data to obtain a difference degree parameter corresponding to the sea chart data;
Calculating a complexity difference between the data complexity corresponding to the chart data and a reference complexity threshold corresponding to the target ocean area;
Calculating the product of the reliability prediction probability corresponding to the chart data and the first weight and the second weight to obtain the reliability parameter of the chart data; optionally, the first weight is inversely proportional to the degree of difference parameter; the second weight is inversely proportional to the complexity difference.
Therefore, through the optional embodiment, the reliability parameter of the sea chart data can be accurately calculated based on the reliability probability of the neural network prediction and the weighting rule based on the sea chart difference degree and the complexity, so that the reliability parameter is used for cross verification of the sea chart data in the follow-up process, the data accuracy and the management efficiency of the sea chart data are improved, and the safety of sea chart data storage is improved.
As an alternative embodiment, the sea chart variability is calculated by the following rule:
For the two sea chart data, calculating the boundary difference degree between land boundary objects in the two sea chart data;
calculating the regional difference degree between the marine barrier regional objects of the two sea chart data;
Calculating a float difference degree between the float objects of the two sea chart data;
calculating a weighted sum value among the boundary difference degree, the region difference degree and the floating object difference degree to obtain a chart difference degree between two chart data; the weights corresponding to the boundary difference degree, the region difference degree and the floating object difference degree are sequentially reduced.
Therefore, through the optional embodiment, detailed calculation rules of the sea chart difference degree are clarified, the sea chart data difference degree can be accurately calculated, and reliability parameters of the sea chart data can be accurately calculated in the follow-up process, so that improvement of data accuracy and management efficiency of the sea chart data is achieved in an auxiliary mode, and safety of sea chart data storage is improved.
As an alternative embodiment, the data complexity is calculated by the following rule:
calculating the data type number of all data in the chart data;
Determining all trivial objects in the chart data; optionally, the size of the data volume of the trivial object is below the data volume threshold or the size of the object area is below the area threshold;
calculating the total number of objects of all trivial objects;
Calculating the discrete degree of the object position distribution of all the trivial objects;
Calculating the weighted sum value of the data type quantity, the object total quantity and the discrete degree to obtain the data complexity corresponding to the chart data; wherein the degree of discretization, the total number of objects and the weight corresponding to the number of data types are sequentially reduced.
Therefore, through the optional embodiment, the detailed calculation rule of the data complexity is clarified, and the data complexity of the sea chart data can be accurately calculated so as to be used for accurately calculating the reliability parameter of the sea chart data in the follow-up process, thereby assisting in realizing improvement of the data accuracy and management efficiency of the sea chart data and improving the security of sea chart data storage.
As an optional embodiment, the verification module performs cross-verification on all chart data according to the reliability parameter to obtain a specific mode of verifying chart data corresponding to the target ocean area, where the specific mode includes:
screening out all chart data with reliability parameters larger than a first reliability threshold value to obtain a plurality of preferable chart data;
Carrying out data average calculation on all the preferable chart data to obtain basic verification chart data;
rejecting all chart data with reliability parameters smaller than a second reliability threshold;
Screening out all chart data with reliability parameters smaller than the first reliability threshold and larger than the second reliability threshold to obtain a plurality of swinging chart data;
judging whether the total quantity of all the swing chart data is larger than a preset quantity threshold value, if not, determining the basic verification chart data as verification chart data corresponding to the target ocean area;
If yes, correcting the basic verification chart data based on the swing chart data to obtain verification chart data corresponding to the target ocean area.
Therefore, through the optional embodiment, the chart data with different credibility can be determined based on the threshold screening rule of the credibility parameter, so that accurate data cross-validation calculation is performed, more accurate validation chart data corresponding to the ocean area is obtained, and therefore data accuracy and management efficiency of the chart data are improved.
As an optional embodiment, the specific manner of modifying the basic verification chart data based on the rocking chart data to obtain the verification chart data corresponding to the target ocean area by the verification module includes:
calculating boundary average values of land boundary objects in all the swing chart data and land boundary objects in the basic verification chart data so as to correct the land boundary objects in the basic verification chart data;
Ignoring floating objects in all rocking chart data;
Judging whether the regional difference degree between the sea obstacle region objects of the swing chart data and the basic verification chart data is larger than a difference degree threshold value or not for each swing chart data, if not, neglecting the sea obstacle region objects in the swing chart data, and if so, determining that the swing chart data is obstacle-related chart data;
Calculating weighted regional boundary average values of the marine obstacle region objects in all the obstacle-related chart data and the marine obstacle region objects in the basic verification chart data so as to correct land boundary objects in the basic verification chart data; wherein the mean calculation weight corresponding to each sea obstacle region is proportional to the credibility parameter of the corresponding chart data.
Therefore, according to the alternative embodiment, the basic verification chart data can be corrected based on the correction rule of the data objects of different types in the swing chart data, so that more accurate verification chart data corresponding to the ocean area can be obtained, and the data accuracy and the management efficiency of the chart data are improved.
As an optional embodiment, the specific manner of the storage module dividing the verification chart data to obtain a plurality of data portions and storing the plurality of data portions to the plurality of server nodes includes:
Dividing the verification chart data according to a preset regional subdivision rule corresponding to the target ocean region to obtain a plurality of regional chart data;
Dividing each region chart data according to a preset data type dividing rule to obtain a plurality of region data subdivision data;
Acquiring the equipment data transmission rate of the current computing equipment and the node space and the node transmission distance corresponding to each server node;
Setting an objective function to the maximum number of different areas and different data types corresponding to the allocation data corresponding to each server node;
Setting limiting conditions, wherein the limiting conditions comprise that the server nodes with larger node space have more distribution data, the server nodes with larger node transmission distance have fewer distribution data, and the sum of transmission time of all distribution data is smaller than a preset time threshold; optionally, the transmission time is calculated according to the equipment data transmission rate and the node transmission distance of the corresponding server node;
Based on a dynamic programming algorithm, carrying out iterative allocation calculation on all area data subdivision data according to an objective function and a limiting condition until an optimal allocation scheme is obtained; optionally, the allocation scheme includes server nodes corresponding to the subdivision data of the different area data;
and sending the subdivision data of each region data to a corresponding server node for storage.
Therefore, through the above optional embodiment, the chart data can be subdivided according to the region and the data type to obtain a plurality of subsets, and then the dynamic programming algorithm model is constructed based on the data diversity index and the limitation of the server node, so as to calculate the optimal allocation scheme, realize the distributed storage of the chart data, thereby realizing the improvement of the data accuracy and the management efficiency of the chart data and the improvement of the safety of the chart data storage.
Referring to fig. 3, fig. 3 is a schematic diagram illustrating another chart data resource management system based on data verification according to an embodiment of the present invention. The data verification-based chart data resource management system described in fig. 3 is applied to a data processing system/data processing device/data processing server (wherein the server comprises a local processing server or a cloud processing server). As shown in fig. 3, the data verification-based chart data resource management system may include:
A memory 301 storing executable program code;
a processor 302 coupled with the memory 301;
wherein the processor 302 invokes executable program code stored in the memory 301 for performing the steps of the chart data resource management method based on data verification described in embodiment one.
In a fourth embodiment, the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program causes a computer to execute the steps of the chart data resource management method based on data verification described in the first embodiment.
In a fifth embodiment, the present invention discloses a computer program product, which includes a non-transitory computer readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps of the chart data resource management method based on data verification described in the first embodiment.
The foregoing describes certain embodiments of the present disclosure, other embodiments being within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. Furthermore, the processes depicted in the accompanying drawings do not necessarily have to be in the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in one or more software and/or hardware elements when implemented in the present specification.
It will be appreciated by those skilled in the art that the present description may be provided as a method, system, or computer program product. Accordingly, the present specification embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present description embodiments may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
The present description is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
Finally, it should be noted that: the embodiment of the invention discloses a chart data resource management method and system based on data verification, which are disclosed by the embodiment of the invention only for illustrating the technical scheme of the invention, but not limiting the technical scheme; although the invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art will understand that; the technical scheme recorded in the various embodiments can be modified or part of technical features in the technical scheme can be replaced equivalently; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.