CN116431616B - Big data model management system and method based on cloud computing - Google Patents
Big data model management system and method based on cloud computing Download PDFInfo
- Publication number
- CN116431616B CN116431616B CN202310694060.7A CN202310694060A CN116431616B CN 116431616 B CN116431616 B CN 116431616B CN 202310694060 A CN202310694060 A CN 202310694060A CN 116431616 B CN116431616 B CN 116431616B
- Authority
- CN
- China
- Prior art keywords
- data
- transmission
- model
- channel
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007726 management method Methods 0.000 title claims abstract description 51
- 238000013499 data model Methods 0.000 title claims abstract description 23
- 238000000034 method Methods 0.000 title claims abstract description 16
- 230000005540 biological transmission Effects 0.000 claims abstract description 147
- 238000012937 correction Methods 0.000 claims abstract description 33
- 238000011156 evaluation Methods 0.000 claims abstract description 22
- 230000002159 abnormal effect Effects 0.000 claims abstract description 16
- 238000003032 molecular docking Methods 0.000 claims abstract description 16
- 230000007246 mechanism Effects 0.000 claims description 13
- 238000010276 construction Methods 0.000 claims description 12
- 238000012795 verification Methods 0.000 claims description 12
- 238000004891 communication Methods 0.000 claims description 9
- 239000013589 supplement Substances 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 3
- 230000008859 change Effects 0.000 claims description 3
- 238000004806 packaging method and process Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 230000004044 response Effects 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 238000013475 authorization Methods 0.000 claims description 2
- 210000001503 joint Anatomy 0.000 claims 2
- 238000013523 data management Methods 0.000 abstract description 2
- 238000007689 inspection Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/211—Schema design and management
- G06F16/212—Schema design and management with details for data modelling support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Quality & Reliability (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to the field of data management, and discloses a big data model management system and method based on cloud computing, wherein the system comprises the following components: the model receiving end is used as a receiving end of model data, and after the upper transmission node is confirmed, the control authority of data writing is opened; the transmission docking unit is used for executing a transmission task of the model data, and continuously transmitting the data after docking is finished; the data acquisition unit is used for acquiring data in transmission and sectional mirror image backup data and submitting the backup data downwards; the evaluation end is used for acquiring backup data, analyzing and evaluating the model data and the operation data thereof, outputting an operation index, and withholding a target lower than a preset index; by evaluating and checking the model data, whether abnormal conditions exist in the data is confirmed, the original data is deduced, the abnormal data is marked, and the deduced prediction result is used as reference data for correction, so that the accuracy of the model data is improved.
Description
Technical Field
The invention relates to the technical field of data management, in particular to a big data model management system and method based on cloud computing.
Background
The final objective of building a big data platform is to serve the business needs, solve existing business problems or create new opportunities. The business department may not care whether to adopt the big data technology or the traditional database technology, the main basis of whether to adopt the big data technology is the data volume, if the situation that the task runs for a long time occurs, or the prior art can not meet because the calculated amount is too large, or when a large amount of semi-structured and unstructured data needs to be processed, big data model management is needed;
however, existing big data model management systems and methods have shortcomings, including:
1. the obtained model data is extremely easy to interfere in the transmission process, so that the final transmission result is deviated, the transmission precision is seriously affected, and correction is difficult to carry out, so that errors or doubt data are directly stored, larger errors can occur in subsequent references, and the performance of a used transmission channel is difficult to control;
2. in the management process, backtracking correction measures are lacking, so that when an error occurs in an introduced adjustment mechanism, correction results are offset and expected, management is affected, and data pollution is caused.
Disclosure of Invention
Aiming at the defects existing in the prior art, the invention provides a big data model management system and a method based on cloud computing, which can effectively solve the problems that model data acquired by the big data model management system and the method in the prior art are extremely easy to interfere in the transmission process, so that the final transmission result is deviated, the transmission precision is seriously influenced, in addition, correction is difficult to be carried out, error or doubt data are directly stored, larger errors can occur in subsequent references, the performance of a used transmission channel is difficult to control, and backtracking correction measures are not available in the management process, so that the correction result is deviated to be expected when the introduced adjustment mechanism is wrong, the management is influenced, and the data pollution is caused.
In order to achieve the above object, the present invention discloses a big data model management system and method based on cloud computing, comprising:
the model receiving end is used as a receiving end of model data, and after the upper transmission node is confirmed, the control authority of data writing is opened;
the transmission docking unit is used for executing a transmission task of the model data, and continuously transmitting the data after docking is finished;
the data acquisition unit is used for acquiring data in transmission and sectional mirror image backup data and submitting the backup data downwards;
the evaluation end is used for acquiring backup data, analyzing and evaluating the model data and the operation data thereof, outputting an operation index, and withholding a target lower than a preset index;
the verification module is used for verifying the withheld data of the evaluation end, analyzing and outputting abnormal items of the withheld data;
the prediction module is used for tracking the original data of the withheld data, predicting the state to be completed according to the original data and outputting a predicted text set;
the marking module is used for matching the predicted text set with withholding data and marking the entry and exit;
the correction matching end is used for correcting the marked data generated by the marking module until the marked data is matched with the predicted text set and then outputting the marked data;
the issuing unit is used for performing secondary issuing on the corrected data and acquiring the permission of re-participation transmission;
the asset management terminal is used for acquiring and displaying meta-model elements and attribute information of different database types and used as a storage library of model data and operation data thereof;
the transmission docking unit is interactively connected with a channel allocation unit through a wireless network, the channel allocation unit is interactively connected with a channel construction unit and a channel assessment unit through the wireless network, the channel construction unit is used for constructing a data transmission channel for transmission, acquiring and authorizing transmission authorities in the existing communication network, the channel assessment unit is used for analyzing the transmission performance of the transmission channel constructed by the channel construction unit and sequencing, and the channel allocation unit is used for analyzing transmission requirements and allocating transmission channels with different performances.
The asset management end is interactively connected with a backtracking module through a wireless network, and the backtracking module is used for backtracking running data of the model data issued by the issuing unit and the existing model data of the asset management end, and resubmitting the backtracking data to the checking module to confirm whether the processing result is unique.
Furthermore, the data acquisition unit edits in a segmentation mode by a manual self-defining mode, and the editing targets comprise: transmission time and data length.
Further, after the correction matching end completes matching with the predicted text set, the correction matching end outputs a work log, performs catalogued management on the data resources to form a hierarchical data resource set, and the attribute of the hierarchical data resource set comprises: execution time and execution goal.
Still further, the model receiving end is connected with the transmission docking unit through a wireless network, the transmission docking unit is connected with the data acquisition unit through a wireless network, the data acquisition unit is connected with the evaluation end through electric signal communication, the evaluation end is connected with the inspection module through a wireless network, the inspection module is connected with the prediction module through a wireless network, the prediction module is connected with the marking module through a wireless network, the marking module is connected with the correction matching end through a wireless network, the correction matching end is connected with the issuing unit through a wireless network, and the issuing unit is connected with the asset management end through electric signal communication.
A big data model management method based on cloud computing comprises the following steps:
step 1: obtaining the receiving authority of the model data, and after distributing a transmission channel, continuously submitting the data in the current transmission period;
step 2: copying the data in transmission in a time-interval and length mode, and temporarily storing the copied data;
step 3: the temporary storage data are called for evaluation, abnormal item verification is carried out on the data which do not reach the evaluation threshold, the data are output as one type of data, the posture data which are needed to be realized by the original data are predicted, the data are taken as two types of data, comprehensive analysis is carried out on the one type of data and the two types of data, and the data are released after additional correction and supplement are carried out on the one type of data;
step 4: backtracking the unfolding data, and judging whether the final release data accords with the adjustment mechanism logic or not;
step 5: if yes, packaging and submitting the release data to a transmission channel to participate in transmission again;
step 6: if not, the adjustment mechanism is deactivated, and a problem report is generated and sent to the management end.
Furthermore, the transmission channel paths allocated in the step 1 are not unique, and the constructed channel transmission delay parameters compensate the anti-interference capability, and the calculation formula is as follows:
；
wherein: q is a channel transmission delay function;attenuation parameters for the transmission channel; />The characteristic parameter is suppressed for the positive order of the transmission channel; />Is a critical beam characteristic of the transmission channel; e is a data transmission interference coefficient; />Carrier frequency components allocated for mobile data transmission.
Furthermore, the change condition of the critical beam characteristics directly affects the data transmission precision and transmission loss, and the data quantity is iteratively distributed by adopting an error compensation method, wherein the calculation formula is as follows:
；
wherein F (K+1) is an iterated function;is a function before iteration; />Carrier frequency components allocated for iterative data transmission; />To allocate data volume; />The frequency response values are filtered for the transmission channel.
Still further, the parameters of the problem report in step 6 include: conflict exceptions, missing exceptions, and duplicate exceptions.
Compared with the prior art, the technical scheme provided by the invention has the following beneficial effects that whether abnormal conditions exist in the data is confirmed by evaluating and checking the model data, the original data is deduced, the abnormal data is marked, and the deduced prediction result is taken as the reference data to correct, so that the accuracy of the model data is improved.
According to the invention, the transmission channel is built by oneself, the transmission capacity of the channel is evaluated, and when the transmission requirement exists, the transmission channel with the best current transmission capacity is allocated, so that the anti-interference capacity of data is improved, errors in the transmission process are avoided, and the purity of the data is ensured.
The invention further avoids the occurrence of correction failure caused by the failure of verification measures by backtracking and tracing the adjustment mechanism, and ensures the stable operation of the management function.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is evident that the drawings in the following description are only some embodiments of the present invention and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art.
FIG. 1 is a schematic diagram of a framework of a cloud computing-based big data model management system;
FIG. 2 is a schematic flow chart of a big data model management method based on cloud computing;
reference numerals in the drawings represent respectively:
1. a model receiving end; 2. a transmission docking unit; 3. a data acquisition unit; 4. an evaluation end; 5. a verification module; 6. a prediction module; 7. a marking module; 8. correcting the matching end; 9. a release unit; 10. an asset management end; 11. a backtracking module; 12. a channel construction unit; 13. a channel assessment unit; 14. and a channel allocation unit.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It will be apparent that the described embodiments are some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention is further described below with reference to examples.
Example 1: the big data model management system based on cloud computing of this embodiment, as shown in fig. 1, includes:
the model receiving end 1 is used as a receiving end of model data, and after the upper transmission node is confirmed, the control authority of data writing is opened;
the transmission docking unit 2 is used for executing a transmission task of the model data, and continuously transmitting the data after docking;
the data acquisition unit 3 is used for acquiring data in transmission and sectional mirror image backup data and submitting the backup data downwards;
the evaluation end 4 is used for acquiring backup data, analyzing and evaluating model data and operation data thereof, outputting an operation index, and withholding targets lower than a preset index;
the verification module 5 is used for verifying the withheld data of the evaluation end 4, analyzing and outputting abnormal items of the withheld data;
the prediction module 6 is used for tracking the original data of the withheld data, predicting the state to be completed according to the original data and outputting a prediction text set;
the marking module 7 is used for matching the predicted text set with withholding data and marking the entry and exit;
the correction matching end 8 is used for correcting the marked data generated by the marking module 7 until the marked data is matched with the predicted text set and then outputting the marked data;
the issuing unit 9 is used for performing secondary issuing on the corrected data and acquiring the permission of re-participation transmission;
the asset management end 10 is used for acquiring and displaying meta-model elements and attribute information of different database types as a storage library of model data and operation data thereof.
The segmentation mode of the data acquisition unit 3 is edited in a manual self-defining mode, and the editing targets comprise: transmission time and data length.
As shown in fig. 1, the asset management end 10 is interactively connected with a backtracking module 11 through a wireless network, and the backtracking module 11 is used for backtracking running data of model data issued by the issuing unit 9 and model data existing in the asset management end 10, and resubmitting the backtracking data to the verification module 5 to confirm whether the processing result is unique.
After the correction matching end 8 completes matching with the predictive text set, outputting a work log, and performing catalogued management on the data resources to form a hierarchical data resource set, wherein the attribute of the hierarchical data resource set comprises: execution time and execution goal.
As shown in fig. 1, the model receiving end 1 is interactively connected with the transmission docking unit 2 through a wireless network, the transmission docking unit 2 is interactively connected with the data acquisition unit 3 through a wireless network, the data acquisition unit 3 is interactively connected with the evaluation end 4 through electric signal communication, the evaluation end 4 is interactively connected with the inspection module 5 through a wireless network, the inspection module 5 is interactively connected with the prediction module 6 through a wireless network, the prediction module 6 is interactively connected with the marking module 7 through a wireless network, the marking module 7 is interactively connected with the correction matching end 8 through a wireless network, the correction matching end 8 is interactively connected with the issuing unit 9 through a wireless network, and the issuing unit 9 is interactively connected with the asset management end 10 through electric signal communication.
In the embodiment, when the method is implemented, a model receiving end 1 is used for butting an upper-level data submitting interface to obtain control authority, a channel construction unit 12 is used for constructing a transmission channel, a channel assessment unit 13 is used for assessing channel transmission capacity, a channel allocation unit 14 is used for recommending a transmission channel with better performance in the current transmission period, a transmission butting unit 2 is used for butting the transmission channel to carry out model data transmission, a data acquisition unit 3 is used for copying data, the data are submitted to an assessment end 4 for assessment, abnormal data are examined by an examination module 5, a prediction module 6 is used for retrieving original data for prediction, an entering and exiting position is marked by a marking module 7, correction and supplement are carried out by a correction matching end 8, after the correction and supplement are finished, the data are submitted to a transmission end again by a release unit 9, the data flow to an asset management end 10 are stored, a backtracking module 11 is used for backtracking the adjustment mechanism, and consistency of the obtained data and correction results are confirmed;
by evaluating and checking the model data, determining whether abnormal conditions exist in the data, deducing the original data, marking the abnormal data, correcting the deducted prediction result as reference data, improving the accuracy of the model data, and by tracing the adjustment mechanism, the occurrence of correction failure conditions caused by the failure of the checking measures is avoided, and the stable operation of the management function is ensured.
Example 2: the embodiment also provides a big data model management method based on cloud computing, as shown in fig. 2, comprising the following steps:
step 1: obtaining the receiving authority of the model data, and after distributing a transmission channel, continuously submitting the data in the current transmission period;
step 2: copying the data in transmission in a time-interval and length mode, and temporarily storing the copied data;
step 3: the temporary storage data are called for evaluation, abnormal item verification is carried out on the data which do not reach the evaluation threshold, the data are output as one type of data, the posture data which are needed to be realized by the original data are predicted, the data are taken as two types of data, comprehensive analysis is carried out on the one type of data and the two types of data, and the data are released after additional correction and supplement are carried out on the one type of data;
step 4: backtracking the unfolding data, and judging whether the final release data accords with the adjustment mechanism logic or not;
step 5: if yes, packaging and submitting the release data to a transmission channel to participate in transmission again;
step 6: if not, the adjustment mechanism is deactivated, and a problem report is generated and sent to the management end.
The transmission channel paths allocated in the step 1 are not unique, and the constructed channel transmission delay parameters compensate the anti-interference capacity, and the calculation formula is as follows:
；
wherein: q is a channel transmission delay function;attenuation parameters for the transmission channel; />The characteristic parameter is suppressed for the positive order of the transmission channel; />Is a critical beam characteristic of the transmission channel; e is a data transmission interference coefficient; />Carrier frequency components allocated for mobile data transmission.
The change condition of the critical wave beam characteristics directly affects the data transmission precision and transmission loss, and adopts an error compensation method to iteratively distribute data quantity, wherein the calculation formula is as follows:
；
wherein F (K+1) is an iterated function;is a function before iteration; />Carrier frequency components allocated for iterative data transmission; />To allocate data volume; />The frequency response values are filtered for the transmission channel.
The parameters of the problem report in step 6 include: conflict exceptions, missing exceptions, and duplicate exceptions.
Example 3: in this embodiment, as shown in fig. 1, the transmission docking unit 2 is interactively connected with a channel allocation unit 14 through a wireless network, the channel allocation unit 14 is interactively connected with a channel construction unit 12 and a channel assessment unit 13 through the wireless network, the channel construction unit 12 is used for constructing a data transmission channel for transmission, the acquisition and the authorization of transmission permission are performed in the existing communication network, the channel assessment unit 13 is used for analyzing the transmission performance of the transmission channel constructed by the channel construction unit 12 and sequencing, the channel allocation unit 14 is used for analyzing the transmission requirement and performing the allocation of the transmission channels with different performances.
Through the setting, the transmission channel is built by oneself, the transmission capacity of the channel is evaluated, and when the transmission requirement exists, the transmission channel with the best current transmission capacity is allocated, so that the anti-interference capacity of data is improved, errors in the transmission process are avoided, and the purity of the data is ensured.
In summary, the control authority of the model receiving terminal 1 is obtained through butting an upper data submitting interface, a channel construction unit 12 constructs a transmission channel, a channel assessment unit 13 assesses the channel transmission capacity, a channel allocation unit 14 recommends a transmission channel with better performance in the current transmission period, a transmission butting unit 2 butting the transmission channel for transmitting model data, a data acquisition unit 3 replicates data, the data are submitted to an evaluation terminal 4 for evaluation, abnormal data are inspected by an inspection module 5, an original data is picked up by a prediction module 6 for prediction, an access position is marked by a marking module 7, correction and supplement are carried out by a correction matching terminal 8, after the supplement is finished, the transmission channel is submitted to a transmission terminal again by a release unit 9, the data flow is stored in an asset management terminal 10, the adjustment mechanism is traced back by a tracing module 11, and consistency of the obtained data and correction results is confirmed;
by evaluating and checking the model data, determining whether abnormal conditions exist in the data, deducing the original data, marking the abnormal data, correcting the deducted prediction result as reference data, improving the accuracy of the model data, and by tracing back an adjustment mechanism, the occurrence of correction failure conditions caused by the failure of check measures is further avoided, and the stable operation of the management function is ensured;
in addition, by constructing the transmission channel by oneself and evaluating the transmission capacity of the channel, when there is a transmission requirement, the transmission channel with the best current transmission capacity is allocated, so that the anti-interference capacity of the data is improved, errors in the transmission process are avoided, and the purity of the data is ensured.
The above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; while the invention has been described in detail with reference to the foregoing embodiments, it will be appreciated by those skilled in the art that variations may be made in the techniques described in the foregoing embodiments, or equivalents may be substituted for elements thereof; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (8)
1. A cloud computing-based big data model management system, comprising:
the model receiving end (1) is used as a receiving end of model data, and after the upper transmission node is confirmed, the control authority of data writing is opened;
the transmission butt joint unit (2) is used for executing a transmission task of the model data, and continuously transmitting the data after the butt joint is finished;
the data acquisition unit (3) is used for acquiring data in transmission and sectional mirror image backup data and submitting the backup data downwards;
the evaluation end (4) is used for acquiring backup data, analyzing and evaluating model data and operation data thereof, outputting an operation index, and withholding targets lower than a preset index;
the verification module (5) is used for verifying the withheld data of the evaluation end (4), analyzing and outputting abnormal items of the withheld data;
the prediction module (6) is used for tracking the original data of the withheld data, predicting the state to be completed according to the original data and outputting a prediction text set;
the marking module (7) is used for matching the predicted text set with withholding data and marking the entry and exit item;
the correction matching end (8) is used for correcting the marked data generated by the marking module (7) until the marked data is matched with the predicted text set and then outputting the marked data;
the issuing unit (9) is used for performing secondary issuing on the corrected data and acquiring the permission of re-participation transmission;
the asset management terminal (10) is used for acquiring and displaying meta-model elements and attribute information of different database types and used as a storage library of model data and operation data thereof;
the transmission docking unit (2) is interactively connected with a channel allocation unit (14) through a wireless network, the channel allocation unit (14) is interactively connected with a channel construction unit (12) and a channel assessment unit (13) through the wireless network, the channel construction unit (12) is used for constructing a data transmission channel for transmission, the acquisition and the authorization of transmission permission are carried out in the existing communication network, the channel assessment unit (13) is used for analyzing the transmission performance of the transmission channel constructed by the channel construction unit (12) and sequencing the transmission performance, and the channel allocation unit (14) is used for analyzing the transmission requirement and carrying out the allocation of the transmission channels with different performances;
the asset management terminal (10) is interactively connected with a backtracking module (11) through a wireless network, the backtracking module (11) is used for backtracking running data of model data issued by the issuing unit (9) and model data existing in the asset management terminal (10), and the model data are submitted to the verification module (5) again to confirm whether a processing result is unique.
2. The cloud computing-based big data model management system according to claim 1, wherein the segmentation mode of the data acquisition unit (3) is edited by a manual self-defining mode, and the editing targets comprise: transmission time and data length.
3. The cloud computing-based big data model management system according to claim 1, wherein the correction matching terminal (8) outputs a work log after completing matching with the predicted text set, performs catalogued management on the data resources to form a hierarchical data resource set, and the attribute of the hierarchical data resource set comprises: execution time and execution goal.
4. The big data model management system based on cloud computing according to claim 1, wherein the model receiving end (1) is interactively connected with the transmission docking unit (2) through a wireless network, the transmission docking unit (2) is interactively connected with the data acquisition unit (3) through a wireless network, the data acquisition unit (3) is interactively connected with the evaluation end (4) through electric signal communication, the evaluation end (4) is interactively connected with the verification module (5) through a wireless network, the verification module (5) is interactively connected with the prediction module (6) through a wireless network, the prediction module (6) is interactively connected with the marking module (7) through a wireless network, the marking module (7) is interactively connected with the correction matching end (8) through a wireless network, the correction matching end (8) is interactively connected with the distribution unit (9) through a wireless network, and the distribution unit (9) is interactively connected with the asset management end (10) through electric signal communication.
5. A cloud computing-based big data model management method, which is an implementation method for the cloud computing-based big data model management system according to any one of claims 1 to 4, and is characterized by comprising the following steps:
step 1: obtaining the receiving authority of the model data, and after distributing a transmission channel, continuously submitting the data in the current transmission period;
step 2: copying the data in transmission in a time-interval and length mode, and temporarily storing the copied data;
step 3: the temporary storage data are called for evaluation, abnormal item verification is carried out on the data which do not reach the evaluation threshold, the data are output as one type of data, the posture data which are needed to be realized by the original data are predicted, the data are taken as two types of data, comprehensive analysis is carried out on the one type of data and the two types of data, and the data are released after additional correction and supplement are carried out on the one type of data;
step 4: backtracking the unfolding data, and judging whether the final release data accords with the adjustment mechanism logic or not;
step 5: if yes, packaging and submitting the release data to a transmission channel to participate in transmission again;
step 6: if not, the adjustment mechanism is deactivated, and a problem report is generated and sent to the management end.
6. The cloud computing-based big data model management method according to claim 5, wherein the transmission channel paths allocated in the step 1 are not unique, and the constructed channel transmission delay parameters compensate for the anti-interference capability, and the calculation formula is as follows:
；
wherein: q is a channel transmission delay function;attenuation parameters for the transmission channel; />The characteristic parameter is suppressed for the positive order of the transmission channel; />Is a critical beam characteristic of the transmission channel; e is a data transmission interference coefficient; />Carrier frequency components allocated for mobile data transmission.
7. The cloud computing-based big data model management method according to claim 6, wherein the change condition of the critical beam characteristics directly affects the data transmission precision and the transmission loss, and the data quantity is iteratively distributed by adopting an error compensation method, and the calculation formula is as follows:
；
wherein F (K+1) is an iterated function;is a function before iteration; />Carrier frequency components allocated for iterative data transmission; />To allocate data volume; />The frequency response values are filtered for the transmission channel.
8. The cloud computing-based big data model management method according to claim 5, wherein the parameters of the problem report in step 6 include: conflict exceptions, missing exceptions, and duplicate exceptions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310694060.7A CN116431616B (en) | 2023-06-13 | 2023-06-13 | Big data model management system and method based on cloud computing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310694060.7A CN116431616B (en) | 2023-06-13 | 2023-06-13 | Big data model management system and method based on cloud computing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116431616A CN116431616A (en) | 2023-07-14 |
| CN116431616B true CN116431616B (en) | 2023-09-01 |
Family
ID=87089411
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310694060.7A Active CN116431616B (en) | 2023-06-13 | 2023-06-13 | Big data model management system and method based on cloud computing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116431616B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20110043292A (en) * | 2009-10-21 | 2011-04-27 | 주식회사 동서리서치 | System and method for customizing research |
| CN104346698A (en) * | 2014-11-05 | 2015-02-11 | 无锡雅座在线科技发展有限公司 | Catering member big data analysis and checking system based on cloud computing and data mining |
| CN106815338A (en) * | 2016-12-25 | 2017-06-09 | 北京中海投资管理有限公司 | A kind of real-time storage of big data, treatment and inquiry system |
| CN111400382A (en) * | 2020-03-03 | 2020-07-10 | 湖南长信畅中科技股份有限公司 | Model-driven data integration middleware and implementation method |
| CN114051182A (en) * | 2022-01-12 | 2022-02-15 | 存灏技术(武汉)有限公司 | Current data acquisition, analysis and transmission system and method based on can protocol |
| CN114205174A (en) * | 2022-02-17 | 2022-03-18 | 深圳市三江电气有限公司 | Sensor for realizing multi-sensing data intersection transmission based on networking |
| CN115391425A (en) * | 2022-08-29 | 2022-11-25 | 上海墅字科技有限公司 | Method and system for checking process consistency based on attribute transfer |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11496521B2 (en) * | 2020-08-12 | 2022-11-08 | International Business Machines Corporation | Feedback loop for security audit logs |
| WO2022221719A2 (en) * | 2021-04-16 | 2022-10-20 | Strong Force Vcn Portfolio 2019, Llc | Systems, methods, kits, and apparatuses for digital product network systems and biology-based value chain networks |
| TW202305694A (en) * | 2021-07-29 | 2023-02-01 | 天脈科技股份有限公司 | Business data processing system, method, and computer program product |
-
2023
- 2023-06-13 CN CN202310694060.7A patent/CN116431616B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20110043292A (en) * | 2009-10-21 | 2011-04-27 | 주식회사 동서리서치 | System and method for customizing research |
| CN104346698A (en) * | 2014-11-05 | 2015-02-11 | 无锡雅座在线科技发展有限公司 | Catering member big data analysis and checking system based on cloud computing and data mining |
| CN106815338A (en) * | 2016-12-25 | 2017-06-09 | 北京中海投资管理有限公司 | A kind of real-time storage of big data, treatment and inquiry system |
| CN111400382A (en) * | 2020-03-03 | 2020-07-10 | 湖南长信畅中科技股份有限公司 | Model-driven data integration middleware and implementation method |
| CN114051182A (en) * | 2022-01-12 | 2022-02-15 | 存灏技术(武汉)有限公司 | Current data acquisition, analysis and transmission system and method based on can protocol |
| CN114205174A (en) * | 2022-02-17 | 2022-03-18 | 深圳市三江电气有限公司 | Sensor for realizing multi-sensing data intersection transmission based on networking |
| CN115391425A (en) * | 2022-08-29 | 2022-11-25 | 上海墅字科技有限公司 | Method and system for checking process consistency based on attribute transfer |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116431616A (en) | 2023-07-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Feylizadeh et al. | Project crashing using a fuzzy multi-objective model considering time, cost, quality and risk under fast tracking technique: A case study | |
| Pinzger et al. | Can developer-module networks predict failures? | |
| Alves et al. | Improving software effort estimation with human-centric models: a comparison of UCP and iUCP accuracy | |
| CN111444106B (en) | Analysis method and system for software testable requirements | |
| CN116431616B (en) | Big data model management system and method based on cloud computing | |
| Feng et al. | Robust approach for air cargo freight forwarder selection under disruption | |
| CN111680083A (en) | Intelligent multi-stage government financial data acquisition system and data acquisition method | |
| Cheng et al. | Reliability allocation method based on maximum entropy ordered weighted average and hesitant fuzzy Linguistic term set | |
| CN111324597A (en) | Main data management method and system | |
| Wang et al. | Detecting difference between process models using edge network | |
| CN108491440B (en) | GNSS non-real-time data tracing visualization method and system | |
| KR102195608B1 (en) | Method for automatically returning using object return rate of crowdsourcing based project for artificial intelligence training data generation | |
| KR102159574B1 (en) | Method for estimating and managing the accuracy of work results of crowdsourcing based projects for artificial intelligence training data generation | |
| Goel et al. | Neuro fuzzy based approach to predict component's reusability | |
| CN113837677A (en) | Method, device and equipment for determining logistics line generation strategy | |
| CN116860871B (en) | Metadata standardization method, device and storage medium for unified processing source data | |
| US20230110007A1 (en) | Assessing if records from different data sources represent a same entity | |
| CN118394313B (en) | Application program rapid construction method suitable for various personnel | |
| Narayan et al. | Quantifying the resilience of ICT-enabled grid services in cyber-physical energy system | |
| Chen et al. | An Information Quality Assessment Framework for Developing Building Information Models | |
| Dong et al. | A study of quality management contracts for complex product supply chains considering quality and on-time delivery | |
| Richards et al. | Military System Costing to Support Requirements and Performance Trades | |
| CN116521173A (en) | Method, device and system for detecting code dependency | |
| CN116049143A (en) | Big data platform construction method, device, equipment, medium and product | |
| CN118115052A (en) | Method, device, equipment and medium for checking enterprise innovation behavior same group effect |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |