CN114968721A - Big data analysis processing system based on neural network - Google Patents
Big data analysis processing system based on neural network Download PDFInfo
- Publication number
- CN114968721A CN114968721A CN202210660802.XA CN202210660802A CN114968721A CN 114968721 A CN114968721 A CN 114968721A CN 202210660802 A CN202210660802 A CN 202210660802A CN 114968721 A CN114968721 A CN 114968721A
- Authority
- CN
- China
- Prior art keywords
- data
- analysis
- application
- tit
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3034—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a storage system, e.g. DASD based or network based
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
- G06F11/324—Display of status information
- G06F11/327—Alarm or error message display
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention belongs to the field of big data, relates to a neural network technology, and is used for solving the problem that the existing big data processing system does not have the function of regulating data storage of different data storage nodes under the same label, in particular to a big data analysis processing system based on a neural network, which comprises a distributed storage platform and an application platform, wherein the distributed storage platform is in communication connection with the application platform, the distributed storage platform is constructed by the neural network technology, the distributed storage platform is also in communication connection with a stability analysis module and a data regulation module, and the application platform is in communication connection with an tit management module and an early warning module; according to the invention, through the constructed distributed storage platform, structured, unstructured and semi-structured data are simultaneously placed on each distributed storage node, and the reasonability of data storage of the analysis node is judged through the analysis result, so that the working stability and the efficiency value of the data storage node are ensured.
Description
Technical Field
The invention belongs to the field of big data, relates to a neural network technology, and particularly relates to a big data analysis processing system based on a neural network.
Background
Big data analysis refers to the analysis of huge-scale data, the big data can be summarized into 5V, the data volume is large, the speed is high, the types are multiple, the value and the authenticity are high, the big data is used as the vocabulary of the IT industry which is the most hot at present, the utilization of the commercial values of the big data such as data warehouse, data safety, data analysis and data mining, which is carried out along with the big data, becomes the profit focus which is struggled by industry people, and the big data analysis also comes along with the coming of the big data era.
The existing big data processing system does not have the function of data storage regulation of different data storage nodes under the same label, so that the stability and the efficiency value of the data storage nodes are affected, and meanwhile, the data early warning analysis of a big data platform through the labeling of the data nodes is not facilitated.
In view of the above technical problem, the present application proposes a solution.
Disclosure of Invention
The invention aims to provide a big data analysis and processing system based on a neural network, which is used for solving the problem that the existing big data processing system does not have the function of regulating data storage of different data storage nodes under the same label;
the technical problems to be solved by the invention are as follows: how to provide a big data analysis processing system which can carry out data storage adjustment on different data storage nodes under the same label.
The purpose of the invention can be realized by the following technical scheme:
the big data analysis and processing system based on the neural network comprises a distributed storage platform and an application platform, wherein the distributed storage platform is in communication connection with the application platform, the distributed storage platform is constructed through a neural network technology, the distributed storage platform is also in communication connection with a stability analysis module and a data regulation module, and the application platform is in communication connection with an tit management module and an early warning module;
marking the distributed storage nodes as analysis nodes i, i is 1, 2, …, n is a positive integer;
the stability analysis module is used for detecting and analyzing the stability of the storage nodes of the distributed storage platform, judging whether the data storage of the analysis node i is reasonable or not, and sending an adjustment signal to the data adjustment module when the data storage of the analysis node i is unreasonable;
after receiving the adjusting signal, the data adjusting module performs data storage adjustment on the analysis nodes to enable the data storage of the analysis node i to be reasonable;
the tit management module is used for generating tit values for the application platform, managing the tit values and analyzing the tit values to obtain the data grade of the distributed storage platform;
the early warning module is used for carrying out early warning analysis on the application objects of the application platform through the data grade of the distributed storage platform.
As a preferred embodiment of the present invention, a specific process of the stability analysis module performing detection analysis on the stability of the storage nodes of the distributed storage platform includes: acquiring the quantity of the storage data of the analysis node i and marking the quantity as SLi, acquiring the memory value of the storage data of the analysis node i and marking the memory value as NCi, acquiring the calling times of the storage data of the analysis node i and marking the calling times as DYi, and obtaining the application coefficient of the analysis node i by carrying out numerical calculation on SLi, NCi and DYi; establishing an application set { YY1, YY2, …, YYn } of the application coefficient YYi of the analysis node i, carrying out variance calculation on the application set of the analysis node i to obtain an application performance value, comparing the application performance value with an application performance threshold value, and judging whether the data storage of the analysis node i is reasonable or not according to the comparison result.
As a preferred embodiment of the present invention, the comparing process of the application performance value and the application performance threshold value includes: if the application performance value is less than or equal to the application performance threshold value, judging that the data storage of the analysis node i is reasonable; and if the application performance value is larger than the application performance threshold value, judging that the data storage of the analysis node i is unreasonable, sending an adjusting signal to the distributed storage platform by the stable analysis module, and sending the adjusting signal to the data adjusting module after the distributed storage platform receives the adjusting signal.
As a preferred embodiment of the present invention, a specific process of the data adjustment module for adjusting data storage between the analysis nodes includes: the application coefficient YYi of the analysis node i is compared with the application thresholds YYmin, YYmax:
if YYi is less than or equal to YYmin, judging that the application of the analysis node is insufficient, and marking the corresponding analysis node as an insufficient node;
if YYymin is less than YYi and less than YYmax, judging that the application of the analysis node is qualified, and marking the corresponding analysis node as a qualified node;
if YYi is larger than or equal to YYmax, the analysis node is judged to apply interference, and the corresponding analysis node is marked as an interference node;
obtaining tit values of the analysis node i, respectively marking the numbers of interference nodes and insufficient nodes under the same tit value as t1 and t2, and carrying out numerical comparison on t1 and t 2:
if t1 > t2, the corresponding tit value is marked as an increment of tit; if t1 is t2, the corresponding tit value is marked as conservation tit;
if t1 < t2, then mark the corresponding tit value as a subtracted value of tit;
dividing new analysis nodes for added value tit to perform data adjustment, wherein the number of the divided analysis nodes is the difference value between t1 and t 2; and recovering the analysis nodes for data adjustment for reducing the value tit, wherein the quantity of the recovered analysis nodes is the difference between t2 and t 1.
As a preferred embodiment of the present invention, the tit management module is configured to generate tit values for an application platform and manage them: generating a plurality of tit values according to the application industry of an application platform, distributing tit values to analysis nodes, forming a data family by data stored by all the analysis nodes under the same tit value, giving an evaluation value to the tit value, marking the product of the number of the analysis nodes in the data family and the evaluation value as a family value ZZ of the data family, marking the data family corresponding to the family value with the maximum value as a family, and comparing the family value of the family with grading threshold values FJmin and FJmax: if ZZ is less than or equal to FJmin, judging the data grade of the distributed storage platform to be a grade; if FJmin is less than ZZ and less than FJmax, the data grade of the distributed storage platform is judged to be two grades; if ZZ is larger than or equal to FJmax, the data grade of the distributed storage platform is judged to be three grades;
tit, the management module sends the data grade of the distributed storage platform to the application platform, and the application platform sends the received data grade of the distributed storage platform to the early warning module.
As a preferred embodiment of the present invention, after receiving the data class of the distributed storage platform, the early warning module performs early warning analysis on the application object of the application platform: if the data grade of the distributed storage platform is three grades, generating a primary early warning signal and sending the primary early warning signal to a mobile phone terminal of an application platform manager; if the data grade of the distributed storage platform is a second grade, generating a second-grade early warning signal and sending the second-grade early warning signal to a mobile phone terminal of an application platform manager; and if the data grade of the distributed storage platform is one grade, not performing early warning.
The invention has the following beneficial effects:
1. through the constructed distributed storage platform, structured data, unstructured data and semi-structured data are simultaneously placed on each distributed storage node, so that collaborative analysis processing of heterogeneous data is achieved, application coefficient calculation is conducted on analysis nodes, the application coefficient deviation degree of each analysis node is analyzed, the rationality of data storage of the analysis nodes is judged through analysis results, data storage adjustment is conducted when data storage is unreasonable, and therefore working stability and efficiency values of the data storage nodes are guaranteed;
2. the data adjusting module redistributes and adjusts the stored data of the analysis nodes according to tit values when the data storage is unreasonable, the data adjusting process is carried out among the analysis nodes with the same tit value, the tit value is the label of the data storage node, the analysis nodes are labeled before the data adjustment, then the labeled analysis nodes are subjected to data storage adjustment, so that the data storage confusion is prevented, the group values of the data group are influenced according to the interference nodes and the deficiency nodes, and the group values of the data group can be ensured to feed back the real data state of the application platform;
3. generate tit value for application platform through tit management module, set for the analysis node through the tit value and label, can carry out data storage between the analysis node under same label and adjust, in order to guarantee that each analysis node all can stabilize high-efficient work, simultaneously, the data state through the family value of data family for application platform feeds back and generates the data grade, the early warning module generates corresponding early warning signal through the data grade of feedback and sends to application platform managers's cell phone terminal, in time early warning when application platform data is unusual, guarantee that application platform can normal operating.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic block diagram of the present invention as a whole.
Detailed Description
The technical solutions of the present invention will be described below clearly and completely in conjunction with the embodiments, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, the big data analysis processing system based on the neural network includes a distributed storage platform, and the distributed storage platform is communicatively connected with a stability analysis module, a data adjustment module, and an application platform.
The artificial neural network is a technological recurrence of biological neural network under a certain simplification meaning, and as a subject, the artificial neural network is mainly used for building a practical artificial neural network model according to the principle of the biological neural network and the requirement of practical application, designing a corresponding learning algorithm, simulating certain intelligent activity of human brain, and then technically realizing the artificial neural network for solving the practical problem. Therefore, biological neural networks mainly study the mechanism of intelligence; the artificial neural network mainly studies the realization of an intelligent mechanism.
The distributed storage platform is constructed by a neural network technology:
the method comprises the following steps of realizing structured data distributed storage by adopting a transversely-expanded MPP relational database;
realizing distributed storage of semi-structured data by adopting an NOSQL database;
the distributed file system is adopted to realize the distributed storage of the unstructured data;
structured, unstructured and semi-structured data are simultaneously placed on each distributed storage node, so that collaborative analysis processing of heterogeneous data is achieved.
A distributed storage system is characterized in that data is stored on a plurality of independent devices in a scattered manner, a traditional network storage system adopts a centralized storage server to store all data, the storage server becomes the bottleneck of system performance, is also the focus of reliability and safety and cannot meet the requirement of large-scale storage application, the distributed network storage system adopts an expandable system structure, a plurality of storage servers are used for sharing storage load, a position server is used for positioning storage information, the reliability, the availability and the access efficiency of the system are improved, and the system is easy to expand.
Marking the distributed storage nodes as analysis nodes i, i is 1, 2, …, n is a positive integer;
the stability analysis module is used for detecting and analyzing the stability of the storage nodes of the distributed storage platform: acquiring the quantity of storage data of an analysis node i, marking the quantity as SLi, acquiring a memory value of the storage data of the analysis node i, marking the memory value as NCi, acquiring the calling times of the storage data of the analysis node i, marking the calling times as DYi, and acquiring an application coefficient YYi of the analysis node i according to a formula YYi ═ α 1 × SLi + α 2 × NCi + α 3 × DYi, wherein the application coefficient is a numerical value reflecting the application degree of the analysis node, and the larger the numerical value of the application coefficient is, the higher the application degree of the analysis node is; wherein alpha 1, alpha 2 and alpha 3 are all proportionality coefficients, and alpha 1 is more than alpha 2 and more than alpha 3 is more than 1; establishing an application set { YY1, YY2, …, YYn } of an application coefficient YYi of an analysis node i, carrying out variance calculation on the application set of the analysis node i to obtain an application expression value, wherein the application expression value is a numerical value reflecting the application deviation degree of each analysis node, and the larger the numerical value of the application expression value is, the larger the application deviation degree of each analysis node is, namely the lower the rationality of data storage is, the higher the requirement of data storage regulation is; comparing the application performance value to an application performance threshold: if the application performance value is less than or equal to the application performance threshold value, judging that the data storage of the analysis node i is reasonable; if the application performance value is larger than the application performance threshold value, judging that the data storage of the analysis node i is unreasonable, sending an adjusting signal to the distributed storage platform by the stable analysis module, and sending the adjusting signal to the data adjusting module after the distributed storage platform receives the adjusting signal; the application performance threshold is a set threshold, and the numerical value of the application performance threshold can be set by a manager; through the constructed distributed storage platform, structured, unstructured and semi-structured data are placed on each distributed storage node at the same time, so that collaborative analysis processing of heterogeneous data is achieved, application coefficient calculation is conducted on analysis nodes, the application coefficient deviation degree of each analysis node is analyzed, the rationality of data storage of the analysis nodes is judged through analysis results, data storage adjustment is conducted when data storage is unreasonable, and therefore working stability and efficiency values of the data storage nodes are guaranteed.
The data storage adjustment is carried out between the analysis nodes after the data adjustment module receives the adjustment signal, and the specific process of the data storage adjustment comprises the following steps: the application coefficient YYi of the analysis node i is compared with the application thresholds YYmin, YYmax: if YYi is less than or equal to YYmin, judging that the application of the analysis node is insufficient, and marking the corresponding analysis node as an insufficient node; if YYymin is less than YYi and less than YYmax, judging that the application of the analysis node is qualified, and marking the corresponding analysis node as a qualified node; if YYi is larger than or equal to YYmax, the analysis node is judged to apply interference, and the corresponding analysis node is marked as an interference node; the application thresholds YYmin and YYmax are set thresholds, and the values thereof can be set by a manager. The method comprises the steps of obtaining tit values of analysis nodes i, wherein tit values are generated by a tit management module and are used for generating labels for the analysis nodes and dividing ranges for the analysis nodes, and all analysis nodes with the same tit value can perform data storage adjustment; the number of interference nodes and insufficient nodes at the same tit value are respectively marked as t1 and t2, and the t1 and t2 are compared numerically: if t1 is greater than t2, the corresponding tit value is marked as an added value tit, the added value tit indicates that a label of the application pressure of the existing analysis node in the incoming link of the new analysis node needs to be absorbed, namely the number of the analysis nodes in the added value tit is increased, and the family value of the added value tit is synchronously increased under the condition that the tit value has a fixed evaluation value; if t1 is t2, the corresponding tit value is marked as conservation tit; if t1 < t2, the corresponding tit value is marked as a subtraction value tit, that is, the number of analysis nodes in the subtraction value tit is reduced, and if a tit value has a fixed evaluation value, the group value of the subtraction value tit is also reduced synchronously; dividing new analysis nodes for added value tit to perform data adjustment, wherein the number of the divided analysis nodes is the difference value between t1 and t 2; recovering analysis nodes for reducing the value tit, and adjusting data, wherein the quantity of the recovered analysis nodes is the difference value between t2 and t 1; the data adjusting module redistributes and adjusts the stored data of the analysis nodes according to tit values when the data storage is unreasonable, the data adjusting process is carried out among the analysis nodes with the same tit value, the tit value is the label of the data storage node, the analysis nodes are labeled before the data adjustment, then the labeled analysis nodes are subjected to data storage adjustment, so that the data storage confusion is prevented, the group values of the data groups are influenced according to the interference nodes and the deficiency nodes, and the data state of the real application platform can be fed back by the group values of the data groups.
The application platform is in communication connection with tit a management module and an early warning module.
tit the management module is to generate tit values for the application platform and manage them: generating a plurality of tit values according to the application industry of an application platform, wherein the tit value is a label of an analysis node, distributing a tit value to the analysis node, distributing the analysis node with the same attribute with the same tit value, and forming a data family by data stored by all the analysis nodes with the same tit value, wherein the data family is a large data storage set which comprises a data storage subset of a plurality of analysis nodes; an evaluation value is given to the tit value, the evaluation value can be set by a manager or automatically generated according to an industry rule, and the evaluation value represents the evaluation level of the analysis node storage data corresponding to the tit value in the application industry; marking the product of the number of the analysis nodes in the data family and the evaluation value as a family value ZZ of the data family, marking the data family corresponding to the family value with the maximum value as a head family, wherein the tit value of the head family is an integral display label of the application platform storage data, and comparing the family value of the head family with grading threshold values FJmin and FJmax: if ZZ is less than or equal to FJmin, judging the data grade of the distributed storage platform to be a grade; if FJmin is less than ZZ and less than FJmax, the data grade of the distributed storage platform is judged to be two grades; if ZZ is larger than or equal to FJmax, the data grade of the distributed storage platform is judged to be three grades; tit, the management module sends the data grade of the distributed storage platform to the application platform, and the application platform sends the received data grade of the distributed storage platform to the early warning module; tit values are generated for the application platform through an tit management module, tags are set for analysis nodes through tit values, data storage adjustment can be carried out among the analysis nodes under the same tag, so that stable and efficient work of all the analysis nodes can be guaranteed, and meanwhile, data states of the application platform are fed back through family values of data families, and data grades are generated.
The early warning module performs early warning analysis on an application object of the application platform after receiving the data grade of the distributed storage platform: if the data grade of the distributed storage platform is three grades, generating a primary early warning signal and sending the primary early warning signal to a mobile phone terminal of an application platform manager; if the data grade of the distributed storage platform is a second grade, generating a second-grade early warning signal and sending the second-grade early warning signal to a mobile phone terminal of an application platform manager; if the data grade of the distributed storage platform is one grade, no early warning is carried out; the early warning module generates corresponding early warning signals through the fed back data grade and sends the early warning signals to a mobile phone terminal of an application platform manager, early warning is timely carried out when the application platform data are abnormal, and normal operation of the application platform is guaranteed.
The present disclosure is specifically explained below with reference to specific examples.
Example one
Several tit values are generated according to the application industry of the application platform: specifically, when the application platform is applied to the traffic management industry, the tit management module generates tit values of red light running, illegal parking, drunk driving, unlicensed driving, no-license driving and the like for the traffic management platform;
automatically generating an evaluation value for tit according to an industry law: it can be understood that the more serious the violation scenario is, the higher the corresponding evaluation value is, and in addition, the traffic violation handling strength may be hooked with the evaluation value of tit, for example: the evaluation value assigned for drunk driving is 10, the evaluation value assigned for drunk driving is 8, and so on;
for example, under the condition that the data adjusting module ensures that the application degrees of all analysis nodes tend to be the same under the same tit value, a data family with the maximum influence degree on the traffic management platform can be obtained after multiplying the evaluation value of tit by the number of the analysis nodes, and the overall data state of the traffic management platform can be fed back by comparing the family value of the family with the grading threshold value, that is, if the data family corresponding to drunk driving is the family and the family value of the family is greater than the maximum grading threshold value FJmax, the traffic management is confused, and the traffic management needs to be strengthened; conversely, if the data family corresponding to the regulated parking is the first family, and the family value of the first family is less than the minimum grading threshold FJmin, the traffic management is ordered.
When the big data analysis processing system based on the neural network works, a distributed storage platform is constructed through the neural network technology, a stability analysis module is adopted to detect and analyze the stability of storage nodes of the distributed storage platform and judge whether the data storage of an analysis node i is reasonable or not, and an adjustment signal is sent to a data adjustment module when the data storage of the analysis node i is unreasonable; the data adjusting module receives the adjusting signal and then performs data storage adjustment among the analysis nodes to enable the data storage of the analysis node i to be reasonable, tit values of the analysis node i are obtained, all analysis nodes with the same tit value can perform data storage adjustment, a tit management module is adopted to generate tit values for the application platform, manage and analyze the tit values to obtain the data grade of the distributed storage platform, and an early warning module is adopted to perform early warning analysis on application objects of the application platform through the data grade of the distributed storage platform.
The foregoing is merely exemplary and illustrative of the present invention and various modifications, additions and substitutions may be made by those skilled in the art to the specific embodiments described without departing from the scope of the invention as defined in the following claims.
The formulas are obtained by acquiring a large amount of data and performing software simulation, and the coefficients in the formulas are set by the technicians in the field according to actual conditions; such as: formula YYi ═ α 1 × SLi + α 2 × NCi + α 3 × DYi; collecting multiple groups of sample data and setting corresponding application coefficients for each group of sample data by a person skilled in the art; substituting the set application coefficient and the acquired sample data into formulas, forming a ternary linear equation set by any three formulas, screening the calculated coefficients and taking the mean value to obtain values of alpha 1, alpha 2 and alpha 3 which are respectively 3.87, 2.54 and 2.12;
the size of the coefficient is a specific numerical value obtained by quantizing each parameter, so that the subsequent comparison is convenient, and the size of the coefficient depends on the number of sample data and the corresponding application coefficient preliminarily set by a person skilled in the art for each group of sample data; as long as the proportional relationship between the parameter and the quantized value is not affected, for example, the application coefficient is proportional to the value of the number of calls.
In the description herein, references to the description of "one embodiment," "an example," "a specific example," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The preferred embodiments of the invention disclosed above are intended to be illustrative only. The preferred embodiments are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best utilize the invention. The invention is limited only by the claims and their full scope and equivalents.
Claims (6)
1. The big data analysis processing system based on the neural network comprises a distributed storage platform and an application platform, and is characterized in that the distributed storage platform is in communication connection with the application platform, the distributed storage platform is constructed through a neural network technology, the distributed storage platform is also in communication connection with a stability analysis module and a data regulation module, and the application platform is in communication connection with an tit management module and an early warning module;
marking the distributed storage nodes as analysis nodes i, i equals to 1, 2, …, n, n is a positive integer;
the stability analysis module is used for detecting and analyzing the stability of the storage nodes of the distributed storage platform, judging whether the data storage of the analysis node i is reasonable or not, and sending an adjustment signal to the data adjustment module when the data storage of the analysis node i is unreasonable;
after receiving the adjusting signal, the data adjusting module performs data storage adjustment on the analysis nodes to enable the data storage of the analysis node i to be reasonable;
the tit management module is used for generating tit values for the application platform, managing the tit values and analyzing the tit values to obtain the data grade of the distributed storage platform;
the early warning module is used for carrying out early warning analysis on the application objects of the application platform through the data grade of the distributed storage platform.
2. The big data analyzing and processing system based on the neural network as claimed in claim 1, wherein the specific process of the stability analyzing module performing the detection and analysis on the stability of the storage nodes of the distributed storage platform includes: acquiring the quantity of the storage data of the analysis node i and marking the quantity as SLi, acquiring the memory value of the storage data of the analysis node i and marking the memory value as NCi, acquiring the calling times of the storage data of the analysis node i and marking the calling times as DYi, and obtaining the application coefficient of the analysis node i by carrying out numerical calculation on SLi, NCi and DYi; establishing an application set { YY1, YY2, …, YYn } of the application coefficient YYi of the analysis node i, carrying out variance calculation on the application set of the analysis node i to obtain an application performance value, comparing the application performance value with an application performance threshold value, and judging whether the data storage of the analysis node i is reasonable or not according to the comparison result.
3. The neural network-based big data analysis processing system according to claim 2, wherein the comparing process of the application performance value and the application performance threshold value comprises: if the application performance value is less than or equal to the application performance threshold value, judging that the data storage of the analysis node i is reasonable; and if the application performance value is larger than the application performance threshold value, judging that the data storage of the analysis node i is unreasonable, sending an adjusting signal to the distributed storage platform by the stable analysis module, and sending the adjusting signal to the data adjusting module after the distributed storage platform receives the adjusting signal.
4. The big data analyzing and processing system based on the neural network as claimed in claim 2, wherein the specific process of the data adjusting module for adjusting the data storage between the analyzing nodes comprises: comparing the application coefficient YYi of the analysis node i with the application thresholds YYmin, YYmax:
if YYi is less than or equal to YYmin, judging that the application of the analysis node is insufficient, and marking the corresponding analysis node as an insufficient node;
if YYymin is less than YYi and less than YYmax, judging that the application of the analysis node is qualified, and marking the corresponding analysis node as a qualified node;
if YYi is larger than or equal to YYmax, the analysis node is judged to apply interference, and the corresponding analysis node is marked as an interference node;
obtaining tit values of the analysis node i, respectively marking the numbers of interference nodes and insufficient nodes under the same tit value as t1 and t2, and carrying out numerical comparison on t1 and t 2:
if t1 > t2, the corresponding tit value is marked as an increment of tit; if t1 is t2, the corresponding tit value is marked as conservation tit;
if t1 < t2, then mark the corresponding tit value as a subtracted value of tit;
dividing new analysis nodes for added value tit to perform data adjustment, wherein the number of the divided analysis nodes is the difference value between t1 and t 2; and recovering the analysis nodes for data adjustment for reducing the value tit, wherein the quantity of the recovered analysis nodes is the difference between t2 and t 1.
5. The big neural network-based data analysis processing system of claim 2, wherein the tit management module is configured to generate tit values for an application platform and manage them: generating a plurality of tit values according to the application industry of an application platform, distributing tit values to analysis nodes, forming a data family by data stored by all the analysis nodes under the same tit value, giving an evaluation value to the tit value, marking the product of the number of the analysis nodes in the data family and the evaluation value as a family value ZZ of the data family, marking the data family corresponding to the family value with the maximum value as a family, and comparing the family value of the family with grading threshold values FJmin and FJmax: if ZZ is less than or equal to FJmin, judging the data grade of the distributed storage platform to be a grade; if FJmin is less than ZZ and less than FJmax, the data grade of the distributed storage platform is judged to be two grades; if ZZ is larger than or equal to FJmax, the data grade of the distributed storage platform is judged to be three grades;
tit, the management module sends the data grade of the distributed storage platform to the application platform, and the application platform sends the received data grade of the distributed storage platform to the early warning module.
6. The big data analysis and processing system based on the neural network as claimed in claim 5, wherein the early warning module performs early warning analysis on the application objects of the application platform after receiving the data levels of the distributed storage platform: if the data grade of the distributed storage platform is three grades, generating a primary early warning signal and sending the primary early warning signal to a mobile phone terminal of an application platform manager; if the data grade of the distributed storage platform is a second grade, generating a second-grade early warning signal and sending the second-grade early warning signal to a mobile phone terminal of an application platform manager; and if the data grade of the distributed storage platform is one grade, not performing early warning.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210660802.XA CN114968721A (en) | 2022-06-13 | 2022-06-13 | Big data analysis processing system based on neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210660802.XA CN114968721A (en) | 2022-06-13 | 2022-06-13 | Big data analysis processing system based on neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114968721A true CN114968721A (en) | 2022-08-30 |
Family
ID=82962025
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210660802.XA Pending CN114968721A (en) | 2022-06-13 | 2022-06-13 | Big data analysis processing system based on neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114968721A (en) |
-
2022
- 2022-06-13 CN CN202210660802.XA patent/CN114968721A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108418841B (en) | Next-generation key message infrastructure network Security Situation Awareness Systems based on AI | |
| CN107180070B (en) | Automatic risk information classification, identification and early warning method and system | |
| CN108510006A (en) | A kind of analysis of business electrical amount and prediction technique based on data mining | |
| US11367002B1 (en) | Method for constructing and training decentralized migration diagram neural network model for production process | |
| WO2021043140A1 (en) | Method, apparatus and system for determining label | |
| CN108470022B (en) | Intelligent work order quality inspection method based on operation and maintenance management | |
| CN113590698B (en) | Artificial intelligence technology-based data asset classification modeling and hierarchical protection method | |
| CN112363896A (en) | Log anomaly detection system | |
| CN116628774A (en) | Data storage integrity supervision system based on cloud computing | |
| CN114862373B (en) | Block chain-based distributed business ledger management method and system | |
| CN112383630A (en) | Distributed pluggable process data reporting system based on production line tangent plane | |
| TWM622216U (en) | Apparatuses for service anomaly detection and alerting | |
| CN109558298A (en) | Alarm based on deep learning model executes frequency optimization method and relevant device | |
| CN113434575A (en) | Data attribution processing method and device based on data warehouse and storage medium | |
| CN111984514A (en) | Prophet-bLSTM-DTW-based log anomaly detection method | |
| CN114968721A (en) | Big data analysis processing system based on neural network | |
| CN109635008B (en) | Equipment fault detection method based on machine learning | |
| CN116756659A (en) | Intelligent operation and maintenance management method, device, equipment and storage medium | |
| CN114912854B (en) | Subway train operation adjusting method and device, electronic equipment and storage medium | |
| CN116628554A (en) | Industrial Internet data anomaly detection method, system and equipment | |
| CN109978038A (en) | A kind of cluster abnormality determination method and device | |
| CN111030855B (en) | Intelligent baseline determination and alarm method for ubiquitous power Internet of things system data | |
| CN115374661B (en) | Operation and maintenance boarding corridor bridge model detection method and system based on environmental parameters | |
| Duan et al. | Reliability analysis of intelligent manufacturing systems based on improved fmea combined with machine learning | |
| CN113298326B (en) | Intelligent electronic event supervision method, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |