CN117371058A - Data processing method and device, electronic equipment and storage medium - Google Patents
Data processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117371058A CN117371058A CN202311384900.6A CN202311384900A CN117371058A CN 117371058 A CN117371058 A CN 117371058A CN 202311384900 A CN202311384900 A CN 202311384900A CN 117371058 A CN117371058 A CN 117371058A
- Authority
- CN
- China
- Prior art keywords
- data
- processing
- signal
- communication
- initial communication
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 28
- 230000006854 communication Effects 0.000 claims abstract description 391
- 238000004891 communication Methods 0.000 claims abstract description 389
- 238000012545 processing Methods 0.000 claims abstract description 303
- 230000005540 biological transmission Effects 0.000 claims abstract description 120
- 238000000034 method Methods 0.000 claims abstract description 46
- 230000008569 process Effects 0.000 claims abstract description 20
- 238000007781 pre-processing Methods 0.000 claims description 35
- 238000012986 modification Methods 0.000 claims description 22
- 230000004048 modification Effects 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 16
- 238000004140 cleaning Methods 0.000 claims description 11
- 238000010606 normalization Methods 0.000 claims description 11
- 230000001360 synchronised effect Effects 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 9
- 238000013500 data storage Methods 0.000 claims description 7
- 238000012805 post-processing Methods 0.000 claims description 7
- 238000013480 data collection Methods 0.000 claims 1
- 230000004913 activation Effects 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 238000012550 audit Methods 0.000 description 6
- 230000009849 deactivation Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000002372 labelling Methods 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 4
- 230000003213 activating effect Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013524 data verification Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000013138 pruning Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/70—Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer
- G06F21/78—Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure storage of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Communication Control (AREA)
Abstract
The invention discloses a data processing method, a data processing device, electronic equipment and a storage medium. The data processing method comprises the following steps: selecting a preset transmission path for transmission processing according to a corresponding transmission rate from an initial communication signal, and taking data output by the preset transmission path as a communication processing signal; determining a comparison result of the initial communication signal and the communication processing signal, and determining the processing type of the communication processing signal according to the comparison result; and storing the related information of the initial communication signal, the related information of the communication processing signal and the comparison result belonging to the same processing type in a data information set corresponding to the processing type. The embodiment of the invention realizes the classified storage of the communication data, is convenient for trace backtracking of the communication data, and avoids the problem of data loss in the data processing process.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data processing method, a data processing device, an electronic device, and a storage medium.
Background
With the continuous development of internet communication technology, the stability and safety requirements of communication in the power grid are higher and higher, so that the barrier-free transmission of communication data is realized, the safety of communication data information transmission is ensured, the accuracy and reliability of data transmission are improved, and the communication data information needs to be monitored to ensure the safe operation of the data.
The existing centralized control function is not perfect enough when the communication data information is monitored, the situation that the communication data information is lack of data or is not accurate enough when being conducted can occur, and once the data information is lack, the information is inconvenient to trace back, so that the information accuracy is affected. Therefore, a data processing method is needed to solve the problems in the prior art.
Disclosure of Invention
The invention provides a data processing method, a data processing device, electronic equipment and a storage medium, which are used for solving the problems that in the prior art, data loss or insufficient conduction can occur in the process of conducting communication data information.
According to an aspect of the present invention, there is provided a data processing method, wherein the method includes:
selecting a preset transmission path for transmission processing according to a corresponding transmission rate from an initial communication signal, and taking data output by the preset transmission path as a communication processing signal;
Determining a comparison result of the initial communication signal and the communication processing signal, and determining the processing type of the communication processing signal according to the comparison result;
and storing the related information of the initial communication signal, the related information of the communication processing signal and the comparison result belonging to the same processing type in a data information set corresponding to the processing type.
According to another aspect of the present invention, there is provided a data processing apparatus, wherein the apparatus comprises:
the signal acquisition module is used for selecting a preset transmission channel for transmission processing according to the corresponding transmission rate of the initial communication signal, and taking data output by the preset transmission channel as a communication processing signal;
the type determining module is used for determining a comparison result of the initial communication signal and the communication processing signal and determining the processing type of the communication processing signal according to the comparison result;
and the data storage module is used for storing the related information of the initial communication signals, the related information of the communication processing signals and the comparison result belonging to the same processing type in a data information set corresponding to the processing type.
According to another aspect of the present invention, there is provided an electronic apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the data processing method according to any one of the embodiments of the present invention.
According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to execute a data processing method according to any one of the embodiments of the present invention.
According to the technical scheme, the initial communication signals are selected to be transmitted through the preset transmission paths according to the corresponding transmission rates, the data output by the preset transmission paths are used as communication processing signals, the comparison results of the initial communication signals and the communication processing signals are determined, the processing types of the communication processing signals are determined according to the comparison results, the related information of the initial communication signals, the related information of the communication processing signals and the comparison results which belong to the same processing type are stored in the data information sets corresponding to the processing types, so that the classified storage of communication data is realized, trace backtracking of the communication data is facilitated, and the problem of data loss in the data processing process is avoided.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a data processing method according to a first embodiment of the present invention;
FIG. 2 is a flow chart of another data processing method according to a second embodiment of the present invention;
FIG. 3 is a flow chart of yet another data processing method according to a third embodiment of the present invention;
FIG. 4 is a schematic diagram of a data processing apparatus according to a fourth embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device implementing a data processing method according to an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
Fig. 1 is a flowchart of a data processing method according to a first embodiment of the present invention, where the method may be applied to a communication signal transmission process and classified and stored, and the method may be performed by a data processing apparatus, where the data processing apparatus may be implemented in hardware and/or software, and the data processing apparatus may be configured in an electronic device. As shown in fig. 1, the method includes:
s110, selecting a preset transmission path for transmission processing according to the corresponding transmission rate of the initial communication signal, and taking data output by the preset transmission path as a communication processing signal.
The initial communication signal refers to an initial signal that is not subjected to data processing. In actual operation, the initial communication signal may be a synchronous digital hierarchy (Synchronous Digital Hierarchy, SDH) signal. Different initial communication signals may correspond to different transmission rates. The preset transmission path refers to a preset path for transmitting a communication signal, and the corresponding transmission rate is different for different preset transmission paths. In an actual operation process, the preset transmission path may be a container path for constructing transmission based on a synchronous digital hierarchy cross processing mode, for example, constructing that VC12, VC3, and VC4 operate in a VC4 service layer.
In the preset transmission path, transmission processing may be performed on the initial communication signal. By way of example, data processing may include, but is not limited to, modifying, deleting, activating, deactivating, etc. The communication processing signals refer to initial communication signals after transmission processing through a preset transmission channel, and each initial communication signal has a corresponding communication processing signal.
In the embodiment of the invention, a corresponding preset transmission channel can be selected according to the transmission rate of the initial communication signal, and the communication processing signal is obtained by transmitting and outputting the initial communication signal through each preset transmission channel. In practical application, the initial communication signals may be divided according to transmission rates, the initial communication signals belonging to each transmission rate are transmitted through corresponding preset transmission paths, and data output by the preset transmission paths are used as communication processing signals.
S120, determining a comparison result of the initial communication signal and the communication processing signal, and determining the processing type of the communication processing signal according to the comparison result.
The comparison result may include a result of whether the communication processing signal is different from the initial communication signal. In one embodiment, the comparison result may include, but is not limited to, the data modification content of the communication processing signal compared to the initial communication signal, whether there is a modification, and the like. The processing type refers to a processing type of the communication processing signal as compared to the initial communication signal, and may include, by way of example and not limitation, adding content, pruning content, activation of information, deactivation of information, and the like.
In the embodiment of the invention, the comparison result of the communication processing signal and the initial communication signal can be determined, and the corresponding processing type of the communication processing signal can be determined. In the actual operation process, the preset identification information of the initial communication signals and the communication processing signals can be extracted, the corresponding relation between the initial communication signals and the communication processing signals is determined, the initial communication signals and the communication processing signals with the corresponding relation are compared, and the comparison result is determined. When the comparison result is the added content, the processing type of the communication processing signal can be determined to be the added content; when the comparison result is the censored content, the processing type of the communication processing signal can be determined to be the censored content; when the comparison result is information activation, the processing type of the communication processing signal can be determined to be information activation; when the comparison result is that the information is deactivated, determining that the processing type of the communication processing signal is that the information is deactivated; when the comparison result is that the communication processing signal is the same as the initial communication signal or the unprocessed communication processing signal, it may be determined that the processing type of the communication processing signal is unprocessed.
And S130, storing related information of the initial communication signals, related information of the communication processing signals and comparison results belonging to the same processing type in a data information set corresponding to the processing type.
The related information of the communication processing signal may include information such as an initial communication signal and preset identification information corresponding to the initial communication signal; the related information of the communication processing signal may include the communication processing signal and preset identification information corresponding to the communication processing signal. The data information set refers to a set of stored data created in advance, and for each processing type, there is a corresponding data information set.
In the embodiment of the invention, the communication processing signals belonging to the same processing type and the corresponding initial communication signals can be determined, the communication processing signals belonging to the same processing type and the corresponding initial communication signals are constructed into a data information set, the related information of the initial communication signals and the related information of the communication processing signals are determined, and the related information of the initial communication signals, the related information of the communication processing signals and the comparison result are stored in the data information set corresponding to the processing type.
In an embodiment, a comparison database may be constructed in the data information set, and the related information of the initial communication signal and the related information of the communication processing signal are recorded respectively, so as to facilitate monitoring of the communication data.
According to the technical scheme, the initial communication signals are selected to be transmitted through the preset transmission paths according to the corresponding transmission rates, the data output by the preset transmission paths are used as communication processing signals, the comparison results of the initial communication signals and the communication processing signals are determined, the processing types of the communication processing signals are determined according to the comparison results, the related information of the initial communication signals, the related information of the communication processing signals and the comparison results which belong to the same processing type are stored in the data information sets corresponding to the processing types, so that the classified storage of communication data is realized, trace backtracking of the communication data is facilitated, and the problem of data loss in the data processing process is avoided.
In an embodiment, before selecting the preset transmission path for transmission processing according to the corresponding transmission rate from the initial communication signal, and taking the data output by the preset transmission path as the communication processing signal, the method further includes:
constructing preset transmission paths with different transmission rates based on a synchronous digital hierarchy cross processing mode;
and constructing data processing codes of all preset transmission paths so that all preset transmission paths perform data processing on the initial communication signals.
The synchronous digital hierarchy cross processing mode can comprise a high-order cross of a synchronous digital hierarchy and a low-order cross of the synchronous digital hierarchy. The high-order crossover is a crossover in VC4, and the low-order crossover is a crossover in VC 12. The data processing coding refers to coding that enables each preset transmission path to perform data processing capability on an initial communication signal, and the data processing coding can be coded in a Non-Return-to-Zero (NRZ) coding manner, wherein each bit represents 0 or 1 at a constant level. Data processing encoding of functions of adding content, deleting content, activating information, deactivating information, and the like may be provided.
In the embodiment of the invention, an initial communication signal is assembled in a standard container C12 to be corresponding through rate adaptation, then the C12 is packed and added with corresponding low-order channel overhead to form an information structure of the VC12, the VC3 and the VC4 of the SDH circuit can be obtained according to conversion rate operation among the VC12, the VC3 and the VC4, and the VC12, the VC3 and the VC4 channels with different transmission rates can be constructed based on a synchronous digital hierarchy cross processing mode to serve as preset transmission paths. After the preset transmission paths are constructed, data processing codes of the preset transmission paths can be constructed, so that the preset transmission paths perform data processing on the initial communication signals.
In one embodiment, after the data processing code is constructed, the composed data processing code may be written to each of the predetermined transmission paths. And debugging each preset transmission channel again, and verifying whether the preset transmission channels can be modified, deleted, activated, deactivated and the like, so that the normal operation of each preset transmission channel is ensured.
In an embodiment, the data processing method further includes:
performing data preprocessing on the original communication signals to generate initial communication signals, and storing the initial communication signals in an initial communication data set; the data preprocessing at least comprises warehousing, cleaning, association, de-duplication, normalization and calculation;
dividing initial communication signals in an initial communication data set into at least two sub-initial communication data sets according to the transmission rate of each initial communication signal, wherein the transmission rates of the initial communication signals in the same sub-initial communication data set are the same;
and determining alarm related information of the initial communication signal, and taking the alarm related information as preset identification information of the initial communication signal.
Where the original communication signal refers to an initial communication signal that has not been data pre-processed. Data preprocessing refers to some data processing that occurs before the main processing. Data preprocessing may include, but is not limited to, warehousing, cleaning, correlation, deduplication, normalization, and computation. Binning refers to storing the original communication signals in an initial communication data set; cleaning refers to the process of rechecking and verifying the data for the original communication signal; the association refers to the process of associating the original communication signals, and because the original communication signals come from different sources, the original communication signals need to be associated together, and the association operation involves database connection, connection inquiry, data combination and other operations; deduplication refers to deleting duplicate content of the original communication signal, leaving a unique original communication signal in the initial communication data set; normalization refers to normalization of data features; calculation refers to the calculation process performed on the original communication signal.
The alarm related information refers to alarm information related to the initial communication signal, and the alarm related information may include alarm source reporting, alarm convergence processing, various alarm channel access, and the like, for example. The preset identification information refers to information for identifying the initial communication signal, and the corresponding initial communication signal can be queried according to the preset identification information.
In the embodiment of the invention, data preprocessing such as warehousing, cleaning, association, de-duplication, normalization and calculation can be performed on the original communication signals to generate initial communication signals, and the initial communication signals are stored in an initial communication data set. Determining the transmission rate of each initial communication signal, dividing the initial communication signals in the initial communication data sets into at least two sub-initial communication data sets according to the transmission rate of each initial communication signal, wherein the transmission rate of the initial communication signals in each sub-initial communication data set is the same. Alarm source reporting, alarm convergence processing, various alarm channel arrival and other alarm related information of each initial communication signal are determined, and the alarm related information is used as preset identification information of the initial communication signals, so that the initial communication data can be supported and positioned to a topology through the preset identification information, and meanwhile, related information such as historical alarms and the like can be checked.
In one embodiment, before the data preprocessing of the original communication signal to generate the initial communication signal, the method further comprises:
creating a data set corresponding to each data preprocessing aiming at each data preprocessing;
wherein the data set comprises at least: the data sets comprise a pre-processing data subset and a post-processing data subset.
In an embodiment of the present invention, before each data preprocessing is performed, a data set corresponding to each data preprocessing may be created for each data preprocessing, for example, a warehouse entry data set, a cleaning data set, an association data set, a deduplication data set, a normalization data set, and a calculation data set. Meanwhile, a pre-processing data subset and a post-processing data subset may be included in each data set, wherein the pre-processing data subset internally includes a pre-processing data set, and the post-processing data subset includes a post-processing data set.
In an embodiment, the subset of processed data in the previous step of data pre-processing data set should be the same as the subset of processed data in the subsequent step of data pre-processing data set. Illustratively, the processed data subset of the warehouse-in data set is the same as the pre-processed data subset of the cleaning data set, so that the backtracking and rollback of the data are facilitated, and the loss of the data is prevented.
In an embodiment, the data processing method further includes:
at least one of date, time and data modification content is noted for each communication signal pair, and a log is generated.
Wherein, the communication signal pair refers to a signal pair generated by an initial communication signal and a communication processing signal with the same preset identification information.
In the embodiment of the invention, after the related information of the initial communication signal, the related information of the communication processing signal and the comparison result which belong to the same processing type are stored in the data information set corresponding to the processing type, the recorded communication signal pair can be marked. Labeling content includes, but is not limited to, date, time, data modification content, etc., and after labeling is completed, a detailed log is formed by the communication signal pair.
In an embodiment, after the log is recorded, the log can be audited periodically, and the reverse verification is performed on the content recorded by the log and whether the change recorded by the log occurs in the communication processing data in an information rollback mode, so that the data monitoring function is improved.
Example two
Fig. 2 is a flowchart of another data processing method according to a second embodiment of the present invention, which is further optimized and expanded based on the foregoing embodiment, and may be combined with various alternative technical solutions in the foregoing embodiment. As shown in fig. 2, the method includes:
S210, selecting a preset transmission path for transmission processing according to the corresponding transmission rate of the initial communication signal, and taking data output by the preset transmission path as a communication processing signal.
S220, extracting preset identification information of each initial communication signal and each communication processing signal.
In the embodiment of the invention, the preset identification information of each initial communication signal can be extracted, and because the communication processing signal is a signal that is transmitted and processed by the initial communication signal through the preset transmission channel, the communication processing signal also has the preset identification information correspondingly.
S230, taking the initial communication signal and the communication processing signal with the same preset identification information as a communication signal pair.
In the embodiment of the invention, an initial communication signal and a communication processing signal with the same preset identification information are acquired, and the initial communication signal and the communication processing signal with the same preset identification information are used as a communication signal pair. In the actual operation process, when the initial communication signal is transmitted and processed through the preset transmission channel, the corresponding preset identification information is not changed, so that the initial communication signal and the communication processing signal with the same preset identification information are communication data before and after processing and serve as a communication data pair.
S240, determining data modification content of the communication processing signals in each communication signal pair relative to the initial communication signals, and generating comparison results.
Wherein data modification content refers to data modification content of the communication processing signal relative to the initial communication signal in the communication signal pair, such as addition of content, deletion of content, activation of information, deactivation or non-modification of information, etc.
In the embodiment of the invention, the communication processing signals in the communication signal pairs can be extracted relative to the initial communication signals, the data modification content of the communication processing signals relative to the initial communication signals is determined, and the comparison result is determined. In the actual operation process, when the data modification content of the communication processing signal relative to the initial communication signal is added content, generating a comparison result as added content; when the communication processing signal modifies the content into the censored content relative to the data of the initial communication signal, generating a comparison result into the censored content; when the data modification content of the communication processing signal relative to the initial communication signal is the deactivation of the information, generating a comparison result to be the deactivation of the information; when the data modification content of the communication processing signal relative to the initial communication signal is the activation of the information, generating a comparison result to be the activation of the information; when the data modification content of the communication processing signal relative to the initial communication signal is unmodified, generating a comparison result as unmodified.
S250, determining the processing type of each communication processing signal according to the comparison result.
In the embodiment of the invention, after the comparison result is determined, the processing type of each communication processing signal can be determined. In an embodiment, when the comparison result is the added content, it may be determined that the processing type of the communication processing signal is the added content; when the comparison result is the censored content, the processing type of the communication processing signal can be determined to be the censored content; when the comparison result is information activation, the processing type of the communication processing signal can be determined to be information activation; when the comparison result is that the information is deactivated, determining that the processing type of the communication processing signal is that the information is deactivated; when the comparison result is unprocessed, it may be determined that the processing type of the communication processing signal is unprocessed.
S260, determining a communication signal pair where the communication processing signals belonging to the same processing type are located, and constructing a data information set by the communication signal pair belonging to the same processing type.
In the embodiment of the invention, the communication signal pairs belonging to the same data processing type can be determined, and the communication signal pairs belonging to the same processing type are constructed into the data information set, that is, the communication signal pairs belonging to the same processing type are stored in the same data information set.
S270, taking the preset identification information corresponding to the initial communication signal as the related information of the initial communication signal.
S280, taking preset identification information corresponding to the communication processing signal as related information of the communication processing signal.
And S290, storing the related information of the initial communication signal, the related information of the communication processing signal and the comparison result in a data information set corresponding to the processing type according to the processing type.
In the embodiment of the invention, after the related information of the initial communication signal and the related information of the communication processing signal are determined, the communication signal pair belonging to the same processing type can be constructed into a data information set, and the related information of the initial communication signal, the related information of the communication processing signal and the comparison result are stored in the data information set corresponding to the processing type.
According to the technical scheme, the preset transmission channel is selected for transmission processing according to the corresponding transmission rate, data output by the preset transmission channel is used as communication processing signals, preset identification information of each initial communication signal and each communication processing signal is extracted, the initial communication signal and the communication processing signal with the same preset identification information are used as communication signal pairs, data modification content of the communication processing signals in the communication signal pairs relative to the initial communication signals is determined, a comparison result is generated, the processing type of each communication processing signal is determined according to the comparison result, the communication signal pair where the communication processing signals belonging to the same processing type are located is determined, the communication signal pair belonging to the same processing type is used for constructing a data information set, preset identification information corresponding to the initial communication signal and the initial communication processing signal is used as related information of the communication processing signal, the preset identification information corresponding to the communication processing signal is used as related information of the communication processing signal, the related information of the initial communication signal, the related information of the communication processing signal and the comparison result are stored in the data information set corresponding to the processing type according to the processing type, and the communication data is prevented from being lost in the data processing track.
Example III
Fig. 3 is a flowchart of yet another data processing method according to a third embodiment of the present invention, where, based on the foregoing embodiment, an original communication signal is taken as an original SDH signal, a communication processing signal is a SDH processing signal, and preset transmission paths are VC12, VC3, and VC4 paths, for further explanation of a data processing method. As shown in fig. 3, the method includes:
s310, carrying out data preprocessing of warehousing, cleaning, association, de-duplication, normalization and calculation on the original SDH signal to generate an initial SDH signal, and storing the initial SDH signal in an initial communication data set.
In an embodiment, before the raw SDH signal is subjected to data preprocessing of binning, cleaning, correlating, deduplication, normalization and computation, the raw SDH signal may be screened to determine the raw SDH signal that needs deduplication.
In an embodiment, a data set corresponding to each data preprocessing is created for each data preprocessing, and the data set at least includes: the data sets comprise a pre-processing data subset and a post-processing data subset. Meanwhile, the processed data subset in the previous step of data preprocessing data set should be identical to the processed data subset in the next step of data preprocessing data set.
S320, determining alarm related information of the initial SDH signal, and taking the alarm related information as preset identification information of the initial SDH signal.
In one embodiment, the network management and monitoring tool can be used to automatically mark and record alarm related information, capture and analyze network data, and then add preset identification information to the original SDH signal according to the device configuration and management policies.
S330, dividing the initial SDH signals in the initial communication data sets into at least two sub-initial communication data sets according to the transmission rate of each initial SDH signal.
Wherein the transmission rates of the initial communication signals in the same sub-initial communication data set are the same.
S340, the initial SDH signals in the sub-initial communication data set are adapted to be arranged in a standard container C12 to be corresponding through the transmission rate, and VC12, VC3, VC4 channels and VC4 service layers with different transmission rates are constructed based on a synchronous digital hierarchy cross-processing mode.
In an embodiment, repeated testing is required for the constructed VC12 channels, so as to ensure that the VC12 path is stable and cannot collapse due to transmission of a data stream, and the VC12 after passing the testing can be used for converting creation of VC12, VC3, VC4 and VC4 service layers required for SDH improvement.
S350, constructing data processing codes of the VC12, VC3 and VC4 paths so that the VC12, VC3 and VC4 paths perform data processing on the initial SDH signals.
The data processing coding refers to coding that enables each preset transmission path to perform data processing capability on an initial communication signal, and the data processing coding can be coded in a Non-Return-to-Zero (NRZ) coding mode, wherein each bit represents 0 or 1 at a constant level. Data processing encoding of functions of adding content, deleting content, activating information, deactivating information, and the like may be provided.
In an embodiment, after constructing the data processing codes of the VC12, VC3 and VC4 channels, a batch of initial SDH signals may be re-entered, and based on the transmission rate of the initial SDH signals, the data is verified to be changed or missing after transmission through the VC12, VC3 and VC4 channels, and thus, whether the data processing coding function is completed is verified to be perfect, and if the SDH path works normally after data verification, the next operation may be performed.
S360, selecting a preset transmission path for transmission processing according to the corresponding transmission rate of the initial communication signal, and taking data output by the preset transmission path as a communication processing signal.
S370, determining a comparison result of the initial communication signal and the communication processing signal, and determining the processing type of the communication processing signal according to the comparison result.
And S380, storing the related information of the initial communication signals, the related information of the communication processing signals and the comparison result belonging to the same processing type in a data information set corresponding to the processing type.
In an embodiment, the operations when the VC12, VC3 and VC4 channels process information data are recorded, the recorded contents include an initial SDH signal of the operations of the VC12, VC3 and VC4 channels, an SDH processed signal after SDH processing, a comparison database is built in a data information set, the initial data and the processed data are respectively recorded and recorded, and the built data information set should include added contents of an SDH path, deleted contents of the SDH path, activation of SDH data information, detailed log audit and rollback of the deactivation operation of the SDH data information.
S390, marking at least one of date, time and data modification content for each communication signal pair.
In an embodiment, after storing the related information of the initial communication signal, the related information of the communication processing signal and the comparison result belonging to the same processing type in the data information set corresponding to the processing type, the recorded initial SDH signal and SDH processing signal may be marked, and the marked content includes, but is not limited to, date, time, data modification content and the like. After the labeling of the archived information is completed, a detailed log is formed, after each information processing, the processing measure and other contents are required to form the log for recording, after the log recording, the recorded log is audited regularly, and the reverse verification is carried out on the recorded content of the log and whether the change recorded by the log occurs or not in the mode of information rollback.
In an embodiment, after the log is recorded, the interval time of the log audit time can be determined, the log is audited regularly, and the reverse verification is performed on the recorded content of the log and whether the change recorded by the log occurs in the communication processing data in an information rollback mode, so that the data monitoring function is perfected. In the actual operation process, rollback is needed each time log audit is carried out, and after audit is completed, the audit content and audit time are formed into a document to be stored.
Example IV
Fig. 4 is a schematic structural diagram of a data processing apparatus according to a fourth embodiment of the present invention. As shown in fig. 4, the apparatus includes: the signal acquisition module 41, the type determination module 42 and the data storage module 43.
The signal obtaining module 41 is configured to select a preset transmission path according to a corresponding transmission rate for transmission processing of the initial communication signal, and use data output by the preset transmission path as a communication processing signal.
The type determining module 42 is configured to determine a comparison result of the initial communication signal and the communication processing signal, and determine a processing type of the communication processing signal according to the comparison result.
The data storage module 43 is configured to store the related information of the initial communication signal, the related information of the communication processing signal, and the comparison result, which belong to the same processing type, in the data information set corresponding to the processing type.
According to the technical scheme, the signal acquisition module is used for selecting the preset transmission channel according to the corresponding transmission rate to carry out transmission processing on the initial communication signal, the data output by the preset transmission channel is used as the communication processing signal, the type determining module is used for determining the comparison result of the initial communication signal and the communication processing signal, the processing type of the communication processing signal is determined according to the comparison result, and the data storage module is used for storing the related information of the initial communication signal, the related information of the communication processing signal and the comparison result which belong to the same processing type in the data information set corresponding to the processing type, so that the classified storage of communication data is realized, the trace backtracking of the communication data is facilitated, and the problem of data loss in the data processing process is avoided.
In one embodiment, the type determination module 42 includes:
an information extraction unit for extracting preset identification information of each of the initial communication signal and the communication processing signal;
a signal pair determining unit configured to take an initial communication signal and a communication processing signal having the same preset identification information as a communication signal pair;
a result generation unit for determining the data modification content of the communication processing signal relative to the initial communication signal in each communication signal pair and generating a comparison result;
And the type determining unit is used for determining the processing type of each communication processing signal according to the comparison result.
In one embodiment, the data storage module 43 includes:
the data set construction unit is used for determining communication signal pairs where communication processing signals belonging to the same processing type are located and constructing a data information set by the communication signal pairs belonging to the same processing type;
the first information determining unit is used for taking the initial communication signal and preset identification information corresponding to the initial communication signal as related information of the initial communication signal;
a second information determining unit, configured to use the communication processing signal and preset identification information corresponding to the communication processing signal as related information of the communication processing signal;
and the data storage unit is used for storing the related information of the initial communication signal, the related information of the communication processing signal and the comparison result in a data information set corresponding to the processing type according to the processing type.
In an embodiment, the data processing apparatus further comprises:
the channel construction module is used for constructing preset transmission channels with different transmission rates based on a synchronous digital hierarchy cross processing mode;
the code construction module is used for constructing data processing codes of all preset transmission paths so that all preset transmission paths can perform data processing on the initial communication signals.
In an embodiment, the data processing apparatus further comprises:
the initial signal determining module is used for carrying out data preprocessing on the original communication signals to generate initial communication signals and storing the initial communication signals in an initial communication data set; the data preprocessing at least comprises warehousing, cleaning, association, de-duplication, normalization and calculation;
the sub-set dividing module is used for dividing the initial communication signals in the initial communication data sets into at least two sub-initial communication data sets according to the transmission rate of each initial communication signal, wherein the transmission rates of the initial communication signals in the same sub-initial communication data set are the same;
and the alarm information determining module is used for determining alarm related information of the initial communication signal and taking the alarm related information as preset identification information of the initial communication signal.
In an embodiment, the data processing apparatus further comprises:
the set creating module is used for creating a data set corresponding to each data preprocessing aiming at each data preprocessing;
wherein the data set comprises at least: the data sets comprise a pre-processing data subset and a post-processing data subset.
In an embodiment, the data processing apparatus further comprises:
and the signal labeling module is used for labeling at least one of date, time and data modification content for each communication signal pair and generating a log.
The data processing device provided by the embodiment of the invention can execute the data processing method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example five
Fig. 5 is a schematic structural diagram of an electronic device 10 implementing a data processing method according to an embodiment of the present invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 5, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as data processing methods.
In some embodiments, the data processing method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. One or more of the steps of the data processing method described above may be performed when the computer program is loaded into RAM 13 and executed by processor 11. Alternatively, in other embodiments, the processor 11 may be configured to perform the data processing method in any other suitable way (e.g. by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. A method of data processing, comprising:
selecting a preset transmission path for transmission processing according to a corresponding transmission rate from an initial communication signal, and taking data output by the preset transmission path as a communication processing signal;
determining a comparison result of the initial communication signal and the communication processing signal, and determining the processing type of the communication processing signal according to the comparison result;
And storing the related information of the initial communication signal, the related information of the communication processing signal and the comparison result belonging to the same processing type in a data information set corresponding to the processing type.
2. The method of claim 1, wherein said determining a comparison of said initial communication signal and said communication process signal, and determining a process type of said communication process signal based on said comparison, comprises:
extracting preset identification information of each initial communication signal and each communication processing signal;
taking the initial communication signal and the communication processing signal which have the same preset identification information as a communication signal pair;
determining the data modification content of the communication processing signals relative to the initial communication signals in each communication signal pair, and generating comparison results;
and determining the processing type of each communication processing signal according to the comparison result.
3. The method according to claim 2, wherein storing the correlation information of the initial communication signal, the correlation information of the communication processing signal, and the comparison result belonging to the same processing type in the data information set corresponding to the processing type comprises:
Determining the communication signal pairs where the communication processing signals belonging to the same processing type are located, and constructing a data information set by the communication signal pairs belonging to the same processing type;
taking the initial communication signal and preset identification information corresponding to the initial communication signal as related information of the initial communication signal;
taking the preset identification information corresponding to the communication processing signal as related information of the communication processing signal;
and storing the related information of the initial communication signal, the related information of the communication processing signal and the comparison result in a data information set corresponding to the processing type according to the processing type.
4. The method according to claim 1, further comprising, before said selecting a predetermined transmission path for transmission processing of the initial communication signal at the corresponding transmission rate and taking data output from the predetermined transmission path as the communication processing signal:
constructing the preset transmission paths with different transmission rates based on a synchronous digital hierarchy cross processing mode;
and constructing data processing codes of the preset transmission paths so that the preset transmission paths can process the data of the initial communication signals.
5. The method according to claim 1, characterized in that the method further comprises:
performing data preprocessing on an original communication signal to generate the initial communication signal, and storing the initial communication signal in an initial communication data set; the data preprocessing at least comprises warehousing, cleaning, association, de-duplication, normalization and calculation;
dividing an initial communication signal in the initial communication data set into at least two sub-initial communication data sets according to the transmission rate of each initial communication signal, wherein the transmission rates of the initial communication signals in the same sub-initial communication data set are the same;
and determining alarm related information of the initial communication signal, and taking the alarm related information as preset identification information of the initial communication signal.
6. The method of claim 5, further comprising, prior to said data preprocessing the original communication signal to generate said initial communication signal:
creating a data set corresponding to each data preprocessing aiming at each data preprocessing;
wherein the data set comprises at least: the data collection system comprises a warehouse data set, a cleaning data set, an association data set, a deduplication data set, a normalization data set and a calculation data set, wherein each data set comprises a pre-processing data subset and a post-processing data subset.
7. The method according to claim 2, characterized in that the method further comprises:
at least one of date, time and data modification content is noted for each of the communication signal pairs, and a log is generated.
8. A data processing apparatus, comprising:
the signal acquisition module is used for selecting a preset transmission channel for transmission processing according to the corresponding transmission rate of the initial communication signal, and taking data output by the preset transmission channel as a communication processing signal;
the type determining module is used for determining a comparison result of the initial communication signal and the communication processing signal and determining the processing type of the communication processing signal according to the comparison result;
and the data storage module is used for storing the related information of the initial communication signals, the related information of the communication processing signals and the comparison result belonging to the same processing type in a data information set corresponding to the processing type.
9. An electronic device, the electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the data processing method of any one of claims 1-7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions for causing a processor to implement the data processing method of any one of claims 1-7 when executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311384900.6A CN117371058A (en) | 2023-10-24 | 2023-10-24 | Data processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311384900.6A CN117371058A (en) | 2023-10-24 | 2023-10-24 | Data processing method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117371058A true CN117371058A (en) | 2024-01-09 |
Family
ID=89403638
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311384900.6A Pending CN117371058A (en) | 2023-10-24 | 2023-10-24 | Data processing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117371058A (en) |
-
2023
- 2023-10-24 CN CN202311384900.6A patent/CN117371058A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11294754B2 (en) | System and method for contextual event sequence analysis | |
| CN116049146B (en) | Database fault processing method, device, equipment and storage medium | |
| CN116225769B (en) | Method, device, equipment and medium for determining root cause of system fault | |
| CN114090647A (en) | Power communication equipment defect relevance analysis method and defect checking method | |
| CN110633318A (en) | Data extraction processing method, device, equipment and storage medium | |
| CN117034149A (en) | Fault processing strategy determining method and device, electronic equipment and storage medium | |
| CN115878421B (en) | Data center equipment level fault prediction method, system and medium | |
| CN117371058A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN115099175B (en) | Method and device for acquiring time sequence netlist, electronic equipment and storage medium | |
| CN115422275A (en) | Data processing method, device, equipment and storage medium | |
| CN117056133B (en) | Data backup method, device and medium based on distributed Internet of things architecture | |
| CN114611155B (en) | Data management node verification method, device, equipment and medium | |
| CN115858325B (en) | Project log adjusting method, device, equipment and storage medium | |
| CN116050351A (en) | Structure comparison method, device and equipment of logic expression and storage medium | |
| CN116149933B (en) | Abnormal log data determining method, device, equipment and storage medium | |
| CN117687816A (en) | Service reliability evaluation method, device, equipment and storage medium | |
| CN117389828A (en) | Power supply server management method, device, system, equipment and storage medium | |
| CN116578646A (en) | Time sequence data synchronization method, device, equipment and storage medium | |
| CN117395145A (en) | SDH transmission network data processing method, device, equipment and storage medium of CMDB | |
| CN115567624A (en) | Message processing method and device, electronic equipment and medium | |
| CN116909921A (en) | Data quality detection method, device, apparatus and storage medium | |
| CN115686517A (en) | Front-end page point burying method and device based on mobile development platform, electronic equipment and storage medium | |
| CN114416881A (en) | Real-time synchronization method, device, equipment and medium for multi-source data | |
| CN118152465A (en) | Data conversion method, device, equipment and storage medium | |
| CN116973668A (en) | Power grid line fault diagnosis method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |