WO2020168675A1 - 一种样本数据的处理方法、系统及装置 - Google Patents

一种样本数据的处理方法、系统及装置 Download PDF

Info

Publication number
WO2020168675A1
WO2020168675A1 PCT/CN2019/096621 CN2019096621W WO2020168675A1 WO 2020168675 A1 WO2020168675 A1 WO 2020168675A1 CN 2019096621 W CN2019096621 W CN 2019096621W WO 2020168675 A1 WO2020168675 A1 WO 2020168675A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
layer
matrix
vector
fault
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/096621
Other languages
English (en)
French (fr)
Inventor
匡立伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fiberhome Telecommunication Technologies Co Ltd
Original Assignee
Fiberhome Telecommunication Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fiberhome Telecommunication Technologies Co Ltd filed Critical Fiberhome Telecommunication Technologies Co Ltd
Publication of WO2020168675A1 publication Critical patent/WO2020168675A1/zh
Priority to PH12021551184A priority Critical patent/PH12021551184A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
    • H04L41/064—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis involving time analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
    • H04L41/065—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis involving logical or physical relationship, e.g. grouping and hierarchies
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network

Definitions

  • the present invention relates to the field of communication technology, in particular to a method, system and device for processing sample data.
  • the purpose of the embodiments of the present invention is to provide a sample data processing method, system and device, which convert the configuration data, alarm data and fault data of the optical network for operation and maintenance experts into sample data for machine learning, and improve the sample data
  • the storage efficiency of different vendors realizes the intercommunication and sharing of cross-domain sample data.
  • an embodiment of the present invention provides a method for processing sample data, which includes:
  • Construct a multi-layer high-dimensional space for at least one target network including the alarm data layer, fault data layer, and configuration data layer of each target network.
  • Each layer is a two-dimensional matrix with the same dimensions, which is used to The converted vector is stored in rows in the sequence of the time points.
  • the method further includes: extracting input vectors of the machine learning model from the alarm data layer and the fault data layer, and extracting the machine learning model from the configuration data layer The output vector.
  • the alarm data layer and the fault data layer are sequentially expanded and expanded by row vectors to obtain the input vector; and the configuration data layer is sequentially modulated. Expand and expand by row vector to obtain the output vector.
  • a subspace including target data is extracted from the multi-layer high-dimensional space, and the subspace includes the alarm data layer, the fault data layer, and the configuration data layer. At least one sub-matrix;
  • the subspace is obtained after the multi-layer high-dimensional space is segmented by a subspace segmentation operator
  • the alarm matrix, the fault matrix and the configuration matrix are respectively stored in the alarm data layer, the fault data layer and the configuration data layer of the multi-layer high-dimensional space, and the matrix elements that do not store values in each layer are filled with zero values.
  • the vector obtained by the conversion is the hadamard product of the basic vector and the dimensional expansion vector, wherein each element of the basic vector is an alarm data or fault at the time point.
  • the value of a field in the data or configuration data, and the element of the dimension expansion vector is the expansion or reduction multiple of the corresponding element of the basic vector.
  • an embodiment of the present invention provides a sample data processing system, which includes:
  • a conversion module which is used to obtain alarm data, fault data and configuration data of at least one target network at multiple time points from the database, and convert them into corresponding vectors respectively;
  • the space building module is used to construct a multi-layer high-dimensional space for at least one target network, including the alarm data layer, the fault data layer and the configuration data layer of each target network; wherein each layer is a two-dimensional space with the same dimensions.
  • the dimensional matrix is used to store the converted vector in rows according to the sequence of the time points.
  • system further includes:
  • the extraction module is used to extract the input vector of the machine learning model from the alarm data layer and the fault data layer, and extract the output vector of the machine learning model from the configuration data layer.
  • the extraction module includes:
  • a sub-matrix processing unit configured to extract a sub-space including target data from the multi-layer high-dimensional space, the sub-space including at least one sub-matrix of the alarm data layer, the fault data layer, and the configuration data layer;
  • the matrix operation unit is used to perform modular expansion and row vector expansion on the alarm data layer and the fault data layer in sequence to obtain the input vector; perform modular expansion and row vector expansion on the configuration data layer in sequence to obtain The output vector; and sequentially performing modular expansion and row vector expansion on the subspace to obtain the input vector and/or output vector.
  • the building module includes:
  • a matrix preprocessing unit which is used to store the converted vectors with the same data source in rows in a two-dimensional empty matrix according to the sequence of the time points to obtain the alarm matrix of each target network. , Fault matrix and configuration matrix;
  • the matrix construction unit is used to store the alarm matrix, the fault matrix and the configuration matrix in the alarm data layer, the fault data layer and the configuration data layer of the multi-layer high-dimensional space, respectively, and there is no matrix for storing values in each layer
  • the element is filled with zero values.
  • an embodiment of the present invention provides a network fault processing device based on a machine learning model, the device including a fault diagnosis system and the sample data processing system described in the second aspect;
  • the fault diagnosis system includes a model construction module and an input and output control module.
  • the model construction module is used to obtain input vectors and output vectors from the processing system to construct the machine learning model, wherein the input vector is obtained from the The alarm data layer and the fault data layer are extracted, and the output vector is extracted from the configuration data layer;
  • the input and output control module is used to obtain alarm data and fault data of the target network, convert them into corresponding vectors and input them into the model building module, and deliver the output vector of the model building module to the target network.
  • the embodiment of the present invention obtains alarm data, fault data, and configuration data of at least one target network at multiple points in time from a database, and converts them into corresponding vectors, thereby constructing a multiple for at least one target network.
  • the layered high-dimensional space provides sample data for the machine learning model for optical network fault handling, converts the configuration data, alarm data and fault data of the optical network for operation and maintenance experts into sample data for machine learning, and improves the sample data
  • the storage efficiency of different vendors realizes the intercommunication and sharing of cross-domain sample data.
  • Figure 1 is a schematic diagram of a cloud network architecture
  • FIG. 2 is a flowchart of a method for processing sample data according to an embodiment of the present invention
  • Figure 3 is a schematic diagram of obtaining data from a database and performing vectorization and matrixization
  • Fig. 4 is a flowchart of a method for processing sample data in another embodiment of the present invention.
  • Figure 5 is an example of a multi-layer high-dimensional space
  • Figure 6 is a schematic diagram of performing modular expansion and row vector expansion on a multi-layer high-dimensional space in a method for processing sample data in another embodiment of the present invention
  • FIG. 7 is a flowchart of modular expansion and row vector expansion of subspaces of a multi-layer high-dimensional space in a method for processing sample data in another embodiment of the present invention.
  • FIG. 8 is a schematic diagram of an implementation manner of performing subspace segmentation on a multi-layer high-dimensional space
  • FIG. 9 is a schematic diagram of another embodiment of performing subspace segmentation on a multi-layer high-dimensional space
  • Figure 10 is a schematic diagram of subspace expansion and row vector expansion
  • Figure 11 is a flow chart of module expansion and subspace segmentation of multi-layer high-dimensional space
  • FIG. 12 is a schematic diagram of a processing system for sample data according to an embodiment of the present invention.
  • Fig. 13 is a schematic diagram of a network fault processing device according to an embodiment of the present invention.
  • the embodiment of the present invention provides a method for processing sample data, provides sample data for a machine learning model used for optical network fault processing, and converts optical network configuration data, alarm data, and fault data for operation and maintenance experts into machine learning-oriented Sample data, and improve the storage efficiency of sample data, facilitate the extraction and comparison of sample data and other operations, which is conducive to efficient construction of machine learning models.
  • the optical network involved in the embodiment of the present invention may be a traditional optical transport network such as (Optical Transport Network, OTN), Packet Transport Network (PTN), and Packet Optical Transport Network (Packet Optical Transport Network, POTN), and also It can be a cloud network.
  • OTN Optical Transport Network
  • PTN Packet Transport Network
  • POTN Packet Optical Transport Network
  • Figure 1 is a schematic diagram of a cloudized network architecture.
  • the lower left part of Figure 1 is a cloudized network base station, including Active Antenna Unit (AAU), Centralized Unit (CU), and Distributed Unit (Distributed Unit, DU), where CU supports non-real-time wireless high-level protocols and some core network sink functions and edge application functions, and DU supports physical layer functions and real-time functions.
  • the lower part of Figure 1 is the cloudized network access ring, aggregation ring and core ring.
  • the alarm data, fault data and configuration data of the network devices in these ring networks are reported to the edge data in the upper part of Figure 1 through the network management platform or the controller platform.
  • the core network functions of the 5G core network are divided into user plane (UP) functions and control plane (CP) functions.
  • UP user plane
  • CP control plane
  • these data centers are responsible for the management, orchestration, and control of cloud-based networks.
  • UP user plane
  • CP control plane
  • these data centers are responsible for the management, orchestration, and control of cloud-based networks.
  • they deploy intelligent platforms for cloud-based networks, and build a cloud-based network operation and maintenance management knowledge base based on massive network data and powerful computing capabilities. , As the brain of the cloud network.
  • each target network may be an access ring, an aggregation ring or a core ring in FIG. 1, but it is not limited thereto.
  • each target network may also be an access network, an aggregation network, or a core network in a traditional optical network (OTN, PTN, and POTN).
  • OTN optical network
  • PTN personal area network
  • POTN public switched telephone network
  • the optical network device reports the alarm data and related fault data to the network management platform, and the network management platform submits it to the data center.
  • the alarms generated by optical network equipment include root-cause alarms and derivative alarms, and there is a correlation between root-cause alarms and derivative alarms.
  • the optical network equipment fails, alarm data and fault data are generated and reported at the same time, and the fault needs to be repaired through the issued configuration data.
  • the data center first cleans the data to remove redundant, low-quality data and obtain high-quality alarm data Sets, fault data sets, and configuration data sets are stored in the database.
  • Fig. 2 shows a flowchart of a method for processing sample data according to an embodiment of the present invention.
  • the method for processing sample data includes:
  • S110 obtains alarm data, fault data and configuration data of a target network at multiple time points from the database, and converts them into corresponding vectors.
  • S120 constructs a multi-layer high-dimensional space for the target network, where each layer is a two-dimensional matrix with the same dimensions, and is used to store the converted vector in rows according to the sequence of time points to obtain each target separately
  • the alarm data layer, fault data layer and configuration data layer of the network is a multi-layer high-dimensional space for the target network, where each layer is a two-dimensional matrix with the same dimensions, and is used to store the converted vector in rows according to the sequence of time points to obtain each target separately.
  • step S110 based on the correlation between the alarm data, fault data, and configuration data in the generation time, all the alarm data, fault data, and configuration data of the target network within a set time period can be obtained from the database, or press Obtain all alarm data, fault data and configuration data of the target network from the database periodically every day, week or month.
  • the set time period or cycle includes alarm data at multiple time points, fault data at multiple time points, and configuration data at multiple time points.
  • Alarm data, fault data, and configuration data are not only heterogeneous data, but these data include various types of fields, and different fields have different dimensions.
  • the vectorized representation methods for heterogeneous data of different dimensions include:
  • each piece of alarm data, fault data or configuration data is converted into a basic vector V b , and each element of the basic vector V b is the value of a field in each piece of alarm data, fault data or configuration data.
  • the sample data set acquired all alarms are constituted M a in a trap data, wherein the alerting data generated at a time point may be one or more pieces, each alarm data field has N a.
  • a piece of alarm data shown in Figure 3 includes eight fields, namely: the sequence number of the alarm data Seq.No., address Addr., line number Line, alarm type AlarmType, alarm start time BeginTime, alarm end Time EndTime, board type BoardType, and network element type NetType, where the alarm start time BeginTime and alarm end time EndTime are accurate to seconds, the address Addr. and alarm type AlarmType are character numbers, and the network element type NetType is an integer value.
  • the values of all fields of the alarm data shown in Fig. 3 are converted into real numbers, and thus expressed as elements of a vector.
  • the integer values of these fields are represented in the vector as element values.
  • the minimum value of all alarm start time BeginTime and alarm end time EndTime can be corresponding to the value 1, and the number of seconds between other times and the minimum time can be added to the value 1, and the corresponding values of the alarm start time BeginTime and the alarm end time EndTime can be obtained respectively .
  • the alarm start time BeginTime is 10 seconds longer than the minimum time
  • the alarm start time BeginTime corresponds to the value 11.
  • the two fields are arranged in lexicographic order, and then numbered from 1, and the string is converted to a value as an element of the vector .
  • the dimension conversion is performed on the basic vector V b , and the converted vector V is the hadamard product of the basic vector V b and the dimension expansion vector V s , namely
  • the element of the dimension expansion vector V s is the expansion or reduction multiple of the corresponding element of the basic vector V b . For example, if the bandwidth unit M is expanded to giga G, the element of the dimension expansion vector V s is 1024.
  • the basic vector can be multiplied by the corresponding elements of the dimension expansion vector to generate sample data suitable for the training requirements.
  • the configuration data and fault data in the lower left part of Fig. 3 are also converted into corresponding vectors.
  • the configuration data includes Num_CPUs: 4, which is the number of CPU cores.
  • the vector group in the lower part of Fig. 3 shows two vectors. The alarm data and configuration data are converted.
  • the above method can also be used to construct the data basis vector and the dimension expansion vector.
  • the number of key/value pairs in XML corresponds to the number of the vector Dimension
  • the value of the vector element corresponds to the Value value in the XML document.
  • step S110 three pairs of vector groups are constructed for a target network, which are the basic vector group of alarm data and the extended vector group of dimensions, the basic vector group of fault data and the extended vector group of dimensions, and the basic vector group and dimension of configuration data.
  • extended vector group to give alarm data comprises vectors obtained by the data conversion in a trap M a M a number alarm data vectors, each data vector having N a warning elements; fault vector data group includes data of M f article failover M f failures resulting data vectors, each data vector having N f fault elements; vector set of configuration data including configuration data obtained by conversion of configuration data vector M c M c of strips, each having the configuration data vector number N c element.
  • step S120 After the vectorized representation of heterogeneous data of different dimensions, step S120 performs matrix representation.
  • the number of rows of the two-dimensional matrix in the multi-layer high-dimensional space is the vector of the alarm data vector group, the fault data vector group and the configuration data vector group.
  • the maximum value of the total number i.e. the number of columns M a, M f and M c the maximum value in the multilayer two-dimensional matrix of high-dimensional space is the total number of vector element data vector set alarm, fault data and configuration data vectors of vectors The maximum value of N a , N f and N c .
  • the chronological order of the point M a number of data vectors alarm, the alarm data stored in the vector set one row multilayer high dimensional space, to give an alarm data layer.
  • the fault data layer and the configuration data layer can be obtained.
  • R(I 1 ,:,:) represents the matrix elements of the I 1st layer, and R(I 1 , I 2 ,:) represents all the elements of the I 1st layer I 2 row vector.
  • R(2,:,:) represents the matrix elements of the second layer, and R(2,3,:) represents all the elements of the third row vector of the second layer.
  • the values, as well as the positions of matrices, vectors, and elements are determined.
  • step S110 is step S110'.
  • step S110' it is also possible to store the converted vectors in rows in a two-dimensional empty matrix according to the sequence of time points to obtain the alarm matrix, fault matrix and configuration matrix of each target network respectively.
  • the vector obtained by the matrix storage conversion can save storage space with the help of the classic sparse matrix storage method.
  • the data in the matrix can be saved based on Compressed Row Storage to save storage space.
  • the vectors of all alarm data are converted in the same manner as in the foregoing step S110, and stored in a two-dimensional empty matrix in the form of row vectors to form an alarm matrix, such as the two-dimensional matrix in the lower right part of Figure 3.
  • an alarm matrix such as the two-dimensional matrix in the lower right part of Figure 3.
  • M a 7000 data in a trap
  • trap matrix is formed 7000 rows and 8 columns.
  • fault matrix and configuration matrix can be constructed.
  • step S120 Another implementation of step S120 is step S120', in which the maximum number of rows and the maximum number of columns of the alarm matrix, the fault matrix and the configuration matrix are obtained, and the maximum number of rows and the maximum number of columns are taken as each layer of the multi-layer high-dimensional model The number of rows and columns of the matrix.
  • the alarm matrix, the fault matrix and the configuration matrix are respectively stored in the alarm data layer, the fault data layer and the configuration data layer of the multi-layer high-dimensional space, and the matrix elements that do not store the values in each layer are filled with zero values.
  • the classic sparse matrix storage method can be used to save storage space.
  • multiple layers of high-dimensional space can be stored based on compressed row storage (Compressed Row Storage).
  • Multi-layer high-dimensional space is used to uniformly represent alarm data, fault data, and configuration data, which facilitates the subsequent construction of input and output variables of the deep learning neural network model.
  • the input and output of the deep learning neural network model adopt vector representation.
  • the multi-layer high-dimensional space constructed based on the embodiment of the present invention can conveniently and accurately convert the alarm data layer and the fault data layer into input vectors, and the configuration data layer into Output vectors to train the deep learning neural network and mine the correlation law between alarm data, fault data and configuration data.
  • the time points of the alarm data, fault data, and configuration data of the optical network are different.
  • the alarm data layer, the fault data layer, and the configuration data layer may all include the same time point. Layer, if there is a vector group at a certain point in time, it will be stored in the corresponding row vector of the layer. If there is no data at a certain point in time, the corresponding row vector of the layer will be filled with zero values.
  • the optical network includes multiple target networks, such as the access ring, aggregation ring, and core ring in Figure 1, where the source domain and target domain represent the access network, aggregation network, core network, and data center network, respectively.
  • the access network as the source domain and the convergence network as the target domain, or the core network as the source domain and the data center network as the target domain, without limitation.
  • a multi-layer high-dimensional space is constructed for the source domain and the target domain.
  • the sample data processing method includes:
  • S200 data collection and preprocessing It specifically includes:
  • the alarm data, fault data and configuration data of the S201 optical network are uploaded to the three types of data centers by the network management platform or the controller platform.
  • S210 successively adopts vectorization and matrix representation methods of heterogeneous data of different dimensions, and converts the alarm data, fault data and configuration data of the source field and target field into one-dimensional vectors, and then respectively expresses them into corresponding two-dimensional matrices. It specifically includes:
  • a two-dimensional alarm matrix, a fault matrix, and a configuration matrix are constructed according to the alarm data, fault data, and configuration data of the source field, and a two-dimensional alarm matrix, fault, and configuration data are constructed according to the alarm data, fault data, and configuration data of the target field.
  • the construction methods of the matrix and the configuration matrix, the one-dimensional vector and the two-dimensional matrix are similar to the foregoing embodiments, and will not be repeated here.
  • S220 constructs a multi-layer high-dimensional space to realize the unified representation of alarm data, fault data and configuration data in the source and target fields.
  • the number of rows and columns of the matrix obtained in step S212 may be different.
  • the alarm data, fault data, and configuration data of the source field and target field are expressed in a matrix
  • the number of rows and columns of the two-dimensional matrix obtained As shown in Table 1:
  • Table 1 Examples of the number of rows and columns of the two-dimensional matrix of the source and target fields
  • Matrix type Number of rows and columns of the alarm matrix Number of rows and columns of the fault matrix Configure the number of rows and columns of the matrix Source field 5000 ⁇ 12 7000 ⁇ 18 3000 ⁇ 32 Target field 3000 ⁇ 8 5000 ⁇ 12 2000 ⁇ 35
  • the maximum number of rows and the maximum number of columns of all alarm matrix, fault matrix, and configuration matrix use the maximum number of rows and the maximum number of columns as the number of rows and columns of each layer of the two-dimensional matrix in the multi-layer high-dimensional space.
  • the number of rows and columns of each layer of a two-dimensional matrix in a multi-layer high-dimensional space are 7000 and 35, respectively.
  • the number of rows 7000 means that the largest number of rows in the six matrices is the number of rows of the source field fault matrix
  • the number of columns 35 means that the largest number of columns in the six matrices is the number of columns of the target field configuration matrix.
  • a 6-layer high-dimensional space representation model is constructed based on the six matrices in Table 1 above, and 6 empty matrices with 7000 rows and 35 columns are generated, and these 6 The data in the matrix is copied to the newly generated empty matrix, and the matrix elements without stored data are filled with zero elements.
  • the multi-layer high-dimensional space constructed for the source field and the target field is shown in Figure 5.
  • the six-layer multi-layer high-dimensional space D R(K 1 , K 2 , K 3 ), the first to the third
  • the layers are the alarm data layer, fault data layer and configuration data layer of the source field, corresponding to the alarm matrix, fault matrix and configuration matrix of the source field respectively.
  • the fourth to sixth layers are the alarm data layer, fault data layer and
  • the configuration data layer corresponds to the alarm matrix, fault matrix and configuration matrix of the target field.
  • a multi-layer high-dimensional space for multiple target networks, such as an access network, a convergence network, a core network, and a data center network, which is not limited.
  • the vectorization and matrix representation methods for heterogeneous data of different dimensions can convert structured and semi-structured optical network data of different dimensions into vectors and matrices, because there are a large number of zero elements
  • multi-layer high-dimensional space is a sparse matrix.
  • the classic sparse matrix storage method can be used to save data to save storage space.
  • constructing a multi-layer high-dimensional space not only realizes the unified representation of sample data in the source domain and the target domain, but also realizes the intercommunication and sharing of cross-domain sample data from different vendors, removing information island barriers for subsequent machine learning.
  • the sample data processing method further includes: extracting the input vector of the machine learning model from the alarm data layer and the fault data layer, and extracting the output vector of the machine learning model from the configuration data layer .
  • the alarm data layer and the fault data layer are sequentially modulus expanded and row vector expanded to obtain the input vector; the configuration data layer is modulus expanded and row vector expanded to obtain the output vector.
  • the sample data processing method includes:
  • S310 performs modular expansion on the multi-layer high-dimensional space to obtain a modular expansion matrix.
  • S320 expands the modular expansion matrix according to the row vector to obtain the input vector and the output vector.
  • the top H-th layer of the multi-layer high-dimensional space is placed on the leftmost side, and the bottom
  • the first layer is placed on the far right to form the modulus expansion matrix in Figure 6, and then the modulus expansion matrix is expanded by row vector to get the input vector/output vector on the right of Figure 6, where the first row of the H-th layer matrix
  • the vector is ranked first, then the second row vector of the H-th layer matrix, and finally the last row vector of the first layer.
  • the first modulus expansion operator f si D s ⁇ V ysi is used to divide the source domain
  • Use the second modulus expansion operator f so : Ds ⁇ V yso to convert the configuration data layer in the multi-layer high-dimensional space D s R(I 1 , I 2 , I 3 ) into a vector V yso .
  • the input vector and/or output vector are extracted from the subspace of the multi-layer high-dimensional space, as shown in FIG. 7, the sample data processing method includes:
  • S410 extracts a subspace including target data from a multi-layer high-dimensional space, where the subspace includes at least one submatrix of an alarm data layer, a fault data layer, and a configuration data layer.
  • the sub-matrix can be a sub-matrix in one layer of the multi-layer high-dimensional space; the sub-matrix can also be two or more layers of the multi-layer high-dimensional space, where each layer of the sub-matrix is a layer of the multi-layer high-dimensional space A sub-matrix of.
  • S420 performs modular expansion on the subspace to obtain a modular expansion matrix, and expands the modular expansion matrix according to row vectors to obtain an input vector and/or an output vector.
  • FIG. 8 shows a schematic diagram of an implementation of obtaining a subspace from a multi-layer high-dimensional space, wherein the left part of FIG. 8 is the H-layer multi-layer high-dimensional space constructed in the foregoing embodiment (as shown in FIG. 5).
  • a sub-matrix is extracted from the first layer and the H-th layer of the multi-layer high-dimensional space.
  • a sub-matrix is extracted from the second layer (for clarity For the purpose of extracting the sub-matrix from the second layer, the dot-dash line is not shown in Fig. 8), and the sub-space in the dot-dash line box on the right of Fig. 8 is obtained.
  • the sub-space includes 3 sub-matrices, and at least One includes target data.
  • the second embodiment can be as shown by the dashed line in Fig. 8.
  • the first layer and the second layer are extracted from the multi-layer high-dimensional space, and one sub-matrix is extracted from the first layer and the second layer, and the dashed line on the right of Fig. 8
  • the subspace within the frame, the subspace includes two submatrices, and at least one of these submatrices includes target data.
  • one sub-matrix can also be extracted from the H-th layer to obtain another sub-space.
  • the above two implementation manners can extract a sub-matrix from any position in each layer of the multi-layer high-dimensional space, and these two implementation manners can be combined arbitrarily to obtain a subspace including target data.
  • the third embodiment may be as shown in FIG. 9, after extracting the sub-matrix from the same position of the designated layer in the multi-layer high-dimensional space, the sub-space is obtained.
  • the designated layer can be one layer, or two or more layers, without limitation. .
  • the subspace segmentation operation operator is defined as g:D ⁇ D′, which extracts the data in the multi-layer high-dimensional space and constructs the subspace D′.
  • the modular expansion matrix is expanded by the row vector through the first modular expansion operator to obtain the input vector; and/or the modular expansion matrix is expanded by the second modular expansion operator to obtain the output vector by the row vector.
  • Fig. 10 refers to the schematic diagram of subspace modular expansion and row vector expansion shown in Fig. 10.
  • R(1,1,3) represents the value of the element in the first row and third column of the first layer.
  • the value is 13, R(2,3,2) indicates that the value of the element in the third row and second column of the second layer is 21.
  • the input vector is obtained by expanding the modular expansion matrix by row vector through the first modular expansion operator, that is, the one-dimensional vector containing 18 elements in the lower right part of Fig. 10, where the first row vector of the upper left matrix of Fig. 10 is located in the input.
  • the leftmost side of the vector is then the first row vector of the upper right matrix of Figure 10, and finally the third row vector of the upper right matrix.
  • Figure 6 shows the modular expansion of the multi-layer high-dimensional space by the modular expansion operator to obtain the modular expansion matrix, and the row vector expansion of the modular expansion matrix to obtain the input vector and output vector.
  • Figure 10 shows the modular expansion of the subspace of the multi-layer high-dimensional space by the modular expansion operator to obtain the modular expansion matrix, and the row vector expansion of the modular expansion matrix to obtain the input vector and output vector.
  • a key operation of optical network fault self-healing migration learning is to formally describe the data in the source field and the data in the target field, and convert the optical network configuration, alarm, and fault data expressed by human operation and maintenance experts into machine learning-oriented samples
  • the sample data needs to be flexibly manipulated, including obtaining the intersection and difference of the sample data of the source field and the target field, and accurately extracting part of the sample data for testing and analysis.
  • the sample data processing method of the embodiment of the present invention can construct a more effective
  • the sample data model facilitates operations such as the extraction and comparison of sample data, and facilitates the efficient construction of a migration learning model.
  • Figure 11 shows a flow chart of subspace segmentation, modular expansion, and row vector expansion of a multi-layer high-dimensional space for transfer learning.
  • FIG. 11 The left side of Figure 11 is the multi-layer high-dimensional space D s in the source domain. Expand operator operated by a first mold and the second mold f si operator f so unfolding operations were constructed and the output vector source input vector field, the operator splitting the source field constructed g s D s by a first subspace subspace '.
  • the right side of Figure 11 is the multi-layer high-dimensional space of the target field.
  • the source field input vector V ysi and the target field input vector V yti in the upper part of Fig. 11 together form the migration learning input vector for the intelligent self-healing of optical network faults.
  • the source domain output vector V yso and the target domain output vector V yto jointly form the migration learning output vector for the intelligent self-healing of optical network faults.
  • the composite function f si og si D s ⁇ V ysi is used to construct the input vector of the source domain, where the operator o indicates that the g si operation is performed first, and then the f si operation is performed.
  • the composite function f so og so Ds ⁇ V yso is used to construct the output vector of the source domain.
  • the embodiment of the present invention uses the composite function f ti og ti :D t ⁇ V yti to construct the input vector of the target field, and uses the composite function f to og to :D y ⁇ V yto to construct the output of the target field vector.
  • an embodiment of the present invention also provides a sample data processing system, which is used to implement the methods of the foregoing embodiments.
  • the sample data processing system includes a conversion module 101 and a space construction module 102.
  • the conversion module 101 is configured to obtain alarm data, fault data, and configuration data of at least one target network at multiple time points from a database, and convert them into corresponding vectors respectively.
  • the space construction module 102 is used to construct a multi-layer high-dimensional space for at least one target network, including an alarm data layer, a fault data layer, and a configuration data layer of each target network; wherein each layer is a two-dimensional space with the same dimensions Matrix is used to store the converted vector in rows according to the sequence of time points.
  • the sample data processing system further includes an extraction module 103, which is used to extract the input vector of the machine learning model from the alarm data layer and the fault data layer, and extract the output vector of the machine learning model from the configuration data layer.
  • an extraction module 103 which is used to extract the input vector of the machine learning model from the alarm data layer and the fault data layer, and extract the output vector of the machine learning model from the configuration data layer.
  • the extraction module 103 includes a sub-matrix processing unit 1031 and a matrix operation unit 1032.
  • the sub-matrix processing unit 1031 is configured to extract a sub-space including target data from a multi-layer high-dimensional space, the sub-space including at least one sub-matrix of an alarm data layer, a fault data layer and a configuration data layer.
  • the matrix operation unit 1032 is used to perform modular expansion and row vector expansion on the alarm data layer and the fault data layer in sequence to obtain the input vector; perform modular expansion and row vector expansion on the configuration data layer in sequence to obtain the output vector; and Perform modular expansion and row vector expansion in sequence to obtain input vector and/or output vector.
  • the space construction module 102 includes a matrix preprocessing unit 1021 and a matrix construction unit 1022.
  • the matrix preprocessing unit 1021 is configured to store the converted vectors with the same data source in rows in a two-dimensional empty matrix according to the sequence of time points to obtain the alarm matrix, the fault matrix and the configuration matrix of each target network.
  • the matrix construction unit 1022 is used to store the alarm matrix, the fault matrix, and the configuration matrix in the alarm data layer, the fault data layer and the configuration data layer of the multi-layer high-dimensional space, respectively, and the matrix elements that do not store values in each layer are filled with zero values.
  • an embodiment of the present invention also provides a network fault processing device based on a machine learning model.
  • the network fault processing device includes a fault diagnosis system 200 and a sample data processing system 100 of the foregoing embodiment.
  • the fault diagnosis system 200 includes a model construction module 201 and an input and output control module 202.
  • the model construction module 201 is used to obtain input vectors and output vectors from the sample data processing system 100 to construct a machine learning model, where the input vector is derived from alarm data The output vector is extracted from the configuration data layer.
  • the input and output control module 202 is used to obtain alarm data and fault data of the target network, and convert them into corresponding vectors respectively, and then input them into the model construction module 201, and deliver the output vectors of the model construction module 201 to the target network.
  • the above embodiments it may be implemented in whole or in part by software, hardware, firmware or any combination thereof.
  • software it can be implemented in the form of a computer program product in whole or in part.
  • the computer program product includes one or more computer instructions.
  • the computer program instructions When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application are generated in whole or in part.
  • the computer can be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices.
  • Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • computer instructions can be transmitted from a website, computer, server, or data center through a cable (such as Coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) transmission to another website, computer, server or data center.
  • the computer-readable storage medium may be any available medium that can be read by a computer or a data storage device such as a server or data center integrated with one or more available media. Available media can be magnetic media (for example, floppy disks, hard drives, tapes), optical media (for example, Digital Video Disc (DVD)) or semiconductor media (for example, Solid State Disk (SSD)), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

本发明公开了一种样本数据的处理方法、系统及装置，涉及通信技术领域。样本数据的处理方法包括：从数据库中获取至少一个目标网络在多个时间点的告警数据、故障数据和配置数据，并分别转换为相应的向量；为至少一个目标网络构建一个多层高维空间，包括每个目标网络的告警数据层、故障数据层和配置数据层；其中，每一层均为具有相同维度的二维矩阵，用于根据时间点的先后顺序按行存储所述转换得到的向量。本发明将面向运维专家的光网络的配置数据、告警数据和故障数据转换为面向机器学习的样本数据，并提高样本数据的存储效率，实现异厂商的跨域样本数据的互通和共享。

Description

一种样本数据的处理方法、系统及装置 技术领域
本发明涉及通信技术领域，具体是涉及一种样本数据的处理方法、系统及装置。
背景技术
光网络设备的当前性能指标越限或者一些潜在性能正在劣化时，会产生一系列的告警数据并上报给网管平台。当光网络设备出现故障时，则会同时产生告警数据和故障数据并上报。目前，运维专家通过分析告警数据和故障数据，定位故障发生位置，制定故障修复策略，然后通过管理平台和控制平台下发相应的配置数据到故障发生位置进行修复，必要时触发保护倒换以保证光网络的正常运行。
随着光网络规模日益增大，光网络设备不断增多，光网络产生的告警数据和故障数据数量越来越多，网络故障的定位和修复日趋复杂和费力，传统的故障处理模式面监巨大挑战，难以满足实际需要。特别是随着通信业务的飞速发展，通信技术的不断演进和变革，传统紧耦合、刚性网络架构转型为松耦合、灵活的云化网络架构是大势所趋。云化网络底层由光网络设备实现数据转发，中上层通过控制平台、管理平台、编排平台实现资源和业务的管理控制，系统运营和维护过程更加复杂，需要实现网络数据融合表示，高效提取数据操作和运算，以解决云化网络出现故障后难以及时恢复的问题。
采用人工智能技术对网络故障进行分析和修复是应对这些挑战的有效方案。但是，一方面，现有的面向运维专家的告警数据、故障 数据和配置数据均包括各种类型的字段，例如告警开始时间、网元类型和中央处理器(Central Processing Unit，CPU)的内核数量等。这些异构数据不能直接用作面向机器学习的样本数据；另一方面，由于网络数据数量巨大以及各种类型网络具有差异等因素，导致无法根据机器学习的需求灵活地提取所需的样本数据。因此，迫切需要灵活、有效的样本数据处理方法，以满足机器学习模型测试和分析的需要。
发明内容
本发明实施例的目的在于提供一种样本数据的处理方法、系统及装置，将面向运维专家的光网络的配置数据、告警数据和故障数据转换为面向机器学习的样本数据，并提高样本数据的存储效率，实现异厂商的跨域样本数据的互通和共享。
第一方面，本发明实施例提供一种样本数据的处理方法，其包括：
从数据库中获取至少一个目标网络在多个时间点的告警数据、故障数据和配置数据，并分别转换为相应的向量；
为至少一个目标网络构建一个多层高维空间，包括每个目标网络的告警数据层、故障数据层和配置数据层；其中，每一层均为具有相同维度的二维矩阵，用于根据所述时间点的先后顺序按行存储所述转换得到的向量。
结合第一方面，在第一种可选的实现方式中，所述方法还包括：从所述告警数据层和故障数据层提取机器学习模型的输入向量，从所述配置数据层提取机器学习模型的输出向量。
在第一方面的第一种可选的实现方式中，对所述告警数据层和故障数据层依次进行模展开和按行向量展开，得到所述输入向量；对所述配置数据层依次进行模展开和按行向量展开，得到所述输出向量。
在第一方面的第二种可选的实现方式中，从所述多层高维空间中提取包括目标数据的子空间，子空间包括所述告警数据层、故障数据层和配置数据层中的至少一个子矩阵；
对所述子空间依次进行模展开和按行向量展开，得到所述输入向量和/或输出向量。
其中，一种可选的实现方式中，通过子空间切分算子对所述多层高维空间进行切分后得到所述子空间；
通过第一模展开操作算子对所述模展开矩阵按行向量展开得到所述输入向量；和/或，通过第二模展开操作算子对所述模展开矩阵按行向量展开得到所述输出向量。
结合第一方面第三种可选的实现方式，根据所述时间点的先后顺序，在二维的空矩阵中按行存储所述转换得到的向量，以分别得到每个所述目标网络的告警矩阵、故障矩阵和配置矩阵；
将所述告警矩阵、故障矩阵和配置矩阵分别存储在所述多层高维空间的告警数据层、故障数据层和配置数据层，每一层中没有存储数值的矩阵元素填充零值。
结合第一方面第四种可选的实现方式，所述转换得到的向量为基础向量与量纲扩展向量的hadamard积，其中，基础向量的每个元素为一个所述时间点的告警数据、故障数据或者配置数据中一个字段的数值，量纲扩展向量的元素为所述基础向量的相应元素的扩大或者缩小倍数。
第二方面，本发明实施例提供一种样本数据的处理系统，其包括：
转换模块，其用于从数据库中获取至少一个目标网络在多个时间点的告警数据、故障数据和配置数据，并分别转换为相应的向量；
空间构建模块，其用于为至少一个目标网络构建一个多层高维空间，包括每个目标网络的告警数据层、故障数据层和配置数据层；其中，每一层均为具有相同维度的二维矩阵，用于根据所述时间点的先后顺序按行存储所述转换得到的向量。
结合第二方面，在第一种可选的实现方式中，所述系统还包括：
提取模块，其用于从所述告警数据层和故障数据层提取机器学习模型的输入向量，从所述配置数据层提取机器学习模型的输出向量。
在第二方面的第一种可选的实现方式中，所述提取模块包括：
子矩阵处理单元，其用于从所述多层高维空间中提取出包括目标数据的子空间，子空间包括所述告警数据层、故障数据层和配置数据层的至少一个子矩阵；
矩阵运算单元，其用于对所述告警数据层和故障数据层依次进行模展开和按行向量展开，得到所述输入向量；对所述配置数据层依次进行模展开和按行向量展开，得到所述输出向量；以及对所述子空间依次进行模展开和按行向量展开，得到所述输入向量和/或输出向量。
结合第二方面，在第二种可选的实现方式中，所述构建模块包括：
矩阵预处理单元，其用于根据所述时间点的先后顺序，在二维的空矩阵中按行存储数据来源相同的所述转换得到的向量，以分别得到每个所述目标网络的告警矩阵、故障矩阵和配置矩阵；
矩阵构建单元，其用于将所述告警矩阵、故障矩阵和配置矩阵分别存储在所述多层高维空间的告警数据层、故障数据层和配置数据层，每一层中没有存储数值的矩阵元素填充零值。
第三方面，本发明实施例提供一种基于机器学习模型的网络故障处理装置，所述装置包括故障诊断系统以及第二方面所述的样本数据的处理系统；
所述故障诊断系统包括模型构建模块和输入输出控制模块，所述模型构建模块用于从所述处理系统获取输入向量和输出向量，以构建所述机器学习模型，其中，输入向量是从所述告警数据层和故障数据层提取的，输出向量是从所述配置数据层提取的；
所述输入输出控制模块用于获取目标网络的告警数据和故障数据，并分别转换为相应的向量后输入所述模型构建模块，将所述模型构建模块的输出向量下发给目标网络。
相对于现有技术，本发明实施例通过从数据库中获取至少一个目标网络在多个时间点的告警数据、故障数据和配置数据，并分别转换为相应的向量，为至少一个目标网络构建一个多层高维空间，为用于光网络故障处理的机器学习模型提供样本数据，将面向运维专家的光网络的配置数据、告警数据和故障数据转换为面向机器学习的样本数据，并提高样本数据的存储效率，实现异厂商的跨域样本数据的互通和共享。
附图说明
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是一种云化网络架构示意图；
图2是本发明实施例样本数据的处理方法流程图；
图3是从数据库获取数据并进行向量化和矩阵化的示意图；
图4本发明另一实施例样本数据的处理方法流程图；
图5是多层高维空间的一个示例；
图6是本发明再一实施例样本数据的处理方法中，对多层高维空 间进行模展开和按行向量展开的示意图；
图7是本发明再一实施例样本数据的处理方法中，对多层高维空间的子空间进行模展开和按行向量展开的流程图；
图8是对多层高维空间进行子空间切分的实施方式示意图；
图9是对多层高维空间进行子空间切分的另一实施方式示意图；
图10是子空间进行模展开和按行向量展开的示意图；
图11是多层高维空间的模展开和子空间切分流程图；
图12是本发明实施例样本数据的处理系统示意图；
图13是本发明实施例网络故障处理装置示意图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
本发明实施例提供一种样本数据的处理方法，为用于光网络故障处理的机器学习模型提供样本数据，将面向运维专家的光网络配置数据、告警数据和故障数据转换为面向机器学习的样本数据，并提高样本数据的存储效率，方便对样本数据的提取和比较等操作，有利于高效地构造机器学习模型。
本发明实施例涉及的光网络既可以是(Optical Transport Network，OTN)、分组传送网(Packet Transport Network，PTN)和分组光传送网络(Packet Optical Transport Network，POTN)等传统的光传送网，还可以是云化网络。
作为一个示例，图1是一种云化网络架构示意图，图1左下部分 是云化网络基站，包括有源天线单元(Active Antenna Unit，AAU)、集中单元(Centralized Unit，CU)和分布式单元(Distributed Unit，DU)，其中，CU支持非实时无线高层协议以及部分核心网下沉功能和边缘应用功能，DU支持物理层功能和实时功能。图1下部是云化网络接入环、汇聚环和核心环，这些环形网络中的网络设备的告警数据、故障数据和配置数据通过网络管理平台或者控制器平台分别上报至图1上部的边缘数据中心、区域数据中心和核心数据中心，基站和边缘应用的告警数据、故障数据和配置数据通过本地网上报至边缘数据中心。5G核心网络的核心网功能分为用户面(User Plane，UP)功能与控制面(Control Plane，CP)功能。这些数据中心一方面承担着云化网络的管理、编排和控制等功能，另一方面部署云化网络的智能化平台，基于海量网络数据和强大的计算能力，构建云化网络运维管理知识库，担任云化网络的大脑。
在本发明实施例中，每个目标网络可以是图1中的接入环、汇聚环或者核心环，但并不局限于此。在其他的实施例中，每个目标网络还可以是传统光网络(OTN、PTN和POTN)中的接入网、汇聚网或者核心网。
在本发明实施例中，光网络设备将告警数据和相关的故障数据上报网络管理平台，由网络管理平台提交至数据中心。光网络设备产生的告警包括根源告警和衍生告警，根源告警和衍生告警之间相关联。光网络设备出现故障时，同时产生告警数据和故障数据并上报，并需要通过下发的配置数据对故障进行修复。
因为海量的光网络告警数据、故障数据和配置数据中包含大量重复冗余、不完备和不一致的数据，数据中心首先对数据进行清洗，去除重复冗余、低质量数据，得到高质量的告警数据集、故障数据集和 配置数据集，并分别保存在数据库中。
图2所示为本发明实施例样本数据的处理方法流程图，样本数据的处理方法包括：
S110从数据库中获取一个目标网络在多个时间点的告警数据、故障数据和配置数据，并转换为相应的向量。
S120为该目标网络构建一个多层高维空间，其中，每一层均为具有相同维度的二维矩阵，用于根据时间点的先后顺序按行存储转换得到的向量，以分别得到每个目标网络的告警数据层、故障数据层和配置数据层。
在步骤S110中，基于告警数据、故障数据和配置数据在产生的时间上具有相关性，可以从数据库中获取设定的时间段内目标网络的所有告警数据、故障数据和配置数据，也可以按天、周或者月等周期性地从数据库中获取目标网络的所有告警数据、故障数据和配置数据。设定的时间段或者周期内包括多个时间点的告警数据，多个时间点的故障数据以及多个时间点的配置数据。
告警数据、故障数据和配置数据不仅是异构数据，而且这些数据包括各种类型的字段，而且不同的字段有不同的量纲。对不同量纲的异构数据的向量化表示方法包括：
首先，每条告警数据、故障数据或者配置数据都被转换为一个基础向量V b，基础向量V b的每个元素为每条告警数据、故障数据或者配置数据中一个字段的数值。
例如，获取的所有告警数据所构成的样本集有M a条告警数据，其中，在一个时间点上产生的告警数据可以是一条或者多条，每条告警数据有N a个字段。
作为一个示例，图3中所示的一条告警数据包括八个字段，分别 是：告警数据的序列号Seq.No.、地址Addr.、线路号Line、告警类型AlarmType、告警开始时间BeginTime、告警结束时间EndTime、板类型BoardType和网元类型NetType，其中，告警开始时间BeginTime和告警结束时间EndTime精确到秒，地址Addr.和告警类型AlarmType为字符号，网元类型NetType为整型值。
将图3所示告警数据的所有字段的值转换为实数，从而表示为向量的元素。在告警数据的向量化过程中，这些字段的整型值作为元素值表示在向量中。可以将所有告警开始时间BeginTime和告警结束时间EndTime中的最小值对应为数值1，其他时间与最小时间相差的秒数加到数值1上，分别得到告警开始时间BeginTime和告警结束时间EndTime的对应值。例如，告警开始时间BeginTime比最小时间多10秒，则告警开始时间BeginTime对应数值11，将这两个字段按字典序进行排列，然后从1进行编号，将字符串转换为数值后作为向量的元素。
其次，对基础向量V b进行量纲转换，转换得到的向量V为基础向量V b与量纲扩展向量V s的hadamard积，即
Figure PCTCN2019096621-appb-000001
量纲扩展向量V s的元素为基础向量V b的相应元素的扩大或者缩小倍数，例如将带宽单位兆M扩大为千兆G，则量纲扩展向量V s的元素为1024。
根据机器学习模型训练的要求，可以将基础向量与量纲扩展向量的对应元素相乘，生成适合训练要求的样本数据。同理，图3左下部分的配置数据和故障数据也转换为相应的向量，配置数据包括Num_CPUs：4，即CPU的内核数量，作为示例，图3下部的向量组显示了两个向量，分别由告警数据和配置数据转换得到。
对于光网络中保存在半结构化XML文档中的故障数据和配置数据，同样可以利用上述方法构建数据基础向量和量纲扩展向量，XML 中键值对(Key/Value)的个数对应向量的维度，向量元素的值对应XML文档中的Value值。
在步骤S110中，为一个目标网络构建三对向量组，分别为告警数据基础向量组与量纲扩展向量组、故障数据基础向量组与量纲扩展向量组、以及配置数据基础向量组与量纲扩展向量组，得到的告警数据向量组包括由M a条告警数据转换得到的M a个告警数据向量，每个告警数据向量具有N a个元素；故障数据向量组包括由M f条故障数据转换得到的M f个故障数据向量，每个故障数据向量具有N f个元素；配置数据向量组包括由M c条配置数据转换得到的M c个配置数据向量，每个配置数据向量具有N c个元素。
对不同量纲的异构数据的向量化表示后，步骤S120进行矩阵化表示，多层高维空间的二维矩阵的行数为告警数据向量组、故障数据向量组和配置数据向量组的向量总数的最大值，即M a、M f和M c中的最大值，多层高维空间的二维矩阵的列数为告警数据向量组、故障数据向量组和配置数据向量组的向量元素总数的最大值，即N a、N f和N c中的最大值。
根据M a个告警数据向量的时间点的先后顺序，将告警数据向量组按行存储在多层高维空间的一层中，得到告警数据层。同样地，可以得到故障数据层和配置数据层。
多层高维空间表示为D＝R(I 1，I 2，I 3)，R表示多层高维空间在实数域上取值，I 1表示层数，I 2表示该层矩阵的行数，I 3表示该层矩阵的列数。例如，R(I 1，:，:)表示第I 1层矩阵元素，R(I 1，I 2，:)表示第I 1层第I 2行向量所有元素。R(2，:，:)表示第2层矩阵元素，R(2，3，:)表示第2层第3行向量的所有元素。
在本发明实施例中，多维高层空间的层数、行数和列数是确定的， 也就是说，D＝R(I 1，I 2，I 3)中I 1，I 2和I 3的值，以及矩阵、向量和元素的位置都是确定的。在后续机器学习模型的构建和分析过程中，便于计算告警数据、故障数据和配置数据的交集和差集，以及进行关联分析处理等操作。
上述样本数据的处理方法中，步骤S110的另一种实施方式是步骤S110′。
在步骤S110′中，还可以根据时间点的先后顺序，在二维的空矩阵中按行存储转换得到的向量，以分别得到每个目标网络的告警矩阵、故障矩阵和配置矩阵。采用矩阵存储转换得到的向量，可以借助经典的稀疏矩阵存储方式节省存储空间。例如，可以基于压缩行存储(Compressed Row Storage)保存矩阵中的数据，以节省存储空间。
具体的，以前述步骤S110中同样的方式转换得到所有告警数据的向量，以行向量的方式存入一个二维的空矩阵中，形成告警矩阵，例如图3右下部分的二维矩阵，假如有M a＝7000条告警数据，则形成7000行8列的告警矩阵。同样的，可以构建出故障矩阵和配置矩阵。
步骤S120的另一种实施方式是步骤S120′，其中，求取告警矩阵、故障矩阵和配置矩阵的最大行数和最大列数，将最大行数和最大列数作为多层高维模型每层矩阵的行数和列数。
然后将告警矩阵、故障矩阵和配置矩阵分别存储在多层高维空间的告警数据层、故障数据层和配置数据层，每一层中没有存储数值的矩阵元素填充零值。
得到多层高维空间后，可以借助经典的稀疏矩阵存储方式节省存储空间。例如，可以基于压缩行存储(Compressed Row Storage)保存多层高维空间。
采用多层高维空间统一表示告警数据、故障数据和配置数据，便于后续构建深度学习神经网络模型的输入和输出变量。深度学习神经网络模型的输入和输出采用向量表示形式，基于本发明实施例构建的多层高维空间可以方便、准确地将告警数据层和故障数据层转换为输入向量，将配置数据层转换为输出向量，从而对深度学习神经网络进行训练，挖掘告警数据、故障数据和配置数据之间的关联规律。
通常情况下，光网络的告警数据、故障数据和配置数据的时间点不同，在其他实施方式中，告警数据层、故障数据层和配置数据层中还可以均包括相同的时间点，对于每一层，如果在某个时间点存在向量组，则存入该层的相应行向量，如果在某个时间点没有数据，则在该层的相应行向量填充零值。
如前所述，光网络包括多个目标网络，例如图1中的接入环、汇聚环和核心环，分别以源领域和目标领域表示接入网、汇聚网、核心网和数据中心网络。例如，可以将接入网定义为源领域，将汇聚网定义为目标领域，或者将核心网定义为源领域，将数据中心网络定义为目标领域，不作限定。
在另一实施例中，为源领域和目标领域构建一个多层高维空间，参见图4所示，样本数据的处理方法包括：
S200数据采集与预处理。其具体包括：
S201光网络的告警数据、故障数据和配置数据由网络管理平台或者控制器平台上传至三类数据中心。
S202因为海量的光网络的告警数据、故障数据和配置数据中包含大量重复冗余、不完备、不一致的数据，三类数据中心会首先对数据进行清洗，去除重复冗余低质量数据，求得高质量的告警、故障、配置数据集，并分别保存在源领域数据库和目标领域数据库中。
S210依次采用不同量纲异构数据的向量化和矩阵化表示方法，将源领域和目标领域的告警数据、故障数据和配置数据分别转换为一维向量，然后分别表示成对应的二维矩阵。其具体包括：
S211一维向量的构建过程。
S212二维矩阵的构建过程。
具体的，根据源领域的告警数据、故障数据和配置数据分别构建二维的告警矩阵、故障矩阵和配置矩阵，根据目标领域的告警数据、故障数据和配置数据分别构建二维的告警矩阵、故障矩阵和配置矩阵，一维向量和二维矩阵的构建方法与前述实施例相似，此处不再赘述。
S220构建多层高维空间，实现源领域和目标领域的告警数据、故障数据和配置数据的统一表示。步骤S212中得到的矩阵的行数和列数可能不同，作为一个示例，假如源领域和目标领域的告警数据、故障数据和配置数据在矩阵化表示后得到的二维矩阵的行数和列数如下表1：
表1 源领域和目标领域的二维矩阵的行数和列数示例
矩阵类型 告警矩阵的行列数 故障矩阵的行列数 配置矩阵的行列数
源领域 5000×12 7000×18 3000×32
目标领域 3000×8 5000×12 2000×35
求取所有告警矩阵、故障矩阵和配置矩阵的最大行数和最大列数，将最大行数和最大列数作为多层高维空间的每层二维矩阵的行数和列数。以表一为例，则多层高维空间的每层二维矩阵的行数和列数分别为7000和35。其中，行数7000是指六个矩阵中最大的行数是源领域故障矩阵的行数，列数35是指六个矩阵中最大的列数是目标领域配置矩阵的列数。
求得最大行数7000和最大列数35后，基于上述表一中的六个矩阵构建一个6层的高维空间表示模型，生成6个7000行和35列的空矩阵，并将这6个矩阵中的数据复制至新生成的空矩阵中，没有存储数据的矩阵元素用零元素填充。
具体的，为源领域和目标领域构建的多层高维空间如图5所示，六层的多层高维空间D＝R(K 1，K 2，K 3)，第一层至第三层为源领域的告警数据层、故障数据层和配置数据层，分别对应源领域的告警矩阵、故障矩阵和配置矩阵，第四层至第六层为目标领域的告警数据层、故障数据层和配置数据层，分别对应目标领域的告警矩阵、故障矩阵和配置矩阵。其中，源领域的三层高维空间还可以表示为D s＝R(I 1，I 2，I 3)，目标领域的三层高维空间还可以表示为D t＝R(J 1，J 2，J 3)。
采用以上实施例中的方法，还可以为多个目标网络构建一个多层高维空间，例如接入网、汇聚网、核心网和数据中心网络，不作限定。
通过本发明实施例，对不同量纲的异构数据的向量化和矩阵化表示方法，能够将不同量纲的结构化、半结构化的光网络数据转换为向量和矩阵，因为有大量零元素填充，多层高维空间是稀疏矩阵，在保存过程中，可以采用经典的稀疏矩阵存储方法进行数据保存，以便节省存储空间。同时，构建多层高维空间不但实现源领域和目的领域的样本数据的统一表示，而且能够实现异厂商的跨域样本数据的互通和共享，为后续机器学习扫除信息孤岛障碍。
在前述实施例的基础上，在再一实施例中，样本数据的处理方法还包括：从告警数据层和故障数据层提取机器学习模型的输入向量，从配置数据层提取机器学习模型的输出向量。
在一种实施方式中，对告警数据层和故障数据层依次进行模展开和按行向量展开得到输入向量；对配置数据层依次进行模展开和按行 向量展开得到输出向量。
参见图6所示，样本数据的处理方法包括：
S310对多层高维空间进行模展开得到模展开矩阵。
S320对模展开矩阵按行向量展开得到输入向量和输出向量。
对于前述实施例中的多层高维空间(可以是一个目标网络或者多个目标网络)，例如，图5所示的多层高维空间D＝R(K 1，K 2，K 3)，使用模展开操作算子从上到下对D＝R(K 1，K 2，K 3)的每一层依次展开，多层高维空间最上面的第H层放在最左边，最下面的第一层放在最右边，形成图6中的模展开矩阵，然后对模展开矩阵按行向量展开，得到图6右部的输入向量/输出向量，其中，第H层矩阵的第一个行向量排在第一位，然后是第H层矩阵的第二个行向量，最后是第一层的最后一个行向量。
作为一个示例，对于源领域的多层高维空间D s＝R(I 1，I 2，I 3)，使用第一模展开操作算子f si:D s→V ysi，将源领域的多层高维空间D s＝R(I 1，I 2，I 3)中的告警数据层和故障数据层的转换为输入向量V ysi。使用第二模展开操作算子f so:Ds→V yso将多层高维空间D s＝R(I 1，I 2，I 3)中的配置数据层转化为向量V yso。
在另一种实施方式中，从多层高维空间的子空间中提取输入向量和/或输出向量，参见图7所示，样本数据的处理方法包括：
S410从多层高维空间中提取包括目标数据的子空间，子空间包括告警数据层、故障数据层和配置数据层中的至少一个子矩阵。
子矩阵可以是多层高维空间的一层中的一个子矩阵；子矩阵也可以是多层高维空间的二层或以上，其中，子矩阵的每层为多层高维空间的一层的一个子矩阵。
S420对子空间进行模展开得到模展开矩阵，并对模展开矩阵按 行向量展开得到输入向量和/或输出向量。
图8所示从多层高维空间得到子空间的实施方式示意图，其中，图8左部是前述实施例所构建的H层多层高维空间(如图5所示)。
第一种实施方式可以如图8中点划线所示，在多层高维空间的第一层和第H层分别提取一个子矩阵，同样地，从第二层提取一个子矩阵(为了清楚起见，从第二层提取子矩阵的点划线未在图8中示出)，得到图8右部点划线框内的子空间，该子空间包括3个子矩阵，这些子矩阵中的至少一个包括目标数据。
第二种实施方式可以如图8中虚线所示，首先从多层高维空间提取第一层和第二层，从第一层和第二层各提取一个子矩阵，得到图8右部虚线框内的子空间，该子空间包括2个子矩阵，这些子矩阵中的至少一个包括目标数据。同样地，还可以从第H层提取一个子矩阵，得到另一个子空间。
以上两种实施方式可以从多层高维空间的每一层的任意位置提取子矩阵，并且这两种实施方式可以任意组合，以得到包括目标数据的子空间。
第三种实施方式可以如图9所示，从多层高维空间的指定层的相同位置提取子矩阵后，得到子空间，该指定层可以是一层，也可以是二层以上，不作限定。
子空间切分操作算子操作定义为g:D→D′，提取多层高维空间中的数据，构建子空间D′。
还是以图5所示的多层高维空间为例，源领域的多层高维空间D s＝R(I 1，I 2，I 3)，子空间切分操作算子定义为g s:D s→D s′，提取源领域的多层高维空间中的数据，构建子空间D s′＝R(I 1′，I 2′，I 3′)。目标领域的多层高维空间D t＝R(J 1，J 2，J 3)，子空间切分操作算子操 作定义为g t:D t→D t′，提取目标领域多层高维空间中的数据，构建子空间D t′＝R(J 1′，J 2′，J 3′)。
得到模展开矩阵后，通过第一模展开操作算子对模展开矩阵按行向量展开得到输入向量；和/或，通过第二模展开操作算子对模展开矩阵按行向量展开得到输出向量。
作为一个示例，参见图10所示的子空间模展开和按行向量展开的示意图，图10左部为从源领域的多层高维空间得到的子空间，该子空间可以表示为D s′＝R(I 1′＝2，I 2′＝3，I 3′＝3)，这是一个二层三行三列的表示模型，其中R(1，:，:)表示源领域的告警数据层的一个子矩阵，R(2，:，:)表示源领域的故障数据层的一个子矩阵，R(1，1，:)表示告警数据层的一个子矩阵的第一个行向量，R(1，1，:)＝(6，5，13)。同理，R(2，:，3)表示故障数据层的一个子矩阵的第三个列向量，R(2，:，3)＝(12，7B，6)。在图10中，通过指定每层、每行和每列的坐标，可以准确定位子空间中的每一个元素，例如R(1，1，3)表示第一层第一行第三列元素的值为13，R(2，3，2)表示第二层第3行第2列元素的值为21。
对子空间进行模展开，将上边的故障数据层的子矩阵放在左边，将下边的告警数据层的子矩阵放在右边，得到图10右上部的模展开矩阵。
通过第一模展开操作算子对模展开矩阵按行向量展开得到输入向量，即图10右下部的包含18个元素的一维向量，其中，图10上部左边矩阵的第一个行向量位于输入向量的最左边，然后是图10上部右边矩阵的第一个行向量，最后是上部右边矩阵的第三个行向量。
图6显示了通过模展开操作算子对多层高维空间进行模展开得到模展开矩阵，并对模展开矩阵按行向量展开得到输入向量和输出向 量。图10显示了通过模展开操作算子对多层高维空间的子空间进行模展开得到模展开矩阵，并对模展开矩阵按行向量展开得到输入向量和输出向量。
采用迁移学习等新技术，实现网络告警智能关联分析以及网络故障自动定位和自动愈合，已经成为当前运营商和厂商的研究重点。光网络故障自愈迁移学习的一个关键操作是对源领域的数据和目标领域的数据进行形式化描述，将面向人类运维专家表示的光网络配置、告警、故障数据转换为面向机器学习的样本数据，并需要灵活操作样本数据，包括求取源领域、目的领域的样本数据的交集和差集，准确提取部分样本数据进行测试和分析，本发明实施例样本数据的处理方法能够构建更有效的样本数据模型，方便对样本数据的提取和比较等操作，有利于高效地构造迁移学习模型。
作为一个示例，图11给出了一种用于迁移学习的多层高维空间的子空间切分、模展开和行向量展开流程图。
图11左边是源领域的多层高维空间D s。通过第一模展开操作算子f si和第二模展开操作算子f so分别构建源领域的输入向量和输出向量，通过第一子空间切分算子g s构建源领域的子空间D s′。
图11右边是目标领域的多层高维空间。通过第三模展开操作算子f ti和第四模展开操作算子f to分别构建目标领域迁移学习输入和输出向量，通过第二子空间切分算子g t构建目标领域的子空间D t′。
图11上部的源领域输入向量V ysi与目标领域输入向量V yti共同组成光网故障智能自愈的迁移学习输入向量。源领域输出向量V yso与目标领域输出向量V yto共同组成光网故障智能自愈的迁移学习输出向量。
在样本数据的处理过程中，还可以根据具体实际应用场景，先从 多层高维模型中提取包含作为目标数据的最关键属性数据的子空间，然后基于子空间构建输入向量和输出向量。采用复合函数f siog si:D s→V ysi构建源领域的输入向量，其中，运算符o表示先执行g si操作，然后执行f si操作。采用复合函数f soog so:Ds→V yso构建源领域的输出向量。对于光网故障目标领域数据，本发明实施例采用复合函数f tiog ti:D t→V yti构建目标领域的输入向量，采用复合函数f toog to:D y→V yto构建目标领域的输出向量。
参见图12所示，本发明实施例还提供一种样本数据的处理系统，用于实现上述各实施例的方法，样本数据的处理系统包括转换模块101和空间构建模块102。
转换模块101用于从数据库中获取至少一个目标网络在多个时间点的告警数据、故障数据和配置数据，并分别转换为相应的向量。
空间构建模块102用于为至少一个目标网络构建一个多层高维空间，包括每个目标网络的告警数据层、故障数据层和配置数据层；其中，每一层均为具有相同维度的二维矩阵，用于根据时间点的先后顺序按行存储转换得到的向量。
在其他实施例中，样本数据的处理系统还包括提取模块103，其用于从告警数据层和故障数据层提取机器学习模型的输入向量，从配置数据层提取机器学习模型的输出向量。
具体的，提取模块103包括子矩阵处理单元1031和矩阵运算单元1032。
子矩阵处理单元1031用于从多层高维空间中提取出包括目标数据的子空间，子空间包括告警数据层、故障数据层和配置数据层的至少一个子矩阵。
矩阵运算单元1032用于对告警数据层和故障数据层依次进行模 展开和按行向量展开，得到输入向量；对配置数据层依次进行模展开和按行向量展开，得到输出向量；以及对子空间依次进行模展开和按行向量展开，得到输入向量和/或输出向量。
具体的，空间构建模块102包括矩阵预处理单元1021和矩阵构建单元1022。
矩阵预处理单元1021用于根据时间点的先后顺序，在二维的空矩阵中按行存储数据来源相同的转换得到的向量，以分别得到每个目标网络的告警矩阵、故障矩阵和配置矩阵。
矩阵构建单元1022用于将告警矩阵、故障矩阵和配置矩阵分别存储在多层高维空间的告警数据层、故障数据层和配置数据层，每一层中没有存储数值的矩阵元素填充零值。
参见图13所示，本发明实施例还提供一种基于机器学习模型的网络故障处理装置，网络故障处理装置包括故障诊断系统200以及前述实施例样本数据的处理系统100。
故障诊断系统200包括模型构建模块201和输入输出控制模块202，模型构建模块201用于从样本数据的处理系统100获取输入向量和输出向量，以构建机器学习模型，其中，输入向量是从告警数据层和故障数据层提取的，输出向量是从配置数据层提取的。
输入输出控制模块202用于获取目标网络的告警数据和故障数据，并分别转换为相应的向量后输入模型构建模块201，将模型构建模块201的输出向量下发给目标网络。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行计算机程序指令时，全部或部分地产生按照本 申请实施例的流程或功能。计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(Digital Subscriber Line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够读取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，数字通用光盘(Digital Video Disc，DVD))或者半导体介质(例如，固态硬盘(Solid State Disk，SSD))等。
本发明不局限于上述实施方式，对于本技术领域的普通技术人员来说，在不脱离本发明原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也视为本发明的保护范围之内。本说明书中未作详细描述的内容属于本领域专业技术人员公知的现有技术。

Claims (12)

  1. 一种样本数据的处理方法，其特征在于，其包括：
    从数据库中获取至少一个目标网络在多个时间点的告警数据、故障数据和配置数据，并分别转换为相应的向量；
    为至少一个目标网络构建一个多层高维空间，包括每个目标网络的告警数据层、故障数据层和配置数据层；其中，每一层均为具有相同维度的二维矩阵，用于根据所述时间点的先后顺序按行存储所述转换得到的向量。
  2. 如权利要求1所述的样本数据的处理方法，其特征在于，所述方法还包括：从所述告警数据层和故障数据层提取机器学习模型的输入向量，从所述配置数据层提取机器学习模型的输出向量。
  3. 如权利要求2所述的样本数据的处理方法，其特征在于：
    对所述告警数据层和故障数据层依次进行模展开和按行向量展开，得到所述输入向量；对所述配置数据层依次进行模展开和按行向量展开，得到所述输出向量。
  4. 如权利要求2所述的样本数据的处理方法，其特征在于：
    从所述多层高维空间中提取包括目标数据的子空间，子空间包括所述告警数据层、故障数据层和配置数据层中的至少一个子矩阵；
    对所述子空间依次进行模展开和按行向量展开，得到所述输入向量和/或输出向量。
  5. 如权利要求4所述的样本数据的处理方法，其特征在于：
    通过子空间切分算子对所述多层高维空间进行切分后得到所述子空间；
    通过第一模展开操作算子对所述模展开矩阵按行向量展开得到所述输入向量；和/或，通过第二模展开操作算子对所述模展开矩阵 按行向量展开得到所述输出向量。
  6. 如权利要求1所述的样本数据的处理方法，其特征在于：
    根据所述时间点的先后顺序，在二维的空矩阵中按行存储所述转换得到的向量，以分别得到每个所述目标网络的告警矩阵、故障矩阵和配置矩阵；
    将所述告警矩阵、故障矩阵和配置矩阵分别存储在所述多层高维空间的告警数据层、故障数据层和配置数据层，每一层中没有存储数值的矩阵元素填充零值。
  7. 如权利要求1所述的样本数据的处理方法，其特征在于：
    所述转换得到的向量为基础向量与量纲扩展向量的hadamard积，其中，基础向量的每个元素为一个所述时间点的告警数据、故障数据或者配置数据中一个字段的数值，量纲扩展向量的元素为所述基础向量的相应元素的扩大或者缩小倍数。
  8. 一种样本数据的处理系统，其特征在于，其包括：
    转换模块，其用于从数据库中获取至少一个目标网络在多个时间点的告警数据、故障数据和配置数据，并分别转换为相应的向量；
    空间构建模块，其用于为至少一个目标网络构建一个多层高维空间，包括每个目标网络的告警数据层、故障数据层和配置数据层；其中，每一层均为具有相同维度的二维矩阵，用于根据所述时间点的先后顺序按行存储所述转换得到的向量。
  9. 如权利要求8所述的样本数据的处理系统，其特征在于，所述系统还包括：
    提取模块，其用于从所述告警数据层和故障数据层提取机器学习模型的输入向量，从所述配置数据层提取机器学习模型的输出向量。
  10. 如权利要求9所述的样本数据的处理系统，其特征在于，所 述提取模块包括：
    子矩阵处理单元，其用于从所述多层高维空间中提取出包括目标数据的子空间，子空间包括所述告警数据层、故障数据层和配置数据层的至少一个子矩阵；
    矩阵运算单元，其用于对所述告警数据层和故障数据层依次进行模展开和按行向量展开，得到所述输入向量；对所述配置数据层依次进行模展开和按行向量展开，得到所述输出向量；以及对所述子空间依次进行模展开和按行向量展开，得到所述输入向量和/或输出向量。
  11. 如权利要求8所述的样本数据的处理系统，其特征在于，所述构建模块包括：
    矩阵预处理单元，其用于根据所述时间点的先后顺序，在二维的空矩阵中按行存储数据来源相同的所述转换得到的向量，以分别得到每个所述目标网络的告警矩阵、故障矩阵和配置矩阵；
    矩阵构建单元，其用于将所述告警矩阵、故障矩阵和配置矩阵分别存储在所述多层高维空间的告警数据层、故障数据层和配置数据层，每一层中没有存储数值的矩阵元素填充零值。
  12. 一种基于机器学习模型的网络故障处理装置，其特征在于：所述装置包括故障诊断系统以及如权利要求7至11任一所述的样本数据的处理系统；
    所述故障诊断系统包括模型构建模块和输入输出控制模块，所述模型构建模块用于从所述处理系统获取输入向量和输出向量，以构建所述机器学习模型，其中，输入向量是从所述告警数据层和故障数据层提取的，输出向量是从所述配置数据层提取的；
    所述输入输出控制模块用于获取目标网络的告警数据和故障数据，并分别转换为相应的向量后输入所述模型构建模块，将所述模型 构建模块的输出向量下发给目标网络。
PCT/CN2019/096621 2019-02-21 2019-07-19 一种样本数据的处理方法、系统及装置 Ceased WO2020168675A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PH12021551184A PH12021551184A1 (en) 2019-02-21 2021-05-24 Sample data processing method, and system and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910128534.5A CN111600735B (zh) 2019-02-21 2019-02-21 一种样本数据的处理方法、系统及装置
CN201910128534.5 2019-02-21

Publications (1)

Publication Number Publication Date
WO2020168675A1 true WO2020168675A1 (zh) 2020-08-27

Family

ID=72143392

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/096621 Ceased WO2020168675A1 (zh) 2019-02-21 2019-07-19 一种样本数据的处理方法、系统及装置

Country Status (3)

Country Link
CN (1) CN111600735B (zh)
PH (1) PH12021551184A1 (zh)
WO (1) WO2020168675A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112152852A (zh) * 2020-09-23 2020-12-29 创新奇智(北京)科技有限公司 根因分析方法、装置、设备及计算机存储介质
CN114629776A (zh) * 2020-12-11 2022-06-14 中国联合网络通信集团有限公司 基于图模型的故障分析方法及装置
CN116915576A (zh) * 2023-03-28 2023-10-20 中国移动通信有限公司研究院 网元告警处理方法、装置、设备及可读存储介质

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117709394A (zh) * 2024-02-06 2024-03-15 华侨大学 车辆轨迹预测模型训练方法、多模型迁移预测方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050146426A1 (en) * 2003-09-30 2005-07-07 Goncalo Pereira Method and apparatus for identifying faults in a network that has generated a plurality of fault alarms
CN101335643A (zh) * 2008-08-06 2008-12-31 烽火通信科技股份有限公司 用于sdh设备告警相关性分析的方法及装置
CN105765461A (zh) * 2013-10-02 2016-07-13 Asml荷兰有限公司 用于获得与工业过程有关的诊断信息的方法和设备
CN106844161A (zh) * 2017-02-20 2017-06-13 重庆邮电大学 一种带状态流计算系统中的异常监控及预测方法和系统
CN107508632A (zh) * 2016-06-14 2017-12-22 中国移动通信集团广东有限公司 一种同路由光缆故障定位方法和装置

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104462846B (zh) * 2014-12-22 2017-11-10 山东鲁能软件技术有限公司 一种基于支持向量机的设备故障智能诊断方法
WO2016115280A1 (en) * 2015-01-14 2016-07-21 Virta Laboratories, Inc. Anomaly and malware detection using side channel analysis
CN105467975B (zh) * 2015-12-29 2018-08-03 山东鲁能软件技术有限公司 一种设备故障诊断方法
CN106295692B (zh) * 2016-08-05 2019-07-12 北京航空航天大学 基于降维与支持向量机的产品早期故障根原因识别方法
CN106302522B (zh) * 2016-09-20 2019-08-06 华侨大学 一种基于神经网络和大数据的网络安全态势分析方法和系统
CN108173670B (zh) * 2016-12-07 2020-06-02 华为技术有限公司 检测网络的方法和装置
CN108256173B (zh) * 2017-12-27 2021-06-22 南京航空航天大学 一种航空发动机动态过程的气路故障诊断方法及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050146426A1 (en) * 2003-09-30 2005-07-07 Goncalo Pereira Method and apparatus for identifying faults in a network that has generated a plurality of fault alarms
CN101335643A (zh) * 2008-08-06 2008-12-31 烽火通信科技股份有限公司 用于sdh设备告警相关性分析的方法及装置
CN105765461A (zh) * 2013-10-02 2016-07-13 Asml荷兰有限公司 用于获得与工业过程有关的诊断信息的方法和设备
CN107508632A (zh) * 2016-06-14 2017-12-22 中国移动通信集团广东有限公司 一种同路由光缆故障定位方法和装置
CN106844161A (zh) * 2017-02-20 2017-06-13 重庆邮电大学 一种带状态流计算系统中的异常监控及预测方法和系统

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112152852A (zh) * 2020-09-23 2020-12-29 创新奇智(北京)科技有限公司 根因分析方法、装置、设备及计算机存储介质
CN112152852B (zh) * 2020-09-23 2023-02-03 创新奇智(北京)科技有限公司 根因分析方法、装置、设备及计算机存储介质
CN114629776A (zh) * 2020-12-11 2022-06-14 中国联合网络通信集团有限公司 基于图模型的故障分析方法及装置
CN114629776B (zh) * 2020-12-11 2023-05-30 中国联合网络通信集团有限公司 基于图模型的故障分析方法及装置
CN116915576A (zh) * 2023-03-28 2023-10-20 中国移动通信有限公司研究院 网元告警处理方法、装置、设备及可读存储介质

Also Published As

Publication number Publication date
PH12021551184A1 (en) 2021-12-06
CN111600735B (zh) 2021-08-03
CN111600735A (zh) 2020-08-28

Similar Documents

Publication Publication Date Title
CN111600734B (zh) 一种网络故障处理模型的构建方法、故障处理方法及系统
WO2020168675A1 (zh) 一种样本数据的处理方法、系统及装置
CN110515931B (zh) 一种基于随机森林算法的电容型设备缺陷预测方法
WO2025002140A1 (zh) 根因定位的方法、装置、设备和存储介质
CN103473307B (zh) 跨媒体稀疏哈希索引方法
WO2023284088A1 (zh) 基于人工智能的电路设计方法与实现系统
CN119382125B (zh) 一种电网拓扑识别方法、系统、终端及存储介质
CN110210620A (zh) 一种用于深度神经网络的通道剪枝方法
CN113824575A (zh) 故障节点识别的方法、装置、计算设备及计算机存储介质
CN115941510A (zh) 一种大规模sdn网络流量预测方法及系统
CN107292751A (zh) 一种时序网络中节点重要性的挖掘方法和装置
CN118035502A (zh) 配电网数据图模型构建方法、装置、电子设备及存储介质
CN106780640A (zh) 一种大规模图数据表示方法
CN104951442A (zh) 一种确定结果向量的方法和装置
CN120146291A (zh) 一种基于时空图神经网络的光伏发电能力预测方法、系统及介质
CN117453937B (zh) 一种电网图数据模型自动生成方法及装置
TW202223759A (zh) 通過分層分析的類神經網路修剪方法與系統
CN109861846A (zh) 应用调用关系获取方法、系统及存储介质
CN114298276B (zh) 基于卷积神经网络的多智能体系统网络鲁棒性评估方法
CN120110940B (zh) 一种基于图神经网络的多层网络流量动态监控与分析方法
CN109063048A (zh) 一种基于知识库图匹配的数据清洗方法及装置
RU2780710C1 (ru) Способ, система и устройство для обработки выборочных данных
CN107122472A (zh) 大规模非结构化数据提取方法、其系统、分布式数据管理平台
CN111768214B (zh) 产品属性的预测方法、系统、设备和存储介质
CN109033603A (zh) 基于源流路径链的智能变电站二次系统仿真方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19915760

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19915760

Country of ref document: EP

Kind code of ref document: A1