CN114745291A - Abnormal data cloud method and device, electronic equipment and storage medium - Google Patents
Abnormal data cloud method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114745291A CN114745291A CN202011545051.4A CN202011545051A CN114745291A CN 114745291 A CN114745291 A CN 114745291A CN 202011545051 A CN202011545051 A CN 202011545051A CN 114745291 A CN114745291 A CN 114745291A
- Authority
- CN
- China
- Prior art keywords
- data
- abnormal
- requirement
- target
- acquisition request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 298
- 238000000034 method Methods 0.000 title claims abstract description 72
- 230000015654 memory Effects 0.000 claims description 37
- 238000013507 mapping Methods 0.000 claims description 11
- 230000002547 anomalous effect Effects 0.000 claims description 4
- 238000004891 communication Methods 0.000 claims description 2
- 238000000605 extraction Methods 0.000 claims description 2
- 239000000284 extract Substances 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 22
- 230000008569 process Effects 0.000 description 12
- 238000004590 computer program Methods 0.000 description 6
- 238000007667 floating Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/02—Capturing of monitoring data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Abstract
The invention relates to a method and a device for cloud application of abnormal data, electronic equipment and a storage medium, wherein the method comprises the following steps: after receiving an abnormal data acquisition request sent by a cloud network, an edge side device extracts a data requirement from the abnormal data acquisition request, then calls target data content corresponding to a data field in a data format from a stored data record related to an abnormal event, generates abnormal data according to the called target data content and the data format, and finally sends the generated abnormal data to the cloud network.
Description
Technical Field
The invention relates to the technical field of data transmission, in particular to a method and a device for enabling abnormal data to appear as cloud, electronic equipment and a storage medium.
Background
Generally, when data loss is caused by shutdown, failure and other reasons of a field data acquisition device or a data server, and events such as abnormal start-stop of a unit, abnormal operation and the like occur, corresponding abnormal data are generated, and in order to better analyze the abnormal data, all data in a period of time before and after the occurrence time of the abnormal data are generally required to be acquired, so that the acquisition frequency is high and the acquired data amount is large during each abnormal time. And the acquisition modes and the acquisition fields of the abnormal data acquired by different devices are different, so that the abnormal data cannot be analyzed in time after a large amount of abnormal data are uploaded to the cloud service.
Disclosure of Invention
In order to solve the technical problems or at least partially solve the technical problems, the invention provides an abnormal data clouding method, an abnormal data clouding device, an electronic device and a storage medium.
In a first aspect, the present invention provides an abnormal data clouding method, which is applied to an edge device side, and includes:
receiving an abnormal data acquisition request sent by a cloud network; the abnormal data acquisition request carries at least one data requirement;
extracting target data requirements from the abnormal data acquisition request; the target data requirement comprises a data format and at least one data field;
calling target data content corresponding to a data field in the target data requirement from a stored data record related to an abnormal event;
generating abnormal data from the called target data content according to the data format;
and sending the abnormal data to a cloud network.
In an optional embodiment of the present invention, the abnormal data obtaining request carries device identifiers, and each device identifier corresponds to a data requirement one by one;
the extracting the data requirement from the abnormal data acquisition request comprises:
searching a target equipment identifier corresponding to the identifier of the edge side equipment;
and extracting a data requirement corresponding to the target equipment identification as a target data requirement.
In an alternative embodiment of the present invention, the data requirement comprises a plurality of data fields;
the calling of the target data content corresponding to the data field in the target data requirement includes:
extracting all data fields in the target data requirement;
determining target data fields supported by the edge side equipment from all the data fields;
and calling the data content corresponding to the target data field as target data content.
In an optional embodiment of the present invention, sending the abnormal data to a cloud network includes:
sending the abnormal data to a sender of the abnormal data acquisition request;
alternatively, the first and second electrodes may be,
and sending the abnormal data to a kafka cluster in a cloud network where a sender of the abnormal data acquisition request is located.
In an alternative embodiment of the invention, the method further comprises:
acquiring all data fields supported by the edge side equipment;
judging whether all data fields supported by the edge side equipment are consistent with the data fields required by the data;
if all data fields supported by the edge side equipment are inconsistent with the data fields required by the data, determining the data fields different from the data requirements in all the data fields supported by the edge side equipment as extended data fields;
and adding the extended data field into the received data requirement, updating the data format, and generating abnormal data by using the updated data format.
In an alternative embodiment of the invention, the method further comprises:
judging whether the edge side equipment stores a local data requirement corresponding to the historical abnormal data acquisition request or not;
if the local data requirement is stored, generating abnormal data according to the local data requirement;
and if the local data requirement is not stored, extracting the data requirement from the abnormal data acquisition request sent by the cloud network.
In an alternative embodiment of the invention, the method further comprises:
if the local data requirement is stored, judging whether the local data requirement is consistent with a new data requirement or not;
and if the local data requirement is inconsistent with the new data requirement, updating the new data requirement to be the local data requirement.
In a second aspect, the present invention provides an abnormal data cloud method, applied to a cloud server, where the method includes:
calling a pre-stored target data requirement, wherein the target data requirement is a data requirement which has a mapping relation with the edge side equipment;
generating an abnormal data acquisition request according to the target data requirement;
sending an abnormal data acquisition request to edge side equipment so that the edge side equipment generates abnormal data according to a target data requirement in the abnormal data acquisition request by stored data records related to abnormal events;
and receiving abnormal data sent by the edge side equipment.
In an alternative embodiment of the invention, the method further comprises:
judging whether the data field in the received abnormal data exceeds the data field required by the target data;
if the data field does not belong to the target data requirement, determining the data field in the abnormal data as an extended data field;
adding the extended data field to the target data requirement.
In a third aspect, the present invention provides an abnormal data clouding apparatus, including:
the first receiving module is used for receiving an abnormal data acquisition request sent by a cloud network; the abnormal data acquisition request carries at least one data requirement;
the extraction module is used for extracting target data requirements from the abnormal data acquisition request; the target data requirement comprises a data format and at least one data field;
the first calling module is used for calling target data content corresponding to a data field in the target data requirement from a stored data record related to an abnormal event;
the first generation module is used for generating abnormal data from the called target data content according to the data format;
and the first sending module is used for sending the abnormal data to a cloud network.
In a fourth aspect, the present invention provides an apparatus for clouding abnormal data, the apparatus comprising:
the second calling module is used for calling a pre-stored target data requirement, wherein the target data requirement is a data requirement which has a mapping relation with the edge side equipment;
the second generation module is used for generating an abnormal data acquisition request according to the target data requirement;
a second sending module, configured to send the abnormal data obtaining request to the edge device, so that the edge device generates abnormal data from the stored data record related to the abnormal event according to the data requirement in the abnormal data obtaining request;
and the second receiving module is used for receiving the abnormal data sent by the edge side equipment.
In a fifth aspect, the present invention provides an electronic device, comprising: at least one processor, and at least one memory, bus connected with the processor; the processor and the memory complete mutual communication through the bus; the processor is configured to call program instructions in the memory to perform the above-mentioned method of cloud on exception data of any one of the first or second aspects.
In a sixth aspect, an embodiment of the present invention provides a storage medium, where the storage medium stores one or more programs, and the one or more programs are executable by one or more processors to implement the above-mentioned method for clouding exception data according to any one of the first aspect and the second aspect.
According to the technical scheme provided by the embodiment of the invention, after receiving an abnormal data acquisition request sent by a cloud network, edge side equipment extracts a data requirement from the abnormal data acquisition request, wherein the data requirement comprises a data format and at least one data field, then calls target data content corresponding to the data field in the data format from a stored data record related to an abnormal event, generates abnormal data according to the called target data content in the data format, and finally sends the generated abnormal data to the cloud network, because the edge side equipment generates the abnormal data according to the data requirement in the abnormal data acquisition request sent by the cloud network, the data format and the data field of the abnormal data received by the cloud network meet the requirement of the cloud network on analyzing the abnormal data, and the data of different acquisition modes and different acquisition fields received by the cloud network in the prior art are acquired, the abnormal data generated by the technical scheme of the embodiment obviously accelerates the analysis speed of the abnormal data, and effectively avoids the problem that the abnormal data cannot be analyzed in time due to the fact that the data format and the data field do not meet the analysis requirement.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive labor.
Fig. 1 is a schematic flow chart illustrating an implementation of an abnormal data cloud method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of an implementation flow of data extraction requirements in an abnormal data cloud method according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an implementation flow of retrieving target data content in an abnormal data cloud method according to an embodiment of the present invention;
fig. 4 is a schematic diagram of an implementation flow of a data field in an extended data requirement in an abnormal data clouding method according to an embodiment of the present invention;
fig. 5 is a schematic diagram of an implementation flow of generating abnormal data following a local data requirement in an abnormal data cloud method according to an embodiment of the present invention;
fig. 6 is a schematic flowchart of an implementation of a method for cloud sharing of abnormal data according to an embodiment of the present invention;
fig. 7 is a schematic diagram of an implementation flow of a data update requirement of a cloud server in an abnormal data clouding method according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of an abnormal data cloud apparatus according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of an abnormal data cloud apparatus according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
As shown in fig. 1, an implementation flow diagram of an abnormal data cloud implementation method provided in an embodiment of the present invention is shown, and taking execution of an edge side device as an example, the method may specifically include the following steps:
step S101, receiving an abnormal data acquisition request sent by a cloud network, wherein the abnormal data acquisition request carries at least one data requirement.
In order to make the abnormal data received by the cloud network better meet the analysis requirement, at least one data requirement is added to the abnormal data acquisition request, wherein the data requirement in the step is the requirement for the format of the data and the data field required to be contained in the data.
Step S102, extracting a target data requirement from the abnormal data acquisition request; the target data requirement contains a data format and at least one data field.
After receiving the abnormal data acquisition request, the edge side device first needs to extract the data requirement in the abnormal data acquisition request, and since there may be a plurality of edge side devices, and the abnormal data acquisition request sent by the cloud network may be simultaneously sent to the plurality of edge side devices, and the data content related to the abnormal event in each edge side device may be different data fields, different data requirements may be set for different edge side devices, specifically, the abnormal data acquisition request may also carry a device identifier, and each device identifier corresponds to the data requirement one to one.
In a specific example, taking 3 edge devices as an example, the abnormal data acquisition request may be { device identifier 1, data requirement 1; device identification 2, data requirement 2; device identification 3, data requirement 3 }.
Correspondingly, if there are a plurality of edge devices, the specific implementation of this step may be as shown in fig. 2, and an implementation flow diagram for extracting a data requirement in the abnormal data cloud method provided in the embodiment of the present invention specifically includes the following steps:
step S201, finding a target device identifier corresponding to the identifier of the edge device.
Because the abnormal data acquisition request includes a plurality of data requests, and each data request corresponds to an equipment identifier, in order to find a data request matched with the edge side equipment, in this step, an equipment identifier corresponding to the identifier of the edge side equipment is searched from the abnormal data acquisition request and is used as a target equipment identifier.
Generally, the identifier of the edge device may be a unique identifier that is different from other edge devices by a certain edge device, such as an address code of the edge device, and the device identifier in this embodiment may be the aforementioned unique identifier, or an identifier that a certain class of edge devices all have.
For the case that the device identifier is the unique identifier of the edge device, the step only needs to search the device identifier that is the same as the identifier of the edge device as the target device identifier.
For the case that the device identifier is an identifier that all edge devices of a certain class have, the identifier of the edge device needs to have an identifier of a device type to which the edge device belongs, for example, the identifier of the edge device includes three fields, i.e., a, b, and c, where the field b identifies the type of the edge device, and then the device identifier in this step may be the field b, and for this case, the step needs to use the device identifier having the same content as the field b in the identifier of the edge device as the target device identifier.
And S202, extracting a data requirement corresponding to the target equipment identification as a target data requirement.
After the target device identifier is determined, the data requirement corresponding to the target device identifier can be extracted from the abnormal data acquisition request as the target data requirement.
It should be noted that the target data requirement may include a data format and at least one data field, where the data format refers to a format to be followed when generating data, for example, which data types are included in the data, and data bits occupied by each data type; and the data field refers to what content needs to be contained in the data, such as pressure, temperature, etc.
Also taking the case that the aforementioned includes 3 edge devices as an example, if the target device identifier determined in step S201 is the device identifier 2, then in this step, the data request 2 corresponding to the device identifier 2 is extracted as the target data request.
Step S103, from the stored data record related to the abnormal event, the target data content corresponding to the data field in the target data request is called.
Since the target data request determined in the foregoing step includes at least one data field, in this step, the target data content corresponding to the data field in the target data request may be retrieved from the stored data record related to the abnormal event.
It should be noted that the abnormal event refers to an event defined by the edge side device, where the event is abnormal, and when the abnormal event occurs, the edge side device stores a data record related to the abnormal event, for example, data of some measurement points collected when the abnormal event occurs, or data of some measurement points collected before and after the abnormal event occurs.
Specifically, the data requirement may include a plurality of data fields, and then a specific execution of the target data content may be as shown in fig. 3, which is an implementation flow diagram of the target data content in the abnormal data cloud method provided in the embodiment of the present invention, and specifically includes the following steps:
and S301, extracting all data fields in the target data requirement.
Step S302, determining target data fields supported by the edge side device from all data fields contained in the data requirement.
Generally, in the target data request, a plurality of data fields are preset, and data corresponding to all the data fields may not exist in a data record related to an abnormal event in the edge device, that is, the edge device does not support the data record, and at this time, the edge device only needs to determine the data fields supported by the edge device included in the data request as the target data fields.
In a specific example, for example, the target data request includes a pressure data field and a temperature data field, and the data record related to the abnormal event in the edge side device only includes pressure data, in this case, the edge side device only supports the pressure data field and does not support the temperature data field, and then the pressure data field may be determined as the target data field.
And step S303, calling the data content corresponding to the target data field as the target data content.
After the target data field is determined, the data content corresponding to the target data field in the stored data record related to the abnormal event may be used as the target data content.
And step S104, generating abnormal data according to the called target data content and the data format.
It should be noted that, the data format has a certain relationship with the data fields, the data format prepares corresponding data bits for each data field, and since the data content in the data field has a corresponding data type, for example, the data type of the data field under pressure is floating point type, that is, float, the data bits to be occupied by the data field are 32 bits.
Of course, the data format may also arrange the front and back order, i.e. the high and low bits, for each data field, in a specific example, the step calls the target data contents of 3 data fields, such as the target data contents of the pressure data field, the target data contents of the temperature data field, and the target data contents of the humidity data field, which are all floating point type, so the data format may be { the pressure data field (32 bits), the temperature data field (32 bits), and the humidity data field (32 bits) }.
Therefore, the exception data generated in this step may be the target data content placed in the data bits of the corresponding data field.
And S105, sending abnormal data to the cloud network.
When the abnormal data is sent to the cloud network, the abnormal data may be directly sent to a sender of the abnormal data acquisition request, such as a cloud server, and of course, in order to relieve network pressure between the edge side device and the cloud server, the abnormal data may be sent to a kafka cluster of the cloud network where the sender of the abnormal data acquisition request is located, and when the abnormal data needs to be acquired by the cloud server, the abnormal data may be directly acquired from the kafka cluster. It should be noted that, the working mechanism of the kafka cluster may refer to the related art, and is not described herein again.
According to the technical scheme provided by the embodiment of the invention, after receiving an abnormal data acquisition request sent by a cloud network, an edge side device extracts a data requirement from the abnormal data acquisition request, wherein the data requirement comprises a data format and at least one data field, then calls target data content corresponding to the data field in the data format from a stored data record related to an abnormal event, then generates abnormal data according to the called target data content and the data format, and finally sends the generated abnormal data to the cloud network, because the edge side device generates the abnormal data according to the data requirement in the abnormal data acquisition request sent by the cloud network, the data format and the data field of the abnormal data received by the cloud network are in accordance with the requirement of the cloud network for analyzing the abnormal data, and for the data of different acquisition modes and different acquisition fields received by the cloud network in the prior art, the abnormal data generated by the technical scheme of the embodiment obviously accelerates the analysis speed of the abnormal data, and effectively avoids the problem that the abnormal data cannot be analyzed in time due to the fact that the data format and the data field do not meet the analysis requirement.
Because the abnormal data acquisition request sent by the cloud network is for most of the edge-side devices, the data fields in the data requirement may not cover the data records related to the abnormal event in the edge-side device, that is, all the data fields supported by the edge-side device are inconsistent with the data fields of the data requirement, for example, the data records related to the abnormal event in the edge-side device have stressed data fields, temperature data fields, and humidity data fields, but the data fields in the data requirement of the abnormal data acquisition request sent by the cloud network only have stressed data fields and temperature data fields, for this case, this embodiment also provides a scheme for extending the data fields in the data requirement.
Specifically, as shown in fig. 4, a schematic diagram of an implementation flow of a data field in an extended data requirement in an abnormal data cloud method provided in an embodiment of the present invention may specifically include the following steps:
step S401, acquiring all data fields supported by the edge side device.
The scheme shown in fig. 3 may be referred to for feature definition of all data fields supported by the edge device obtained in this step, and details are not described here.
It should be noted that all the data fields acquired in this step are data fields that can be supported by the edge side device, and the supported meanings may refer to the meanings defined in step S302, which is not described herein again.
Step S402, determining whether all data fields supported by the edge side device are consistent with the data fields of the data requirement.
It should be noted that, in this step, if all the data fields supported by the edge side device are completely the same as the data fields of the data requirement, it is determined that the data fields are the same; and if all the data fields supported by the edge side equipment have data fields different from the data fields required by the data, judging that the data fields are inconsistent.
In step 403, if all the data fields supported by the edge device are not consistent with the data fields of the data requirement, the data fields different from the data requirement in all the data fields supported by the edge device are determined as the extended data fields.
When the determination result is that all data fields supported by the edge side device are inconsistent with the data fields of the data requirement, that is, if there are data fields different from the data fields of the data requirement in all the data fields supported by the edge side device, in order to improve the comprehensiveness of the data record related to the abnormal event and covered by the abnormal data in the cloud network, the distinguished data fields may be added to the data requirement.
Step S404, adding the extended data field into the received data requirement, updating the data format, and generating abnormal data by using the updated data format.
Specifically, the process of adding the extended data field to the data requirement in this step may be divided into two processes, one is to add the extended data field to a set formed by original data fields of the data requirement, so that the updated data requirement includes not only the original data field but also the extended data field; secondly, the extended data field needs to be added into the data format, so that the data format has a storage location of the extended data field, and at this time, the data type of the extended data field needs to be determined first, and still taking a floating point type as an example, the extended data field needs to occupy 32 data bits, and the location of the extended data field can be, but is not limited to, behind the original data field, and at this time, the data format needs to add 32 data bits at the rear side, so as to store the data content corresponding to the extended data field.
Since the abnormal data is not repeatedly sent to the edge device instead of being once-through, a historical abnormal data acquisition request may be stored in the edge device, and in order to improve the sending efficiency of the abnormal data, the edge device may also follow the data requirement in the historical abnormal data acquisition request.
A specific procedure for continuing to use may be as shown in fig. 5, and is an implementation flow diagram of generating the abnormal data according to the requirement for continuing to use the local data in the abnormal data cloud method provided in the embodiment of the present invention, and specifically may include the following steps:
step S501, determining whether the edge device stores a local data request corresponding to the historical abnormal data acquisition request.
In the step, whether the edge side equipment stores the local data request corresponding to the historical abnormal data acquisition request or not is judged, and the local data request can be directly inquired from a memory of the edge side equipment; and if the abnormal data acquisition request is not inquired, judging that the historical abnormal data acquisition request is not stored.
Step S502, if the local data requirement is stored, generating abnormal data according to the local data requirement.
The method comprises the steps that after the edge side equipment receives an abnormal data acquisition request sent by a cloud network each time, the abnormal data acquisition request is stored in a memory of the edge side equipment according to received time, so that historical abnormal data acquisition requests received at different times may exist in the memory at the moment, and a local data requirement according to which abnormal data are generated in the step can be a data requirement in the historical abnormal data acquisition request which is received at the latest time.
It should be noted that the process of generating the abnormal data in this step is different from the process of generating the abnormal data only according to the data requirement, and therefore, reference may be made to the process of generating the abnormal data, which is not described herein again.
In addition, because the local data requirement is sent to the edge side device by the cloud network, if the requirement of the cloud network for the abnormal data changes, the local data requirement stored by the edge side device may no longer meet the current requirement, and in order to avoid the situation, when the local data requirement is judged to be stored, before the abnormal data is generated according to the local data requirement, whether the local data requirement is consistent with the new data requirement or not may be judged; and if the data request is inconsistent with the local data request, updating the new data request to the local data request so as to ensure that the local data request of the edge side equipment is in the latest state.
Step S503, if the local data requirement is not stored, extracting the data requirement from the abnormal data acquisition request sent by the cloud network.
It should be noted that, the process of generating the abnormal data in this step may refer to the foregoing process of generating the abnormal data, and is not described herein again.
As shown in fig. 6, an implementation flow diagram of the abnormal data cloud method provided in the embodiment of the present invention is applied to a cloud server, and specifically includes the following steps:
step S601, a pre-stored target data requirement is called, where the target data requirement is a data requirement having a mapping relationship with the edge side device.
It should be noted that, at one end of the cloud server, a default data requirement is set, where the default data requirement is for devices on all edges, and if the cloud server wants to obtain abnormal data of a certain edge device, it may be determined first whether the edge device is a new edge device, that is, an edge device that has not requested abnormal data before, where the determination is based on whether there is a data requirement having a mapping relationship with the edge device at one end of the cloud server, and if not, the default data requirement is determined as a target data requirement, and the target data requirement is mapped with the edge device (where the identifier of the edge device may be used to map the identifier of the edge device with the target data requirement); and if so, determining the data request having the mapping relation with the edge side equipment as the target data request.
Step S602, generating an abnormal data acquisition request according to the target data requirement.
Step S603, sending the abnormal data obtaining request to the edge device, so that the edge device generates the abnormal data according to the target data requirement in the abnormal data obtaining request, where the data record is stored and is related to the abnormal event.
For the description of the abnormal data obtaining request and the process of generating the abnormal data by the edge device according to the abnormal data obtaining request, reference may be made to the foregoing embodiments, which are not described herein again.
Step S604, receiving the abnormal data sent by the edge side device.
In this step, the received abnormal data can be distinguished according to different sending modes of the edge side equipment, and if the edge side equipment directly sends the abnormal data to a sender (namely a cloud server) of the abnormal data acquisition request, the cloud server can directly receive the abnormal data; if the edge side device sends the abnormal data to the kafka cluster in the cloud network where the sender of the abnormal data acquisition request is located, the cloud server can receive the abnormal data through the kafka cluster.
In view of the foregoing situation that all data fields supported by the edge-side device for which the scheme of fig. 4 is directed are not consistent with the data fields of the data requirement, the present embodiment also provides a scheme for updating the data requirement at the cloud server.
As shown in fig. 7, an implementation flow diagram of a data updating requirement of a cloud server in an abnormal data cloud method provided in an embodiment of the present invention may specifically include the following steps:
step S701, determining whether a data field in the received abnormal data exceeds a data field required by the target data.
And step S702, if the abnormal data exceeds the target data, determining the data fields which do not belong to the target data requirement in the abnormal data as the extended data fields.
It should be noted that the term "exceeding" refers to that the data field in the received abnormal data has a data field that is not included in the data field in the target data requirement, that is, a data field that is not included in the target data requirement, and at this time, the data field that is not included in the target data requirement in the abnormal data may be determined as the extended data field.
And step S703, adding the extended data field into the target data requirement.
For the specific added process, reference may be made to the relevant description of step S404, which is not described herein again.
Corresponding to the foregoing method embodiment, an embodiment of the present invention further provides an abnormal data cloud apparatus, as shown in fig. 8, which is a schematic structural diagram of the abnormal data cloud apparatus provided in the embodiment of the present invention, and the apparatus includes:
a first receiving module 801, configured to receive an abnormal data acquisition request sent by a cloud network; the abnormal data acquisition request carries at least one data requirement;
an extracting module 802, configured to extract a target data requirement from the abnormal data obtaining request; the target data requirement comprises a data format and at least one data field;
a first retrieving module 803, configured to retrieve, from the stored data record related to the abnormal event, target data content corresponding to the data field in the target data request;
a first generating module 804, configured to generate abnormal data from the called target data content according to a data format;
a first sending module 805, configured to send the exception data to a cloud network.
The abnormal data cloud apparatus includes a processor and a memory, the first receiving module 801, the extracting module 802, the first retrieving module 803, the first generating module 804, the first sending module 805, and the like are all stored in the memory as program modules, and the processor executes the program modules stored in the memory to implement corresponding functions.
Corresponding to the foregoing method embodiment, an embodiment of the present invention further provides an abnormal data cloud apparatus, as shown in fig. 9, which is a schematic structural diagram of the abnormal data cloud apparatus provided in the embodiment of the present invention, and the apparatus includes:
a second retrieving module 901, configured to retrieve a pre-stored target data requirement, where the target data requirement is a data requirement having a mapping relationship with the edge-side device;
a second generating module 902, configured to generate an abnormal data obtaining request according to the target data requirement;
a second sending module 903, configured to send the abnormal data obtaining request to the edge device, so that the edge device generates abnormal data from the stored data record related to the abnormal event according to the data requirement in the abnormal data obtaining request;
a second receiving module 904, configured to receive the abnormal data sent by the edge-side device.
The embodiment of the invention also provides a storage medium (computer readable storage medium). The storage medium herein stores one or more programs. Among others, the storage medium may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, a hard disk, or a solid state disk; the memory may also comprise a combination of memories of the kind described above.
When the one or more programs in the storage medium are executable by the one or more processors, the above-described method for clouding the abnormal data executed on the electronic device side is implemented.
The processor is used for executing the abnormal data cloud program stored in the memory to realize the following steps of the abnormal data cloud method executed on the electronic equipment side:
receiving an abnormal data acquisition request sent by a cloud network; the abnormal data acquisition request carries at least one data requirement;
extracting a target data requirement from the abnormal data acquisition request; the target data requirement comprises a data format and at least one data field;
calling target data content corresponding to a data field in a target data requirement from a stored data record related to an abnormal event;
generating abnormal data from the called target data content according to a data format;
and sending the abnormal data to the cloud network.
In an optional embodiment of the present invention, the abnormal data acquisition request carries an equipment identifier, and each equipment identifier corresponds to a data requirement one by one;
extracting data requirements from the anomalous data acquisition request, comprising:
searching a target equipment identifier corresponding to the identifier of the edge side equipment;
and extracting the data requirement corresponding to the target equipment identification as a target data requirement.
In an alternative embodiment of the invention, the data requirement comprises a plurality of data fields;
calling the target data content corresponding to the data field in the target data requirement, wherein the calling comprises the following steps:
extracting all data fields in the target data requirement;
determining target data fields supported by the edge side equipment from all the data fields;
and calling the data content corresponding to the target data field as the target data content.
In an optional embodiment of the present invention, sending the exception data to the cloud network includes:
sending the abnormal data to a sender of the abnormal data acquisition request;
alternatively, the first and second electrodes may be,
and sending the abnormal data to the kafka cluster in the cloud network where the sender of the abnormal data acquisition request is located.
In an alternative embodiment of the invention, the method further comprises:
acquiring all data fields supported by edge side equipment;
judging whether all data fields supported by the edge side equipment are consistent with the data fields required by the data;
if all data fields supported by the edge side equipment are inconsistent with the data fields required by the data, determining the data fields different from the data requirements in all the data fields supported by the edge side equipment as extended data fields;
and adding the extended data field into the received data requirement, updating the data format, and generating abnormal data by using the updated data format.
In an alternative embodiment of the invention, the method further comprises:
judging whether the edge side equipment stores a local data requirement corresponding to the historical abnormal data acquisition request or not;
if the local data requirement is stored, generating abnormal data according to the local data requirement;
and if the local data requirement is not stored, extracting the data requirement from the abnormal data acquisition request sent by the cloud network.
In an alternative embodiment of the invention, the method further comprises:
if the local data requirement is stored, judging whether the local data requirement is consistent with a new data requirement or not;
and if the local data requirement is inconsistent with the new data requirement, updating the new data requirement to be the local data requirement.
Alternatively, the first and second electrodes may be,
calling a pre-stored target data requirement, wherein the target data requirement is a data requirement having a mapping relation with the edge side equipment;
generating an abnormal data acquisition request according to the target data requirement;
sending the abnormal data acquisition request to the edge side equipment so that the edge side equipment generates abnormal data according to the data requirement in the abnormal data acquisition request by the stored data record related to the abnormal event;
and receiving abnormal data sent by the edge side equipment.
In an alternative embodiment of the invention, the method further comprises:
judging whether the data field in the received abnormal data exceeds the data field required by the target data;
if the data field does not belong to the target data requirement, determining the data field in the abnormal data as an extended data field;
an extended data field is added to the target data requirement.
The embodiment of the invention provides a processor, which is used for running a program, wherein the program executes the following steps during running: receiving an abnormal data acquisition request sent by a cloud network; the abnormal data acquisition request carries at least one data requirement; extracting a target data requirement from the abnormal data acquisition request; the target data requirement comprises a data format and at least one data field; calling target data content corresponding to a data field in a target data requirement from a stored data record related to an abnormal event; generating abnormal data from the called target data content according to a data format; and sending the abnormal data to the cloud network. Or, calling a pre-stored target data requirement, wherein the target data requirement is a data requirement having a mapping relation with the edge side equipment; generating an abnormal data acquisition request according to the target data requirement; sending the abnormal data acquisition request to the edge side equipment so that the edge side equipment generates abnormal data according to the data requirement in the abnormal data acquisition request by the stored data record related to the abnormal event; and receiving abnormal data sent by the edge side equipment.
Fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, where the electronic device 100 shown in fig. 10 includes: at least one processor 1001, and at least one memory 1002, bus 1003 connected to the processor 1001; the processor 1001 and the memory 1002 communicate with each other through the bus 1003; the processor is used for calling the program instructions in the memory so as to execute the abnormal data cloud method. The electronic device herein may be a server, a PC, a PAD, a mobile phone, etc.
The invention also provides a computer program product adapted to perform a program for initializing the following method steps when executed on a data processing device:
receiving an abnormal data acquisition request sent by a cloud network; the abnormal data acquisition request carries at least one data requirement;
extracting a target data requirement from the abnormal data acquisition request; the target data requirement comprises a data format and at least one data field;
calling target data content corresponding to a data field in a target data requirement from a stored data record related to an abnormal event;
generating abnormal data from the called target data content according to a data format;
and sending the abnormal data to the cloud network.
In an optional embodiment of the present invention, the abnormal data acquisition request carries an equipment identifier, and each equipment identifier corresponds to a data requirement one by one;
extracting data requirements from the anomalous data acquisition request, comprising:
searching a target equipment identifier corresponding to the identifier of the edge side equipment;
and extracting the data requirement corresponding to the target equipment identification as a target data requirement.
In an alternative embodiment of the invention, the data request comprises a plurality of data fields;
calling the target data content corresponding to the data field in the target data requirement, wherein the calling comprises the following steps:
extracting all data fields in the target data requirement;
determining target data fields supported by the edge side equipment from all the data fields;
and calling the data content corresponding to the target data field as the target data content.
In an optional embodiment of the present invention, sending the exception data to the cloud network includes:
sending the abnormal data to a sender of the abnormal data acquisition request;
alternatively, the first and second electrodes may be,
and sending the abnormal data to the kafka cluster in the cloud network where the sender of the abnormal data acquisition request is located.
In an alternative embodiment of the invention, the method further comprises:
acquiring all data fields supported by edge side equipment;
judging whether all data fields supported by the edge side equipment are consistent with the data fields required by the data;
if all data fields supported by the edge side equipment are inconsistent with the data fields required by the data, determining the data fields different from the data requirements in all the data fields supported by the edge side equipment as extended data fields;
and adding the extended data field into the received data requirement, updating the data format, and generating abnormal data by using the updated data format.
In an alternative embodiment of the invention, the method further comprises:
judging whether the edge side equipment stores a local data requirement corresponding to the historical abnormal data acquisition request or not;
if the local data requirement is stored, generating abnormal data according to the local data requirement;
and if the local data requirement is not stored, extracting the data requirement from the abnormal data acquisition request sent by the cloud network.
In an alternative embodiment of the invention, the method further comprises:
if the local data requirement is stored, judging whether the local data requirement is consistent with a new data requirement;
and if the local data requirement is inconsistent with the new data requirement, updating the new data requirement to be the local data requirement.
Alternatively, the first and second electrodes may be,
calling a pre-stored target data requirement, wherein the target data requirement is a data requirement which has a mapping relation with the edge side equipment;
generating an abnormal data acquisition request according to the target data requirement;
sending the abnormal data acquisition request to the edge side equipment so that the edge side equipment generates abnormal data according to the data requirement in the abnormal data acquisition request by the stored data record related to the abnormal event;
and receiving abnormal data sent by the edge side equipment.
In an alternative embodiment of the invention, the method further comprises:
judging whether the data field in the received abnormal data exceeds the data field required by the target data;
if the data field does not belong to the target data requirement, determining the data field in the abnormal data as an extended data field;
the extended data field is added to the target data requirement.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a device includes one or more processors (CPUs), memory, and a bus. The device may also include input/output interfaces, network interfaces, and the like.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip. The memory is an example of a computer-readable medium.
Computer-readable media, including both permanent and non-permanent, removable and non-removable media, may implement the information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The above description is only an example of the present invention and is not intended to limit the present invention. Various modifications and alterations to this invention will become apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.
Claims (10)
1. The abnormal data cloud method is applied to edge side equipment, and comprises the following steps:
receiving an abnormal data acquisition request sent by a cloud network; the abnormal data acquisition request carries at least one data requirement;
extracting target data requirements from the abnormal data acquisition request; the target data requirement comprises a data format and at least one data field;
calling target data content corresponding to a data field in the target data requirement from a stored data record related to an abnormal event;
generating abnormal data from the called target data content according to the data format;
and sending the abnormal data to a cloud network.
2. The method according to claim 1, wherein the abnormal data acquisition request carries device identifiers, and each device identifier corresponds to a data requirement one by one;
the extracting the data requirement from the abnormal data acquisition request comprises:
searching a target equipment identifier corresponding to the identifier of the edge side equipment;
and extracting a data requirement corresponding to the target equipment identification as a target data requirement.
3. The method of claim 1, wherein the data request comprises a plurality of data fields;
the calling of the target data content corresponding to the data field in the target data requirement includes:
extracting all data fields in the target data requirement;
determining target data fields supported by the edge side equipment from all the data fields;
and calling the data content corresponding to the target data field as target data content.
4. The method of claim 1, further comprising:
acquiring all data fields supported by the edge side equipment;
judging whether all data fields supported by the edge side equipment are consistent with the data fields required by the data;
if all data fields supported by the edge side equipment are inconsistent with the data fields required by the data, determining the data fields different from the data requirements in all the data fields supported by the edge side equipment as extended data fields;
and adding the extended data field into the received data requirement, updating the data format, and generating abnormal data by using the updated data format.
5. The abnormal data cloud method is applied to a cloud server, and comprises the following steps:
calling a pre-stored target data requirement, wherein the target data requirement is a data requirement which has a mapping relation with the edge side equipment;
generating an abnormal data acquisition request according to the target data requirement;
sending an abnormal data acquisition request to edge side equipment so that the edge side equipment generates abnormal data according to a target data requirement in the abnormal data acquisition request by stored data records related to abnormal events;
and receiving abnormal data sent by the edge side equipment.
6. The method of claim 5, further comprising:
judging whether the data field in the received abnormal data exceeds the data field required by the target data;
if the data field does not belong to the target data requirement, determining the data field in the abnormal data as an extended data field;
adding the extended data field to the target data requirement.
7. An anomalous data clouding apparatus, the apparatus comprising:
the first receiving module is used for receiving an abnormal data acquisition request sent by a cloud network; the abnormal data acquisition request carries at least one data requirement;
the extraction module is used for extracting target data requirements from the abnormal data acquisition request; the target data requirement comprises a data format and at least one data field;
the first calling module is used for calling target data content corresponding to a data field in the target data requirement from a stored data record related to an abnormal event;
the first generation module is used for generating abnormal data from the called target data content according to the data format;
and the first sending module is used for sending the abnormal data to a cloud network.
8. An anomalous data clouding apparatus, the apparatus comprising:
the second calling module is used for calling a pre-stored target data requirement, wherein the target data requirement is a data requirement which has a mapping relation with the edge side equipment;
the second generation module is used for generating an abnormal data acquisition request according to the target data requirement;
the second sending module is used for sending the abnormal data acquisition request to the edge side equipment so that the edge side equipment can generate the stored data record related to the abnormal event into abnormal data according to the data requirement in the abnormal data acquisition request;
and the second receiving module is used for receiving the abnormal data sent by the edge side equipment.
9. An electronic device, comprising: at least one processor, and at least one memory, bus connected with the processor; the processor and the memory complete mutual communication through the bus; the processor is used for calling the program instructions in the memory to execute the method of any one of claims 1-6.
10. A storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011545051.4A CN114745291A (en) | 2020-12-23 | 2020-12-23 | Abnormal data cloud method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011545051.4A CN114745291A (en) | 2020-12-23 | 2020-12-23 | Abnormal data cloud method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114745291A true CN114745291A (en) | 2022-07-12 |
Family
ID=82273851
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011545051.4A Pending CN114745291A (en) | 2020-12-23 | 2020-12-23 | Abnormal data cloud method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114745291A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109313687A (en) * | 2016-01-24 | 2019-02-05 | 赛义德·卡姆兰·哈桑 | Computer security based on artificial intelligence |
| CN109710817A (en) * | 2018-12-14 | 2019-05-03 | 深圳壹账通智能科技有限公司 | Electronic device information input method, device, computer equipment and storage medium |
| WO2019149019A1 (en) * | 2018-01-30 | 2019-08-08 | 深圳壹账通智能科技有限公司 | Data interaction method and apparatus, computer device, and storage medium |
| US20190324797A1 (en) * | 2018-04-18 | 2019-10-24 | Alibaba Group Holding Limited | Task processing method, apparatus, and system |
| CN110472114A (en) * | 2019-07-03 | 2019-11-19 | 平安科技（深圳）有限公司 | Abnormal data method for early warning, device, computer equipment and storage medium |
| CN111177178A (en) * | 2019-12-03 | 2020-05-19 | 腾讯科技（深圳）有限公司 | Data processing method and related equipment |
| CN111835562A (en) * | 2020-07-01 | 2020-10-27 | 北京爱笔科技有限公司 | Log uploading method, log query method and log query system |
| CN112003686A (en) * | 2020-07-24 | 2020-11-27 | 新华三大数据技术有限公司 | Message format negotiation method and device |
-
2020
- 2020-12-23 CN CN202011545051.4A patent/CN114745291A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109313687A (en) * | 2016-01-24 | 2019-02-05 | 赛义德·卡姆兰·哈桑 | Computer security based on artificial intelligence |
| WO2019149019A1 (en) * | 2018-01-30 | 2019-08-08 | 深圳壹账通智能科技有限公司 | Data interaction method and apparatus, computer device, and storage medium |
| US20190324797A1 (en) * | 2018-04-18 | 2019-10-24 | Alibaba Group Holding Limited | Task processing method, apparatus, and system |
| CN109710817A (en) * | 2018-12-14 | 2019-05-03 | 深圳壹账通智能科技有限公司 | Electronic device information input method, device, computer equipment and storage medium |
| CN110472114A (en) * | 2019-07-03 | 2019-11-19 | 平安科技（深圳）有限公司 | Abnormal data method for early warning, device, computer equipment and storage medium |
| CN111177178A (en) * | 2019-12-03 | 2020-05-19 | 腾讯科技（深圳）有限公司 | Data processing method and related equipment |
| CN111835562A (en) * | 2020-07-01 | 2020-10-27 | 北京爱笔科技有限公司 | Log uploading method, log query method and log query system |
| CN112003686A (en) * | 2020-07-24 | 2020-11-27 | 新华三大数据技术有限公司 | Message format negotiation method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106874328B (en) | Data processing method and equipment | |
| CN105760199B (en) | A kind of application resource loading method and its equipment | |
| CN107844274B (en) | Hardware resource management method, device and terminal based on super-fusion storage system | |
| CN109582470B (en) | Data processing method and data processing device | |
| CN111399764B (en) | Data storage method, data reading device, data storage equipment and data storage medium | |
| CN104113576A (en) | Method and device for updating client | |
| CN107040576B (en) | Information pushing method and device and communication system | |
| CN110781149A (en) | Method, device, equipment and storage medium for managing live broadcast room information | |
| CN110069406B (en) | Automatic triggering TPC-DS test method and system | |
| CN112363980A (en) | Data processing method and device for distributed system | |
| CN109271097A (en) | Data processing method, data processing equipment and server | |
| CN114745291A (en) | Abnormal data cloud method and device, electronic equipment and storage medium | |
| CN110688201B (en) | Log management method and related equipment | |
| CN110989929A (en) | MON service migration method, device, equipment and readable storage medium | |
| CN110908876B (en) | Method and device for acquiring hardware performance data | |
| CN112162951A (en) | Information retrieval method, server and storage medium | |
| CN113055243B (en) | DPI interface data processing method and device | |
| CN111158873A (en) | System state machine implementation method and device, storage medium and processor | |
| CN113051143A (en) | Detection method, device, equipment and storage medium for service load balancing server | |
| CN113326004B (en) | Efficient log centralization method and device in cloud computing environment | |
| CN108427625B (en) | Method, client, server and system for sending and recording logs | |
| CN113495932A (en) | General aerospace big data efficient caching method and device based on OGC standard service | |
| CN115017191A (en) | Data processing method, device, server and storage medium | |
| CN111694611B (en) | Data processing method and device and electronic equipment | |
| CN112579605A (en) | Data storage method and device, storage medium and server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |