CN112069242B

CN112069242B - Data processing method based on big data and cloud computing and big data service platform

Info

Publication number: CN112069242B
Application number: CN202010957749.0A
Authority: CN
Inventors: 鲍强; 杨伟深
Original assignee: Yingkou Guoshen Network Technology Co ltd
Current assignee: Liaoning minweixian IOT Technology Development Co.,Ltd.
Priority date: 2020-09-14
Filing date: 2020-09-14
Publication date: 2021-10-01
Anticipated expiration: 2040-09-14
Also published as: CN113312417A; CN112069242A; CN113312417B

Abstract

According to the data processing method and the big data service platform based on big data and cloud computing, firstly, target cloud service data are determined, then, a plurality of groups of device communication logs with different interaction frequencies are determined, communication protocol messages are detected from the device communication logs to obtain a communication protocol message set, the communication protocol message set is mapped into the target cloud service data according to a time sequence correlation coefficient to obtain a plurality of mapping message fields, then, access authority identification is carried out according to field correlation among the plurality of mapping message fields to obtain a cross authority authentication result, and finally, a data access authority list is extracted according to the cross authority authentication result. The data access permission list is used for indicating an access path of the first service equipment in the second service equipment, so that the second service equipment is prevented from intercepting an access request of the first service equipment by the second service equipment when the first service equipment accesses the second service equipment, and normal data interaction between the first service equipment and the second service equipment is further ensured.

Description

Data processing method based on big data and cloud computing and big data service platform

Technical Field

The disclosure relates to the technical field of big data processing, in particular to a data processing method and a big data service platform based on big data and cloud computing.

Background

With the development of science and technology, the application of cloud service communication is more and more extensive. For example, cloud business communication can be applied to the fields of smart cities, intelligent medical treatment, intelligent homes, intelligent transportation, industrial automation, big data analysis, big data mining, data platform management and the like, and the working efficiency and the working flexibility are greatly improved.

At present, cloud service communication with multi-terminal interaction becomes a mainstream, and timely and efficient information intercommunication can be realized through multi-terminal interaction.

However, as the scale of the multi-end interaction is continuously enlarged, the stability of the cloud service communication of the multi-end interaction also faces challenges, and one of the challenges is that the normal and stable data interaction between the service devices is difficult to be ensured.

Disclosure of Invention

In order to solve the technical problems in the related art, the present disclosure provides a data processing method and a big data service platform based on big data and cloud computing.

Based on the first aspect, a data processing method based on big data and cloud computing is provided, and includes:

acquiring cloud service data to be verified, and completing the cloud service data to be verified at the data generation time to obtain the completed target cloud service data;

determining a plurality of groups of equipment communication logs with different interaction frequencies based on the target cloud service data; respectively detecting the communication protocol messages in the multiple groups of equipment communication logs with different interaction frequencies to obtain communication protocol message sets in the multiple groups of equipment communication logs with different interaction frequencies; mapping communication protocol message sets in the multiple groups of equipment communication logs with different interaction frequencies into the target cloud service data according to the time sequence correlation coefficient between the multiple groups of equipment communication logs with different interaction frequencies and the target cloud service data to obtain multiple mapping message fields;

according to the field relevance among the mapping message fields, carrying out access authority identification on the mapping message fields to obtain a cross authority authentication result among the mapping message fields;

extracting a data access authority list corresponding to the target cloud service data according to a cross authority authentication result between the mapping message fields;

wherein:

when a first service device corresponding to the target cloud service data accesses a second service device, the data access permission list is used for indicating an access path of the first service device in the second service device.

Based on the second aspect, a big data service platform is provided, which comprises a data processing device, wherein the data processing device realizes the above method through at least a plurality of functional modules when running.

Based on a third aspect, a big data service platform is provided, comprising a processor, a memory and a bus; wherein: the processor and the memory communicate over the bus; the processor retrieves a computer program from the memory via the bus and runs the computer program to implement the method described above.

Based on a fourth aspect, a computer-readable storage medium is provided, having stored thereon a computer program which, when executed, implements the above-described method.

The data processing method and the big data service platform based on big data and cloud computing provided by the embodiment of the invention firstly generate target cloud service data according to the acquired cloud service data to be verified, secondly, determining a plurality of groups of device communication logs with different interaction frequencies, detecting communication protocol messages from the device communication logs to obtain a communication protocol message set, mapping the communication protocol message set to target cloud service data according to a time sequence correlation coefficient between the device communication logs and the target cloud service data to obtain a plurality of mapping message fields, and then, according to the field relevance among the mapping message fields, carrying out access authority identification on the mapping message fields to obtain a cross authority authentication result among the mapping message fields, and finally extracting a data access authority list corresponding to the target cloud service data according to the cross authority authentication result among the mapping message fields. In this way, when the first service device corresponding to the target cloud service data accesses the second service device, the data access permission list is used to indicate an access path of the first service device in the second service device, so that the false interception of the access request of the first service device by the second service device when the first service device accesses the second service device is avoided, and the normal data interaction between the first service device and the second service device is further ensured.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

FIG. 1 is an architectural schematic diagram of a big data and cloud computing based data processing system according to the present disclosure;

FIG. 2 is a flow diagram illustrating a data processing method based on big data and cloud computing, according to an example embodiment;

FIG. 3 is a block diagram illustrating a big data and cloud computing based data processing apparatus in accordance with an exemplary embodiment;

FIG. 4 is a diagram illustrating a hardware architecture of a big data services platform, according to another example embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

After finding the problems of the background art, the inventors have studied and analyzed the problems of the background art, and innovatively find that in cloud service communication, service devices have interactive data with different privacy levels, and correspondingly, access rights of the interactive data are different.

The inventor also finds that the service device configures an authority detection mechanism in advance to intercept the detected access request with abnormal access authority. Therefore, there is a problem that if the service device a does not obtain the access permission list of the service device B in advance when accessing the service device B, the service device B may perform a false interception on the access request of the service device a, thereby affecting normal data interaction between the service device a and the service device B.

In order to solve the above technical problems, embodiments of the present invention provide a data processing method and a big data service platform based on big data and cloud computing, which can ensure the authority security of a target service device when accessing other service devices, avoid the false interception of an authority detection mechanism of other service devices on an access request of the target service device, and ensure that the target service device can perform normal data interaction with other service devices.

To facilitate the overall description, please first refer to fig. 1, which shows a communication architecture diagram of a big data and edge computing based data processing system 100, wherein the data processing system 100 may include a big data service platform 200 and a plurality of business devices 400. Wherein the big data service platform 200 and the plurality of business devices 400 are communicatively connected to each other. On the basis, please refer to fig. 2 in combination, which shows a flowchart of a data processing method based on big data and edge calculation, where the data processing method may be applied to the big data service platform 200 in fig. 1, and specifically may include the contents described in the following steps 21 to 24.

Step 21, obtaining cloud service data to be verified, and completing the cloud service data to be verified at the data generation time to obtain the completed target cloud service data.

For example, in practice, in order to ensure the timeliness of acquiring the cloud service data to be verified, the data generation time of the cloud service data to be verified is not always acquired together. The service data to be verified in this embodiment may be service data of a first service device, and is used to determine access right information of a second service device corresponding to the first service device.

Step 22, determining a plurality of groups of device communication logs with different interaction frequencies based on the target cloud service data; respectively detecting the communication protocol messages in the multiple groups of equipment communication logs with different interaction frequencies to obtain communication protocol message sets in the multiple groups of equipment communication logs with different interaction frequencies; and mapping the communication protocol message sets in the multiple groups of equipment communication logs with different interaction frequencies into the target cloud service data according to the time sequence correlation coefficient between the multiple groups of equipment communication logs with different interaction frequencies and the target cloud service data to obtain multiple mapping message fields.

For example, the timing correlation coefficient is used to characterize the synchronicity or asynchronization of the device communication log with the target cloud traffic data in time.

And step 23, according to the field relevance among the mapping message fields, performing access authority identification on the mapping message fields to obtain a cross authority authentication result among the mapping message fields.

For example, the field relevance is used to represent whether the different mapping packet fields have interaction and overlap in service behavior, and the cross-authorization verification result is used to represent authorization verification behavior when service interaction is performed between different service devices.

And 24, extracting a data access authority list corresponding to the target cloud service data according to the cross authority authentication result between the mapping message fields.

For example, when a first service device corresponding to the target cloud service data accesses a second service device, the data access permission list is used to indicate an access path of the first service device in the second service device.

Therefore, the first service equipment can access the second service equipment through a correct access path under the indication of the data access authority list, and the access to a sensitive database of the second service equipment to trigger an authority detection mechanism of the second service equipment is avoided, so that the access interception of the second service equipment to the first service equipment is also avoided, and the normal data interaction between the first service equipment and the second service equipment is ensured. It should be understood that the second service device may be plural.

Based on the contents described in the above steps 21 to 24, firstly, target cloud service data is generated according to the obtained cloud service data to be verified, secondly, a plurality of groups of device communication logs with different interaction frequencies are determined, a communication protocol message set is detected from the device communication logs to obtain a communication protocol message set, the communication protocol message set is mapped into the target cloud service data according to a time sequence correlation coefficient between the device communication logs and the target cloud service data to obtain a plurality of mapping message fields, then, access authority identification is performed on the plurality of mapping message fields according to field correlation between the plurality of mapping message fields to obtain cross authority authentication results between the mapping message fields, and finally, a data access authority list corresponding to the target cloud service data is extracted according to the cross authority authentication results between the mapping message fields.

In this way, when the first service device corresponding to the target cloud service data accesses the second service device, the data access permission list is used to indicate an access path of the first service device in the second service device, so that the false interception of the access request of the first service device by the second service device when the first service device accesses the second service device is avoided, and the normal data interaction between the first service device and the second service device is further ensured.

In the specific implementation process, the inventor finds that the cross authority authentication result has more service interaction behaviors, and if the matching condition of the access authority is not analyzed when the cross authority authentication result is determined, the cross authority authentication result is difficult to be determined accurately and completely. To achieve the technical purpose, the performing, according to the field association between the mapping packet fields and described in step 23, access right identification on the mapping packet fields to obtain a cross right authentication result between the mapping packet fields may specifically include the following steps: determining an access authority matching list based on the protocol authentication confidence of the communication protocol message sets corresponding to the plurality of mapping message fields; and determining a cross authority authentication result between the mapping message fields according to the determined access authority matching list. Therefore, the matching condition of the access authority can be analyzed when the cross authority authentication result is determined, so that the cross authority authentication result can be accurately and completely determined.

Based on the above description of step 23, determining an access right matching list based on the protocol authentication confidence of the communication protocol message set corresponding to the plurality of mapping message fields; determining the cross-authority authentication result between the mapping message fields according to the determined access authority matching list further comprises the following contents of steps 231-237.

Step 231, if an absolute value of a difference between the protocol authentication confidence of any two communication protocol message sets in the plurality of mapping message fields and the protocol authentication confidence of one communication protocol message set in the any two communication protocol message sets exceeds a first set threshold, generating an authority comparison list of the any two communication protocol message sets, and taking the authority comparison list of the any two communication protocol message sets as an access authority matching list of the any two communication protocol message sets.

Step 232, if the message coding modes of any two communication protocol message sets in the plurality of mapping message fields are the same, and the absolute value of the difference between the protocol authentication confidence of any two communication protocol message sets and the protocol authentication confidence of one communication protocol message set in any two communication protocol message sets exceeds a second set threshold, taking the authority comparison list of any two communication protocol message sets as the access authority matching list of any two communication protocol message sets; wherein the first set threshold is greater than the second set threshold.

Step 233, counting all the determined access right matching lists, determining a right consistency comparison result of the right comparison lists of the two corresponding communication protocol message sets in each group of access right matching lists, and extracting a comparison similarity rate of the right events from the right consistency comparison result; the comparison similarity rate is used for representing the ratio of the number of the same authority events to the total number of the authority events in two corresponding communication protocol message sets, and each authority event corresponds to one service access behavior.

Step 234, extracting the list characteristics of each group of access right matching lists, converting the characteristic queues of the list structured information used for representing the access right matching lists in each group of list characteristics into characteristic queue description values according to a set conversion mode, and normalizing the characteristic queue description values to obtain normalized description values; and the value range of the normalization description value is 0-1.

Step 235, assigning a first ordering weight to each normalized description value based on the field association among the plurality of mapped message fields, and assigning a second ordering weight to each normalized description value based on the mapping path node parameters between the communication protocol message set and the plurality of mapped message fields; and sequencing the access authority matching list according to the descending order of the first sequencing weight to obtain a first sequencing queue, and sequencing the access authority matching list according to the descending order of the second sequencing weight to obtain a second sequencing queue.

Step 236, for the current access right matching list in all the access right matching lists, determining a first queue position of the current access right matching list in the first sorting queue and a second queue position of the current access right matching list in the second sorting queue, and calculating a relative position difference value between the first queue position and the second queue position of the current access right matching list; and judging whether the relative position difference is larger than a preset value, marking the current access permission matching list when the relative position difference is larger than the preset value, and taking the next access permission matching list of the current access permission matching list as the current access permission matching list and returning to the step of determining the first queue position of the current access permission matching list in the first sorting queue and the second queue position in the second sorting queue when the relative position difference is smaller than or equal to the preset value.

And 237, determining the cross authority authentication result between the mapping message fields according to the access authority matching list obtained by marking.

It can be understood that, by performing the above steps 231-237, the cross-authority authentication result can be accurately and completely determined.

In one possible embodiment, the determining of the cross-right authentication result between the mapping message fields according to the marked access-right matching list described in step 237 specifically includes the following contents described in steps 2371 to 2374.

Step 2371, after performing authority event classification on the access authority matching list obtained by the marking to obtain a first event set and a second event set, obtaining a first message signature key of the first event set and a second message signature key of the second event set, where the first event set includes a first authority event generation time list, and the second event set includes a second authority event generation time list.

Step 2372, obtaining each group of key units in the first message signing key and each group of key units in the second message signing key to obtain a public key and private key distribution queue; determining key authentication logic information between any two groups of key units in the public key and private key distribution queue to obtain a key authentication logic list; and adjusting the number of the logic nodes of the logic topology graph of the key authentication logic information in the key authentication logic list to be smaller than the set number, and updating two groups of key units corresponding to the key authentication logic information to obtain a target logic list corresponding to the key authentication logic list.

Step 2373, analyzing the logical intersection of the target logic lists corresponding to the key authentication logic list to obtain an analysis result, where the analysis result is used to indicate whether graph data characteristics of a logic topology graph of the key authentication logic information corresponding to the intersection time between the first authority event generation time list and the second authority event generation time list are similar.

Step 2374, when the analysis result indicates that the graph data features of the logical topology graph of the key authentication logic information corresponding to the crossing time between the first authority event generation time list and the second authority event generation time list are similar, obtaining a crossing authority authentication result between the mapping message fields according to the authority events in the access authority matching list corresponding to the key authentication logic information corresponding to the crossing time; and the mapping time of the mapping message field is associated with the crossing time.

When the contents described in the steps 2371 to 2374 are applied, the access right matching list can be parsed from the message signing key layer, so that the reliability of the cross-right authentication result is ensured.

In a specific implementation, in order to accurately determine the data access permission list, the step 24 may extract the data access permission list corresponding to the target cloud service data according to the cross permission authentication result between the mapping packet fields, which specifically includes the following contents:

acquiring cross authentication path information in a result distribution track of cross authority authentication results among the mapping message fields, extracting path parameters of the cross authentication path information through a logic node corresponding to the maximum heat value in the current logic topology of the cross authentication logic information corresponding to the cross authority authentication results, and determining a first path characteristic sample matched with the cross authentication path information;

determining a second path characteristic sample matched with the cross authentication path information through the logic node corresponding to the minimum heat value in the current logic topology in the cross authentication logic information corresponding to the cross authority authentication result based on the first path characteristic sample;

extracting service interaction characteristics in the target cloud service data through the data characteristic extraction node of the cross authentication logic information corresponding to the cross authority authentication result based on a second path characteristic sample matched with the cross authentication path information to obtain a service interaction track corresponding to the target cloud service data;

and determining a data access authority list corresponding to the target cloud service data according to the equipment identity information of the service equipment carried in the service interaction track.

Therefore, the data access authority list can be accurately determined according to the equipment identity information of the service equipment carried in the service interaction track.

Further, the step of extracting path parameters from the cross authentication path information through the logical node corresponding to the maximum heat value in the current logical topology of the cross authentication logical information corresponding to the cross authority authentication result, and determining the first path feature sample matching the cross authentication path information may include the following steps 2411 to 2415 in detail.

Step 2411, extracting nodes according to the first data characteristics in the cross authentication logic information corresponding to the cross authority authentication result, and determining classifier parameters corresponding to the cross authentication path information.

Step 2412, determining simulation thread data of the executive function simulator corresponding to the classifier parameters based on the logic executive function in the logic node corresponding to the maximum heat value.

Step 2413, responding to the simulation thread data of the executive function simulator, performing multidimensional feature clustering on the resource configuration information of any configuration parameter in the classifier parameters through the executive parameter queue of the logic executive function in the logic node corresponding to the maximum heat value, and determining a first clustering information set.

And 2414, according to the feature sample screening script in the logic node corresponding to the maximum heat value, sequentially carrying out feature sample screening and multi-dimensional feature clustering on the first clustering information set, and determining a second clustering information set.

Step 2415, importing a second classification information set corresponding to the cross authentication logic information into a preset authority authentication thread, and outputting a first path characteristic sample matched with the cross authentication path information by adjusting thread parameters of the preset authority authentication thread.

Still further, the step of determining, based on the first path feature sample, a second path feature sample matching the cross authentication path information through the logical node corresponding to the minimum heat value in the current logical topology in the cross authentication logical information corresponding to the cross authority authentication result may further include the following steps 2421 to 2425.

Step 2421, constructing a sample distribution map data set corresponding to the first path feature sample, and constructing a logic authentication map data set corresponding to a logic node corresponding to the minimum heat value in the current logic topology in the cross authentication logic information corresponding to the cross authority authentication result; wherein the sample distribution map data set and the logic authentication graph data set respectively comprise a plurality of graph data nodes with different graph data association degrees.

Step 2422, extracting graph data connection information of the first path feature sample in any graph data node of the sample distribution graph data set, and determining the graph data node with the smallest graph data association degree in the logic authentication graph data set as a target graph data node.

Step 2423, mapping the graph data link information to the target graph data nodes according to the cross authentication sequence of the cross authentication logic information corresponding to the cross authority authentication result, obtaining graph data mapping information in the target graph data nodes, and generating a sample update list between the first path feature sample and the logic node corresponding to the minimum heat value in the current logic topology in the cross authentication logic information corresponding to the cross authority authentication result according to the graph data link information and the graph data mapping information.

Step 2424, obtaining information to be updated in the target graph data node by taking the graph data mapping information as reference information, mapping the information to be updated to the graph data node where the graph data link information is located according to the updating time table corresponding to the sample updating list, obtaining updating result information corresponding to the information to be updated in the graph data node where the graph data link information is located, and determining the updating result information as the information to be matched.

Step 2425, obtaining the graph data connection information and mapping the graph data connection information to a node function container in the target graph data node; according to the cosine distance between the update result information and the sample information corresponding to the plurality of sample functions in the node function container, traversing the matching path parameters corresponding to the information to be matched in the logic authentication graph data set until the node centrality of the graph data node where the obtained matching path parameter is located is consistent with the node centrality of the information to be matched in the sample distribution graph data set, stopping obtaining the matching path parameters in the next graph data node, and adding the information to be matched to the first path feature sample according to the matching path parameters to obtain the second path feature sample.

In this way, it can be ensured that the samples in the second path feature sample have a higher feature recognition degree based on the above steps 2421 to 2425.

In an alternative embodiment, the completing of the cloud service data to be verified at the data generation time described in step S21 to complete the cloud service data to be verified into cloud service data carrying timing information, so as to obtain completed target cloud service data, which may specifically include the contents described in steps S211 to S214 below.

Step S211, locating a service node identifier from the cloud service data to be verified.

Step S212, determining whether the current service node identifier in the cloud service data to be verified is updated relative to the last service node identifier in the cloud service data to be verified.

Step S213, if yes, determining the current service node identifier positioned from the cloud service data to be verified as a service node to be supplemented of the cloud service data to be verified; otherwise, carrying out weighted sum on the node identification of the current service node identification positioned in the cloud service data to be verified and the node identification of the last service node identification of the cloud service data to be verified, and determining the service node to be completed of the cloud service data to be verified according to the weighted sum result.

Step S214, for each service node to be complemented, performing data generation time complementation on each service node to be complemented according to the data log corresponding to the cloud service data to be verified, so as to complement the cloud service data to be verified into the cloud service data carrying the time sequence information, and obtain the complemented target cloud service data.

Therefore, accurate and efficient data generation time completion of the cloud service data to be verified can be guaranteed.

In an alternative embodiment, the determining, based on the target cloud traffic data, the device communication logs of multiple groups of different interaction frequencies in step S22 includes: extracting a plurality of groups of device communication logs with different interaction frequencies corresponding to communication link identifications from a preset database according to the communication link identifications carried in the target cloud service data; the interaction frequency is used for representing the interaction times among different service devices in a set time period.

In an alternative embodiment, the step S22 of detecting the communication protocol messages in the multiple sets of device communication logs with different interaction frequencies respectively to obtain the sets of communication protocol messages in the multiple sets of device communication logs with different interaction frequencies includes: and extracting a target message with a protocol layer identifier from the equipment communication log, determining the target message as a communication protocol message and obtaining a communication protocol message set in the equipment communication log with the plurality of groups of different interaction frequencies when detecting that a message field carrying a communication protocol authentication signature in the target message is detected.

In an alternative implementation manner, the mapping, according to the time-sequence correlation coefficient between the multiple sets of device communication logs with different interaction frequencies and the target cloud service data, the communication protocol packet sets in the multiple sets of device communication logs with different interaction frequencies to the target cloud service data to obtain multiple mapping packet fields, which is described in step S22, includes: determining a log text having a service connection with the target cloud service data from each group of device communication logs, determining a time sequence correlation coefficient between the group of device communication logs and the target cloud service data according to a time sequence characteristic of a data stream carried in the log text, and sequentially mapping communication protocol message sets in the multiple groups of device communication logs with different interaction frequencies to the target cloud service data according to a descending order of the time sequence correlation coefficient to obtain multiple mapping message fields.

Based on the same inventive concept, there is also provided a data processing apparatus 300 based on big data and cloud computing as shown in fig. 3, including:

the data completion module 310 is configured to acquire cloud service data to be verified, perform data generation time completion on the cloud service data to be verified, complete the cloud service data to be verified into cloud service data carrying timing sequence information, and obtain completed target cloud service data;

the message mapping module 320 is configured to determine, based on the target cloud service data, multiple groups of device communication logs with different interaction frequencies; respectively detecting the communication protocol messages in the multiple groups of equipment communication logs with different interaction frequencies to obtain communication protocol message sets in the multiple groups of equipment communication logs with different interaction frequencies; mapping communication protocol message sets in the multiple groups of equipment communication logs with different interaction frequencies into the target cloud service data according to the time sequence correlation coefficient between the multiple groups of equipment communication logs with different interaction frequencies and the target cloud service data to obtain multiple mapping message fields;

the authority authentication module 330 is configured to perform access authority identification on the multiple mapping message fields according to field relevance among the multiple mapping message fields, so as to obtain an intersection authority authentication result among the mapping message fields;

and the list extraction module 340 is configured to extract a data access permission list corresponding to the target cloud service data according to the cross permission authentication result between the mapping packet fields.

On the basis of the above, the data processing device 300 is integrated in a big data service platform.

On the basis, please refer to fig. 4 in combination, a schematic diagram of a hardware structure of a big data service platform 200 is also provided, where the big data service platform 200 includes a processor 210, a memory 220, and a bus 230; wherein: the processor 210 and the memory 220 communicate via the bus 230; the processor 210 retrieves a computer program from the memory 220 via the bus 230 and runs the computer program to implement the method shown in fig. 2.

It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Based on the above technical solutions, the embodiments of the present invention further provide the following alternative implementations.

Alternative embodiment one:

A1. a data processing method based on big data and cloud computing comprises the following steps:

acquiring cloud service data to be verified, and completing the cloud service data to be verified at the data generation time to obtain the completed target cloud service data; the method specifically comprises the following steps: positioning a service node identifier from the cloud service data to be verified; judging whether the current service node identification in the cloud service data to be verified is updated relative to the last service node identification in the cloud service data to be verified; if so, determining the current service node identifier positioned from the cloud service data to be verified as a service node to be supplemented of the cloud service data to be verified; otherwise, carrying out weighted sum on the node identification degree of the current service node identification positioned in the cloud service data to be verified and the node identification degree of the last service node identification of the cloud service data to be verified, and determining a service node to be completed of the cloud service data to be verified according to the weighted sum result; for each service node to be complemented, performing data generation time complementation on each service node to be complemented according to a data log corresponding to the cloud service data to be verified, complementing the cloud service data to be verified into cloud service data carrying time sequence information, and obtaining complemented target cloud service data;

wherein:

A2. The method according to a1, wherein identifying access rights to the mapping message fields according to the field associations between the mapping message fields to obtain cross-rights authentication results between the mapping message fields includes:

determining an access authority matching list based on the protocol authentication confidence of the communication protocol message sets corresponding to the plurality of mapping message fields;

and determining a cross authority authentication result between the mapping message fields according to the determined access authority matching list.

A3. The method according to a2, wherein determining an access right matching list based on the protocol authentication confidence of the communication protocol message set corresponding to the plurality of mapping message fields, and determining the cross-right authentication result between the mapping message fields according to the determined access right matching list includes:

if the absolute value of the difference between the protocol authentication confidence degrees of any two communication protocol message sets in the plurality of mapping message fields and the protocol authentication confidence degree of one communication protocol message set in any two communication protocol message sets exceeds a first set threshold, generating an authority comparison list of any two communication protocol message sets, and taking the authority comparison list of any two communication protocol message sets as an access authority matching list of any two communication protocol message sets;

if the message coding modes of any two communication protocol message sets in the plurality of mapping message fields are the same, and the absolute value of the difference between the protocol authentication confidence degrees of any two communication protocol message sets and the protocol authentication confidence degree of one of the any two communication protocol message sets exceeds a second set threshold, taking the authority comparison list of any two communication protocol message sets as an access authority matching list of any two communication protocol message sets; wherein the first set threshold is greater than the second set threshold;

counting all the determined access authority matching lists, determining authority consistency comparison results of authority comparison lists of two corresponding communication protocol message sets in each group of access authority matching lists, and extracting comparison similarity rates of authority events from the authority consistency comparison results; the comparison similarity rate is used for representing the ratio of the number of the same authority events to the total number of the authority events in two corresponding communication protocol message sets, and each authority event corresponds to a service access behavior;

extracting the list characteristics of each group of access right matching lists, converting the characteristic queues of the list structured information used for representing the access right matching lists in each group of list characteristics into characteristic queue description values according to a set conversion mode, and normalizing the characteristic queue description values to obtain normalized description values; wherein the value range of the normalization description value is 0-1;

assigning a first ranking weight to each normalized description value based on field associations between the plurality of mapped message fields, and assigning a second ranking weight to each normalized description value based on mapping path node parameters between the communication protocol message set and the plurality of mapped message fields; sequencing the access authority matching lists according to the descending order of the first sequencing weight to obtain a first sequencing queue and sequencing the access authority matching lists according to the descending order of the second sequencing weight to obtain a second sequencing queue;

for the current access authority matching list in all the access authority matching lists, determining a first queue position of the current access authority matching list in the first sorting queue and a second queue position of the current access authority matching list in the second sorting queue, and calculating a relative position difference value between the first queue position and the second queue position of the current access authority matching list; judging whether the relative position difference is larger than a preset value or not, marking the current access permission matching list when the relative position difference is larger than the preset value, and taking the next access permission matching list of the current access permission matching list as the current access permission matching list and returning to the step of determining the first queue position of the current access permission matching list in the first sorting queue and the second queue position in the second sorting queue when the relative position difference is smaller than or equal to the preset value;

and determining the cross authority authentication result between the mapping message fields according to the access authority matching list obtained by marking.

A4. The method according to a3, wherein determining the cross-authority authentication result between the mapping packet fields according to the marked access authority matching list includes:

after carrying out authority event classification on an access authority matching list obtained by marking to obtain a first event set and a second event set, obtaining a first message signature key of the first event set and a second message signature key of the second event set, wherein the first event set comprises a first authority event generation time list, and the second event set comprises a second authority event generation time list;

acquiring each group of key units in the first message signature key and each group of key units in the second message signature key to obtain a public key and private key distribution queue; determining key authentication logic information between any two groups of key units in the public key and private key distribution queue to obtain a key authentication logic list; adjusting the number of the logic nodes of the logic topology graph of the key authentication logic information in the key authentication logic list to be smaller than the number of the logic nodes of the key authentication logic information of a set number, and updating two groups of key units corresponding to the key authentication logic information to obtain a target logic list corresponding to the key authentication logic list;

analyzing the logic cross of the target logic list corresponding to the key authentication logic list to obtain an analysis result, wherein the analysis result is used for indicating whether the graph data characteristics of the logic topological graph of the key authentication logic information corresponding to the cross time between the first authority event generation time list and the second authority event generation time list are similar or not;

when the analysis result indicates that the graph data characteristics of the logic topological graph of the key authentication logic information corresponding to the cross moment between the first authority event generation moment list and the second authority event generation moment list are similar, obtaining a cross authority authentication result between mapping message fields according to the authority events in the access authority matching list corresponding to the key authentication logic information corresponding to the cross moment; and the mapping time of the mapping message field is associated with the crossing time.

A5. The method according to any one of a1-a4, wherein extracting a data access right list corresponding to the target cloud service data according to the cross right authentication result between the mapping packet fields includes:

acquiring cross authentication path information in a result distribution track of cross authority authentication results among the mapping message fields, extracting path parameters of the cross authentication path information through a logic node corresponding to the maximum heat value in the current logic topology of the cross authentication logic information corresponding to the cross authority authentication results, and determining a first path characteristic sample matched with the cross authentication path information; determining a second path characteristic sample matched with the cross authentication path information through the logic node corresponding to the minimum heat value in the current logic topology in the cross authentication logic information corresponding to the cross authority authentication result based on the first path characteristic sample; extracting service interaction characteristics in the target cloud service data through the data characteristic extraction node of the cross authentication logic information corresponding to the cross authority authentication result based on a second path characteristic sample matched with the cross authentication path information to obtain a service interaction track corresponding to the target cloud service data; and determining a data access authority list corresponding to the target cloud service data according to the equipment identity information of the service equipment carried in the service interaction track.

A6. The method according to a5, wherein the determining, by performing path parameter extraction on the cross authentication path information through the logic node corresponding to the maximum heat value in the current logic topology of the cross authentication logic information corresponding to the cross authorization authentication result, the first path feature sample matching the cross authentication path information includes:

extracting nodes according to first data characteristics in the cross authentication logic information corresponding to the cross authority authentication result, and determining classifier parameters corresponding to the cross authentication path information; determining simulation thread data of an execution function simulator corresponding to the classifier parameters based on a logic execution function in a logic node corresponding to the maximum heat value; responding to simulation thread data of the execution function simulator, performing multi-dimensional characteristic clustering on resource configuration information of any configuration parameter in the classifier parameters through an execution parameter queue of a logic execution function in a logic node corresponding to the maximum heat value, and determining a first clustering information set; according to a feature sample screening script in a logic node corresponding to the maximum heat value, successively performing feature sample screening and multi-dimensional feature clustering on the first clustering information set, and determining a second clustering information set; and importing a second classification information set corresponding to the cross authentication logic information into a preset authority authentication thread, and outputting a first path characteristic sample matched with the cross authentication path information by adjusting thread parameters of the preset authority authentication thread.

A7. The method according to a6, wherein determining, based on the first path feature sample, a second path feature sample matching the cross-authentication path information through the logical node corresponding to the minimum hot value in the current logical topology in the cross-authentication logical information corresponding to the cross-authorization authentication result includes:

constructing a sample distribution map data set corresponding to the first path characteristic sample, and constructing a logic authentication map data set corresponding to a logic node corresponding to the minimum heat value in the current logic topology in the cross authentication logic information corresponding to the cross authority authentication result; wherein the sample distribution map data set and the logic authentication graph data set respectively comprise a plurality of graph data nodes with different graph data association degrees;

extracting graph data connection line information of the first path feature sample in any graph data node of the sample distribution graph data set, and determining the graph data node with the minimum graph data association degree in the logic authentication graph data set as a target graph data node;

mapping the graph data connection information to the target graph data nodes according to the cross authentication sequence of the cross authentication logic information corresponding to the cross authority authentication result, obtaining graph data mapping information in the target graph data nodes, and generating a sample updating list between the first path feature sample and the logic nodes corresponding to the minimum heat value in the current logic topology in the cross authentication logic information corresponding to the cross authority authentication result according to the graph data connection information and the graph data mapping information;

obtaining information to be updated in the target graph data node by taking the graph data mapping information as reference information, mapping the information to be updated to the graph data node where the graph data connecting line information is located according to an updating time table corresponding to the sample updating list, obtaining updating result information corresponding to the information to be updated in the graph data node where the graph data connecting line information is located, and determining the updating result information as information to be matched;

obtaining the graph data connection information and mapping the graph data connection information to a node function container in the target graph data node; according to the cosine distance between the update result information and the sample information corresponding to the plurality of sample functions in the node function container, traversing the matching path parameters corresponding to the information to be matched in the logic authentication graph data set until the node centrality of the graph data node where the obtained matching path parameter is located is consistent with the node centrality of the information to be matched in the sample distribution graph data set, stopping obtaining the matching path parameters in the next graph data node, and adding the information to be matched to the first path feature sample according to the matching path parameters to obtain the second path feature sample.

The correspondingly provisioned big data service platform may be used to perform the method of any of A1-A7 above.

Alternative embodiment two:

B1. a data processing method based on big data and cloud computing comprises the following steps:

determining a plurality of groups of device communication logs with different interaction frequencies based on the target cloud service data, specifically comprising: extracting a plurality of groups of device communication logs with different interaction frequencies corresponding to communication link identifications from a preset database according to the communication link identifications carried in the target cloud service data; the interaction frequency is used for representing the interaction times among different service devices in a set time period;

respectively detecting the communication protocol messages in the multiple groups of device communication logs with different interaction frequencies to obtain communication protocol message sets in the multiple groups of device communication logs with different interaction frequencies, and the method specifically comprises the following steps: extracting a target message with a protocol layer identifier from the device communication log, determining the target message as a communication protocol message and obtaining a communication protocol message set in the device communication log with the plurality of groups of different interaction frequencies when detecting that a message field carrying a communication protocol authentication signature in the target message is detected;

mapping communication protocol message sets in the multiple groups of equipment communication logs with different interaction frequencies into the target cloud service data according to the time sequence correlation coefficient between the multiple groups of equipment communication logs with different interaction frequencies and the target cloud service data to obtain multiple mapping message fields;

wherein:

B2. The method according to B1, wherein identifying access rights to the mapping packet fields according to the field association between the mapping packet fields to obtain a cross-rights authentication result between the mapping packet fields includes:

B3. The method of B2, determining an access right matching list based on protocol authentication confidence of a communication protocol message set corresponding to the plurality of mapping message fields, and determining a cross-right authentication result between the mapping message fields according to the determined access right matching list, includes:

B4. The method according to B3, wherein determining the cross-authority authentication result between the mapping packet fields according to the access-authority matching list obtained by the marking includes:

B5. The method according to any one of B1-B4, wherein extracting a data access right list corresponding to the target cloud service data according to the cross right authentication result between the mapping packet fields includes:

B6. The method according to B5, wherein the determining, by performing path parameter extraction on the cross authentication path information by the logic node corresponding to the maximum heat value in the current logic topology of the cross authentication logic information corresponding to the cross authorization authentication result, a first path feature sample matching the cross authentication path information includes:

B7. The method according to B6, wherein determining, based on the first path feature sample, a second path feature sample matching the cross-authentication path information through the logical node corresponding to the minimum hot value in the current logical topology in the cross-authentication logical information corresponding to the cross-authorization authentication result includes:

B8. A big-data and cloud-computing based data processing system, comprising: a big data service platform and a business device which are communicated with each other; wherein the big data service platform is configured to:

wherein:

It should be understood that the detailed description of the two alternative embodiments can refer to the description of the method shown in fig. 2, and will not be described herein.

Claims

1. A data processing method based on big data and cloud computing is characterized by comprising the following steps:

wherein:

when a first service device corresponding to the target cloud service data accesses a second service device, the data access permission list is used for indicating an access path of the first service device in the second service device;

wherein, according to the field relevance among the multiple mapping message fields, the access authority identification is carried out on the multiple mapping message fields to obtain the cross authority authentication result among the mapping message fields, and the method comprises the following steps:

determining a cross authority authentication result between the mapping message fields according to the determined access authority matching list;

determining an access authority matching list based on the protocol authentication confidence of the communication protocol message sets corresponding to the plurality of mapping message fields, and determining a cross authority authentication result between the mapping message fields according to the determined access authority matching list, wherein the method comprises the following steps:

2. The method of claim 1, wherein determining cross-authority authentication results between the mapped message fields according to the marked access authority matching list comprises:

3. The method according to any one of claims 1-2, wherein extracting a data access permission list corresponding to the target cloud service data according to a cross permission authentication result between the mapping message fields comprises:

4. The method as claimed in claim 3, wherein the determining the first path feature sample matching the cross authentication path information by performing path parameter extraction on the cross authentication path information by the logical node corresponding to the maximum heat value in the current logical topology of the cross authentication logical information corresponding to the cross authority authentication result comprises:

5. The method of claim 4, wherein determining, based on the first path feature sample, a second path feature sample matching the cross-authentication path information by a logical node corresponding to a minimum hot value in a current logical topology in the cross-authentication logical information corresponding to the cross-authority authentication result comprises:

6. A big data service platform, characterized in that it comprises data processing means, which when running implement the method of any of claims 1-5 by at least a plurality of functional modules.

7. A computer-readable storage medium, on which a computer program is stored which, when executed, implements the method of any of claims 1-5.