CN117724891B - Service data processing method and service data processing system - Google Patents
Service data processing method and service data processing system Download PDFInfo
- Publication number
- CN117724891B CN117724891B CN202410174604.1A CN202410174604A CN117724891B CN 117724891 B CN117724891 B CN 117724891B CN 202410174604 A CN202410174604 A CN 202410174604A CN 117724891 B CN117724891 B CN 117724891B
- Authority
- CN
- China
- Prior art keywords
- service
- data
- point
- abnormal
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 94
- 238000003672 processing method Methods 0.000 title claims abstract description 15
- 230000002159 abnormal effect Effects 0.000 claims abstract description 286
- 238000004458 analytical method Methods 0.000 claims abstract description 211
- 238000010586 diagram Methods 0.000 claims abstract description 74
- 230000005856 abnormality Effects 0.000 claims abstract description 42
- 238000000034 method Methods 0.000 claims description 91
- 230000008569 process Effects 0.000 claims description 44
- 238000003860 storage Methods 0.000 claims description 37
- 238000004364 calculation method Methods 0.000 claims description 26
- 238000009877 rendering Methods 0.000 claims description 24
- 238000004140 cleaning Methods 0.000 claims description 16
- 230000002776 aggregation Effects 0.000 claims description 15
- 238000004220 aggregation Methods 0.000 claims description 15
- 238000013507 mapping Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 13
- 238000013524 data verification Methods 0.000 claims description 12
- 230000000295 complement effect Effects 0.000 claims description 11
- 238000013075 data extraction Methods 0.000 claims description 11
- 230000006399 behavior Effects 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 4
- 230000002441 reversible effect Effects 0.000 claims description 2
- 238000012512 characterization method Methods 0.000 claims 4
- 238000004891 communication Methods 0.000 description 34
- 238000007726 management method Methods 0.000 description 29
- 230000006870 function Effects 0.000 description 24
- 238000005070 sampling Methods 0.000 description 21
- 238000005259 measurement Methods 0.000 description 20
- 230000002829 reductive effect Effects 0.000 description 18
- 238000012423 maintenance Methods 0.000 description 17
- 230000005540 biological transmission Effects 0.000 description 14
- 238000010295 mobile communication Methods 0.000 description 13
- 238000004080 punching Methods 0.000 description 11
- 230000000007 visual effect Effects 0.000 description 9
- 238000013500 data storage Methods 0.000 description 7
- 238000009826 distribution Methods 0.000 description 7
- 238000009933 burial Methods 0.000 description 5
- 238000013461 design Methods 0.000 description 5
- 238000012360 testing method Methods 0.000 description 5
- 239000002699 waste material Substances 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 4
- 229920001621 AMOLED Polymers 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 238000007621 cluster analysis Methods 0.000 description 3
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000013144 data compression Methods 0.000 description 3
- 238000012216 screening Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000003416 augmentation Effects 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 2
- 239000002775 capsule Substances 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000007599 discharging Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 239000000725 suspension Substances 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010367 cloning Methods 0.000 description 1
- 239000013078 crystal Substances 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000010006 flight Effects 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000002096 quantum dot Substances 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a business data processing method and a business data processing system, which relate to the technical field of data processing and are applied to a business processing system. Comprising the following steps: the electronic equipment acquires abnormal data from the embedded point data of the service and sends the abnormal data to the server. The server performs abnormal characteristic analysis on the abnormal data to obtain an analysis result corresponding to each business data stream in the abnormal data. The server displays a service link diagram corresponding to one or more service data streams. In the scheme, the server performs abnormal feature analysis on abnormal data, which is sent by the electronic equipment and contains buried point data of points in the service data stream, the points in the service data stream can represent the execution position of the service data stream, and the service abnormality analysis is performed based on the operation data corresponding to the points, so that the efficient and accurate positioning of the service abnormality can be realized.
Description
Technical Field
The embodiment of the application relates to the technical field of data processing, in particular to a business data processing method and a business data processing system.
Background
In order to meet the requirements of users on various services in different scenes, the number of services which can be provided for the users by the electronic equipment is increased, and the complexity and the intelligent degree of the services are also increased. For example, when the electronic equipment detects that the electronic equipment enters the geofence corresponding to the subway station, information such as subway cards/subway riding codes and the like can be output on any display interface (such as a current display interface) in the form of a bullet card and the like, so that more intelligent and more convenient travel business service is provided for the user. For example, through interaction with an application, when a video update message focused by a user is acquired, the electronic device can output video update information on any display interface (such as a current display interface), so as to prompt the user to update the video in time, and provide more intelligent reminding business service for the user.
If the electronic equipment executing service is abnormal, the corresponding intelligent service cannot be provided for the user. In the prior art, for executing a service with an abnormality, the operation log of the service is obtained, and the operation log is traversed to locate the abnormality of the service. However, the method for traversing the running log has low abnormality positioning efficiency and inaccurate abnormality positioning.
Disclosure of Invention
The embodiment of the application provides a service data processing method and a service data processing system, wherein electronic equipment can acquire buried point data of a service and send abnormal buried point data to a server. The server analyzes the data according to the abnormal buried data, so that abnormal positioning of the service is realized, the efficiency and accuracy of abnormal positioning of the service are improved, and the operation and maintenance cost of the service is reduced.
In order to achieve the above object, the following technical solution is adopted in the embodiments of the present application.
In a first aspect, a service data processing method is provided, which is applied to a service processing system, where the service processing system includes a server and one or more electronic devices, and one or more services are deployed in the electronic devices.
The method comprises the following steps:
The electronic equipment acquires abnormal data from the buried point data of the service. The embedded point data comprises normal operation data or abnormal operation data generated by a plurality of preset point positions in the execution process of the service. The anomaly data comprises at least one service data stream of the service, and each service data stream is composed of anomaly operation data of at least one point location.
The electronic device sends the anomaly data to the server.
And the server performs abnormal characteristic analysis on the abnormal data to obtain an analysis result corresponding to each business data stream.
The server displays one or more traffic link graphs. Each service link diagram comprises a plurality of points of one service data stream and an analysis result corresponding to the service data stream.
In the application, because the abnormal data in the buried point data comprises the abnormal operation data of each point in the service execution flow, the point has the function of positioning the service execution process. The server performs abnormal feature analysis on abnormal operation data of each point in service execution, so that the position of the abnormality generated in the service execution process can be obtained, the efficient and accurate positioning of service abnormality is realized, and the complexity of performing service abnormality scanning by traversing the service operation log in the traditional scheme is solved. In addition, the electronic equipment and the server do not need human intervention in the process of processing abnormal data, so that the timeliness of service maintenance is improved under the condition that the user perceives weak or even not perceives the communication cost between the user and the maintenance personnel is reduced; the abnormal nodes are pertinently improved and maintained, and the reliability of service operation in the electronic equipment is improved. The server visually presents the service link diagram comprising the point positions of the service data streams and the analysis results of the service data streams, so that the analysis results of the service abnormality of the electronic equipment are more visual.
In another possible implementation manner of the first aspect, the analysis result includes business index data. The server performs abnormal characteristic analysis on the abnormal data to obtain an analysis result corresponding to each service data stream, including:
and the server calculates service indexes of each service in the abnormal data by taking the service as a unit, and acquires service index data corresponding to each service. Wherein the business index data is used for representing at least one point position causing business abnormality.
In the application, the server can calculate the service index of each service and can acquire the point position with larger abnormal probability in each service, thereby providing data support for subsequent abnormal analysis.
In another possible implementation manner of the first aspect, the calculating, by using the service as a unit, a service index for each service in the abnormal data by the server, to obtain service index data corresponding to each service includes:
And the server calculates the execution times of each point in the service by taking the service as a unit.
The server acquires the point positions with the execution times smaller than a preset first threshold value as service index data of the service.
The preset first threshold value is smaller than N times of the number of service data flows included in the service, and N is a number larger than 0 and smaller than 1.
In the application, the server can determine the service index data based on the execution times of each point location. For example, the point with small execution times is a point with no execution, and the point is either an abnormal point or a point after the abnormal point execution sequence, and the point with normal execution can be rapidly discharged through the execution times, so as to obtain a point with possible abnormality.
In another possible implementation manner of the first aspect, the calculating, by using the service as a unit, a service index for each service in the abnormal data by the server, to obtain service index data corresponding to each service includes:
And the server calculates the execution times of each point in the service by taking the service as a unit.
And if the first execution times of the first point location is larger than the second execution times of the second point location, and the difference between the first execution times and the second execution times is larger than a preset second threshold value, taking the second point location as service index data of the service.
Wherein the first point location and the second point location are adjacent points; the preset second threshold is greater than M times of the number of service data flows included in the service, and M is a number greater than 0 and less than 1.
In the application, the server can determine the service index data based on the execution times of each point location. For example, the point with small execution times is a point with no execution, and the point is either an abnormal point or a point after the abnormal point execution sequence, and the point with normal execution can be rapidly eliminated through the execution times, so as to obtain a point with possible abnormality.
In another possible implementation manner of the first aspect, the analysis result includes an anomaly tag; the server performs abnormal characteristic analysis on the abnormal data to obtain an analysis result corresponding to each service data stream, including:
The server takes the service data stream as a unit, performs abnormal characteristic analysis on each service data stream of each service in the abnormal data, and obtains an abnormal label corresponding to each service data stream; the anomaly tags are used for representing the point positions and the reasons of anomalies of each business data stream.
In the application, the server can perform abnormal feature analysis on each service data stream, and according to the buried data of each point in the service data stream or according to the number of the points in the service data stream, the server can acquire the abnormal label matched with the service data stream, thereby realizing the abnormal analysis and abnormal positioning of the service data stream. The problems of complex flow and low efficiency of the conventional business anomaly positioning by traversing the operation log of the business are avoided.
In another possible implementation manner of the first aspect, the server performs an anomaly characteristic analysis on each service data flow of each service in the anomaly data by taking the service data flow as a unit, and obtains an anomaly tag corresponding to each service data flow, including:
And the server takes the service data stream as a unit, and matches the buried point data of each point in the service data stream with a plurality of preset reference data to obtain an abnormal label corresponding to the service data stream.
The plurality of reference data comprise a plurality of abnormal reference data, each abnormal reference data corresponds to one abnormal label, and the abnormal labels corresponding to different abnormal reference data are different from each other.
Or alternatively
The server takes the service data stream as a unit to determine the number of point positions in each service data stream; based on the number of the point positions in the service data flow, the abnormal labels corresponding to the number are matched.
In the application, because the preset reference data comprises a plurality of abnormal reference data, each abnormal reference data corresponds to an abnormal label respectively, and the server can determine the abnormal label corresponding to the buried data of the service data stream under the condition that the buried data is matched with the preset reference data in a mode of matching the buried data of each point in the service data stream with the preset reference data. Or the server is preset with the abnormal labels corresponding to the number of different points corresponding to each service, and the server can also match the abnormal labels corresponding to the number of the points contained in each service data flow according to the number of the points. The abnormal label can realize the abnormal positioning of the business data flow.
In another possible implementation manner of the first aspect, the analysis result includes service complementary point data of the service data flows, the server performs abnormal feature analysis on the abnormal data, and the obtaining an analysis result corresponding to each service data flow includes:
And the server determines the complementary point position of each business data stream in the abnormal data by taking the business data stream as a unit. The complementary point positions are positions where buried point data are not collected.
And the server acquires the point filling data corresponding to the point filling point positions, performs point filling operation, and obtains the service point filling data of the service data stream.
The point filling data are buried point data corresponding to point filling points preset in the server.
In the application, some point positions may not collect the embedded point data, in order to carry out the integrity of each service data stream of the abnormal feature analysis, the server can acquire the embedded point data corresponding to the embedded point positions from the preset embedded point data to carry out the point filling operation, so that each point position of the service data stream has the embedded point data, and the analysis result of the embedded point data based on the service data stream is more accurate and reliable.
In another possible implementation manner of the first aspect, before the server displays one or more service link graphs, the method further includes:
and the server constructs a service link diagram of each service data stream according to each service data stream and the analysis result of each service data stream.
In the application, the server can construct the service link diagram capable of representing the specific execution condition of the service data stream according to the operation data of each point location and the analysis result of the service data stream. The service link diagram can comprise the point positions of the service data flow, so that the analysis result of the service data flow is more visual.
In another possible implementation manner of the first aspect, the server constructs a service link diagram of each service data flow according to each service data flow and an analysis result of each service data flow, including:
the server acquires a service link map corresponding to the service data flow; the service link map is used for representing the execution sequence and the number of the points in the service data flow.
And the server performs link rendering and node mapping on the service link map according to the operation data of each point position of the service data stream and the analysis result of the service data stream to obtain a service link map of the service data stream.
In the application, the server can construct the service link map corresponding to each service, so that when each service data stream of the service is obtained, the service link map of the service can be directly obtained to construct the service link map of the service data stream. The process of constructing the service link map corresponding to the service data flow is not required to be repeatedly executed, so that the waste of computing resources is reduced, and the waste of the service link map on storage space is also reduced.
In another possible implementation manner of the first aspect, the server performs link rendering and node mapping on the service link map according to the operation data of each point location of the service data stream and an analysis result of the service data stream to obtain a service link map of the service data stream, including:
The server performs link rendering on the executed point positions in the service data flow in a first mode in the service link map, and performs link rendering on the unexecuted point positions in the service data flow in a second mode to obtain a service link map after link rendering.
And the server carries out assignment on each point bit in the service link map after the link rendering according to the operation data of each point bit in the service data stream and the analysis result of the service data stream, so as to obtain a service link map of the service data stream.
According to the application, the server can perform link rendering and node mapping on the constructed service link map according to the operation data of each point location and the analysis result of the service data stream, so as to obtain the service link map capable of representing the specific execution condition of the service data stream. The service link diagram can comprise the point positions of the service data flow, so that the analysis result of the service data flow is more visual.
In another possible implementation manner of the first aspect, the server displays one or more service link graphs, including:
the server displays a target interface; the target interface includes options for the service and options for the service data flow in the service.
And the server responds to the selection operation of the user on the options of the target service data flow in the target interface, and displays a target service link diagram corresponding to the target service data flow. The target service data stream is one or more service data streams contained in the service.
In the application, the server can display the target service link diagram corresponding to the service data flow, the executed node and the unexecuted node in the service execution process are differentially rendered in the target service link diagram, and the target service link diagram can intuitively display the execution condition of the service data flow for a maintainer or a user, so that the analysis result of the service data flow is more intuitive.
In another possible implementation manner of the first aspect, after displaying the target service link diagram corresponding to the target service data flow, the method further includes:
and the server responds to the selection operation of the user on the target point position in the target service link diagram, and displays the operation data and the abnormal label of the target point position.
In the application, if the user moves the cursor to the target point position in the target service link diagram, the server can respond to the operation to display the operation data and the abnormal label of the point position of the cursor on the display interface of the target service link diagram, so that the analysis result of each point position in the service data flow is more visual and more convenient to display in the display interface.
In another possible implementation manner of the first aspect, the electronic device obtains abnormal data from buried point data of a service, including:
The electronic equipment acquires buried point data generated by executing the service. The electronic equipment performs first data processing on the embedded data, and stores the first embedded data after the first data processing into a database. Wherein the first data processing includes data checksum data serialization; the data verification comprises the length and the integrity verification of the original buried point data; the data sequence number includes a data format conversion for the buried point data.
The electronic equipment reads the first buried point data from the database, and performs second data processing on the first buried point data to obtain first abnormal data. Wherein the second data processing includes data deserialization and exception cleaning; the data deserialization comprises data format reverse conversion of the first buried point data, the abnormal cleaning comprises deleting normal operation data in the first buried point data, and retaining the abnormal operation data in the first buried point data.
Then, the electronic device sends the anomaly data to the server, including:
the electronic device sends the first exception data to the server.
In the application, after the electronic equipment acquires the buried point data, the first data processing can be carried out on the buried point data, so that the buried point data conforming to the data verification is converted into a format required by a database and is stored in the database. Further, before the electronic equipment reads the embedded point data from the database and prepares to send the embedded point data to the server, operations such as abnormal cleaning are carried out, only abnormal data are sent to the server, normal operation data in the embedded point data are filtered, interference of the normal operation data on abnormal feature analysis of the server is reduced, and efficiency and accuracy of the server on the abnormal feature analysis can be improved; the data volume of data transmission between the electronic equipment and the server is reduced, and the transmission efficiency between the electronic equipment and the server is improved.
In another possible implementation manner of the first aspect, the electronic device obtains buried point data generated by executing the service, including:
The electronic equipment determines the point positions configured by each service according to a preset service point burying rule. The preset service embedded point rule comprises configuration information of point positions of each service. The electronic equipment acquires buried point data generated when the business is executed to each point location.
In the application, because the point positions of the buried point data required to be collected by each service are configured in the service buried point rule, the electronic equipment can collect the buried point data in real time in the service execution process according to the point positions corresponding to each service configured in the service buried point rule. Thereby obtaining buried point data for at least one service execution at least once. The point location is often set at a position where a behavior or an event is generated in the service execution process, and specific conditions of service execution can be effectively represented based on buried point data collected by the point location. And when the server performs abnormal feature analysis based on the buried data, the business abnormality based on the point location can be rapidly positioned.
In another possible implementation manner of the first aspect, the electronic device obtains buried point data generated by executing the service, including:
And the electronic equipment determines the shielding point positions corresponding to the services according to the preset point position shielding rules. The shielding point positions are point positions in the point positions of the service, and the point positions do not need to collect buried point data. The electronic device acquires buried point data of each point except the shielding point where the service is executed.
In the application, because the point positions which do not need to collect the buried point data are configured in the point position shielding rule, the electronic equipment can not collect the buried point data of the shielded point positions, and the waste of collection resources in the process of collecting the electronic equipment can be reduced; in addition, the buried point data of the shielding points are already existing in the server, the electronic equipment does not collect the buried point data of the shielding points, and data redundancy caused by repeated collection of the buried point data can be avoided. Meanwhile, the data volume of buried point data transmitted between the electronic equipment and the server is reduced, and the data transmission efficiency between the server and the electronic equipment is improved.
In another possible implementation manner of the first aspect, the electronic device obtains buried point data generated by executing the service, including:
And the electronic equipment determines the target service according to the preset service reporting rule. Wherein the target service is a service of a plurality of services. The electronic equipment acquires buried point data generated by executing the target service.
According to the application, the electronic equipment only acquires the buried point data generated by the target service, screens the buried point data acquired by the electronic equipment from the service dimension, and can effectively remove the buried point data which is not needed by the abnormal analysis of the service, so that the data interference caused by useless buried point data and the waste of calculation resources and transmission resources in the data processing process are avoided.
In another possible implementation manner of the first aspect, the sending, by the electronic device, the exception data to the server includes:
The electronic equipment performs data aggregation on the abnormal data to form a data packet corresponding to the abnormal data. The electronic device sends the data packet to the server.
In the application, the data packet is generally a data packet after compression processing, the electronic equipment performs data aggregation processing on the abnormal data to be sent, and sends the abnormal data to the server in the form of the data packet, so that the size of the abnormal data sent to the server can be reduced, the transmission resource occupation redundancy caused by overlarge abnormal data is avoided, the compressed packet with smaller size is transmitted, and the data transmission rate between the server and the electronic equipment can be improved.
In another possible implementation manner of the first aspect, the method further includes:
And the server receives the data packet sent by the electronic equipment. And the server performs third data processing on the data packet to obtain second abnormal data after the third data processing. Wherein the third data processing includes data extraction and data splitting; the data splitting comprises splitting the data packet into service data streams in service units; the data extraction comprises extracting a preset proportion of service data streams from the split service data streams.
Then, the server performs an anomaly characteristic analysis on the anomaly data, including:
And the server performs abnormal characteristic analysis on the second abnormal data.
In the application, after the server receives the data packet sent by the electronic equipment, the server can split the data packet to obtain the service data flow of each service, thereby extracting the service data flow with the preset proportion from the data packet to perform the abnormal feature analysis, reducing the data volume of the abnormal feature analysis and improving the efficiency of the abnormal feature analysis.
In another possible implementation manner of the first aspect, the method further includes:
The server updates a preset service embedded point rule of the electronic equipment; and/or the number of the groups of groups,
The server updates a preset point location shielding rule of the electronic equipment; and/or the number of the groups of groups,
And the server updates the preset business reporting rule of the electronic equipment.
In the application, the server can update the preset business embedded point rule, the preset point location shielding rule, the preset business reporting rule and the like according to the abnormal data of each business received from the electronic equipment, so that the business embedded point rule, the business reporting rule and the point location shielding rule are more in accordance with the actual running state of each business, and the embedded point data of the business acquired by the electronic equipment according to the rules are more effective.
In a second aspect, a business data processing system is provided, the exception handling system comprising an electronic device and a server. The electronic device comprises a maintenance Software Development Kit (SDK), and one or more services are deployed in the electronic device. The server comprises a big data platform and a business anomaly analysis platform.
The maintenance SDK acquires abnormal data from buried point data of the service; the buried data comprises normal operation data or abnormal operation data generated by a plurality of preset point positions in the execution process of the service. The anomaly data comprises at least one service data stream of the service, and each service data stream is composed of anomaly operation data of at least one point location.
The dimension SDK sends the abnormal data to the server.
And the big data platform performs abnormal characteristic analysis on the abnormal data to obtain an analysis result corresponding to each business data stream.
The business anomaly analysis platform displays one or more business link graphs. Each service link diagram comprises a plurality of points of one service data stream and an analysis result corresponding to the service data stream.
In a third aspect, an apparatus is provided that includes a memory, a processor, and a computer program stored on the memory; the device includes an electronic device and a server. Wherein the processor executes a computer program to implement the steps of the method of the first aspect.
In a fourth aspect, there is provided a computer readable storage medium having stored thereon a computer program/instruction which when executed by a processor implements the steps of the method of the first aspect.
In a fifth aspect, there is provided a computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the method of the first aspect.
In a sixth aspect, an embodiment of the application provides a chip comprising a processor for invoking a computer program in memory to perform a method as in the first aspect.
It will be appreciated that the advantages achieved by the service data processing system according to the second aspect, the device according to the third aspect, the computer readable storage medium according to the fourth aspect, the computer program product according to the fifth aspect, and the chip according to the sixth aspect provided above may refer to the advantages in any one of the possible designs of the first aspect and the advantages will not be repeated here.
Drawings
FIG. 1 is a scene diagram for analyzing business anomalies provided by an embodiment of the application;
fig. 2 is a schematic diagram of a service flow according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a business data processing system formed by a plurality of electronic devices and a server according to an embodiment of the present application;
Fig. 4 is a schematic hardware structure of an electronic device according to an embodiment of the present application;
Fig. 5 is a schematic hardware structure of a server according to an embodiment of the present application;
Fig. 6 is a schematic diagram of a software architecture of an electronic device and a schematic diagram of a software architecture of a server in a service data processing system according to an embodiment of the present application;
Fig. 7 is a schematic software architecture of a more detailed electronic device according to an embodiment of the present application;
FIG. 8 is a schematic diagram of a software architecture of a server according to an embodiment of the present application;
fig. 9 is a schematic flow chart of a first stage of a service data processing method according to an embodiment of the present application;
Fig. 10 is a schematic flow chart of a part of a first stage and all second stages of a service data processing method according to an embodiment of the present application;
fig. 11 is a schematic diagram of a service link diagram of a service a displayed by a display module of a server in a display interface of a display screen according to an embodiment of the present application.
Detailed Description
In the description of embodiments of the present application, the terminology used in the embodiments below is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification of the present application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include, for example, "one or more" such forms of expression, unless the context clearly indicates to the contrary. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" means one or more than two (including two). The term "and/or" is used to describe an association relationship of associated objects, meaning that there may be three relationships; for example, a and/or B may represent: a alone, a and B together, and B alone, wherein A, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship.
Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise. The term "coupled" includes both direct and indirect connections, unless stated otherwise. The terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated.
In embodiments of the application, words such as "exemplary" or "such as" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "e.g." in an embodiment should not be taken as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.
In order to meet the requirements of users on various services in different scenes, the number of the services deployed in the electronic equipment is increased, and the different services in the electronic equipment can be services provided by the users in different scenes, so that the complexity and the intelligent degree of the services are also increased. After the user starts the intelligent service function of the electronic equipment, the electronic equipment can display corresponding information in a preset form according to different scenes and corresponding bound system applications or third party applications in different scenes, and provides convenient business service for the user. The preset form can be a negative one-screen card set, a desktop card set, a suspension capsule, a notification bar and the like.
Illustratively, when the electronic device is a mobile phone, the business services that can be provided by the mobile phone include travel services, life services, and the like.
The following embodiments provide some scenarios in which a mobile phone provides travel services for a user.
For example, the mobile phone detects that the position of the mobile phone enters the geofence corresponding to the subway station, obtains the subway card/subway riding code provided by the third party application providing the subway travel service, and outputs the subway card/subway riding code in a preset mode so as to be read by a gate of the subway station, thereby providing more intelligent and more convenient subway travel service for users. For example, when the mobile phone detects that the position of the mobile phone enters a geofence corresponding to an airport, the mobile phone acquires the flight information provided by a third party application providing the flight travel service, and can output the flight information in a preset form so that a user can acquire the flight information in time or perform operations such as the flight check-in time, and the like, thereby providing more intelligent and more convenient flight travel service for the user. The flight travel service comprises services such as flight change information reminding, check-in seat reminding, check-in and the like. For example, the mobile phone detects that the position of the mobile phone enters the geofence corresponding to the railway station, acquires train number information provided by third party application providing train travel service, and can output the train number information in a preset form so as to enable a user to acquire the train number information in time and provide more intelligent and more convenient train travel service for the user. The train travel services comprise train number waiting reminding, train number ticket checking reminding and the like.
The following embodiments provide some scenarios in which a mobile phone provides a life service for a user.
For example, the mobile phone detects that the position of the mobile phone enters the geofence corresponding to the express post, acquires the express information provided by the third party application providing the express reminding service, and can output the express information in a preset form so that a user can acquire the express information in time, and provides more intelligent and more convenient express service for the user. The express delivery pickup service comprises the services of checking pickup codes, picking up one piece and the like. For example, the mobile phone can output calendar information in a preset mode, so that more intelligent and more convenient schedule reminding service, meeting reminding service and the like are provided for the user. For example, the mobile phone detects that the position of the mobile phone enters a geofence corresponding to a company, and according to the current system time, the opening reminding information can be output in a preset mode, so that a user can punch a card in time, and more intelligent and more convenient attendance card punching service is provided for the user. For example, the mobile phone can acquire video update information provided by the video application, output video update reminding information in a preset form, timely remind a user that the video is updated, and provide more intelligent reminding service for the user. For example, the mobile phone detects that the position of the mobile phone enters the geofence corresponding to the common merchant, obtains information such as a payment code provided by a third party application providing a payment service, outputs the information such as the payment code in a preset form, so that merchant equipment can read the payment code to pay, and provides more intelligent and more convenient payment service for a user.
The intelligent service function provided by the electronic equipment aims at enabling a user to quickly and conveniently use corresponding services without opening a third party application. If abnormal business execution occurs, a user is usually required to feed back to a maintenance personnel to analyze and repair the abnormal business. The business anomaly analysis and repair based on the manual feedback is poor in timeliness and high in manual communication cost. Moreover, maintenance personnel often determine the place where the business is abnormal and the reason for the abnormality by traversing and analyzing the operation log of the corresponding business in the electronic equipment. Because the operation log of the service is often data with longer time, and as the complexity of the service is higher, the nodes included in the service are more and more, the data volume of the operation log generated by executing the service once or for a plurality of times in a day is larger, and the data transmission efficiency is low when a maintainer exports the operation log. In addition, the method for traversing the operation log is long in traversing time, and can not accurately locate the abnormality of the service, so that the abnormality analysis efficiency is low.
For example, refer to the traffic anomaly analysis scene graph shown in FIG. 1. The business abnormality analysis in the prior art is described by taking the case of abnormality in the business card punching business. Illustratively, step 1: the electronic equipment does not output the prompt information of the card punching of going to and off duty for several continuous days. The user finds out the problem and feeds back the problem of the business card punching business. Step 2: and the maintenance personnel of the electronic equipment communicate with the user to know the actual use condition of the electronic equipment in the business trip card punching scene. For example, whether to start a business trip card punching prompt (intelligent service function), the service duration of the electronic equipment, whether the electronic equipment has cloning operation, etc. Step 3: and the maintenance personnel sign the user agreement, and export the database of the electronic equipment to obtain the related data of the business card punching business. The related data comprise data contained in operation logs such as card punching data, portrait labels, fence data and the like. Step 4: based on the related data of the business card punching business, the maintenance personnel preliminarily determines the abnormal situation and communicates with the user whether human intervention exists or not. Step 5: and the maintenance personnel acquires the operation log of the business card punching business of the business trip in the electronic equipment, and performs business abnormality and positioning based on the operation log. Step 6: and repairing the problem by a maintainer aiming at the business abnormality. The manual communication cost is high in the whole process, the abnormality analysis efficiency of the business card punching business on duty and off duty is low, and the business abnormality positioning is performed by traversing the operation log, so that the abnormality positioning is inaccurate.
In the face of increasingly complex services, buried point technology may be employed to monitor the execution of the service. Specifically, the buried point technology refers to a technology for capturing and processing data of a point location using a specific user behavior/event as the point location. The point location (buried point position) may be any node of the service execution. For example, the service is a bullet card, and refer to the service flow diagram shown in fig. 2. Fig. 2 shows a schematic of all possible business processes corresponding to the bullet train. Illustratively, the nodes included in the service include, a node that triggers a start time, a node that registers a geofence, a node that enters a geofence, a node that goes out of a card, a node that registers a payment fence, a node that registers a card-pinning fence, a node that triggers a card-pinning fence, a node that pins a card, a node that does not go out of a card, a node that regularly pays time, a node that leaves a geofence, a node that registers an end time fence, a node that triggers an end time fence, a node that registers a return desktop fence, and a node that registers a return desktop, etc.
In some embodiments, the electronic device may set a point location at each node to monitor the execution condition of each node (point location) in the service execution flow, and obtain the buried point data generated by each node (point location). The buried data characterizes the operational data of the node (point location) during the execution of the service. Wherein, the operation data comprises user behavior/event/operation data and the like which occur in the service executing process. For example, a point location is set at the card discharging node, and if the electronic equipment is successful in discharging the card, the subway card is output on the display interface in any form. The buried point data of the card-out node may include information that characterizes the success of the card-out. If the electronic equipment fails to output the card, the electronic equipment fails to trigger the display interface to output the operation of the subway card. The buried point data of the card-out node may include information characterizing card-out failure.
And setting point positions for one or more nodes in the service execution flow, and acquiring buried point data corresponding to each node, so that the node execution can be primarily judged to be normal or abnormal based on the buried point data.
In some embodiments, for a bullet subway card service, for example, the key nodes may include nodes such as triggering a fence, clicking a bullet card, etc., the node event corresponding to the key node triggering the fence may be above/below ground, the node event corresponding to the key node bullet card clicking may be clicking/not clicking/clicking a desktop application, etc. The service-based key nodes may set point positions, and the buried point data corresponding to the point positions may include node events.
In view of the above problems, an embodiment of the present application provides a service data processing method. The method is applied to a service data processing system, and the service data processing system comprises electronic equipment and a server. The electronic device may acquire the abnormal data from the buried point data generated from the service. The embedded data refer to normal operation data or abnormal operation data generated by a plurality of preset point positions in the execution process of the service. The abnormal data comprises at least one service water flow of the service, and each service data flow consists of abnormal operation data of at least one point location. The electronic device sends the anomaly data to the server. The server performs abnormal characteristic analysis on the abnormal data to obtain an analysis result of each business data stream in the abnormal data. The server may display one or more traffic link graphs. The service link diagram comprises a plurality of points of one service data stream and an analysis result of the service data stream.
By adopting the scheme, because the abnormal data in the buried point data comprises abnormal operation data of each point in the service execution flow, the point has the function of positioning the service execution process. The server performs abnormal characteristic analysis on abnormal operation data of each point in service execution, so that the position of the abnormality in the service execution process can be obtained, and the efficient and accurate positioning of the service abnormality is realized. In addition, the electronic equipment and the server do not need human intervention in the process of processing abnormal data, so that the timeliness of service maintenance is improved under the condition that the user perceives weak or even not perceives the communication cost between the user and the maintenance personnel is reduced; the abnormal nodes are pertinently improved and maintained, and the reliability of service operation in the electronic equipment is improved. The server visually presents the service link diagram comprising the point positions of the service data streams and the analysis results of the service data streams, so that the analysis results of the service abnormality of the electronic equipment are more visual.
The business data processing method provided by the embodiment of the application can be applied to a business data processing system. The business data processing system includes a server and one or more electronic devices. Referring to FIG. 3, a schematic diagram of a business data processing system formed by a plurality of electronic devices and servers is shown. The electronic equipment is used for acquiring buried point data of each service, screening abnormal data from the buried point data and sending the abnormal data to the server. The server is used for carrying out abnormal characteristic analysis according to the received abnormal data to obtain an analysis result of each service data flow in the abnormal data, and displaying a service link diagram corresponding to the service data flow.
The electronic device in this embodiment may be an electronic device with an intelligent service function. It should be noted that, the intelligent service function related to the embodiment of the present application may be turned on through the system setting interface. For example, when the electronic device is a mobile phone, the smart service function can be started by a smart assistant of the system setup interface. Further, when the user starts the function of 'participating in experience and improving plan', the electronic device can report the buried point data corresponding to the acquired service to the server for positioning and analyzing the abnormal service.
The electronic device may also be called a terminal (terminal), a User Equipment (UE), a Mobile Station (MS), a Mobile Terminal (MT), or the like. The electronic device may be a mobile phone, a wearable device, a tablet computer (Pad), a computer with wireless transceiving function, a Virtual Reality (VR) device, an augmented reality (augmented reality, AR) device, etc.
The server may be a cloud server, an independent server, a server cluster, or the like. The embodiment of the application does not limit the specific technology and the specific equipment form adopted by the electronic equipment and the server.
The following describes a hardware structure of the electronic device with reference to fig. 4, taking the electronic device as an example of a mobile phone.
Fig. 4 shows a schematic hardware structure of the electronic device 100 according to an embodiment of the present application. As shown in fig. 4, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a display 194, and the like.
The processor 110 may include one or more processing units, such as: processor 110 may include a controller, an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (IMAGE SIGNAL processor, ISP), memory, a video codec, a digital signal processor (DIGITAL SIGNAL processor, DSP), a baseband processor, and/or a neural-network processor (neural-network processing unit, NPU), etc. The controller may be a neural center or a command center of the mobile phone 100. The controller can generate operation control signals according to the instruction operation codes and the time sequence signals to finish the control of instruction fetching and instruction execution. A memory may also be provided in the processor 110 for storing instructions and data.
A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to reuse the instruction or data, it may be called directly from memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby improving the efficiency of the system.
The wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.
The electronic device 100 implements display functions through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering.
The display screen 194 is used for displaying an operation interface of the projection screen APP, a projection screen image, a projection screen video, and the like. The display 194 includes a display panel. The display panel may employ a Liquid Crystal Display (LCD) CRYSTAL DISPLAY, an organic light-emitting diode (OLED), an active-matrix organic LIGHT EMITTING diode (AMOLED), a flexible light-emitting diode (FLED), a quantum dot LIGHT EMITTING diodes (QLED), or the like. In some embodiments, the cell phone 100 may include 1 or N display screens 194, N being a positive integer greater than 1.
In this embodiment, for example, when executing the subway card flicking service, if the service execution is successful, the display screen 194 may display the subway card; for example, when executing the flight travel reminder service, if the service execution is successful, the display 194 may display flight information, etc.
The external memory interface 120 may be used to connect external memory cards to enable expansion of the memory capabilities of the electronic device 100. The external memory card communicates with the processor 110 through an external memory interface 120 to implement data storage functions.
The internal memory 121 may be used to store computer-executable program code that includes instructions. The processor 110 executes various functional applications of the electronic device 100 and data processing by executing instructions stored in the internal memory 121. The internal memory 121 may include a storage program area and a storage data area. The storage program area may store an application program (APP) required for at least one function (e.g., a camera APP, a gallery APP, and third-party video editing software, etc.) and the like of an operating system. The storage data area may store data created during use of the mobile phone 100 (e.g., photographs or videos taken, screenshots taken by the mobile phone, screen recordings of the mobile phone, images downloaded from other devices, and video sets generated using a one-touch sheeting function), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may further include a nonvolatile memory such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (universal flash storage, UFS), and the like.
The wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.
The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 100 may be used to cover a single or multiple communication bands. Different antennas may also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed into a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.
The mobile communication module 150 may provide a solution for wireless communication including 2G/3G/4G/5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA), etc. The mobile communication module 150 may receive electromagnetic waves from the antenna 1, perform processes such as filtering, amplifying, and the like on the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation. The mobile communication module 150 can amplify the signal modulated by the modem processor, and convert the signal into electromagnetic waves through the antenna 1 to radiate. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be provided in the same device as at least some of the modules of the processor 110.
The modem processor may include a modulator and a demodulator. The modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs sound signals through an audio device (not limited to the speaker 170A, the receiver 170B, etc.), or displays images or video through the display screen 194. In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be provided in the same device as the mobile communication module 150 or other functional module, independent of the processor 110.
The wireless communication module 160 may provide solutions for wireless communication including wireless local area network (wireless local area networks, WLAN) (e.g., wireless fidelity (WIRELESS FIDELITY, wi-Fi) network), bluetooth (BT), global navigation satellite system (global navigation SATELLITE SYSTEM, GNSS), frequency modulation (frequency modulation, FM), near field communication (NEAR FIELD communication, NFC), infrared (IR), etc., as applied to the electronic device 100. The wireless communication module 160 may be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signals, filters the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive a signal to be transmitted from the processor 110, frequency modulate it, amplify it, and convert it to electromagnetic waves for radiation via the antenna 2.
In some embodiments, antenna 1 and mobile communication module 150 of electronic device 100 are coupled, and antenna 2 and wireless communication module 160 are coupled, such that electronic device 100 may communicate with a network and other devices through wireless communication techniques. The wireless communication techniques can include the Global System for Mobile communications (global system for mobile communications, GSM), general packet radio service (GENERAL PACKET radio service, GPRS), code division multiple access (code division multiple access, CDMA), wideband code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC, FM, and/or IR techniques, among others. The GNSS may include a global satellite positioning system (global positioning system, GPS), a global navigation satellite system (global navigation SATELLITE SYSTEM, GLONASS), a beidou satellite navigation system (beidou navigation SATELLITE SYSTEM, BDS), a quasi zenith satellite system (quasi-zenith SATELLITE SYSTEM, QZSS) and/or a satellite based augmentation system (SATELLITE BASED AUGMENTATION SYSTEMS, SBAS).
In this embodiment, the electronic device 100 may communicate with a server through the wireless communication module 160.
The electronic device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like.
It should be understood that the illustrated structure of the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the application, electronic device 100 may include more or fewer components than shown, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
Fig. 5 shows a schematic hardware structure of a server.
Exemplary, a schematic diagram of the structure of the server 200 in fig. 5 is shown. The server 200 shown in fig. 5 may include: processor 201, memory 202, communication interface 203, display 204, and bus 205. The processor 201, the memory 202, the communication interface 203, and the display 204 may be connected via a bus 205.
The processor 201 is a control center of the server, and may be a general-purpose central processing unit (central processing unit, CPU), or may be another general-purpose processor. Wherein the general purpose processor may be a microprocessor or any conventional processor or the like.
As an example, processor 201 may include one or more CPUs, such as CPU 0 and CPU 1 shown in fig. 5.
Memory 202 may be, but is not limited to, read-only memory (ROM) or other type of static storage device that can store static information and instructions, random access memory (random access memory, RAM) or other type of dynamic storage device that can store information and instructions, or electrically erasable programmable read-only memory (EEPROM), magnetic disk storage or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.
In one possible implementation, the memory 202 may exist independent of the processor 201. The memory 202 may be coupled to the processor 201 via the bus 204 for storing data, instructions or program code. The processor 201, when calling and executing instructions or program codes stored in the memory 202, can implement the service data processing method provided by the embodiment of the present application.
In another possible implementation, the memory 202 may also be integrated with the processor 201.
Communication interface 203 for the electronic device to connect with other devices through a communication network, which may be an ethernet, a radio access network (radio access network, RAN), a wireless local area network (wireless local area networks, WLAN), etc. The communication interface 203 may include a receiving unit for receiving data, and a transmitting unit for transmitting data.
In this embodiment, the server 200 may communicate with the electronic device 100 through the communication interface 203, and receive the abnormal data sent by the electronic device 100.
And the display screen 204 is used for information such as a service link diagram corresponding to the abnormal data. The display 204 includes a display panel. The display panel may be LCD, OLED, AMOLED, FLED, QLED, etc.
In this embodiment, the display screen 204 may display a service link map corresponding to each piece of abnormal data of each service. In some other embodiments, the display 204 may also display relevant information such as the buried data corresponding to each node in the service link graph; the display screen 204 can also display other data needed to be displayed in the business anomaly analysis process according to actual requirements.
Bus 205 may be an industry standard architecture (industry standard architecture, ISA) bus, an external device interconnect (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus, or an extended industry standard architecture (extended industry standard architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in fig. 5, but not only one bus or one type of bus.
It should be noted that the structure shown in fig. 5 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than those shown in fig. 5, or may combine some components, or may be arranged with different components.
Referring to fig. 6, fig. 6 shows a schematic diagram of the software architecture of the electronic device 100 and a schematic diagram of the software architecture of the server 200 in a service data processing system. The software architecture of the electronic device 100 and the software architecture of the server 200 involved in the business data processing system are described below in connection with fig. 6.
Referring to fig. 6, illustratively, a business module and a non-functional attribute design module (DFX) are included in the software architecture of the electronic device 100. And the service module reports the embedded point data of the service to the DFX when executing the service once. The DFX may perform buried point control, data processing, service processing, and data transmission on buried point data of the service according to a preset correlation rule. The preset relevant rules comprise a preset service embedded point rule, a preset service reporting rule, a preset service sampling rule, a preset query frequency, a preset point location shielding rule, a preset abnormal data cleaning rule and the like. The electronic device may compress and package the buried data to be sent to the server, and send the compressed and packaged data packet to the server for service exception analysis (exception feature analysis).
Referring to fig. 6, illustratively, a business anomaly analysis platform and a big data platform are included in the software architecture of the server 200. The big data platform is used for carrying out abnormal feature analysis on the buried point data of the business in the data packet according to the data packet transmitted by the electronic equipment, and obtaining an analysis result of the buried point data of the business. Alternatively, the big data platform may store the analysis results into a storage module. The storage module may include, for example, a mysql database, an elastic search storage engine (es storage engine), and a storage configuration unit. The storage configuration unit may configure parameters such as a structure of the data storage, a length of the data storage, and the like. The business anomaly analysis platform is used for carrying out rule calculation and generating related rules such as business embedded point rules, business reporting rules and the like. In some embodiments, when an update of a rule occurs, the server may send the updated relevant rule to the electronic device. Or the electronic device may obtain a preset correlation rule from the server according to a preset query frequency. The business anomaly analysis platform is also used for displaying a business link diagram. In some embodiments, the business anomaly analysis platform may display a portal (portal) of business anomaly analysis. And the business anomaly analysis platform constructs a business link diagram corresponding to the business data stream according to the analysis result of the buried point data of the business transmitted by the big data platform and displays one or more business link diagrams corresponding to the business data stream.
In particular, referring to fig. 7, fig. 7 shows a more detailed software architecture diagram of the electronic device 100. Illustratively, the business module of the electronic device 100 in fig. 7 includes a plurality of business scenarios. The business scenario includes, for example, regular payments, payment assistants, subways, trains, flights, etc. The non-functional attribute design module of the electronic device 100 in fig. 7 may be a dimension software development kit (software development kit, SDK). The process of performing embedded point control on the service by the dimension measurement SDK includes obtaining embedded point data generated by each node in the service execution process by the dimension measurement SDK according to a preset service embedded point rule. The service processing process of the dimension measurement SDK comprises the steps that the dimension measurement SDK obtains a service identifier corresponding to the service reported by the service module, so that buried data of the service are filtered based on the service identifier, and a corresponding relation between the service identifier and the buried data of the service is established. The service identification may be represented, for example, as a tracking identification number (TRACE IDENTITY documents, traceID). The service may be executed multiple times, and in each execution process, buried point data of a point location can be obtained when the service is executed to the point location; each time a service is executed, a service data stream including buried data of the executed point location can be obtained. The service is performed a plurality of times, then one traceID corresponds to a plurality of service data flows. The process of carrying out data processing on the dimension measurement SDK comprises the steps of carrying out data verification, point location shielding, data serialization, data storage and the like on the buried data reported by the service module by the dimension measurement SDK. Alternatively, the dimension SDK may store the processed data in a database. The database may be, for example, SQLite. The process of data transmission by the dimension measurement SDK comprises the steps of obtaining buried point data of a service from a database by the dimension measurement SDK, carrying out data extraction, data anti-sequence, data point location aggregation, rule matching, data cleaning, data aggregation, compression encryption and the like on the buried point data of the service, and transmitting the processed data packet to a server through a transmission channel.
Alternatively, the service module of the electronic device may be deployed at an application layer, and the application layer of the electronic device may further include other services, such as sports health, communication office, and the like.
Specifically, referring to fig. 8, fig. 8 shows a more detailed software architecture diagram of the server 200. Illustratively, the server 200 in fig. 8 includes a big data platform and a business anomaly analysis platform. The large data platform performs data preprocessing and anomaly analysis processing on the received data packets. The data preprocessing comprises the steps of storing buried data of a business received by a big data platform; the big data platform performs data extraction on the buried data of the stored business; the big data platform decompresses the data of the extracted buried data of the business; the big data platform performs data splitting on the buried data of the decompressed service; and the big data platform performs data analysis on the buried data of the split business. The process of carrying out exception analysis processing on the big data platform comprises the steps of carrying out service index synchronization, service configuration synchronization, service point complement synchronization and service feature synchronization on buried point data of each service. After the big data platform performs service index synchronization, service index calculation can be performed on the buried data, so that service index data of the buried data is obtained. After the big data platform performs service point filling synchronization, the buried data can be subjected to service point filling extraction, service point filling calculation, service data sampling and service data aggregation, so that service point filling data is obtained; after service data aggregation and service feature synchronization, the big data platform performs service feature calculation, so as to obtain service feature data corresponding to the buried data. Wherein the business feature data comprises an anomaly tag.
The data of the data processing is formed into a data lake by the big data platform. After the service index data, the service point filling data and the service characteristic data are obtained, the big data platform transmits the service index data, the service point filling data and the service characteristic data to the service abnormality analysis platform through the lake outlet channel for further data processing.
The business anomaly analysis platform comprises a business access module, a link management module, a feature management module, a rule calculation module, a group analysis module, a lake outlet channel, a storage module and a display module. The lake outlet channel is used for receiving service index data, service complement point data and service characteristic data transmitted by the large data platform, splitting the data into one or more service data streams by taking traceID of service as a unit, and storing the service data streams in a storage module in a distributed manner.
The link management module is used for carrying out link drawing, node mapping and association point compensation on the service to obtain a service link map. The service link map is used for representing the topological structure among all nodes in the service, and the operation data of all the nodes are not included. In some embodiments, the link management module may also store the traffic link map to the storage module.
The display module is used for constructing and displaying a service link diagram corresponding to the service data flow according to the service data flow in the storage module. One way may be that the display module performs link rendering and node mapping on the service link map provided by the link management module according to the service data flow, so as to obtain a service link map corresponding to the service data flow. Still another possible way includes that the display module draws a service link map according to the service data flow, and further performs link rendering and node mapping on the service link map to obtain a service link map corresponding to the service data flow. Optionally, the display module may further store the service link map in a link file or a storage module.
The feature management module is used for carrying out feature management and service identification on the buried point data of the service, so as to obtain service features of the buried point data of each service. The characteristic management module returns the service characteristics to the big data platform, and the big data platform can carry out service characteristic synchronization according to the service characteristics of the buried point data of the service.
The rule calculation module is used for carrying out rule calculation and generating relevant rules required by business anomaly analysis. For example, the relevant rules include a service embedded point rule, a service reporting rule, a service sampling rule, a query frequency, a point location masking rule, an abnormal data cleaning rule, and the like.
The service access module is used for acquiring service information, establishing a lake outlet channel, determining a lake outlet proportion, determining a point location shielding rule and providing a visual interface. For example, the service access module acquires service information and can provide data support for the large data platform to perform service index synchronization.
The group analysis module is used for carrying out error distribution analysis on batch businesses and carrying out buried point monitoring on the businesses. The batch services may be the same type of services, or services executed in the same time period, or services in the same type of equipment, or services under the same version of system operation, and so on.
In connection with the software modules of the electronic device presented in fig. 7 and the software modules of the server presented in fig. 8, fig. 9 and 10 provide a flow chart of interactions between the software modules of the electronic device and the software modules of the server. The service data processing method provided by the embodiment of the application is further described with reference to fig. 9 and 10. Fig. 9 is a schematic flow chart of a first stage of a service data processing method, where the first stage includes an interaction portion where data transmission exists between a server and an electronic device, and the method includes:
s01, the service access module of the server performs parameter configuration of service abnormality analysis.
The service access module may preset service information included in the electronic devices corresponding to different models and different system versions respectively. For example, the services included in the electronic device of the model 1 system version a include services such as bullet cards, flight reminders, meeting reminders, express reminders, and the like. The service access module can preset the service information of each service corresponding to the electronic equipment of the model 1 system version A. The service information may include a node of the service, a service event of each node, operation data of each node, and the like. For example, the service is a bullet card, and the nodes of the service include, for example, the respective nodes referred to in fig. 2. Illustratively, the node is a card-out node, and the corresponding service event is outputting the subway card in any form. The subway card is output in any form, including outputting the subway card in a negative one-screen card set, outputting the subway card in a flick form, outputting the subway card in a suspension capsule form and the like. Accordingly, the operation data of the node may include an output subway card or an output subway card.
The service access module may also configure other parameters involved in the process of analysis of the service anomalies. For example, the service access module may determine the point location mask rules for each service. The point location mask rules include unnecessary point locations in each service. In some embodiments, the points for which no buried point data needs to be collected include at least two types of points. One of which is a preset point location that is not important for business anomaly analysis. The embedded point data of the point location has little effect on the business anomaly analysis, so the embedded point data can be not collected even though the embedded point location is arranged, thereby reducing the data processing quantity and cost. And the other is a point position set in other scenes. Other scenes are different from the business anomaly analysis scene, the other scenes are used for burying point setting for the business in the business anomaly analysis scene, and the server obtains burying point data of the point positions of the business required by the business anomaly analysis scene in the other scenes. For example, other scenarios may be scenarios such as a server making predictions of user behavior of an electronic device. And assuming that the business anomaly analysis scene and the user behavior prediction scene need to acquire buried point data of the bullet subway card business. When predicting the user behavior of the electronic equipment, the server acquires the buried point data of the business of playing the subway card, and the buried point data comprise buried point data of some points needed in a business anomaly analysis scene. In this case, when the server performs the business anomaly analysis, the buried point data of the required point can be extracted from the buried point data acquired from the scene of the user behavior prediction. The point positions are preset in the point position shielding rule, after the server sends the point position shielding rule to the electronic equipment, the electronic equipment does not collect the buried point data of the point positions configured in the point position shielding rule according to the point position shielding rule, so that the problems of repeated collection and repeated uploading of the electronic equipment are avoided, and the redundancy of data collection and the redundancy of data quantity are reduced.
In some embodiments, the service access module may also preset sampling weights of different services. The sampling weight can represent the acquisition tendency of buried point data of the service. For example, the sampling weight of a new service or an important service (such as service a, service B and service C) in a plurality of services is increased, so that the buried point data of the service a, the service B and the service C are collected preferentially. Or adding sampling weights of new or important services (such as service A, service B and service C) in the plurality of services, wherein the acquisition frequency of the buried point data of the service A, the service B and the service C is higher than that of the buried point data of the service D and the service E. Or increasing sampling weight of new or important service (such as service A, service B and service C) in the plurality of services, wherein the sampling proportion of the embedded point data of the service A, the service B and the service C is higher than that of the embedded point data of the service D and the service E. Or increasing the sampling weight of a service (such as service A) with high failure rate in a plurality of services, and preferentially collecting buried point data of the service A. Wherein the failure rate characterizes the probability of abnormal business occurrence.
In some embodiments, the service access module may further preset sampling weights of different users. Wherein the sampling weights may characterize the propensity of a user to collect buried point data of the service. For example, the sampling weight of the internal staff is increased, and the buried point data of the business in the electronic equipment of the internal staff is preferentially acquired. For example, adding the sampling weight of the new user, preferentially acquiring the buried point data of the service in the electronic device of the new user, and so on.
In some embodiments, the service access module may preset a buried point setting of the point location in the service flow. Illustratively, the node of the service may be set as a point location of the buried point; part of nodes can be selected from the service nodes as the point positions of the buried point setting. It is understood that the point location of the buried point may include all or part of the nodes of the traffic.
The service access module can also preset analysis parameter configuration related to the process of service abnormality analysis. The analysis parameter configuration refers to parameter configuration related to different layers of analysis of buried data of a service. For example, a lake outlet channel corresponding to analysis of different layers of buried point data of a service can be preset in the service access module; the service access module can preset the lake-out proportion of the buried point data of the service, and the like. Illustratively, presetting a lake channel 1 for index calculation of buried point data of a service; presetting a lake outlet channel 2 for configuration synchronization of buried point data of a service; presetting a lake channel 3 for complementary point calculation of buried point data of a service; the lake-out channel 4 is preset for characteristic analysis of buried point data of the business, and the like.
The buried data of the executed point location related to the service execution once forms a service data stream, and the index calculation refers to counting the execution times of each point location in the service data stream under the same service traceID. The more execution times of the point location, the greater the probability of normal execution of the point location. The fewer the execution times of the point location, the greater the probability that the point location is abnormally executed or not executed. That is, when the execution times of the point location is smaller than the preset first threshold, the point location is used as the business index data. The preset first threshold value is smaller than N times of the number of service data flows included in the service, and N is a number larger than 0 and smaller than 1. For example, the number of service data flows included in the service a acquired by the server is 50. N is 0.5. If the execution number of the point location 1 is smaller than the preset first threshold 25, for example, the execution number of the point location 1 is 5, the abnormal probability of the point location 1 is considered to be larger, and the point location 1 is used as the service index data. Illustratively, the number of executions of a point location is significantly less than the number of executions of other point locations, indicating a greater probability that the point location is abnormally executed or not executed. That is, for example, the first point location and the second point location are adjacent points, the first execution frequency of the first point location is greater than the second execution frequency of the second point location, and if the difference between the first execution frequency and the second execution frequency is greater than the preset second threshold, the point location is used as the business index data. The preset second threshold value is larger than M times of the number of service data flows included in the service, and M is a number larger than 0 and smaller than 1. For example, the number of service data flows included in the service a acquired by the server is 50. N is 0.8. If the first execution count of the point bit 1 is 48, the second execution count of the point bit 2 is 5. The difference between the execution times of the point location 1 and the point location 2 is 43 and is larger than the preset second threshold 40. Then the abnormal probability of the point location 2 is considered to be larger, and the point location 2 is used as business index data.
The configuration synchronization refers to synchronizing the data out of the lake proportion of each service traceID for the buried point data of all the services received at this time. For example, the out-lake ratio is 80%, and business anomaly analysis is performed for extracting buried data of 80% of businesses from each business traceID.
The point filling calculation refers to the point filling operation of the point filling data for the point filling point positions in each service data stream. The point filling point positions refer to points where buried point data are not collected. For example, the mask points configured in the point mask rule may be. The server performs the point filling operation of the point filling data, which can be that the server performs the point filling operation of corresponding point filling data from the embedded point data corresponding to the same service uploaded by the electronic equipment received under other scenes; the server may perform the data point filling operation from the preset point filling data of the same service.
The feature analysis refers to feature matching of the service data stream with a plurality of preset reference data to obtain an abnormal label corresponding to the service data stream. The preset plurality of reference data come from the server characteristic management module. For example, the feature management information in the feature management module may include a plurality of preset reference data. The preset plurality of reference data may include a plurality of abnormal reference data, each abnormal reference data corresponds to an abnormal tag, and the abnormal tags corresponding to different abnormal reference data are different. The server can perform feature matching on buried point data of each point in the service data stream and a plurality of reference data, and can obtain an abnormal label of each point. For example, the service is executed to the card-out node, and buried point data of the card-out node is generated. And matching the embedded point data of the card-out node with parameter data of corresponding service and corresponding point positions in a plurality of preset reference data, and determining the reference data corresponding to the embedded point data of the current card-out node, thereby determining that the abnormal label of the reference data is the abnormal label of the card-out node. The feature analysis may also be to match an anomaly tag corresponding to the number according to the number of points included in the service data stream. For example, the feature management module includes anomaly tags corresponding to different point positions of each service, and the server can directly determine the anomaly tags corresponding to the point positions of the service data stream according to the point positions of the service data stream, and so on.
The service access module can also be provided with a visual interface which is communicated with the display module. The data to be displayed can be displayed through the display module through the visual interface.
S02, a rule calculation module of the server calculates rules and determines relevant rules of business anomaly analysis.
The related rules comprise a service embedded point rule, a service reporting rule, a service sampling rule, a query frequency, a point location shielding rule, an abnormal data cleaning rule and the like.
Illustratively, the service burial point rule refers to the burial point setting of the point location for the service node of each service preset in the service access module. For example, the service burial point rule may characterize each node of the service as a point location; or the service burial point rule may characterize a portion of the nodes in the service as points.
The service reporting rule refers to the setting of the service requiring buried data acquisition. For example, based on parameters such as the type of the electronic equipment, the type of the equipment, the version of the equipment system, the type of the service and the like, determining that the service corresponding to the type of the preset electronic equipment needs to collect buried point data; or determining that the service corresponding to the preset equipment model needs to acquire buried point data; or determining that the service corresponding to the system version of the preset equipment needs to collect buried point data; or determining that the service corresponding to the type of the preset equipment system needs to acquire buried point data; or determining that the corresponding service corresponding to the preset service type needs to collect buried point data. If the system version of the electronic equipment is not the preset equipment system version, reporting of the embedded point data of the service is not performed.
The service sampling rule refers to setting of sampling frequency, sampling proportion and the like for the service requiring acquisition of the buried data.
The query frequency refers to the frequency at which the electronic device sends a query request to the server. The query request is used for acquiring preset relevant rules generated by the server. For example, the query frequency may be set such that the electronic device sends query requests at regular daily times, e.g., the electronic device sends query requests to the server at 0 o' clock per day. Or the electronic device periodically sends a query request to the server, for example, the electronic device sends the query request to the server every 48 hours.
The point location shielding rule refers to a point location, which is related to the service in the service anomaly analysis scene and does not need to collect buried data, of the service which is preset by the service access module according to the point location of the service and the point location of the service which is set in other scenes by the server.
The abnormal data cleaning rule refers to a relevant setting for screening abnormal operation data and normal operation data for the obtained buried data of the service.
In some embodiments, the server needs to perform S01-S02 for parameter and rule updates when generating new electronic device traffic, or when generating version updates for electronic device traffic. Or the server may update the parameter configuration and related calculation rules for each service in the electronic device based on the historical analysis results. When the server updates the rule once, the updated relevant rule can be sent to each electronic device according to the address of each electronic device. Or the electronic device may send a query request to the server to obtain the latest relevant rule.
If the service of the electronic device is not changed or the server does not need to update the parameters and rules of the service, S01-S02 are not executed.
For electronic devices, the electronic device needs to acquire the relevant rule before executing the buried data of the acquisition service. Taking the example of the electronic device sending a query request to the server at regular time, the related rule is obtained for illustration.
S03, the electronic equipment sends a query request to the server to acquire preset relevant rules.
In this embodiment, the electronic device may actively send a query request to the server for the first time to obtain a preset relevant rule.
For example, the preset correlation rule includes a query frequency, and the dimension SDK of the electronic device may send a query request to the server according to the query frequency to obtain the latest preset correlation rule.
For example, the preset relevant rules include a service embedded point rule, and the dimension measurement SDK may collect embedded point data of the point location related to the service according to the service embedded point rule.
For example, the preset relevant rules include a service reporting rule, and the dimension measurement SDK may determine a service that needs to perform buried point data acquisition according to the service reporting rule, so as to perform buried point data acquisition.
For example, the preset correlation rule includes a service sampling rule, and the dimension measurement SDK may obtain buried point data of the service under the sampling proportion according to the service sampling rule.
For example, the preset relevant rules include a point location shielding rule, and the dimension measurement SDK may determine, according to the data shielding rule, a point location in each service where buried point data does not need to be collected.
For example, the preset relevant rules include an abnormal data cleaning rule, and the dimension measurement SDK can perform data processing on the collected buried point data of the service according to the abnormal data cleaning rule, so that abnormal data in the buried point data is sent to the server for service abnormal analysis.
After acquiring the preset correlation rule, the electronic device can send a query request to the server at regular time according to the query frequency in the subsequent interaction process with the server so as to acquire the latest preset correlation rule.
Alternatively, the electronic device may store the acquired correlation rule in a designated storage space. For example, the storage space may be in a database, memory, or register.
In some embodiments, if the electronic device has acquired the preset correlation rule, the electronic device may not execute the S01-S03, directly respond to the operation of the service module based on the existing correlation rule, and collect the buried point data of the service. Or in other scenarios, the preset relevant rule may be a rule preset locally for the electronic device. In this case, the electronic device does not need to acquire the preset correlation rule from the server, and S01-S03 are not executed. The electronic equipment directly responds to the operation of the service module to collect the buried point data of the service.
S04, responding to triggering operation of the service module, and acquiring buried point data of the service by the dimension SDK according to a preset service buried point rule.
In the execution process of the service module, when the service module executes to one point location, the service module can report the embedded point data of the point location to the dimension test SDK. Refer to the flow numbered 1 in fig. 7. Illustratively, the buried point data may include information such as a point location identifier, a point location event, operation data corresponding to a point location, and the like. The point location identifier may be a point location ID, which may be composed of numbers and/or characters, the length of the point location ID is a preset length, and the point location identifier is a unique identifier of the point location. A point location event refers to information describing an action performed by a point location. For example, the point location is a card-out node, and the point location event may include outputting a subway card in a card-flicking form; correspondingly, the operation data corresponding to the point location may include outputting the subway card in a card ejection form at the time point XXX. Optionally, the operation data corresponding to the point location may further include that the subway card is not output in the period XXXX. In some embodiments, the service module may return the reason for the non-output subway card to the dimension SDK, where the dimension SDK obtains the abnormal reason corresponding to the non-output subway card.
And each time the service is executed, the dimension measurement SDK can acquire buried point data of each point location according to the point location set by the service, and all buried point data related to the point location of the service is executed once, so that a service data stream can be formed.
In this embodiment, the dimension SDK may directly package the buried data reported by the service module and send the buried data to the server for service exception analysis (exception feature analysis). However, if the buried point data of all the services of the service module are processed and transmitted to the server for service anomaly analysis, the data volume is large, and the transmission cost and the data processing cost are high.
In some embodiments, the dimension SDK may also perform:
S05, the maintenance SDK receives the embedded point data reported by the service module, and acquires the embedded point data of the target service conforming to the preset service reporting rule.
The preset service reporting rule may be obtained from a server (e.g., the service reporting rule generated by the server in S02), or may be a rule preset locally for the electronic device. The preset service reporting rule comprises a service identifier of a target service needing to collect buried point data.
For example, refer to the flow numbered 2 in fig. 7. The service identity may be traceID of the service. One service identity may correspond to one or more service data flows. If traceID of the service corresponding to the embedded point data reported by the service module does not exist in the service identifier included in the preset service reporting rule, the embedded point data is considered to be not in accordance with the preset service reporting rule, the service corresponding to the embedded point data is not the service requiring acquisition of the embedded point data, and the embedded point data is discarded. If traceID of the service corresponding to the embedded point data reported by the current service module exists in the service identifier contained in the preset service reporting rule, the embedded point data is considered to be in accordance with the preset service reporting rule, the service corresponding to the embedded point data is the target service requiring acquisition of the embedded point data, and the embedded point data is acquired.
The preset service reporting rule comprises a designated service identifier which is determined based on parameters such as the type of the electronic equipment, the type of the equipment, the version of the equipment system, the type of the service and the like, and the buried point data corresponding to the designated service identifier is collected based on the preset service reporting rule, which is equivalent to the service screening of the buried point data of all service modules, so that the processing amount of the quantity of the service anomaly analysis can be reduced.
In some embodiments, the service module and the dimension measurement SDK may perform buried point data transmission through a preset reporting interface.
S06, the dimension SDK performs first data processing on the buried point data of the target service which is acquired to meet the preset service reporting rule, and acquires the first buried point data.
The first data processing refers to data verification, data cleaning, data format conversion and the like before buried data are stored in a database.
For example, refer to the flow numbered 3 in fig. 7. When the dimension SDK receives buried point data of one point position of the target service, the buried point data of the point position is subjected to data verification. The data verification is a verification operation for ensuring the validity and the integrity of the data. Illustratively, the data verification includes a validity check, a length check, and an integrity check of the buried data. The validity check may include checking whether the format of the data meets the requirement, checking whether there are unreasonable characters in the data content, whether the length of the buried point data meets the preset length requirement, and so on. Generally, the length of the buried point data should be less than or equal to a predetermined maximum length. For example, the preset maximum length of buried point data of one dot is set to 20 characters. If the length of the buried point data 1 of the point location 1 is 19 characters, the buried point data 1 is considered to be in accordance with the length check. If the length of the buried data 2 corresponding to the point location 2 is 21 characters, the buried data 2 is considered to be not in accordance with the length check.
And carrying out data verification on the buried point data by the dimension test SDK, deleting the buried point data which does not meet the data verification requirement, and reserving the buried point data which meets the data verification requirement. For example, the dimension SDK may delete buried data 2 of point location 2.
Alternatively, the dimension SDK may perform a point-to-point mask check on the buried data that meets the data check, for example, refer to the flow number 4 in fig. 7. The dimension measurement SDK deletes the buried point data corresponding to the shielding point positions related in the preset point position shielding rule, and reserves the buried point data of the non-shielding point positions. For example, if the point location 3 is a mask point location included in the point location mask rule, the buried point data 3 of the point location 3 is deleted.
After the above data verification and the point location mask verification are performed on the buried data of each point location, optionally, the dimension SDK may further determine whether the buried data of all the points where the service is executed once has been obtained in an accumulated manner. For example, refer to the flow numbered 5 in fig. 7. If the buried point data of all the points where the primary service is executed are acquired, determining the buried point data corresponding to the primary service as a piece of service buried point data by the dimension test SDK. The service burial point data is subjected to data serialization processing, for example, refer to a flow of number 6 in fig. 7. And stores the serialized service buried data, illustratively, with reference to the flow of fig. 7, numbered 7. The dimension measurement SDK may receive a plurality of service data streams of a plurality of services, and the dimension measurement SDK may perform first data processing on the buried point data corresponding to each service data stream to obtain first buried point data corresponding to the plurality of service data streams.
S07, the dimension SDK stores the first buried data into a database.
The databases may be different types of databases such as SQLite, mysql, etc. The dimension test SDK can perform data serialization processing on the business buried data according to the requirements of databases of different types on the data storage format, and store the first buried data meeting the requirements of the database data storage format into the database. For example, refer to the flow numbered 8 in fig. 7.
In some embodiments, the dimension SDK may initialize a table base of the database according to the service related in the service module, and determine parameters such as a data storage format of the database. For example, refer to the flow numbered 9 in fig. 7. The dimension test SDK can also perform duplicate removal detection on the first buried data when the first buried data is stored in the database, and delete repeated buried data so as to reduce the waste of redundant data on the storage space of the database.
S08, the dimension measuring SDK acquires first buried point data from the database, performs second data processing on the buried point data, and acquires second buried point data.
The second data processing refers to data processing before the first buried data is sent to the server after being sent out of the database. Illustratively, the second data processing includes data extraction, data deserialization, point location aggregation, rule matching, anomaly cleaning, traffic aggregation, data compression encryption, and the like.
Illustratively, data extraction refers to the wizard SDK reading data from a database. For example, reference is made to the flow numbered 10 in fig. 7. And the dimension measuring SDK reads the first buried point data pair from the database and the data after the data serialization processing. Thus, the dimension SDK may perform data deserialization processing on the read buried data, and convert the data in a format required by the database (e.g., a key value pair format) into data in a service data stream format (e.g., a character string format). For example, refer to the flow numbered 11 in fig. 7.
After the data is deserialized, the dimension SDK may perform a point location aggregation process on the obtained buried data. For example, refer to the flow numbered 12 in fig. 7. The point location aggregation refers to that the dimension measurement SDK obtains point location buried data of points belonging to the same service data stream, and forms the point location buried data of a plurality of points into a finished service data stream according to the execution sequence of the points.
Further, the dimension SDK may perform rule matching and anomaly cleaning for each service data flow. For example, refer to the flow numbered 13 in fig. 7. The maintenance SDK performs rule matching, namely, each service data stream is matched with the data stream which is normally executed by the corresponding service, and if the service data stream is matched with the data stream which is normally executed by the service, the service data stream is determined to be a normal service data stream; the buried point data in the service data stream is normal operation data. For example, if the traffic data stream 1 is uniform with the point location through which the corresponding normal data stream passes, the buried point data of the point location, and the like, the traffic data stream 1 is considered to be the normal data stream, and the buried point data of the point location in the traffic data stream 1 is considered to be the normal operation data. If the service data stream is not matched with the data stream normally executed by the service, determining that the service data stream is an abnormal service data stream; the buried point data in the service data stream is abnormal operation data. For example, when the traffic data stream 1 is consistent with the point location and/or the buried point data of the point location through which the corresponding normal data stream passes, the traffic data stream 1 is considered to be an abnormal data stream, and the buried point data of the point location in the traffic data stream 1 is considered to be abnormal operation data. Each traffic data stream can be identified as either an abnormal data stream or a normal data stream by rule matching.
The maintenance of the SDK for abnormal cleaning refers to deleting normal operation data and reserving abnormal operation data for business abnormality analysis. Therefore, the second buried point data after the second data processing is the abnormal data, and the abnormal data includes abnormal business data streams of each business.
After obtaining the service data stream after the normal data stream is removed, the current service data stream comprises an abnormal data stream. The dimension SDK may aggregate traffic for the abnormal data stream. For example, reference is made to the flow of fig. 7 numbered 14. And taking the service as a unit, attributing the abnormal data flow belonging to the same service as a class, and forming a piece of data corresponding to the service. Each service traceID includes at least one abnormal data stream.
Further, the abnormal data stream may be subjected to service aggregation in units of a collection period and traceID. For example, refer to the flow of fig. 7 numbered 15. For example, 2 periods of the period are collected, and for the abnormal data stream of each period, service aggregation is performed by traceID to obtain a data stream 1 corresponding to a service 1 (traceID 1) corresponding to the period 1 and a data stream 2 corresponding to a service 2 (traceID 2); one data stream 3 corresponding to the service 1 (traceID 1) corresponding to the period 2.
The point location aggregation and the service aggregation of the embedded data by the dimension measurement SDK are both performed on the embedded data in a structural conversion way, so that the embedded data form a data stream form, and the storage space redundancy caused by a data structure is reduced.
After the dimension SDK obtains a plurality of data streams in units of periods and services traceID, the dimension SDK performs data compression and data encryption on all the data streams. For example, refer to the flow numbered 16 in fig. 7. The data compression may be converting the format of the data stream into zip, RAR, etc. In this embodiment, the dimension SDK may encrypt data by using a hash encryption, a symmetric encryption, an asymmetric encryption, a Base64 encryption, or the like.
In some embodiments, the data flows of the plurality of service identities may be represented as:
["traceId1","traceId2",…];
the data flows of names corresponding to the plurality of service identifiers can be expressed as:
[["traceId1-name1","traceId1-name2",…],
["traceId2-name1","traceId2-name2",…],
["traceId3-name1","traceId1-name3",…]]。
the data flow of the information corresponding to the plurality of service identifiers can be expressed as:
[["traceId1-factor1","traceId1-factor2",…],
["traceId2-factor1","traceId2-factor2",…],
["traceId3-factor1","traceId1-factor3",…]]。
the data streams of the compression ratios corresponding to the plurality of service identifiers can be expressed as:
[["traceId1-extend1","traceId1-extend2"],
["traceId2-extend1","traceId2-extend2"],
["traceId3-extend1","traceId1-extend2"]]。
in some embodiments, after the dimension SDK obtains the embedded data once from the database, the database clears the embedded data obtained this time, and releases the storage space in the database in time.
S09, the dimension SDK sends the second embedded data to the server.
In this embodiment, the dimension SDK may send the second buried data to the big data platform of the server through a pre-established transmission channel. The second embedded point data is the abnormal data (first abnormal data) to be sent.
Before the electronic device sends the second embedded data, a communication connection with the server can be established according to the address of the server. The server can establish a binding relationship with the electronic device according to the identification of the electronic device.
In some embodiments, the electronic device may periodically send the second buried point data to the server. Or the electronic device may send the second buried point data to the server according to a preset sending timing. It will be appreciated that the electronic device may perform the generated buried point data on the acquisition service module in real time, but that the first data processing and the second data processing for the buried point data may be performed during non-use of the electronic device. For example, when the electronic device is a mobile phone, the first data processing and the second data processing of the buried point data can be performed during off-screen charging, so that the power consumption of the data processing on the mobile phone is reduced.
And S10, performing third data processing on the second buried point data by the big data platform of the server to acquire third buried point data.
The third data processing is data preprocessing before business anomaly analysis is carried out on the buried data.
Illustratively, the data preprocessing includes the big data platform storing the received service data; the big data platform performs data extraction on the stored business data; the large data platform decompresses the extracted business data; the big data platform splits the decompressed business data; and the big data platform performs data analysis on the split business data.
The third data processing is directed to the abnormal data, and the third buried point data is the abnormal data (second abnormal data) after the third data processing.
In some embodiments, the big data platform may store the received second embedded data to a designated storage space, such as to a database, a designated file, a cloud, and so forth.
And the big data platform acquires second buried point data from the storage space, and performs data extraction on the second buried point data. Data extraction refers to acquiring buried point data related to business anomaly analysis. The data sent by the electronic device may have buried data required by other scenes, and for carrying out the business anomaly analysis, the effective value of the other buried data is smaller, so that the other buried data can be filtered, and only the buried data related to the business anomaly analysis is reserved for carrying out the business anomaly analysis.
After the big data platform filters out irrelevant buried point data, the data decompression can be carried out on relevant buried point data, and decompressed buried point data is obtained. The decompressed buried point data may include buried point data for a plurality of cycles and a plurality of services. The big data platform can take the service identifier as a unit, split the decompressed buried point data into one or more service data streams under the same traceID, and use the service data streams as third buried point data.
After the third buried data is acquired, the big data platform performs business anomaly analysis on the third buried data. Since the electronic devices transmit the abnormal data streams, the third embedded data streams are also abnormal data streams (abnormal traffic data streams).
In some embodiments, the big data platform performs business anomaly analysis on the third buried data, and may obtain the buried data of the point by determining the last executed point of the anomaly data stream. And matching the buried point data with the business abnormal characteristics so as to obtain an abnormal label of the buried point data. Or in some embodiments, the big data platform performs business exception analysis on the third buried point data, so that the buried point data of the last point position which is normally executed can be obtained, and the exception reason of the point position which is failed to be executed or is not executed is determined based on the buried point data of the point position which is normally executed, so that the exception label of the buried point data is obtained.
In some embodiments, the business anomaly analysis may include, illustratively, a multidimensional analysis of the third buried point data, including, in particular:
s11, the big data platform performs service index synchronization on the third buried data to obtain service index data.
For example, refer to the flow numbered 1 in fig. 8. The third buried point data is data in traceID units. The service index synchronization includes determining points involved in the service flow as service indexes, thereby performing service index calculation. The service index calculation process comprises counting the execution times of each point position in each service data stream by the big data platform aiming at a plurality of service data streams corresponding to each traceID. The execution times are suddenly reduced, and the probability that the point is an abnormal point is high. That is, for example, the first point location and the second point location are adjacent points, the first execution frequency of the first point location is greater than the second execution frequency of the second point location, and if the difference between the first execution frequency and the second execution frequency is greater than the preset second threshold, the point location is used as the business index data. The preset second threshold value is larger than M times of the number of service data flows included in the service, and M is a number larger than 0 and smaller than 1. For example, in the multiple abnormal service data flows corresponding to traceID1, the service data flow 1 includes a point location 1, a point location 2, a point location 3, and a point location 4; the service data flow 2 comprises a point position 1, a point position 2 and a point position 3; the service data stream 3 comprises a point location 1, a point location 2 and a point location 3; the service data stream 4 comprises a point location 1, a point location 2 and a point location 3. In the 4 service data flows, the execution times of the point location 1 is 4, the execution times of the point location 2 is 4, the execution times of the point location 3 is 4, and the execution times of the point location 4 is 1. The execution times of the point location 4 are suddenly reduced, which indicates that the point location 4 has a high probability of abnormality for a plurality of abnormal business data streams corresponding to traceID < 1 >. Point 4 may be a failure point that causes an exception to the execution of the service.
Or index calculation refers to counting the execution times of each point in the service data stream under the same service traceID. The more execution times of the point location, the greater the probability of normal execution of the point location. The fewer the execution times of the point location, the greater the probability that the point location is abnormally executed or not executed. That is, when the execution times of the point location is smaller than the preset first threshold, the point location is used as the business index data. The preset first threshold value is smaller than N times of the number of service data flows included in the service, and N is a number larger than 0 and smaller than 1. For example, the number of service data flows included in the service a acquired by the server is 50. N is 0.5. If the execution number of the point location 1 is smaller than the preset first threshold 25, for example, the execution number of the point location 1 is 5, the abnormal probability of the point location 1 is considered to be larger, and the point location 1 is used as the service index data.
And calculating the service index of the service data flow corresponding to each traceID by using the big data platform, wherein the big data platform takes the points as index data corresponding to traceID. It is understood that the index data corresponding to traceID may include one or more points.
And S12, carrying out service configuration synchronization on the third buried data by the big data platform.
For example, refer to the flow numbered 2 in fig. 8. In this embodiment, the third buried point data may include buried point data corresponding to a plurality of different services (different traceID). The service configuration synchronization refers to that a big data platform determines the service to be subjected to the service abnormality analysis currently. For example, the big data platform determines that the service to be subjected to the service anomaly analysis is a service corresponding to traceID. Further, the big data platform may extract traceID the corresponding service data flow to perform the service anomaly analysis.
And S13, carrying out service point filling synchronization on the third buried point data by the big data platform to obtain service point filling data.
After determining that the service to be subjected to the service anomaly analysis is a service corresponding to traceID3 in S12, the service complementary point synchronization is performed on the service corresponding to traceID3, for example, refer to the flow of number 3 in fig. 8. The service point-filling synchronization refers to that the big data platform determines the service needing point-filling processing. Some service nodes of the service do not set a point location, but the buried point data of the point location is needed to perform service anomaly analysis. At this time, it is necessary to perform complementary point calculation, and to acquire and multiplex buried point data of the point from other scenes. Other scenes are different from the business anomaly analysis scenes, the other scenes also carry out buried point setting on the business in the business anomaly analysis scenes, and the large data platform of the server already acquires buried point data in the other scenes. The buried point data in other scenes comprise buried point data required by the business anomaly analysis scene, and the large data platform can acquire and multiplex the buried point data of corresponding points in other scenes. If no service node with no point position exists in the service corresponding to traceID, that is, each service node in the service corresponding to traceID3 has a point position, the service point compensation process is not needed.
In some embodiments, the big data platform may sample the traffic after completing the calculation of the traffic patch. For example, refer to the flow numbered 4 in fig. 8. For example, the big data platform extracts service data streams of some appointed users, or some types of service anomalies, or some types of electronic equipment from service data streams of which the service point-filling calculation is completed, so that the big data platform aggregates the service data streams belonging to the same service identifier according to the service identifier, and service point-filling data is obtained.
The big data platform can perform abnormal feature matching on the service data flow, so that an abnormal label of the service data flow is obtained. Before that, the big data platform may acquire preset abnormal feature information from the feature management module of the traffic abnormality analysis platform.
S14, the big data platform acquires preset abnormal characteristic information from the characteristic management module.
The preset abnormal characteristic information comprises information such as abnormal labels of corresponding service nodes of each service. The abnormal characteristic information may include a plurality of preset reference data. The preset plurality of reference data may include a plurality of abnormal reference data, each abnormal reference data corresponds to an abnormal tag, and the abnormal tags corresponding to different abnormal reference data are different. For example, in a bullet subway card scenario, when a service is executed to a card-out node, the buried point data of the service includes buried point data of a point location from a service start node to the card-out node. The anomaly tags corresponding to the traffic data streams consisting of the buried data at these points may include network anomalies, not clicked cards, not entered into subway pens, etc. Or the abnormal characteristic information may include abnormal labels corresponding to different point positions of each service. The feature management module may count the number of points involved in the execution of the service to the card-out node. For example, if the number of the points included in the service execution to the card-out node is 5, when the service data stream of the service includes buried point data of 5 points, it may be determined that the corresponding abnormal label is an untracked card. For example, after the service is executed to the payment fence node after the card-out node is registered, the service data stream contains buried point data of 6 points, and it can be determined that the corresponding abnormal label is that the payment fence is not registered successfully when the service data stream contains buried point data of 6 points. That is, the feature management module may configure a corresponding anomaly tag for each service node/point location according to anomalies that may be generated by each service node/point location; or the feature management module can also configure corresponding abnormal labels for the number of the points involved in the service data flow. It will be appreciated that the anomaly tags may be pre-labeled tags, which may be empirically set.
Before the abnormal characteristic analysis of the third buried point data is performed, the big data platform can acquire abnormal characteristic information corresponding to the service from the characteristic management module so as to perform matching of abnormal labels of all service data flows in the third buried point data. For example, refer to the flow numbered 5 in fig. 8.
S15, the big data platform performs abnormal feature analysis on the third buried point data to obtain service feature data of the third buried point data.
Wherein, the business feature data refers to an abnormal label. For the same service, the big data platform can acquire abnormal characteristic information from the characteristic management module. Wherein the anomaly signature may characterize the cause of the anomaly. For example, in the bullet subway card service, the abnormal label of the service data stream 1 is "card-out failure".
When the service data stream is a service stream with a single-line serial execution sequence, the abnormal characteristic information can also include information such as abnormal labels corresponding to the number of the points in the service data stream. For example, when the service data stream contains P points, the corresponding exception label 1; the service data flow comprises Q points corresponding to the abnormal labels 2. Wherein P and Q are natural numbers greater than 0 and less than L. L is the total number of points contained in the normal traffic data stream.
The big data platform performs service feature synchronization on each abnormal service data stream in the third buried point data, performs service feature calculation on each abnormal service data stream in the third buried point data based on the abnormal feature information, and obtains an abnormal tag (service feature data) corresponding to each service data stream, for example, refer to a flow numbered 6 in fig. 8. The anomaly tags may characterize the cause of anomalies in the traffic data stream. Illustratively, when a service is handled, for example, the service is a bullet card, and the service traceID is traceID. And the big data platform acquires all the service data streams corresponding to traceID < 1 > in the third buried point data. For example, the service data flows corresponding to traceID a include service data flow 1, service data flow 2, and service data flow 3. The big data platform obtains traceID abnormal characteristic information corresponding to the characteristic management module. And the big data platform matches each business data stream with the abnormal characteristic information corresponding to traceID < 1 >, and determines the abnormal label of each business data stream.
Illustratively, the anomaly characteristic information may include corresponding anomaly tags for each service node/point in the service process. The big data platform matches the buried point data of each point in the service data stream with the operation data of each point in the abnormal characteristic information, and obtains the abnormal label matched with the buried point data in the abnormal characteristic information as the abnormal label of the service data stream.
Illustratively, the anomaly characteristic information may include anomaly tags corresponding to a number of points contained in the traffic data stream. The service data stream 1 is a single-line serial execution sequence service stream. The big data platform determines that the service data stream 1 comprises 5 points, and can query the abnormal labels corresponding to the point number of 5 points of the service data stream from the abnormal characteristic information corresponding to traceID as the abnormal labels of the service data stream 1.
Therefore, the big data platform can obtain the abnormal label corresponding to the business data flow contained in each business in the third buried point data.
In some embodiments, S11, S12, S13, S15 described above may be performed synchronously or asynchronously. In the same step, that is, the big data platform may perform service index calculation, service configuration, service complementary point calculation, and abnormal feature analysis based on the third buried point data, so that relevant information of the third buried point data needs to be synchronized, for example, traceID for synchronously determining a service to be subjected to service abnormality analysis.
The large data platform obtains service index data, service complementary point data and abnormal labels of service data streams of each service, and the data form analysis results of each service. The analysis results may characterize the cause of anomalies in the traffic data stream. For example, the service is a bullet subway card, and the abnormal reason of the service data flow may be card-out failure caused by network abnormality. The server may transmit the analysis result to the construction of the traffic link map of the traffic anomaly analysis platform.
Fig. 10 is a schematic flow diagram of a part of a first stage and a whole second stage of a service data processing method, and in combination with fig. 9 and fig. 10, a service anomaly analysis platform of a server performs construction and display of a service link diagram according to an analysis result, and specifically includes:
S16, the big data platform transmits the analysis result to the business anomaly analysis platform for storage through the lake outlet channel.
For example, refer to the flow numbered 7 in fig. 8. And the big data platform transmits the business index data, the business dotting data and the abnormal labels in the analysis result to the kafka system of the business abnormal analysis platform through the lake outlet channel, and the business index data, the business dotting data and the abnormal labels are stored through the kafka system. For example, refer to the flow numbered 8 in fig. 8. The kafka system is a distributed storage system, and is responsible for transmitting analysis results obtained from a large data platform to a display module to construct a service link diagram corresponding to each service data stream.
Before the display module builds the service link map, the link management module of the service anomaly analysis platform may build a service link map corresponding to each service flow.
S17, a link management module of the business anomaly analysis platform creates a corresponding business link map for each business data stream in the third buried point data.
In some embodiments, for each service, the link management module draws a service link map corresponding to the service according to an execution flow of the service preset in the service access module. Refer to the link management module portion shown in fig. 8. A traffic link map may also be understood as a topology corresponding to a traffic flow. The link management module performs node mapping on the service link structure diagram according to all service nodes and point positions related to the service flow, and creates a service link map corresponding to the service. The service link map contains each service node of the service and a default execution sequence, and does not contain operation data of each service node and buried point data of points involved in the service flow.
If the service flow has point location shielding, point location compensation operation is needed for the point locations related in the service flow.
Optionally, in some embodiments, the link management module may store a service link map corresponding to each service, so that the service link map may be used multiple times, reducing redundant operations in creating the service link map.
S18, the group analysis module of the business anomaly analysis platform performs clustering analysis of business anomalies to obtain clustering analysis results.
In some embodiments, the group analysis module is configured to perform error distribution analysis on the bulk traffic and perform buried point monitoring on the traffic. Refer to the population analysis module section shown in fig. 8. The batch services may be the same type of services, or services executed in the same time period, or services in the same type of equipment, or services under the same version of system operation, and so on.
In some embodiments, the group analysis module may further generate a group analysis result according to preset aggregate tags, statistics of the duty ratios of different tags, statistics of distribution situations of business anomaly problems of all the bound electronic devices, and the like. The group analysis results may be transmitted to a display module to display the group analysis results in a display screen of the server.
For example, the business is a bullet subway card, and the group distribution result may include a duty ratio distribution corresponding to different anomaly tags, for example, the group distribution result may include a duty ratio of card ejection failure, a duty ratio of registered payment fence failure, and the like. Based on the group analysis result, the abnormal probability and abnormal distribution of the service can be better obtained. In this embodiment, cluster analysis is performed on the service anomalies by combining the system results of the service data flows, so that anomaly classification with each service problem can be achieved. Further, the group analysis module may also send the cluster analysis result to the display module, so that the display module displays the cluster analysis result on the display interface.
In some embodiments, the display module of the service anomaly analysis platform may draw a service link map corresponding to each service data stream according to each service data stream in the third buried point data and an analysis result corresponding to each service data stream.
For example, after the display module obtains the analysis result of the third buried point data, the execution sequence, the number of point positions, the operation data of each point position, the abnormal label and other data of each service data stream in the third buried point data are directly drawn and rendered to obtain the service link diagram corresponding to each service data stream.
Optionally, in some embodiments, the display module may also obtain a service link map from the link management module, so as to reduce drawing operations on each service data flow and improve efficiency of generating the service link map.
S19, the display module of the business anomaly analysis platform performs link rendering and node mapping on the business link map corresponding to the business data stream according to each business data stream in the third buried point data and the analysis result corresponding to each business data stream, and generates a business link map of the business data stream.
In some embodiments, the display module may, for each service data stream in the third buried point data, be according to the execution condition of each point in the service data stream. The execution condition of each point location comprises executed or unexecuted, and each node in the service link map is subjected to link rendering according to the execution condition of each point location. For example, the executed point positions in the service data stream are subjected to link rendering in a first mode, and the unexecuted point positions in the service data stream are subjected to link rendering in a second mode, so that the executed or unexecuted nodes in the service link diagram are distinguished. At this time, the service link map includes executed points (nodes) and unexecuted points (nodes). The first mode and the second mode may be different in color, for example, the first mode is a first color (such as green), the second mode is a second color (such as gray), which is not limited in this embodiment. For example, the first and second modes may be different modes of line specification. For example, the first mode is a solid line, and the second mode is a broken line. For example, the first way is to thicken lines, the second way is to plain lines, etc. The embodiment is not limited to the specific implementation manner of the first mode and the second mode.
Further, according to buried point data of each point in the service data stream and an analysis result of the service data stream, node mapping is carried out on the service link map, and assignment is carried out on each point in the service link map, so that a service link map of the service data stream is obtained. At this time, the service link diagram includes buried point data and analysis results of each point (node).
The buried data comprises operation data of the point location, and the analysis result can comprise an abnormal label. And carrying out link rendering and node mapping on each service data stream to obtain an amateur link map corresponding to each service data stream in the third buried data. Reference is made to the display module portion shown in fig. 8.
In some embodiments, the service link graph constructed based on the buried point data of each point in the service data stream and the analysis result of the service data stream may include operation data of each point in the service data stream and an anomaly tag of the service data stream.
In some embodiments, the display module of the server may further analyze the buried data of each point in the service data stream to determine an abnormal identifier of each point. The anomaly identification includes a first value and a second value, wherein the first value indicates that the point location is operating normally and the second value indicates that the node is operating abnormally. Constructing a service link diagram corresponding to the service data flow based on the abnormal identifications of the various points of the service data flow, wherein the obtained service link diagram comprises the various points of the service data flow and the corresponding abnormal identifications.
In some other embodiments, the traffic link map may include one or more data for various points of the traffic data stream. This embodiment is not limited thereto.
S20, the display module displays one or more service link diagrams in the display screen.
And the server receives the selection operation of the user on the service link graphs to be displayed, and displays one or more service link graphs selected by the user in a target interface of the display screen through the display module.
The display module displays the target interface on the display screen. The target interface may include each index data corresponding to an analysis result of the abnormality feature analysis of the abnormality data uploaded by the electronic device by the server. Such as abnormal positive-going profiles, business index data statistics, etc. The target interface comprises options of the service and options of each service data flow under the service. And the display module responds to the selection operation of the user on the options of the target service data flow, and displays a target service link diagram corresponding to the target service data flow in the target interface. Optionally, the display module may display the target service link diagram in a certain display area in the target interface; or the display module can also display the target business link diagram on the new page; or the display module can also display the target business link diagram and the like on a popup window interface of the target interface.
After the display module displays the target service link diagram, the operation data corresponding to a certain point (node) of the service link diagram, the data such as an abnormal label (analysis result) and the like can be displayed in response to the selection operation of the user moving the cursor on the point (node) of the service link diagram. The display module can display operation data and abnormal labels of the nodes on a display interface of the target service link diagram in the forms of popup window, floating window, newly-built page and the like.
Fig. 11 shows a schematic diagram of a service link diagram of a display module displaying a service a in a display interface of a display screen. Wherein the solid line portion represents an executed traffic link in the traffic link graph; the dashed line portion represents an unexecuted traffic link in the traffic link graph. When the cursor moves to the service node 5, data such as operation data of the service node 5 and an anomaly flag (analysis result) are displayed.
In some embodiments, the display module may further store a service link map corresponding to each service in the third buried point data. For example, refer to the flow numbered 9 in fig. 8.
In the present embodiment, the electronic apparatus may acquire the abnormal data from the buried point data generated from the service. The embedded data refer to normal operation data or abnormal operation data generated by a plurality of preset point positions in the execution process of the service. The abnormal data comprises at least one service water flow of the service, and each service data flow consists of abnormal operation data of at least one point location. The electronic device sends the anomaly data to the server. The server performs abnormal characteristic analysis on the abnormal data to obtain an analysis result of each business data stream in the abnormal data. The server may display one or more traffic link graphs. The service link diagram comprises a plurality of points of one service data stream and an analysis result of the service data stream.
By adopting the scheme, because the abnormal data in the buried point data comprises abnormal operation data of each point in the service execution flow, the point has the function of positioning the service execution process. The server performs abnormal characteristic analysis on abnormal operation data of each point in service execution, so that the position of the abnormality in the service execution process can be obtained, and the efficient and accurate positioning of the service abnormality is realized. In addition, the electronic equipment and the server do not need human intervention in the process of processing abnormal data, so that the timeliness of service maintenance is improved under the condition that the user perceives weak or even not perceives the communication cost between the user and the maintenance personnel is reduced; the abnormal nodes are pertinently improved and maintained, and the reliability of service operation in the electronic equipment is improved. The server visually presents the service link diagram comprising the point positions of the service data streams and the analysis results of the service data streams, so that the analysis results of the service abnormality of the electronic equipment are more visual.
It should be noted that, the personal information used in the technical solution of the present application is limited to information that is only agreed by the individual, including but not limited to notifying and reminding the user to read the related user protocol (notification) and signing the protocol (authorization) including the information of the authorized related user before the user uses the function.
In the technical scheme disclosed by the application, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing and the like of the personal information of the user accord with the regulations of related laws and regulations, and the public order is not violated.
Embodiments of the present application also provide a computer readable storage medium, where the computer readable storage medium includes computer instructions, which when executed on an electronic device, cause the electronic device to perform the functions or steps performed by the electronic device 100 in the method embodiments described above; the computer instructions, when executed on a server, cause the server to perform the functions or steps performed by server 200 in the method embodiments described above.
Embodiments of the present application also provide a computer program product which, when run on a computer, causes the computer to perform the functions or steps performed by the electronic device 100 in the method embodiments described above. For example, the computer may be the electronic device 100 described above. The computer instructions, when executed on a server, cause the server to perform the functions or steps performed by server 200 in the method embodiments described above.
It will be apparent to those skilled in the art from this description that, for convenience and brevity of description, only the above-described division of the functional modules is illustrated, and in practical application, the above-described functional allocation may be performed by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to perform all or part of the functions described above.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another apparatus, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and the parts displayed as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed in a plurality of different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a readable storage medium. Based on such understanding, the technical solution of the embodiments of the present application may be essentially or a part contributing to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, including several instructions for causing a device (may be a single-chip microcomputer, a chip or the like) or a processor (processor) to perform all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk, etc.
The foregoing is merely illustrative of specific embodiments of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present application should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (21)
1. A business data processing method is characterized by being applied to a business processing system, wherein the business processing system comprises a server and one or more electronic devices, and one or more businesses are deployed in the electronic devices; the method comprises the following steps:
The electronic equipment acquires abnormal data from the buried point data of the service; the embedded data comprises normal operation data or abnormal operation data generated by a plurality of preset point positions in the execution process of the service, wherein the abnormal data comprises at least one service data stream of the service, and each service data stream consists of abnormal operation data of at least one point position; the point location is used for representing a behavior or an event generated in the service execution process, and one point location is a node for the service execution;
the electronic equipment sends the abnormal data to the server;
The server performs abnormal characteristic analysis on the abnormal data to obtain an analysis result corresponding to each business data stream;
The server constructs a service link diagram of each service data stream according to each service data stream and the analysis result of each service data stream: the server acquires a service link map corresponding to the service data flow; the service link map is used for representing the execution sequence and the number of the points in the service data flow; the server performs link rendering and node mapping on the service link map according to the operation data of each point position of the service data stream and the analysis result of the service data stream to obtain a service link map of the service data stream; the link rendering characterization respectively renders the executed point positions in the service link diagram and the unexecuted point positions in the service link diagram differently; the node mapping characterization maps each point in the service link diagram with each point in the service data flow;
The server displays one or more service link graphs; each service link diagram comprises a plurality of points of one service data flow and an analysis result corresponding to the service data flow.
2. The method of claim 1, wherein the analysis results include business metric data; the server performs abnormal characteristic analysis on the abnormal data to obtain an analysis result corresponding to each service data stream, and the method comprises the following steps:
The server calculates service indexes of each service in the abnormal data by taking the service as a unit to acquire service index data corresponding to each service; the business index data is used for representing at least one point position causing the business abnormality.
3. The method according to claim 2, wherein the server performs service index calculation on each service in the abnormal data by using a service as a unit, and obtains service index data corresponding to each service, including:
the server calculates the execution times of each point location in the service by taking the service as a unit;
The server acquires point positions with execution times smaller than a preset first threshold value as service index data of the service; the preset first threshold value is smaller than N times of the number of service data flows included in the service, and N is a number larger than 0 and smaller than 1.
4. The method according to claim 2, wherein the server performs service index calculation on each service in the abnormal data by using a service as a unit, and obtains service index data corresponding to each service, including:
the server calculates the execution times of each point location in the service by taking the service as a unit;
If the first execution times of the first point location is larger than the second execution times of the second point location, and the difference between the first execution times and the second execution times is larger than a preset second threshold, the second point location is used as service index data of the service; the first point location and the second point location are adjacent points; the preset second threshold is greater than M times of the number of service data flows included in the service, and M is a number greater than 0 and less than 1.
5. The method of any one of claims 1-4, wherein the analysis result comprises an anomaly tag; the server performs abnormal characteristic analysis on the abnormal data to obtain an analysis result corresponding to each service data stream, and the method comprises the following steps:
The server takes the service data stream as a unit, performs abnormal characteristic analysis on each service data stream of each service in the abnormal data, and obtains an abnormal label corresponding to each service data stream; the anomaly tags are used for representing the point positions and the anomaly reasons of the anomaly of each service data stream.
6. The method according to claim 5, wherein the server performs an anomaly feature analysis on each service data stream of each service in the anomaly data by using a service data stream as a unit, and obtains an anomaly tag corresponding to each service data stream, including:
The server takes a service data stream as a unit, and matches buried point data of each point in the service data stream with a plurality of preset reference data to obtain an abnormal label corresponding to the service data stream; the plurality of reference data comprise a plurality of abnormal reference data, each abnormal reference data corresponds to one abnormal label, and the abnormal labels corresponding to different abnormal reference data are different from each other; or alternatively
The server takes the service data stream as a unit to determine the number of point positions in each service data stream;
and matching the abnormal labels corresponding to the number of the point positions in the service data flow based on the number of the point positions.
7. The method according to any one of claims 1 to 4, wherein the analysis result includes service point-complementary data of a service data stream, and the server performs an anomaly characteristic analysis on the anomaly data to obtain an analysis result corresponding to each service data stream, including:
The server takes the service data stream as a unit to determine the point filling point position of each service data stream in the abnormal data; the complementary point positions are positions where buried point data are not collected;
The server acquires the point filling data corresponding to the point filling point positions and performs point filling operation to obtain service point filling data of the service data stream; the point filling data are buried point data which are preset in the server and correspond to the point filling points.
8. The method according to claim 1, wherein the server performs link rendering and node mapping on the service link map according to the operation data of each point location of the service data stream and the analysis result of the service data stream, to obtain a service link map of the service data stream, including:
The server performs link rendering on the executed point positions in the service data flow in the service link map in a first mode, and performs link rendering on the unexecuted point positions in the service data flow in a second mode to obtain a service link map after link rendering;
and the server carries out assignment on each point bit in the service link map after the link rendering according to the operation data of each point bit in the service data stream and the analysis result of the service data stream, so as to obtain a service link map of the service data stream.
9. The method of any of claims 1-4, wherein the server displaying one or more of the traffic link graphs comprises:
the server displays a target interface; the target interface comprises options for the service and options for service data flow in the service;
the server responds to the selection operation of the user on the options of the target service data flow in the target interface, and displays a target service link diagram corresponding to the target service data flow; the target service data stream is one or more service data streams contained in the service.
10. The method of claim 9, wherein after said displaying the target traffic link map corresponding to the target traffic data stream, the method further comprises:
and the server responds to the selection operation of the user on the target point location in the target service link diagram, and displays the operation data and the abnormal label of the target point location.
11. The method of claim 1, wherein the electronic device obtaining anomaly data from buried point data of the service comprises:
The electronic equipment acquires buried point data generated by executing the service;
The electronic equipment performs first data processing on the buried data, and stores the first buried data after the first data processing into a database; the first data processing includes data checksum data serialization; the data verification comprises a length and integrity verification for the buried point data; the data serialization comprises data format conversion of the buried point data;
The electronic equipment reads the first buried point data from the database, and performs second data processing on the first buried point data to obtain first abnormal data; the second data processing comprises data deserialization and exception cleaning; the data deserialization comprises data format reverse conversion of the first buried point data, the abnormal cleaning comprises deleting normal operation data in the first buried point data, and retaining the abnormal operation data in the first buried point data;
the electronic device sending the anomaly data to the server, comprising:
The electronic device sends the first abnormal data to the server.
12. The method of claim 11, wherein the electronic device obtaining buried point data generated by executing the service comprises:
the electronic equipment determines the point positions configured by each service according to a preset service embedded point rule; the preset service embedded point rule comprises configuration information of point positions of each service;
and the electronic equipment acquires buried point data generated when the business is executed to each point position.
13. The method of claim 11, wherein the electronic device obtaining buried point data generated by executing the service comprises:
the electronic equipment determines shielding point positions corresponding to the businesses according to a preset point position shielding rule; the shielding point positions are point positions which do not need to collect buried point data in the point positions of the service;
the electronic device acquires buried point data of each point except the shielding point where the service is executed.
14. The method of claim 11, wherein the electronic device obtaining buried point data generated by executing the service comprises:
The electronic equipment determines a target service according to a preset service reporting rule; the target service is a service in a plurality of services;
and the electronic equipment acquires buried point data generated by executing the target service.
15. The method of any of claims 1-4, wherein the electronic device sending the anomaly data to the server comprises:
The electronic equipment carries out data aggregation on the abnormal data to form a data packet corresponding to the abnormal data;
And the electronic equipment sends the data packet to the server.
16. The method of claim 15, wherein the method further comprises:
The server receives the data packet sent by the electronic equipment;
the server performs third data processing on the data packet to obtain second abnormal data after the third data processing; the third data processing comprises data extraction and data splitting; the data splitting includes splitting the data packet into service data flows in service units; the data extraction comprises extracting a service data stream with a preset proportion from the split service data stream;
The server performs abnormal feature analysis on the abnormal data, and the method comprises the following steps:
And the server performs abnormal characteristic analysis on the second abnormal data.
17. The method according to any one of claims 11-14, further comprising:
the server updates a preset service embedded point rule of the electronic equipment; and/or the number of the groups of groups,
The server updates a preset point location shielding rule of the electronic equipment; and/or the number of the groups of groups,
And the server updates preset business reporting rules of the electronic equipment.
18. A business data processing system, wherein the business data processing system comprises an electronic device and a server; the electronic equipment comprises a maintenance Software Development Kit (SDK), and one or more services are deployed in the electronic equipment; the server comprises a big data platform and a business anomaly analysis platform;
The maintenance software development kit SDK acquires abnormal data from the buried point data of the one or more businesses; the buried data comprises normal operation data or abnormal operation data generated by a plurality of preset point positions in the execution process of the service; the abnormal data comprises at least one business data stream of the business, and each business data stream consists of abnormal operation data of at least one point location; the point location is used for representing a behavior or an event generated in the service execution process, and one point location is a node for the service execution;
The maintenance software development kit SDK sends the abnormal data to the server;
the big data platform performs abnormal characteristic analysis on the abnormal data to obtain an analysis result corresponding to each business data stream;
The business anomaly analysis platform constructs a business link diagram of each business data stream according to each business data stream and the analysis result of each business data stream: the business anomaly analysis platform acquires a business link map corresponding to the business data flow; the service link map is used for representing the execution sequence and the number of the points in the service data flow; according to the operation data of each point position of the service data stream and the analysis result of the service data stream, carrying out link rendering and node mapping on the service link map to obtain a service link map of the service data stream; the link rendering characterization respectively renders the executed point positions in the service link diagram and the unexecuted point positions in the service link diagram differently; the node mapping characterization maps each point in the service link diagram with each point in the service data flow;
The business anomaly analysis platform displays one or more business link graphs; each service link diagram comprises a plurality of points of one service data flow and an analysis result corresponding to the service data flow.
19. An apparatus comprising a memory, a processor, and a computer program stored on the memory, the apparatus comprising an electronic device and a server;
The processor executes a computer program to implement the steps of the method of any one of claims 1-17.
20. A computer readable storage medium having stored thereon a computer program/instruction, which when executed by a processor, implements the steps of the method of any of claims 1-17.
21. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the method of any of claims 1-17.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202410174604.1A CN117724891B (en) | 2024-02-07 | 2024-02-07 | Service data processing method and service data processing system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202410174604.1A CN117724891B (en) | 2024-02-07 | 2024-02-07 | Service data processing method and service data processing system |
Publications (2)
Publication Number | Publication Date |
---|---|
CN117724891A CN117724891A (en) | 2024-03-19 |
CN117724891B true CN117724891B (en) | 2024-06-28 |
Family
ID=90207355
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202410174604.1A Active CN117724891B (en) | 2024-02-07 | 2024-02-07 | Service data processing method and service data processing system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN117724891B (en) |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110912738A (en) * | 2019-11-19 | 2020-03-24 | 支付宝(杭州)信息技术有限公司 | Business exception processing method, device, platform and electronic equipment |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FR2720530B1 (en) * | 1994-05-27 | 1996-08-23 | Sextant Avionique | Device for optimizing the performance of a processor. |
CA2214511A1 (en) * | 1995-03-27 | 1996-10-03 | Donald K. Forest | Method of and apparatus for data entry |
JP2008033849A (en) * | 2006-08-01 | 2008-02-14 | Hitachi Ltd | Fault analysis system |
CN101562540B (en) * | 2009-05-08 | 2011-11-02 | 华为技术有限公司 | Business monitoring method and device |
CN102354298A (en) * | 2011-07-27 | 2012-02-15 | 哈尔滨工业大学 | Software testing automation framework (STAF)-based fault injection automation testing platform and method for high-end fault-tolerant computer |
CN109873717A (en) * | 2019-01-18 | 2019-06-11 | 深圳壹账通智能科技有限公司 | Monitoring method, device, computer equipment and storage medium |
CN110362475A (en) * | 2019-06-17 | 2019-10-22 | 北京奇艺世纪科技有限公司 | A kind of information acquisition method, device, terminal device and storage medium |
CN115473839B (en) * | 2021-06-11 | 2024-03-05 | 北京字跳网络技术有限公司 | Buried point-based data processing method, device, equipment and storage medium |
CN115118631B (en) * | 2022-06-27 | 2023-09-01 | 平安银行股份有限公司 | Link exception handling method and device, electronic equipment and storage medium |
-
2024
- 2024-02-07 CN CN202410174604.1A patent/CN117724891B/en active Active
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110912738A (en) * | 2019-11-19 | 2020-03-24 | 支付宝(杭州)信息技术有限公司 | Business exception processing method, device, platform and electronic equipment |
Also Published As
Publication number | Publication date |
---|---|
CN117724891A (en) | 2024-03-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN102289771A (en) | Auto insurance claims system and method based on mobile communication network | |
CN108875328B (en) | Identity verification method, device and storage medium | |
CN110969215A (en) | Clustering method and device, storage medium and electronic device | |
CN111798664B (en) | Illegal data identification method and device and electronic equipment | |
EP2495696A1 (en) | Management server, population information calculation management server, zero population distribution area management method, and population information calculation method | |
CN117893215A (en) | Risk control method, electronic device, server and storage medium | |
CN117724891B (en) | Service data processing method and service data processing system | |
CN113626624A (en) | Resource identification method and related device | |
CN116226114B (en) | Data processing method, system and storage medium | |
CN110084467B (en) | Mobile label verification method, mobile label verification device, computer equipment and storage medium | |
CN105306891A (en) | Monitoring and early warning method, device and system | |
CN113114752A (en) | Data acquisition method, device, terminal and server based on public traffic scene | |
CN116861198A (en) | Data processing method, device and storage medium | |
CN108765823B (en) | Alarm method, device and system | |
CN107027142B (en) | Method and device for processing pseudo base station information | |
CN115562772B (en) | Scene recognition and preprocessing method and electronic equipment | |
CN113128299B (en) | People stream density detection method and device, terminal equipment, server and storage medium | |
CN111339939B (en) | Attendance checking method and device based on image recognition | |
CN112989078A (en) | Data processing method and device, computer equipment and storage medium | |
CN107169670B (en) | Law enforcement record evaluation system and method thereof | |
CN111091155A (en) | ID information pairing method, fusion method, device and storage medium | |
CN112818745B (en) | Method and device for determining correspondence between objects, electronic equipment and storage medium | |
CN115664982B (en) | Network resource management system based on cloud computing | |
CN115712745B (en) | Method, system and electronic device for acquiring user annotation data | |
CN116092203B (en) | Travel method, device and storage medium based on riding card |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |