WO2023241366A1

WO2023241366A1 - Data processing method and system, and electronic device and computer-readable storage medium

Info

Publication number: WO2023241366A1
Application number: PCT/CN2023/097671
Authority: WO
Inventors: 陈登月; 莫元武
Original assignee: 易保网络技术(上海)有限公司
Priority date: 2022-06-13
Filing date: 2023-06-01
Publication date: 2023-12-21
Also published as: CN115242433B; CN115242433A

Abstract

The present application relates to the technical field of computers, and in particular to a data processing method and system, and an electronic device and a computer-readable storage medium. The method comprises: a routing module sending, to a data management and control module, acquired first request data for a target business; the data management and control module determining whether the first request data meets a first condition, wherein the first condition is used for performing validity and security checking on the first request data that a target service module is requested to process; when confirming that the first request data meets the first condition, the data management and control module sending the first request data to the target service module; and when confirming that the first request data does not meet the first condition, the data management and control module modifying the first request data, so as to obtain second request data which meets the first condition, and the data management and control module sending the second request data to the target service module. By means of the solution in the present application, the security performance and the scenario adaptability performance of each service module can be effectively enhanced, and the renovation cost of a service function can be reduced.

Description

Data processing methods, systems, electronic devices and computer-readable storage media

This application claims the priority of the Chinese patent application submitted to the China Patent Office on June 13, 2022, with the application number 202210660992.5 and the application title "Data processing method, system, electronic device and computer-readable storage medium", and its entire content incorporated herein by reference.

Technical field

The invention relates to the field of computer technology, and in particular to a data processing method, system, electronic equipment and computer-readable storage medium.

Background technique

With the development of computer technology, more and more clients rely on corresponding business platforms to provide products and services to users. That is, clients can respond to user operations and obtain products and services provided by the connected business platform to process corresponding businesses. , and then the client can display the corresponding business processing results to the user based on the business processing data fed back by the business platform. For example, the insurance business client can rely on the online insurance platform to provide insurance-related service modules to users who need insurance. It can be understood that the client relying on the business platform can be a business system application (application, APP) running on an electronic device such as a mobile phone, or a web-based business system application, which is not limited here.

As a business platform, in order to provide clients with more comprehensive products and services, it is usually necessary to integrate some third-party services, such as link query services and visualization service third-party open source service modules. In other embodiments, third-party services can also be microservices running under Kubernetes, and the business platform can connect business requests for such third-party services to corresponding third-party services for processing, where Kubernetes is used An open source system that automatically deploys, scales and manages containerized applications. However, some third-party services may be software developed under some old frameworks, that is, the development framework may be incompatible with the system framework of the business platform; some third-party services may have authorization restrictions, etc. Therefore, when the business platform integrates or accesses these third-party services, it often needs to make some modifications to the software source code of the third-party services based on individual case needs. Among them, these modifications are, for example, some adaptive modifications to adapt to the system framework of the business, or modifications to lift authorization restrictions, etc. Moreover, when third-party services upgrade or update some entry parameters, etc., the above code modifications based on individual case requirements cannot be automatically completed to adapt to the updates, and developers can only make corresponding code modifications again.

Therefore, when the business scenario changes, or in order to meet the security requirements of some service modules provided by the business platform, it is necessary to enhance the functions of some third-party services integrated on the business platform, such as security performance, scene adaptability, etc., if it is also passed The above-mentioned integration of third-party services requires functional enhancement and transformation based on individual case needs, which will undoubtedly incur greater development costs. Therefore, a data processing solution is currently needed to solve the problem of functional enhancement of third-party services integrated or accessed by the business platform.

Contents of the invention

The embodiments of the present application provide a data processing method, system, electronic device and computer-readable storage medium to solve It solves the current problems of difficulty and large development volume in functional enhancement and transformation of service modules such as third-party services integrated on the business platform. It can effectively enhance the security performance and scene adaptability of each service module, so there is no need to modify third-party services. Higher-cost functional enhancements and transformations save service development costs. Moreover, the business platform implemented based on this application solution can easily connect to third-party services and perform some security or scene adaptability function upgrades without modifying the relevant code of third-party services, which is conducive to improving the client, development side and third-party services. A multi-device user experience.

In the first aspect, embodiments of the present application provide a data processing method, which method is applied to a business platform including a routing module, a data management and control module, and a service module, where the service module includes third-party services. The method includes:

The routing module obtains the first request data for the target service;

The routing module sends first request data to the data management and control module, where the first request data at least includes identification information of the target service module and target data acquisition parameters for the target service;

The data management and control module determines whether the first request data satisfies the first condition, where the first condition is used to check the validity and security of the first request data processed by the request target service module;

After confirming that the first request data meets the first condition, the data management and control module sends the first request data to the target service module;

After confirming that the first request data does not meet the first condition, the data management and control module modifies the first request data to obtain the second request data that meets the first condition, and the data management and control module sends the second request data to the target service module.

That is, through the data management and control module with the data management and control function, the business request-related data accessed by the business platform is screened, for example, including the above-mentioned validity and security check of the first request data based on the preset first condition. Finally, the request data that meets some inspection requirements related to the target service module (ie, the above-mentioned first request data) is sent to the target service module. The above-mentioned target service module includes a third-party service. The third-party service may be, for example, the visualization service (Kibana) illustrated in Embodiment 2 below. The above-mentioned first request data may be, for example, visualization service request data that needs to be processed by Kibana.

In a possible implementation of the first aspect, the first condition is dynamically determined based on at least one of the normative requirements of the API entry parameters of the target service module and the service data security requirements of the target service module.

For example, the first condition corresponding to the preset of the visualization service (Kibana) can be determined based on Kibana's various API entry parameter thresholds and other requirements, as well as Kibana's requirements for the source of data content for visual display and the security of the data content. . In other embodiments, the first condition may also include restrictions on the data table format corresponding to the data content to be visually displayed, etc., which is not limited here.

In a possible implementation of the above first aspect, the first condition includes a parameter threshold judgment condition preset for at least one API entry parameter of the target service module, and the data management and control module judges whether the first request data satisfies the first Conditions include: the data management and control module determines whether the value of the first parameter in the first request data is within the preset first parameter threshold range; if the value of the first parameter is within the first parameter threshold range, the data management and control module confirms The first request data satisfies the first condition; if the value of the first parameter is not within the first parameter threshold range, the data management and control module confirms that the first request data does not satisfy the first condition.

In a possible implementation of the above first aspect, it is confirmed that the first request data does not satisfy the first condition, and the data management and control module modifies the first request data to obtain the second request data that satisfies the first condition, including: The value of the first parameter within a parameter threshold range is adjusted to within the first parameter threshold range.

That is, for business request data that does not meet the API entry parameter threshold requirements of the target service module (i.e., the above-mentioned first request data), the corresponding API entry parameters in the request data can be adjusted to within the threshold requirement range, so that the business request data In the subsequent process, the target service module is successfully connected for processing.

In a possible implementation of the above first aspect, the target service module is used to process the request data sent by the data management and control module. The request data includes first request data and second request data, and the above method includes:

The target service module processes the received request data to obtain the first processing data;

The target service module returns the first processing data to the data management and control module;

The data management and control module determines whether the first processed data satisfies the second condition, where the second condition is used to perform a security check on the first processed data to be returned to the service requesting end, which is the client that initiates the target service;

After confirming that the first processed data meets the second condition, the data management and control module sends the first processed data to the routing module;

After confirming that the first processed data does not meet the second condition, the data management and control module modifies the first processed data to obtain second processed data that meets the second condition, and the data management and control module sends the second processed data to the routing module.

That is, the business processing data obtained by the target service module processing the corresponding business request data (i.e., the above-mentioned first request data), that is, the above-mentioned first processing data, can also perform some security checks through the data management and control module to improve the target service module in providing corresponding services. As well as the security of the processed data, the business requester receiving the business processing data can also receive higher security guarantees.

In a possible implementation of the above first aspect, the second condition is dynamically determined based on at least one of a security verification parameter, a permission verification parameter, and a data protection verification parameter of the service requesting end.

In a possible implementation of the above first aspect, the second condition includes sensitive data identification parameters as security verification parameters; and the data management and control module determines whether the first processed data satisfies the second condition, including:

The data management and control module determines whether the first processed data contains sensitive data based on the sensitive data identification parameters;

If the first processed data does not contain sensitive data, confirm that the first processed data meets the second condition;

If the first processed data contains sensitive data, it is confirmed that the first processed data does not meet the second condition.

In a possible implementation of the above first aspect, it is confirmed that the first processed data does not satisfy the second condition, and the data management and control module modifies the first processed data to obtain the second processed data that satisfies the second condition, including: deleting the first processed data. Sensitive data in the processed data; or, encrypting sensitive data in the first processed data.

In a possible implementation of the above first aspect, the second condition includes authorization information verification parameters as authority verification parameters; and the data management and control module determines whether the first processed data satisfies the second condition, including:

The data management and control module verifies parameters based on the authorization information to confirm whether the terminal recipient of the first processed data has the right to obtain the entire data content of the first processed data;

If the terminal recipient of the first processed data has the right to obtain the entire data content of the first processed data, it is confirmed that the first processed data satisfies the second condition;

If the terminal recipient of the first processed data does not have the right to obtain the entire data content of the first processed data, it is confirmed that the first processed data does not meet the second condition.

In a possible implementation of the above first aspect, it is confirmed that the first processed data does not meet the second condition, and the data management and control module modifies the first processed data to obtain the second processed data that satisfies the second condition, including:

If the terminal recipient of the first processed data does not have the right to obtain the entire data content of the first processed data, the first processed data is deleted.

In a possible implementation of the above first aspect, the second condition includes a preset time threshold for protecting business platform data as a data protection verification parameter; and the data management and control module determines whether the first processed data satisfies the second conditions, including:

The data management and control module determines whether the first processed data includes data whose collection time is earlier than a preset time threshold;

If the first processed data does not include data whose collection time is earlier than the preset time threshold, it is confirmed that the first processed data meets the second condition;

If the first processed data includes data whose collection time is earlier than the preset time threshold, it is confirmed that the first processed data does not meet the second condition.

In a possible implementation of the above first aspect, it is confirmed that the first processed data does not satisfy the second condition, and the data management and control module modifies the first processed data to obtain the second processed data that satisfies the second condition, including: Data in the processed data whose collection time is earlier than the preset time threshold are deleted.

In a possible implementation of the first aspect above, the routing module is any one of Nginx, Traefik, Envoy, and Kong.

It can be understood that in other embodiments, the above-mentioned routing module can also be other service modules that can access business data and have routing functions, which are not limited here.

In the second aspect, embodiments of the present application provide a data processing system, including:

The routing module is used to obtain the first request data for the target service and send it to the data management and control module, where the first request data at least includes the identification information of the target service module and the target data acquisition parameters for the target service;

The data management and control module is used to determine whether the first request data satisfies the first condition, where the first condition is used to check the validity and security of the first request data processed by the request target service module; and, after confirming the first request data When the first condition is met, it is used to send the first request data to the target service module; and when it is confirmed that the first request data does not meet the first condition, it is used to modify the first request data to obtain the second request data that satisfies the first condition. , and sends the second request data to the target service module;

The target service module is configured to process the first processing data according to the received first request data or the second request data, and return the first processing data to the data management and control module.

It can be understood that the above-mentioned target service module can be either a local service module developed in the business platform introduced in the embodiment below, or a third-party service module integrated or accessed by the business platform, such as the chain example in Embodiment 1 below. The road query service and the visualization service (Kibana) exemplified in Embodiment 2 are not limited here.

In a possible implementation of the second aspect above, the data management and control module is also used to determine whether the first processed data satisfies a second condition, where the second condition is used to perform a security check on the first processed data to be returned to the business requesting end. , the service requesting end is the client that initiates the target service; and, when confirming that the first processing data meets the second condition, is used to send the first processing data to the routing module;

When it is confirmed that the first processing data does not meet the second condition, it is used to modify the first processing data to obtain second processing data that meets the second condition, and send the second processing data to the routing module.

In a third aspect, embodiments of the present application provide an electronic device, including: one or more processors; one or more memories; one or more memories store one or more programs. When one or more programs are When one or more processors are executed, the electronic device is caused to execute the data processing method provided in the first aspect.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium that includes a computer program/instruction. When the computer program/instruction is executed by a processor, the data processing method provided in the first aspect is implemented.

Description of the drawings

Figure 1 shows a schematic diagram of an interaction scenario between a client and a business platform provided by an embodiment of the present application.

Figure 2a shows a processing process of business interaction data between a client and a business platform provided by an embodiment of the present application. Process diagram.

Figure 2b shows a schematic diagram of the processing process of business interaction data between another client and the business platform provided by the embodiment of the present application.

Figure 3 shows a schematic flowchart of the implementation of a data processing method provided by an embodiment of the present application.

Figure 4 shows a schematic flowchart of the implementation of a data processing method corresponding to the link query service provided in Embodiment 1 of the present application.

Figure 5 shows a schematic implementation flow diagram of a data processing method corresponding to Kibana provided in Embodiment 2 of the present application.

Figure 6 shows a schematic structural diagram of an electronic device 600 for running a business platform or client provided by an embodiment of the present application.

Detailed ways

Figure 1 shows a schematic diagram of an interaction scenario between a client and a business platform according to an embodiment of the present application.

As shown in Figure 1, this scenario includes a client program running on device 100a (referred to as client 100a), a development-side program running on device 100b (referred to as development-side 100b), and a third-party service running on device 100c. The provider (referred to as the third-party server 100b) and the business platform running on the device 200 (referred to as the business platform 200).

The client 100a may be a program product developed by an insurance company and other merchants and oriented to terminal consumer groups, and is used to provide insurance-related business services or other non-insurance-related business services to terminal consumer groups (ie, users).

The development terminal 100b is used to provide a development platform for developers. Developers can develop service modules that provide various services in the business platform 200 through the development terminal 100b, including technical services that provide functions such as analysis or visual display, and functions such as business processing. business services, etc. In addition, the service module includes not only local services developed for the business platform 200, but also third-party services that are integrated into the business platform 200 or access the business platform 200 to provide corresponding service content through correspondingly developed program interfaces, which are not limited here.

For example, the third-party server 100c can provide some open source and accessible third-party services to dock corresponding business requests transferred by the business platform 200, or some third-party services provided by the third-party server 100c can also be integrated into the business platform 200 for corresponding processing. Corresponding business request. The third-party services that are accessed or integrated into the business platform 200 may be, for example, email services, payment services, link query services, visualization services, etc., which are not limited here.

The service platform 200 is used to access the service request initiated by the client 100a, and arrange the corresponding service module to process the service request. It can be understood that the business platform 200 can communicate and connect with the client 100a, the development 100b and the third-party server 100c respectively to form a business service system or a data processing system.

It can be understood that the device 100a running the client program or the device 100b running the development program can be, for example, a mobile phone, a laptop, a tablet or other electronic device, or run a third-party service. The device 200 running the business platform may be, for example, a server, a desktop computer, a laptop computer, a handheld computer, a netbook, or other electronic devices that are embedded or coupled with one or more processors or that can access the network. Here, No restrictions.

Continuing to refer to Figure 1, the user can initiate a service request through the client 100a. For example, the user can fill in an insurance order, or initiate a query for an insurance service order, or other service request through the insurance client 100a. Correspondingly, the service platform 200 can implement a routing function to access service requests from each client 100b. If the service module required by the business request is a local service module of the business platform 200 or a third-party service integrated into the business platform 200, the business platform 200 can call the corresponding service module to process the business request; if the service module required by the business request service module If it is a third-party service running on the device 100c, the service platform 200 can forward the accessed service request to the corresponding third-party service. In this way, the service platform 200 accesses the service request of the client 100a and sends it to the corresponding service module for processing.

The routing function implemented by the above-mentioned business platform 200 can be implemented by Nginx, Traefik, Envoy, Kong and other open source software with edge router functions, and is not limited here. In the embodiment of this application, the service module with routing function implemented in the business platform 200 based on the above-mentioned open source software is called a routing module. That is to say, the routing module can forward the corresponding service request to the corresponding service module in the service platform 200 according to the obtained target service identification information of each service request.

Continuing to refer to Figure 1, the service module provided on the service platform 200 responds to the service request sent by the client, and after performing corresponding service processing, the service processing data can be returned to the corresponding client through the routing module.

As mentioned above, the third-party services currently integrated or accessed on the business platform 200 have undergone necessary transformations according to the actual needs of the business platform during integration or program interface development. However, in the process of using third-party services to process client business requests, if the business platform 200 is to meet business processing security requirements or to adapt third-party programs to different business scenarios, the service module needs to perform corresponding functions. Enhanced code transformation, including code transformation in aspects such as security performance enhancement and scene adaptability enhancement. In this case, the third-party services integrated by the business platform 200 need to face secondary transformation with relatively large development costs, while the third-party services connected to the business platform 200 cannot achieve such function-enhancing transformations. For example, developers of the business platform 200 may not understand the existing code of third-party services, so it takes time and effort to understand the existing codes of integrated third-party services; for example, the technology stack used by some third-party services does not match the development business platform. 200 uses different technology stacks, so developers of the business platform 200 need to spend time and effort to learn the technology stacks corresponding to some third-party services, etc., so that it is possible to enhance the functions of third-party services. In other words, all of the above will greatly increase the cost of functional enhancement and transformation of third-party services.

In order to solve the above technical problems, this application provides a data processing method, which adds a data management and control module to the business platform to implement data processing on the request data sent to the service module and the business processing data returned by the service module. Inspection and control. That is, the added data management and control module can perform data inspection and control on the unified routing module of the business platform access request data, the request data and business processing data transmitted between the service module, including identifying whether the requester of the request data has access rights. , whether the API entry parameters and other parameters of the requested data meet the legality requirements, and whether the business processing data returned by the service module in response to the business request meets the security requirements, and whether data filtering is required. It can be understood that the above-mentioned service modules include local services developed for the business platform, third-party services integrated into the business platform, and third-party services accessed by the business platform through program interfaces. In this way, through the data processing method provided by the embodiment of the present application, the security performance and scene adaptability of each service module can be effectively enhanced. Therefore, there is no need to carry out costly functional enhancements and modifications to third-party services, saving service development costs.

It can be understood that the above-mentioned data management and control module can customize some data management and control functions according to the service module requirements provided by the business platform 200 to the client. For example, it can also add data processing functions supported by multi-tenants to adapt third-party services to multi-tenants. Scenes etc. There are no restrictions here.

For example, after the business platform performs a small amount of necessary modifications to the third-party service when integrating the third-party service, based on the data processing method provided by the embodiment of this application, the business request form or content requirements that originally did not meet the third-party service processing can be processed. Request data and access the corresponding third-party service after being processed by the data management and control platform, so that the business platform can Use third-party services to smoothly connect business requests initiated by each client. Moreover, in the data processing method provided by the embodiment of the present application, the above-mentioned data management and control module can also perform security screening on the business processing data returned by the service module, such as deleting some sensitive data and filtering historical data before a preset time length. etc., thus effectively improving the security of the corresponding service modules provided by the business platform.

As an example, Figure 2a shows a schematic diagram of the processing process of business interaction data between the client and the business platform.

As shown in Figure 2a, the business platform 200 receives, for example, a business request from the client 100a through the unified routing module 201a. After the routing module 201a identifies the target service of the corresponding business request, it forwards it to the corresponding service module 202a for processing, that is, Figure The “data entry” process shown in 2a. After processing the corresponding service request, the service module 202a returns the corresponding service processing data to the routing module 201a, and the routing module 201a forwards it to the client 100a, that is, the "data return" process shown in Figure 2a.

Figure 2b shows another schematic diagram of the processing process of business interaction data between the client and the business platform according to an embodiment of the present application.

As shown in Figure 2b, the business platform 200 receives, for example, a business request from the client 100a through the unified routing module 201b. After the routing module 201b identifies the target service of the corresponding business request, it first sends the request data of the business request to the data management and control module. 202b. The data management and control module 202b performs a data check on the requested data, such as identifying whether the requester of the requested data has access rights and whether parameters such as API entry parameters of the requested data meet legality requirements. The data management and control module 202b can also modify the request data that does not meet the data inspection requirements, such as adaptively modifying the request data parameters that do not meet the legality requirements. After that, the data management and control module 202b sends the request data that passes the data check or is qualified after modification to the corresponding service module 202b in the business platform 200 for processing. That is, the “data entry” process shown in Figure 2b.

Continuing to refer to Figure 2b, after processing the corresponding business request, the service module 202b first sends the corresponding business processing data to the data management and control module 202b, and the data management and control module 202b screens the returned business processing data based on the preset return conditions. , the preset return condition is, for example, used to confirm whether the returned business processing data meets security requirements, including whether it is necessary to filter out historical data that may easily cause security issues, whether it involves sensitive fields, etc. For business processing data that does not meet the preset return conditions, the data management and control module 202b can make some adaptive modifications, such as deleting historical data before the preset time length for the returned business processing data, or deleting the returned business processing data. Sensitivity treatment, etc. After that, the data management and control module 202b sends the business processing data that satisfies the preset return conditions or the modified return conditions to the routing module 201b, and the routing module 201b forwards it to the client 100a, that is, the "data return" shown in Figure 2b process.

It can be seen that compared with the business interaction data processing process shown in Figure 2a, in the business interaction data processing process shown in Figure 2b, the security of "entering" or "returning" data is higher, and the services provided by the business platform 200 The module handles various business data with greater security and scenario adaptability. And in the process of handing over the business data to the corresponding third-party service for processing, if it is necessary to realize the security function or scene adaptability of the third-party service, etc., it only needs to set the corresponding parameters for the data management and control module of the business platform 200 Adjustments or some adaptive code modifications are enough, and there is no need to modify third-party services. In this way, the development amount of some functional enhancements can be greatly reduced. In addition, the business platform 200 can also more flexibly access third-party services based on the data management and control module. When accessing a third-party service, you only need to adjust the data management and control module to process the accessed business request to comply with the entry parameters and other requirements of the third-party service. The request data can be sent to the third-party service for processing, and the data management and control module can also filter some useless data or data that threatens security returned by the third-party service, which can also ensure the access to the third-party service process. Safety.

Figure 3 shows a schematic flowchart of the implementation of a data processing method according to an embodiment of the present application. Among them, the process shown in Figure 3 shows the interaction between the unified routing module 201, the data management and control module 202 and the various service modules 203. As mentioned above, the service module 203 includes third-party services integrated or accessed by the business platform 200. .

Specifically, as shown in Figure 3, the process includes the following steps:

301: The routing module 201 obtains the service request data for the target service (as the first request data).

For example, the routing module 201 is Traefik software, which is one of the reverse proxy tools. It has functions such as HTTP reverse proxy and load balancing, and can intercept the HTTP request data sent by the client to the service module 203, that is, the business Request data.

The aforementioned "interception" can be understood as acquisition. Referring to the scenario shown in Figure 1 above, the routing module 201 can obtain the service request data sent to the service platform 200 by the client 100a or 100b. The service request data may include, for example, requester identification information, service module identification information, and specifically requested business content. The specific requested business content may be defined, for example, by a target data acquisition parameter, which is not limited here. Among them, the requester identification information, service module identification information, etc. can be sent in the form of domain names or paths, for example, and are not limited here.

302: The routing module 201 forwards the service request data to the data management and control module 202.

For example, before sending the service request data to the corresponding service module 203, the routing module 201 may first send the service request data to the data management and control module 202 preset in the service platform 200 for processing. It can be understood that for the service request data that needs to be forwarded to the data management and control module 202, the service module related information corresponding to the corresponding service request can be pre-configured in the routing module 201, so that when the routing module 201 receives the corresponding service request data, it can be based on The configuration first sends the received business request data to the data management and control module for processing.

It can be understood that generally, after intercepting the service request data, the routing module 201 (for example, Traefik) can forward it to the corresponding service module 203 based on the domain name or path in the request data and based on the corresponding preset forwarding rules. For example, the routing module 201 forwards service requests prefixed with /web1/ to the service module 203-1, forwards service requests prefixed with /web2/ to the service module 203-2, and so on. For another example, the routing module 201 can also confirm the target service that receives the request data according to the API path of the request data. That is, the routing module 201 can match the corresponding forwarding rules according to the API path and forward the business request data to the corresponding service module. There are no restrictions here. In this embodiment of the present application, when forwarding the service request data, the routing module 201 can first redirect the request data that is originally intended to be sent to the corresponding target service module 203 to the data management and control module 202 . It can be understood that when forwarding the service request data to the corresponding data management and control module 202, the routing module 201 can also forward the identification information corresponding to the identified target service module 203 to the data management and control module 202, so that the data management and control module 202 can perform corresponding processing. After data inspection, modification and other management and control processing, the business request data is sent to the target service module 203.

303: The data management and control module 202 confirms the access permission of the requester based on the received request data.

For example, the data management and control module 202 analyzes and processes the received service request forwarded by the routing module 201 . For example, the data management and control module 202 may first confirm whether the requesting party has access rights based on the requesting party identification information in the request data. That is to say, the data management and control module 202 can first authenticate the requester who initiates the service request data, for example, check whether the requesting user has permission authentication and whether he has access permission to the requested service content, etc. The authority authentication refers to whether the requesting user has completed legal identity authentication in the corresponding business system. Users who have completed legal identity authentication can verify their legal identity information by providing user names and passwords. If the username and password are consistent, it is considered that the user has passed the authentication, has authority authentication, and has access rights to the corresponding business content.

Referring to the scenario shown in Figure 1 above, as an example, if user A queries the history of user B from the client 100a, If the policy data is stored, the routing module 201 forwards the service request data initiated by the client 100a to the data management and control module 202. The data management and control module 202 can first identify whether user A has a right to the request based on the requester identification information in the request data. Access rights to user B’s historical policy data. If user A is the administrator of the insurance business platform, for example, has the authority to query the historical policy data of user B in the past week, it can be confirmed that the account of user A logged in by client 100a has access permission, that is, client 100a belongs to the authorized requester .

If user A is another insurance user and is not an administrator, he does not have the right to query the historical policy data of user B. At this time, the data management and control module 202 can confirm that the client 100a used by user A, as the service request initiator, does not have Access permission, that is, the client 100a belongs to the requester without permission.

It can be understood that in other embodiments, user A may also have restricted permissions. For example, user A's permissions may limit user A to only query policy business data for the past week. In this case, the data management and control module 202 may accordingly Add this restrictive query condition to the business request data, and then continue to perform the following steps 305 to 308. After completing the data check, the business request data with the restrictive query condition added is then sent to the corresponding service module 203 for processing. No further details will be given here.

304: For the unauthorized requester, the data management and control module 202 returns an access denial message to the routing module 201.

For example, if the data management and control module 202 determines that the requester of the received business request data does not have the corresponding access rights, that is, if the requester is confirmed to be an unauthorized requester, the data management and control module 202 may return an access denial message to the requester through the routing module 201. information. In other embodiments, the data management and control module 202 may also feed back error prompt information to the requesting party through the routing module 201, which is not limited here. It can be understood that the routing module 201 can forward the received access denial message returned by the data management and control module 202 to the requester that initiates the response to the service request data.

305: For the authorized requester, the data management and control module 202 checks whether the received request data meets the data inspection requirements (as the first condition).

For example, if the data management and control module 202 determines that the requester of the received service request data is an authorized requester, it may further perform a data check on the received service request data. This data check includes, for example, checking whether relevant parameters in the business request data meet legality requirements or are compliant with regulations. If the requested data meets the data inspection requirements, for example, all parameters are compliant and legal, you can continue to perform the following step 306 to send the request data to the corresponding service module for processing; if the requested data does not meet the data inspection requirements, for example, some If the parameters do not meet the legality requirements or are not compliant, you need to perform the following step 307 to modify the relevant parameters.

As an example, the data management and control module 202 can check whether the API entry parameters in the request data meet the legality requirements. For example, a certain API entry parameter queries the reasonable age range of the insured person should be between 0 and 65. If a certain API entry parameter is received, In the request data, if the value of the corresponding API entry parameter is 70, it can be considered that this API entry parameter in the received request data is illegal. And if the value of the corresponding API entry parameter in a certain request data received is 55, it can be considered that the API entry parameter of the received request data is legal.

In other embodiments, in addition to checking whether the API entry parameters in the request data meet the legality requirements, the content of the data check can also be other, such as checking whether the option parameters about the insurance type in the request data are legal, etc., specific data The first condition based on which the management and control 202 performs data inspection can be customized, preset, or adjusted according to the needs of the business scenario, and is not limited here.

306: The data management and control module 202 sends the request data that meets the data inspection requirements to the service module 203.

Exemplarily, after completing the data check, the data management and control module 202 will, for the service request data that passes the check, that is, the request data that meets the data check requirements, for example, the request data whose relevant parameters meet the legality requirements, the data management and control module 202 202 can be sent to the corresponding service module 203 for corresponding business processing.

307: The data management and control module 202 modifies the request data that does not meet the data inspection requirements.

For example, for request data that does not meet the data inspection requirements, for example, request data whose API entry parameters do not meet the legality requirements, the data management and control module 202 can modify the relevant parameters. For example, for the above example, if in a certain request data received, the value of the corresponding API entry parameter is 70, which does not meet the legality requirements, the data management and control module 202 can modify the parameter value according to the preset reasonable age range. , for example, change "70" to "65". There are no restrictions here.

For another example, if there is no age parameter in the request data, and the API entry parameter of the target service module needs to match the age parameter, the data management and control module 202 can complete the age parameter in the request data, for example, a preset wildcard value, such as 50. As a completion value for the age parameter in the request data.

For another example, if the received request data includes an age parameter, but the API entry parameter needs to match the birth year parameter, the data management and control module 202 can convert the corresponding birth year parameter based on the age parameter in the request data, and then update to in request data.

In other embodiments, the content of the data check can also be other, for example, the option parameters for insurance types in the check request data, the option parameters for personal insurance types are mixed with the option parameters for enterprise group insurance, etc., which also need to be modified. , there is no restriction here.

308: The data management and control module 202 sends the modified request data (as the second request data) to the service module 203.

For example, for request data that does not meet the data check requirements, the data management and control module 202 can modify the checked illegal parameters and other data, so that the request data meets the data check requirements corresponding to the corresponding service module, for example, make the parameters For illegal request data, modify relevant parameters to within the legal range, etc.

309: The service module 203 responds to the received request data and performs corresponding business processing.

For example, after receiving the service request data from the data management and control module 202, the service module 203 can perform corresponding service processing in response to the requested service content corresponding to the request data.

In this embodiment, the service module 203 may serve a third party. Since the data management and control module 202 has processed the request data, the obtained request data meets the API interface parameter requirements of the service module 203. That is to say, regardless of whether the request data sent by the client meets the requirements of the API interface parameters of the service module 203, the service module 203 can respond to the request sent by the client without modifying the service module 203. In other embodiments, the service module 203 may also be a service module provided by the platform itself (not a third-party service).

In addition, it can be understood that due to the data processing function of the data management and control module 202, when the business platform 200 integrates a new third-party service or develops a new program interface to access the new third-party service, the business platform 200 does not need to Modifying the program code or related entry parameters of a third-party service, or the interface program code or related entry parameters of a third-party service, does not require modifying the program code or related parameters of the routing module, thus reducing the amount of development.

It can be understood that the service request data received by the service module 203 may be the request data that meets the data inspection requirements sent by the data management and control module 202 in the above-mentioned step 306, or may be the modified request data sent by the data management and control module 202 in the above-mentioned step 308. The request data is not limited here.

310: The service module 203 returns the business processing data (as the first processing data) to the data management and control module 202.

For example, after the service module 203 completes the corresponding business processing, the processed business processing data can be returned to the data management and control module 202. That is, the service processing data returned by the service module 203 in response to the corresponding service request can also be After further processing by the data management and control module 202, it is returned to the requesting party through the routing module 201. Please refer to the detailed description below for details and will not go into details here.

311: The data management and control module 202 checks whether the returned business processing data meets the preset return conditions (as the second condition). If the judgment result after the check is yes, it indicates that the business processing data can be returned, that is, the following step 312 can be performed; if the judgment result after the check is no, it indicates that the business processing data needs to be further checked and processed before returning. That is, the following step 313 can be performed.

For example, after receiving the business processing data returned by the service module 203, the data management and control module 202 can check the business processing data according to the preset return conditions. Among them, the preset return conditions can be set according to specific business content and business scenarios. For example, in the embodiment of this application, the preset return conditions may include checking whether the returned business processing data meets the corresponding business security requirements, such as checking whether it is necessary to filter some historical data, business data, etc. in the business processing data that may cause security problems. Process whether there are sensitive fields in the data, etc.

312: The data management and control module 202 returns the business processing data that meets the preset return conditions to the routing module 201.

For example, for business processing data that meets the preset return conditions, the data management and control module 202 can directly send it to the routing module 201, and the routing module 201 forwards it to the corresponding client, that is, the service requester.

313: The data management and control module 202 modifies the business processing data that does not meet the preset return conditions.

For example, for business processing data that does not meet the preset return conditions, the business processing data needs to be further checked and processed. As an example, for the above example, if there is historical data that needs to be filtered that may cause security problems, the data management and control module 202 can filter this part of the historical data. For example, it can delete the returned policy data with invalid validity period. to a portion of policy data for 3 months, etc., to complete data filtering. For another example, if there are sensitive fields in the business processing data, the data management and control module 202 can encrypt or desensitize the returned business processing data, for example, replace necessary sensitive fields with encrypted data, or replace unnecessary Sensitive fields can be deleted to desensitize them, etc., and there are no restrictions here.

As an example, for the above example, for example, the returned business processing data is the processing result corresponding to user A's query of user B's historical policy data, where user A's permission can only query user B's policy business data for the past week, then the corresponding service When the module 203 returns the business processing data to the data management and control module 202 after querying the results, the data management and control module 202 can filter and desensitize the returned business processing data according to user A's permissions. For example, the policy business data from one week ago and sensitive information such as User B's bank account in the returned business processing data are deleted, and then the processed business processing data is sent to the client through the routing module 201.

314: The data management and control module 202 returns the modified business processing data (as the second processing data) to the routing module 201.

For example, after the data management and control module 202 completes the modification of the business processing data that does not meet the data inspection requirements, the modified business processing data can be sent to the routing module 201, and the routing module 201 returns it to the service requester.

315: The routing module 201 returns the received service processing data to the service requester.

For example, the routing module 201 can return to the requester that initiates the service request, such as the client 100a or 100b shown in FIG. 1, the business processing data that meets the preset return conditions after being checked by the data management and control module 202, or the data The management and control module 202 modifies the processed business processing data, etc., which are not limited here.

It can be understood that the data processing method implemented based on the process shown in Figure 3 above can achieve a certain degree of data control, including security control and control of legal compliance of data-related parameters, etc., to realize the data control of this data control function. Module 202 can also preset (or customize) adaptive data management and control strategies according to different service modules. Including data inspection strategies for business request data and data inspection strategies for business processing data returned by the service module. In this way, if it is necessary to enhance the functions of the third-party services integrated or accessed by the business platform 200 in terms of security performance, scene adaptability, etc., the corresponding data management and control conditions in the above-mentioned data management and control module 202 can be adjusted, and then based on data management and control The transfer transition processing of module 202 is used to realize the above functional enhancement. Based on this application solution, there is no need to upgrade the code execution logic of the third-party service itself, which is conducive to reducing the cost of manpower and resources spent on functional upgrades of service modules such as third-party services and some local services of the business platform 200.

In order to understand the technical solution of the present application more clearly, based on the above-mentioned interaction process shown in Figure 3 and the structure of the data management and control module 202 shown in Figure 4, the following describes the data processing method provided by the embodiment of the present application in combination with specific business scenarios. Specific implementation processes in some business scenarios.

Example 1

This embodiment of the application takes the service module 203 as the link query service (jaeger-query) 203-1 as an example to introduce the specific implementation process of the data processing method provided by the embodiment of the application, and then implements the link query service 203-1. For the purpose of functional enhancement. Among them, the link query service 203-1 mainly provides the query capability for application call link information.

It can be understood that in the scenario of distributed microservices, a business request initiated by a user through the client may undergo response processing by multiple service modules after accessing the business platform. If the user wants to query in which link (that is, which service module) the business request takes more time, the user can query through the link query service 203-1, and the business platform can query according to the link query service 203-1. The time consuming of business requests in the processing of each service module is analyzed, and whether each service module operates abnormally. It can be understood that the link query service 203-1 is mainly used to provide the query capability for application call link information. This service can implement analysis of application performance (or service performance) and the degree of rationalization of the call link.

Figure 4 shows a schematic implementation flow chart of a data processing method corresponding to the link query service 203-1 according to an embodiment of the present application. It can be understood that the process shown in Figure 4 involves the interaction between the routing module 201, the data management and control module 202 and the link query service 203-1.

Specifically, as shown in Figure 4, the method specifically includes the following steps:

401: The routing module 201 obtains link analysis request data.

For example, the requester who initiates the link analysis request to request the link query service 203-1 to provide the corresponding link information query service may be, for example, the administrator account of the developer corresponding to a certain client program. When detection is required, When the business platform 200 calls various service modules that process business requests and whether they are running normally, the administrator account can log in to the client 100a and initiate the above-mentioned link analysis request to the business platform 200. In other embodiments, some clients may also initiate the above link analysis request to the service platform 200 based on other business requirements, which is not limited here.

It can be understood that the data used to call the link query service to request link analysis usually includes some query parameter options used to determine query conditions. The parameter types corresponding to these options may include, for example, Traceld (used to mark monitoring objects), service call interface name, client application name, client IP (that is, the IP of the service call initiator), the service name being queried and called, time-consuming threshold (for example, the call takes more than the specified number of milliseconds), call type, whether the call is abnormal, The business primary key (that is, the field based on which the corresponding business event is searched), response code, etc. will not be described in detail here.

For the specific process of obtaining the request data, please refer to the above-mentioned step 301, which will not be described in detail here.

402: The routing module 201 forwards the link analysis request data to the data management and control module 202.

For the specific process of forwarding the request data, please refer to the above-mentioned step 302, which will not be described in detail here.

403: The data management and control module 202 analyzes the request data according to the received link and confirms the access permission of the requester.

For example, the data management and control module 202 can, for example, according to the received link analysis request data, confirm whether the account logged in by the client that initiated the link analysis request is a management account authorized by the business platform 200. If so, it can confirm that the account logged in is a management account authorized by the business platform 200. The requester is a requester with authority; if not, it can be confirmed that the requester is a requester without authority.

For the specific process of confirming the access permission of the requesting party, please refer to the above-mentioned step 303, which will not be described in detail here.

404: For the requester without permission, the data management and control module 202 returns an access denial message to the routing module 201.

For example, if the data management and control module 202 confirms that the requester initiating the link analysis request is not a management account authorized by the business platform 200 and confirms that the requester is an unauthorized requester, then the requester's link analysis request can be rejected. .

For the specific process of denying access, please refer to the above-mentioned step 304, which will not be described in detail here.

405: For the authorized requester, the data management and control module 202 performs data inspection on the received link analysis request data.

For example, if the data management and control module 202 confirms that the requester who initiated the link analysis request is a management account authorized by the business platform 200 and can confirm that the requester is an authorized requester, the data management and control module 202 can further receive the link. Road analysis request data to perform corresponding data checks. For example, the data management and control module 202 can perform data inspection through preset cookies and the content of the link analysis request. If the link analysis request data meets the data inspection requirements, the following step 406 can be continued to send the link analysis request data to The link query service 203-1 performs processing.

If the link analysis request data does not meet the data inspection requirements, for example, the link analysis request data lacks the interface name of the service call as a query parameter, or the interface name is inaccurate, and the link query service 203-1 queries the corresponding service data. It needs to be based on the interface name, and it cannot support fuzzy search on the query parameter of the interface name, that is, related parameters that lack necessary information. At this time, you need to perform the following step 407 to modify the parts of the request data that do not meet the inspection requirements, such as matching the interface name of the corresponding service based on other relevant parameters in the link analysis request data, or modifying the ambiguous interface name. Fuzzy search, get the accurate interface name, replace the original interface name in the request data, etc.

After executing the following step 407, the following step 408 is executed to send a link analysis request to the link query service 203-1. For the specific data checking process, please refer to the above-mentioned step 305, which will not be described in detail here.

406: The data management and control module 202 sends the link analysis request data that meets the data inspection requirements to the link query service 203-1.

407: The data management and control module 202 modifies the link analysis request data that does not meet the data inspection requirements.

For example, the modification of the link analysis request data that does not meet the data inspection requirements includes information completion for parameters that lack necessary information in the request data. For details, please refer to the above-mentioned step 307, which will not be described again here.

408: The data management and control module 202 sends the modified link analysis request data to the link query service 203-1.

409: In response to the received link analysis request data, the link query service 203-1 queries the time-consuming data of each service module in the service link. Illustratively, the time-consuming data includes the time it takes for each service module to receive the corresponding service request and perform the corresponding business processing, and the time it takes for each service module to perform the business processing process, etc., which are not limited here.

410: The link query service 203-1 returns the queried time-consuming data to the data management and control module 202.

411: The data management and control module 202 checks whether the returned time-consuming data meets the preset return conditions.

Exemplarily, the data management and control module 202 receives the original data (ie, time-consuming data) returned by the link query service 203-1 Check whether the preset return conditions are met, such as checking whether there is sensitive data. If there is sensitive data, desensitization processing is required, such as deleting sensitive data that does not need to be returned. For another example, the data management and control module 202 can also check whether the returned time-consuming data contains relevant authorization information of the requester, for example, based on the preset client IP in the link analysis request data, to confirm that the link analysis service 203-1 Whether the returned time-consuming data corresponds to the client IP information has corresponding customer authorization information. If there is no authorization information, it means that the client IP has not yet obtained authorization from the corresponding client, that is, the requester is not authorized, and the time-consuming data returned by the link query service 203-1 is deleted.

For the specific inspection and judgment process, please refer to the above-mentioned step 311, which will not be described in detail here.

412: The data management and control module 202 returns the time-consuming data that satisfies the preset return conditions to the routing module 201.

For example, the time-consuming data that meets the preset return conditions after checking can be returned to the requesting client and displayed on the corresponding link analysis page. Among them, the preset return conditions may not only include the conditions such as no sensitive data and authorization information of the requesting party as exemplified in step 411 above, but may also include other preset return conditions. There are no restrictions here.

For example, the relevant parameters of the link analysis request data responded by the link query service 203-1 may also include a time-consuming threshold, the name of the service called by the query, etc., then the link query service 203-1 responds to the link analysis Among the time-consuming data returned by the request, the corresponding call time should be greater than the specified number of milliseconds, and the time-consuming data should correspond to the service name being queried and called. However, the time-consuming data obtained by the link query service 203-1 may include the time-consuming data of calling the service within the last year. The time-consuming data may be, for example, the time-consuming data of calling the policy data management service. The business platform 200, for example, a business platform that provides corresponding services for the insurance business, may only allow the service call time-consuming data of the last three months to be called based on security considerations. In this case, the preset return conditions set in the data management and control module 202 will include time filtering conditions, that is, the data management and control module 202 can control the filtering of time-consuming data 3 months ago, and only return the latest 3 months to the routing module. Months of time-consuming data.

413: The data management and control module 202 modifies the time-consuming data that does not meet the preset return conditions.

For example, the data management and control module 202 detects that there is sensitive data in the time-consuming data returned by the link query service 203-1, and it needs to perform desensitization processing, such as deleting the corresponding sensitive data, or encrypting the corresponding sensitive data, that is, the above modification. process. For another example, referring to the example in step 412 above, if the default return condition set in the data management module 202 includes the time filter condition of "last 3 months", then the time-consuming time returned by the link query service 203-1 Among the data, the time-consuming data 3 months ago belongs to the time-consuming data that does not meet the preset return conditions, and the data management and control module 202 can delete it from the time-consuming data to be returned.

For the specific modification process, please refer to the relevant description in step 313 above, and will not be described again here.

414: The data management and control module 202 returns the modified time-consuming data to the routing module 201.

415: The routing module 201 returns the received time-consuming data to the requester.

It can be understood that the data processing method corresponding to the link query service 203-1 implemented based on the above-mentioned process shown in Figure 4 can perform authentication and control on the link analysis request requesting the link query service 203-1, and can also perform authentication and control on the link query service 203-1. The time-consuming data returned by the link query service 203-1 is subjected to sensitive data inspection and desensitization processing, or whether it meets the security requirements of some service module-related data, etc., thereby improving the link query analysis provided by the link query service 203-1. Security of the Service. Moreover, this enhancement of security performance can be achieved without upgrading the link query service 203-1.

Example 2

This embodiment of the present application takes the service module 203 as a visualization service, namely Kibana 203-2, as an example to introduce how to implement the function enhancement of the data visualization service by implementing the data processing method provided by the embodiment of the present application. Among them, Kibana 203-2 is a visualization platform that searches, views, stores and retrieves data through Kibana. (Elasticsearch) indexes data and interacts with it to achieve data analysis and visualization, such as displaying the searched data in the form of charts.

It is understandable that in the operation, maintenance and management scenarios of some distributed business systems, some users of the distributed business system may want to display some business statistics in the form of charts and other forms on the client interface to facilitate analysis and management. At this time, the user can initiate a visualization service request for Kibana 203-2 to the business platform 200 through the corresponding client to request the target data for visual display through Kibana 203-2 search and the page for visual display of the data.

Figure 5 shows a schematic implementation flow diagram of a data processing method corresponding to Kibana 203-2 according to an embodiment of the present application. It can be understood that the process shown in Figure 5 involves the interaction between the routing module 201, the data management and control module 202 and Kibana 203-2.

Specifically, as shown in Figure 5, the method specifically includes the following steps:

501: The routing module 201 obtains the visualization service request data.

For example, the requester who initiates a visualization service request to request Kibana 203-2 to provide corresponding data search and visualization services can be, for example, the administrator account of a developer corresponding to a certain client program. When the client program needs to be served, When analyzing user-related business data, you can initiate a Kibana 203-2 visualization service request to the business platform 200 through the operation and maintenance client of the business system to request that the client program be provided with corresponding service processing through Kibana 203-2 search. The operation log or related link information of the service module of the corresponding business request is used to obtain the corresponding business data, and a page for visual display of the searched corresponding business data is provided.

In other embodiments, some clients may also initiate the above visualization service request to the business platform 200 based on other business requirements, which is not limited here.

502: The routing module 201 sends the visualization service request data to the data management and control module 202.

503: The data management and control module 202 confirms the access permission of the requester based on the received visualization service request data.

For example, the data management and control module 202 can confirm, based on the received visual service request data, whether the client that initiated the visual service request has the authority to obtain data such as the corresponding service module operation log. For example, the visual service request is a service request to visually display the number of new policyholders, the number of intended policyholders, and the number of policyholders being maintained on the insurance business platform, then the client that initiated the service request or the The account logged in on the client (i.e. the requesting party) should have the management authority to view and obtain policyholder-related data on the insurance business platform. If the requesting party has the authority to obtain the policyholder-related data to be visualized, the requesting party can be confirmed to be a requesting party with authority; if the requesting party does not have permission to obtain, the requesting party can be confirmed to be a requesting party without authority.

For the specific process of confirming the access permission of the requesting party, you can also refer to the above-mentioned step 303, which will not be described in detail here.

504: For the requester without permission, the data management and control module 202 returns an access denial message to the routing module 201.

For example, if the data management and control module 202 confirms that the requester who initiates the visualization service request does not have the authority to obtain data such as the corresponding service module operation log and confirms that the requester is an unauthorized requester, it can deny the visualization of the requester. Request for service.

For the specific process of denying access, you can also refer to the above-mentioned step 304, which will not be described again here.

505: For the authorized requester, the data management and control module 202 performs data inspection on the received visualization service request data.

For example, if the data management and control module 202 confirms that the requester who initiated the visualization service request has the authority to obtain the corresponding service module operation log and other data, and can confirm that the requester is an authorized requester, the data management and control module 202 can further receive The visualization service requests the data for corresponding data inspection. For example, the data management and control module 202 can perform data checking through preset cookies and the content of the visual service request, such as checking whether relevant request parameters lack necessary information, etc. If the visualization service request data meets the data inspection requirements, for example, the descriptive parameters set corresponding to the requested relevant data in the visualization service request data are complete and accurate, and there is no lack of necessary information. Then you can continue to perform the following step 506 to send the visualization service request data to Kibana 203-2 for processing.

If the visual service request data does not meet the data inspection requirements, for example, the descriptive parameters corresponding to the requested relevant data in the visual service request data are missing or inaccurate, for example, the policy data time series parameters to be obtained are missing, that is, The collection time of policy data must correspond to the set start time and end time being missing or incorrectly set. For example, the start time or end time may be set to a time in the future. In this case, necessary information is missing. Then you need to perform the following step 507 to modify the portion of the request data that does not meet the inspection requirements. For example, the data management and control module 202 can set the missing start time to a date 6 months ago based on the 6-month time span usually set by Kibana 203-2 to provide visualization services, and set the end time to the same date as the start time. The starting time span is a certain date of 6 months, that is, it is set to obtain the policy data of the past 6 months to extract the number of policyholders for visual display to complete the missing information.

Afterwards, the data management and control module 202 can continue to perform the following step 508 to send a visualization service request to Kibana 203-2. For the specific data checking process, you can also refer to the above-mentioned step 305, which will not be described again here.

506: The data management and control module 202 sends the visualization service request data that meets the data inspection requirements to Kibana 203-2.

507: The data management and control module 202 modifies the visualization service request data that does not meet the data inspection requirements.

For example, the modification of the visualization service request data that does not meet the data inspection requirements includes information completion for parameters that lack necessary information in the request data.

For the aforementioned example, for example, the data management and control module 202 can complete the visualization service request data requesting to display the analysis results of the number of policyholders based on the 6-month time span usually set by Kibana 203-2 to provide visualization services. For example, the data management and control module 202 sets the missing start time to a date 6 months ago, and sets the end time to a date that spans 6 months from the starting time, that is, setting the acquisition The policy data of the past 6 months are used to extract the number of policyholders for visual display to complete the missing information.

For the specific process of modifying the visualization service request data that does not meet the data inspection requirements, you can also refer to the above step 307, which will not be described again here.

508: The data management and control module 202 sends the modified visualization service request data to Kibana 203-2.

509: In response to the received visual service request data, Kibana 203-2 searches the running log or link information of the corresponding service module to process the visual data of the corresponding business system.

For example, the requested business data corresponding to the visualization service request data may be policy data, for example. Kibana 203-2 may obtain the operation log of the corresponding policy data management service and/or the policy data management service in the insurance business platform. Call link information, etc., and then process the visual data used to form a visual page based on the policyholder-related information extracted from the operation log or call link information. Among them, the visual data includes the corresponding business data obtained based on the operation log of the corresponding service module, and the relevant parameters of the visual page obtained by processing the corresponding business data through the data visualization processing capability provided by Kibana 203-2, such as the used histogram, Pie charts and other graphs Table path parameters, etc. are not limited here.

510: Kibana 203-2 returns the processed visual data to the data management and control module 202.

511: The data management and control module 202 checks whether the returned visual data meets the preset return conditions.

For example, the data management and control module 202 checks whether the original data (i.e., visual data) returned by Kibana 203-2 meets the preset return conditions, for example, checking whether there is insurance amount data or insurance in the visual data corresponding to the analysis result of the number of policyholders. Sensitive data such as a person’s ID number. If there is sensitive data, desensitization processing is required, such as deleting sensitive data that does not need to be returned, etc. For another example, the data management and control module 202 can also check whether the returned visual data contains relevant authorization information of the requester. If there is no authorization information, for example, the requester is not authorized, the visual data returned by Kibana 203-2 will be deleted.

For the specific inspection and judgment process, you can also refer to the above-mentioned step 311, which will not be described again here.

512: The data management and control module 202 returns the visual data that meets the preset return conditions to the routing module 201.

For example, visual data that meets the preset return conditions after inspection can be returned to the requesting client for visual display on the corresponding client business interface.

513: The data management and control module 202 modifies the visual data that does not meet the preset return conditions.

For example, the data management and control module 202 detects that there is sensitive data in the visual data returned by Kibana 203-2. For example, the visual data corresponding to the analysis result of the number of policyholders includes the insurance amount data of some policyholders, or some of the insured persons. A person’s ID number and other information. Desensitization processing is required, such as deleting sensitive data such as the insurance amount data and the ID number of the policy holder in the visual data corresponding to the analysis results of the number of policyholders, or encrypting the corresponding sensitive data, which is the above modification process.

For the specific modification process, you can also refer to the relevant description in step 313 above, and will not be described again here.

514: The data management and control module 202 returns the modified visual data to the routing module 201.

515: The routing module 201 returns the received visualization data to the requester.

It can be understood that the data processing method corresponding to Kibana 203-2 implemented based on the process shown in Figure 5 above can authenticate and control the visualization service request data requesting Kibana 203-2, and can also perform visualization returned by Kibana 203-2 The data undergoes sensitive data inspection and desensitization processing, thereby improving the security of data search and visualization services provided by Kibana 203-2. Moreover, this security performance enhancement can be achieved without upgrading Kibana 203-2.

Figure 6 shows a schematic structural diagram of an electronic device 600 according to an embodiment of the present application. In this embodiment of the present application, the electronic device 600 can run the above-mentioned business platform 200. In other embodiments, the electronic device 600 can also run clients of some business systems, such as running insurance business clients, etc., which is not limited here.

As shown in Figure 6, in some embodiments, server 200 may include one or more processors 604, system control logic 608 connected to at least one of the processors 604, system memory 612 connected to the system control logic 608, A non-volatile memory (NVM) 616 coupled to the system control logic 608, and a network interface 620 coupled to the system control logic 608.

In some embodiments, processor 604 may include one or more single-core or multi-core processors. In some embodiments, processor 604 may include any combination of general-purpose processors and special-purpose processors (eg, graphics processors, applications processors, baseband processors, etc.). In an embodiment where the server 200 adopts an eNB (Evolved Node B, enhanced base station) or a RAN (Radio Access Network, radio access network) controller, the processor 604 may be configured to execute various conforming embodiments, for example, One or more of the various embodiments shown in Figures 2-5.

In some embodiments, system control logic 608 may include any suitable interface controller to provide any suitable interface to at least one of processors 604 and/or any suitable device or component in communication with system control logic 608 .

In some embodiments, system control logic 608 may include one or more memory controllers to provide an interface to system memory 612 . System memory 612 may be used to load and store data and/or instructions. Memory 612 of server 200 may include any suitable volatile memory in some embodiments, such as suitable dynamic random access memory (DRAM).

NVM/memory 616 may include one or more tangible, non-transitory computer-readable media for storing data and/or instructions. In some embodiments, NVM/memory 616 may include any suitable non-volatile memory such as flash memory and/or any suitable non-volatile storage device, such as HDD (Hard Disk Drive), CD (Compact Disc) , CD) drive, DVD (Digital Versatile Disc, Digital Versatile Disc) drive at least one.

NVM/storage 616 may comprise a portion of storage resources on the device on which server 200 is installed, or it may be accessed by the device but is not necessarily part of the device. For example, NVM/storage 616 may be accessed over the network via network interface 620.

In particular, system memory 612 and NVM/storage 616 may include temporary and permanent copies of instructions 624, respectively. Instructions 624 may include instructions that, when executed by at least one of processors 604, cause server 200 to implement the methods illustrated in Figures 3-4. In some embodiments, instructions 624, hardware, firmware, and/or software components thereof may additionally/alternatively be located in system control logic 608, network interface 620, and/or processor 604.

Network interface 620 may include a transceiver for providing a radio interface for server 200 to communicate with any other suitable devices (such as front-end modules, antennas, etc.) over one or more networks. In some embodiments, network interface 620 may be integrated with other components of server 200. For example, network interface 620 may be integrated with at least one of processor 604, system memory 612, NVM/storage 616, and a firmware device (not shown) with instructions that when at least one of processor 604 executes said When instructed, the server 200 implements the above-mentioned methods shown in FIGS. 2 to 5 .

Network interface 620 may further include any suitable hardware and/or firmware to provide a multiple-input multiple-output radio interface. For example, network interface 620 may be a network adapter, a wireless network adapter, a telephone modem, and/or a wireless modem.

In one embodiment, at least one of the processors 604 may be packaged with logic for one or more controllers of the system control logic 608 to form a system in package (SiP). In one embodiment, at least one of the processors 604 may be integrated on the same die with logic for one or more controllers of the system control logic 608 to form a system on a chip (SoC).

Server 200 may further include input/output (I/O) devices 632. The I/O device 632 may include a user interface that enables a user to interact with the server 200; the peripheral component interface is designed to enable peripheral components to also interact with the server 200. In some embodiments, the server 200 further includes a sensor for determining at least one of environmental conditions and location information related to the server 200 .

In some embodiments, the user interface may include, but is not limited to, a display (e.g., a liquid crystal display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and/or video camera), a flashlight (e.g., LED flash) and keyboard.

In some embodiments, peripheral component interfaces may include, but are not limited to, non-volatile memory ports, audio jacks, and Power interface.

In some embodiments, sensors may include, but are not limited to, gyroscope sensors, accelerometers, proximity sensors, ambient light sensors, and positioning units. The positioning unit may also be part of or interact with network interface 620 to communicate with components of the positioning network (eg, Global Positioning System (GPS) satellites).

Reference in the specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one example implementation or technology disclosed in accordance with the embodiments of the present application. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

The disclosure of the embodiments of the present application also relates to devices for performing the operations in the text. This apparatus may be specially constructed for the required purposes, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such computer programs may be stored on a computer-readable medium such as, but not limited to, any type of disk including floppy disk, optical disk, CD-ROM, magneto-optical disk, read-only memory (ROM), random access memory (RAM) , EPROM, EEPROM, magnetic or optical card, application specific integrated circuit (ASIC), or any type of medium suitable for storing electronic instructions, and each may be coupled to a computer system bus. Furthermore, the computers referred to in the specification may include a single processor or may employ an architecture involving multiple processors for increased computing power.

Additionally, the language used in this specification has been selected primarily for readability and instructional purposes and may not have been selected to delineate or limit the disclosed subject matter. Accordingly, the present disclosure of examples is intended to illustrate, but not to limit, the scope of the concepts discussed herein.

Claims

A data processing method, characterized in that the method is applied to a business platform including a routing module, a data management and control module, and a service module, wherein the service module includes third-party services, and the method includes:

The routing module obtains the first request data for the target service;

The routing module sends the first request data to the data management and control module, where the first request data at least includes identification information of the target service module and target data acquisition parameters for the target service;

The data management and control module determines whether the first request data satisfies a first condition, wherein the first condition is used to check the validity and security of the first request data processed by the request target service module;

Confirming that the first request data satisfies the first condition, the data management and control module sends the first request data to the target service module;

After confirming that the first request data does not satisfy the first condition, the data management and control module modifies the first request data to obtain the second request data that satisfies the first condition, and the data management and control module reports to The target service module sends the second request data.
The method according to claim 1, characterized in that the first condition is based on at least one of normative requirements of API entry parameters of the target service module and service data security requirements of the target service module. Items are determined dynamically.
The method according to claim 2, wherein the first condition includes a parameter threshold judgment condition preset for at least one API entry parameter of the target service module, and,

The data management and control module determines whether the first request data meets the first condition, including:

The data management and control module determines whether the value of the first parameter in the first request data is within a preset first parameter threshold range;

If the value of the first parameter is within the first parameter threshold range, the data management and control module confirms that the first request data satisfies the first condition;

If the value of the first parameter is not within the first parameter threshold range, the data management and control module confirms that the first request data does not meet the first condition.
The method according to claim 3, characterized in that, after confirming that the first request data does not satisfy the first condition, the data management and control module modifies the first request data to obtain the first request data that satisfies the first condition. The second request data includes:

Adjust the value of the first parameter that is not within the first parameter threshold range to within the first parameter threshold range.
The method according to claim 1, characterized in that the target service module is used to process the request data sent by the data management and control module, and the request data includes the first request data and the second request data. , and the method includes:

The target service module processes the received request data to obtain the first processing data;

The target service module returns the first processing data to the data management and control module;

The data management and control module determines whether the first processed data satisfies a second condition, wherein the second condition is used to perform a security check on the first processed data to be returned to the service requesting end, which is the initiator of the target. Business client;

Confirming that the first processed data meets the second condition, the data management and control module sends the first processed data to the routing module;

After confirming that the first processed data does not meet the second condition, the data management and control module modifies the first processed data to obtain second processed data that meets the second condition, and the data management and control module reports The routing module sends the second processing data.
The method of claim 5, wherein the second condition is dynamically determined based on at least one of a security verification parameter, an authority verification parameter, and a data protection verification parameter of the service requesting end. of.
The method of claim 6, wherein the second condition includes a sensitive data identification parameter as the security verification parameter; and the data management and control module determines whether the first processed data satisfies the second condition. conditions, including:

The data management and control module determines whether the first processed data contains sensitive data based on the sensitive data identification parameters;

If the first processed data does not contain sensitive data, confirm that the first processed data meets the second condition;

If the first processed data contains sensitive data, it is confirmed that the first processed data does not meet the second condition.
The method according to claim 7, characterized in that, after confirming that the first processing data does not satisfy the second condition, the data management and control module modifies the first processing data to obtain the data that satisfies the second condition. The second processing data of the condition includes:

Delete the sensitive data in the first processed data; or,

Sensitive data in the first processed data is encrypted.
The method of claim 6, wherein the second condition includes authorization information verification parameters as the authority verification parameters; and the data management and control module determines whether the first processed data satisfies the second condition. ,include:

The data management and control module confirms whether the terminal recipient of the first processed data has the right to obtain the entire data content of the first processed data based on the authorization information verification parameter;

If the terminal recipient of the first processed data has the right to obtain the entire data content of the first processed data, it is confirmed that the first processed data satisfies the second condition;

If the terminal recipient of the first processed data does not have the right to obtain the entire data content of the first processed data, it is confirmed that the first processed data does not satisfy the second condition.
The method according to claim 9, characterized in that, after confirming that the first processing data does not satisfy the second condition, the data management and control module modifies the first processing data to obtain the data that satisfies the second condition. The second processing data of the condition includes:

If the terminal recipient of the first processed data does not have the right to obtain the entire data content of the first processed data, the first processed data is deleted.
The method of claim 6, wherein the second condition includes a preset time threshold for protecting business platform data as the data protective verification parameter; and the data management and control module determines that the Whether the first processed data meets the second condition, including:

The data management and control module determines whether the first processed data includes data whose collection time is earlier than the preset time threshold;

If the first processed data does not include data whose acquisition time is earlier than the preset time threshold, confirm that the first processed data meets the second condition;

If the first processed data includes data whose acquisition time is earlier than the preset time threshold, it is confirmed that the first processed data does not satisfy the second condition.
The method according to claim 11, characterized in that, after confirming that the first processing data does not satisfy the second condition, the data management and control module modifies the first processing data to obtain the data that satisfies the second condition. The second processing data of the condition includes:

Delete data in the first processed data whose collection time is earlier than the preset time threshold.
The method according to any one of claims 1 to 12, characterized in that the routing module is any one of Nginx, Traefik, Envoy, and Kong.
A data processing system, characterized by including:

A routing module, configured to obtain the first request data for the target service and send it to the data management and control module, where the first request data at least includes the identification information of the target service module and the target data for the target service. Get parameters;

A data management and control module, used to determine whether the first request data satisfies a first condition, wherein the first condition is used to check the validity and security of the first request data processed by the request target service module; and, When it is confirmed that the first request data satisfies the first condition, used to send the first request data to the target service module; and when it is confirmed that the first request data does not satisfy the first condition, used to Modify the first request data to obtain second request data that satisfies the first condition, and send the second request data to the target service module;

The target service module is configured to process the first processing data according to the received first request data or the second request data, and return the first processing data to the data management and control module.
The data processing system according to claim 14, characterized in that the data management and control module is also used to determine whether the first processed data satisfies a second condition, wherein the second condition is used to return the service requesting end. The first processed data undergoes a security check, and the service requesting end is the client that initiates the target service; and,

When it is confirmed that the first processing data meets the second condition, be used to send the first processing data to the routing module;

When it is confirmed that the first processing data does not meet the second condition, it is used to modify the first processing data to obtain second processing data that meets the second condition, and send the said processing data to the routing module. Second process the data.
An electronic device, characterized in that it includes: one or more processors; one or more memories; the one or more memories store one or more programs. When one or more programs are processed by the one or more When executed by multiple processors, the electronic device is caused to execute the data processing method described in any one of claims 1 to 12.
A computer-readable storage medium, characterized in that it includes a computer program/instruction. When the computer program/instruction is executed by a processor, the data processing method according to any one of claims 1 to 12 is implemented.