CN116707846A - Data processing method and device - Google Patents

Data processing method and device Download PDF

Info

Publication number
CN116707846A
CN116707846A CN202210180527.1A CN202210180527A CN116707846A CN 116707846 A CN116707846 A CN 116707846A CN 202210180527 A CN202210180527 A CN 202210180527A CN 116707846 A CN116707846 A CN 116707846A
Authority
CN
China
Prior art keywords
data
request
request data
request response
proxy
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210180527.1A
Other languages
Chinese (zh)
Inventor
李宛儒
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Wodong Tianjun Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN202210180527.1A priority Critical patent/CN116707846A/en
Publication of CN116707846A publication Critical patent/CN116707846A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
    • H04L63/0281—Proxies
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
    • H04L63/0227—Filtering policies
    • H04L63/0245—Filtering by information in the payload
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
    • H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/20—Network architectures or network communication protocols for network security for managing network security; network security policies in general

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer And Data Communications (AREA)

Abstract

The invention discloses a data processing method and device, and relates to the technical field of computers. One embodiment of the method comprises the following steps: intercepting request data sent by a front end, and carrying out safety monitoring on the request data; transmitting the request data passing through the safety monitoring to a server side, and intercepting request response data from the server side, wherein the request response data is the response of the server side to the request data passing through the safety monitoring; desensitizing the request response data to generate request response desensitization data; the request response desensitization data is sent to the front end. The implementation mode can realize the solution of unified safety protection and data management aiming at the front end application, avoids the defect that the nginx proxy cannot protect, reduces the complexity of the code of the interface secondary packaging and protection proxy, and has high maintainability.

Description

Data processing method and device
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a data processing method and apparatus.
Background
At present, three security methods are applied to the front end: the scheme one is to repackage the back end interface with nodeJs to provide a layer of page, and make security protection (such as interface check, network protection, etc.) on the back end interface layer; the second scheme is that the java language is utilized to convert the rear-end interface into a restful interface to be provided for the page, and the rear-end interface layer is subjected to safety protection (such as interface verification, network protection and the like); and thirdly, the front end is deployed under the nginx server as a static resource, and an interface request of the front end is forwarded to the back end service by adopting a proxy forwarding mode. The nodeJs is a development platform for enabling JavaScript to run on a server, the restful is a design style and development mode of a network application program, and the nginx is a high-performance http and reverse proxy server.
In the process of implementing the present invention, the inventor finds that at least the following problems exist in the prior art:
the solution of unified safety protection and data management aiming at the front end application cannot be realized, the nginx agent has the defect of incapability of protection, meanwhile, the interface is secondarily packaged, the code complexity of the protection agent is higher, and the maintainability is low.
Disclosure of Invention
In view of this, the embodiment of the invention provides a data processing method and device, which can realize a solution for unified security protection and data management for front-end application, avoid the defect that the nginx proxy cannot protect, reduce the complexity of the interface secondary packaging and code of the protection proxy, and have high maintainability.
To achieve the above object, according to one aspect of an embodiment of the present invention, there is provided a data processing method.
A data processing method, comprising: intercepting request data sent by a front end, and carrying out safety monitoring on the request data; transmitting the request data passing through the safety monitoring to a server side, and intercepting request response data from the server side, wherein the request response data is the response of the server side to the request data passing through the safety monitoring; desensitizing the request response data to generate request response desensitization data; the request response desensitization data is sent to the front end.
Optionally, the security monitoring of the request data includes: acquiring monitoring rules from a first configuration file or a database; judging whether the request data is an illegal request with default or rule violating based on the monitoring rule, if so, the request data does not pass the safety monitoring, otherwise, the request data passes the safety monitoring.
Optionally, the method further comprises: discarding and logging the request data which does not pass the safety monitoring, and adding the request data into a blacklist.
Optionally, the request data includes a virtual path; the sending the request data passing the security monitoring to the server side includes: acquiring a real request path corresponding to the virtual path from a second configuration file or database; and sending the request data passing through the safety monitoring to the server according to the real request path.
Optionally, the request data sent by the front end is intercepted by a passive interception mode, and the passive interception is realized by one of the following modes: mode one: providing a proxy service domain name to the front end, so as to receive and intercept request data sent by the front end when the front end accesses the proxy service domain name; mode two: and intercepting the request data sent by the front end through a plug-in.
Optionally, the request data sent by the front end is intercepted by an active interception mode, and the active interception is realized by the following modes: detecting whether the front end is a special front end or not, wherein the special front end forwards the request data by a service agent; and under the condition that the front end is the special front end, intercepting the request data sent by the front end by changing the proxy configuration file of the special front end.
Optionally, the modifying the proxy configuration file of the special front end includes: and establishing a nodeJs communication pipeline with the special front end, and replacing a real proxy path in the proxy configuration file with the virtual path through the nodeJs communication pipeline, wherein the virtual path is used for intercepting the request data.
Optionally, the desensitizing the request response data to generate request response desensitization data includes: decoding and converting the request response data to obtain a character string of the request response data, and converting the character string of the request response data into a JSON object of the request response data by utilizing a JSON function; after performing dimensionality reduction analysis on the JSON object of the request response data, determining sensitive data through sensitive data detection analysis, and filtering and/or encrypting the sensitive data to generate the request response desensitization data.
According to another aspect of an embodiment of the present invention, there is provided a data processing apparatus.
A data processing apparatus comprising: the request data monitoring module is used for intercepting request data sent by the front end and carrying out safety monitoring on the request data; the request data sending module is used for sending the request data passing through the safety monitoring to a server side and intercepting request response data from the server side, wherein the request response data is the response of the server side to the request data passing through the safety monitoring; the request response data desensitization module is used for carrying out desensitization processing on the request response data so as to generate request response desensitization data; and the request response desensitization data sending module is used for sending the request response desensitization data to the front end.
Optionally, the request data monitoring module is further configured to: acquiring monitoring rules from a first configuration file or a database; judging whether the request data is an illegal request with default or rule violating based on the monitoring rule, if so, the request data does not pass the safety monitoring, otherwise, the request data passes the safety monitoring.
Optionally, the request data monitoring module is further configured to: discarding and logging the request data which does not pass the safety monitoring, and adding the request data into a blacklist.
Optionally, the request data includes a virtual path; the request data sending module is further configured to: acquiring a real request path corresponding to the virtual path from a second configuration file or database; and sending the request data passing through the safety monitoring to the server according to the real request path.
Optionally, the request data sent by the front end is intercepted by a passive interception mode, and the passive interception is realized by one of the following modes: mode one: providing a proxy service domain name to the front end, so as to receive and intercept request data sent by the front end when the front end accesses the proxy service domain name; mode two: and intercepting the request data sent by the front end through a plug-in.
Optionally, the request data sent by the front end is intercepted by an active interception mode, and the active interception is realized by the following modes: detecting whether the front end is a special front end or not, wherein the special front end forwards the request data by a service agent; and under the condition that the front end is the special front end, intercepting the request data sent by the front end by changing the proxy configuration file of the special front end.
Optionally, the modifying the proxy configuration file of the special front end includes: and establishing a nodeJs communication pipeline with the special front end, and replacing a real proxy path in the proxy configuration file with the virtual path through the nodeJs communication pipeline, wherein the virtual path is used for intercepting the request data.
Optionally, the request response data desensitization module is further configured to: decoding and converting the request response data to obtain a character string of the request response data, and converting the character string of the request response data into a JSON object of the request response data by utilizing a JSON function; after performing dimensionality reduction analysis on the JSON object of the request response data, determining sensitive data through sensitive data detection analysis, and filtering and/or encrypting the sensitive data to generate the request response desensitization data.
According to yet another aspect of an embodiment of the present invention, an electronic device is provided.
An electronic device, comprising: one or more processors; and the memory is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are enabled to realize the data processing method provided by the embodiment of the invention.
According to yet another aspect of an embodiment of the present invention, a computer-readable medium is provided.
A computer readable medium having stored thereon a computer program which, when executed by a processor, implements a data processing method provided by an embodiment of the present invention.
One embodiment of the above invention has the following advantages or benefits: intercepting request data sent by a front end, and carrying out safety monitoring on the request data; transmitting the request data passing through the safety monitoring to a server side, and intercepting request response data from the server side, wherein the request response data is the response of the server side to the request data passing through the safety monitoring; desensitizing the request response data to generate request response desensitization data; the request response desensitization data is sent to the front end. The method can realize the solution of unified safety protection and data management aiming at the front end application, avoid the defect that the nginx proxy cannot protect, simultaneously reduce the complexity of the code of the interface secondary packaging and protection proxy, and have high maintainability.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of the main steps of a data processing method according to one embodiment of the invention;
FIG. 2 is a schematic diagram of an architecture for data processing according to one embodiment of the invention;
FIG. 3 is a flow diagram of data processing according to one embodiment of the invention;
FIG. 4 is a schematic diagram of the main modules of a data processing apparatus according to one embodiment of the present invention;
FIG. 5 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
fig. 6 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of main steps of a data processing method according to an embodiment of the present invention.
As shown in fig. 1, the data processing method according to an embodiment of the present invention mainly includes the following steps S101 to S104.
Step S101: and intercepting the request data sent by the front end, and carrying out safety monitoring on the request data.
Security monitoring of the requested data may include: acquiring monitoring rules from a first configuration file or a database; judging whether the request data is an illegal request with default or rule violations based on the monitoring rule, if so, the request data does not pass the security monitoring, otherwise, the request data passes the security monitoring.
The first profile is a pre-stored profile regarding the configuration of the monitoring rules. A request for a breach refers, for example, to a request issued for malicious transmission of a probe. Typically, proxy requests are normally proxy at the interface level, while requests containing a large number js or css belong to conflicting requests. Illegal requests such as forged or unsafe requests. The criteria for determining the violation or the default may depend on the service requirement, which is not limited by the embodiment of the present invention.
And discarding and logging the request data which do not pass the security monitoring, and adding the request data to a blacklist.
The request data may include a virtual path.
In one embodiment, the request data sent by the front end may be intercepted by passive interception, which may be implemented by one of the following ways: mode one: providing a proxy service domain name to the front end, so as to receive and intercept request data sent by the front end when the front end accesses the proxy service domain name; mode two: and intercepting the request data sent by the front end through the plug-in.
In another embodiment, the request data sent by the front end may be intercepted by an active interception manner, where the active interception may be implemented by: detecting whether the front end is a special front end, and forwarding request data by a service agent by the special front end; and under the condition that the front end is a special front end, intercepting request data sent by the front end by changing the proxy configuration file of the special front end. The special front end can be the front end needing the safety guarantee.
Altering the proxy profile for a particular front end may include: and establishing a nodeJs communication pipeline with the special front end, and replacing a real proxy path in the proxy configuration file with a virtual path through the nodeJs communication pipeline, wherein the virtual path is used for intercepting the request data.
Step S102: and sending the request data passing through the safety monitoring to the server side, and intercepting request response data from the server side, wherein the request response data is the response of the server side to the request data passing through the safety monitoring.
The sending of the request data passing the security monitoring to the server may include: acquiring a real request path corresponding to the virtual path from a second configuration file or database; and sending the request data passing the safety monitoring to the server according to the real request path. The second configuration file is configured with a corresponding relation between the virtual path and the real request path. The monitoring rules, and the correspondence between the virtual path and the real request path, may be configured in the same configuration file or database.
Step S103: the request response data is desensitized to generate request response desensitized data.
Desensitizing the request response data to generate request response desensitization data may include: decoding and converting the request response data to obtain a character string of the request response data, and converting the character string of the request response data into a JSON object of the request response data by utilizing a JSON function; after performing dimensionality reduction analysis on the JSON object of the request response data, determining the sensitive data through sensitive data detection analysis, and filtering and/or encrypting the sensitive data to generate request response desensitization data.
The corresponding decoding mode can be determined according to the encoding mode in the request response data header file to decode, and the conversion is to convert the decoded data into character strings, namely the character strings of the request response data.
Step S104: the request response desensitization data is sent to the front end.
FIG. 2 is a schematic diagram of an architecture for data processing according to one embodiment of the invention.
As shown in fig. 2, an embodiment of the present invention may include a security proxy service, and the security proxy service upstream may be a front-end service of an nginx proxy, a java web service, a nodejs service, or a different client (PC side, mobile side, applet side, etc.). The security proxy service has the core functions of performing security monitoring and scanning on the entrance request, and if an illegal request exists in the entrance, adopting strategies such as packet loss, blacklist, request restriction and the like to prevent malicious requests of a user side, and simultaneously avoiding the problems of exposing intranet server data or files (such as data or files in Node service side and Java service side) in an external network and security transparent transmission. Meanwhile, the service can detect and filter the outlet data for the second time, so that sensitive information such as an intranet account, an intranet organization structure and the like is prevented from being exposed to the outer network.
FIG. 3 is a flow diagram of data processing according to one embodiment of the invention.
As shown in fig. 3, the data processing flow of one embodiment of the present invention may be implemented by a security agent apparatus, which mainly includes an agent auto-detector, a request monitor, an agent converter, and a response assembler. The request monitor processes all the requests (or request data, such as front-end request 1-front-end request 4) sent by the front-end page, intercepts and analyzes all the requests of the page, processes security monitoring to all the requests, for example, can retrieve monitoring rules from configuration files or databases, judges whether there is an illegal request in the requests according to the monitoring rules, discards and records the illegal request if there is the illegal request, and can also be added into a blacklist according to the need. For a request that is not violated, whether the request data is violated is detected, for example, all special characters in the data are ignored, the file in the request is uploaded to a file parser for security detection, and the file parser can detect whether the file is a forged executable file. If the request is not a offending request and is not an offending request, it is passed to a proxy request parameter, such as proxyseq (service proxy request parameter), which processes the reverse proxy, which can forward the request to the real service. The proxy request may pass the security monitor pass request to the proxy converter.
The proxy converter performs true direction on the proxy request of the user, namely, the proxy request is directed to a true back end or BFF service.
The response assembler performs secondary verification on the data which is requested back, namely the request response data, and because a lot of intranet data is not allowed to be exposed to the users of the external network when the internal network and the external network interact, the data is subjected to desensitization processing, which can specifically comprise the processing of decoding and analyzing the request response data, converting and filtering or encrypting and the like.
The following describes the security agent device in further detail.
In one embodiment, the request monitor may intercept the request data sent by the front end in two ways: active interception and passive interception. Passive interception may be achieved by one of the following: mode one: providing a proxy service domain name to the front end, so as to receive and intercept request data sent by the front end when the front end accesses the proxy service domain name; mode two: and intercepting the request data sent by the front end through the plug-in. Specifically, in the first mode, URLs (Uniform Resource Locator, i.e., uniform resource locators or web addresses) of all interface requests from the front end are domain names of proxy services, so the security protection proxy device can intercept request data sent by the front end, and the user end cannot know addresses of services of the back end interface or the BFF (Backend For Frontend API, a special back end API designed for the front end), so that the function of protecting the back end domain names is achieved, and BFF service 1 and BFF service 2 are exemplarily shown in fig. 3. The second mode is to install plug-ins provided by the security protection agent device, the plug-ins can be divided into node plug-ins and java plug-ins, so that different plug-ins can be selected according to different languages, the plug-ins are used for intercepting all requests of front-end pages, and then the page requests are uniformly processed.
Active interception may be achieved by invoking an agent auto-detector, or service agent auto-detector, which may detect the front-end services that currently need to be monitored. Specifically, the application with the service agent may be determined by the agent auto-detector reading a profile or database, or the agent auto-detector may autonomously discover the application with the service agent, where the application refers to the front end. Detecting whether the front end is a special front end, and forwarding request data by a service agent by the special front end; and under the condition that the front end is a special front end, intercepting request data sent by the front end by changing the proxy configuration file of the special front end. The special front-end is a front-end application with a service agent, typically a front-end requiring security, for example by special marking of the front-end system or page. Specifically, the configuration file, the database or the front end which autonomously discovers the request data forwarded by the service agent is read, the configuration file path of the special front end which needs to be processed is added into the resource pool, and the resource pool is gradually consumed according to the processing speed set by the system. And establishing a nodeJs communication pipeline between the proxy automatic detector and the special front end, replacing a real proxy path in the proxy configuration file with a virtual path through the nodeJs communication pipeline so as to intercept all requests of the special front end, and directing all requests of the special front end to a domain name of the safety protection through the virtual path. After the proxy configuration file is changed, the front-end service can be restarted, the nodeJs pipeline is closed, and the next round of service detection and interception is performed. Because the embodiment of the invention aims at the front-end system, the nodeJs is used for carrying out service architecture, and the service computing performance is improved by utilizing the characteristic of high concurrency of nodes.
In one embodiment, the request monitor intercepts the request data sent by the front end, acquires the monitoring rule from the first configuration file or the database, and judges whether the request data is an illegal request with default or rule violation based on the monitoring rule. Specifically, all requests of the page are intercepted, monitoring rules are investigated from a first configuration file or a database, and the monitoring rules are stored in a cache for other request monitors to use. Judging whether the request data has an illegal request according to the monitoring rule, if so, discarding the illegal request and recording a log if not passing the security monitoring, and if not, passing the security monitoring if necessary, in a blacklist. The monitored request data is processed by Buffer (binary data type in nodeJs) and sent to the proxy converter through proxy request parameter (proxyseq). The monitoring rules can be various, one-to-one matching can be performed when the security monitoring is performed based on the monitoring rules, and whether the request data meets the requirement of the monitoring rules is judged, for example, the request data contains a large number of js or css requests, and at the moment, the request data can be judged to be a malicious transparent probe. And when judging whether the request data has an default request, all special characters in the data can be escaped, and the file in the request is uploaded to a file analyzer for safety monitoring.
In one embodiment, the proxy converter sends the request data passing the security monitoring to the server side and intercepts the request response data from the server side. Specifically, the second configuration file or the database is read, the real request path corresponding to the virtual path (for example, virtual URL) in the request data is obtained, the request data passing the security monitoring is sent to the server according to the real request path, and the service agent can be used to send the request data to the real server URL of the real request path. The server side generates request response data in response to the request data passing through the security monitoring, the request response data from the server side is intercepted through the proxy converter, namely the real server side responds to the request data, and the proxy converter sends the request response data to the response assembler.
In one embodiment, desensitizing the request response data by the response assembler to generate request response desensitized data includes: decoding and converting the request response data to obtain a character string of the request response data, and converting the character string of the request response data into a JSON object of the request response data by utilizing a JSON function; after performing dimensionality reduction analysis on the JSON object of the request response data, determining the sensitive data through sensitive data detection analysis, and filtering and/or encrypting the sensitive data to generate request response desensitization data. Specifically, request response data is intercepted, an encoding mode of an agent header file is detected, the request response data is decoded by an encoding mode of an encoding mode corresponding to the encoding mode, the decoded request response data is converted into a character string, and the character string is converted into a JSON object by using a JSON function. And performing dimension reduction analysis on the JSON object, and then performing detection analysis on the analyzed data. After all sensitive data are filtered or encrypted, the data are reassembled, and the JSON object data after assembly are subjected to Buffer (binary data type in nodeJs) data conversion to generate request response desensitization data. And sending the request response desensitization data to the real response parameters of the front-end page, and presenting the front-end page data. The embodiment of the invention carries out secondary verification on the request response data through the response assembler, and filters or encrypts the request response data so that the intranet data can not be exposed to the users of the external network when the intranet and the external network interact. The dimension reduction analysis can filter all sensitive data, for example, dimension reduction processing is performed on sensitive fields with deeper layers (such as array data with two dimensions or more than three dimensions, etc.), so as to improve the speed of data calculation and processing.
Fig. 4 is a schematic diagram of main modules of a data processing apparatus according to an embodiment of the present invention.
As shown in fig. 4, a data processing apparatus 400 according to an embodiment of the present invention mainly includes: a request data monitoring module 401, a request data transmitting module 402, a request response data desensitizing module 403, and a request response desensitizing data transmitting module 404.
The request data monitoring module 401 is configured to intercept request data sent by the front end, and perform security monitoring on the request data.
The request data sending module 402 is configured to send request data that passes the security monitoring to the server, and intercept request response data from the server, where the request response data is a response of the server to the request data that passes the security monitoring.
The request response data desensitization module 403 is configured to perform desensitization processing on the request response data to generate request response desensitization data.
The request response desensitization data sending module 404 is configured to send the request response desensitization data to the front end.
In one embodiment, the request data monitoring module is further to: acquiring monitoring rules from a first configuration file or a database; judging whether the request data is an illegal request with default or rule violations based on the monitoring rule, if so, the request data does not pass the security monitoring, otherwise, the request data passes the security monitoring.
In one embodiment, the request data monitoring module is further to: and discarding and logging the request data which do not pass the security monitoring, and adding the request data to a blacklist.
In one embodiment, the request data includes a virtual path; the request data sending module is further configured to: acquiring a real request path corresponding to the virtual path from a second configuration file or database; and sending the request data passing the safety monitoring to the server according to the real request path.
In one embodiment, the request data sent by the front end is intercepted in a passive interception manner, and the passive interception is realized in one of the following manners: mode one: providing a proxy service domain name to the front end, so as to receive and intercept request data sent by the front end when the front end accesses the proxy service domain name; mode two: and intercepting the request data sent by the front end through the plug-in.
In one embodiment, the request data sent by the front end is intercepted in an active interception manner, and the active interception is realized in the following manner: detecting whether the front end is a special front end, and forwarding request data by a service agent by the special front end; and under the condition that the front end is a special front end, intercepting request data sent by the front end by changing the proxy configuration file of the special front end.
In one embodiment, modifying the proxy profile for a particular front end includes: and establishing a nodeJs communication pipeline with the special front end, and replacing a real proxy path in the proxy configuration file with a virtual path through the nodeJs communication pipeline, wherein the virtual path is used for intercepting the request data.
In one embodiment, the request response data desensitization module is further configured to: decoding and converting the request response data to obtain a character string of the request response data, and converting the character string of the request response data into a JSON object of the request response data by utilizing a JSON function; after performing dimensionality reduction analysis on the JSON object of the request response data, determining the sensitive data through sensitive data detection analysis, and filtering and/or encrypting the sensitive data to generate request response desensitization data.
The data processing apparatus 400 according to the embodiment of the present invention corresponds to the same function as the security protection proxy apparatus described in the above embodiment, specifically, the function of the request data monitoring module 401 may be referred to in detail also in the description of the proxy automatic detector and the request monitor, the function of the request data transmitting module 402 may be referred to in detail in the description of the proxy converter, and the functions of the request response data desensitizing module 403 and the request response desensitizing data transmitting module 404 may be referred to in detail in the description of the response assembler.
In addition, the specific implementation of the data processing apparatus in the embodiments of the present invention has been described in detail in the above data processing method, and thus the description thereof will not be repeated here.
Fig. 5 illustrates an exemplary system architecture 500 in which a data processing method or data processing apparatus of an embodiment of the present invention may be applied.
As shown in fig. 5, the system architecture 500 may include terminal devices 501, 502, 503, a network 504, and a server 505. The network 504 is used as a medium to provide communication links between the terminal devices 501, 502, 503 and the server 505. The network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 505 via the network 504 using the terminal devices 501, 502, 503 to receive or send messages or the like. Various communication client applications may be installed on the terminal devices 501, 502, 503, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 501, 502, 503 may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 505 may be a server providing various services, such as a background management server (by way of example only) providing support for shopping-type websites browsed by users using the terminal devices 501, 502, 503. The background management server may analyze and process the received data such as the product information query request, and feedback the processing result (e.g., the target push information, the product information—only an example) to the terminal device.
It should be noted that, the data processing method provided by the embodiment of the present invention is generally executed by the server 505, and accordingly, the data processing apparatus is generally disposed in the server 505.
It should be understood that the number of terminal devices, networks and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 6, there is illustrated a schematic diagram of a computer system 600 suitable for use in implementing a terminal device or server in accordance with an embodiment of the present invention. The terminal device or server shown in fig. 6 is only an example, and should not impose any limitation on the functions and scope of use of the embodiments of the present invention.
As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, mouse, etc.; an output portion 607 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The drive 610 is also connected to the I/O interface 605 as needed. Removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on drive 610 so that a computer program read therefrom is installed as needed into storage section 608.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 609, and/or installed from the removable medium 611. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 601.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, as: a processor includes a request data monitoring module, a request data transmitting module, a request response data desensitizing module, a request response desensitizing data transmitting module. The names of these modules do not limit the modules themselves in some cases, for example, the request data monitoring module may also be described as "a module for intercepting request data sent by a front end and performing security monitoring on the request data".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include: intercepting request data sent by a front end, and carrying out safety monitoring on the request data; transmitting the request data passing through the safety monitoring to a server side, and intercepting request response data from the server side, wherein the request response data is the response of the server side to the request data passing through the safety monitoring; desensitizing the request response data to generate request response desensitization data; the request response desensitization data is sent to the front end.
According to the technical scheme of the embodiment of the invention, the request data sent by the front end is intercepted, and the request data is monitored safely; transmitting the request data passing through the safety monitoring to a server side, and intercepting request response data from the server side, wherein the request response data is the response of the server side to the request data passing through the safety monitoring; desensitizing the request response data to generate request response desensitization data; the request response desensitization data is sent to the front end. The method can realize the solution of unified safety protection and data management aiming at the front end application, avoid the defect that the nginx proxy cannot protect, simultaneously reduce the complexity of the code of the interface secondary packaging and protection proxy, and have high maintainability.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims (18)

1. A method of data processing, comprising:
intercepting request data sent by a front end, and carrying out safety monitoring on the request data;
transmitting the request data passing through the safety monitoring to a server side, and intercepting request response data from the server side, wherein the request response data is the response of the server side to the request data passing through the safety monitoring;
desensitizing the request response data to generate request response desensitization data;
the request response desensitization data is sent to the front end.
2. The method of claim 1, wherein the security monitoring of the request data comprises:
acquiring monitoring rules from a first configuration file or a database;
judging whether the request data is an illegal request with default or rule violating based on the monitoring rule, if so, the request data does not pass the safety monitoring, otherwise, the request data passes the safety monitoring.
3. The method as recited in claim 2, further comprising:
discarding and logging the request data which does not pass the safety monitoring, and adding the request data into a blacklist.
4. The method of claim 1, wherein the request data comprises a virtual path;
the sending the request data passing the security monitoring to the server side includes:
acquiring a real request path corresponding to the virtual path from a second configuration file or database;
and sending the request data passing through the safety monitoring to the server according to the real request path.
5. The method of claim 4, wherein the request data sent by the front end is intercepted by passive interception, and the passive interception is implemented by one of the following ways:
mode one: providing a proxy service domain name to the front end, so as to receive and intercept request data sent by the front end when the front end accesses the proxy service domain name;
mode two: and intercepting the request data sent by the front end through a plug-in.
6. The method of claim 4, wherein the request data sent by the front end is intercepted by active interception, the active interception being implemented by:
Detecting whether the front end is a special front end or not, wherein the special front end forwards the request data by a service agent;
and under the condition that the front end is the special front end, intercepting the request data sent by the front end by changing the proxy configuration file of the special front end.
7. The method of claim 6, wherein said altering the proxy profile of the special front end comprises:
and establishing a nodeJs communication pipeline with the special front end, and replacing a real proxy path in the proxy configuration file with the virtual path through the nodeJs communication pipeline, wherein the virtual path is used for intercepting the request data.
8. The method of claim 1, wherein desensitizing the request response data to generate request response desensitized data comprises:
decoding and converting the request response data to obtain a character string of the request response data, and converting the character string of the request response data into a JSON object of the request response data by utilizing a JSON function;
after performing dimensionality reduction analysis on the JSON object of the request response data, determining sensitive data through sensitive data detection analysis, and filtering and/or encrypting the sensitive data to generate the request response desensitization data.
9. A data processing apparatus, comprising:
the request data monitoring module is used for intercepting request data sent by the front end and carrying out safety monitoring on the request data;
the request data sending module is used for sending the request data passing through the safety monitoring to a server side and intercepting request response data from the server side, wherein the request response data is the response of the server side to the request data passing through the safety monitoring;
the request response data desensitization module is used for carrying out desensitization processing on the request response data so as to generate request response desensitization data;
and the request response desensitization data sending module is used for sending the request response desensitization data to the front end.
10. The apparatus of claim 9, wherein the request data monitoring module is further configured to:
acquiring monitoring rules from a first configuration file or a database;
judging whether the request data is an illegal request with default or rule violating based on the monitoring rule, if so, the request data does not pass the safety monitoring, otherwise, the request data passes the safety monitoring.
11. The apparatus of claim 10, wherein the request data monitoring module is further configured to:
Discarding and logging the request data which does not pass the safety monitoring, and adding the request data into a blacklist.
12. The apparatus of claim 9, wherein the request data comprises a virtual path;
the request data sending module is further configured to:
acquiring a real request path corresponding to the virtual path from a second configuration file or database;
and sending the request data passing through the safety monitoring to the server according to the real request path.
13. The apparatus of claim 12, wherein the request data sent by the front end is intercepted by passive interception, the passive interception being implemented by one of:
mode one: providing a proxy service domain name to the front end, so as to receive and intercept request data sent by the front end when the front end accesses the proxy service domain name;
mode two: and intercepting the request data sent by the front end through a plug-in.
14. The apparatus of claim 12, wherein the request data sent by the front end is intercepted by active interception, the active interception being implemented by:
Detecting whether the front end is a special front end or not, wherein the special front end forwards the request data by a service agent;
and under the condition that the front end is the special front end, intercepting the request data sent by the front end by changing the proxy configuration file of the special front end.
15. The apparatus of claim 14, wherein said altering the proxy profile of the special front end comprises:
and establishing a nodeJs communication pipeline with the special front end, and replacing a real proxy path in the proxy configuration file with the virtual path through the nodeJs communication pipeline, wherein the virtual path is used for intercepting the request data.
16. The apparatus of claim 9, wherein the request response data desensitization module is further configured to:
decoding and converting the request response data to obtain a character string of the request response data, and converting the character string of the request response data into a JSON object of the request response data by utilizing a JSON function;
after performing dimensionality reduction analysis on the JSON object of the request response data, determining sensitive data through sensitive data detection analysis, and filtering and/or encrypting the sensitive data to generate the request response desensitization data.
17. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs,
when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-8.
18. A computer readable medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the method according to any of claims 1-8.
CN202210180527.1A 2022-02-25 2022-02-25 Data processing method and device Pending CN116707846A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210180527.1A CN116707846A (en) 2022-02-25 2022-02-25 Data processing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210180527.1A CN116707846A (en) 2022-02-25 2022-02-25 Data processing method and device

Publications (1)

Publication Number Publication Date
CN116707846A true CN116707846A (en) 2023-09-05

Family

ID=87834392

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210180527.1A Pending CN116707846A (en) 2022-02-25 2022-02-25 Data processing method and device

Country Status (1)

Country Link
CN (1) CN116707846A (en)

Similar Documents

Publication Publication Date Title
US11593484B2 (en) Proactive browser content analysis
US9544318B2 (en) HTML security gateway
US8286250B1 (en) Browser extension control flow graph construction for determining sensitive paths
US9081961B2 (en) System and method for analyzing malicious code using a static analyzer
US8365291B1 (en) Browser extension control flow graph based taint tracking
CN101223562A (en) Immunizing HTML browsers and extensions from known vulnerabilities
Liu et al. MR-Droid: A scalable and prioritized analysis of inter-app communication risks
US8407766B1 (en) Method and apparatus for monitoring sensitive data on a computer network
US8789177B1 (en) Method and system for automatically obtaining web page content in the presence of redirects
US20120102541A1 (en) Method and System for Generating an Enforceable Security Policy Based on Application Sitemap
WO2021154724A1 (en) Metadata-based detection and prevention of phishing attacks
US11082445B1 (en) Preventing phishing attacks via document sharing
US11023590B2 (en) Security testing tool using crowd-sourced data
CN112751900B (en) Network request processing method and device
CN114626061A (en) Webpage Trojan horse detection method and device, electronic equipment and medium
US20140068771A1 (en) Transforming User-Input Data in Scripting Language
CN116707846A (en) Data processing method and device
Mun et al. Secure short url generation method that recognizes risk of target url
Cvitić et al. Defining Cross-Site Scripting Attack Resilience Guidelines Based on BeEF Framework Simulation
CN114491356B (en) Data acquisition method and device, computer storage medium and electronic equipment
CN112637171A (en) Data traffic processing method, device, equipment, system and storage medium
KR100977150B1 (en) Method and system for testing web site
CN113535322A (en) Form verification method and device
KR102311119B1 (en) Method for automatic diagnosis vulnerability of web and apparatus for performing the method
CN111737624B (en) Page redirection protection method and device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination