CN113472785B - Data processing method and device, electronic equipment and readable storage medium - Google Patents

Data processing method and device, electronic equipment and readable storage medium Download PDF

Info

Publication number
CN113472785B
CN113472785B CN202110740642.5A CN202110740642A CN113472785B CN 113472785 B CN113472785 B CN 113472785B CN 202110740642 A CN202110740642 A CN 202110740642A CN 113472785 B CN113472785 B CN 113472785B
Authority
CN
China
Prior art keywords
data
return
call instruction
service system
batch
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110740642.5A
Other languages
Chinese (zh)
Other versions
CN113472785A (en
Inventor
林振溪
屠方轫
李辉
李龙龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial and Commercial Bank of China Ltd ICBC
Original Assignee
Industrial and Commercial Bank of China Ltd ICBC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial and Commercial Bank of China Ltd ICBC filed Critical Industrial and Commercial Bank of China Ltd ICBC
Priority to CN202110740642.5A priority Critical patent/CN113472785B/en
Publication of CN113472785A publication Critical patent/CN113472785A/en
Application granted granted Critical
Publication of CN113472785B publication Critical patent/CN113472785B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
    • H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/51—Discovery or management thereof, e.g. service location protocol [SLP] or web services
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources

Abstract

The disclosure provides a data processing method, a data processing device, an electronic device and a readable storage medium, which can be applied to the technical field of data processing, the financial field or other fields. The data processing method comprises the following steps: configuring interface parameters of a data interface so that the data interface can be connected with a target data source and receive return data from the target data source; acquiring a first call instruction from a service system, preprocessing the first call instruction according to the data type of a target data source, and generating a second call instruction, wherein the first call instruction comprises at least one of a first batch call instruction and a first request message; and acquiring the returned data, and performing post-processing on the returned data according to the data type set by the service system to generate a standard information format.

Description

Data processing method and device, electronic equipment and readable storage medium
Technical Field
The present disclosure relates to the field of data processing technologies, and in particular, to a data processing method and apparatus, an electronic device, and a readable storage medium.
Background
As the form of data service is more and more, the data service standard, the data service category and the data format provided by different data service providers are different. When data is acquired, each pair corresponds to one data service, and customized data service access development is required to be carried out so as to realize data access to a specific data service. Furthermore, for different target users, data accessed from a service provider needs to be packaged into a format that satisfies the target users to ensure that the target users can use the data. The process has the problems of long period, repeated processing and the like, and cannot meet the requirement of rapid data transmission and calling between different service providers and target users, so that the efficiency is low.
Disclosure of Invention
In view of the above problems, the present disclosure provides a data processing method, an apparatus, an electronic device, and a readable storage medium.
According to a first aspect of the present disclosure, there is provided a data processing method, including but not limited to: configuring interface parameters of a data interface so that the data interface can be connected with a target data source and receive return data from the target data source; acquiring a first call instruction from a service system, preprocessing the first call instruction according to the data type of the target data source, and generating a second call instruction, wherein the first call instruction comprises at least one of a first batch call instruction and a first request message; and acquiring the returned data, and performing post-processing on the returned data according to the data type set by the service system to generate a standard information format.
According to an embodiment of the present disclosure, the second call instruction includes a second batch call instruction, the obtaining a first call instruction from a service system, and preprocessing the first call instruction according to a data type of the target data source, and the generating the second call instruction includes: the method comprises the steps of obtaining a first batch call instruction from a service system, encrypting the first batch call instruction according to the data type of a target data source, and generating a second batch call instruction.
According to the embodiment of the present disclosure, the second call instruction includes a second request packet, the obtaining a first call instruction from a service system, preprocessing the first call instruction according to the data type of the target data source, and generating the second call instruction includes: the method comprises the steps of obtaining a first request message from a service system, configuring request parameters of the first request message according to the data type of a target data source, and generating a second request message.
According to an embodiment of the present disclosure, the configuring, according to a data type of a target data source, a request parameter of the first request packet includes: setting at least one of a request head, a request body and a request format of the first request message; and authenticating and encrypting the first request message.
According to an embodiment of the present disclosure, the return data includes batch return data, the obtaining of the return data performs post-processing on the return data according to a data type set by the service system, and generating a standard information format includes: acquiring batch return data, decrypting the batch return data, and generating first decrypted data; responding to the first calling instruction, and acquiring the first decryption data and public data from the service system; and processing the first decrypted data and the public data to generate a standard information format.
According to the embodiment of the disclosure, after the first decrypted data is generated, the first decrypted data is stored persistently, and is updated in real time in a set period of time, so that the first decrypted data and the target data source keep synchronous.
According to an embodiment of the present disclosure, the return data includes a return packet, the obtaining of the return data performs post-processing on the return data according to a data type set by the service system, and generating a standard information format includes: configuring the format of the return message and acquiring the return message, wherein the format of the return message corresponds to the data type set by the service system; decrypting the return message to generate second decrypted data; and performing structure conversion on the second decrypted data to generate a standard information format.
According to an embodiment of the present disclosure, wherein the second decrypted data is persistently stored after being generated.
According to the embodiment of the disclosure, the data processing method further comprises monitoring data information of the data interface, and generating a data report view based on a monitoring result; the report view comprises at least one of a data link, data availability, data transmission success rate, data transmission time and data flow.
According to the embodiment of the present disclosure, the interface parameter includes at least one of URL information, an interface request protocol, and an interface request method.
According to an embodiment of the present disclosure, the configuring the interface parameters of the data interface includes determining a cache and a cache time of the data interface.
A second aspect of the present disclosure provides a data processing apparatus comprising: the configuration module is used for configuring interface parameters of a data interface so that the data interface can be connected with a target data source and receive return data from the target data source; the first generation module is used for acquiring a first calling instruction from a service system, preprocessing the first calling instruction according to the data type of the target data source and generating a second calling instruction, wherein the first calling instruction comprises at least one of a first batch calling instruction and a first request message; and the second generation module is used for acquiring the return data, performing post-processing on the return data according to the data type set by the service system and generating a standard information format.
According to an embodiment of the present disclosure, the first generation module includes a first generation submodule and a second generation submodule. The second calling instruction comprises a second batch calling instruction, the first generation submodule is configured to acquire a first batch calling instruction from the service system, and encrypt the first batch calling instruction according to the data type of the target data source to generate the second batch calling instruction. The second calling instruction comprises a second request message, the second generation submodule is configured to obtain the first request message from the service system, configure the request parameter of the first request message according to the data type of the target data source, and generate the second request message.
According to an embodiment of the present disclosure, the second generation module includes a third generation submodule and a fourth generation submodule. The third generation submodule is configured to acquire batch return data, decrypt the batch return data, and generate first decrypted data; responding to the first calling instruction, and acquiring the first decryption data and public data from the service system; and processing the first decrypted data and the public data to generate a standard information format.
According to an embodiment of the disclosure, after generating the first decrypted data, the third generating sub-module is configured to persistently store the first decrypted data and update the first decrypted data in real time for a set period of time to keep the first decrypted data synchronized with the target data source.
According to the embodiment of the present disclosure, the return data includes a return packet, the fourth generation sub-module is configured to configure a format of the return packet and obtain the return packet, and the format of the return packet corresponds to a data type set by the service system; decrypting the return message to generate second decrypted data; and performing structure conversion on the second decrypted data to generate a standard information format.
According to an embodiment of the disclosure, after generating the second decrypted data, the fourth generation submodule is configured to persistently store the second decrypted data.
According to an embodiment of the disclosure, the configuration module is configured to determine a cache and a cache time of the data interface.
A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; storage means for storing executable instructions that, when executed by the processor, implement the data processing method according to the above.
A fourth aspect of the present disclosure provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, implement a data processing method according to the above.
According to a fifth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements a data processing method according to the above.
According to the embodiment of the disclosure, the data interface is configured to preprocess the first call instruction from the service system, so that the first call instruction can be called according to different data formats of the target data source, and post-process the return data of the target data source to generate a standard information format, thereby facilitating data exchange and data management between the target data source and the service system.
Drawings
The foregoing and other objects, features and advantages of the disclosure will be apparent from the following description of embodiments of the disclosure, taken in conjunction with the accompanying drawings of which:
fig. 1 schematically shows a schematic diagram of a system architecture to which the data processing method of the embodiments of the present disclosure may be applied;
FIG. 2 schematically shows a flow chart of a data processing method according to an embodiment of the present disclosure;
fig. 3 schematically shows a flowchart for processing request parameters of a first request message according to a data processing method of an embodiment of the present disclosure;
FIG. 4 schematically illustrates a flow diagram for post-processing batch return data according to a data processing method of an embodiment of the disclosure;
FIG. 5 is a flow chart schematically illustrating post-processing of a return packet according to a data processing method of an embodiment of the present disclosure;
FIG. 6 schematically shows a schematic diagram of an implementation of a data processing method according to an embodiment of the present disclosure;
FIG. 7 schematically shows another implementation of a data processing method according to an embodiment of the disclosure;
FIG. 8A schematically shows a block diagram of a data processing apparatus according to an embodiment of the present disclosure;
FIG. 8B schematically shows a block diagram of a first generation module according to an embodiment of the disclosure;
FIG. 8C schematically shows a block diagram of a second generation module according to an embodiment of the present disclosure; and
fig. 9 schematically shows a block diagram of an electronic device adapted for a data processing method according to an embodiment of the present disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.
Where a convention analogous to "at least one of A, B, and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B, and C" would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.).
The embodiment of the disclosure provides a data processing method and a device, wherein the data processing method comprises the following steps: configuring interface parameters of a data interface so that the data interface can be connected with a target data source and receive return data from the target data source; acquiring a first call instruction from a service system, preprocessing the first call instruction according to the data type of a target data source, and generating a second call instruction, wherein the first call instruction comprises at least one of a first batch call instruction and a first request message; and acquiring the returned data, and performing post-processing on the returned data according to the data type set by the service system to generate a standard information format.
According to the embodiment of the disclosure, the data interface is configured to preprocess the first call instruction from the service system, so that the first call instruction can be called according to different data formats of the target data source, post-process the return data of the target data source, generate the standard information format, and facilitate data exchange and data management between the target data source and the service system.
Fig. 1 schematically shows a schematic diagram of a system architecture to which the data processing method of the embodiment of the present disclosure can be applied. It should be noted that fig. 1 is only an example of a system architecture to which the embodiments of the present disclosure may be applied to help those skilled in the art understand the technical content of the present disclosure, and does not mean that the embodiments of the present disclosure may not be applied to other devices, systems, environments or scenarios. It should be noted that the data processing method and apparatus provided by the embodiment of the present disclosure may be used in the related fields of the data processing technology field and the financial field, and may also be used in any field other than the financial field.
As shown in fig. 1, an exemplary system architecture 100 to which the data processing method may be applied may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is the medium used to provide communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may have installed thereon various data processing clients or communication client applications, such as a data analysis-type application, a shopping-type application, a web browser application, a search-type application, an instant messaging tool, a mailbox client, social platform software, and the like (by way of example only).
The terminal devices 101, 102, 103 may be various electronic devices having information collection and data processing functions, including but not limited to smart phones, smart televisions, tablet computers, laptop portable computers, desktop computers, and the like.
The server 105 may be a server that provides various services, such as a background management server (for example only) that provides support for data information acquired by users using the terminal devices 101, 102, 103. The background management server may analyze and perform other processing on the received data such as the user request, and feed back a processing result (e.g., a webpage, information, or data obtained or generated according to the user request) to the terminal device.
It should be noted that the data processing method provided by the embodiment of the present disclosure may be generally executed by the terminal devices 101, 102, 103 or the server 105. Accordingly, the data processing apparatus provided by the embodiments of the present disclosure may be generally disposed in the terminal devices 101, 102, 103 or the server 105. The data processing method provided by the embodiment of the present disclosure may also be executed by a server or a server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Accordingly, the data processing apparatus provided by the embodiment of the present disclosure may also be disposed in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and/or the server 105.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
The data processing method of the disclosed embodiment will be described in detail below with reference to fig. 2 to 7.
Fig. 2 schematically shows a flow chart of a data processing method according to an embodiment of the present disclosure.
As shown in fig. 2, the data processing method 200 of the embodiment of the present disclosure includes operations S201 to S203.
In operation S201, interface parameters of the data interface are configured to enable the data interface to connect to the target data source and receive return data from the target data source.
In an embodiment of the present disclosure, the interface parameter includes at least one of URL information, an interface request protocol, and an interface request method. For example, URL information of the configuration data interface, such as 122.18.173.204:80/api/RcQuery. And configuring a request protocol of the data interface, such as supporting http and https protocols. And the request method for configuring the data interface supports methods such as get, post, put, delete and the like.
In the embodiment of the present disclosure, the target data source includes a plurality of target data sources, and the information called by different target data sources differs according to different protocols and formats. In the related art, when information of different target data domain sources is called, the information needs to be adjusted according to the target data domain sources, which is very inconvenient. According to the method and the device, the interface parameters are configured, and the interface protocol in the interface parameters can adapt to information retrieval of different target data sources. Thereby enabling the data interface to connect to the target data source and receive return data from the target data source.
For example, configuring interface parameters of a data interface includes determining a buffering and a buffering time of the data interface.
In operation S202, a first call instruction from the service system is obtained, the first call instruction is preprocessed according to the data type of the target data source, and a second call instruction is generated, where the first call instruction includes at least one of a first batch call instruction and a first request message.
For example, the second call instruction may be a second batch call instruction or a second request message. And when the first batch calling instruction is sent to the target data source, processing the first batch calling instruction to generate a second batch calling instruction. And when the first request message is sent to the target data source, processing the first request message to generate a second request message. The batch calling instruction can be used for calling the data of the target data source in batches, so that the batch transmission and processing of the data are facilitated. The request message instruction can acquire the specified information, so that the information content can be accurately acquired.
In an embodiment of the present disclosure, the second call instruction includes a second batch call instruction, the first call instruction from the service system is acquired, the first call instruction is preprocessed according to a data type of the target data source, and the generating the second call instruction includes: and acquiring a first batch call instruction from the service system, and encrypting the first batch call instruction according to the data type of the target data source to generate a second batch call instruction.
For example, when data of the target data source needs to be called in batch, the service system generates a first batch call instruction, and after the first batch call instruction is obtained, performs preprocessing, such as encryption, to generate a second batch call instruction. In the processing process, the first call instruction needs to be processed according to the data type of the target data source, for example, the data types of different target data sources are encrypted in different ways, and after the first batch of call instructions are encrypted, a second batch of call instructions are formed.
In an embodiment of the present disclosure, the second call instruction includes a second request packet, the first call instruction from the service system is obtained, the first call instruction is preprocessed according to a data type of the target data source, and the generating the second call instruction includes: the method comprises the steps of obtaining a first request message from a service system, configuring request parameters of the first request message according to the data type of a target data source, and generating a second request message.
For example, the service system needs to call data of the target data source, configure the first request packet according to different data types, and generate the second request packet. The second request message can process the first request message according to the data type of the target data source to generate a second request message, so that the second request message can adapt to the scenes of interface authentication and message encryption of the target data source.
Fig. 3 schematically shows a flowchart for processing request parameters of a first request message according to a data processing method of the embodiment of the present disclosure.
In an embodiment of the present disclosure, configuring request parameters of the first request packet according to a data type of the target data source includes: and setting at least one of a request head, a request body and a request format of the first request message. And authenticating and encrypting the first request message.
For example, as shown in fig. 3, configuring the request parameters of the first request packet includes operations S301 to S303, and in operation S301, the request header of the first request packet is set. In operation S302, a request body and a request format of the first request packet are set. For example, XML, JSON, form-data, etc. formats may be supported. In operation S303, the first request packet is authenticated and encrypted. The process of processing the request parameter may be, for example, processing the request parameter through a Groovy script.
In operation S203, the return data is acquired, post-processed according to the data type set by the service system, and a standard information format is generated.
For example, the return data may be batch return data or a return message, and after the target data source responds to the second batch call instruction, the batch return data is returned to the data interface, and the data interface processes the batch return data or the return message according to the data type set by the service system to generate a standard information format that can be processed or directly used by the service system.
FIG. 4 is a flow chart schematically illustrating post-processing of batch return data by a data processing method according to an embodiment of the present disclosure.
In the embodiment of the present disclosure, the returning data includes batch returning data, the returning data is acquired, post-processing is performed on the returning data according to a data type set by a service system, and generating the standard information format includes: acquiring batch return data, decrypting the batch return data, and generating first decrypted data; responding to the first call instruction, and acquiring first decryption data and public data from a service system; and processing the first decrypted data and the public data to generate a standard information format.
For example, as shown in fig. 4, after the batch return data is acquired, the post-processing includes operations S401 to S403.
In operation S401, batch return data is obtained, and the batch return data is decrypted to generate first decrypted data.
For example, a second batch of call instructions is obtained in response to the preprocessed first batch of call instructions. And after the data interface receives the batch return data, decrypting the batch return data to generate first decrypted data.
In an embodiment of the present disclosure, after the first decrypted data is generated, the first decrypted data is persistently stored, and the first decrypted data is updated in real time for a set period of time, so that the first decrypted data and the target data source are kept synchronized.
For example, the first decrypted data is stored persistently, facilitating subsequent calls. The first decrypted data may be stored in a storage location set by the data interface, or may be stored in another storage location. After the first decrypted data is stored, the first decrypted data is updated in real time, for example, in an asynchronous updating manner, so that the stored first decrypted data and the data of the target data source are ensured to be synchronous. In alternative embodiment modes, the updating mode may also be other modes.
The first decryption data is updated in real time in a set time period, for example, the data update may be performed at 12 points per day, or the data update may be performed every hour, or other feasible updating manners.
In operation S402, in response to the first call instruction, first decrypted data and common data from the service system are acquired.
For example, the first call instruction is an instruction sent from the business system to the target data source or the data interface, and after receiving the first call instruction, the data interface acquires the first decryption data from the location where the first decryption data is stored and acquires the common data from the business system in response to the first call instruction. The common data of the service system may be stored in the same location as the first decryption data, or may be stored in a different location from the common data of the service system. And after the data interface receives the first call instruction, acquiring the first decryption data and the public data at the same time.
In operation S403, the first decrypted data and the common data are processed to generate a standard information format.
For example, after the first decrypted data and the common data are acquired, the data interface processes the acquired first decrypted data and common data. For example, processing can be performed according to the type of data required by the service system, and a standard information format is generated according to the requirement of the service system, so that the service system can call the data conveniently.
According to the embodiment of the disclosure, by combining the public data of the service system and the data of the target data source, the service system can call the data at any time conveniently, and the efficiency is improved.
Fig. 5 schematically shows a flowchart of post-processing a return packet according to the data processing method of the embodiment of the present disclosure.
In the embodiment of the present disclosure, the returning data includes a returning message, the returning data is acquired, post-processing is performed on the returning data according to a data type set by a service system, and the generating of the standard information format includes: configuring the format of a return message and acquiring the return message, wherein the format of the return message corresponds to the data type set by the service system; decrypting the return message to generate second decrypted data; and performing structure conversion on the second decrypted data to generate a standard information format.
As shown in fig. 5, the flow of post-processing the return packet includes operations S501 to S503.
In operation S501, a format of the return packet is configured and the return packet is obtained, where the format of the return packet corresponds to a data type set by the service system.
For example, before the return message is acquired, the format of the return message is configured, so that the return message supports XML and JSON formats, and the format of the return message is adapted to the data type set by the service system, so that the service system can directly read or call the return message when calling data, and the efficiency is improved.
In operation S502, the return packet is decrypted, and second decrypted data is generated.
For example, the Groovy script may be used to process the return packet, for example, perform operations such as decryption, and generate second decrypted data.
In an embodiment of the present disclosure, after the second decrypted data is generated, the second decrypted data is persistently stored.
In operation S503, the second decrypted data is subjected to structure conversion, and a standard information format is generated.
After the second decrypted data is generated, the structure of the second decrypted data needs to be converted to generate a standard information format. The standard information format can be conveniently called by a service system without data processing and the like, and the efficiency can be improved.
In the embodiment of the disclosure, the data report further comprises data information of a monitoring data interface, and a data report view is generated based on a monitoring result; the report view comprises at least one of data link, data availability, data transmission success rate, data transmission time and data flow.
For example, when data transmission is performed, the data information transmitted by the data interface is monitored, so that a data report view can be generated according to the monitoring result of the data information. For example, success rate of data transmission, time used for data transmission, different links for data transmission, traffic used for data transmission, availability of target data sources, and so on. During the data management process, according to the data report view, the contents and the transmission process of each data in the transmission process can be clearly seen, and the data transmission management is convenient.
According to the embodiment of the disclosure, the data interface is configured to preprocess the first call instruction from the service system, so that the first call instruction can be called according to different data formats of the target data source, and post-process the return data of the target data source to generate a standard information format, thereby facilitating data exchange and data management between the target data source and the service system.
According to the embodiment of the disclosure, the efficiency of target data source access is improved by configuring the data interface. The method can realize the quick butt joint of the data service interfaces of target data sources with different protocols, different formats and different data sources of the 0 code, output the standardized service, can be used after online debugging, automatic generation of interface documents and verification are finished without restarting any service, provides standard data interface service for a business system, and supports the calling permission and the current limit strategy of flexibly distributing the applied service interfaces. Effectively solving the problems of investment cost and time cost. In addition, the efficiency of service opening is improved. The method has the advantages that the common data of the business system and the target data source data are persisted, the fusion of the data of the common data domain and the target data source is realized, new data service is mined, and the data value is fully released. Data management is effectively carried out by monitoring data information of the data interface. And the target data source is managed uniformly, a uniform data standard is established, and the complexity of data service management is simplified.
Fig. 6 schematically shows an implementation process diagram of a data processing method according to an embodiment of the present disclosure. Fig. 7 schematically shows another implementation process diagram of the data processing method according to the embodiment of the present disclosure.
The data processing method according to the embodiment of the present disclosure will be described in detail below with reference to fig. 6 and 7.
As shown in FIG. 6, target data source 610 includes a plurality of target data sources 611, business system 620 includes a plurality of business systems 621, and data interface 630 includes a preprocessing portion 631 and a post-processing portion 632, which may generate a data link list view 640.
In the embodiment of the present disclosure, the first call instruction is in the form of a first request packet, and the data processing method includes operations S601 to S605. Specifically, in operation S601, a first request message is sent, that is, the service system 620 sends the request message to the data interface 630. After the data interface 630 receives the first request message, the preprocessing section 631 preprocesses the request message according to the data type of the target data source. And after the first request message is preprocessed, generating a second request message. Next, operation S602 is performed, the data interface 630 sends the second request message to the target data source 610, which may send the second request message to different target data sources 611 according to different data types or requirements. Next, operation S603 is performed, that is, the target data source sends a return message to the data interface 630 in response to the second request message. After the data interface 630 receives the return packet, the post-processing section 632 performs post-processing on the return packet to generate a standard information format. Next, operation S604 is performed to transmit the standard information format to the service system 620. In the embodiment of the present disclosure, operation S605 is further included, that is, by monitoring the data information of the data interface 630, a data link table view 640 is generated.
As shown in fig. 7, the target data source 710 includes a plurality of target data sources 711, the service system 720 includes a plurality of service systems 721, a preprocessing portion 730, and the data interface 740 includes a post-processing portion 741, the data interface 740 is further configured to call service system common data 750 from the service systems, and the data interface 740 may generate a data link list view 760.
In the embodiment of the present disclosure, the first call instruction is in the form of a first batch call instruction, and the data processing method includes operations S701 to S706. Specifically, in operation S701, a first batch call instruction is sent, that is, the business system 720 sends the first batch call instruction to the target data source 710, and the preprocessing portion 730 preprocesses the first batch call instruction. After the preprocessing portion 730 receives the first batch call instruction, the preprocessing portion 730 preprocesses the request message, and the preprocessing process is performed according to the data type of the target data source. And after preprocessing the first batch of call instructions, generating second batch of call instructions. Next, operation S702 is performed, the preprocessing portion 730 sends the second batch of call instructions to the target data source 710, which may send the second batch of call instructions to a different target data source 711 according to a different data type or requirement. Next, operation S703 is performed, that is, the target data source sends batch return data to the data interface 740 in response to the second batch call instruction. After the data interface 740 receives the batch return data, the post-processing part 741 performs post-processing on the batch return data to generate a standard information format. In the embodiment of the present disclosure, operation S705 is further included, in response to the first call instruction, after preprocessing the batch return data, acquiring the preprocessed data, that is, acquiring the first decrypted data, and simultaneously acquiring the common data of the service system in response to the first call instruction, where the post-processing section 741 further includes processing the first decrypted data and the common data to generate the standard information format. Next, operation S704 is performed to transmit the standard information format to the service system 720. In the embodiment of the present disclosure, operation S706 is further included, that is, by monitoring the data information of the data interface 740, the data link list view 760 is generated.
Fig. 8A schematically shows a block diagram of a data processing apparatus according to an embodiment of the present disclosure. Fig. 8B schematically illustrates a block diagram of a first generation module according to an embodiment of the present disclosure. Fig. 8C schematically shows a block diagram of a second generation module according to an embodiment of the present disclosure.
As shown in fig. 8A, the data processing apparatus 800 of the embodiment of the present disclosure includes a configuration module 810, a first generation module 820, and a second generation module 830.
The configuration module 810 is configured to configure interface parameters of the data interface, so that the data interface can connect to the target data source and receive return data from the target data source. The configuration module 810 may be configured to perform operation S201 of the embodiment of the present disclosure, and is not described herein again.
The first generating module 820 is configured to obtain a first call instruction from the service system, pre-process the first call instruction according to the data type of the target data source, and generate a second call instruction, where the first call instruction includes at least one of a first batch call instruction and a first request message. The first generating module 820 may be configured to perform operation S202 in the embodiment of the disclosure, and is not described herein again.
The second generating module 830 is configured to obtain the return data, perform post-processing on the return data according to the data type set by the service system, and generate a standard information format. The second generating module 830 may be configured to perform operation S203 according to the embodiment of the disclosure, and is not described herein again.
According to an embodiment of the present disclosure, as shown in fig. 8B, the first generation module 820 includes a first generation submodule 821 and a second generation submodule 822. The second call instruction includes a second batch call instruction, and the first generation submodule 821 is configured to obtain the first batch call instruction from the service system, encrypt the first batch call instruction according to the data type of the target data source, and generate the second batch call instruction. The second call instruction includes a second request message, and the second generation submodule 822 is configured to obtain the first request message from the service system, configure the request parameter of the first request message according to the data type of the target data source, and generate the second request message.
According to an embodiment of the present disclosure, as shown in fig. 8C, the second generation module 830 includes a third generation submodule 831 and a fourth generation submodule 832. The third generation submodule 831 is configured to obtain the batch return data, decrypt the batch return data, and generate first decrypted data; responding to the first call instruction, and acquiring first decryption data and public data from a service system; and processing the first decrypted data and the public data to generate a standard information format.
According to an embodiment of the present disclosure, after generating the first decrypted data, the third generating sub-module 831 is configured to persistently store the first decrypted data and update the first decrypted data in real time for a set period of time so as to keep the first decrypted data synchronized with the target data source.
According to the embodiment of the present disclosure, the return data includes a return message, the fourth generating sub-module 832 is configured to configure a format of the return message and obtain the return message, and the format of the return message corresponds to a data type set by the service system; decrypting the return message to generate second decrypted data; and performing structure conversion on the second decrypted data to generate a standard information format.
After generating the second decrypted data, the fourth generation submodule 832 is configured to persist the second decrypted data, in accordance with embodiments of the present disclosure.
The configuration module 810 is used to determine the buffering and buffering time of the data interface according to embodiments of the present disclosure.
According to an embodiment of the present disclosure, any plurality of the configuration module 810, the first generation module 820, the second generation module 830, the first generation submodule 821, the second generation submodule 822, the third generation submodule 831, and the fourth generation submodule 832 may be combined and implemented in one module, or any one of them may be split into a plurality of modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the configuration module 810, the first generation module 820, the second generation module 830, the first generation submodule 821, the second generation submodule 822, the third generation submodule 831, and the fourth generation submodule 832 may be at least partially implemented as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or implemented in any one of three implementations of software, hardware, and firmware, or in a suitable combination of any of them. Alternatively, at least one of the configuration module 810, the first generation module 820, the second generation module 830, the first generation submodule 821, the second generation submodule 822, the third generation submodule 831 and the fourth generation submodule 832 may be at least partially implemented as a computer program module which, when executed, may perform a corresponding function.
Fig. 9 schematically shows a block diagram of an electronic device adapted to implement a data processing method according to an embodiment of the present disclosure. The electronic device shown in fig. 9 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 6, an electronic apparatus 900 according to an embodiment of the present disclosure includes a processor 901 which can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM) 902 or a program loaded from a storage portion 908 into a Random Access Memory (RAM) 903. Processor 901 can include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or related chipset(s) and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), and/or the like. The processor 901 may also include on-board memory for caching purposes. The processor 901 may comprise a single processing unit or a plurality of processing units for performing the different actions of the method flows according to embodiments of the present disclosure.
In the RAM 903, various programs and data necessary for the operation of the electronic apparatus 900 are stored. The processor 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. The processor 901 performs various operations of the method flows according to the embodiments of the present disclosure by executing programs in the ROM 902 and/or the RAM 903. Note that the programs may also be stored in one or more memories other than the ROM 902 and the RAM 903. The processor 601 may also perform various operations of the method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.
Electronic device 900 may also include input/output (I/O) interface 905, input/output (I/O) interface 905 also connected to bus 904, according to an embodiment of the present disclosure. The electronic device 900 may also include one or more of the following components connected to the I/O interface 905: an input portion 906 including a keyboard, a mouse, and the like; an output section 907 including components such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 908 including a hard disk and the like; and a communication section 909 including a network interface card such as a LAN card, a modem, or the like. The communication section 909 performs communication processing via a network such as the internet. The drive 910 is also connected to the I/O interface 905 as necessary. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 910 as necessary, so that a computer program read out therefrom is mounted into the storage section 908 as necessary.
The present disclosure also provides a computer-readable storage medium, which may be embodied in the device/apparatus/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The above-mentioned computer-readable storage medium carries one or more programs which, when executed, implement a data processing method according to an embodiment of the present disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, a computer-readable storage medium may include the ROM 902 and/or RAM 903 described above and/or one or more memories other than the ROM 902 and RAM 903.
Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the method illustrated in the flow chart. When the computer program product runs in a computer system, the program code is used for causing the computer system to realize the data processing method provided by the embodiment of the disclosure.
The computer program performs the above-described functions defined in the system/apparatus of the embodiments of the present disclosure when executed by the processor 901. The systems, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.
In one embodiment, the computer program may be hosted on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed in the form of a signal on a network medium, and downloaded and installed through the communication section 909 and/or installed from the removable medium 911. The computer program containing program code may be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 909, and/or installed from the removable medium 911. The computer program, when executed by the processor 901, performs the above-described functions defined in the system of the embodiment of the present disclosure. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.
In accordance with embodiments of the present disclosure, program code for executing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, these computer programs may be implemented using high level procedural and/or object oriented programming languages, and/or assembly/machine languages. The programming language includes, but is not limited to, programming languages such as Java, C + +, python, the "C" language, or the like. The program code may execute entirely on the user computing device, partly on the user device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
It will be appreciated by a person skilled in the art that various combinations or/and combinations of features recited in the various embodiments of the disclosure and/or in the claims may be made, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.
The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the disclosure, and these alternatives and modifications are intended to fall within the scope of the disclosure.

Claims (12)

1. A data processing method, comprising:
configuring interface parameters of a data interface so that the data interface can be connected with a target data source and receive return data from the target data source;
acquiring a first call instruction from a service system, preprocessing the first call instruction according to the data type of the target data source, and generating a second call instruction, wherein the first call instruction comprises at least one of a first batch call instruction and a first request message;
acquiring the returned data, and performing post-processing on the returned data according to the data type set by the service system to generate a standard information format;
wherein the return data comprises batch return data,
the obtaining of the return data and the post-processing of the return data according to the data type set by the service system, and the generating of the standard information format includes:
acquiring batch return data, decrypting the batch return data, and generating first decrypted data;
responding to the first calling instruction, and acquiring the first decryption data and public data from the service system;
processing the first decrypted data and the public data to generate a standard information format;
or
The return data may comprise a return message that includes,
the obtaining the return data, performing post-processing on the return data according to the data type set by the service system, and generating a standard information format includes:
configuring the format of the return message and acquiring the return message, wherein the format of the return message corresponds to the data type set by the service system;
decrypting the return message to generate second decrypted data;
and performing structure conversion on the second decrypted data to generate a standard information format.
2. The data processing method according to claim 1, wherein the second call instruction includes a second batch of call instructions, the obtaining a first call instruction from a business system, preprocessing the first call instruction according to a data type of the target data source, and generating the second call instruction includes:
the method comprises the steps of obtaining a first batch call instruction from a service system, encrypting the first batch call instruction according to the data type of a target data source, and generating a second batch call instruction.
3. The data processing method according to claim 1, wherein the second call instruction includes a second request packet, the obtaining a first call instruction from a service system, and preprocessing the first call instruction according to a data type of the target data source, and the generating the second call instruction includes:
the method comprises the steps of obtaining a first request message from a service system, configuring request parameters of the first request message according to the data type of a target data source, and generating a second request message.
4. The data processing method according to claim 3, wherein the configuring the request parameter of the first request packet according to the data type of the target data source comprises:
setting at least one of a request head, a request body and a request format of the first request message;
and authenticating and encrypting the first request message.
5. The data processing method according to claim 1, wherein after the first decrypted data is generated, the first decrypted data is persistently stored and updated in real time for a set period of time so as to keep the first decrypted data synchronized with the target data source.
6. The data processing method according to claim 1, wherein the second decrypted data is persistently stored after being generated.
7. The data processing method according to any one of claims 1 to 6, further comprising monitoring data information of the data interface, generating a data report view based on the monitoring result;
the report view comprises at least one of data link, data availability, data transmission success rate, data transmission time and data flow.
8. The data processing method of any of claims 1 to 6, wherein the interface parameters comprise at least one of URL information, interface request protocol, interface request method.
9. The data processing method of any of claims 1 to 6, wherein the configuring of the interface parameters of the data interface comprises determining a buffering and a buffering time of the data interface.
10. A data processing apparatus comprising:
the configuration module is used for configuring interface parameters of a data interface so that the data interface can be connected with a target data source and receive return data from the target data source;
the first generation module is used for acquiring a first calling instruction from a service system, preprocessing the first calling instruction according to the data type of the target data source and generating a second calling instruction, wherein the first calling instruction comprises at least one of a first batch calling instruction and a first request message;
a second generating module, configured to obtain the returned data, perform post-processing on the returned data according to the data type set by the service system, generate a standard information format,
wherein the return data comprises batch return data,
the obtaining of the return data and the post-processing of the return data according to the data type set by the service system, and the generating of the standard information format includes:
acquiring batch return data, and decrypting the batch return data to generate first decrypted data;
responding to the first call instruction, and acquiring the first decryption data and public data from the service system;
processing the first decrypted data and the public data to generate a standard information format;
or
The return data may comprise a return message,
the obtaining the return data, performing post-processing on the return data according to the data type set by the service system, and generating a standard information format includes:
configuring the format of the return message and acquiring the return message, wherein the format of the return message corresponds to the data type set by the service system;
decrypting the return message to generate second decrypted data;
and performing structure conversion on the second decrypted data to generate a standard information format.
11. An electronic device, comprising:
one or more processors;
storage means for storing executable instructions which, when executed by the processor, implement a data processing method according to any one of claims 1 to 9.
12. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, implement a data processing method according to any one of claims 1 to 9.
CN202110740642.5A 2021-06-30 2021-06-30 Data processing method and device, electronic equipment and readable storage medium Active CN113472785B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110740642.5A CN113472785B (en) 2021-06-30 2021-06-30 Data processing method and device, electronic equipment and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110740642.5A CN113472785B (en) 2021-06-30 2021-06-30 Data processing method and device, electronic equipment and readable storage medium

Publications (2)

Publication Number Publication Date
CN113472785A CN113472785A (en) 2021-10-01
CN113472785B true CN113472785B (en) 2022-10-04

Family

ID=77876867

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110740642.5A Active CN113472785B (en) 2021-06-30 2021-06-30 Data processing method and device, electronic equipment and readable storage medium

Country Status (1)

Country Link
CN (1) CN113472785B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104615534A (en) * 2015-01-28 2015-05-13 广州酷狗计算机科技有限公司 Interface test method and device
CN112153152A (en) * 2020-09-28 2020-12-29 深圳壹账通智能科技有限公司 Management method and system of algorithm service interface
CN112491659A (en) * 2020-12-02 2021-03-12 广州品唯软件有限公司 Flow playback test method and device, computer equipment and storage medium
CN112671705A (en) * 2020-11-23 2021-04-16 中信银行股份有限公司 Message processing method and device, electronic equipment and computer readable storage medium

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11695800B2 (en) * 2016-12-19 2023-07-04 SentinelOne, Inc. Deceiving attackers accessing network data

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104615534A (en) * 2015-01-28 2015-05-13 广州酷狗计算机科技有限公司 Interface test method and device
CN112153152A (en) * 2020-09-28 2020-12-29 深圳壹账通智能科技有限公司 Management method and system of algorithm service interface
CN112671705A (en) * 2020-11-23 2021-04-16 中信银行股份有限公司 Message processing method and device, electronic equipment and computer readable storage medium
CN112491659A (en) * 2020-12-02 2021-03-12 广州品唯软件有限公司 Flow playback test method and device, computer equipment and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
网络管理中的ping命令;熊宗炬;《电脑知识与技术(经验技巧)》;20150405(第04期);第116-117页 *

Also Published As

Publication number Publication date
CN113472785A (en) 2021-10-01

Similar Documents

Publication Publication Date Title
US11588642B2 (en) Method and apparatus for sharing and acquiring information
CN110377440B (en) Information processing method and device
US10164970B1 (en) Secure cross-domain page generation
CN114826733B (en) File transmission method, device, system, equipment, medium and program product
US10831754B2 (en) Using metadata to take action on an SMS message on a proprietary system
CN112866385B (en) Interface calling method and device, electronic equipment and storage medium
CN112039826A (en) Login method and device applied to applet terminal
CN112015383A (en) Login method and device
CN112489760B (en) Prescription processing method and system based on distributed identity authentication
CN113472785B (en) Data processing method and device, electronic equipment and readable storage medium
CN116244682A (en) Database access method, device, equipment and storage medium
CN112257039B (en) Identity attribute adding method and device and electronic equipment
CN114584378A (en) Data processing method, device, electronic equipment and medium
CN113626848A (en) Sample data generation method and device, electronic equipment and computer readable medium
CN111752625A (en) Method and device for interface mock
CN111163156A (en) Data processing method, device and storage medium based on block chain
CN111835846B (en) Information updating method and device and electronic equipment
CN113783835B (en) Password sharing method, device, equipment and storage medium
CN113572763B (en) Data processing method and device, electronic equipment and storage medium
CN114553570B (en) Method, device, electronic equipment and storage medium for generating token
CN110851754A (en) Webpage access method and system, computer system and computer readable storage medium
CN116112172B (en) Android client gRPC interface security verification method and device
CN113676482B (en) Data transmission system and method and data transmission system and method based on double-layer SSL
CN114785560B (en) Information processing method, device, equipment and medium
US11201856B2 (en) Message security

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant